How The Olympics Were Streamed Online: Q&A With Microsoft & iStreamPlanet

Last week StreamingMedia.com hosted a webinar with Microsoft and iStreamPlanet talking about the backend technology and platforms that powered the online streaming of the Winter Olympics. During the Q&A portion of the event, the companies gave out details on server technology, protocols, encoding bitrates, transport services and other pieces of information. Below are their answers to some of the most frequently asked questions during the webinar.

Question: How many peak concurrent connections were recorded during the Sochi Olympics?
NBC Sports hasn’t officially confirmed their peak concurrent viewership numbers, but the New York Times did report that the USA vs. Canada men’s hockey game viewership peaked at 850,000 concurrent viewers before the end of the game.

Question: What was your highest & lowest bit rate per device?
The highest bit rate was 3.5 Mbps (1280×720) and the lowest bit rate was 200 kbps (340×192). Alex Zambelli’s blog contains additional information about the Olympics encoding specifications.

Question: How many total GBs of live video were delivered from opening to closing ceremonies?
NBC Sports reported that 10.8 million hours of streamed video were consumed during the Sochi Winter Olympics, of which 80% were live streams consumed via NBCOlympics.com and NBC Sports Live Extra apps. Those numbers are cumulative and NBC hasn’t officially stated how many hours or gigabytes of video were actually delivered.

Question: What is the latency you have for the live streaming?
Latency depends on multiple variables in the workflow, including the geographical distance between the content acquisition and the content delivery, the encoding buffer size, the duration of the media segments (fragments), CDN caching and edge structure, and the player buffer duration. Typical end-to-end latency observed in HTTP-based adaptive streaming can range anywhere between 15 seconds to over a minute.

Question: Why use Flash Player and not Silverlight since you guys used Windows Azure Media Services?
The customer will often determine the playback platforms and formats and in this case NBC Sports chose Adobe Primetime and Flash for the client experiences. Both Aventus and Windows Azure Media Services are capable of delivering live video in multiple HTTP-based adaptive streaming formats, such as Smooth Streaming, HDS and HLS.

Question: What kind of redundancy was used to ensure that feeds continued streaming despite any technical failures?
There were multiple levels of redundancy built into every aspect of the live video workflow, from the content acquisition to the content delivery. A few examples: content was acquired via IP over fiber networks but for key events satellite back up was ready. iStreamPlanet replicated the streams at Switch SuperNAP and sent them to Azure U.S. East and West data centers for geographical redundancy. Finally, within each data center Aventus used full VM redundancy to ensure uninterrupted publishing even in the case of partial channel failures.

Question: Can you give us some detail about the transport services necessary to convey the live streams from ingest to CDN?
The live feeds from Sochi were delivered over a private IP network from Sochi Olympics Broadcasting Services (OBS) to NBC’s facility in Stamford, CT; to iStreamPlanet’s data center at Switch SuperNAP in Las Vegas, NV; and to Microsoft’s Windows Azure U.S. East and West data centers.

Question: What is the orchestration layer used to deploy and configure additional channel instances in the cloud?
Aventus has built in orchestration layer to deploy and launch Aventus channels while the details of the Azure cloud orchestration layer used to launch Window Azure Media Services channels will be made available when Windows Azure Media Services officially launches their Live service.

Question: How much consideration is given to content protection/DRM and how is it achieved?
Our customers give a lot of thought to content protection and content security since most of the content we stream is considered premium content. Aventus supports PlayReady DRM and AES-128 encryption although NBC Sports chose not to use these technologies for live streaming the Olympics. NBC Sports used CDN token authentication in addition to TVE authentication to prevent unauthorized access to video streams.

Question: What KPIs were used in the monitoring? Was the dashboard simply viewed by humans?
Both Aventus and Windows Azure Media Services relied on their own telemetry systems, respectively, to monitor system performance. Due to the high profile nature of the event additional human monitoring was provided by iStreamPlanet and Microsoft operations teams.

Question: Where were the ad triggers inserted (SCTE?) and did the player insert ads or the stream leaving the cloud?  Was Freewheel the ad insertion tool?
iStreamPlanet built a video CMS tool used for ad insertion. Ad markers were inserted into encoded streams via Aventus web-based API, and converted to HLS and HDS-compatible ad markers by Windows Azure Media Services. Ad insertion was then performed client-side by the respective player apps.

Question: What was the origin encoder at iStreamPlanet?  Elemental?  Cisco Media Processor?  Digital Rapids?
None of the above. The Olympics video feeds were ingested and transcoded to multiple bitrates and packaged into Smooth Streaming for delivery to Windows Azure using iStreamPlanet’s proprietary video encoding solution Aventus. Aventus is a cloud-based live video encoding solution, built for the cloud from the ground up, capable of delivering live events and live linear channels in a scalable fashion while utilizing commodity x86-64 hardware.

Question: How was Android supported? Was RTSP in the mix for the native android browser or was HLS in Chrome utilized  or was an app solely relied on?
Android was supported via a Google Play store app which consumed HLS streams.

Question: What server technology was used to deliver the live multiprotocol streams?
Windows Azure Media Services was used to dynamically remux video into Apple HLS and Adobe HDS format for delivery to compatible devices.

Question: What processing is done at Switch?
In the case of the Olympics the streams were aggregated and replicated at the Switch SuperNAP data center and then sent to Azure U.S. East and West for processing. iStreamPlanet also uses Switch SuperNAP for hosting its own private Aventus SaaS offering.

Question: Was there a way to prevent viewers from recording live or on demand content?
Using content protection solutions such as PlayReady DRM it is possible to restrict media playback to only secure playback platforms over certified output paths (e.g. HDCP/HDMI). This type of content protection was not used by NBC Sports for streaming the Sochi Olympics.

Question: Was this technology used solely for NBC or was it the streaming infrastructure for all global streaming (i.e. CBC in Canada, Network Ten in Australia, etc)?
iStreamPlanet Aventus was used solely for NBC Sports live streaming video coverage of the Sochi Olympics.

Question: What device or appliance was used to deliver the video from NBC Sports Stamford to iStreamPlanet?
NBC Sports used Ericsson video encoders to create broadcast-quality MPEG-2 Transport Streams for delivery to iStreamPlanet.

Question: Where there any major issues or failover situations during the games? I know the infrastructure was built to span multiple datacenters for redundancy.
The solution performed extremely well and there were no failovers required, however, the solution was architected to fail over to new VMs and servers with minimal to no interruption to the streams.

Question: What kind of CMS was used and how do they relate to each other?
iStreamPlanet worked with NBC Sports to create a custom CMS for the Olympics to perform ad and slate insertion. Other CMS solutions were used by NBC Sports to manage live and on-demand video assets.