Recent Live Event Failures Due To Technical Issues With Vendors, Not “Excessive Traffic”

We’ve seen quite a few web events have some pretty big streaming failures lately including the Oscars, Golden Globes, WWE Network and last night’s True Detective finale on HBOGO. After these kinds of failures, the companies involved are quick to point to a large volume of traffic as the reason for the failure when in fact, that’s not the case at all. Technical issues with vendors, a breakdown in technology and lack of planning are the true cause of viewer’s frustration. Today HBO said that the problems viewers encountered last night was due to “an excessive amount of traffic“, which is not the real reason there was a problem.

Like almost all content providers, HBO relies on third-party CDNs for the delivery of their videos and these CDNs are in the business of handling large scale live events. That’s what they do. So making it sound like these CDNs can’t handle the traffic caused by these events, from a bandwidth or server capacity perspective is not the case. They have the resources to handle it, but many times, technical issues in the video ecosystem keep these events from happening without problems and more often than not, poor planning also plays a part in the failure.

However, it’s not always CDNs that are the cause of the problem, it can also be issues with other platforms in the video chain that all have to work perfectly in order to make the live streaming event a success. In the case of the Oscars, ABC said the live streams were down nationwide “due to a traffic overload/greater than expected“, but that’s not what happened. In fact, the live stream going over CDN provider EdgeCast and Akamai (Updated 3/12: Akamai did not stream the Oscars but was the CDN for the website) was fine, but Verizon had an issue with the signal acquisition portion of the event. Verizon’s Uplynk software runs on Amazon Web Services platform, which encountered a problem during the broadcast. So it had nothing to do with a “traffic overload” on the CDNs or them not being able to handle the capacity of the live stream itself.

In the case of the recent problems the WWE had when they launched their new streaming platform WWE Network, the company outsourced the infrastructure requirements to Major League Baseball Advanced Media (MLBAM), which had problems with their ecommerce infrastructure that kept people from being able to sign up for the service. The day the service launched, WWE said that MLBAM “was overwhelmed and its systems have been unable to process most orders since 9 am due to demand for WWE Network.” In reality, the “demand” is not what caused the issue, this wasn’t even a live event, but rather technical issues with MLBAM infrastructure and software. And keep in mind, MLBAM does not own their own infrastructure, they run their software on top of other cloud and CDN providers.

For the media reports that say these live events are failing because there is so much demand, that’s not true. Remember that the live webcast of the Oscar’s was only available in eight cities in the U.S. and you had to be a subscriber to cable TV. In the case of HBOGO, last week HBO’s CEO said that only a small percentage of subscribers to HBO actually use the HBOGO service. So user demand and traffic to these live and on-demand events is not what’s causing them to fail and the number of people trying to watch the video streams aren’t that large. It simply comes down to the technology platforms that are being used which are not as reliable as broadcast TV. Media reports implying that a “broadband shortage” or “overwhelming popularity” is what’s causing these failures is not accurate.

The fact is the Internet will never be as reliable or scalable as broadcast TV distribution. Some don’t like to hear that, but that’s reality and you can’t argue with it. The Internet was not built to handle large scale live video events with the same reliability, quality and viewership numbers as broadcast TV. We’ve been live streaming events over the Internet for 20 years now and in that time, the average live video stream has only grown in quality from about 300Kps in 1999 2000/2001, to an average of 1.5Mbps today. I would argue that’s not a lot of quality improvement over the past 15 years, especially when compared to how much broadcast TV has improved quality wise during the same time.