The Challenges With Ultra-Low Latency Delivery For Real-Time Applications

For the past 20+ years, CDNs have dominated the content delivery space when it comes to delivering video on the Internet. Third-party CDNs focus on layers 4-7 in the OSI stack which includes things like caching and TCP protocol optimization. But they still depend on the underlying routers of the providers they buy bandwidth from, and latency could still vary and be only as good as the underlying network. For ultra-low latency applications, layers 4-7 don’t help, which is why many CDNs are looking at other technologies to assist them with delivering video with only a few hundred milliseconds of delay. For the delivery of real-time applications, there is a debate currently taking place within the industry as to whether or not you need a private core with optimal routing on layers 2-3, to truly succeed with low-latency delivery.

Historically there have been two different ecosystems in networking: private networks and the public internet. Private networks were used by corporations and enterprises to connect their headquarters, branch offices, data centers and retail locations. The public internet was used to reach consumers and have them connect to websites. In the past, these two ecosystems lived separately, one behind the firewall and the other beyond it.

With the rise of the cloud, the world changed. While Amazon deserves much of the credit for ushering in the era of cloud computing (and the demise of physical server data centers), it may have been Microsoft who triggered the tipping point when they shifted from client-server based Exchange to cloud-based Office365. For all of the enterprises dependent on Microsoft software, the cloud was no longer something they could delay. Once they started to shift to the cloud, their entire architecture needed to change from hardware-based solutions to cloud-based solutions. The success of companies like ZScaler and Box shows this continuing trend.

Lately we’ve seen new vendors come to the market, taking products from the private networking space and using it to try and solve problems with the public internet. As an example, last week Mode launched in the market with $24M in funding with an MPLS Cloud, used in combination with SD-WAN and last-mile internet. MPLS is a 20-year-old technology used by enterprises to create private dedicated circuits between their branch offices (ex: Walmart connects all of their stores back to Headquarters using dedicated MPLS circuits). Enterprises buy these lines because they have guaranteed bandwidth and QoS. MPLS costs 10x-20x the price of Internet bandwidth, but you get guaranteed quality, which is why business still pay for this difference, because the latency variability on the internet is far higher.

However the problem with MPLS is that it was built for the enterprise market. It is very rigid and inflexible and often takes months to provision. It was designed for a hardware-centric world, where you have fixed offices and you buy multi-year contracts to connect offices together using fixed circuits. But what if you could make MPLS an elastic cloud-like solution, and bring the great qualities of MPLS over to the public internet.

While many vendors in the public internet space continue to evangelize that the public internet is great at everything, the reality is that it is not. If you are thinking in terms of “seconds” like 2-6 seconds, then the public Internet + CDNs solve this problem. However, when you think in “milliseconds” – like 30 milliseconds being the maximum allowed jitter for real-time voice calls – the public internet is not nearly good enough. If it were, enterprises wouldn’t pay billions of dollars a year and growing at rates 20x more than regular internet prices for private networks. Instead of rewriting all of the issues, read the awesome blog post by the team over at Riot Games entitled “Fixing The Internet For Real Time Applications”.

As many have pointed out, the current public internet has been tuned for a TCP/HTTPS world were web pages are the common payload. However, as we transition into an ultra low latency world where the internet is being used for WebRTC, sockets, APIs, live video and voice, the public internet is definitely not a “good enough” solution. If you’ve ever complained about a terrible video call experience, you know this to be true. Sectors like gaming, financial trading, video conferencing, and live video will continue to leverage the public internet for transport. The growth in ultra low latency applications (IoT, AR, VR, real-time machine learning, rapid blockchain etc.) will continue to make this problem worse.

When I talk to CIOs in charge of both sides of the house: consumer/end-user facing, as well as in-house employee facing, they have said they are interested in a way to utilize MPLS level QoS and reliability for millions of their end-users back to their applications (gaming, financial trading, voice calls, video calls). But is has to be super flexible just like the cloud, so they have total control over the network – create it, manage it, get QoS, define policy –  and have it be way more flexible than traditional hardware-defined networks. It can’t be a black box. The customer has to be able to see everything that’s happening, so they have real transparency for traffic and flow and have it be elastic, growing or shrinking depending on their needs. This is the service Mode has just launched with in the market, which you can read more about here. I’d be interested to hear from those who knows a lot more about this stuff than I do, what they think of the idea.