A few weeks ago, Verizon received an email from a customer in Los Angeles asking why he was not getting a good experience watching Netflix on his 75 Mbps FiOS connection. He was understandably confused by some of the misleading public accounts that inaccurately suggest widespread congestion that could affect Netflix traffic on Verizon’s network. Worse still were claims that Verizon is deliberately “throttling” Netflix traffic. This customer wanted to know what was going on and why his performance wasn’t what he hoped. We, too, wanted to get to the bottom of the problem.
Review Shows No Congestion on Verizon Network
After receiving the letter, our network operations team studied the network connection for this customer for the week preceding the date that he emailed us. They measured the utilization – or the percentage of total capacity used – at every link in the Verizon network – from the customer to the edge of our network, where we receive Netflix traffic – to determine where, if at all, congestion was occurring.
This review confirmed again what I’ve explained before (here and here): there was no congestion anywhere within the Verizon network. There was, however, congestion at the interconnection link to the edge of our network (the border router) used by the transit providers chosen by Netflix to deliver video traffic to Verizon’s network.
While the links chosen by Netflix were congested (congestion occurs when use approaches or reaches 100% capacity during peak usage periods), the links from other transit providers (carrying non-Netflix traffic) to Verizon’s network did not experience congestion and were performing fine. The maximum amount of capacity used (or peak utilization) over the links between these other networks and Verizon’s network ranged from 10% to 80% (with an average peak utilization of 44%).
Below is a chart that illustrates the connections; we also note the peak utilization on key links on Verizon’s network as Netflix traffic travelled to our FiOS customer in L.A.
So Netflix Could Cut Buffering By Making Simple Adjustments?
One might wonder why Netflix and its transit providers were the only ones that ran into congestion issues. What it boils down to is this: these other transit and content providers took steps to ensure that there was adequate capacity for their traffic to enter our network. In some cases, these are settlement-free peering arrangements, where the relative traffic flows between an IP network provider and us remain roughly equal, and both parties invest in sufficient facilities to match these roughly equal flows. That is the traditional basis for such deals. In other cases there may be traffic imbalances, but the networks or content providers have entered into paid arrangements with us to ensure connections and capacity to meet their needs for their out-of-balance traffic.
That has not recently been the case with Netflix and the networks carrying its traffic. Netflix sends out an unprecedented amount of traffic. Sandvine recently noted that Netflix now accounts for more than one-third of all North American downstream traffic during peak hours. For whatever reason (perhaps to cut costs and improve its profitability), Netflix did not make arrangements to deliver this massive amount of traffic through connections that can handle it.
Instead, Netflix chose to attempt to deliver that traffic to Verizon through a few third-party transit providers with limited capacity over connections specifically to be used only for balanced traffic flows. Netflix knew better. Netflix is responsible for either using connections that can carry the volume of traffic it is sending, or working out arrangements with its suppliers so they can handle the volumes. As we’ve made clear before, we regularly negotiate reasonable commercial arrangements with transit providers or content providers to ensure a level of capacity that accommodates their volume of traffic. Such arrangements have been common practice for content delivery networks in the Internet ecosystem for many years, and Netflix is fully capable of taking the necessary and customary steps to ensure that its connections match its traffic volumes.
We’re Not Satisfied Until Our Customers Are
Even though there is no congestion on our network, we’re not satisfied if our customers are not. We fully understand that many of our customers want a great streaming experience with Netflix, and we want that too. Therefore, we are working aggressively with Netflix to establish new, direct connections from Netflix to Verizon’s network. This doesn’t “prioritize” Netflix traffic in any way, but it ensures that their traffic gets on our network through direct connections—not middleman networks—that are up to the task.
The benefit of these direct connections will be two-fold. First, Verizon customers who use Netflix will have a significantly improved experience as Netflix traffic flows over non-congested links. Early tests indicate that this is the case. The other benefit will be that the congestion that we are seeing today on those links between these middleman networks and our L.A. border router will likely go away once the huge volume of Netflix traffic is routed more efficiently. This will improve performance for any other traffic that is currently being affected over those connections.
As always, Verizon is focused on providing its customers with the best Internet experience possible. We appreciate their patience as we work with Netflix to ensure our shared customers have an improved viewing experience as soon as possible.