My third post on IP interconnection – innovation in networks

This is the third post in my series on IP interconnection. The first focused on the technological underpinnings regarding IP interconnection. The second covered key economic aspects of IP interconnection. In this post, I focus on the innovation that’s been happening in IP interconnection over the last decade.

Innovation happens throughout the Internet ecosystem, from the physical networks and broadband connections, to computers and devices, to software. While innovation in some aspects of the Internet is readily visible – such as new web applications or content sites (think Facebook or Netflix) – innovation in how the physical networks are structured and how they function is equally important, if less visible. In fact, many new services – such as massive distribution of video – would not be possible today without innovation in how the physical networks interconnect and exchange traffic.

First, let’s put things in perspective: The volume, variety and complexity of traffic on the Internet have been evolving and growing for many years. Just fifteen years ago, Internet video was virtually non-existent. Today, the fastest-growing type of Internet traffic by “volume” (i.e., how much capacity is consumed in sending the traffic) is real-time streaming video. According to Sandvine, nearly 60% of all traffic during peak periods on wireline residential networks is video. Though other kinds of traffic – from peer to peer file sharing to web browsing – are also growing at high rates, video consumes far and away the most capacity on the Internet today.

This wave of increasing traffic volume is not a new phenomenon. We’ve seen similar waves of growth in the past. In the late 1990s, simple web traffic (a new phenomenon on the Internet at the time) was growing massively, and many "experts" feared that a congestive failure of the Internet was imminent. In fact, Robert Metcalfe, the originator of Ethernet protocols, famously predicted that the Internet would collapse in 1995, promising that if it did not, he would eat his words. (Later on, he did quite literally do so).

In each case where a new service or application caused traffic to surge, the Internet has adapted to the new levels of bandwidth demand. Part of this adaptability is due to the billions of dollars of investment by network companies – I’m talking fiber optics, VDSL, and cable DOCSIS modems and networks – that have expanded bandwidth by several orders of magnitude in the last decade.

But an equally important part of this story of network adaptation has to do with innovations in how networks interconnect and exchange traffic. Remember, the term “Internet” refers to a network of multiple networks, each differently owned and operated, often using different underlying technologies, that interconnect and exchange traffic via agreements that benefit all parties. The chart below shows the way that video traffic might have moved under the old “hierarchical” model of the Internet back in 2003.

This hierarchical, tree-like organization worked fine for traffic that did not have latency constraints or did not require virtually instantaneous display. But it didn’t work well for high-capacity, latency-sensitive traffic like video. Traffic had to travel all the way from the consumer up through the backbone, down to the website, and then all the way back again (see the red line), carrying a continuous stream of video traffic. It simply did not run smoothly, and video delivery was increasingly degraded as more and more video was consumed.

But two major examples of network innovation changed this:

First, a more efficient way to manage web video traffic has taken hold with the gradual restructuring toward a more distributed network system, and with the emergence of Content Delivery Networks (CDNs). With the rise of CDNs – and there are many competitive providers of these services – traffic operates differently. It does not travel hundreds or thousands of miles up and down the “stack.” Instead, content is stored on several local servers, closer to end-users, making the transmission much more rapid and greatly improving quality.

As a result, here is what the Internet looks like today:

A second example of innovation and network adaptability is the emergence of Internet Exchange Points (IXPs). In many parts of the world, traffic on the old, hierarchical Internet model had to travel from a country in, say, Africa, all the way to the U.S. and back in order to serve a simple web page. Just a little more than a decade ago, 75% of all European and Asian traffic passed through the U.S., where a lion’s share of the web’s servers were located at the time.

However, in recent years, interconnection arrangements called IXPs began to evolve. IXPs are hubbing agreements that allow several ISPs to locate routers together and interconnect with one another. These hubs are located near a country or region served by the ISPs. As a result, when a request for information, content, or an email is sent that can be served by an ISP in a nearby IXP hub, it can be routed quickly to that ISP and not have to travel thousands of miles to the US or Europe and back. Today, according to FCC Commissioner Robert McDowell, only 5% of Africa’s internet traffic now touches the U.S. The number of IXPs operating globally today has grown from just a handful to over 350 today, a solid indication that IXPs are at the heart of a new and more efficient way to interconnect networks.

The important point to keep in mind is that all of this restructuring did not just happen by accident or regulatory decree. A whole new set of interconnection relationships had to develop in order to make CDNs and IXPs possible. Did anyone order this to happen? Did regulators mandate it? No. Rather, growing market demand for bandwidth forced companies to collaborate in order to find a better way to serve that demand. It wasn’t always easy to do and there were inevitable hiccups, but it worked out and network operators continue to work together to meet the evolving bandwidth needs of the 21st century.

Given how fast the Internet is growing, and how effectively it has adapted to growing bandwidth demand with new forms of interconnection, the question I’ve been addressing in these three posts about IP interconnection is this: what could regulation or government intervention do to improve the current state of network organization, without undermining the innovation and rapid adaptation we’ve experienced over the last 15 years?

I'd wager not much. The Internet is growing faster than anyone ever imagined. More people around the world own a mobile phone than own a toothbrush. Although not all of these phones are connected to the Internet, given the speed of adoption, they undoubtedly will be in the near future. Despite this growth – in fact, because of it – the Internet continues to adapt to meet that swelling demand without governments having to play any significant regulatory role. On balance, the three posts I’ve offered make a strong case that allowing the Internet to evolve as it has – with minimal government involvement – is the best means of ensuring its continued success.