The Internet consists of some 40.000 administratively separate networks linked together. How does this system of systems operate on the physical layer? Is it flakey and unreliable as some corporations still preferring their private connections seem to think? Can it handle the relentless growth of data volumes? Is it expanding to reach the billions of poorly connected people in developing countries? To answer these questions I spoke with Henk Steenman CTO of the Amsterdam Internet Exchange (AMS-IX) and James Cowie co-founder and CTO of Renesys, an Internet measurement and analytics company.

Internet exchange

In the early ‘90s much of the local European Internet traffic was routed over submarine cables across the Atlantic to Virginia, USA. There MAE-East, one of the world’s first Internet Exchanges (IXs), hosted physical connections to route traffic from one network to another. For many small European Internet Service Providers (ISPs) it was the only exchange point available.

In 1997 twenty competing ISPs and carriers established AMS-IX to interconnect their networks locally [1]. AMS-IX brought down the cost of data exchange, reduced latency and eased traffic congestion on the heavily overloaded American hub.

Henk Steenman has been part of the Dutch not-for-profit organization from the start. With his help it has grown out to be one of the largest exchange points in the world. Constantly in a neck and neck race for first place with DE-CIX in Frankfurt, AMS-IX currently comes in second with 595 participating networks and traffic peaking at 2.3 Tb per second.

   Internet intelligence
 
Henk-Steenman
Henk Steenman, CTO AMS-IX

Renesys is an American company that collects and analyzes data about both the logical and the physical structure of the Internet [2]. ‘The logical map tells you how the Internet believes it should be routing traffic’, says James Cowie. ‘It basically says if you needed to reach this person and you were some place else, what chain of organizations would help you carry it there. The physical map is more detailed and involves figuring out which IP-addresses, which routers, are connected to each other and which ones of those are actually the most useful in moving traffic closer to its destination. We take active measurements of millions points of hundreds of places worldwide to make an accurate map of what the Internet is doing.

 

‘We use that informationfor customers who need to figure out how to use the Internet effectively as a tool of business. People tend to study their own part of the Internet very carefully. But nobody worries about what is over the horizon. One of the things we provide is that big picture because more and more companies have a global interest. The Internet is not a managed system so we provide some of that missing transparency.’

Physical layer

‘It’s interesting what we learn about the physical layer from the logical grid and these performance readings on the sensors’, says Cowie. ‘There is a good example of a case where we saw a number of networks in Iran and Iraq simultaneously disappear. We thought this was strange so the next day we looked at the media to see what this might have been. It turns out there is a gas pipeline that goes from Iran across the frontier into Turkey and then on to European markets for the energy. Pipelines take a lot of effort to negotiate because you have to get the rights of way, secure it and bury the pipeline. So when people do that they typically also put some fiber optics next to the pipeline because its basically zero marginal cost. That must have been the case here because that day there was a roadside bomb that had broken the pipeline.

‘The heartening thing is that the Internet did not permanently get impaired by that because the Internet works around things like that all the time. There was probably another fiber route that could be used and was failed over to. So in our data you see a problem and then a recovery. The Internet is much more resilient to damage even than it’s fabled.

Government regulation

Governments the world over increasingly want to regulate the Internet at the end user level. I asked the two men if they see the same trend at the infrastructural level.

Cowie: ‘The ITU, the UN agency responsible for global telecommunications regulations, took their eye off the ball during a critical window of inflation where the Internet stepped beyond something that could be lightly regulated. Which was, in my opinion, an excellent stroke of luck because it now becomes much more difficult to retroactively put things back in the bottle.

‘It is always possible government intervention will cause a mounting regulatory burden, you’re always operating in some jurisdiction. But I think that governments realize that the Internet’s fluidity makes it possible for IT services to go anywhere. The people are going to be reluctant to do things that will cause their local market to look less favorable from an investment standpoint.

‘I often am asked by people if their part of the Internet can be taken offline as happened during the blackouts in Egypt and Syria. I think that in Western Europe and the United States there aren’t really many threats left to the Internet. The Internet has grown so wonderfully diverse in these places that in terms of being attacked or people taking the Internet offline it really can’t happen anymore. It’s beyond that stage.’

The AMS-IX CTO isn’t keen on increasing regulations either: ‘Currently the Dutch regulators are keeping their distance with respect to AMS-IX but that could change. If regulations and bureaucracy were imposed on us it would be at the cost of the flexibility and simplicity with which we run our operation. One of our most important qualities is that we’re a carrier-neutral IX which means that any ISP can connect to exchange traffic. We’d like to propagate our neutrality as much as possible and I am afraid that if the government steps in we will lose some of that.’

 

Data flood
 

jim cowie
James Cowie, CTO Renesys
AMS-IX deals with a doubling of traffic volume roughly every two years. The challenge for Henk Steenman and his colleagues is to find technical solutions to deal with that growth. ‘We’re now implementing 100 Gb/s Ethernet equipment which became available last year’, says Steenman. ‘Up until then we used the 10GbE standard so we’re increasing the throughput rate of our network by a factor 10. As one of the largest exchanges we’re always running up against the limit of what is technically possible. We’re participating in the IEEE standard body where the next  standard is being developed which is going to be 400GbE. The data rate of each new Ethernet standard has always been increased by a factor 10 but the technology simply isn’t ready to make the jump to 1Tb. Although in terms of growth we could really use that. On the other hand growth is a two-way street, traffic can’t grow faster than the available infrastructure allows so I don’t foresee any serious shortage.’

James Cowie isn’t worried about capacity either. ‘If you look at the total amount of subsea fiber that interconnects the various continents a very small portion of that is actually lit and available for use. There is enormous amounts of reserved bandwidth. And inside continents, especially in Europe, there is just amazing amounts of available bandwidth that could be lit if the traffic grows. I don’t think that is ever going to be a problem.’

Digital divide

In most developed countries a well-established Internet infrastructure provides fast and cheap connections. In developing countries, however, infrastructure has lacked behind causing a digital divide. Is the gap closing?

James Cowie: ‘The trend is that the countries that had the least Internet do the best as soon as the Internet does arrive. East Africa is a perfect example. The only Internet that had been available was very limited, much of it was over satellite connections which are very slow and super expensive. Until the submarine cables came on shore. Within three months we could see the whole market turning upside down. People were cancelling their satellite contracts and they were moving all of their things onto this cable and the data rates went from tens of kilobits per second to  a gigabit network. Exactly like that in the space of weeks and months.

What happened there ultimately was that people were leapfrogging generations of technology. Maybe they never get a desktop computer they just move right ahead an get a smart phone. Not having had the continuous evolution through all the stages that Western Europe went through they get to cherry pick the very best technology at the very lowest prices. So it is actually very positive. The digital divide is still quite deep but the Internet is a great leveler.’

Henk Steenman: ‘Now that the networks are rapidly growing in East Africa the next thing needed in terms of infrastructure are regional IXs. In Kenya, for instance, much of the local traffic destined for neighboring countries is routed via Europe for lack of a well-established regional exchange point. They are facing the same problem that motivated us to establish AMS-IX in the 90’s. So we thought: ‘we’ve done this before why not do it again?’. We’re now developing an IX in Mombasa in collaboration with the Kenyan Telecommunications Service Providers Association of Kenya (TESPOK) to improve regional connectivity.’

When I asked James Cowie where he thought the Internet would go from here he answered: ‘Yeah that’s the part where I stop making predictions. The only thing we can be sure of is that it will be unexpected. It will be something completely different. We’re always wrong. I am guessing the innovation is going to come from all these people in East Africa who are on the Internet and have real need. And it will be something that would have never have occurred to us because we don’t need things, really. We have most of our needs met. So these folks will figure this out.’

Image: Submarinecablemap.com

This article was first published in the Elektor September issue.