FINISHED - 2014 09 02 - WS65 - The Role of IXPs in Growing the Local Digital Economy - Room 8

FINISHED COPY



NINTH ANNUAL MEETING OF THE

INTERNET GOVERNANCE FORUM 2014

ISTANBUL, TURKEY

"CONNECTING CONTINENTS FOR ENHANCED

MULTI-STAKEHOLDER INTERNET GOVERNANCE"



02 SEPTEMBER 2014

10:15

WS 65

THE ROLE OF IXPS IN GROWING THE LOCAL DIGITAL ECONOMY





***

The following is the output of the real-time captioning taken during the IGF 2014 Istanbul, Turkey, meetings.  Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid to understanding the proceedings at the session, but should not be treated as an authoritative record. 

***



>> Our starting time has passed at this point so I would like to ask that people try to find a seat and we will start to get ready to go here.  

>> BILL WOODCOCK:  Okay.  So my name is Bill Woodcock I'm the Executive Director for Packet Clearing House.  To my right, your left is my colleague Nishal.  Here, I've got slides for this.  Nishal runs the South African Internet exchange points and was recently until recently the CTO of the Internet registry for Africa.  There's Sam Paltridge with OECD.  To his right, your left, is Pindar wrong who was the first among the equals of the people who got the Hong Kong Internet exchange started.  As he said this is all about community.  So not about who did the work.  

And to his right your left aerial grazer who runs the Latin American and Caribbean exchange.  As I said my name is Bill Woodcock.  With that I'll discuss briefly the themes we are going to be covering today.  The role and importance of IXPs and encouraging development of local content and developing the domestic Internet economy.  So what role does an Internet exchange point play in the economy and culture of the country in which it's located.  Then the technical policy and economic principles surrounding Internet exchange.  So what are the norms under which this is done?  And last the role of IXPs in making regions economically autonomous.  So IXPs essentially are factories where Internet Bandwidth is produced.  How do countries become net exporters, why are some countries net importers?  Our experts here today will be able to share with you their experiences in a variety of regions.  African region, Asia Pacific region, Latin American region.  And Sam can give a global economic overview of the activity in the area.  We don't have any particular order and no one has prepared remarks.  But I thought that in our hour here we would try to spend about 40 minutes, about ten minutes per person talking about their experiences and issues and save the last 20 minutes or so for an open discussion.  Ariel, would you like to begin?  

>> ARIEL:  Good morning, everyone.  I mentioned a case that we have at the very south of the world in Argentina.  There's a town in the middle of the mountains, the latitude is south 41 and it's 71 east, just to be clear where it is.  Very south and deep.  

This is a small tourist town, a typical mountain with a lake.  And the main activity is tourism.  And they start connection to Internet with 40 megs for all the town.  And they are almost 500 kilometers to the nearest city.  So how we improve this connective?  How we change the reality for them?  And the challenge was to connect to the IXP which is in the north Patagonia in Argentina, and try to get them from the $1,800 per meg for these 40 megs for the whole town so the situation to have more than 500 megs to a cost of almost $30 per meg.  

I'm talking about the wholesale.  The wholesale business and how this affects the town, because at the beginning with the 40 meg for the town, it's almost impossible to connect.  It's more or less 20,000 people, inhabitants in this town plus the visitors.  So it was very difficult for them to rate the IXP because there was nothing for a starter.  And I guess and I think that the very important point is how we resolve the problem of the last mile for this kind of towns.  And we decided to develop a microwave radio connection for almost the 500 kilometers crossing the mountains.  And when we did it they increased the first step from the 40 megs to 150 megs.  And they operate the whole capacity in a day.  

So it was satellited, the link.  Then they realized they need a better connection and we develop a fiber optic infrastructure and in a year we reached the point where the link on the fiber was ready and they increase the Bandwidth to 500 megs.  

And at that point we get in terms of the end users; five times better connections and they now are able to view the systems at least in the 40 megs for the whole town for a whole town was impossible.  So the challenge was to change the reality of this town, connect into an Internet exchange point where the infrastructure and the wholesale Bandwidth is available.  Because for towns in the middle of nowhere like Patagonia you need to reach the infrastructure.  The infrastructure does not reach you.  So this was the challenge that we give to them.  And they did it.  And now they have a very, very good connection and they are developing a solution and a system because it's affordable for them now.  So as an example of how the reality of one small town in the middle of the mountains we can see the difference from the coast and the amount of Bandwidth from one point to the other when the infrastructure is ready.  Just as a trigger to the discussion for everyone, this is the first comment.

>> BILL WOODCOCK:  You're talking about this in the context of a country which has been rapidly adding exchange points.  So if it costs I think you said 1800 US dollars per mega bit.

>> Per month?

>> Per month in the small town originally, that at that time may well have been in the context of a country as a whole that was still importing a lot of its Bandwidth.  But at the same time you're connecting this small town better to the rest of the country and the exchange points in the country those exchange points were producing more Bandwidth in Argentina.  Could you talk about how the prices have come down not only in that town but in Argentina as new exchange points have been brought online?

>> Yeah.  The IXP point was starting in operation in 1997 and with ten members.  And that point very in the beginnings the Argentinian connection for the world.  We came in the early 90's for 64 K, that was the starting point.  $40,000 for 64 K at the early 90's.  Yeah.  When we switched from X 25 IP, that was the starting point of that.

>> BILL WOODCOCK:  And that would have been Bandwidth being hauled from Washington D.C., right?  

>> Yes.  And then we connect to one point in the -- I guess at the end of the '90s we get one point to Florida.

>> BILL WOODCOCK:  Yeah.  So and in Florida obviously there's a large exchange point called NOTA that serves a lot of Bandwidth to Latin America intraregional competition led to the exchange in Sao Paolo being started in the same month in 1997.  Could you talk a little bit about how long it took before Brazilian ISP's participating in Buenos Aries?  

>> We have different models.  The first thing is the IXPs in Argentina are all interconnected by ourselves because there was a situation with the carriers that they don't want to connect the IXPs.  And so we did it.  In the Brazilian model, they are not interconnected and the carriers sell different traffic between them so this is the main difference.  Now we have 15 in Argentina IXP and Brazil is almost 22, yes?  Or more now.  I don't know, maybe 25.  And the model is different because also the model is different in Brazil.  You participate in one exchange point and there is no obligation to exchange traffic with all of the members.  And in the Argentinian model we have an agreement if you have connected to one exchange point you must accept all the traffic for the rest of the members of all the Internet at this point and put your traffic for all of them.  

>> BILL WOODCOCK:  Nishal, could you discuss your experiences in South Africa and in Africa general and perhaps pick up in the area of multi laterally and what you've seen there.

>> NISHAL GOBURDHAN:  Thanks.  Good morning, everybody, my name is Nishal.  I'm also responsible for three Internet exchange points in South Africa and I work closely with a lot of the exchanges in our region.  We have a slightly different model just drawing on what we do.  We have always believed that the model for peering should be on an as determined by the ISP perspective.  

So we operate a highly‑available neutral exchange platform and people that connect to the Internet exchanges are free to choose who they want to appear and how they want to appear with them.  We are slightly different from the start.  That was 1996.  Now we have even allowed people to sell transit if they want to.  Our focus has been solely on running a platform and let people interconnect and provided they follow the nice polite rules, don't do the nasty, technical things that are in place.  You can literally do whatever it is that you want.  We are not quiet, quiet the most southern part of the world.  I think that goes to you.  But Cape Town, for example, is pretty far down there if you know anything about geography.  Coming from such a southern part of the world establishing Internet infrastructure.  If I had to work out the return of investment, in fact I was doing the math when my colleague was speaking, I think I would put it down to two weeks.  So in two weeks related to the exchanges we manage to pay off what we otherwise would have been spending on at that stage transmission costs to get our traffic transferred and then it was in the United States via Sprint and BBN if I remember correctly.  It wasn't in Europe at that stage.  So I would definitely mark it off as totally a gain for us.  Yeah.

>> CHS has a survey that's still in progress but basically what we have observed is that 90% of Internet exchange points are built for a cost between $4,000 US and $40,000 US and almost all of them pay for themselves.  Somewhere between two or three days and two or three weeks.  It's very rare to find an exchange point that doesn't earn back its initial investment within a few weeks.  Pindar, do you want to spend time talking about your early exchanges?  

>> Let's start off with the difference here is that many people have the concept of Internet rather than just providing service.  You have to go through the service of Internet connectivity.  In the early 90's it was a very, very competitive significant it did cost $50,000 ‑‑ $40,000 actually to get from Hong Kong to the United States for 164 kilo bit.  It was a period at that point in time the fact that you had a bigger pipe going across the Pacific was for commercial advantage so the one with the deeper pocket would get multi honed across the Pacific.  It was only until I think 1995 with the decommissioning and also there's controllers in Hong Kong that run IXPs didn't need to have a license.  In Hong Kong we have one of the world's best access for broadband and what I would like to focus on is not so much the economics.  I think keeping your currency in your country and not shipping it overseas is of interest.  This morning I wasn't quite sure why the video feeds ‑‑ is there an Internet exchange point in Turkey?  

>> Yes, there is but it's relatively small.

>> So the experience of trying to keep traffic local to Hong Kong we also have content industry which uses Hong Kong Internet exchange point run by the local university so at that point in time we couldn't trust each worry, we are all commercial competition but we could just a third‑party which until this day runs the exchange point.  The good news they only came to ask for funds for the government until after 50 years.  At that point in time it goes to show the recuperation of the costs can be done if there's community support.  What I would like to point to is the social and economic dimensions.  If you want to drive a digital economy you need the engineers to do that.  And those engineers need to know the Internet physics of how the Internet is actually built.  They need to have full command of that.  You're going to have a lot of mistakes like we did in Hong Kong.  One way to get that experience is trial and error and to do so at a rapid iteration.  We need to try to keep local traffic local and also increase the usability of that.  Within Asia it wasn't quite clear exactly how to build one of these.  We have in Hong Kong in 96 the first meeting that we taught was how to build an Internet exchange point and we built an Internet exchange point in the hotel.  The point is how do you have a project whereby it's easy to see the economic benefits but it's not easy to see the social benefits that have an operator's group that gets around regularly to meet and discuss the Internet exchange point.  Did it really discuss the Internet exchange point?  Yeah, perhaps for about five minutes, and then talk about something else.  

All of our different roles change.  In those early days there was another gentleman, Charles Mark who is sitting right there, he's our IT legislator.  Way back when we were competitors and I hated him and he hated me because we were competitors.  We are now sitting around the table wearing different hats.  These engineers will change companies, will change jobs; but their trust within each other over the years, that really cements the community.  As their roles change it's that community which will drive a society.  Where we have moved from Hong Kong as much as talking about Internet exchange points but looking for other forms of exchange.  In Hong Kong over the last year we built five international property exchanges.  

The concept here is quite similar which is where the money is.  We have gone up the stack routing that property here.  So the point being is how do you build the engineering trust to build your local industry if that's what you're going to do and I would suggest an Internet exchange point if we haven't got one is a very good starting point.  However there's a down side.  Last year at the IGF you mentioned post Snowden.  One of the things out of the Snowden revelation if you gather all your aspects out of one spots that a good point for mass surveillance.  My question is where are the boundaries?  Any equipment that goes into an exchange point needs to be basically vouched for by the manufacturer.  And there needs to be blow back because that whole incident in Hong Kong I was upset about.  It breaks the trust.  And that trust is what I think is most important about building and driving the economy not just trust within Hong Kong but that's what is important.  Thank you.

>> BILL WOODCOCK:  I think the note that Pindar brings up about trust and the community between the engineers who do this work is important.  One of this things we see going into a developing economy is don't yet produce their own Bandwidth and are not thriving is the Internet service providers are all at each other tooth and nail and none of them trust each other because they are competing directly against each other for customers and they don't really share any common interests other than maybe some legislative or regulatory ones.  But Internet Bandwidth is produced collaboratively; Internet service providers come together at exchange points to produce the Bandwidth.  Once they realize that, once the ISP's realize that they have to make this huge mental shift from the other ISP is my competitor who I do not trust and will not break bread with, to they're the people with whom I collaboratively produce the product I sell to my commerce and if they sell to another customer I win and if they sell to another customer they win.  One of the things we saw in the US is that as soon as we all figured this out we started going out for sushi together on Thursday nights.  You have this intense dynamic to a very community‑based collaborative one.

>> If I can illustrate this further because you think the other competition is the other ISP but what we didn't realize was it was actually one of the customers.  It can be that your downstream customer grows so large to eat you up or become your competitor.  The competition is everywhere.  You want to be very, very focused on where your value is and it's probably not going to be on the cost unless you want to aggravate really large really quick.  I want to touch on D‑peering.  

>> I want to mention also that not only the ISP are part of it, content providers are very important for us.  Content providers means for us not to need to not go outside our country to get the content.  So develop content inside our countries is very important to reduce the costs and increase the quality and improve the quality.  And so content providers like universities, like any producer of media are very important members and in our case are connected to our Internet exchange point.  

>> BILL WOODCOCK:  And that can be television stations, local newspapers, all kinds of things like that that in a global marketplace may not be competitive but if you have a local exchange point suddenly they have an economic advantage over the global providers like CNN or the New York times.  Nishal?  

>> I wanted to add to that point about local content creation.  You'll listen to lots of talk.  There's an 80‑20 proposition by the African union to get 80% of Africa's housed in Africa.  I don't know how familiar most of you are with Africa, if you look at Africa we are literally buying services from the rest of the world w the exception of the three more developed economies, and that's Egypt, Kenya and South Africa.  The one thing they all have in common is they have all been running well‑run Internet exchange points for a very long time.  If you look at the participants at the exchanges, one of the largest at the Kenyan is the Kenyan revenue society, the ones that collect tax, the ones that we love the hate.  They play such a big role that they have pushed the exchange point operators to diversify their services and pushed for the addition of a second site so they could be always on so people would always be paying their taxes.  I know, like I said, love to hate, but it's working for the Kenyans.  One of the busiest peers we have at the Johannesburg exchange point is a bank.  A bank because it's convinced that its future lies in providing e‑services.  You have no hope as you said of being able to compete in the global economy without actually having and running well‑managed local Internet exchange points or extending that into more connected economies around you.  That's exchange points.  

>> BILL WOODCOCK:  Sam, I wonder if you can tell us about your experiences with carrier interaction and perhaps lead into a discussion with Pindar about peering and why people connect and choose not to.

>> Good morning.  I wanted to make a point about local content.  We have been doing these workshops for quite a while.  One of the most interesting stories was early on.  It was after Packet Clearing House helped establish an Internet exchange point in Nepal.  Just by the by, it was mentioned what the day with the highest traffic exchange was during the first year.  And it was when the local high school results came out.  Now if you think about it, before that time those local high school results would have gone via Paris, London, San Francisco, without any real need to do so.  And the cost of that was passed on to the Internet users.  So as I listened to the people on this panel that have much more experience with establishing IXPs because I'm probably the interloper that hasn't established an IXP, there's one on self‑governance.  So various tweaks to the model but good governance model to establish Internet exchange points with a best practice model and it's really built upon self‑governance and that's what they're talking about when they're talking about sushi.  One thing you hear here is not about how much money we are going to make.  

Across all the IXPs and the whole Internet ecosystem it's built around customers at the end of their model paid for whatever the service or whatever the activity is.  But this discussion today is about how we reduce that cost, how do we get the cost down?  And that is built around what is a very successful model for Internet traffic exchange.  And if you look at how the Internet has scaled, it's because of that successful model where everyone is trying to reduce their costs.  These guys are trying to pass their costs to other people; they're trying to reduce their costs.  The danger to that is and this is Pindar's point about the honey pot and he mentioned security issue, because there's also the area of how this money flows.  And it's also a honey pot for other people because they see this as one of the potential bottle necks or choke points or where ever you can find one on the Internet.  Try and apply governance externally to that activity and somehow try and extract rents from that.  

And that's one of the main dangers facing Internet exchange points.  Think of been so successful because they do have a self‑governance model.  But the threats to that come as more attention as more ‑‑ actually attention.  It comes towards the Internet as it becomes such a central part of the economy and that could be the issue of surveillance, it could be how the money flows, it could be the Internet supplying the old telecoms network and when they converge you have this whole debate about trying to take previous models for telecommunication exchange of traffic and apply them to the Internet.  

And you can see them all the time.  You can see I believe in the Caribbean this year, because people saw that as somehow bypassing what they regard as appropriate termination.  You can see it in countries where they try and create white lists for VPN's.  All this is about trying to force that traffic through either an international gateway or an Internet exchange point or some other place where you can apply a different form of governance.  And I guess that leads into the discussion about the peering and so forth.  Over to you, Pindar.  

>> PINDAR WONG:  Sure.  Well, I think two words.  De-peering and renasis (phonetic).  They have great studies that illustrate the political and business dimension of when it made sense for us to peer initially but then I realize that I'm getting a bad deal or it's not in my strategic example and I suddenly de-peer.  If that only affected the two of us that's probably fine.  But sometimes it was done ungracefully and has cascade affects.  So it's not all one way.  As long as we agree to agree and peer, sometimes it can go terribly wrong from a de-peering aspect.  So that's it.

>> BILL WOODCOCK:  We have seen a lot of politics around that particularly in the United States in the last six months from Netflix which is one of the largest content networks in the world versus Comcast which is one of the largest eyeball networks.  The largest eyeball network in the United States which is Netflix's primary market.  So there's disagreement about whether Comcast is exercising an abuse of market power because they have a near monopoly on the last mile, whether Netflix is somehow trying to take advantage of Comcast's infra structural investment without making a similar investment of their own which that's kind of a misunderstanding of how the Internet gets built.  The exchange point is where networks meet and so unlike a network where a payment might cross an exchange point, in the Internet the exchange point is where the buck stops as it were.  

Another thing that Sam brought up is the danger of excess rents being, extracted at exchange points because this is where the value is coming from.  People who look at money carefully, they rob the banks because that's where the money is.  So just a quick anecdote about that.  Large developed economies; India is one of the largest ones that produce a tiny, tiny fraction of the Bandwidth that it uses.  It still imports almost all of that, produces very small amount domestically.  A big part of the reason why India is so far behind in developing the production capacity is because in 2000 many of you may remember a company called Enron.  The first en change point in India was built by Enron and they thought they were building an Internet trading platform for Packets.  And so of course India winds up sort of ten years behind as a result of all of the kind of misconceptions about traffic exchange that came out of that.  Fortunately the rest of the world Internet service providers have kind of stuck up for themselves and worked collaboratively.  Pindar?  

>> I think you brought up an important point.  I came back from a session about Internet terminology.  The question in the back of my mind is why is it important for policy makers to understand Internet physics, because it can have long‑term consequences if your policy is wrong.  

>> Sorry, I just wanted to give an operational comment.  Could you please speak to the microphone.  It is very difficult to hear in this noisy room.  Thank you.

>> BILL WOODCOCK:  Sorry I realize we are up against a coffee break outside.  We are I guess a bit more than halfway through our session.  Perhaps we can enliven conversation with some comments and questions from the rest of the room.  Anyone have anything?  

>> Thank you.  The cost of the Internet exchange point is not so high, not expensive.  But if you look to the global, many countries do not have the exchange points.  And who should lead for the establishment of the exchange points?  The countries?  Governments?  Internet organisations?  ISPs?  I don't know.  Thank you.  

>> Can I understand your question correctly you said the cost of establishing the Internet exchange point is not high and who should set that up, is that correct?  I would disagree with you.  The physical cost is not high.  The trust cost is huge.  The level of trust you have to build between the ISPs who are competing to say okay let's start an Internet exchange point, that cost is very high.  

And in Hong Kong as I mentioned we did not trust each other, we were in competition.  But we all agree that my enemy is my friend.  And so we kind of agreed that the local university ‑‑ in Hong Kong's case was the one to trust.  Before the commercial Internet service providers in Hong Kong was the academic networks so I dropped out of university at that point in time to do the commercial stuff.  So the academic networks new how the commercials worked.  So I would say rephrase the question of who in your local community do you all agree you can trust?  In some cases it may be the government.  In some cases the government is the last thing you want.  And it's very difficult to answer that question because the politics and there is politics is always local.  

>> I totally agree.  Let me tell our history.  The physical cost was just a switch for us at the beginning, but it took more than a year of 12 different ISPs sitting around a table discussing the way to start.  And this was the extremely huge cost.  The discussion over more than a year, 14 months, discussions, once a week, from different ISP in the main city of Argentina, how to start around putting the rules on the table that all will agree.  That was the main point.  The second one that we opened took us nine months.  Now we are running 15.  It's fastest, the process, now.  We learned the lesson.  

>> BILL WOODCOCK:  So just speaking from my own organisation because this is the primary thing that we do, we have been through this process more than 200 times now.  And what we have observed is that most exchange points take at least eight months to get started, eight months of the Internet service providers talking with each other and reaching an appropriate level of trust and also investment in the outcome that they will have confidence and they would invest themselves in the process of building.  

Even though the monetary cost is very low as you've all noted, the internal cost for a network is much higher than the shared costs so you have to build out your network to reach that exchange point.  If you don't of confidence that after you've made that investment the exchange point will be productive for you, then you're unlikely to make that investment in the first place so that's one of the big hurdles to overcome the initial trust there.  We have seen it happen in as little as 3 or 4 months but it's very rare.  We have also seen it take 5, 6, 7 years before the ISPs come to trust each other enough.  You ask who should take the lead.  Ultimately it's the ISPs, the network operators who have to reach that trust and work together.  There are places where they simply have not yet gotten it together to do that.  One interesting example, Jamaica just got their first exchange point in the last month.  This was the end point of a process where 6, 7 ‑‑ okay.  Three years on this particular effort.  But I remember we were doing workshops there 6 or 7 years ago and that was a process led by the regulator.  I remember the first time we went in to do a workshop the government, the regulator brought in all the entities and sat them down and said listen to this.  They said we are not going to listen to this.  So, you know, there are very different entry points for this conversation.  

>> Can I draw on this point, because I think one of the beauties of the Internet Governance Forum is trust building.  Within the Internet physics you have the ITF and all those groups doing standards but they have to understand before they can go home and implement things.  How you have a trust process is how I use the aspect of the multistakeholder approach.  If I see the people again and again I know where they're coming from I may not agree with them but over time you can build trust.  There's a very clear business model, clear proposition that makes that hopefully quite easily.  And we have been discussing this for 15 years.  

>> BILL WOODCOCK:  Next question.  Sir?  

>> AUDIENCE:  Hi.  My name is Adley, one of the regional IRR's in Asia Pacific.  I would like to go back on the topic of capability development which is one of the requisites for building IXPs.  They're quite expensive, not only in terms of getting the technical know‑how to come to the training but also in terms of requiring people to travel to places to get the knowledge itself.  One of the things that we find quite useful is actually in terms of trying to get knowledge across to a lot of people on a low‑cost basis is one partnering with the local network operator's group which also sometimes means helping the local community to build because this will allow you to accomplish few objectives, one is to create that trust.  So get people from different places, different companies to sit together and discuss how to make things better but also at the same time we can address the issue much getting the knowledge across on a very economical basis.  

There are many countries where recently they started knocks, so this year I think there was one UNOG in Indonesia.  If they can't establish their own NOG they would go to regional one so we find there are places you can get these messages across and help support with relevant information or knowledge.  And on the other thing with regard to distributing knowledge we do provide E‑learning on a weekly basis so on topics like routing, security, the success or the measurement for success is very hard to see.  But of course this is also another avenue for people to acquire knowledge.  There is value in I think supporting the NOG's and getting them in this type of discussion and also this would facilitate the development of IXPs and the development of access to Internet in the long run.

>> BILL WOODCOCK:  I couldn't agree more strongly.  We teach about 120 ‑‑ the organisation that Nishal and I work for teach about 120 workshops each year where we can get a bunch of people in from different places to share information.  I know there's NSRC also does training of that sort.  Nishal or Malcolm, do you want to talk about European outreach efforts in that area?  

>> AUDIENCE:  So, I'm from URX.  It's an association for Internet exchanges.  What we do ‑‑ I mean there are lots of initiatives in Europe already working together actually with other bodies to do some outreach.  Not necessarily training but just more case of passing knowledge, explaining from ‑‑ going through what the experiences that we have had and seen throughout Europe.  Like you were talking about earlier, like trust and community building is key.  The Internet exchanges that come about which are community‑lead, there's one in the UK, IX leads, for example.  Leads is a very small city and they felt the need there was enough network operators that came together and felt the need for an Internet exchange.  It's now been running for over two years and very successful and growing.  So talking about the trust and the community aspect is really important and bringing the community together and doing that by NOG's, other things like the RIR, URX we have two meetings a year as well.

>> BILL WOODCOCK:  Thank you.  We have got some questions online and maybe I'll come back to you.

>> AUDIENCE:  I just want to add some note about this.  Our experience in Russia, we have dozens of XP's.  And the largest one has like 400 members.  So it's pretty large.  And each of IXP I know in Russia on some stage became the enabler of local Internet business.  If some city or small town has own Internet exchange, in this city we have first local providers.  And new local providers, because they understand how to do this business.  ISP allows them to understand this business easily and this is very important for us.  And local Internet business.  I can illustrate on very similar cities in one we have IXPs and in the other city nearby we don't have IXPs.  And in the first one we have strong content providers and in the second.  No.  So this is way to enable our business.  It's very important point for me to understand it.

>> BILL WOODCOCK:  And do you find that data centres have been springing up around exchanges as well.

>> AUDIENCE:  Yeah.

>> BILL WOODCOCK:  Which is where the content ‑‑

>> AUDIENCE:  Yeah.  This is usually data centre which will not exactly to IXP but one of member usually large member that came from Moscow or something like that because, you know, it's huge money to build data centre itself.  So data centre usually belongs to one of members.  But for largest IXPs and so on, they do have own large data centres of course.  

>> BILL WOODCOCK:  So I think obviously on opposite sides of a conflict right now Russia and Ukraine are both really interesting because both countries if you look on a per country basis, Russia and Ukraine are moving up on band width and they're on the top of the list right now and that trend appears to be increasing.  One huge exchange point in a country and a bunch of very small ones whereas in Russia and Ukraine you find a medium number of exchange points and a medium number of Internet service providers and that creates a much more competitive environment than you find in western Europe or the US and that appears to be driving a huge amount of growth.  So Russian and Ukraine are shooting up in the amount of Bandwidth.

>> AUDIENCE:  Yeah, you're right.

>> BILL WOODCOCK:  Remote questions?  

>> AUDIENCE:  Fazil from Bangladesh, there are examples of exchange points based on software defined networks?  

>> BILL WOODCOCK:  Well, so in Japan there are both commercially operated exchange points and there's also a very, very healthy academically operated exchange point by operated by Wide.  And Wide experimented with visual exchange based on MPLS and that's about as close as we get to software‑defined exchange points.  People are playing with it but that was actually routing production traffic for a while and was not particularly successful.  There's the question of sort of security and stability.  And it's getting better and better.  For now though in an Ethernet switch not a lot of complexity is needed and people trust the idea of just a piece of hardware that has well‑understood code on it.  

The same time we see more and more open source operating systems for switching and routing running on top of open source hardware so we are headed in that direction but it's not going to wind up being like the virtual switch that comes in a VM wear hyper visor.  It's going to be something more purpose‑specific than that when it hits.  You looked like you were about to say something?  

>> AUDIENCE:  IXPs have been talked about I think for quite a while now in the last couple of years you were hearing about XDN's and IXPs.  There is one currently being built in Toulouse, it's due to be completed in a couple of months so it will be interesting to see.  We don't know how it's going to work out or what the status of that is going to be like but we will find out in a couple of months.  

>> BILL WOODCOCK:  400 exchangers out there in the world, there are always experiments happening and always examples of success and failure.  So if you don't want to experiment, if you want something that will be successful, the trick is to go and look at examples of success and see how it is they got started.  Don't copy what they are right now; copy their method for getting there.  Pindar?  

>> AUDIENCE: I just want to observe I really enjoyed that last question about SDN's.  Networks are difficult to upgrade wholesale so yes there's a very attractive promise but I'm not aware of any SDN exchange point or if SDN's develop to their promise we are looking at a different Internet at a whole.  Will it be the same engineers who built the exchange point?  Probably.  

>> BILL WOODCOCK:  Malcolm?  

>> AUDIENCE: Thank you, Bill.  Malcolm from the London Internet Exchange.  Bill, before we had those last two questions you invited me to come in on some issues of building exchanges from the first steps and the trust and so forth.  And so coming back to that, I mean, for those that are not familiar with links, links is now enjoying 20 years, its 20th year was establishing Internet exchange was one of the largest exchanges in Europe.  But we are very committed to the idea that it needs to be a bottom‑up process.  And still very established and substantial by any measure exchange we still are very committed ourselves to being run in that bottom‑up model where our own members are in control of what we do and don't do to be neutral between all our members and so we cannot be a strategic competitor to our members.  

And also so we are not becoming part of disputes between our members about their commercial models and so forth.  We are just there to facilitate the exchange of traffic.  We have recently in the last couple of years engaged in a programme of establishing new Internet exchanges in cities of the United Kingdom.  We established an exchange in the city of Manchester in England, about 50 participants.  More recently one in Scotland, and one thing that I think might interest this meeting is we were approached by the government of the developed government of Wales about could we help establish an Internet exchange point in south Wales on their behalf?  

Now, our answer to them I think may interest you.  It was we would be delighted to do so but... and it's a big but.  It can only work if this has the whole hearted buy‑in of the network and peer community.  So any active work that we could do would be only that which was designed towards building and establishing that support and we weren't to put in switches or building a data centre or anything like that until we were sure there was a critical mass of networks that wanted to us do this.  This was challenging for a government body that had a budget that needed to be spent by end of year.  We didn't really need the money or care about the money.  As you say, the equipment is cheap.  Certainly the equipment for links isn't cheap but the equipment for an IX in south Wales would be cheat but the investment is exactly what you said about building a community that want to work that way.  

Other areas of outreach that links has been involved in is largely towards explaining our process, our experience to others to establishing mutual exchanges so they can learn in as much as they think is appropriate to their own situation from our experience.  And there we would say self‑governing membership models are very important.  We believe that really helps establish that trust because it gives the potential confidence that they are building something that will always be responsive to them.  Internet exchange points are a scale business.  The bigger it gets, the more you want to join it.  It's very difficult to move away.  There's no reason to move away.  

So before you join something that started, you really need to have the confidence that this is always going to be working for you in your interest.  A membership model there that is totally committed to neutrality to its members that has the members in control of the organisation and in control of the all of the organisation we believe is crucial to that success.  As a result of that I know there are a good number of IXPs around the world that have simply taken our constitution and applied it with their own little mode and so forth from.  Time to time when we hear this happened and we make our own updates or clarifications to our constitution we to have send out an e‑mail saying just to let you know we found this in the constitution, it's up to you what you do.  It's your constitution now but there you have it.  Anyway I hope that was of some interest to you.  

>> BILL WOODCOCK:  I think we have time for one last remark or comment?  

>> AUDIENCE:  Yes, I want just to share with all of you that two years ago we signed the Internet exchange federation, Asia Pacific IX, and then Africa is joining us and I expect the North American association will join us in the near future just so everyone knows that.  

>> BILL WOODCOCK:  Any other comments or questions from the floor?  Okay.  Well, thank you all, very, very much for participating this morning and I'm sure our panelists would be happy to answer any further questions you have informally or if you catch them throughout the rest of the week.  Thank you all, very much.  

(Applause)



***

This text is being provided in a rough draft format.  Communication Access Realtime Translation (CART) is provided in order to facilitate communication accessibility and may not be a totally verbatim record of the proceedings.

***