The following are the outputs of the real-time captioning taken during the Thirteenth Annual Meeting of the Internet Governance Forum (IGF) in Paris, France, from 12 to 14 November 2018. Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid to understanding the proceedings at the event, but should not be treated as an authoritative record.
***
>> EDUARDO BARASAL MORALES: Hello. We ask you to be a bit more patient because we have some technical issues with the slides and with a remote participant. So in a few instance we shall start.
(Pause).
>> EDUARDO BARASAL MORALES: Hello, everyone. Before we start I would like to thank you all for coming to our workshop. And also I would like to thank the panelists for accepting our invitation and above all, I would like to thank NIC.br, CGI.br and IGF for letting us discuss this here.
Well, my name is Eduardo Barasal Morales. And I'm a computer engineer at NIC.br, a non‑profit organization that tries to improve the Internet quality in Brazil. And I will be the Moderator for this Round Table. Online participation, Nick will be Moderator. Our Round Table is composed by Antonio Marcos Moreiras who will represent the technical community and then we have Lee Howard who will give an Internet service provider's perspective, and up next is Klaus Nieminen and then we have Barbara Prado Simao who will bring the perspective of digital consumer rights. And finally Darrin Veit from Xbox who will present a gaming company's perspective.
So let me show you how this workshop is organized. Our Round Table will start with a lean introduction to the theme of migration, IPv6 and IPv4 address exhaustion and online games. And our panelists will have up to five minutes each to introduce themselves and their points of view about the problem. After that we will have 15 minutes of open microphone both for the audience and remote participants.
Next the panelists will have another five minutes to discuss solutions and possibilities for collaboration. And finally, we will have 15 more minutes for another open microphone.
So let's start with the problem. In our country Brazil we have a lot of Internet service providers. Almost 6,000. In this year many of them have reported connectivity problems with online games. Our team decided to investigate this problem. And we discovered that many online games don't use IPv6. And some of them even haven't started to plan this migration. Apple makes them use IPv6. And we also discovered that most of the games requires good connectivity and some of them requires incoming connections.
From the Internet service provider's point of view they are suffering from IPv4 exhaustion. In some regions, ISPs cannot receive any more IPv4 addresses. In other regions like North America they can buy it but it is too expensive. And because of that they have started to use CGNAT and the difficulty in finding the right proportion between clients and port. So as you may see we have a problem here because online games companies only use IPv4 and the ISPs are suffering from IPv4 exhaustions. What's the best solution? The perfect solution is to deplore an IPv6 which has many advantages. It is faster than IPv4 and it allows end‑to‑end connectivity and each user has their own public IPv6 address. All because it doesn't need any network address later in the middle of the communication. Like CGNAT. However to use IPv6 we need both sites to deploy ISP in online games, because I am a machine. IPv6 will only talk directly to another machine that has IPv6 and a machine with IPv4 will only talk directly to another machine that has IPv4. That's a compatibility issue. That's ‑‑ and that's why we need to improve IPv6 deployment. Why IPv6 is not widely deployed? We need to discover what problems there are with IPv4 and games. So that's why we have some questions here that I hope our panelists could discuss.
First do game companies use Blacklist or any kind of an IP filter? Imagine how many clients might be affected if you filter the IPv4 address from CGNAT. If you do that you will be filtering the whole ISP. Second, how many ports should a client receive? We have heard that some online games use more than 1,000 ports. So if an ISP gives 2,000 ports to a single client that number might be enough. Third, do online games usually require incoming connections? And four, how worse is the double NAT connection? So now we are going to hear our specialists talk about this problem. So we will start with Antonio Marcos Moreiras.
>> ANTONIO MARCOS MOREIRAS: Thank you, Eduardo. Very well. I'm Antonio Marcos Moreiras. I'm a project and development manager at NIC.br. As Eduardo already said NIC.br is a private organization, not‑for‑profit related to the Brazilian Internet steering committee. We manage the .br domain names. I am a computer engineer. And I have been working with IPv6 dissemination in Brazil for the last ten years. Next slide. Okay.
Very well. So ISPs approached me and my team at NIC.br and asked our help regarding one problem that was bothering them a lot. Their customers who are complaining about the gaming online or more precisely about not being able to play online. These ISPs were using CGNAT for sharing IPv4. And they were also using IPv6, but some gaming platforms seem to have problems with both technologies. So we started to dig in to these issues. It is not a problem unique to Brazil. Maybe it is more evident because our market for Internet is growing, is growing very fast. There are a lot of new small ISPs in Brazil. They have to share IPv4 addresses in some way. We know that similar situations could come to smart TVs, security cameras and other kind of home appliances and Internet applications in general.
I would like to give some data about ISPs in Brazil that you see in this slide. Brazil is a very big country. With more than 5,500 cities, municipalities. Less than 10% of these cities are attended by the major Internet providers. We have more than 7,000 ISPs. Some of these ISPs are very small. We call them regional ISPs. 70% of these ISPs have less than 1,000 subscribers. Sometimes some of these regional ISPs lack technical knowledge and sometimes this can be part of the problem. So I'm not here just to blame the game platforms. Part of the problem can be of ISPs sometimes.
We conducted a small survey in the last two weeks asking the ISPs a series of questions about issues with games. And we had 172 answers. 172 ISPs answered the survey. 90.2% of them say that this issue is a relevant problem for their business. And a bit more than half of them are already delivering IPv6 to their users.
But what's the issue anyway? Very well. The Internet infrastructure nowadays is working basically in paperclip and hot glue. Okay? It is dependent of a very big work around that is CGNAT. We don't have more IPv4 addresses. It is impossible, at least not viable. They are sharing IPv4 addresses, normally what we call CGNAT. They are also using IPv6.
When we have a problem, we have a problem if first, the gaming platform or the application platform in general does not support CGNAT. For instance, it needs common connections or if it cannot recognize two different user sessions coming from the same IPv4 address or if it has poor criteria, blacklisting IPv4 addresses is used in the NAT pool of the ISP.
Two, second situation, when we have a problem, if the gaming platform lacks also IPv6 support. And the third situation, if the ISP has a broken IPv6 implementation or if the ISP has a broken CGNAT implementation. It is our understanding that how these three situations are happening simultaneously.
Very well. Most of the complaints are about Sony Play Station or PSN. But we also have complaints about other platforms such as Xbox that do have IPv6 support, LOL. Tibia, Cabal. We invited Sony to this workshop, but they declined explaining they could not discuss publicly their roadmap of future technologies.
It seems that they were talking about the IPv6 when they told future technologies. Maybe it can be their future, but it is present technology for at least one in each four Internet users in Brazil.
And this number is growing fastly. Sony is answering Brazilian user complaints by saying I will read it. The words could not be exact because I translated it in Portuguese to English and we received the same response from some different sources.
So Sony is saying Play Station is not yet compatible with IPv6. However your Internet provider must offer a public IPv4 address if you request them. Or they should offer a platform that uses IPv4 and IPv6 at the same time. In Brazil only 8% of Internet providers are using IPv6. So Play Station is not yet using IPv6. In the U.S. only 25% and here we still do not use IPv6. All Internet providers have until the year 2020 to switch from IPv4 to IPv6.
Very well. I have no idea where they get this major ‑‑ magical date of 2020. Seen by the point of view of ISPs this is very, very bad, this answer I mean. Because the ISPs are doing the best they can. They are delivering IPv6. They are doing the decision the best way they know. And the game platforms say should their customer that they are blaming because they have to take new IPv4 addresses from some magical hat until 2020.
So that's it. That's the problem in my point of view. Please.
>> EDUARDO BARASAL MORALES: Thank you. Let's continue with Lee Howard.
>> LEE HOWARD: Hi. My name is Lee Howard. That last quote was just shocking and wrong. 50% of U.S. has IPv6 and Google says 45%. 25% is worldwide.
>> ANTONIO MARCOS MOREIRAS: Yeah, worldwide.
>> LEE HOWARD: So my name is Lee Howard. I have a small company, working on making a larger company that does IPv4 ‑‑ IPv6 translation. We also do some consulting and training. And why I got involved in IPv6 because I was responsible for the IPv6 deployment for one of the world's largest IPv6 networks and Working Group co‑Chair at V4 ops Working Group for IPv6 operations at the IETF. When we started deploying IPv6 management said we must not provide a bad user experience. We need to measure how people are using the Internet. We did a lot of performance testing. Comcast provided a speed test where you can compare the download and latency over IPv6 and IPv4. This is a screen shot showing that we are getting higher throughput. APNIC did some measurement comparing the roundtrip time and 2012 they found that IPv4 was slightly move or ‑‑ and a little faster than IPv6. And in 2013 IPv6 was a little bit faster and that continues to be the case.
I wanted to show there are lots of different measurements showing what's going on. Cable labs did some studies. Let's see, is there a difference between IPv4 and IPv6 performance in the lab? IPv6 is a little faster. And they also tested two layers of network address translations. One at the home and one at a carrier grade address translation. And IPv4 got faster when it went through two layers. That was bizarre. Cisco did tests similar to Jeff Houston's tests showing there is a slightly ‑‑ more often IPv6 is slightly faster. This was mine. We did some testing. I put probes in to ten different hubs around the country. I did a TCP roundtrip. IPv4 was more than 20% faster than IPv6. In general either a website is faster or slower over one Protocol, or in general one of our hubs, the columns are ‑‑ our hubs for one of our hubs was slower over IPv6. It turns out that routing matters and there is congestion. And you move traffic around so that you can relieve that congestion. It turns out sometimes the engineers don't remember to move both IPv4 and IPv6 traffic. And therefore one protocol family ends up being happier than the other. They tend to normalize.
Academia has also done some research in to IPv6. In this one it is a little hard to see here. But most of the time, most of the top websites were faster with IPv6. And actually a very high percentile was within one millisecond or faster. And the happy eyeballs timer of 150 milliseconds gives you a really nice sort of performance measurement.
Apple later did some testing and they saw that 90 milliseconds was a sweet spot. Akamai did some measurements in the median. Selected websites were now to be fairly tested. For this particular test they tested one device on one mobile network in the U.S. because that's where they had the best instrumentation. And they thought in general IPv6 was in the median, IPv6 was 5% faster. And in the 95 percentile it was 15% faster. So there is consistent reporting that IPv6 is just a few milliseconds faster.
This was a more scholarly paper that they did. The dark line is the IPv6 page load time. And the dotted line is the IPv4 page load time. There are only so many ways that you can put graphics on a slide, but the dark line is to the left in every case. So the scale shows that generally just for Web page load time we see one or two seconds of faster speed for downloading websites. If you maintain a website and you can make ‑‑ you can get it to download a second or two faster you know that's money. Facebook has done similar kinds of performance. They saw IPv6 being 30 to 40% faster. They have kind of backed off of this in later studies. There is evidence that there is statistically a significant advantage over IPv6 but no longer willing to necessarily stand by the 30 to 40% statement, but I don't have a graphic for that.
And LinkedIn provided some measurements over a bunch of different networks in several different countries. And so this is the percentage faster that a Web page downloaded on various different networks. So these were U.S. networks. And it goes from about 2% faster to 20% faster. The UK was 12, 15, 20% faster. Germany significantly faster. Since we are in France I thought you would be particularly interested that it was 17% to 40% faster. If you can improve your speed by 40% that's something that everyone is interested in and especially gamers. The most recent and ongoing measurement we see is from Jeff Houston at APNIC where he is doing side by side measurements. And he shows there is a significant difference, how significant it is and it varies significantly by network, too.
And so something that is hard to see in most of these slides but I am sure online available later, I have included the source for every slide down at the bottom, but in many slides it was covered by the transcription. So you can download the slides and find the sources yourself. And you can also go test speed for yourself. And you can test whether IPv6 is faster than IPv4. You can't do that now because the IGF network doesn't have IPv6. That's it. Thank you.
>> EDUARDO BARASAL MORALES: So thank you, Lee. Let's continue with Klaus Nieminen.
>> KLAUS NIEMINEN: Okay. Good afternoon, everyone. I'm Klaus Nieminen from the Finnish Communications Regulatory Authority. And I have been sharing the national IPv6 Working Group for some years. First I had to ask from myself that do we actually care about gaming. I mean the gaming is not our top priority, but I think it is a pretty good example of Internet applications that can benefit from IPv6.
So I'm actually going to the story that why do we actually care about the IPv6. We believe that the IPv6 actually offers a better Internet for users mainly because of the end‑to‑end connectivity. According to the European TSM regulation, for example, the end users to use and provide information services, applications and content. And basically if you are behind NAT, it is much harder for yourself to actually start providing services and applications. So we have some legal background also to this topic. However I mean we are not watching the topic from their regulatory perspective mainly. We are trying to get the better services by having a collaboration and cooperation with the industry, not mandating them to, for example, implement IPv6. We had this discussion nearly ten years ago. And we decided against regulating the ISPs to implement the IPv6. We believe that IPv6 is essential. It is not that direct need in Finland because we have quite a legacy and ISPs and mainly quite much of the IPv4 addresses.
So we are not really running out of the IP addresses in Finland. Of course, partly due to the fact that the mobile networks are using Great Net. And then again the carrier Great Net, of course, is some problems. And we believe that having the different net devices in between actually harms the end user's possibility to use services. We believe that the Internet connectivity it is much better. And we saw from the last presentation also the end user experience from quality wise would be much better.
And basically if I'm thinking a bit further about the problem because this was really about the problem statement, well, we had also figures from Google that the latency for IPv6 is actually quite much lower than for IPv4. And I would say that this is a really good benefit for the users of IPv6. We also believe that the IPv4 with the NAT provides a lot of problems for the content providers. First of all, they can't use the IP address to identify users and makes the implementation much more complex.
Let's think about the voice applications. The basic idea was to have a simple product or simple tools to build applications, but then you have to implement different types of the net reversal techniques that makes it much harder and more complex for the application providers and content providers. So it actually also harms the Internet being the engine of innovation in this sense. It is not made ‑‑ I would say it is not maybe the major problems, but it is one of the factors that may impact. And then about the real problem statement, because we have been talking about IPv6 for a long time. I mean my universal teacher was teaching in the late '90s that IPv6 will come in a few years.
And it took a bit longer. He didn't need to change the slides for ten years or something like that. And basically the problem is that we don't really have a deadline for the implementation of IPv6. And we don't really have a clear return of investment. Let's say we have the ISP or some companies or the experts that claim we now have to take a stand and implement IPv6. It requires resources. It requires some work and money. And then you talk your management that basically how we are going to get the money back well, we can always put that a bit later because there is something in the roadmap that actually gives you money back. And I think that's one of the main problems. Everyone knows that they have to implement IPv6 some day. But then that day is going to come, that has been the question.
Now I think we are in a much better situation than a few years ago because IPv6 is already mainstream. It is not a niche that you can say that only a few companies have implemented it. However I mean we are still lacking the association pressure, market pressure, many markets. The gaming market could be one. Some national markets could be the case. You have the market, the penetration is already pretty good. You have the pressure there. But well, globally we still have work to be done.
And well, also I would like to address the security because I have heard security many times as an excuse or real problem why the companies are not going to implement IPv6. Of course, it is part of the lack of education, lack of resources, lack of infrastructure, replacements, but it could be also an excuse that because of the security, we are not going to touch the topic at all. And that's a shame I would say. My message is that IPv6 is mainstream. It is really widely implemented. And it is now time to look at the topic.
>> EDUARDO BARASAL MORALES: So thanks. Let's continue with Barbara.
>> BARBARA PRADO SIMAO: Hello. First of all, thank you for the invitation to be here. I'm Barbara Prado Simao. And I work at IDEC which is a Civil Society Organization in Brazil that deals with consumer defense, mainly claiming for justice and balance within consumer relations. IDEC works with different themes such as health and food but works with telecom and Human Rights and to improve network access and equality for the Brazilian customers. The theme has a lot to do with quality of access. People are not able to profit from the Internet they hired due to problems from transition of IP4 and IPv6. When IDEC was invited to talk a month ago, we did a small research and we discovered a few interesting things. Since last year, 28 complaints were filed and that's not a large number but that's ‑‑ this does not mean that the problem is of small proportion. These complaints were able to show us some particularities of this problem that are really interesting.
First the complaints demonstrated a lot of technical expertise on the topic. People who were brought there already knew how the problem should be solved by the ISP and describing resolutions from a Working Group of Antel which is telecom regulator in Brazil and it should work as a solution to the problem. A resolution to explain for you say that any consumer that had problems with a shared IPv4, the IPv4, the NAT, has the right to demand for the ISP to change it for a public IPv4.
So this shows that consumers who complain on the official databases already know what is happening and how the ISPs or game providers could fix it. But when we look at a global picture, the scenario is not that simple. As I already said here before, this problem does not affect only gamers, but also people who use surveillance cameras, VPNs, smart TVs, et cetera. And most consumers do not have the technical knowledge to fully understand what's wrong and to properly complain. They mainly think there is a problem with the project and not with the connection involved. So I think the main problem from a consumer perspective is this structure barrier knowledge that can make customers give up to complaints when they don't understand what is happening. This should be minimized by the condition's customer service. But the majority of consumers who complained also mentioned that they achieved not success on talking to those responsible for the service. The attendants were not prepared to explain or to present any kind of solution.
So I imagined a number of people that just give up when the situation is so difficult like that. And just concluding, I think the problem for customers in general is mainly of information and proper means to find a resolution for their problems. Information regarding any aspects that may impact the quality of a service should be well provided by the companies responsible because this is as I said before a problem with the quality of access they are having and they should be informed of that. Thank you. I think that's it.
>> EDUARDO BARASAL MORALES: Thank you, Barbara. So let's hear our remote participant, Darrin Veit.
>> DARRIN VEIT: Hi. Thanks for having me. My name is Darrin Veit. I work in Xbox in the multi‑player networking platform. We are responsible for providing gaming companies for the network connectivity. Thanks for presenting. There is some really interesting information presented so far. I can say from a Xbox perspective we are actually seeing close to 50% worldwide for IPv6 connectivity. It has been trending up year over year. And we do see bumps here and there with large ISPs rolling out connectivity. CGNAT is an issue that we are aware of. We have been aware of it since the previous generation with Xbox 360.
And when we originally were designing Xbox One we need needed to be able to address connectivity issues when we started seeing larger ISPs starting to roll out CGNAT. To take it a step back further is that there are two classifications of games. Primarily the peer to peer games where we see issues with CGNAT that can cause problems with peer to peer connectivity. So back in almost two years ago there is an application on Xbox called party chat which is a means of communicating with other players within the same game or even across games. Its architecture defaulted to a peer to peer. And back in late 2016 we actually enabled IPv6 connectivity for that application. And then there is also the default platform for games doing peer to peer connectivity on Xbox. And we saw that there was a ‑‑ you can see on the slide diagram a bit of a funneling behavior when we started seeing what it takes to actually connect to consoles peer to peer over IPv6.
Starting at a little under 50% of consoles with IPv6 connectivity, when we actually tried to connect two players we need to have IPv6 on both ends of the connection. So that drops down your base to 50% filtering effect. We also use IPSec for network security. It is native IPSec exchange. One of the challenges we run in to there is that many home routers whether they are consumer purchased or provided by their ISP don't follow the guidance of RFC6092 and block either IC and/or ISP protocols. We are not able to establish a peer to peer connection using IPSec. We have been working with IPSec and router manufacturers to address that issue. If you have IPv6 on both ends you are able to traverse the IPv6 wall, and then you also need to have an instance where the IP path or latency is less or equal to IPv4.
We will start the connection process for IPv6 before IPv4 but it is the ‑‑ whichever path connect first and is able to establish by bidirectional activity.
Going back to the CGNAT and peer to peer as well as client server, from a consumer standpoint the client server is not something that becomes problematic just because NATs typically don't come in to play when you are talking with client server applications. But even then it is ‑‑ there is a bit of a mix. When I take ‑‑ when I take a look at different game titles there is often a multi‑facetted approach to connectivity even within game modes. I can give an example of one of our other titles. Those are all hosted with the Microsoft basher. They are client server based. And so incompatible (inaudible) types doesn't come in to play. There is also an online campaign co‑op which is hosted by one of the peer machines when you are joining a game session together. And so NAT types do come in to play there. And that's where we have been able to leverage IPv6 to help address instances where before the NAT types are incompatible. CGNAT is something that we are definitely aware of. It has gone from an issue that can be present where we were mostly seeing it within small to medium network operators so now we are seeing larger ISPs rolling it out because of IPv4 address exhaustion. And we have taken steps because of double NATs being a cause of peer to peer connection problems. And the Xbox console will alert the customer to the presence of a double NAT if incompatible or challenges NAT type is detected. And we also detect that there is a double NAT present that will display that to the customer in the network settings. It is a common pain point.
And then also we took some steps to change the network settings to also inform the customer that if we only detect IPv4 to give them some information about IPv6 to help them ‑‑ help educate as far as that there is benefits to have IPv6. It is not uncommon in some markets that even if the customer has IPv6 from the ISP, that they need to take some additional steps to enable that. And when it comes to kind of a wrap‑up a little bit, when we talk about IPv6 and gaming, when you talk about the peer to peer connectivity it is an interesting consumer problem that is fairly unique.
When we talk about the pain points B4 addresses exhaustion and IPv6 benefits. It's done from an operator perspective that we are concerned with the monetary cost of trying to allocate B4 addresses. It is a fairly unique scenario where the customer can see benefit from IPv6 and can help justify network operators that want to further invest in IPv6 to actually have customers that are interested in it as long as we provide them with that information to show them why it is helpful in these scenarios.
>> EDUARDO BARASAL MORALES: That's all?
>> DARRIN VEIT: Yes.
>> EDUARDO BARASAL MORALES: Thank you. We don't have much time. So let's move on to the second part where we discuss solutions and possibilities for collaboration. And then we will have the open microphone session, okay? So Antonio.
>> ANTONIO MARCOS MOREIRAS: Well, so what we have to do to go towards a solution? First of all, we should remember that ISPs and game platforms have the same customers, at least in this situation in this context. So we, ISP and game platforms have a common problem and why not look for a solution together. I really don't understand why it is being so difficult to put together the game platforms and the ISPs to talk about a solution in this problem.
The second idea to remember is that IPv6 is the current Internet protocol. Internet standards that IETF and Internet in Geneva at ISPs are being done considering IPv6.
IPv4 is still the most used protocol, but it is legacy. How to work together. What we have to do. Well, I think we have to share information. First regarding IPv6 implementation, what's the real status for the ISPs for the gaming platforms, what the ‑‑ what's the planning, what's the scheduling to do it. And what are the difficult, how can we do it. And second, sharing information about CGNAT, about shared IPv4. How ISPs are doing it. How gaming platforms are supporting it. How IPv4 being compared, shared by ISPs. How subscribers by IPv4, what should be a best practice regarding this. How many open ports a game platform needs simultaneously. Does this depend on the title on the game being played or is it a characteristic of the platform? I don't know it. Do the platform or the games need incoming connections?
What should be the best practices on this? What are the criteria for a gaming platform to insert an IPv4 address in a blacklist? It is one of the common complaints of the ISPs. How to avoid this? How to avoid IPv4 to be inserted in a blacklist and how to delete it from a blacklist.
Well, again I have serious difficulty to understand why it has been so difficult to get ISPs and gaming platforms to talk at least in a technical level so we could easily improve the situation. It would be very good to hear from Darrin's perspective on this, on the ‑‑ of the gaming platforms if you can in your time.
And if you can't get all parties to cooperate, maybe we need some kind of enforcement, right? I don't like the idea of enforcement. I don't think it is really needed. But I have to ask and it is one of the reasons we bring this problem to the IGF. To put more people or more perspectives on the table. And ask what could Governments and regulatory agencies do. What could Civil Society groups enabled to defend users rights do? At least maybe Governments and regulatory agencies and Civil Society groups, defenders of users could help us to invite the game platforms and the ISPs to the same table to disclose ‑‑ to discuss technically the question because it is something that the technical community and the ISPs are not succeeding. We can't do it. At least I won't. That's it.
>> EDUARDO BARASAL MORALES: Thank you. Lee Howard can continue.
>> LEE HOWARD: I want to respond to Antonio's question. I did like what Barbara said earlier about Civil Society, noticing ‑‑ the regulator noticing that there is a consumer impact, but we saw the panel this morning on how regulators can influence IPv6. And generally the consensus there seemed to be that convening meetings between stakeholders was a more effective practice than providing any regulation. When I have studied Governments that have a regulation around IPv6, regulating private companies to use IPv6, those tend to be the countries with the least IPv6 deployment. But in the case that I think Barbara had described where a consumer can request their only IPv4 address, if a customer has to call the ISP and ask for an IPv4 address, that customer is no longer profitable because you had to spend the money to answer the phone and you had to give them an address that costs $20. So that's probably at least one year of the profit for that customer.
So that's really the worst scenario for an ISP is to lose money on customers. But then on the other hand, you were talking about blacklisting. If ‑‑ something that we see in the gaming community is you start to lose to somebody ‑‑ to another player. People will actually go out and pay for denial of service attacks against an IP address of a rival player. If it is a shared IPv4 address, they might be attacking hundreds of users rather than the one player that offended them. There are even worse cases than the carrier Great Net. I think that more convening like this is useful to show that there are issues here, but I am also interested to hear what everyone else has to say.
>> EDUARDO BARASAL MORALES: Thank you.
>> KLAUS NIEMINEN: Yes. I mentioned before we were discussing before regulator IPv6. So enforcing the ISPs to implement it and basically we decided against it. Didn't see that a really good choice and then well, what we did. Well, first we published a recommendation for ISPs to implement IPv6 regarding the technical implementation like the prefix size and lifetime or the do‑not‑block the extensions. So basically try to make the IPv6 working. Because we know that if you make broken implementation you face problems.
And well, but that wasn't really the major task. We actually decided to launch the national IPv6 launch in 2015. And before that we had basically 0% coverage for IPv6. We managed to get eight ISPs onboard. All major ISPs joined. We managed to get 16 caps joining the initiative basically also including free hosting providers. We got our top websites joining national broadcast to have some problems with the publishing systems and what a ‑‑ complex background systems. So they are not fully IPv6 yet. But as you can see we managed to get more than 5 million subscriptions IPv6 enabled in one day and that's one subscription per capita in Finland. We got a pretty good coverage. But as you can see that map still just to the 7% of the utilization. Because even though the ISP switches IPv6 on, it doesn't mean that everyone actually starts using it.
There, of course, still are generally equipment dependent issues, so the service dependent issues. So that's the reason why not everyone is using IPv6 yet. But I mean it is a continuous process and we are improving all the time. I know that a few other companies have enabled IPv6, but now we are seeing coverage more than 20%. So basically the rate is increasing all the time. And basically I believe the most important thing that we manage to build this kind of ‑‑ the social pressure and get companies to actually implement it. We got this launch date that we had a deadline. I assume it would have taken much longer to get IPv6 ongoing. But because we had the ‑‑ we could advertise that the national broadcast is on, your main computer is on, but it is much easier to ask from a player why you are not ‑‑ I mean basically that really helped to convince the management of those companies to do it.
So that was my reason to emphasize the social and peer pressure basically to get the management of the companies to understand the importance. And also, of course, were giving a lot of free advertisements and goodwill. So that also helps. And, of course, well, I mean we are really talking about the connectivity issues. We are talking about the how to enable end users to connect each other.
And basically the other thing what we did was really already discussed here by Barbara, so we actually told the ISPs to give the end users a public IPv4 address when they are requesting it. I mean it doesn't have to be that you call your assistance, you may do it from the portal. By yourself, so you don't necessarily need to have a human contact there. But anyway, I mean it is up to the ISP to arrange how they do it. And we believe that, of course, this kind of public address it helps on the connectivity side and because the IPv6 isn't yet dominant. But still there are a lot of many applications that doesn't IPv6 and that's the reason we decide to require public IPv4. I am sure this requirement would change once the IPv6 comes. We need to have a common protocol. You need to talk to IPv4, to another party talking IPv4 or IPv6, both of them but we don't want to have translations in between. It doesn't really help.
So basically yeah, my message is that try to build momentum. Try to get let's say the certain game companies or the national framework to actually do it. Define a deadline and then you are actually in a nice process. And, of course, as a regulatory authority we could also impose requirements but we don't really see them that important compared to the collaboration. Thank you.
>> EDUARDO BARASAL MORALES: Thank you.
>> BARBARA PRADO SIMAO: For me it is kind of difficult to come up with a technical solution since I am just a lawyer. But I think we are in a transition moment and what works in transition moments is dialogue and cooperation between all the parties involved. And one idea we had if ‑‑ if channels of customer service could be created for this issue. Responsible for ISP companies or the game for providers. That could properly indicate for the customers what's wrong. And then what should he do to just solve the problem. And the customer that already said here are the same for the ISPs and for the game providers. And so this is kind of a shared responsibility between them.
And they should cooperate for this work. And as if what's Civil Society could ‑‑ Civil Society could do with ‑‑ with pressure the regulators. We can do information campaigns, but that's mainly what we can do, but the responsibility still is ‑‑ is still of the companies that provide the service. And that should be resolved and customers would be ‑‑ so that customers could provide 100% of the Internet they hired.
>> EDUARDO BARASAL MORALES: Thank you, Barbara. So Darrin, can you hear us? Darrin?
>> DARRIN VEIT: Yes, I can hear you. Can you hear me okay?
>> EDUARDO BARASAL MORALES: So can you continue?
>> DARRIN VEIT: Yes. So just looking at the policy questions, so there is already a forcing function that we are well aware of the impact that has to provide traditional IPv4 addresses for clients connecting. There will be natural forcing functions for client server based games to start working towards more IPv6 limitation. We have made a good amount of progress but there is still a lot more progress to bring all of our services and games to connect over IPv6 from a client server topology. As far as the issues that we'll face with IPv6 not ‑‑ so I would actually ‑‑ I would add a qualifier. I would say it is more than just IPv6 being available. I would say that high quality low latency IPv6. That's one of the big things that you look at for games, is that multi‑player gamers are very latency sensitive. We have done some of our own testing with a lot of in‑home routers a couple years back. And it wasn't uncommon to see in‑home network devices that with IPv6 would actually perform worse for IPv6 when it come to latency or throughput versus IPv4. We have seen this sometimes in the case of enterprise networking equipment that IPv4 has and connectivity were optimized, but IPv6 was more of a hey it is a feature. But it may perform worse because not many people have been paying too close attention to what those paths look like except for, you know, folks here on the panel and folks that are deeply embedded in this.
So yeah, because of the problems that we have with CGNAT, continue to see progress and connectivity and low latency, that's something that will become even more growing problems as far as gamers being connected. As far as connecting peer to peer, there are two primary methods that we can use to work around IPv4 and NAT issues that can be increased in CGNAT environments. One is the IPv6 path and the other is using relay servers in order to work around incompatible NAT types. We have provided game developers with a platform that has automatic fallback to using relay servers to work around NAT problems. I wouldn't say that is a cure‑all solution. It is not uncommon that you are going to increase latency. They may be within the same region. And if they are able to connect directly to each other they will have an optimal experience from a latency perspective, but if they can't connect natively over IPv4 or IPv6 and if I have to connect them, now I could be increasing the latency in order to work around those problems.
Overall challenges for V6 deployment I would say it is still probably when I talked about routers that don't have IPv6 enabled by default, it is something that it is not uncommon. We have been working with router manufacturers for awhile. And that's just the step of having IPv6 enabled by default. Then there is that additional step of hey, is your IPv6 performance as good or better than IPv6 performance, as good or better as IPv4. And if not, if there is more work to be done in that space.
There was an earlier question about blacklisting CGNAT. I have seen some of those problems in the community that folks may be unaware of CGNAT and the implications that it has. If you have methods of blacklisting that don't take in to account that it is not a one‑to‑one mapping between user to IP address, by blacklisting static addresses that you could actually be blacklisting 50, 100, 1,000 people.
Lee also mentioned about the DDOS problem. It is a problem in online, especially peer to peer and impacts are greater with CGNAT. It needs to be a collaborative effort. I spend a good amount of my time working with network operators and router manufacturers helping educate about things with CGNAT. Like hairpining support or how to configure CGNAT devices to be less problematic. But it is something that we have done and continue to do and will need to continue to do to help educate both game developers as well as network operators about some of the challenges that we are facing and will continue to face at a growing rate because of the before address exhaustion problem.
>> EDUARDO BARASAL MORALES: So thank you, Darrin. Let's start our open microphone session.
>> AUDIENCE: Thank you very much. I am from the French regulator. Your question, would be some kind of remarks. Just to put things in context. I have been always for IPv6. I will explain to you why I am saying this right now. In general I totally agree with Antonio and Barbara. We are in a multistakeholder situation. Coconstruction approach is necessary so that we can work together at the same pace and try to accelerate this transition. That's what we are doing locally at ARCEP. That's what we did last month. We did a workshop with different ISPs and hosting providers and administration. And we worked together to list the different issues and different solutions. And we can provide some solutions to motivate top management to take the risk even if there is no direct ROI to this implementation, but I may say that I do not really agree with you saying that IPv6 provides better quality of service or less latency.
Let me explain this. Because for me in order to accelerate this transition which should be critical towards IPv6 and not only say positive things. First thing related to quality of service we took statistics related to Akamai and Google. If I take the example of tier 1, there are issues between cogent and hurricane electrics or Verizon. There are other issues related to the fact that we don't peer the same way on IPv4 and IPv6. There is also another issue related to the equipment that root IPv4 hard coded but not in IPv6. So we don't have the same SLA regarding IPv6 and IPv4. This is an issue that we should tackle and it would motivate the actor to do so.
To come back to the security issue, it is true that we need more training, et cetera, related to the security of it, of IPv6, but there is also issues related to the fact that if I take the example of anti‑DDOS for IPv6 there are still solutions that are not mature. I think that we should have like a critical vision for what is the actual state of IPv6 so that we can take care of this not small issues but if we take them one by one we can fix them so we don't tell things that are not 100% accurate. Thank you very much.
>> EDUARDO BARASAL MORALES: Thank you.
>> AUDIENCE: Hello. Jonas. I am an activist from Electron Transfer of Finland. And what I would like to know is there any kind of reliable results so far on how easy it is to migrate to IPv6 between platforms. Because if you are playing on Xbox or Play Station or a Windows desktop you are going to have to wait for X amount of years for it to work. Any current good knowledge on how things are going platform wise? We already saw something how it is going with Play Station as in no information yet.
>> LEE HOWARD: I think part of your question is there a list someplace describing how various kinds of devices or what their progress is for IPv6 implementation. And if that's the question, the answer is no. Whenever they start to make a list they find it is impossible to maintain because it changes so fast.
>> AUDIENCE: We don't know how the process of migrating to IPv6 is going on different platforms?
>> LEE HOWARD: Not to a consumer electronics platform. From a general purpose operating system I would say it is going spectacularly well. In terms of electronics you have in your home that have embedded operating systems, probably poorly, but I don't know of any metrics.
>> EDUARDO BARASAL MORALES: I think this lack of information is part of the problem.
>> AUDIENCE: Hello. My name is Gustavo. I am from Brazil's order of attorneys. Competitive sports, it is a profitable and viable career. And my question is would this performance drop from the IPv4 and IPv6 issue, could this make it unviable for them to participate competitively? Or if not unviable, at least hinder their performance?
>> EDUARDO BARASAL MORALES: Let me see if I understood the question. You are asking if they don't do the IPv6 implementation in some time they will suffer for lack of performance, is it?
>> AUDIENCE: Yeah, my point was about IPv4. Because since we discussed that the IPv4 can bring a few issues could this end their career, maybe?
>> LEE HOWARD: Isn't that why we are here? Isn't that the concern that the gamers with IPv6 have an unfair advantage over gamers with IPv4 and that that would be a problem for gamers that only have IPv4? Is that a fair characterization of the problem?
>> EDUARDO BARASAL MORALES: Sometimes we have heard that some gamers with IPv6 have an unfair advantage. And the other part of the problem is that even, with IPv6 with ISP offering IPv6 the gamers don't ‑‑ aren't able to use it because the platform does not support IPv6 yet.
Last comment.
>> AUDIENCE: Hi. Good afternoon. I'm Machaela and I run a posting provider in ISP Ireland. Trying to push for greater adoption. I agree with the comments about regulation. I think that Government can play a role in encouraging ISP, to roll it out. We across Europe have public tenders and I don't see many tenders where the Government is saying we will give you extra points for IPv6. We will give you extra points for something along those lines. I know that the French Government have been doing a lot of work around. You still can't access most of the services over IPv6. And the costs around replacing all of the end user equipment is scarey. To the gentleman from Finland about the equipment, the ICANN SSAC did a study about nine years ago but it is completely out of date. Nobody updates on the equipment that supports it.
>> EDUARDO BARASAL MORALES: I think that Darrin has something to say.
>> DARRIN VEIT: I wanted to add. The impacts of IPv6 access, I would just say it is incumbent on platform providers like ourselves like Xbox to help educate the customer not only do they have IPv6 connectivity but also what is the quality of that connectivity. The gaming population is so attuned to what the quality of their connection is. It can be a forcing function that customers that can see that they have a better quality connection with one ISP versus another with IPv6 that it can actually help as far as being used as a competitive advantage and help justify further investments in V6. So it is something that we have been looking at that we need to spend some more time investing and to help show customers that not only do you have V6 connectivity but here is your throughput and latency. My friend has 30 milliseconds than I do on this other ISP. I can help build some critical mass behind that option and high quality V6 connectivity.
>> EDUARDO BARASAL MORALES: So that's it. Thank you all for coming to Game Over IPv4.