The following are the outputs of the captioning taken during an IGF intervention. Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid, but should not be treated as an authoritative record.
***
>> PABLO HINOJOSA: Hello. How are you? Just double‑check that you are in the right room. It is great to welcome you to the Internet Resilience: Securing a Stronger Supply Chain. If you are sitting at the back, please don't be shy. Come closer. This is a roundtable by design. Please don't spare any seat over there. Come. Don't be shy. Take a seat. You are welcome.
The closer that we sit, the more resilient our dialogue becomes, you know? I'm Pablo. I'm supporting the Marconi Society Internet Resiliency Institute. I will be your moderator for this session. Let's dive straight into the substance. I would like to begin by turning to someone who truly needs no introduction. Vint Cerf. Are you there?
>> VINTON CERF: I am here.
>> PABLO HINOJOSA: Help us set the scene of where this comes from. And what would be your advice for what would be a good outcome for this discussion? Put some stress on us.
>> VINTON CERF: The short summary, many of us have been asking very hard questions about internet and its resilience. Because we've been so dependent upon it. And with the arrival of new applications of artificial intelligence, that dependence is only going to increase.
The implication of dependence is that when it doesn't work, there are all kinds of consequences. Serious problems. This is true of a lot of infrastructure. You don't think about it until it doesn't work. Like when the power goes out. Now what doesn't work?
Or when the roads are all blocked. When your mobile doesn't work. That's become a very important infrastructure. When it doesn't work, there are consequences. So, the internet has become so woven into a significant part of our ecosystem. That we are very concerned about its resilience and reliability.
We've talked about that in a number of different contexts. And the specific one for this meeting has to do with the supply chain. Since the internet is composed of many, many pieces, coming from many, many different sources. A very significant part of the internet is software. After all, it's the protocols that make all these things work. Running on, of course, physical equipment that they depend upon.
I'm going to suggest to you that the word prominence will turn out to have a very heightened importance in this conversation. Where did things come from? How do we know that they have integrity? Have they been altered sort of in-flight? Do we know who is responsible for them? Do we know where to turn when things are not working properly?
So, if we ask ourselves, can we document the supply chain that comes from a particular product or service, can we know it has integrity? This is particularly applicable to open-source libraries. Which have become a major component of development of products and services in the internet space. Open source is wonderful. Because it's accessible.
But it's also hazardous. Because sometimes it's not well maintained. And so we have to ask questions about is the bill of materials used to create a product. And the order in which those products have been applied.
Do we know whether the software is unaltered? Does it have integrity? Do I know the parties who digitally signed it? So I can trust the integrity of that.
And, second, security is a huge issue as well. Not only in open-source software. But in all the components of the internet. How can we be assured that security has been properly attended to? Not only in the fabrication, creation of the software. But also its configuration and installation.
So, tracking all of that. Having the ability to audit where things came from and how they were assembled and who assembled them is a very important aspect of supply chain evaluation. So, as you know, I have to go off to another meeting soon. But I hope that you will all take copious notes. And there will be something concrete coming out of the discussion that the rest of us can share.
That would be my opening homily for you, Pablo.
>> PABLO HINOJOSA: Vint, thank you so much. It's very strange to hear myself. So, I think we have a good scene to work on. Let me introduce, more or less, how this panel is going to work. So we will have ‑‑ there are a few lines there. But basically we will have three sections. The first one, to talk about the current state of resilience. The second, to talk about what we mean by Internet Resilience supply chain, interdependencies and developing solutions. And is the third part will be real‑time experience and practical examples.
Our panelists, it's a stellar group of people and experts. And on the first block, we will have Ram. We will have Olaf. And we will have Anriette. Ram from Digital Identity. Olaf from the Internet Society. Anriette, former MAG chair, and APC.
Fiona, American University. John Crain. He's joining us remotely. He is the CEO of ICANN. Manal, I hope you are here. Welcome. She is from Egypt. From the regulator. Former chair of the Government Advisory Committee of ICANN. And we have Mark from Cloudflare, an IPF. We will be having a conversation.
You are welcome to have that conversation. We will get straight into it. In terms of the current state of Internet Resilience. Ram, why don't you start to talk about what has happened recently in this work?
>> RAM MOHAN: Pablo, thank you so much. Just to frame this issue for us, the internet, as Vint said, is no longer just a convenience. It's an invisible infrastructure that supports nearly every aspect of our lives from critical services to daily activities.
We know this. But we also perhaps don't understand fully well just how much our modern world is built on interconnectivity. From enabling critical services ‑‑ healthcare or financial transactions, transportation, energy grids, emergency response. A resilient internet ensures that these vital services remain operational. Even in the face of disruptions.
The other part of the criticality of what we're looking at is economic stability. From global corporations to local shops, they rely on the internet for every aspect of what they do. We also know over the last decade that it's been significant for social communication and social connectivity. And for all of us who endured the years of COVID. Without the internet, would have been quite a different experience all together.
Certainly, the internet is critical for national security and public safety. It's also a key part of trust and confidence. When you don't have a resilient internet, it erodes both public and business trust in digital systems. And conversely, a resilient internet fosters confidence. It encourages further digital innovation, investment and adoption. Right?
And, finally, it's about mitigating disruptions, right? Resilience is not about preventing outages, which are inevitable. It's about the ability to withstand, adapt to and rapidly recover from them. Right? Without resilience, minor glitches or cyber attacks could have disproportionately severe or long-lasting consequences. So in November of 2024, the Marconi Society convened a group of 30 or so experts in Washington, D.C. To review the state of Internet Resilience.
And that group provided a report for 2024. The Internet Resilience Report. That report highlights a critical assessment of the internet's infrastructure. It identifies key threats. It also offers recommendations. But here are the pressing challenges in Internet Resilience. There are really four key challenges.
The first is just the increasing system complexity with tangled systems. Modern applications and services rely on a vast web of interdependencies. And inside those interdependencies are often single points of failure. They're hidden within complex supply chains.
An example is dependency on power. Dependency on water. Or over‑reliance on a few APIs, software APIs. A second part is just intensifying regulatory pressures. The relationship between operators and government policymakers is becoming strained.
Governments demand accountability and quick responses to incidents. Given the internet's critical role in not only national security, but the economy. And that clashes, often, with the technical community's traditional reactive and problem‑solving approach. Try what's going on. Solve it. If it doesn't work well, try the next thing. Meanwhile, you have a government regulator saying I don't want you to try the next thing. Fix it now. Right?
So, there's a growing friction. Growing clash that's happening there. The third part is insufficient funding for preventive measures. So if you look at funding for resilience, there's almost none.
What happens is when incidents occur, when problems happen, those things readily attract funding. And they readily attract attention. But the crucial work of prevention, which is about good operational practices, proper training, systematic thinking about infrastructure ‑‑ those things often lack adequate investment.
And resilience is a prevention problem. Prevention does not attract money. The last part is the supply chain vulnerabilities in software. The reliance on complex software supply chains introduces significant points of fragility. And the potential for cascading failures if a single link is compromised.
And there are a great number of examples of this happening. And that are public. There are a much greater number of them that are happening that are not public. We're at a moment where it's not just a technical challenge.
Resilience is an increasing urgent, global, and political challenge in front of us. Which is why we're here at the IGF to talk about it. Right? What we need is a fundamental mind shift change.
We need to start to talk about proactive prevention. We have to really start thinking about how can you enhance collaboration? Not inside a sector. But across sectors.
An anecdotal and funny story is that when you talk to regulators and they talk about, you know, how terrific the resilience plan is for internet service providers. The common thread is they have great backup plans. Then you start to talk to them about what is the common assumption. The common assumption is power will be available. Water will be available. Air will be available. Right?
You go to those in had the power sector. You talk about their backup plans. They have resilience plans. They have very good plans in there. But a common assumption for them is that the internet will be available. So that they can communicate. Right?
So, you have this expectation and assumption that other parts of critical infrastructure are going to be available. Except those parts aren't actually talking with each other. Right? So we need enhanced collaboration across sectors.
Really, the last part is a much deeper understanding of the interdependencies that really sustain global connectivity. These things are essential. We have to wake up to these challenges now. Or we'll wake up to a world that doesn't have adequate resilience for a foundational part. Of not just infrastructure, but of life.
>> PABLO HINOJOSA: Thank you. Tangled systems, regulatory friction, funding gap, software Achilles heel. What am I thanking you for is that you used enhanced collaboration and not enhanced cooperation. If not, we were going to get into a lot of trouble. That's an inside joke.
Well, Olaf, do you want the clicker for your slides? Or do you want me to do it?
[ Speaker off mic ]
>> OLAF KOLKMAN: There it is. First I want to apologize to Maarten Botterman, who was initially scheduled to moderate the panel. But he had to leave due to family emergency. That brings us to matters of life.
What you see here on the picture is a staple food for the global minority. Bread. This is something that is delivered to our grocery stores. And this is a little bit of a global minority description of the issue.
But this is being delivered to our shelves with our groceries on almost a daily basis. I'm telling this a little bit to make the story of resiliency a little bit more ‑‑ give it a little bit more life. The way this works is that a grocery store will place an order. That will go into a logistic system.
If you Google logistic system software ‑‑ specifically, if you do an image source, you will see enormously complicated architects with all kinds of building blocks of interconnected systems that go all over the place. But that was the interconnectedness that Ram was talking about. One of those blocks fail, and your grocery will not be delivered.
And all these types of things depend on the internet connecting. Not only connecting in your locality. Not connecting in your city. Not connecting between the warehouse and the grocery store. But also connecting to distant APIs that may be housed somewhere at the other end of the globe.
Everybody in that supply chain is doing their best to maintain up time. Seriously doing their best to maintain up time. But once systems get more complex, they become more fragile. A webpage nowadays will do 100, 200 queries before the page is actually visible. Going to all kinds of different locations.
And these websites are housed probably in data centers. These data centers rely on electricity. Of course, the internet providers and the data center maintainers are doing their best. Next slide. To maintain their power situation.
This, my friends, is a Cummings DFLE 1500 kilowatt diesel generator. I got this picture from a secondhand diesel generator site. This model is available from $139,000. This is the type of thing that sits in a data center. It slurps about 392 liters of diesel per hour. To generate 1500 kilowatts of power. That is one truckload, a 30‑ton digital truckload of diesel every three days.
If your internet goes down or if power goes down in a data center, it takes three days before these babies are out. Now, of course, these things are tested all over the place. I actually have a story. Or I'm telling a story. But I have a sub‑story.
I know of a case where the diesel was tested every month for two hours. They turn it on. Made sure everything was working. And after five, six years, they really had an emergency.
They turned the thing on. It worked for five hours. And it was out of diesel. They forgot to refuel the tanks. You laugh about this. But this is important. Because this is the type of thing that can go wrong, if you don't think through your system.
And if you think about resiliency, I think that is the main question that we have to ask ourselves in these approaches. Next slide. What can possibly go wrong? And what can possibly be broken?
And that informs your risk‑based approach. That also informs what can I do when things go wrong? Do I have to have a satellite phone so that I can communicate to somebody? To my diesel provider, for instance? So I can get diesel.
If the logistics system is down at the same time, there's no way I can refuel my generators to keep up time. And things will really break down. So, these are the things that we need to think about when we think about resiliency. What can possibly go wrong? What can possibly be broken?
Take risk‑based approaches on the evaluation. And understand, for instance, what are risks that might be coupled to each other? What if an anchor goes over the seabed and breaks both the electricity cable as well as the fiber cable? Coupled risks.
And with that, I hope I set a stage for the continued discussion.
>> PABLO HINOJOSA: Olaf, thank you very much, indeed. We are thinking about you, Maarten, if you are following us. Maarten said to all of us, could you moderate? And they agreed I would moderate. If not, we would have missed this wonderful presentation that you just did. I think it was awesome.
Anriette, I would like if you could, thread this to the IGF. Why are we discussing this in the IGF?
>> ANRIETTE ESTERHUYSEN: My mic had to be switched on. Thanks, Mark. Because the IGF is the place where we connect people. People at the other end of the breakages to the processes that involve both technical coordination, policy and development, as well as other policy coordination development. As well as practice. As well as innovation.
This is the place we try to talk about how does all of this make a difference or not make a difference in people's lives? You asked me to say what is my take coming from the Global South on Internet Resilience. I'm from South Africa. I live in a big city.
But it happens to be a big city that has electricity outages. If not every day, every few days. And I live in a region where there's been drought recently. So, all the countries along the Zambezi River ‑‑ Malawi, Zambia, Zimbabwe ‑‑ have power outages as a result of drought. They rely on hydro. If the dam levels are too low, you just don't have electricity.
Many people, Internet Resilience is something that other people have. And what you have is this divide. Where you have financial institutions, for example, who will make sure they have backup. Upper middle‑class people would have moved to solar by now.
So, when it comes to dealing with these disruptions, which are often not easy to control, there's a vast difference in how people are able to respond to that. And then when you look at industry and businesses, obviously, the larger the company, the better it's going to be. To be able to address disruptions.
You have sectors. You have data centers approximate which communicate very effectively with their clients. Their clients would probably get advanced warning. Or they would be rerouted. And they would manage.
Data centers are kind of a small sector of the economy, relatively. Although they work with financial institutions. So the banks might carry on working. But if you rely on a mobile network that needs to use a tower. That relies on electricity. And the towers just go down.
So, even though theoretically, the bank services are still up. You are not, as an end user, able to access it. Because your mobile operator ‑‑ firstly, repairs ‑‑ you're a customer. So you get no communication from your service provider that there are disruptions.
It's like there's a two‑tier system of how ‑‑ as Ram said, what really matters is how do you respond? How do you do recovery? I think in many countries of the Global South a two‑tier system.
Some people deal with Internet Resilience as an act of God. Sometimes you have. Sometimes you don't. Then there are sectors of economy and society who are empowered enough to invest in routing around it.
I think that's what's so frightening in many ways. You've all talked about how the technical community does the routing around the disruptions. I think what happens in many developing countries is that the end user is expected to do the routing.
They have to have two SIM cards. Three or four SIM cards. Because there's different affordability. And different coverage in different parts of the country, for example. They have to have backups. They always have to have batteries.
If the undersea cable disruptions that have hit much of Africa ‑‑ much of the world in the last year. You have no idea. There's no communication. There's no information. Unless you actually are an internet service provider that belongs to an internet service provider's association or repairing point. They're very good at communicating.
If you're any other business, never mind an end user, you will not know what the current status is of undersea cable breakages and repair. You basically have to wait and see. And hope for the best. So, I think just my last input here, Ram talked about enhanced cooperation.
I'm very happy say b saying enhanced cooperation, by the way. I think enhanced communication. I think we need to ‑‑ at the moment, there's a security sector. The companies that provide the security and the backup are very good at communicating with one another. There's a lot of cross‑industry, cross‑discipline collaboration.
But there is no communication with end users. Which, as Ram said, is very dependent. And takes for granted this will be available. Exactly as Ram said, it breaks trust. And it disrupts innovation. It's a ‑‑ it disincentivizes people from the beginning to embed the internet in daily life and daily services on a more routine basis.
>> PABLO HINOJOSA: Thank you, Anriette. We are just on time. We are doing well. We have participation online. From Nandipha Ntsaluba, what a great topic on resilience. Do we have business continuity strategy for unplanned disruption leading to total global unavailability of internet? They are very switched on. I would like to know where you are from. Welcome to this panel.
I see some empty seats. I welcome people to join us. Don't be shy. If you would like to move up to participate later on for questions. Let's go on to the second block about supply chain, interdependencies and developing solutions. And I would like to ask Fiona if you could help us with some mapping. And the work you are doing with identifying gaps and the policy concerns.
>> FIONA ALEXANDER: Sure. Hi, everyone. Thanks, Pablo. You're doing a great job stepping in to moderate. Thank you for the invitation to participate and speak and to Ayesha and others at the Marconi Society for this interesting project.
Mark talked about the session we had in Washington. Time flies for me. We've been having a couple of conversations. We have a working group trying to put together an Internet Resiliency mapping exercise. And actually putting together a map of what that supply chain looks like.
The goal ‑‑ I think there might be a slide. But the goal of our group is to produce a map that identifies the actors, institutions and dependencies across the internet ecosystem, infrastructure supply chain. As Anriette's story highlights, that's not just what we normally think about. It's not just traditional telecom, providers, ISP, or other players.
We're trying to demonstrate clearly that our internet ecosystem infrastructure supply chain includes everybody. Hoping to do that with this map. Again, hard to follow if we only focus on the people we think of and some people who are here. We're hoping to go through this exercise and confirm it's much broader than that.
Not all actors or institutions fully appreciate their role. Corresponding resilient responsibilities. Not all these different silos are in a routine habit of talking to each other. Back to enhanced communication or whatever we're going with this cycle. We're hoping to do that as well.
Our working group is doing, we're trying to do this all online through Google Doc. That will be an interesting test of our ability to work together. But a map that follows the path of an IT packet. And an internet user action initiates.
This is an accessible tool for everybody and anyone, regardless of their technical skill set. I should also offer the observation that I'm the least technical person in the working group. Which is great for me. I get to rely on all these great engineering minds.
We'll follow the route a packet takes at a 30,000‑foot level. And try to identify what infrastructure layers the packet hits as it goes from you typing something. To getting to its destination. And delivering what you've asked it to do.
Obviously, we'll talk power and water. I think we all kind of know. And highlight the most recent broad‑based internet outages we've seen in Europe and Spain most recently. But what other sectors of the world we're hitting with all of that.
We did talk about and have discussed that we want to find a way to acknowledge cross‑cutting issues that we're not going to address. So we kind of scope this carefully. And don't try to take on everything at once. We're not going to try to take on the policy and regulatory landscape at the outset. And acknowledge that at the first instance.
If this works for us. Which I'm optimistic that it will. We're hoping to have a first draft of the map for the Marconi Society meeting happening in November. What we think is useful, and we think we're finding the value ‑‑ and we like the map ‑‑ by we, I mean all of us collectively. We like it.
I can envision next iterations of the map that dig into each bucket. Maybe there's a resiliency map for just energy. And one just for power. At the outset, we're trying to get everyone together. And get all these smart engineering minds together. To figure out where does the packet go? What might need to work? And what happens when those things don't work?
That's what our group is doing. Look forward to talking about it more. And others that want to be involved. And answering any other questions. I thought it was an interesting exercise. And one worth spending time on.
I'm really happy to be part of the exercise.
>> PABLO HINOJOSA: Thank you, Fiona. Those were a lot of words per second.
[ Laughter ]
>> PABLO HINOJOSA: It's always ‑‑ I will be like you when I grow up. And we'll have that speed of thought translated into language. That's amazing. I'm not sure if you have John Crain. He is at a very odd hour. I'm sure he will be much slower than you at this time. John is online.
>> JOHN CRAIN: I am.
>> PABLO HINOJOSA: He will talk about part of that map, the unique identifiers. Happy morning, John. Hope have you a cup of coffee.
>> JOHN CRAIN: Can you hear me?
>> PABLO HINOJOSA: We cannot hear you.
>> JOHN CRAIN: Can you hear me when you put your headphone on?
>> PABLO HINOJOSA: Yes, we can hear you.
>> JOHN CRAIN: Good morning, everybody. It's early morning in southern California. I work for an organization called the internet corporation for assigned names and numbers. Many of you may have heard of us. Often just referred to as ICANN by its acronym. As tech folks, we love our acronyms.
So, resilience is interesting. Even if you look at the identifier systems in the role of ICANN. ICANN has existed approximately 25 years. A little bit longer.
Our bylaws talk a lot about security and stability. Those are the oversights, in my mind, of resilience. Resilience being the ability to bounce back once one of those two things don't happen. Now, identifiers is not just the Domain Name System.
Many people think ICANN, they think DNS. But ICANN is also responsible for the Assigned Numbers Authority. They register, literally, thousands of different types of identifiers that they use on the internet. For the internet to work and for that packet that Fiona was talking about to get from one place to the other, all of those need to have certain attributes.
Often, that is a uniqueness to the attribute, to the identifier. Or it is some other special relevance to the protocols. And how the protocols work. Now, the interesting thing is, of course, that may not be obvious. Most end users, the people who use the internet, are blissfully unaware of how the internet works on most of these things.
When we talk about resilience of the network or of the internet, it's important to remember that the users aren't aware of a lot of what is happening. And also that it's not "a" internet. You'll hear people say the internet is a network of networks. That means even in the identifier space, there are many, many people that have a role to play.
It's not just ICANN. ICANN's role is really about administering the top registration databases. The initial information of many of those identifiers. Now, if we look at the DNS, the Domain Name System. And you look at the history. And go back to the early days of ICANN or pre‑ICANN.
That was a system and the route of the DNS is a critical system for the internet to work. Where the domain names start. Where the resolution from that web browser to getting you somewhere starts. Back in the late '90s and into the early 2000s, you were talking about 13 systems around the globe.
Now what the industry was able to do was to notice that that was an issue. We said to ourselves, this is an issue that could affect the resiliency and the stability of the ecosystem. New ways of doing routing were developed. Something called Any Cast. Now you have 2,000, roughly, locations around the globe where this infrastructure is.
You can build resilience. It actually takes you thinking about it. It takes time. And it takes what Ram talked about. It takes investment in resilience.
Building something like this out is not something you just do overnight. Now, ICANN is also interesting. Because very early on, we realized that this is a global asset or a global ecosystem. And it required global policy making. So when it comes down to the Domain Name System, any of you who are familiar with ICANN know there is actually a global, what we call, multi‑stakeholder system for the policy setting around that.
In some ways ‑‑ and I've been with ICANN for quite a few years. In some ways, we spent a lot of effort and time over the years to develop ways that we can include the user. And the ISP. And the governments. And everybody else in the discussions.
But we also work on the technical side with our technical partners. Like the ITF, the route server operators and many others. To actually increase resilience. But you don't just increase resilience and stop.
The internet is a growing thing. You have to keep looking at how you're going to continue to increase and keep that resilience. And you have to be aware of all the things that you rely on for that resilience. That's why the Marconi Society's work is interesting to me personally.
I do sit and think about what if we have a major power outage? What if we have a major code flaw? How will that affect the identifier systems? And, further, how will that affect the internet?
So, it's really good that our friends at Marconi ‑‑ and I thank them for allowing me to take part in this ‑‑ are looking at the broader scope. Of what does it take to provide resilient communication? Remember, the internet is a communication device across the globe. So identifiers are a big part of that.
It's what makes myself, in my role as CEO at ICANN, lose sleep at night. Thinking about how we keep this stuff resilient. But it's more than just what we do. It's so much broader. Power. Water. Protocols.
Just think about everything it takes. To not only move a packet, like Fiona was saying, from one place to another. But do the right thing with that piece of data when it gets to the other end. I'm looking forward to hearing the rest speak. I'm going to hang around to answer questions, if there are any. Thank you, everybody.
>> PABLO HINOJOSA: This is very nice. Thank you, John. This is all about collective risk management in a collaborative way. It's really good to converge here to discuss these matters. Manal, I'm not sure if you are online.
Welcome. She's in Egypt. This is another part of the map. Which is the governmental aspect of things. And would love if you could talk about the regulatory challenges and the role of governments. Manal?
>> MANAL ISMAIL: Yes, I am, Pablo. Can you hear me? When you put on your headset? Okay.
[ Laughter ]
>> PABLO HINOJOSA: I'm ready.
>> MANAL ISMAIL: Great. Thank you very much, Pablo. Thanks to everyone for the inspiring interventions so far. So as was already mentioned by everyone, the internet has become more than just a communication tool.
It's part of nation's critical infrastructure. Backbone for digital economies. And the fundamental need for society's development. So dependency on the internet and its infrastructure is rising exponentially. And it's increasing even more with IoT, AI and other emerging technologies.
This exposes the need to work on securing the internet against the escalating risks that endanger not only critical services running on the internet. But the mere functioning of the internet itself. Such risks include unintentional errors. Like infrastructure failure, power outages, misconfigurations. But also sometimes intentional disruptions. Such as internet shutdowns due to political tensions resulting in cyber attack. And disruptions caused by wars and organization of the internet in the cyber space.
All of this, of course, in addition to the very well‑known natural and climate disasters. As well as crises that may trigger unforeseen traffic spikes. Like what we have all experienced with COVID. In this context, the role of governments is both vital and evolving. As today's digital interconnected world poses unprecedented challenges on how nations operate.
This ranges from simple awareness and incentivized encouragement to enforced regulations. Accordingly governments, as owners of national digital infrastructure, should promote conscious investment and resilient networks with (?) redundancy. And also benefiting from technologies diversity like satellite versus land or undersea cables.
Properly secure and redundancy store registry devices. Regulations or frameworks that balances national interests with global interoperability.
Be aware of and minimize dependencies that were already mentioned. And carefully manage third‑party elements in the network. But also encourage keeping local traffic local through ISP.
As we've already heard, also, cooperation and collaboration with other sectors is extremely important to align efforts towards having a national digital resilience strategy. That is constantly being updated. And also not to overlook the human factor. Not only in terms of capacity development. But also in terms of changing the culture of the internet.
Already taking for granted the internet's underlying infrastructure and its ongoing responsiveness. Of course, governments cannot do all this without embracing the expertise and contributions of other stakeholders. Depending on the issue and stage we're at. The internet is already a global shared resource.
And its governance must reflect that reality through what we refer to as multi‑stakeholder approaches. Where a government's private sector, civil society, academia and international organizations all work together. In order to reach well‑informed, sustainable decisions. Devise more effective people‑centered organizations. And achieve feasible and realistic solutions. That are mindful of the global public interest.
On the other hand, stakeholders should also understand and respect government's concerns. In order to be able to reach a constructive way forward. And since processes followed are equally as important achieved, in that respect I would like to refer to the Sao Paolo multi‑stakeholder guidelines. That serves as a valuable reference for strengthening internet governance and digital policy.
They provide comprehensive and operational framework for establishing, implementing and assessing inclusive, transparent and multi‑stakeholder processes. In conclusion, it's important to note that the internet was not developed with security in mind. But now the transition to an increasingly digital society and economy prioritizes Internet Resilience as a technical and policy challenge. That is both national and global in nature.
Hence, strengthening Internet Resilience is the responsibility of everyone. Caretakers and beneficiaries. And it should be ongoing for access. And in order for the internet to continue to function as we expect. I leave it at this. And hand it back to you, Pablo. Thank you.
>> PABLO HINOJOSA: Thank you, Manal. Wonderful. And it seems that we're really working on the basis of resiliency in spite of major construction happening at the back. So if you see Manal, our faces ‑‑ a lot is going on in the back room of this. I don't know exactly what. Are they preparing for the music night? I don't know.
[ Speaker off mic ]
>> PABLO HINOJOSA: Okay. Imagine a major concert of the Rolling Stones preparing at the back.
>> So are the musicians.
>> PABLO HINOJOSA: Let's be resilient and let's move on. We are soon to the opening of a conversation. We have Vint stating a provocative question. Ram providing some background. Then Olaf providing some "what if" scenarios.
Anriette on the collaborative and linkage between the internet governance arena and the Internet Resilience conversation. We have Fiona on the mapping. John on the unique identifiers. We have Manal on a global perspective on government regulatory. But also cooperation. Cooperation. Cooperation. Cooperation.
I think that's what we try to do. Let's not say how. Just cooperation. Let's get into the real deal with you, Mark. If you could, tell us a bit how this works in the real world.
>> MARK NOTTINGHAM: Sure. Hello. I'm going to take my headphones off. Should I take the control? Lovely. Thank you.
So, I was asked to talk a little bit about how my company conceptualizes and deals with Internet Resilience. I work for a company called Cloudflare. And Cloudflare is a company that provides internet infrastructure in a particular way. The internet itself is inherently unreliable.
There's nothing that guarantees that a packet will get to its destination. We build abstractions on top of that. So we can pretend it is reliable and resilient. We have TCP. So we have connections. We pretend that is reliable.
But sometimes it's not. Sometimes that abstraction breaks down. Web browsers will retry requests, if they fail. So they paper over that unreliability. So that the user, the browser, has a nice experience.
And websites and other services use companies, like mine, to further provide resilience. So what we are is a global network of servers. Everywhere. Providing services to users. Whether it's in the Global South. Whether it's in the minority world.
They bring content and services closer to the users. To improve the resilience and the availability. To improve the security of those services. And also to make them faster. To make them so that they seem like they're just next door. Even though you're actually talking to someone around the world.
And what that does is make the internet more seamless for everyone. That means that resilience is really an existential threat for our business. We have to be available all the time. Or our customers are very unhappy. We use several systems to make sure they're accessible.
The data plane is decentralized. So every node can operate on its own. It doesn't need the connectivity of our systems. So if there's a partial failure somewhere, they can still provide data.
There are other parts that need to be centralized. So we use other techniques to make sure they're available. Usually that's redundancies and things like that. Despite all those efforts, engineering, and hiring some of the best engineers in the world to do this, we still have problems. As has been said a few times.
You can never make something perfectly available. We had an incident last year, just as one example. We called code orange. It was an alarm bell situation in the company. One of the generators that Olaf was talking about earlier ‑‑ thank you, by the way, for that illustration, Olaf.
It was used by one of our power suppliers at a key data center. It failed in a particular way. And then some other systems failed. That caused a cascade of failures where the data center went offline completely. That shouldn't happen.
We take extraordinary lengths to keep them highly available. That's something that shouldn’t have happened. Or a series of events. Because it was that particular data center, those centralized systems I talked about ‑‑ the ones that needed to be in one place ‑‑ also failed.
And we designed them to fail over to another data center when that happened. It turns out that didn't happen as well as we anticipated. And so it made those control systems unavailable. For about a day and a half. Six hours was the length of the real outage there. But it kind of trailed on for a day and a half where there was partial availability.
That's a horrific situation for a company like ours. And we resolved not to let that happen again. We went to this effort to re‑engineer our systems. Those key systems that were centralized. So they were more resilient. And that took a couple of months, basically.
We had another big incident at the same time. Which caused some distraction. Which I won't go into. But there were a lot of lessons that we learned from that.
Even though we had a lot of very talented eyeballs on these problems, making sure these systems were redundant. You have to continuously improve your resiliency. And follow the best practices. Define what those best practices are.
Interrogate how you're using these systems. Provision for failure is it situations. Test the failure. Make sure you intentionally fail your systems. To understand how they behave.
And consider what happens when there are these cascading failures. And, finally, be transparent about your failures. We are very proud every time we have one of these incidents, our CEO and CTO writes a blog entry that explains what happened and why. So we're transparent to our customers.
Indeed, we had another incident about a week ago where we did that. Unfortunately, you know, this is not just an issue that happens within one company. That would be a much simpler problem to deal with. It's a systemic issue. It's a whole of internet issue for resilience.
And in one example, there is routing security. The routing system is critical to Internet Resilience. We need to route packets around the world. It's one of those places where everybody has to agree on what the reality of routing is.
Cloudflare spent a tremendous amount of resource. We're committed to RPKI, routing security issues. We're collecting routing data in our radar platform. Which is our internet observatory for statistics that we're able to observe from that worldwide network of servers that we have. That helps us identify and mitigate these systemic issues.
It's that cooperation that we've been talking about across different businesses and the ecosystem that's so key. To make sure, as a whole, we're able to have those abstractions. So people don't have to worry about this in most cases.
>> PABLO HINOJOSA: Mark, thank you. We are moving from theory to practice. I think this is a very good moment to start opening the conversation. I would love to open the floor, if you have any questions. If not, by all means ‑‑ okay.
>> PARTICIPANT: If I may, to tie what Mark was saying to what Ram was saying, basically, the previous failure is prevented.
>> PARTICIPANT: Pablo, thank you. And thank you, everybody, for being here. I wanted to ask a question to the panel about a topic that I think the first speaker mentioned about the economics and financial. I'm going to make a brief introduction, to put it in context.
Before the internet, the international communication was through the telephone network. And international communication was long‑distance telephone calls. And the business model there is that the call, it was originated by the company or country. It was finalized by some other company.
And they shared the revenue of that call. That was a very sound business model. In which everybody that cooperated ‑‑ maybe there was some other intermediary on that call. But everybody got a fair share of the revenue here.
By historic reasons, when you know, when internet first moved out of the continental United States, there was a set of rules. Those who want to connect have to pay the whole share of the communication. In the last years of the past century ‑‑ oh, my God. So many times. It was a big deal.
It was called the international internet communication cost. And it was raised in the ITU. In study group three. There was recommendation from those days, the D50 recommendation. It was a big issue. Even for some European countries and companies.
Even for Australia. They were very vocal about that. As the time passed, the companies from these developed countries are solved the problem. By putting their connection into what's called the core or backbone of the internet.
Eventually, this communication problem cost has been alleviated in some of the places through ISPs. And companies like the one (?). But alleviated. Because still the problem is that these revenues are not external to the country.
I forgot to say that during the telephone long‑distant model, those income, especially for developing countries, was the one who financed. You mentioned the financing infrastructure and development. Worth billions every year.
When internet came, all those yearly billions that was going from outside to developing countries disappeared. Now that's not only happening. This uneven distribution of revenue. It's not only happening through the communication infrastructure. Also about the information.
We see information data, even knowledge that is created in many countries ‑‑ even if it's flowing two directions, the money is flowing in one direction. That is no surprise. That a few companies, only a fistful of companies have amassed a massive amount of money from resources. That comes from everywhere.
So, my question to the panel is the following. Don't you think that that uneven flow of revenues could be a cost in the long run of unsustainability of the internet? Thank you.
>> PABLO HINOJOSA: Those were the days. International charging arrangements for services. '88, Australia, and all those debates. It reflects some part of the economics of the infrastructure. And I wonder how this is linked to Internet Resilience. So that's a provocative question, for sure. I'm not sure if any of the panelists would like to take it.
Fiona, I knew it!
>> FIONA ALEXANDER: For sure. I'm happy to talk about the telephone accounting rate regime. And two decades of arguments we had. One about accounting rates and how that system could not be applicable to the internet. If you tried to apply traditional telephone circuit switch model of accounting and revenue to the internet, it would require breaking the internet infrastructure we are trying to preserve.
I would dispute all those accounting rate revenue and passing money that went to other countries was not always used to build the infrastructure in those countries. We could talk about that a lot, too, offline. What I will say, in terms of whether the fair share debate ‑‑ it's back again. I can't believe ‑‑ is relevant in the context of this. I'm not quite so sure.
One of the ways of dealing with Internet Resilience ‑‑ maybe Olaf wants to talk about the development of ISPs. And Mark wants to talk about peering and things like that. But the traffic interchange is through peering and transit. That's the way commercial parties privately resolve these revenue constructs you're talking about.
But there has been a lot of work ‑‑ John Crain talked about the route server instances. There's been lots and lots of work to get ISPs deployed. That's shifted and changed the traffic patterns. So we no longer see unequally traffic patterns as the ones from the 1990s.
If you wanted something on the internet in other part of the world, it always had to come back to the United States or western Europe. That's no longer the case because of all the work on Internet Resilience that's been done. We could talk about this for hours, as you know. Maybe the more technical people want to talk about how it relates to the Internet Resilience construct.
>> PABLO HINOJOSA: For hours or decades, indeed.
>> FIONA ALEXANDER: It has been decades.
>> PABLO HINOJOSA: Please.
>> PARTICIPANT: Thank you. I am from Kuwait. We are located in the Middle East. You know the circumstances in our region. It's a question.
Can we consider the safety and the security of the internet infrastructure part of the resilience? Part of the concept of the resilience of the internet? When we talk about the safety or security of the infrastructure.
Whether it's a submarine cable or area of hostilities. Where sometimes the complication of the sector (?) gets limited in operation. Or gets halted. Or gets targeted.
Would we consider the safety and the security of the internet infrastructure part of the resilience approach? Or this is a different angle? It's just a question to the panelists. Thank you.
>> PABLO HINOJOSA: Thank you. Anriette. Then Ram.
>> ANRIETTE ESTERHUYSEN: Of course, I have a view on this. It's not necessarily a broadly accepted view. I was a member of the global commission on the cyber space. So was Olaf. We debated this together. We spent a long time talking about was it ecological resources or the physical information infrastructure?
We decided it has to include the physical transmission infrastructure. I haven't even touched on that. But I think it really ‑‑ having ‑‑ actually, I think that we have different ‑‑ resilience is treated differently by different parts of the internet. The technical community understands resilience. And has good practices and procedures for ensuring resilience. Or striving towards greater resilience.
I do not necessarily think that telcos have the same approaches to the technical community. I don't think governments have the same approaches. There's not common definitions of what critical infrastructure. What is critical information infrastructure.
I think we also see the destruction of infrastructure in conflict. We have the case of Gaza. Where infrastructure has been completely destroyed. And even when Gaza had internet, they were restricted to 2G. There are so many different factors.
I certainly think that the physical transmission infrastructure does have to be considered. Because without that ‑‑ I think as Mark and others have said as well, even electricity, which is an external factor. It disables both the physical and ecological resources. I do think that infrastructure is part of it. But that's not a universal view.
In fact, we asked the ICANN board a few years ago to look at the public resources, physical infrastructure being part of that. In fact, even look at the DNS as being part of the public core. And even that, I think, we struggled to achieve.
>> PABLO HINOJOSA: Thank you, Anriette. We need to hurry. We still have some content to cover. I would let Mallory to have a question. Ram, a quick response. And then John to wrap the conversation, if that's okay. Mallory?
>> PARTICIPANT: Thanks. Thanks for hosting this conversation. Ram, I really liked your first slide. I don't know if you can go back to it. But I don't know if folks are like me. But when we were having these conversations in the framing of fragmentation. And now it came up here. I love the bread picture as well. Thank you, Olaf.
We obviously see an interconnection between electricity and food systems, and shipping lines. And I feel a bit odd when we have those conversations. Because it's really clear that we don't think the internet is maybe more important than eating, or drinking water. But we don't quite make the connection, in my view, that every single one of these vulnerabilities is actually a crisis of capitalism.
And we aren't thinking holistically in terms of the political economy of the internet. And I can say if we think that we can solve those problems without solving power or without ‑‑ you obviously haven't gotten that, right? We can't solve these problems without solving electricity, power and other things. We can't solve these problems without thinking about what happens when some states invade others. Or there are internet shutdowns.
We have to think of it holistically. I wonder if we imagine the project we're trying to take on together. To keep the internet resilient is a project that will transform our global capitalist system. Or it's a project that depends on some other sector sorting those things out.
>> PABLO HINOJOSA: Thank you, Mallory. Ram?
>> RAM MOHAN: Thank you, Mallory. That's a great, provocative question. I think it will transcend rather than transform. And I don't want to talk necessarily about ‑‑ I think there's quite a bit to discuss about the system. And whether the system itself is primarily at fault.
I would point out these resilient problems happen in noncapitalist systems as well. So that's just one thing. One very brief response to the gentleman from Kuwait, who had the question. He's at the table there.
You've spoken about security and safety. I want to say, you should be thing about security and stability. Not safety. Because safety is often what the user perceives. Stability is something you can actually effect as a regulator or as a government.
>> PABLO HINOJOSA: Thank you, Ram. I would love for the John Janowiak, the president of the Marconi Society, help us wrap this up.
>> JOHN JANOWIAK: Thank you, Pablo. Thank you to the panel and all of you for attending today. This is a near and dear subject to the Marconi Society. I was approached not too long ago by Vint to take this on. As one of our core issues for the Society.
Since we did, we've just gotten a great amount of support from the industry. And so this is one of the first readouts that we're providing publicly. So, thank you for attending today's session.
Raising the awareness of this critical issue that's often unappreciated is the topic of Internet Resilience. It's going to take all of us to ensure that this matter gets the attention that's required. We look forward to having you all involved.
The responsibilities are cross sectors, cross geographies and cross sectors, as you heard today. One of the things we were fortunate to do was bring together the top experts of the internet. We have a photo up here of our activity together in Washington, D.C. last November. Where we really sat down and looked at what are some of the ‑‑ where are some of the critical issues in resiliency of the network?
Reliability. Stability. And resiliency. And the report that we ultimately came up with is right here. This yellow report that's on the internet. On our website.
Please, go ahead and download that report. Take a look at it. We welcome your organizations to get involved in this. The more people we get involved in looking at these issues and helping solve these issues, the more resilient the internet will become.
So, we're really looking forward to working with all of you on this issue. So, on behalf of this group here, which most of the members on the panel here are part of the Internet Resilience Advisory Council as well as the Marconi board of directors. We look forward to working with all of you on making the internet more reliable, stable and resilient. So, thank you for attending today. Pablo?
>> PABLO HINOJOSA: With that, I wish we have a really good concert. Thank you very much for attending the workshop.
