IGF 2019 – Day 4 – Convention Hall I-C – Meeting on Cyber-Accountability: Building Attribution Capability

>> MODERATOR: Shall we get started? Hello, everybody. This was supposed to be kind of an informal working group environment. I don't know why they put us in this cavernous room. If you're on the sidelines, come to the table, come closer. We can all talk perhaps a bit easier and yeah. No reason to sit back. You are participants, not an audience. So let me introduce the people behind this workshop. I'll let Serge Droz from ICP4 peace foundation.

>> SERGE DROZ: I'm the chair of the security team. This is a topic that pops up all the time and we need to start working on it.

>> MODERATOR: Okay. So I'm Milton Mueller from the Internet Governance Project at the Georgia Institute of Technology. We're in the school of public policy. My colleague Hans Klein is here and also a former full bright scholar from our institute and a Ph.D. student I Lana is here. Not sure we have time for everybody to introduce themselves but maybe later on we can get a sense of who else is here. Really glad that you are interested in this topic and we have been pursuing this topic for about two years now. So let me explain why we think cyber attribution is an Internet governance, a critical issue in Internet governance and then we'll give you some updates and let Serge take over then as to what's happening in this space and talk about what we can do next, okay?

Our interest in attribution actually started with the Microsoft digital Geneva Convention proposal. We're not part of Microsoft, not funded by Microsoft in any way. We looked at this digital Geneva Convention proposal. There were three elements to it. One was an international treaty. The other was a tech accord among the private sector. We were not too keen on either one of those ideas. We didn't think the treaty was feasible or the tech accord would be too easy for two or three private actors to defect from it. And we didn't think it would be a big effect. We appreciated the sentiment behind both of those initiatives, however, and thought it was great that a major private sector actor like Microsoft was taking such a global public interest perspective on Internet security.

However, the attribution organization proposal that they made we found extremely interesting. We thought it was something that was quite feasible and they commissioned a follow-up study from the RAND Corporation about how it should be stateless attribution. The attribution organizations should try to be independent of nation states. And we looked at that and we did some more research and investigation on the concept and we realized this is really something worth pursuing. For one thing, you know, when you are talking about attribution, the -- many people think it's a highly technical thing. If you're dealing with cybersecurity there are the technical details that are extremely important and understanding tools and having access to data that's generated by operating networks and logs and so on is extremely important. But fundamentally attribution is more like a court coming to a decision than it is a technical process. It is about putting together pieces of evidence and having a what we might call a scientific process or if you want to be philosophical creating intersubjective validity. What is it that you can prove that will make your conclusion appear to be valid to other people? And it was this intersubjective credibility that we thought was really needed in cyber attribution. Insofar as it's controlled by nation states entirely, then they have the problem of international anarchy, the fact that no nation state can really assert authority over any other nation state. And every nation state has its own interests to look at. So we thought the original idea of an attribution organization and independent non-state actor was very interesting and started to try to develop organizational capacity to do that. One of the things we did was we proposed a workshop at the last IGF in Paris which was not accepted.

Apparently they didn't think it was an important enough issue at the time. Actually, this workshop was initially not accepted and somebody must have canceled and they put that in. It's kind of a sideline. I'm digressing. I'm not whining, I'm just seeming to.

But anyway, we did some things in the context of Georgia Tech which made it clear that we have severe trust problems in putting together such an organization. We have, you know, the people in the west will say it's fine to create an independent attribution organization but don't invite the Russians and Chinese. The your answer and Chinese will be if it's closely connected to Microsoft or the U.S. government will be dominated by them. Really it's not a minor task to try to come up with an independent and neutral attribution organization. But the model we went to based on our own experience was we wanted domain name system to be independent of states as well so we created iCann, a nonprofit multi-national governance institution that is more or less independent of states and thought maybe something like this could eventually happen with cyber attribution. So we do think it's very much an Internet governance problem and we think we need some kind of independent source to go to to resolve attribution issues. There are all kinds of subtleties about this that we'll get into. At this point I want to turn it over to Serge to talk about what's been happening in this space since our initial efforts.

>> SERGE DROZ: As Milton pointed out, it sounds easy but it is a really hard problem. What's really hard about it is that there is a lot of asymmetry. You mentioned political things and attribution always plays into the political area. Then you have resource problems. Not everybody has the same amount of resources. Microsoft has a lot more than a small garment may have. These are challenges. We don't talk to each other and particularly we don't talk to each other across trust borders. People often conclude that attribution is a technical problem. You do the forensics and then you find out who the bad guy was. It's not that easy. What you find out is which computer was used to commit a certain operation. And sometimes you don't even find that out. Making the step to who is actually behind kind of this last meter who is behind the keyboard, that typically requires intelligence that the technical community certainly doesn't have. And that states are not willing to give up. Typically that's information that intelligence agencies have. So you need human intelligence and most people don't have access to that.

And -- but what the reasoning was there are a lot of individual organizations that have parts of the puzzle and that can maybe work together. An inspiration to me came from looking at how the people do this. If there is an incident that involves chemical or biological weapons different independent labs are tasked with analyzing certain samples. So you cannot translate it to cyber but the idea of having independent organizations, independent labs that peer review each other and come up with common standards of what makes a solid analysis seemed intriguing. That's I think where we started to meet up because you were following on this kind of network idea.

What we did last summer generously funded by the German Department of foreign affairs, we conducted a workshop where we invited a broad range of stakeholders. So we had people from Garmin center, private industry, academia in there and we had people from Civil Society in there. It's maybe not a surprise in here but most people are surprised when we say you need to have Civil Society on board. Why is that? Because states actually in the Internet go after members of the Civil Society and vice versa. Some individual can actually take down a state if he has a bit of luck. You have a huge symmetry and we feel all of these people need to be on board.

The workshop is really interesting. A couple of conclusions that we found were that it probably doesn't really make sense to create a network that focuses solely on attribution. But it should actually focus on what we call fact-finding at the workshop. The reason for that is that first of all a lot of the states said hey, attribution is really a state activity, it is not up to the private sector or Civil Society through attribution. There was a big argument about whose authority that is. But then especially members from Civil Society said hey, we are not really interested in attribution. Interested in the facts because it allows us to find victims. If we make these facts turn them into attribution, we are going to run into problems because then the victims are going to go into hiding and that's not what we want. And the conclusion really was that let's try to focus on finding facts, finding what was establishing what happened, and then decide what to do with this knowledge. You can argue if this makes sense or not. That's still very open.

But I think those were the main findings and the question is now how to we move forward from here and one of the proposals and ideas that we are going to pursue is that we want to do a proof of concept where we find a couple of volunteers to take on an incident that is not too political and play this through. How does it work? If you have different stakeholders collaborating, working on analysis, peer reviewing each other, can we do it? What are the obstacles? What are the challenges? That's where we plan to go. And we plan two workshops. One organized by you guys, by Milton. And then later in the year one again organized by ICT4Peace in Switzerland where we follow up on the idea. Right now we have more questions than answers but we do feel we need to move away from cyber attribution being a single state or U.N.-run organization. We feel that it just doesn't have any future.

>> AUDIENCE: Can you give a reference to the workshop you mentioned funded by the Germans.

>> SERGE DROZ: ICT4Peace. We have a public report about the workshop. Also the participants and outcomes in a little more detail.

>> MODERATOR: So I would jump in here to flesh out the debate about fact finding versus attribution. We, I think, were pushing for more of a focus on attribution as opposed to fact finding because what we fear is you'll lay the facts out and then governments and others will use those sets of facts to draw whatever conclusions they want. It is kind of like the missile Ukrainian missile that shot down the airplane, you know. You just lay out the facts and the Russians will say well, you can see that this clearly did not come from our forces and then the other people will say you can clearly see that this came from the Russians. So obviously even if you went the additional step and made an attribution, that will be contested. But we think the whole point of having this independent network is to make that final conclusion to -- so that others, you know, at least have to engage with the responsibility in a way. The other issue I think that is an interesting one that we've debated is Serge made an excellent analogy when we were talking when he said that the search that he works with are like firefighters and firefighters don't try to decide who is the arsonist or find the arsonist, they put out the fire, right? And so by the same token, the attribution function is not necessarily fully integrated with the cert kind of incident response capability. So even though certs could help with a lot of evidence, we don't want them to get involved in attribution because it kind of -- it could undermine actually what they want to do.

So -- yeah.

>> AUDIENCE: Here is a reflection on this. I puzzled about the distinction is attribution scientific or otherwise. If you make that it's attribution as a science and forensic act. Forensic invoking -- being similar to a judgment in a courtroom, a judge in a court has to be endowed with authority. It is a position of authority off a generally public authority. It does make some sense when states claim that the act of attribution is a state function, only they can do it, that makes sense if the active attribution is like a judge in a courtroom making a forensic act. So my own thinking out loud a little bit helps me perhaps understands why states are adamant they possess the attribution function. Attribution is an act of public authority.

>> MODERATOR: I agree and also disagree with that way of putting it. And again I think this is a very interesting debate. So when I hear states say we should be doing attribution, what I hear and I've heard it from three or four different states, is that attribution is a political act. And what they mean by that is that the governments want the strategic flexibility to make an attribution or not insofar as it serves their interests, right? So they may actually think they know who did it but the statement are we going to make a public attribution is going to be a political calculation. And it could be also a false attribution based on political characterizations. What I like to tell the states who say this is that it's not a political -- it's not a political function, it is a scientific function in the sense that there is a truth about who did it and who didn't do it and we want to get at that truth. Whether you choose to do something about that, what you do about it politically is a completely different question. It is kind of like the firefighters attribution of the arsonist distinction. You might use it to organize collective action among other states, you might do nothing. A whole range of different options that you could use. But the actual attribution is not political. I think it's scientific. And we I think, at least our perspective on this, we want to take away from states the ability to manipulate or play politically with attributions and make it more scientific. I have a question over here.

>> AUDIENCE: A comment to this. If you look at the --

>> MODERATOR: Identify yourself first.

>> AUDIENCE: The point of the judge should be is meant to be independent of the parties and by this creates authority. That's what the state should insure that the judge is independent. It is not happening in all the countries. It is not true everywhere, but that's a cool idea. So this is where the authority of the judge is coming from that is not a party. And that's actually answering this. It shouldn't be political but independent. Should be no interest of the person who or the organization who is attributing. By this you can go to the fact finding analogy with the judge. But if you look at interstate actions where the example was coming from, then it is probably more like an interstate action or interstate -- more like arbitration or something which doesn't have to be state-based. It could be with the U.N. but it doesn't have to. It could be actually a good civil -- yeah, process from somewhere else.

>> SERGE DROZ: This touches on one of the big issues. In the Internet not everything is between two states. What keeps me and my day job the most busy is not state attacks it's criminals. And we do see certain actors or countries pushing out a lot of crime. And I don't want to go to an international court and accuse a certain country of fostering crime but I want to demonstrate to this country hey, maybe you should clean up the mess you have in your place. And typically countries that push a lot of cybercrime are also the ones that suffer the most from crime. So there is this mutual interest. There is a lot of levels of attribution. There is just the one state versus state is a very small one. And then also again when we say we do only technical attribution versus human intelligence, it depends what you want to do with this attribution and why we chose fact finding. If you just want to take down a botnet maybe it doesn't really matter who was behind it, you just want to take this thing out. If you want to arrest someone you better know who it is.

>> AUDIENCE: I'm working on a blockchain context that we have no idea who the person is so it doesn't really matter if it's a state, criminal, somebody, was it correct, incorrect or what actually happened and all. So -- but I wanted to -- it is something that what you have to do to get to the point where you can attach an attribute.

>> SERGE DROZ: It is part but not all of it.

>> AUDIENCE: I'm a government architect for blockchain. I'm a German. I have a background-in-law and computer science.

>> MODERATOR: Governance architect. Okay. So we have two people that want to speak. First I'll go to Juan.

>> AUDIENCE: I'm from the ministry of communication of Cuba and advisor to the designated expert on the GTE of 2016 and 17 and doing the same now for the working group. Milton, I believe you gentlemen are from ICT4Peace. I'm a good friend of Daniel many years back and I'm following your work in this because we feel and we said so in the 16-17GTE that attribution is essential in the discussions going on. Especially when it's dealing with how we integrate international law in the face of -- cyberspace. There is a big argument. Well, you will see it because Cuba contribution will be public and also some other country has the same concern that we have concerns when it was stated in the previous GGE that by the way, it was close groups and Cuba was not a member. So we have to take with a grain of salt what was agreed in the 13 and 15GTE when they said that international law and the charter on the U.N. plus it's entirety. When they put us in entirety it is in a sense it gives the idea as is. We say -- we argue that in the case of article 51 we have two main problems. First, when it's equivalence of an attack in cyberspace. The equivalent of an attack in order to treat article 51. Then is the question that deals with attribution. Who is really the culprit for this attack? And this as you know is -- it's really in cyberspace is especially important. Even in kinetic world we are familiar with the false flag operations in order to trigger some counter responses. And this is more easily done in cyberspace in which computers from one country are used from other country to attack a third country. We feel that that link with the possibility of invoking the article 51 is a threat to the international security in the world. Not only in cyberspace but in kinetic world. Having said that I really appreciate the work that you are doing and we should put this topic of attribution at the top of the list.

Having said that I have two comments. My first comment is that as you may know from the technical point of view or science point of view, attribution is the tools are getting more effective and maybe snot so but really been sold from a technical point of view. The problem that we have. We have a sort of catch 22 thing that the evidence that are acquired with those tools, those who gathered them either be states, their own forces of states or some independent organization that for-profit do this. Don't want to show it because then show the capability and that could trigger a way of circumventing and create a sort of arms race in cyberspace of hiding and attributing. We have a catch 22. I mentioned during that 17 GTE maybe we should do something like independent organization that function like an escrow in which the evidence are presented but are not disclosed. They only rule on the validity of that evidence. Of course this is only one way of doing it.

But whatever the way that you do it, then we have as you said before, Milton, the science part. It is a science part. But then the rest that goes from there is totally political. The next one is trust. Because if this mechanism is not trusted, then how will we deal with denial? You mentioned these things that happen even in kinetic world where there is denial of evidence. So we have to deal with trust. In that case we will have to devise a way in which every country or every interested party should be represented in a way to insure trust in the rulings, and even if we -- if that is achieved, we have to think or this institution a way of dealing with denial. And that would be the end of this mechanism because this mechanism is -- you make the parallel of the court. Maybe you rule only the culprit but is not a sentencing court. The sentencing court should be United Nations or maybe the security council or whatever. Is out of the purview of this organization. I again thank you very much. I am sorry I don't have answers. More questions than answers, but well, it is a good, important work to do.

>> MODERATOR: You have raised a lot of interesting questions so I'll try to summarize those because I think they will feed into what other people said before I get to was it you that had your hand up? Okay. So number one, yes, this is a big problem. I think the article 51 debates on the GGE is way over our heads. That's war and peace and we're again not trying to solve that problem. We will let you solve that in the U.N. But the more important point I think you made and I think it relates to same point that the governance architect made who refuses to give us her name. Eva. So the states are developing better tools and method, this is true, Georgia Tech's computer science department is a contractor for some of these. They do a lot of research. But you're right. They will not give you the data and because they don't want to reveal their sources and methods. So one of the good developments that's happening in this space is the formation of the Cyber Peace Institute which is -- has support from Microsoft but is not controlled by Microsoft and other big players in like financial -- like MasterCard I think is supporting it. And so these entities would have data and may be able to help an independent and neutral process with data that would be drawn from their own substantial forensic capabilities. Hopefully an independent source could use this data and make independent attributions. But again to go back to the point that Eva said she was saying a domestic judge is supposed to be independent but when you get to interstates we have the problem of anarchy and no judge that sits above them all which is precisely why we're proposing this neutral entity. Enough from me. Now let's go to the other question and then we'll go to --

>> AUDIENCE: My name is -- I'm here in my capacity as the member of -- (inaudible) We're part of the Civil Society organization in Germany. The questions you have raised about attribution is -- I find them really interesting because you were speaking about fact finding and in my technical understanding I'm not a forensics expert. But many people in our organization are. And speaking with those people, what they always say attribution is hard. And hard as in it's insolvably hard because every evidence you have is incidental at best. And every evidence you have could be in theory falsified easily. So you could have an actor based in Pretoria that pretends to be an actor sitting in -- or sitting in ALABAMA. NATO says that cyberattacks warrant retaliation. I think only thinking that you can attribute a cyberattack to a single actor, especially a state-sponsored actor, is a -- is a very, very dangerous way to go. And will jeopardize in the end the lives of humans in this world. And I think that it's aside from it may be that that actor wouldn't go so far as to say it is a judge or an international organization but definitely 100% says it was actor XYZ. We see that as a very dangerous step. Thank you very much.

>> SERGE DROZ: I know we always say attribution is really hard because on the Internet you can claim to be a doc and no one notice. In some cases it may be true. We have criminal gangs and criminal cases that never have been cleared out and we will probably have more in cyberspace than physical space. People make mistakes and you can find and arrest people. The confidence you need the terms of attribution will determine the response you want to give. You better be really super sure. If you are just going to take down an adverse infrastructure maybe you needless. The fact is you can do a lot of technical attribution and then moving from the technical to the human attribution finding the player behind it, that will be a challenge. But again it is not only states versus states. In many cases it's crimes. I've dealt with incidents where the entire infrastructure of the company I work was taken down and we could to attribution and it turned out it was a student that did an experiment he shouldn't have done. We don't want to nuke him but it was good to know what was happening because it de-escalates. It's only a student, not a state-sponsored action. If we talk about norms. If we talk about cyberspace is not free of law, I mean we need to be able to find out what the actors are. I think -- have an interest in keeping everything nebulous because it gives them the possibility to do all sorts of things they would never do in physical space. Yes, it is hard. But no, it's not an unsolvable problem. It is unsolvable if you want to have it in every case for every act, for every operation. You may -- we need to move forward and that's the whole point of this next group that we actually have not just one entity with vested interest saying it was that guy over there and that guy over there says I don't believe you. It is a community of peer review organizations yes, we agree it's a solid analysis and it is very, very likely it was that actor and not this one.

>> MODERATOR: Our queue for speaking is getting very large. You see, Joy? Don't go yet. We have James and then what is your name? Torson. We have -- wait, wait, wait. This gentleman with the mustache back there. What is your name?

>> Vladimir.

>> MODERATOR: Andrew Konac, right? I forgot your name. Tell us your name.

>> AUDIENCE: I'm from China.

>> MODERATOR: We have Martin. So that's one, two, three, four, five, six, seven people and then we have Eva again so that's eight. So keep in mind that we have about another 40 minutes. Actually we only have 20 minutes but I'm assuming we can just overstay our thing here. Keep it brief. Thank you.

>> AUDIENCE: I'm from the South Korean Alliance. I had an opportunity to give a talk in Thailand several months ago. I covered the issue how the IP address and cross border cooperation is important to resolving cyber threats. I happened to meet a guy who was doing attribution in Thailand. He is based in Thailand. He said that when I'm introducing -- (inaudible) he said the actor was in Thailand and it took a lot of time to trace up who did it and there are so many -- the evidence occurred to his knowledge collected several facts and evidence and clue. It was a lot of time. How we cut it down to kind of the inefficiency. Attribution comes with very much intolerable inefficiency. That's the point I want to make. When it comes to -- you know there is no norm globally when it comes to attribution. My point is that if there is norm, how can we -- because I -- so many IP addresses. IP address does not comply with any jurisdiction cross border at this moment. IP is just numbers. Where is this number located at physically? Change it by some tools moving IP, anything. IP address can be high, the biggest issue of the cyber attribution must solve. IP address is very unclear and does not comply with any institution and even though for example South Korea have -- attribution process in -- IP address does not match inside South Korea. There is an issue today. I want to ask about this identification. Thank you.

>> MODERATOR: IP addresses are assigned to particular Internet service providers and we know what block of numbers is given to which ISP. Various ways of spoofing them and unraveling how they are  spoofed. It is not an unsolvable problem.

>> AUDIENCE: James Gannon. A question or challenge maybe. First of all. States may want to be the main actor doing attribution. The reality is at the moment most attribution is done by private sector organizations. Talking about state to state attacks, with the exception of maybe four state actors the private sector is by far more sophisticated and advanced if you look at some of the cybercrime actors out there. They're by far the best paid and most capable threat actors out there at the moment. Thirdly. Somebody who went through a number of attribution exercises with private sector companies, what is really needed -- and me and Milton have debated the attribution piece for a number of years so I'm happy to see it is getting more operationally focused. But if you want to really know what the industry and if there is anybody else from the private sector that works on this, it's the framework that's missing. And really I would make the suggestion if you haven't already if you think about it in terms of what the miter corporation does for CVSS where they have a high-level framework and database of tracking actors and vulnerabilities, if you applied those same principles to attribution where you came up with a high-level framework how it could be analyzed or assessed and also had a centralized way to record those indicators of attribution. At the moment if you talk to Crowdstrike one day and fire ride the next day you'll have the same -- there is no centralized way to record it. That's a big problem for those actually being attacked. So I think something in the realm of the governance of CFSS that miter corps does could be a good framework to base something around.

>> MODERATOR: That's a really good suggestion, James. I know that's in fact something that has been talked about. In the originate Microsoft proposal we were talking about standardizing these kinds of principles and methods. And I think I'm sorry to say that the idea to do that is there. I haven't seen any progress on actually doing it. That's something we would really like to get underway and maybe the Cyber Peace Institute would be able to help with that. Thorson.

>> AUDIENCE: Thank you very much. I have a couple of comments. The first was, director of Thailand institute think-tank focusing on China and all various aspects including cybersecurity and AI governance, ethics, etc. The open/closed debate that some of the agencies would not disclose their methods and tools and research. Actually I think last week or two weeks ago there was a cybersecurity activity in London and there was NCSC, Martin and they from France were leading institutions in Europe and in fact they actually reversed that. In the past security has been always a patch and always secretive and it was always by default everything is secret and you had to apply if you wanted to publish something in a journal. Now they reversed it and said everything we do is public and you have to justify why you don't want to publish it. There is a hierarchy of research they're doing in the cybersecurity space but they have found out that they were left behind if they keep it secretive and so it's -- they made a risk analysis and found even with their published things it has not a negative impact on the cybersecurity issue. The second comment is the state-sponsored versus cybercrime. I think it's very difficult to differentiate, right? States it could be a crime and you have a lot of this. So it makes it also difficult. In terms of attribution the third comment. It's a big business. In the U.S. over 60% of businesses are insured against cyber intrusion, cybercrime, and if we don't know how to formalize and how to have rigid systems in place you have to ask these insurance companies because they evaluate those companies. It's a big risk for the insurance companies so there are -- it becomes a business and usually in terms of attributing I would agree it is not easy and still probably not easy. But the combination, to find the computer where, you know, virus has been launched. It is a series of variables. What kind of attack it was and what did they want to achieve, etc. And then with these variables it becomes quite clear who is behind that if it is about the state-sponsored activity.

Last but not least the fourth point I would like to raise I have noticed and probably most of you as well that maybe it's changing now, hopefully, that the cybersecurity community and this emerging AI community, AI security are two different sets of people. And you see if you look at many of the cybersecurity frameworks, discussions, and even a couple of weeks ago when the EU launched -- released its 5G assessment report and then Germany a week later, 5G assessment report and you search through -- look for AI there. It doesn't show up a single time. If you look at the -- the U.S. National Institute of technology standardization they've launched a draft concept on zero trust architecture, right? When you look there about AI they already have identified it as something which will shift the game also in the cyber securities, cyber tech space. I think it is also maybe to consider that attribution will become again much more complex with AI. And both communities really need to work together. AI lacks the cybersecurity knowledge, cybersecurity community has not yet really embraced the AI threat here in this space. Thank you.

>> MODERATOR: Just a note about your last point. So actually the use of what you might call artificial intelligence or AI to detect anomalies is a big part of the technical research on attribution that's going on now. Who was next? Let me go down to my list. Rogmire?

>> AUDIENCE: Thank you very much. I have a brief question about the trust. As you know, there in many countries they have developed their attribution capabilities and if we have established such an impartial attribution system, how can we deal with the countries that already have the attribution capabilities? Since they can have their own reports on attribution and I think sometimes it goes against the results that are impartial systems. I think I would like to hear about your opinion on how we deal with the issues, that system and the countries.

>> SERGE DROZ: A very brief. They have to join the network. If they choose not to do so they have a different opinion. Hopefully the network is trustworthy enough so that opinion based on facts -- that's the whole point that we stop saying it was you, no, it was you.

>> MODERATOR: That's an excellent question and I think that's precisely the problem we're trying to solve is that every state is developing its own attribution capability and ultimately they probably won't just abandon them. They will have their own. But there will be kind of a shared space in which these attributions can be made jointly. So Andrew.

>> AUDIENCE: Thanks, Andrew from the U.K. which makes the discussion about courts quite interesting because in the U.K. it's the jury that find the fact, not the judge. Something to think about. I wondered a better framing for what you're trying to do is develop a fact finding or fact agreeing protocol. One of the use cases for which could be attribution. And very much coming back to the last question the existing attribution schemes can then choose to play in that protocol or not. The reason I'm saying multiple use cases is I suspect and it hasn't been made explicit there are at least two. One of which is malware. I suspect those are quite different and might themselves be interesting illustrations if you can do one of each rather than just saying attribution or cyberattack. I think creation of malware, if you can come up with a protocol that works for attribution and malware and attribution. It suggests it's interesting.

>> SERGE DROZ: Certainly part of the process of building up this network is coming up with a protocol. I like the name but I have to call it standards of what do we feel is an acceptable attribution? We need to establish this. They don't exist. That's the laying the foundation here. We don't have the answers. We're collecting all the important questions. But that certainly is one of the keys. If you have no clue what you're talking about then everything goes.

>> MODERATOR: Right. What you are saying is reinforcing what James Gannon said which is we need to have the steps defined and agreed upon more universally in terms of what constitutes an attribution. It's interesting you're saying the attribution would be one use case. What would be another one?

>> AUDIENCE: I don't know, but I think there are at least two different kinds of attribution and I'm a mathematician. If you come up with a solution to one problem, that's really, really boring. If you come up with a solution to multiple problems, I might wake up.

>> MODERATOR: It may be mathematically boring but politically or security not so boring. James, --

>> SERGE DROZ: Just one. I think just Civil Society participants in the workshop make it clear. For them it's identifying victims and helping them. That's a different use case. And it's not up to me to decide what's more worse but different actors may have different use case. That is something we just need to accept and it would be foolish to restrict ourselves to just one if we can cover more. All the better.

>> AUDIENCE: Coming back to the previous point that will be perfect second use case.

>> MODERATOR: Do we have a Martin?

>> AUDIENCE: Martin on security teams. Thank you for the great session. One thing I wanted to ask about is what degree you have already been able to work through openness of evidence. The reason I ask is because one of the biggest challenges today with attribution is actually -- pretty much every attribution that happens in contested and that can happen by the commercial sector. They have benefits at showing they're smarter, it can be a state who has benefit at reshaping the narrative and show they're not at fault. And the way that you typically deal with that is by making all of the evidence open so attribution have for instance indictions attached to them and get more value that attributions that go without. The problem there when you're investigating an incident that's one state attacking another one you can probably get access to evidence that you can share in some way. When you are investigating an incident that involves a human rights defender, for instance, sharing some of that information can put the individual at risk. So I'm just wondering it's a thorny issue and no good solutions but I'm curious if there has already been a lot of work there.

>> SERGE DROZ: Absolutely. Again it is a question how do we deal -- there is a lot of confidential information in there and how do we deal with this, especially if there is collateral damage. Courts know the concept of having part of a process in a closed setting. I can't really answer this question in its full entity. We just can hope maybe we don't needful amount of data. Maybe all the members have access to it. A lot of attribution studies are so conclusive that people say we are not at fault but no one really believes them so again, a work in progress.

>> MODERATOR: I would just add to that, that is recognized as one of the problems we have to deal with and it is a tough one, I agree with you. Eva, you were up and probably -- Vladimir, I'm sorry. I knew I missed a name in there. Vladimir, go ahead.

>> AUDIENCE: Thank you. I'm -- I agree with Martin thank you for the amazing discussion. The first point is usually when we think about a consequences and we discuss it also with Serge we think about big powers. U.S. is going to fire back against whoever or whether China is going to do that. Small countries in the regions that are tense they will be more using cyber for pointing fingers. If there is no methodology or clear understanding how this is done, the level of mistrust and even the conflicts can actually explode around the world. We aren't just talking about big powers. The consequence can be much bigger. Now, trying to focus on our role. As non-governments whether it's academics, private sector or NGOs. A lot of work for us. Whatever the governments are doing, open-ended working group. I'm trying to structure the thinking. What do we need? The first one is the framework for data and evidence collected and so on. We've discussed the problems of that where something is business, something is secret, so on. How do we make a trusted framework where the evidence can be shared, collected and hopefully can be the venue but we have to find the buy in with the parties. Why would they actually share?

The second level is to develop procedures and standards. If we look at the courts back to the analogy of the courts you have very specific methodology and standards how to collect evidence and manage them and present them and how do you do the analysis? This is a huge work for technological and legal experts to develop such methodology which is mainly I guess the academic work to some extent as well.

The third one is the ability to provide some sort of Martin also mentioned the ability to explain or respond to an accusation which is a political aspect. So we have a mechanism like you would have in a court that you can actually say contest the decision and say no, no, this is not me, let me explain. That's again a political level but I think that all of the scientists there to suggest that the jury or the judge is again our role. I would like to underline that this is actually the role of us and let's try to move on with the academic and expert discussions at least on these three lines and we can do a lot while the governments are discussing the political levels.

>> MODERATOR: Those are all very good suggestions. Our model was sort of the scientific process and in that process the contestability or debatability of the conclusions should be open so you should be able to contest and say yeah, it wasn't me and here is what we're saying and that should be taken into account. I like that addition to the discussion. And yeah, I think we've seen several suggestions we need a framework for sharing and collecting data and methodology for arriving at the conclusion. Eva, you are our last. I don't think anybody will chase us out. We will need to wrap up.

>> AUDIENCE: I don't think it's always necessary to actually have one person to point to or one state or one individual or something. Sometimes it is just also sensible to say it's not clear, we don't know. But there is some probability or maybe nothing. Even then we can establish that something happened and what are the risks and damages. And who was the probability to look at. But to establish that other answers are less likely.

>> MODERATOR: All right. So to wrap up, we would like anybody who is seriously interested in helping us do this to give us a card and we will put you on a mailing list. And otherwise I think we're done and I appreciate you all participating. Thank you.

(Applause)