IGF 2025 - Day 3 - Workshop Room 6 - WS 123 Responsible AI in Security Governance, Risks and Innovation

>> YASMIN AFINA: Thank you very much, everyone. Good afternoon from Oslo. Or good morning, wherever you are tuning in from. My name is Yasmin Afina. I have the pleasure of moderating today's session. Responsible AI in Security: Governance, Risks and Innovation.

For those who are joining us in person, may I please highly encourage you to come to the front? To this beautiful almost roundtable? To allow us to have a free‑flowing roundtable discussion? Because this forms the part of our roundtable.

In the spirit of a roundtable, I do encourage everyone in the room, who have just joined us today, to join us in the front. Because I would like this to be very interactive and highly engaging. For those who are joining online, thank you very much for joining us wherever you are. And as we are using Zoom, I encourage you to use the raise hand function.

If you would like to take the floor. Again, this is a very highly interactive discussion. Again, my name is Yasmin Afina. I am pleased to be joined by three excellent speakers.

Those in the room, we do not see them yet. But those on Zoom, will you see them. Dr. Jingjie He from the Chinese Academy of Social Sciences. Michael Karimian from Microsoft. And Dr. Alexi Drew from the ICRC.

And before we get into the remarks from out excellent panelists, I would like to spend five minutes to introduce you a little bit to the Roundtable for AI, Security and Ethics. And my institute, the United Nations Institute for Disarmament Research. UNIDIR you can think of us like a think tank. We are independent from the secretariat.

We were established in 1980 at the height of the cold war. To ensure that the deliberations of states are well informed and evidence based in the area of disarmament. Of course, today, the landscape of disarmament is different than it was in 1980. We are conducting research, stakeholder engagement. We also want to make sure we facilitate dialogue, where there is none. Including on sensitive issues such as AI and security.

So, one of the priority areas of UNIDIR and our work is the ‑‑ is really to AI in autonomy, security and defense, including the military domain. What we've noticed is that in a lot of these technologies, highly unique nature, we understand the importance of multi‑stakeholder engagement and perspectives to obtain input on the implications of AI. For international peace and security.

So, we need to provide a platform for open, inclusive and meaningful dialogue. We saw this need as well to warrant public trust and legitimacy. And make sure that they're not just coming from the bottom‑up, but the top‑down approach. We need to make sure we improve cross‑disciplinary literacy, so on and so forth. And you'll see why multi‑stakeholder perspective is important on this issue.

So, that is why in March 2024, UNIDIR joined forces with Microsoft. And in partnership with a series of other stakeholders. The establishment of the Roundtable for AI, Security and Ethics or RAISE. We have, for example, experts from China, from Russia, from the United States and United Kingdom. But also from Namibia, Ecuador, Kenya, India.

We want to bridge divides. And bridge the conversation. When there is none. On these issues of AI and security.

We aspire to lay the foundation for responsible global AI government grounded in cooperation, transparency and mutual learning. With the idea that we should overcome any sense of distrust. And where there is need to build trust, this is where it would be. We would also like to use RAISE to foster and facilitate compliance through international and ethical laws in the age of innovation, welfare insecurity and destabilization.

And finally, we would like to complement and reinforce responsible and ethical AI practices in security and defense domains. In an area where we're hoping to disrupt monopolies in the hands of the few. And make sure all voices are heard, from all layers of society. Before we hear from our excellent panelists, I would like to open the floor ‑‑ I would like to provide an opportunity for both participants joining online but also in person to share their thoughts via Slido.

For those unfamiliar with Slido ‑‑ I may please ask our technicians to share the Slido presentation on screen. Thank you very much. First, before we start, I wanted to get your sense of what you think AI and international peace and security means for you. There is no right or wrong answer.

And for that, what I would encourage you to do is go to Slido.com. And put in the code 179‑1812. For those who see the screen, use the QR code to join the conversation. You'll see a text box. Where you'll be able to provide your input on what you think AI in international peace and security means for you. Again, no right or wrong answer. It is really for us to understand your thoughts and perspectives. And really set the scene. And see where things are at.

Of course, it is important for us to share with you the work that we're doing. But it's also important for us to engage with the incredibly diverse IGF community. To see what you think about this issue. I will leave this poll open as you put in your contributions. As to what you think international peace and security means to you.

And there should be results showing in. Perhaps. If I can ask our technicians to see if there's any input that has been added. It is not showing on the screen. Sorry, I'm bugging the technicians with my requests.

I can see that there's quite a few responses already. We see, for example, censorship, fake news generated by AI. AI could be used for good or for bad. I really appreciate this balanced approach to looking at AI for international peace and security.

I also see the use of AI in the military, law enforcement. And how they're used responsibly in their respective fields. Facial recognition. The proliferation of AI weapons systems. And automated target selection. A very wide range of responses. Please keep adding your responses to this question.

May I please ask our colleagues from IT to share, again, Slido? And this time for the next question.

[ Speaker away from mic ]

>> YASMIN AFINA: Perfect. Now we can see them. This is great. Thank you very much to our IT team. I heard there were connectivity issues. Please bear with us. As we navigate the hybrid space in discussions for this session.

I'm going to get us to the second question. What should be the role of the multi‑stakeholder community in the governance of AI and international peace and security? For those on Slido, it's the same link. If you refresh your page. Or it should be refreshing on its own.

If you could, please add your responses. And they should start appearing one by one. For those who have just joined us, may I encourage you to open Slido.com using your laptop or your phone? By scanning the QR. To provide your input on what you think should be the role of the multi‑stakeholder community in the governance of AI and international peace and security.

I see one response on agree and implementing norms. I do encourage everyone to keep sharing their discussions and their reflections on what they think should be the role of the multi‑stakeholder community. That will help us at UNIDIR. To inform our work on this. And how to better engage with the multi‑stakeholder community.

I see big commitments. I would love to hear your thoughts. When we open the floor on what sort of commitments do you think whether the multi‑stakeholder community could have a role in. I also see industry and AI.

Perhaps, again, when we open the floor for discussions, I would love to hear your thoughts on what they mean. For those who are joining us in the room physically, may I encourage you highly to come on to the stage. To join us in the middle. To have a roundtable discussion. It is interactive.

I see a lot of input into the Slido. I do see, for example, trust building, proposed solutions. Technical standards. Again, actionable legislation. Responsible. And peace. I do appreciate you really putting a lot of input into this discussion.

And now that we've had this little warm‑up exercise, I will please ask our IT team to get us back to the Power Point. For me to introduce, once again, our speakers for today's discussions. So, the way it will work for the discussion, as we have 45 minutes, I will be sharing ‑‑ I will be providing the floor to three speakers joining us online for kick‑off remarks.

It is supposed to be introductory. And generate more questions than answers, perhaps. On issues related to AI, international peace and security. And I will open the floor for those joining us online, but also in‑person. For a discussion.

Perhaps on reactions from what you've heard. Or discussion on the answers you shared with us. Or perhaps if you have any questions for our panelists and speakers, who are joining us today. Again, for those who have just joined, we are joined virtually by three excellent speakers.

Dr. Jingjie He from the Chinese Academy of Social Sciences. Dr. Alexi Drew from the International Committee of the Red Cross. And Dr. Michael Karimian from Microsoft. For those of you in person, I assure you, they are online.

They should be appearing on screen when it is their time to speak. So now, may I please turn online and ask Jingjie to provide us with her opening remarks? May I please ask the IT team to show her on the screen? Jingjie, you have the floor. Thank you very much.

>> JINGJIE HE: Thank you, Yasmin. Very nice to be meeting you all. Thank you for the invitation. Always a pleasure to join in the conversation. Just to see your face on screens.

So, I think the inclusive engagement across stakeholders is essential for the effective global governance of artificial intelligence. And the main reason will be that technological challenges, I believe, can be addressed by technical solutions. However, the identification of the true nature of these challenges criers an interdisciplinary and multi‑stakeholder approach.

Such an inclusive approach ensures a wide range of knowledge, expertise and perspectives, often complimentary in nature, are taken into account. In shaping quibble understanding, norms, and policies for AI development and deployment. So here, I want to take the opportunity to really underscore the importance of the U.N.‑sponsored platform, such as UNIDIR's RAISE, that Yasmin just introduced, IGF, and the Global Digital Compact.

They play a critical role in engagement. What sets them apart from most state‑centric mechanism is that they ‑‑ it's their unique ability to provide neutral, depoliticized and inclusive spaces. So, within those platforms, knowledge sharing and confidence building can take place beyond the constraints of geopolitical tensions and national interest. Allowing for more constructive, balanced and, therefore, more promising outcomes.

But, of course, one dilemma that I want to point out is that such platforms, especially like RAISE, do face funding issues. And problems about questions about how to make the project more sustainable. I remember the first time I attended RAISE. And Yasmin was sharing the concern that this project should be more sustainable.

I do believe that Yasmin and Michael have done a great job in supporting this program. But I do believe that this should be a more collective effort for all of us. To bring resources and contribute to this project and these communities. So, Yasmin also asked me to provide some predictions for how AI fosters international peace and security.

One of my recent projects is about AI and satellite remote sensing. Satellite remote sensing has been increasingly recognized as a critical tool for international peace and security. In recent years, there has been a very interesting applied in AI and machine learning for enhanced efficiency of satellite imageries.

One example is Amnesty International, in collaboration with a company called Element AI. As well as almost 29,000 volunteers. So they developed tools to automatically analyze imagery for monitoring conflicts, like in Darfur.

This is one of the many examples of how AI can benefit international peace and security and nonproliferation missions. Of course, I always care about the challenges. So one potential challenges, as my previous research shows, is that there's always a challenge for adversarial attacks. Which will make the system more vulnerable. And our discussion more interesting in challenges.

I will stop for now. I will be happy to answer questions. Yasmin?

>> YASMIN AFINA: Thank you very much, Jingjie, for this very short and crisp, but also very provocative introductory remarks. I do appreciate your noting as well the difficulty that the U.N. is currently facing on fund‑raising. And, of course, UNIDIR as a funded institute. We do rely on volunteer contributions.

I appreciate you acknowledging what we're facing today to allow such dialogue to happen. I appreciate your also sharing the importance of AI to enhance the ability to analyze and to monitor conflicts, including by civil society organizations. It does show the potential of AI. To enhance international peace and security. While, of course, being balanced by the risks that may resurface. Including adversarial attacks on these AI technologies.

So, I think one key aspect that you also shared with us is the importance of engaging all kinds of stakeholders. We're very fortunate today to be joined by Michael Karimian from Microsoft. Michael, may I please now ask that you provide us with your kick‑off remarks? Particularly to see what do you think is the role of industries in supporting responsible AI practices for international peace and security? Michael, over to you.

>> MICHAEL KARIMIAN: Thank you, Yasmin. It's a pleasure to join you all. And thank you, Yasmin. Not just for facilitating today's discussion. But, of course, for being an essential partner in the work of the Roundtable for AI, Security and Ethics.

As we heard and probably already know, AI is and will rapidly reshape international security dynamics. And the governance frameworks needed to ensure responsible use require multi‑stakeholder engagement, as Jingjie outlined.

Industry, in particular, has a critical role to play, as developers and employers of AI technology. But also I think as proactive stakeholders in establishing norms and standards. And safeguards to mitigate risks associated with AI in security context.

And the Roundtable for AI, Security and Ethics has clearly highlighted that while states and international organizations are vital in setting norms and regulations, industry in particular has quite practical contributions to governance. Which I don't think can be overstated. For example, industry actors often are the first to encounter and understand AI risks and vulnerabilities.

In part, due to their direct involvement in developing and deploying these technologies. That can put industry players in a unique position. To provide expertise on technical feasibility, operational impacts, and risk mitigation strategies. Which, of course, are essential for effective governance.

And through RAISE, industry stakeholders, including Microsoft, have already identified several key contributions that can be made. Firstly, transparency and accountability. Industry must develop and adhere to AI standards. To ensure AI systems, use and security applications are transparent in their capabilities and limitations. With accountability mechanisms clearly stated.

That involves quite robust documentation practices. As well as continuous monitoring. And the capability to audit AI systems. Which together provide greater predictability and trust.

Second, and relatedly, is the topic of due diligence. The Secretary General's upcoming report and ongoing U.N. general assembly discussions, will likely continue to underscore the importance of due diligence. Industry actors have a responsibility to implement robust due diligence processes across the AI life cycle. From design and development through to deployment and eventual decommission.

This aligns closely with life cycle management approaches already being emphasized by UNIDIR and by ICRC in its submission to the Secretary General's upcoming report. Third is the topic of proactive collaboration. Industry should actively contribute technical expertise and build initiatives. Particularly to regionals where regulatory frameworks are still emerging.

Effective governance requires global equity in knowledge and resources. So initiatives such as RAISE but also REAIM, responsible AI in the military domain process. We see them promote practical and inclusive government strategies. Which serve as a strong foundation.

And industry collaboration through those platforms amplify these efforts. I think on the topic of reducing disparities and capacity building and knowledge transfer, industry really does have a significant technical and expertise resources that are needed to support governance, civil society, and international organizations. Particularly those from the Global South. And understanding, assessing, and mitigating AI risks.

Because strengthening global capacity is really key to ensuring inclusive governance and avoiding exacerbating already existing inequalities and insecurity capabilities. As we look ahead, industries in engagement should continue to be structured. It should continue to be sustained. Of course, it could be substantive.

That means participating in frameworks established through the United Nations and other multi‑lateral venues. And initiatives such as RAISE to shape AI governance and security. I think we can ensure our collaborative efforts lead not only to innovative but enhanced global stability, resilience and trust. I look forward to the discussion.

>> YASMIN AFINA: Thank you very much, Michael, for, again, a very comprehensive overview of what you think should be the role of industries in promoting and enhancing responsible AI practices through international peace and security. Both as developers and as deployers. I do appreciate your remarks as well, your points on the industries needing to be proactive actors. To mitigate the risks and harms that may emerge from these technologies.

I also note from your remarks the importance of implementing feasible and effective risk mitigation mechanisms throughout the life cycle of technologies. And for AI and for international peace and security. We're very fortunate to be joined by Dr. Alexi Drew from the International Committee of the Red Cross. Who has been promoting relentlessly on life cycle management approach. To the governance of AI in security. May I please ask Alexi to come to the floor? And also share her remarks on this point. Thank you very much, Alexi.

>> ALEXI DREW: Thank you very much, Yasmin. Thank you very much, Michael. For setting the stage for me. It makes it a lot easier to continue my crusade to make life cycle management a feature that everyone is aware of.

And be more aware of the necessity of why it needs to be approached and understood. And actually engaged with as a second feature. That's one of the key reasons life cycle management is critical.

We've been talking about governance quite a bit. We've been talking about the need to be responsible and ethical in how we design, develop and deploy these systems. Governance is not something that can be added on after the fact. It's not an after‑thought. It should be designed to fit into each stage of the life cycle.

For the purpose of this discussion, I'm going to break life cycles down into very simple segments. In this case we're going to talk about the development stage, the validation stage, and the deployment stage.

I thought it would be helpful if I gave you a particular set of series of risks with hypothetical contexts. Whether these things are actually producing risks now. So we can understand why governance at each stage is important. So, one of these risks that I see and how the ICRC approaches them is the trend in localization of aid and assistance is reversed to the utilization system preponderance.

Which, by their default and design are not local. In the development stage, you might use data taken from the Global North. Train a model, which is designed to be deployed in the Global South. It doesn't reflect the realities.

Predictive model, for example, based on unitarian aid is going to prioritize aid to certain groups, as opposed to others. Based upon the data that's been selected for it. Which is not applicable to our local context. That's going to effectively create a compounding problem.

At the validation stage, that can also create problems. If it's not properly taken into account with regards to localization. If you test something outside of the local context in which you tend to deploy it, you're not actually testing for the scenario and circumstance, and the context which your thing is going to be used in.

So, your ability to be sure it's going to be used as expected is undermined. You're ignoring the social, economic, political dynamics of the context in which it will be ultimately deployed in. So our clean test beds, which might be suitable for some circumstances will not likely be suitable for those in certain places.

At the deployment stage, for example, we might be using algorithms that work in context that systemically exclude minoritized communities in another. A refugee processing system, which trained on one population, works perfectly fine.

But is catastrophic when applied to a different. With different social characteristics, economic needs and requirements. When you take these across these localized issues of the development stage, the validation and deployment stage, there's compounding problems and risks. Which you can't then remove by a set of governance. Which is attached to the end of a life cycle.

It's something which has to be addressed at each of these stages. To ensure that these risks are avoided and not compounded. And there's also the problem of inscrutability. That's almost the opposite of transparency.

But sometimes inscrutability is a design choice that takes place at certain parts of the life cycle. At the development stage, rather than choose something which is open sourced, understood as a model, you might choose a proprietary algorithm. Which is more niche, more sophisticated. But a complex network because it seems more appropriate and more complicated. When a simpler, more explainable model could do the job. Is going to introduce inscrutability into the system at the development stage.

Further on at the stage where you're validating or generating a model, you'll create a system that is so complex that not only can the end users, the subjects of the system, not understand decisions being made. But the users themselves may not be able to. Particularly if they haven't been designers. They simply purchased these systems in the procured form.

What's the real‑world impact? It means unitarian or aid supplies on the ground can't explain to individuals why the decisions are being made as they are. They can't explain why aid isn't being delivered to one group. While it is to another. They can't explain why some resources are available in one place and not another. That undermines trust. In both the humanitarian sector. But also the systems being used.

Which further means in long‑term, this life cycle of redeployment and design will have less of an effected impact on the very communities and peace building it's designed to develop. And the final point I would raise is life cycles often ‑‑ we use the term cycle. What do we mean by cyclical? What does that actually imply to how things are used?

The problem is if you look at a life cycle as a series of stages that are at one end and produced at the other. And perhaps cycle round again. It seems like a conveyor belt. It could be seen operationally by the designers, procurers and the ultimate deployers of these systems as a set of check boxes.

But what that means is that we have, rather than a series of checks and balances and means of ensuring these risks are not compounded, we have a series of things which are simply checked off as complete. Without sufficient evidence to the fact. Without the ability to understand, is the system suitable? Is it being useful?

They're recycled. The requirements might be changed. And deployed in a different context through a different purpose. Then we find ourselves further compounding the issues that we saw before.

So, what I would like to say is what we need to be aware of, this life cycle, finally taking away from it. If we are to ensure these systems being used in a manner that is humane, ethical. And adding to our security and building peace. Rather than creating or recreating, rather, the conditions that have led to insecurity, unethical practice, and a risk to both civilians, combatants and other already highly impacted and at‑risk individuals. We need to ensure not only do we have a shared understanding of how these are made in the different stages of their life cycle.

We need to understand and come up with a means of technical, ethical, and humanitarian governance. Which intersects with all of these stages effectively. And I'll leave it there. And look forward to your questions.

>> YASMIN AFINA: Thank you very much, Alexi, for, again, this very comprehensive overview of why the life cycle management approach of the governance of AI is really important. I particularly like the way you ended the discussion and your remarks. By noting that this is a prerequisite to ensure that these technologies will build peace. Rather than exacerbating the sources of insecurity and instability.

On that note, we have a bit ‑‑ around 20 minutes, I would say, for an open discussion. I would highly encourage, for those who are both in the room in Oslo, but also who are virtual, joining us virtually, to ask questions to our panelists. But also building on the Slido discussions that we had earlier. Where we collected your responses of what AI in international peace and security means.

But also the role of the multi‑stakeholder community. I would encourage you also to take the floor to elaborate a bit more on these answers. But also if you have anything else to add based on ‑‑ for example, we heard from Alexi the importance of local context. How is AI being deployed and used, for example, in your respective regions or states? Or your organization? To build peace and enhance international peace and security.

On that note, I would like to open the floor now for those joining in‑person and online. For those online, I will keep an eye on the Zoom. For those joining in person, I believe there's a microphone on the side. For those who are joining from the floor. Or perhaps those who are joining on the center table. If they would like to take the floor, I think there are microphones in front of you.

On that note, I'm opening the floor now. Perhaps give a few seconds, as well, for to you collect your thoughts or your questions. Please introduce yourself. Share your name. Where you're coming from. And if it is directed to a speaker, please do so.

>> PARTICIPANT: My name is Frances Alomay from the dominion Registry. It's more of a comment. So I know AI is widely used. AI is something that a lot of persons had jumped into one (?) context of AI.

So, part of the things that the AI adoption is driving at is trying to make imaginary things come real. And they should look at ways to actually, you know, make AI‑generated content have more of a signature look. That people can actually easily identify what AI is generated and what humans have generated. In a way you look at some video content. You see a lot of ‑‑ there are video content that you see. You would think they are real.

Those kinds of contents can be used to pass some kind of false information. Can be used to actually instigate some kind of violence in some places. Where you see some kind of content that is actually not culture friendly. Or something that can actually instigate some kind of thoughts in the minds of people.

I think that should be more ‑‑ that's kind of a signature or that kind of thing to identify AI‑generated content and human content. Thank you.

>> YASMIN AFINA: Thank you very much. So ensuring some sort of signature or at least a means to verify what is AI generated, what is not AI generated. And perhaps the security implications of not being able to differentiate between the two. I see we have a hand raised virtually by Bagus Jatmiko. Who is joining us from Indonesia. Could I ask the IT technicians to display him on the screen? You have the floor now. Thank you very much.

Bagus, if you can, please unmute yourself. And turn on your camera, if you would like to intervene.

>> PARTICIPANT: Can you hear me now?

>> YASMIN AFINA: Yeah, we can hear you.

>> PARTICIPANT: Okay. So ‑‑

>> YASMIN AFINA: Although we cannot see you. I'm so sorry.

>> PARTICIPANT: Okay. There you go. Sorry for the connection. And also the technical issues. So, I see very familiar faces in this conference. And would also like to bring some concern.

And also would like to maybe bring some question to the panelist also in a way that I'm working in the defense sector. And AI is being used exponentially. And I talked about this during the ICRC conference last week, if I'm not mistaken. I bring concern about how AI is being used in a way that some of the commander or user within the military domain is unaware of the possibility that AI might be corrupted during the use.

Like what we call as emergent misalignment. There's also the misalignment with the system itself. And I would like to bring the concern about the possibility of the ‑‑ maybe it's not possibility. The tendency of AI being cycle panned in a way that AI would also maybe provide the answers that the users would like to bring or would like to see.

And being in the battlefield, that kind of tendency would be very, in a way, very risky and maybe dangerous. And how they can actually misalign the user or the commander in the battlefield in taking the whatchamacallit, the decision that might increase the risk for the humanity. And also for the civilian population.

This goes to my question. How would you all maybe provide the what you call, maybe attention. And how the AI is being used. Especially in the military domain. This is for all the panelists.

How would you maybe encourage more into the use of AI, responsible AI within the military arena? Because if I relate it to the humanitarian, somehow in the uncertainty, mostly commander would like to see quick answers provided by AI, DSS. And maybe they ignore the possibility or the existence of law, or humanitarian law in this case. Question back to you, Yasmin. Thank you.

>> YASMIN AFINA: Thank you, Bagus. May I ask that you please introduce yourself? For those who are not familiar with your work. Or where you're coming from?

>> PARTICIPANT: Sorry for not introducing myself. My name is Commander Jatmiko. I'm actually an Indonesian Navy officer. And I also research in AI and warfare. Which brings me to the concern about use of AI in the military sector. Thank you.

>> YASMIN AFINA: Thank you so much, Bagus. I see a gentleman here would like to ask a question. Or perhaps share some comments. And then I'll get back to our panelists for answers or reactions. Gentleman, please.

>> PARTICIPANT: Good afternoon, everybody. Allowing to raise very short question at the beginning. Who is responsible for the mitigation of AI risks? This is a very short question for me. Is it the high‑tech big companies who are creating AI and develop AI?

Because it is not in the hand of the governments, especially in the developing countries, right now. So, let me have that big issue here. While I'm following the development, address‑ment of AI, especially in fields which are related to security, I am terrorized. You know?

I am not going to mention or name any country now. But we can see how AI is being used in current, ongoing wars. And the victims behind the use of AI technology and autonomous weapon, for example. How civilians are being killed without accountability.

So, for this reason, looking from a developing country's perspective, which has nothing to do in their hands right now. It is all in the hands of the big tech companies. Which exists in the powerful countries. So this is my issue here. How we are going to mitigate ourselves of this risk. Thank you.

>> YASMIN AFINA: May I please ask that you introduce yourself? Sorry. Can you please introduce yourself in the microphone? Just so that we know who you are and where you're coming from?

>> PARTICIPANT: My name is Zsa Zsa (?) and honorary professor of law at university in the UK.

>> YASMIN AFINA: Thank you very much, Sir. In the interest of time, I realize that we have 10 to 15 minutes left. I just want to make sure. And check in the room virtually or online ‑‑ virtually or in‑person, if there are any further questions, comments or remarks for our panelists. Or to add to the discussions today.

If not, I know that Alexi, you've also put in the chat that there is an ongoing project on adding signatures to AI‑generated content. The Content Authenticity Initiative. And perhaps, Alexi, you will be able to elaborate more.

Before I do give the floor back to the panelists, I know a question from Rowan Wilkinson. Hello, Rowan.

Many policymakers are discussing the importance of AI openness in civilian contexts. Including in meeting safety commitments through OSS and community oversight. Do the panel foresee a policy shift around openness in the AI peace and security domain? We had a question surrounding AI authenticity. And implications of not knowing what is generated and the destabilizing effects.

We had a question from Bagus on the commander and perhaps the human interaction in the battlefield. And perhaps also how do we make sure that the use of AI remains responsible in the hands of the commander. Particularly under situations of pressure. Such as in the battlefield.

We have a question of who should be responsible for the mitigation of risk of AI. Particularly in the light of ongoing conflicts today. And the implications of civilians. And, finally, we have a question on openness in the AI peace and security domains.

Perhaps, may I ask, in the interest of time, Jingjie, to start us off with three to four minutes? Please feel free to answer any questions, or any other element you would like to add based on what you've heard today. Jingjie, please, you have the floor.

>> JINGJIE HE: Thank you for the question. First the question from Bagus. In the military when you deploy AI system. You developed it first as a project. And you deploy it.

So, many times, based on my experience from the civilian field and industry, many times the one who makes the decision whether to use, deploy or complete a project may not always be the one who understands the technology. So knowledge sharing is very important. Transparency is important. Those people who make the decisions need to understand the technology perspective.

And also the second point I want to make is the importance of incentives. It is very important for the military to understand that AI is not only a force multiplier but also a stress multiplier. It is not only about the risk of civilians. It is also about increasing risks of your own combatants when you have poorly designed, unverified AI system with uncertainties. And you cannot be confident about it. There's a whole black box.

So, this kind of incentive is very important. With this understanding, I believe many militaries will be more incentivized to improve their systems. A quick answer for the second question. I'm sure Michael will talk more about from the industry point of view. Everyone is responsible for raising a voice. Being sensitive about the importance of AI governance. You know, incentivize or promote dialogue about the AI risks.

And the third question about AI openness. I'm not exactly sure what is AI openness. If you are talking about openness about the algorithm, I think it's very difficult. Because when we're in the industry, when we go for due diligence or technological scouting, we ask the company, what is it? What's your core technology?

They were reluctant to tell us. That's air own IP. They did not want to reveal it. But, look, we have a system. It works perfectly. Just believe in our results. This is what happens.

If you're talking about openness about AI algorithm, I have a huge question mark about feasibility and plausibility about these kinds of solutions. Thank you.

>> YASMIN AFINA: Thank you, Jingjie, for your very sharp response. And also the fact that you're actually joining us from very far. And it's very late. And you're at home. So thank you very much for this.

Michael, would you like to intervene now?

>> MICHAEL KARIMIAN: Thank you, Yasmin. Happy to do so. I should flag, Zoom keeps telling me that my internet connection is unstable. If I pause at any moment, that will be the reason why.

In answer to the question on AI signature, I appreciate the question. I think one way of thinking about this is, are there specific use cases? Where we really need AI signatures. Or would we be comfortable with other use cases where we don't need them? I suspect that's possibly the direction we will go in.

Of course, the proliferation of AI solutions means that there will always be solutions or actors who would circumvent that anyway. That doesn't downplay the importance of having AI signatures in the first place. To Bagus' question on misalignment and AI‑supported DSS, I think your question really points to something that we recently discussed in the Roundtable for AI, Security and Ethics.

That's the challenge that exists at the moment in having access to meaningful and trustworthy use cases. To understand, in very effective ways, actually how AI is being used. I think the academic community, society, industry, governments ‑‑ actually, at the moment, we are relying on a number of examples. Which partly comes from hearsay or possibly that reflective on how AI is being used in security domains.

I'm hopeful that as AI is further adopted in various security domains, transparency around use cases will improve. And then we'll be better able to understand implications of them. To the question from our colleague from Egypt, the University of Durham. Jingjie is right.

Everyone, from a human rights perspective, has the responsibility to respect human rights. Industry has a corporate responsibility to respect human rights. Individuals have a right to remedy. When their rights have been harmed.

Focusing specifically on the role of industry. What that means is that all companies under the U.N. guided principles on business human rights have a responsibility. To ensure that their products and services are not being used in ways to facilitate or contribute to serious human rights abuses. This means any engagement with a government, military defense, armed forces ‑‑ especially in the context of ongoing conflict or credible allegations of international law violations ‑‑ must be subject to rigorous due diligence.

Red lines on misuse. And where it should not be mitigated, there should be a refusal to provide or maintain support. Actually, that's not new. This has been an established permission for a number of years now. Of course, what matters there is implementation.

Lastly to Rowan's question, I hope we will see more openness. One example of that is the REAIM process. Which I mentioned earlier. Responsible AI in the Military domain process.

Hosted last year in South Korea. And in six months will be hosted in Spain. Anyone in the audience, stakeholders, who are interested in this should keep an eye on the REAIM process.

>> YASMIN AFINA: Thank you very much, Michael, for this. And the importance of differentiating principles. But also actual implementation. And the importance, as well, of human rights. And providing a framework to ensure that everyone is, indeed, held accountable. But also to ensure that civilians have a right to remedy. Including in the context of AI for international peace and security.

Finally, Alexi, would you like to have any concluding remarks and responses to any of the questions raised?

>> ALEXI DREW: Thank you. I'll do this nice and quickly. In the interest of giving people their time. The silver lining to the mis‑signatures and kind of demarking of inauthentic content or AI‑generated content from human‑generated content, as somebody who used to work at arms control, there's a great thing to bear in mind.

Every time a new threat arises, or new innovation creates a threat, it's very quick for (?) to be developed. It's just as true in identification of inauthentically generated or machine‑generated content. So I'm encouraged to see it's not just CIA that exists in this space. There are a number of initiatives going on technical and nontechnical means.

To give us the means, as Michael says, in critical circumstances, being able to identify when content is being generated. As opposed to when it has been created and is authentic. On the question that Bagus referenced, compliance commander's component. This is part of what I was referencing. When I was talking about the need to ensure that governance, both ethical, legal, and economic, is built in at every stage of the life cycle.

A system should be designed, trained, tested, authenticated and verified with the data selected. With its need to be compliant. If it isn't, that's when you're introducing the risks that something could be designed. Which is either completely incompatible. Or is open to being ‑‑ to be used in a manner which is not compliant. You should actually treat the life cycle effectively in how you incorporate across it. In the assurance stage. Or treat it as a check box. Then you can constrain the risks of that going wrong.

That being said, there are other components to that. The fact that any system should be trained to understand what it can and what it cannot do. What does it look like when it fails? What are the circumstances which have led to its failure in testing?

What's the influence of its level of accuracy? So they can make informed decision of how much, or whether, in fact, to trust an AI‑based tool system or weapon system. Be it a decision‑making, strategical, tactical, or be it a direct weapons system. It should also be, in some cases, that these tools simply aren't used.

Because it's understood, because of how these systems have been designed with IHL baked into each part of it, that they simply cannot be compliant with IHL. On the subject of trust around these tools and how particularly LLM has been found noncritical of their human uses and how they might influence ‑‑ yes, that is a problem.

They're not designed to be critical and push back. They're designed to be supportive, administrative assistants that say yes a lot. And that should be something that's understood as a potential failing. And implications to how a military should design, create doctrine and deploy at all.

Moving quickly on to who owns it. Where is the responsibility. I agree with both speakers, Jingjie and Michael. Everyone has a responsibility here. There are levers in the ownership structures between the private sector, public sector, Global North and south. And those with less.

Everyone has a lever they can use here. Be it taking part in globalized standard setting organizations, technical or nontechnical. Or be it in procurement strategies and procurement standards. If governance is critical ‑‑ IHL, ethical, economic, social ‑‑ then it should be a commissioned procurement from government to suppliers. So even if they don't own the system or the services required to operate the system of AI as a service, that it has been designed to fit these standards. It may be legally necessary to do so to meet procurement methods.

Finally, I'm trying to be positive amongst the negativity here. I think we're at a point here where innovation is being posed as a solution to our increasing state of insecurity and risk to peace. It's being positive as a zero sum game between innovation and security. Or insecurity and constraint on innovation. That is not the case.

You can, in fact, have security and innovation with adherence to values and governance. Which protects all individuals from all societies. And delivers the economic and social value we're trying to seek here. It's important we push back on those trying to stress that zero sum dynamic between those two.

And I'll leave it there. Hopefully on a semi‑positive note.

>> YASMIN AFINA: Thank you very much, indeed, Alexi, for ending us on a semi‑positive note. But also for Jingjie to also add a point on the fact that Chinese social media also adds signatures to AI‑generated content. And I think that also adds to the importance of collective responsibility. To ensure responsible AI in international peace and security.

I do know the importance of incentivization, noted by Jingjie. Compliance with IHL. And, yeah. On that, hopefully, positive note that we are ending this workshop.

Thank you very much, everyone, for joining us today. Either online or in‑person. Please join me in giving a round of applause to our speakers online. Thank you very much.