The following are the outputs of the captioning taken during an IGF intervention. Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid, but should not be treated as an authoritative record.
***
>> KENNETH LEUNG: Good morning fellow Internet practical significance. Good to see you here. Especially this is the first session of the today. We have people from around the room and across the world joining us online. Throughout this hour we are responding to different sectors as well as stakeholder perspectives about the determination of sexual deepfakes across the Internet. Dramatic rise, this is what an Austria Internet studies called on the number of AI tools specifically designed to create deepfake images of identifiable people. This very timely study formally just published this Monday, unvailed nearly 35,000 AI models available for public download on one platform service for generative AI, many of which are even marketed with the intention to generate NCIIs, nonconsensual intimate imagery. The digital service of concern has responded to the Austrian study and taking action. And some governments are also taking action. Take for example the UK20205 data use act has just become effective last Thursday with the provision criminalizing the creation and requesting the creation of purported intimate images. However the additional safeguarding only applies to adults as such behaviors as explained by the government are targeting minors have already been covered by the law. But is that enough?
Especially safeguards in the age of generative AI for teenagers, the so‑called in between phase of the innocence of childhood to adulthood. This workshop will directly and exactly be focusing on this with three questions. Fm one, what legal and educational measures are most effective in addressing the creation and spread of sexual deepfakes among school‑going teens
Number two, how can different stakeholder collaborate to ensure that school curricula incorporate digital literacy and awareness about the dangers of sexual defects
Number three, what proactive policies can countries implement and anticipate technological changes and prevent sexual defects and harms against teenagers globally.
My name is Kenneth. Joining me on stage today we have from my left Ms. Oh Won jism bringing the youth perspective. And Mr. Yi s Teng AU, and Ms. Janice Richardson from Inside SA offering views from the education field and Ms. Juliana Cunha giving insight from the NGO standpoint. After initial remarks from our speakers, we invite you to chime in with other questions, comments, perspectives on the topic. You have two minutes to share your thoughts either in front of this mic or online by using the raise‑hand function, so if you have more thoughts and resources that you would like to share after the session, we'd love to have them and please share these on this website, CSIT.TK by will be closed after one week of this workshop for us to synthesize the discussion right now and on the comments platform to consolidate into a report next month. So it's csit.tk on this platform will be up right now and I will show it up one more time later
So hold your thoughts for now in the first 20 minutes and let's hear from Oh Ji Won and Yi Teng AU. Oh Ji s Won has a master's degree in political science and international relations and Yi s Teng is a technical major in ‑‑ scoins and Microsoft certified AI engineer over to you
>> JI WON OH: Okay. Hello, everyone, today I will talk about how deepfakes affects students ‑‑ first let me explain what a defect is. Defects are highly realistic video, audio, or image for ‑‑ replaces using generative AI. The tech following that create defects include generate ‑‑ virtual networks and tsh so with deepfake tools people can copy someone's face and voice without permission. In August of last year, a big problem was found in Korea schools. Security chat room on telegram were using deep fake sexual videos and using real face of middle and high school students and so this news shocked the whole country with the technology and social media platforms, anyone can make a sexual video easily, it's not just older. Many young people are involved, so in 2021, there was only 156 reports about deepfake news crimes in Korea. In 2024 the number increased to 1, 022 cases. About two times more. Most people who made the deepfakes are teenagers.
So one student heard the news will they be scared and think that ‑‑ think that they should never do it again? Not sure. Many students now use deepfake tools. These tools are easy to find online, and even young students can make deepfakes, but here is the problems, actually many students don't care ‑‑ do not be care because they think it just funny, and laugh and share the videos. They don't think about the pain these cause. We can think many students shocked, scared, and frustrated when they see deepfakes but victims feel anxious and unsafe and suffer from social stigma because of fear. ‑‑ deepfake can hurt expectation and make them feel helpless.
So therefore main reason why deepfake problems are growing. There are many reason why sexual deepfake crimes continue to happen in Korea even after the law was changed. First, it is still unclear whether the new laws are strong enough to stop these crimes. Some people say that more action is needed, especially to mechanics Internet companies act master, even when video, they can spread for long time if the removal takes too long. Experts say that Internet service providers must be more responsible, and they should block, monitor and prevept deepfakes before they spread. Also the law should not only punish people who make deepfakes, but should also punish others who want to create them because technology change so fast, we need laws that are clear, strong, and stable even for new types of crimes.
Second, these issues is not only about sex crimes. It is part of a bigger problem. Misinformation affects contents. For example affects videos and politicians and can affect democracy by spreading lies. We need a national response with all working together. It's finally one of the most important to education. Many young people don't really understand how serious of these crimes are. Some even think it's funny or harmless. That's why schools need to ‑‑ what are deep fakes and how dangerous they are. And also people and better digital literacy to better be able to understand what is real or fake. We must help other young people and others to be more smarter online. What can we do? Where deepfakes can happen, start investigations, and then try to find who may share the video. A school separate the victim and offender. They also give counseling and supporting to the victims. There are some legal protections but they are not enough right now. We need more stronger laws to punish the defect crimes in all economies, so we also need to educate students and raise awareness that's important to support the victims and protect their safety. Okay. Thank you so much.
>> YI TENG AU: Hi. To put things in context here are numbers I want to share. In South Korea over 500 schools affected. Many taken photos seriously as school often social media. 2145 middle and high schoolers showed that 54% of offenders just did it for money. Other reasons include curiosity or thinking the consequences were minor, highlighting a lack of awareness about the seriousness. In response, the Ministry of Education published five guidebooks this April tailored to different age groups, there is a cartoon version for elementary students and separate editions for middle and high schoolers, teachers and caregivers, they discover three case situations, where you find the victim or if someone around me is, and what if something I did cause harm.
For the technical folks here, I'll highlight three innovations. First, the Korean deepfake data set released in twup contains 2.3Terabyte of video. It's ‑‑ with this some national university students achieve 96% accuracy in detecting videos. Economies without such datasets might consider developing one tailored to their local needs.
Second, the metropolitan government develop detection tool enhanced in May 2025, targeting elicit content involving minors, it identifies items such as school books, uniforms, and even ‑‑ flagging under content even when face is unvisible, and also scans systems and drafts multilingual reports depending on the site host country.
Lastly, I will share about a phenomenon called platform hopping, many deepfake crimes in South Korea begin on telegram but as the company now actively cooperates with software authorities, on a form ininsight, it is noted perpetrators have shifted on platforms making detection harder to lower hopping. Thank you.
>> KENNETH LEUNG: Thank you. Thank you for your sharing the Korean case but as what is the Korean government after the case happens. And I just want to also plug here on the Asia‑Pacific policy, as this particular case was also heavily debated and spotlighted in one of our latest analyses on how the recent advancement of AI capabilities are transforming online safety and security, which includes challenges in the search of AI‑generated child sexual abuse material and AI‑powered gender‑based violence, so I would invite you to take a look when it's out this week during IGF and I will be sure to send a link up at that commenting platform, csit.tk.
So, and now we'd love to hear from Janice for thoughts in education and the private sectors perspective. Janice Richardson has been an educator for 50 plus years in different countries including Australia, Europe, and on the safety advisory both of Meta and Snapchat and partner in the European Commission and Council of Europe projects focusing on AI, misinformation ‑‑ democracy on all levels of education. Janice?
>> JANICE RICHARDSON: Thank you. First of all, thank you very much to all of those people I reached out to in UK, Poland, France, Netherlands, who gave me information about what are the solutions. Let's begin with what are the challenges. Schools still have a tendency to post the face, the image of their pupils in sports, in all sorts of activities. This is the first problem. There are so many images out there, maybe they were con secht iewl in the beginning but they can very easily be used when creating deepfakes.
Secondly, it's a availability of tools such as Noodafy, undress, dressed, I don't know, there are so many out there, and I find it absolutely amazing that they can still exist on the market. Then of course we have the enforcement challenge. We've done quite a bit of work Merocco training the judiciary so they understand how to collect electronic proof and how you use electronic proof but not all law are adapted to this type of proof, and therefore we do need some legal amendments in many countries.
The media the way they report these things, we've also started training the media to make them understand that being spectacular may be a way to make them come to your website, but it's certainly not helping the victims and it's actually calling for a lot of copycat behavior.
Then I would cite the lack of cross‑platform collaboration. I know there are projects such as Lentin where companies, social media in particular, come together to share knowledge. But the problem with this is that it's across platforms, it happens with many different layers. And until industry joins up, I think it's going to be very difficult to find a solution.
Industry needs to be a partner education. It shouldn't just be there supplying tools or pushing their tools, I could say, on to the education sector and then throwing out bits of education to help young people. It should be their when we're developing curricula, finding ways together that we can use real‑life cases, real‑life resources in a way that will be much more impactful for learning.
There are, of course, many issues when we look at education systems because few teachers have the training that's necessary to be able to tackle this issue. We've started an innovative project in Morocco where we've trained two from every one of the the 86 regions and they trained in cascade, and our idea is to have two teachers that fully understand the issue, who receive regular updates in every school so that if there are two teachers in every school, two resource persons, there should be a very fast way to escalate the issue.
There are lots of very interesting programs out there. I can cite what's going on in France, for example, where law students at a university have come together and created a poster competition because they feel if they create the framework for a post are competition, providing little bits of the law so the public becomes more familiar with the law, then everyone can create posters which will be meaningful and informative and reach the young people who are very much concerned by this issue.
I really like something that happens in the Netherlands, too. They have lots of television programs, debates, but every school there has the freedom to choose the way that they're going around this issue. But when a young person goes from elementary school to middle school, they go through a six‑week course on health, on technology, on all of the issues that can really help them tackle these issues, these problems, because you know the big shift when they go from elementary school through to secondary school.
We have a very innovative project running in Scandinavia, thanks to the support of Heuwii. I train trick to put in a position. Then it's the magician that in one‑hour sessions in schools, deliver what I've prepared and what helps them to tackle very, very interesting projects in a fun way, a problem in a fun way so that kids are on board, on interested but they don't feel threatened in any way.
There is one big problem that I've noticed in all of the countries that I work in, and it's reporting. The humiliation if you have to report that an image of you has been shared, especially in countries like Morocco or Tunisia, so we need to take another approach to reporting. We need to educate children from the cradle, understand, one, the importance of human dignity; two, when something is not going right, there are ways to say it that are not hurtful, that are not telling tales, they're the right ways to do it.
Should they mention the help lines, this has been a super initiative across Europe. I actually got figures from the Netherlands which shows me that in about six and a half,000 cases are about AI generated sexual fake profiles, but it's only days, it's going to get worse.
Therapy, support for the victims, these are all extremely important areas. But when I talk to young people, they say actually, it's not the nude profiles that are bothersome, if someone puts a a backini but sexual stance is it a fake profile, is it a sexual profile, this is where it really raises issues, if there is a little bit of clothing, then they think they're out of the category of a sexual fake profile. Young people also come to me and say you should be educating the adults about this because very often, it's them the ones that are suffering, the ones at that are doing this, and the ones that don't have the education to tackle it. So I would say that's a broad sweep of where I think we should go of the education projects we should set up as many of you know. I believe it's all about digital citizenship and understanding that you are a citizen, you have an obligation for all of those around you regardless of what it is, you also have an obligation to share your knowledge because if we're here today, we have knowledge about this subject so we need to share it so that it really becomes a grassroots movement to educate everyone on how to tackle this plague. Thank you.
>> KENNETH LEUNG: Thank you. Thank you, Janice. It's indeed a broad sweep of issues that you've mentioned and lots of good solution, and you also mentioned about cases from Scandinavia, Morocco, Netherlands, Tunisa, that means this issue, sexual defects are really transcending the border and it's a global issue, so this is why we are here and I would like to now turn the floor to Ms. Juliana Cunha for Latin American perspective and some of the work that you do. Juliana holds a Bachelor's in psychology and masters in culture and society. At Safety Net Brazil, Juliana coordinates the national helpline safety interacting with and counseling children and teenagers and adults with sexiting, and other risks online. In this regard she also collaborates with media outlets and tiding bodies to create national and international award‑winning campaigns. So after Jul iana's presentation and sharing her perspective, I invite everyone who wants to share to come to the floor in front of the mic on this side or for student participant, please do type in your question and we'll consolidate and address those questions. So, the floor is yours. Over to you. Thank you.
>> JULIANA CUNHA: Thank you. Good morning. I'd like just to do a brief introduction about Safer Network. Safer Net is a 20‑year long nonprofit organization dedicated to establish a multistakeholder approach to protect the human rights in digital environments. We work as a safer Internet center which coordinates actions in three pillars of online safety. The Brazilian national cyber tip line, hotline which users can report anonymously crimes against human rights. The national web base national helpline which offers online conversation about different risks and provides support to children and people and families and educators about the online safety.
And then the other pillar is the country's awareness and education hub which is responsible for the educational activities involving workshops with students, educators, families, developing materials and carry out campaigns on digital citizenship.
About the context, Brazilian context, in in 2023, Safernet reported a historic spike reported to child sexual abuse material online in Brazil, and the key factor was the rising of the use of AI tools such as notifying apps as mentioned in bots by young people to generate and share fake notes of classmates. This problem already affected large schools in Brazil, especially private schools with several cases being reported by media outlets. The new trend challenges us to find inappropriate response, especially due to the fact that victims and perpetrators are minors and the boundary between sexual experimentation and sexual abuse is becoming a little bit blurred. And this phenomenon I think it is reshaping the way young people perceive sexuality, relationship and consent.
I would like to highlight increasingly in reports in Brazil, especially involving telegram, as I mentioned before, this is a huge challenge for us in Brazil right now because in 2023, 24 90% of the reporters related to messaging appears in Brazil involving telegram reports. WhatsApp and Single together account for the remaining 90% during the same periods.
When you notice ‑‑ when you notify the company limits digital response to the ‑‑ evidence with little cooperation with law enforcement agencies, it was noting that at least 38 countries have legislation requiring legal ‑‑ requiring digital platforms like telegram to report to authorities when they are aware of hosting child sexual abuse material.
In the United States for example the national center of missing and exploited center received 20‑million industries to be reported in 2023. None of them from Telegram. Last year, they saw increase of 1, 325 increasing reporting involving generative AI going from the 4, 700 in 2023. In 6700 reports in 2024. It's increasing, a large increase number.
This alarming trend is also noted in Brazilment the persistence is the response is inadequate response, moderation of content, and Telegram non‑comply with Brazilian chat protection laws led Safer net to file a formal complaint against the company with the federal prosecutor office in October last year, and this is the report based on this complaint, and I think it's interesting to show the proportion of the problem in Brazil, so this is the code to access the report.
So to deep dive, we run ongoing projects from safe online coalition wil we investigate how teens in Brazil are affected by the use of generative AI to create and spread deepfake sexual images also known as deep nodes ‑‑ we are listening for key groups in these incidents, those who image was manipulated and shared, those ro kret and stair, bistanders, educators, caregivers, we'll conduct one‑on‑one interview with survivors and perpetrators. Co‑creation workshops with bystanders and one session with with educators. Our expected outcomes include qualitative insights into how things perceive and to engage in AI‑generated sexual images, practical recommendations to inform child center safety policies in Brazil, research for platforms to improve their trusted and safe responses, a national awareness campaign to help identify, report, and resist this form of abuse and a step‑by‑step methodological manual to replicate this work in other countries and contexts.
One of the main challenges, I'm concluding right now. One of the main challenges is going beyond legal and punitive measures. Why accountability is essential, this phenomenon is deeply embedded in a culture that reproduced gender‑based social norms as mentioned previous. Is the misuse of AI to create sexual images of peers is not a tech issue or legal gap. It's a reflection of a broader systemic gender inequality. That's why prevention must go ‑‑ must also be cultural. We need long‑term interventions focused on education, awareness, digital literacy in schools as Janice mentioned where social norms are being form. This project is not only to ebbing pose the harm caused by AI generated sexual image content but as to empower teens, educators, and communities for critically engage with these issues. Our core pleaf is the best which to pro teblght children is by listening to them, this project challenges usually top‑down approach instead of responding with panic or punitive measures, we're asking how do teens actually experience this and what do they think would help.
And to conclude my thoughts, to effectively address this issue, we need a coordinated multistakeholder approach, response, bringing together tech companies, public institutions, and society in academia. We have seen some initial steps; for example, Meta took action against a company.(?) advertising rules to promote notifying apps in these platforms. This is the first step that must be followed by other measures. We need the entire industry across sectors to take stronger action especially against pickup trucks deliberately designed to violate rights, exploit vulnerabilities and bypass existing safety standards. This requires clear accountability for fables, stronger safeguards built in platform by design, and about child‑center innovation that respect human rights.
Just to conclude, I bring an example of education resource in Brazilian Portuguese. To address the issue in Brazil in partnership with UNICEF Brazil, it combines legal guidance, real‑life cases, and interactive activities. It's a tool toes for tr dialogue in classroom and with groups, and the type of resource helps to equip youth with the tools to understand their rights, recognize harm and build a culture of respect online. Thank you very much.
>> KENNETH LEUNG: Thank you so much, Juliana. Thank you for the Brazilian case and how there was misuse and how to disseminate information and how to create a on the platform. I also like how you have been stressing the importance of cross‑platform and cross sectoral collaborate rigs in combating sexual deepfakes.
Now I open the floor to everyone. I would love to hear your statement, questions, comments both here and online so and you have two minutes to have your statement and please state your name and affiliation when you do so, please. Please head to the mic.
>> AUDIENCE MEMBER: Okay. I just understood this. Can you hear me? Okay. Great. Hello. Great to meet you all. Thank you for the great panel. My name is Mariana, Columbian, I work at an initiative, very happy to hear the Brazil case. I work and I'm leading one of our big, big projects that's actually very close to my heart, which is called the Youth for our Data Future project, we've been engaging young people in different parts the world and pretty much in the conversation around data governance and right now we're wrapping up that project and we're starting to shape an initiative focused on influencers so one of my big questions and I think it's for all of you actually, is if you've touched on this topic, and how have you start thinking about the role that not only influencers themselves as adults that are influencing the online space, but young people themselves and in Brazil there is a very interesting case around kid influencers, so 4 or 5‑year‑olds that are famous because of their parents influence and they're put online and sharing content and creating content themselves or parents creating content with them. So one of my questions is how do you see this interplay and what's the role of the influencer industry in shaping safety online, and have you thought about any kind of actions that could help us build a safe are and more inclusive Internet. And there I just close my question and this is more of an invitation to anybody interested in the topic. I'll be very much ‑‑ very excited to talk about this later after the workshop is over because I'm very excited about the work that you're all doing. That's it. Thank you.
>> KENNETH LEUNG: Thank you so much. I'm seeing ‑‑ I think I see there is a long queue and we do want to get through every statement, so I would suggest to have maybe three statements or interventions at a time, and then the panel will address it clctively if that's all right. Thank you very much for your first question. Thank you.
>> AUDREY AZOULAY: Hi, my name is Francis. Part of Youth dig. I have two questions. Janice I think you highlighted this. Why not just have a real crack down on these kind ofapplications because I think a big reason young children are using these apps is because it's very easy. It's easy to access. You just download them. And stores providing the applications, whether it's advertisers being noodifying or not, necessarily means young children think it's the same as Instagram or snake game or things like this. The message you send to young people allowed on the app store is this is equivalent and equivalent to other fun apps. So why can't we outright ban it, I think this is something that charities in the UK are trying to push now and what difficulty do you have with trying to draw the line about what apps you do ban depending on what kind of too manies they allow young people or people in general to have. The second thing I just wanted to raise was I think I very much agree about education in societal and tiewd nal changes is problem about gender‑based violence and just another form of it, right, and I think in krung children we're talking about deepfakes today but I also think educating young children, perhaps, especially young women that when you share even real content with people who you think are trusted individuals as a young person, which it's easy to feel this way if you're having a relationship online or quasi‑online, then it's very important that these young people know that content that gets shared online is always online, so even when you have WhatsApp as end‑to‑end encrypted and can send one‑time images they're never one‑time and the technology is very different in person relationship, I would also say that. Yep.
>> KENNETH LEUNG: Thank you.
>> AUDIENCE MEMBER: Good morning from the Netherlands as Janice already shared our numbers thank you very much. I won't go too much into the statements. I was wondering two things. I went just sitting here, I we want to Google, I went to Bing and I went to Duck Duck Go when you put in best nudify apps, you just get them that's how easy it is. So a global legislation answer might be difficult. We've seen it with Mr. deepfakes in the Netherlands they're criminalized and privacy law and you can act VPN and act like you're from a different country, that's difficult. Then I think can you do something about the search engines, curious about that.
Second as is it so difficult to have a legal global answer? I do think, but I'm a kind of glass half full kind of guy, it is possible to have a more global awareness answer, very curious how you look at these two questions. Thank you very much.
>> KENNETH LEUNG: Thank you very much. Since we have three speakers to intervention ready, I guess we can address very ‑‑
>> YI TENG AU: I'll address why can't we ban and results. So actually in many countries the laws against deepfakes is very muddy because it's a picture that is modified. Although it's a picture of a face and a body, it's a different ‑‑ it's not ‑‑ a lot of times it's very hard to persecute the perpetrators as well as there is many societal issues as well, so in South Korea there was a anecdote from a teacher whereby an intern ‑‑ there was a photo, sexual deepfake of a photo uploaded to X formerly Twitter and perpetrator found out it was her student while she had to find the student herself because the police did nothing. And then there is also many ‑‑ in short the law needs to improve to actually be effectively punishing those perpetrators. Yeah.
>> JANICE RICHARDSON: I would like to respond to a couple of points. Influencers, kids are actually sick of them. Maybe this is a trend which is not growing, which perhaps will start dwinding out, but once again it's up to education to show that there are real places and real people who understand their subject, and let's go to those people and not to someone trying to sell me a product.
Secondly, yeah, I asked the same question. But in fact we come here, we talk, and then we don't do anything about it. And if people like us came together, formed a group, started putting pressure on the search engines to delete access to these type of apps, then I think we may get somewhere. It's very easy for industry. They have the money, they're getting our data, they're seeing where people are going, but why don't people join together? Why don't people make an international action against this. That's my call, let's keep working on it. Yeah. Once online, always online, there is nothing more that we can say, except teach our children from age 1, 2, 3, Internet's great to explore but they have to know that what's mine is mine and my privacy is the most precious possession I have.
>> JULIANA CUNHA: About the influencers in Brazil, they play a key role in the culture, especially because we have big influencers right now, we are discussing related to gang ling, for example and some influencers went to public hearing to explain, all the model, the business model related to publicize gaming apps ‑‑ but of course in Safer net we have on the other hand an experience involving influencers in campaigns and it's very interesting too because it's effective way to communicate with young people, so you have the two sides of the coin, the influencers that gain millions and publicizing, for example, bad products. And the or hand we have the influencers that is a role model for many young people. I think you have to take advantage to how to avoid this and regulators has the challenge to regulate influencers in Brazil, but on the other hand, it's important to involve them.
And about the other question, I think technically is it possible,py of course there is challenges, for example, how they, these companies bypassing the example of the Meta even when you ban this kind of advertising, companies try to bypass the rules, so I think the challenge is how to technically implement it and anticipate ways the players can try to bypass. And of course the awareness, I really think that if you ‑‑ if it's rooted in the culture, it's really hard, of course, we can do better if terms of awareness as a global trend. It's a new trend. Many people in Brazil doesn't ‑‑ aren't aware about it, but I think that the culture will play an important role so the awareness maybe can can address this culture, because in Brads I will especially, we have the spike of humans ‑‑ human violence so it's very hard to change this reality.
>> KENNETH LEUNG: Right. Thank you so much for your responses. Let's I suggest because we're closing in on the session, but we still want to get through so if everyone can have like maybe just one minute for your intervention and let's do a very quick response, that would be gorgeous. So again please you have the floor for one minute. Thanks.
>> AUDIENCE MEMBER: Thank you very much. My name is Mati fl publisher ‑‑ and also Digi net.pl. First of all I thank you very much for this organizing this session, it's very important. The deepfake phenomena is not something new, but you know accessibility is completely new and makes you know the big difference especially you know the new apps for addressing people, they are rulemaking democratizing this technical science which was, you know, very difficult to make, you know, these things 20 years ago and even two years ago or three. Now it's accessible on a phone, and now so from lawyers because I can say you know about our regulation system, it was now it's very difficult because we have a completely new situation, you know ‑‑ the adults are transform into the child and child transform tolling adult and there are the new situation which are not easy to tackle, especially in the form of protesting in the court, so this is the big challenge.
Also, the big challenge is the victim and perpetrators also can be the children, and for example in our law system this has something like public ‑‑ some pet file register, and it would be it's not true that you know the children are not pedophiles, so we have to change the law, and we are preparing, we are in the process of preparing, you know, the new criminal, new part of the criminal code and change some new regulations. Thank you very much, I think.
>> KENNETH LEUNG: Can I have first this gentlemen for one minute.
>> AUDIENCE MEMBER: Surely. I'll be brief. Hi. My name is Andrew Kemp ling a trustee with the Internet watch. I put in the chat where people with report illegal images, including AI generated and you can report anonymously and added to the block list if they're deemed to be illegal and that will prevent through many services them being shared further which is at least progress.
And that includes by the way on the public spaces on Telegram which has recently joined the IWF after the CEO was placed under house arrest in France.
Question for the panel, I think the biggest issue we have is with end to end encrypted messaging services, you can technically block, prevept the sharing of CSAM on those services but it needs technical action by the service operate ertsers, it doesn't break encryption but that's where we should focus attention because we know that's where CSAM including deepfake generated images is very widely shared. There is lots of research that shows that. Thank you.
>> KENNETH LEUNG: Thank you so much. I guess we can run through all the questions and then panelists can respond with final remarks. Please go ahead.
>> AUDREY AZOULAY: Hello. My name is ‑‑ with the digital foundations based in Germany. I would like to touch on two points. It was mentioned that pictures and photos of pupils, members of thought communities and so on are part of the challenge and I get this point and I understand it, but I'm wondering what will be the solution because I'm ‑‑ I will not imagine an Internet and digital environment without photos, experience of children and young persons online because they also have the right to see their experience reflected in the Internet and to find, yeah, or a connection to themselves, so what will be the solution on that?
Secondly, it's connected to what Andrew Kempling asked, and in the European Union we are discussing since three years of it should be allowed or if it should be okay to compromise privacy and detect CSAM known and unknown and also grooming in encrypted environments, and I would like to know maybe especially from the youth Ambassadors what's your perspective on that? Thanks.
>> Hi, my name is Claire from Hong Kong, what are implementle measures for students encountering deepfakes? Thank you.
>> Hello I'm Sana from ‑‑ my question is given the rapid advancement of AI tools that can generate hyperillustrated content, so what kind of reactions are proactive mechanism are currently being developed, especially about we are talking about the NGOs and also when we are discussing the Korean rapidly increasing video wiring and image wiring, so at that upon the I just have a question of should we prioritize to identify and stop the preed of sexual defects targeting minors before they go viral? Thank you.
>> Yuri from the finish green party. My first question is specifically to Janice, it's about how to improve awareness of this ‑‑ of these issues especially in education when the government rotary legislators in education departments at least in our country are quite omp made up of quite conservative voices which dismiss these issues as harmless, and the second question is more general about we've seen a lot of these regulations that are supposed to protect children be misused by more authoritarian governments to silence free speech and expression elsewhere, most recently today and yesterday in Hungary. What can we do to safeguard against misuse of these regulations that are supposed to protect children.
>> KENNETH LEUNG: Thank you so much. We actually have one last comment on Zoom I'll just read it out. So it's a statement that she wants to chime in on the Korea case and if there is any innovation measures against actually fct effective. I guess in the last 30 seconds everyone can make a remark and end the session.
>> JULIANA CUNHA: Yes, I have no time and I would like to thank you and happy to talk to everyone to continue the conversation about the panel. Thank you very much.
>> JANICE RICHARDSON: If we're so clever with technology, why can't we make something that once we've put an image online, it becomes indel bl, it becomes unchangeable, I'd like more efforts from the tech industry improving awareness, there is a great initiative in Latin America where they actually educate the ministers, the ministers of education so that we get not only a bottom up from awareness razors but a top down from the people who meant to be there who are elected to look ever after us sphok me the problem I really feel is the accessibility of the two, for example even if we outright have safeguards for ChatGPT, especially sexual explicit, people really into it feepped ways to download models, the real thing is how to educate enough and make accessibility of creating such content not accessible.
>> Okay. In closing deepfake sex crimes are not ‑‑ the problem is study Internet governance and actor had a big role but now thinking everyone has a big role, studying, I think institutions need to experimented, but I think that opinions of various sectors are important to protect something and deepfake crimes may start online, but they destroy real lifes so let's have the programs together not just with punishment but with prevention of illness ‑‑ thank you.
>> KENNETH LEUNG: Thank you so much. I guess we will be concluding the session but we would love to have more of your thoughts, so if you can please comment on this platform and once again I guess to conclude and takeaway is we should must act together in order to combat sexual deepfakes and to be a good digital citizen. Thank you very much. I hope you enjoy the rest of your day. Thank you.
(Applause).
