The following are the outputs of the captioning taken during an IGF intervention. Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid, but should not be treated as an authoritative record.
***
>> SABRINA VORBAU: Wow, good morning, everyone. Nice to see you all and welcome to the workshop on protecting children from online sexual exploitation, including live streaming spaces: Technology, policy, and prevention.
My name is Sabrina Vorbau, I'm representing here the in safe network of European safer Internet centers.
I will be co-moderating this session together with my colleague Dhanaraj Thakur for the centre for democracy and technology, and we are also joined by our colleague Deborah Vassallo who is the coordinator of the safer Internet centre in Malta and she will be supporting us with the online moderation.
Welcome, everyone, here in the room and also online for joining our session and for your participation.
The session aims of exploring how technology, policy, and education can work together to tackle the evolving threat of online child sexual exploitation and abuse, also known as CSEA or CESA. We're really fortunate to be joined by people from the tech industry who will share their insight from different regions and perspective truly in the multi-stakeholder approach the IGF.
We have structured our session that will run for 90 minutes in three parts where we will tackle different angles of the subject.
Before we dive into the first round, let me briefly introduce our speakers. Here in the room with us we have next to me Robbert Hoving who is president of Off-limits. The safer Internet centre in the Netherlands and he's president of the board.
And we have Kate Ruane with us, she's also from the centre of democracy and technology.
Virtually and online joining us from various parts of the world we have Sean Litton who is President and chief Executive Officer at the Tech Coalition.
We have Dr. Sabine Witting, professor at Leiden University. Pratishtha a. And Awo Aidam Amenyah, and Lisa Robinson safety analyst policy.
So I see loads of expertise in the room, so let's begin by exploring how technology tools are currently being developed and implement to dedicate and to prevent CSEA in live streaming spaces but not exclusively, and how can they ensure that these tools align with children's rights and safety.
So we will turn our first question to our speakers. And after each round we will take a pause and invite you to share your intervention and your questions with us and if you would like to do so, please just line up here in front of the microphone.
So our first question, how can technology to prevent CSEA be integrated in trust and safe systems in a way that protects and promotes children's rights. We will start with our first speaker joining us online, and I give the floor over to Lisa that we can see on the screen.
Hi, Lisa. Welcome. Thanks for joining us.
>> LISA ROBINSON: Hi, thanks so much. Hopefully you can hear me well.
>> SABRINA VORBAU: Wonderful, yes.
>> LISA ROBINSON: Thank you so much. It's great to be here on this really rich panel for this super important conversation.
So just for a tiny little bit of context for those of you who don't know the OECD, we have a 38 country membership and we work on a multilateral -- on a consensus basis to develop research and evidence policy recommendations on lots of issues, but including digital policy and including within that digital safety and specifically work on children in the digital environment.
My first response to the question is that I think it's really important to acknowledge up front that protecting children from CSEA should not be done in a way that protects their rights, but I think it's important to recognise protecting children from CSEA is protecting their rights.
A save online space is important for children to access a large number of rights in today's reality, such as opinion, assembly, information, education health.
While it's important to recognise there are solutions and policy solutions which is the angle I come from, it's important to recognise there's a direct child rights implementation on a number of layers of CSEA.
Again, just to break down the question of what it is that we mean by technology, I guess in the question are we referring to companies themselves that provide live streaming or safety technologies in and how can this be int -- and how safety technologies could be integrated into those services to provide safeguards for children.
Or are we talking about a combination both?
For the interest of time, I'll tackle the question really from the perspective of both. And I'm going to highlight the importance of taking a safety by design approach. I know that that's not a technology in itself, but there are technologies that underline safety by design.
Here at the OECD we've done a little bit of work here to really understand what this concept means. And we last year published a report that posits eight key components for safety by design.
These roughly fall into three buckets. I'm not going to talk about one of them which is more about corporate social responsibility and environment of safety. But two of the buckets, one of which is putting in technologies and tools in the front end to prevent harm from occurring.
And then putting in tools and technologies at the back end to detect harm should it occur otherwise really does require incorporating certain technologies into service design and delivery.
So two, we also have done some research here which can shed some light on how companies are actually doing many this space and what is actually occurring in relation to the incorporation of technologies into companies to meet those sort of two aspects of what's at the front end and also as a safety net.
Firstly, we've done a series of reports looking at company practices for transparency reporting and other public-facing policies and governing documents relative to child sexual exploitation and abuse.
We look at a wide number of things, such as how CSEA is defined on a platform, how then force their policies, transparency reporting practices. But one of the things we do look at is how they detect the CSEA. We've done two reports relevant to this.
One report was published in 2023 and another one will be published this coming Monday. And in the one that's to be coming on Monday, we -- we -- it shows that companies really rarely provide detailed information on their moderation practices. And that way we don't specifically break this down in the report. I did a quick scan of the some 80 services we looked at relative to live streaming and only two who is sold businesses live streaming, only a few provide metrics that breakdown their live streaming incidences.
For policymakers to react and understand what targeted action and deployment of safeguarding technologies needed, better and more granular transparency is key. I realize I'm about to run out of time, but we've done a similar exercise relevant to age-related policies. This paper is out today.
And just to say really briefly, it's clear that age research -- this is a hot topic on a lot of people's minds, but we're not quite there in achieving age assurance. We identify this as a key component of safety by design because companies need to know who their users are to put in place child protective safeguards.
But of the 50 services we looked at here, only two systematically assure age on account creation. So there still gaps to be done in relation to age assurance. I'll leave it there.
>> SABRINA VORBAU: Thank you so much, Lisa, for kicking us off and sharing with us the incredible work the OECD is doing at global level, mentioning a lot of publications and we will also make sure to include this in our session report.
And you mentioned very interesting areas already that we will definitely come back to safety by design and age assurance, really crucial topics that are on debate right now. But what strike out for me what you said, protection means already promoting children's rights.
I think that's a very good kickoff for this first round.
We will now turn to Awo who is from Child Online Africa, also very big advocate on children's rights and we will hear a little bit more from Awo who is happening in the space particular in the Africa region.
I hope Awo, you can hear us. And the floor is yours.
>> AWO AIDAM AMENYAH: Yes, I can hear you. And --
>> SABRINA VORBAU: Morning.
>> AWO AIDAM AMENYAH: Hello, everyone. Pleasure connecting with you from Ghana.
I'm going to build on what the background Lisa gave because we don't want to bend time, we want to build on what each other is presenting.
So as much as possible, as you all are aware, Africa has a unique situation where we have an even situation of digital interest and the rapid growth of technology is helping to some extent, but is also posing a number of incidents.
And at Child Online Africa our focus is to give children what they want -- what they need in order to be online. So by so doing, we interact with these young people a lot to get things done. And one of the things that make us understand is virtually almost everywhere there's something that they encounter in the space.
So we're asking, okay, so what can be done in order to safeguard your interest and your engagement in the space?
And they made us understand that it's important that now that AI is there, the platform is made in such a way that it's sensitive to detect, prevent, and also report streaming of live situations that affect children by extension.
And the moment we mention AI, you ask them, okay, so what can you do yourself? Okay, we can try to code something. But we haven't got them there yet.
But then we are of the view that as a growing environment where young people are largely involved in the space, it would be important for the -- for us to look at protocols that allow laws to be enforced effectively.
Because one of the things they came up with is the fact that the laws are not binding and it looks like they're not working effectively, as far as our country -- our region is concerned.
So they're looking at the involvement of children in local communities to design solutions which affect them so that it will be full proof.
It wouldn't be like adults sitting to come up with designs that do not really, really impact on children. And they also recommended that there should be human oversight on AI moderation to prevent false positives, if possible.
And again, there's also the need -- they said we should put a lot of responsibility on Internet service providers, because they should be able to block some of the live streams of children. But they're not doing that enough. And all because there are no laws holding them accountable.
So they will feel that if the service providers were made a lot more responsible, the system will be good enough and conducive enough for them. But their involvement is very key in assuring that the design for the protection solution is fit for purpose and is intuitive for them to interact with.
So by extension, they felt that it's important for platforms to be faster in identifying incidents of abuse and rescue -- rescuing young people if they should fall to things, prosecution of perpetrators, and also creating -- making -- having at the back of the mind of designers that children are using the platform.
Even though they don't give money, they didn't buy anything, they are also consumers and their interests should be taken on board and taken seriously.
That's what I can say for them. Thank you.
>> SABRINA VORBAU: Thank you so much, Awo, and also for highlighting already youth participation in the whole process. So really making young people part of the development process, but also policy making, the design as well, and definitely call for action for more responsibilities for the platforms.
Thank you. For now we will now go to our next speaker, Pratishtha Arora. We talked about platforms already. Pratishtha works for Social Media Matters specifically in the India region.
I hope Pratishtha, you can hear us and the floor would be yours now. Thank you for joining us.
>> PRATISHTHA ARORA: Thank you, Sabrina, thank you for the opportunity. I'm so happy to be here giving the context of our child safety work in India and what we have been doing and how India's basically shaping the North around child protection in India.
Given that, just a brief introduction about who I am for the audience over there. So I'm Pratishtha Arora, I'm a [?] an organization that's working at the intersection of digital safety, and online harm prevention in India. Basically as an engaged [?] child protection, digital [?] framework.
And having these on setting the context. I think so now this adds value when we talk about the online initiative that we have been doing in India for over a decade now.
And this has given us a clear standing that, you know, what we need as children and their voices to be heard, right?
Right now in India what we can see, many children who are facing any form of abuse or harm, they -- they do not know the part of how to kind of report the situation, even if they know it is a form of online harm like cyberbullying.
This is one area which is talked about across cyberbullying against minors is a major issue in India. And when we're talking about technology, and when we're talking about policies, I think that [?]. Why? Because when we are giving the liberty to children to access Internet, which is a right for them in individual spaces and we're talking individual rights in India, but [?] what we are seeing is that they're failing in a form that we do not have some standard mechanism to protect children.
And mechanisms are not well defined because we do not have a global framework. We are borrowing from sections from various laws currently to protect children online. Which was not even in the speculation until we -- you know, we realized that children -- how children are [?] in the online spaces.
From [?] to grooming it has taken lives of many young children as well because of the [?] in the online spaces.
Now coming to live streaming, because live streaming is one place where [?] monetized for their own benefit. This is coming from across platforms.
The user of child content, child abusive content is something which is at its -- is at its peak right now. And some stats from the report that has been released and also the report that stated that 9 million children are [?] of India have been targeted. So this gives a very big number for us to even think about what is happening, especially when we talk about the cultural aspect of it.
How we're looking into this content and from the live streaming perspective as well, how we're able to add safety for them in the online spaces.
How do -- are we giving them adequate mechanism that they can freely access Internet or that freedom is turning that a bad shape for them.
So these are some points that I want to think about. And also think about do we need to redefine these policies?
Do we need to rethink about them?
Do we need to imagine what is happening?
Do we need to retrain markets?
Do we need have more collaborative efforts where organizations, tech companies, where regulatories can come together and create a full package where we're just thinking about [?]
At this point, at this juncture when we're talking about the [?] I think many of you have heard about the digital first and data protection act that is talking very specific about child safety in India.
There was panel on legal framework on the digital India act which was only to take forward the information technology act where India's trying to define all sorts of online [?] protecting digital users.
So these two important acts we are also hopeful that that will reshape the whole cultural context of child safety in India. Including live streaming spaces that will prevent children from CSEA and situations that children are encountering in the online spaces. I'll leave it there.
>> SABRINA VORBAU: Thank you very much, Pratishtha, for your intervention and also bringing us closer a bit to the national context and the current situation in India and also the, yeah, very shocking numbers you have shared with us million but I think -- and we will come back to this also in our second round where we will talk a bit more about cross-platform and multi-stakeholder collaboration. But reporting is, of course, crucial to make that more accessible to children and young people.
But the public in general build more awareness, but also more confidence to take the step in reporting.
I would remind everyone if you would like to make an intervention or ask a question, please line up towards the microphone and also online there's the possibility to intervene, you can just raise your hand.
We will conclude this first round by giving the floor to Sean, president from the Tech Coalition.
Sean, I hope you're with us and the floor is yours now.
>> SEAN LITTON: Yeah, thank you.
It's great to be with you all. By way of introduction, the Tech Coalition is a global association of leading tech companies. We have social media, search engines, we have many live streaming platforms, some that's their primary business and then many -- many platforms it's a component of their service. They'll have a live streaming component.
And we are 100% focused on preventing, disrupting child sexual exploitation and abuse and building our members' capacity to do this.
As we heard from the other speakers, there's alarming trends with respect to online live streaming of child sexual abuse.
It is perhaps the number one way that children are now exploited commercially in terms of -- we would call it sex trafficking. These trends underscore the need to develop and deploy technologies that prevent child sexual exploitation and abuse online on live streaming platforms while also ensuring children's safety, well-being, and privacy.
The Tech Coalition and our members are prioritizing ways to do this. Two examples I'd like to talk about today, first of all, we are working with one of our members, it's a major live streaming platform, to develop a tool to detect child sexual exploitation and abuse in the live streaming context.
It uses session metadata and third-party signals to generate a risk score for the particular broadcast. And because this tool operates without analysing actual content of the live stream, privacy standards are preserved.
But based on that score, the child safety team can take a closer look and decide whether to shut down or intervene in some way in that broadcast.
So in practice, this approach relies on participant characteristics like country of origin, the use of anonymization services, et cetera. And my combining these metadata signals, the system indicates the likelihood of child sexual exploitation and abuse activity occurring within a given live stream session for further investigation.
Now, development of this tool began in the fall of 2024. In the summer or members will test and formally evaluate its feasibility for this approach for broader industry adoption.
Our goal is to detect methods of behavioral signals and metadata rather than relying solely on content scanning and preserve privacy of the conversation.
So another example is Apple's communication safety feature for example. Apple's a member of the tech Coalition. This uses on-device machine learning to detect nudity in photos and video counts. And because analysis happens on the device, neither Apple or any other third party observes the content or is aware that nudity was shared.
When nudity is detected, the image is blurred and the child sees age-appropriate safety information and help resources.
And recently Apple announced the expansion of this feature to Face Time calls. It's available by API for free to all developers who develop apps for IOS. This enables any app to check for nudity in a conference stream for devices signed into the call.
These are just two examples of how we can combine both children's safety and to reduce abuse in live streaming environments.
Thank you very much.
>> SABRINA VORBAU: Thank you very much, Sean. I'm also again for giving us the perspective and the work that is going on in the space of tech companies. Certainly a lot.
We will come back to Sean also in the next round where we are going to also take a little bit more about multi-stakeholder collaboration and also Sean mentioned already the promotion of well-being for children and young people when using technology.
Before we get there, I see we have a few people lining up. Just please before you intervene or ask your question, quickly introduce yourself. Please go ahead.
>> Hello, good morning. [?] I'm associate professor and work on Internet Governance and I've taken part in a few research projects on online youth safety.
One of the things that when we do focus groups in middle and high schools with young people is that a lot of the time they feel that partly they can be compelled by adults and institutions to use digital tools. Of course I'm not talking about live streaming, but just having to use mobile and data mine a lot, even when sometimes they might wish to disconnect.
And when they're faced with this kind of content or with cyberbullying problems or any of the things that you have talked about, the main problem that comes up is where do I find a safe space where adults can hear my problems and act on them.
And I feel like a lot of the proposals coming from the tech company side is to say, well, we're going to develop a technology that's going to detect nudity and things like that without infringing on privacy, which is something that I would like maybe question sometimes.
But actually what happens then? If some content is flagged or if something really bad happens, what happens if the police doesn't act upon it?
So my question is you is, if your work advocating for children's rights and the online here, how much effort are you targeting towards law enforcement agencies and educational institutions so that when something happens they act upon it?
Because you can flag content as much as you like, platforms can act as much as they'd like. If at some point there's no law enforcement coming up, problems are not going to be solved.
Thank you.
>> SABRINA VORBAU: Thank you so much for your question. We haven't heard from Robbert yet, but maybe this could be a question for you to briefly answer but I know you later on will give a bit more context on the work That Off-limits does.
>> ROBBERT HOVING: Thank you, everyone. I would like to bring in the safer Internet systems that we have in Europe, because that actually is more safe space. Both for parents, caretakers, but also for people who are a victim to find help. And a lot of times that help can be that they just have someone that listens to them.
They can also take the content down. And if they want to take more action, for instance, go to the police, they can help with that as well. But they can also provide them towards the right helping parties.
So for me, it would be really crucial to strengthen such a network, because there are these spaces where people can go. And a third aspect of them is the awareness raising to ensure -- to help schools and to give schools material to ensure that it doesn't happen.
You will always have people with that intent, so you cannot stop everything. But I think you could step up much more in the prevention looking at the data that's actually coming in at the safer Internet centers.
>> SABRINA VORBAU: And I think we will hear a little bit more about this later on in the session. Maybe for this round one more question or intervention and Jutta, please.
>> JUTTA CROLL: Thank you for giving me the floor. I'm Jutta Croll from the Dynamic Coalition on children's rights. And since age assurance was mentioned as a tool to protect children from online sexual exploitation, I just wanted to intervent and turn your attention to the global standards summit on age assurance. Which is a child rights' based approach to such tools, making sure not only that children question the an age-appropriate space on the Internet, but also it's possible with age assurance instruments to make sure that adults cannot draw on these spaces that are made for children.
And that global assurance standard summit came to the conclusion that this can be done data minimizing and privacy preserving without being intrusive and gathering data from children as well as from adults.
And this is all based on the ISO standard 27566. So if anybody is interested how that can be done, you will find the communique. We have some -- brought some copies and just come to the Dynamic Coalitions booth in the IGF and we can talk about that.
Thank you.
>> SABRINA VORBAU: Thank you very much, Jutta. Also for mentioning already the great work you and colleagues are doing on children's rights.
I think we will move to the next round, but please stay with us, we will bring you in for intervention later.
I will hand over now to Dhanaraj to guide us through a second round more looking into cross-platform and multi-stakeholder collapses.
>> DHANARAJ THAKUR: Thank you so much, I'm research director at the of democracy centre of technology based in Washington, D.C.
So one aspect of the problem that was already raised is this issue of cross-platform prevalence of content related to child sexual exploitation and abuse.
This is a significant problem, I'm part of the reality of how this kind of exploitation on abuse occurs.
So the question I want to raise to start with our experts and then we can move into Q&A at the end as well is how can we promote -- better promote cross-platform efforts, including live stream platforms to prevent the spread of this kind of child sexual exploitation and abuse content.
And to do that, I'm going to start with you, Robbert, to hear your thoughts first on this issue. On this cross-platform problem.
>> ROBBERT HOVING: Thank you very much. As we're from the safer Internet centre here in the Netherlands Off-Limits, we have the hotline for the child abuse to be reported. We have the help line for the other transgressive behavior and where caretakers can call for help. And we have Stop It Now which is for people actually watching this material.
When I bring that perspective in, when I start for instance, with the perpetrator, we did research and we discovered that the people calling Stop It Now more than half are males under 26 and they watch this type of behavior. We know how easy it is to get access to an adult website, we know that looking at heavy material desensitized looking at this.
Young children making sexualized images themselves, might be alone, might be with other children, attributes, but they're making them themselves through webcams, phones, et cetera. And 75% is prepubescent. And then we look at platforms and there are different types of platforms, there might be live streaming platforms or also social media, there are a lot of good aspects about social media. And some social media are really veered towards sharing content, but there's also social media that is designed to meet people and to connect with people.
Now when we go back to that 75% prepubescence, we know it might be grooming, but risk attention. It might be because of previous abuse or stuff they've seen online at a young age and enacting that.
When you have these three ingredients, it's been managed age verification. I think it's very important to insert in online cases age verification to ensure that old people who want to pose as younger people and younger people who want to pose as older people cannot connect.
And to chime in what Jana said session about deep fakes, cross-platform efforts, connecting with education, I think we could also really have companies stepping up to collaborate more with schools to give material instead of just pushing their tools towards schools, but also help build up the curriculum.
Because we always say education, but when you put everything at schools, then there should also be the capacity at schools to be able to do that.
And I think combining it with the tech sector, for instance, like Jana's mentioned, could be a very good idea.
That would be my reaction to your question.
>> DHANARAJ THAKUR: Great. Thank you for those points, particularly as you mentioned around prepubescent self-generated content.
Sean, I want to turn to you now, because given your particular perspective with the coalition and therefore the opportunity to work across actual platforms and engage across platforms, I'm curious to hear your thoughts on this issue of solution -- cross-platform solutions.
>> SEAN LITTON: Thank you for the question.
So as the other speakers have noted, bad actors typically exploit multiple services across the tech ecosystem in their attempts to groom children and distribute CSAM or other things. So for example the bad actor might contact a child on gaming platform, move them to a private messaging platform, and perhaps use a live streaming platform down the road.
So the abuse stands on social media, gaming, live streaming, payment apps and more. But the gaming company is obviously unaware of what happened on all the other platforms. They don't have all the information and it makes them difficult without the complete picture to adequately grasp what's going on and take action.
So that's why industry collaboration is essential at the Tech Coalition. We recently launched a programme Lantern which is the first cross-platform signal sharing programme that helps companies strengthen enforcement of their child safety policies.
We launched Lantern so companies so securely share signals with one another about accounts and activity that violate their own child sexual exploitation abuse policies. Until Lantern, there was no consistent way for companies to share in information in a secure and privacy-preserving way. Lantern fills that gap by a fuller picture of the harm.
Working with Lantern companies can increase their detection capabilities, build awareness of threats, bad actors, and hotlines and other authorities.
We know this approach works and last year members shared hundred of thousands of signals through Lantern. This led to account actions, content removal, and the disruption of offender networks and CSAM circulation.
Signals helped flag contact and trafficking cases as well that may not have been identified otherwise. It's really important.
And crucially these outcomes come in additional to the original action taken by the company that first detected the abuse, showing how Lantern enables a ripple effect of protection across the ecosystem.
So together initiatives like Lantern are helping close detection gaps enabling faster action and providing -- and proving that collaboration really does make a difference. It's not just possible, it's powerful.
So thank you.
>> DHANARAJ THAKUR: Thank you, Sean. Very interesting to hear about project Lantern and I'm sure that may come up again in the Q&A.
Very good. So also next I want to turn to one of our online speaks, Sabine Witting, Leiden University. Sabine, hopefully you can hear us as well since you've joined us now and speaking for the first time from you want to say more about yourself, please good ahead.
>> SABINE WITTING: Thank you so much. I hope you can hear me all right. Yeah, my name's Sabine Witting, I'm an assistant professor for digital technologies but I'm with a consultancy firm specializing in tech and dealing with age assurance and we do a lot of work on these topics across the world.
Thanks so much for the question around cross-platform collaboration. And I think this kind of collaboration is essential because platforms all deal with the same human rights and children's rights issues, especially issues competing for rights. And this is trying to square the circle and this assistance can assist with that.
I think cross-collaboration is not enough for Tech Coalition to really collaborate. I think a multi-stakeholder effort is always crucial to all people affected by technology at the table. Human rights advocates, child rights advocates, academia, but also parents and children themselves.
And I also want to use an example of one of the technologies that's often put forward as a solution to child sexual abuse in the digital space, but for live streaming which is age assurance which has come up a few times.
I think when we approach age assurance from a multi-stakeholder collaboration, one of the key gaps that's always criticized about the age assurance is the lack of clear evaluation criteria and how we can assess, for example, the effectiveness and robustness of these technologies and also to really understand better who is adversely impacted by these technologies.
And at the moment, there are some standards, some industry standards which have played a very important role for quite a long time. But a problem is that a lot of industry standards are not accessible and drafted in a multi-stakeholder way. They're drafted by industry and age assurance providers themselves.
And that of course begs the question of might these standards be biased to a certain extent. Have human rights concerns from across the world been take mean it consideration. Not only talk about the Global North, but especially the Global South and to representatives of the Global South are usually underrepresented in these industry standard making processes.
Which is a huge problem because of the important role that industry standards are playing here.
And maybe I want to point at a good practice example that I'm lucky enough to be part of, which is the current drafting of the IEEE standard on the prevention of CSEA generative AI. And this group is really a combination of all the actors that we need around the table. We have human rights advocates, industry, tech experts, academia, and it's really enriching to see how an industry standard can be developed with all of these different stakeholders at the table and also we have a strong representation from the Global South.
And of course also generative AI, we've heard it from Pratishtha and Awo is a different story in India and African countries than the Global North. So it's really important to have only a cross-platform collaboration, but multi-stakeholder and regional representation especially of vulnerable groups.
Thank you.
>> DHANARAJ THAKUR: Thank you so much, Sabine. Thank you for raising this point about the relevance of standards settings particularly around age assurance tech and the utility of having multi-stakeholder approaches with that.
So I now want to turn to Kate Ruane, my colleague. We heard about industry around cross-platform but we heard from Robbert and Sabine the importance of multi-stakeholderism in these effort. I'm curious on your thoughts on how we can better address the cross-platform proliferation of these kinds of content.
>> KATE RUANE: Thanks to everybody who's spoken to far. I echo what's been said. I want to pick up on the point that cross-platform efforts need multi-stakeholder engagement in order to work best in part because cross platform efforts are going to create more significant risks for human rights like free expression and privacy.
And you know, that is actually a necessary thing. Child sexual abuse and exploitation is such a large and clear are harm, it is a crime around the world for a reason.
And that means efforts to restrict it are necessarily proportionate to the harm. So therefore when mistakes get made, we can see really significant impacts on the lives of innocent people. And in order to ensure that our responses to the crime of CSEA is proportionate, multi-stakeholder engagement can ensure that that harms do not propound beyond the criminal activity.
And multi-stakeholder engagement can be really helpful for ensuring that there are things like transparency and appeals processes.
The Tech Coalition has done impact assessments to combat CSEA that's really positive and it would be helpful to have more spaces in which children themselves, survivors Civil Society, technologists and privacy experts can engage in the ways that platforms are developing their information sharing efforts to ensure that human rights are respected throughout their development and their execution.
And I specifically want to call out a couple of things. So the Tech Coalition has talked about its development of a tool to detect signals across platforms for a child sexual abuse and exploitation. This is a really interesting and valuable tool, I think, and it would be helpful, particularly to have transparency into the tools themselves.
Not just for the -- not just for signals development, but also for things like content detection, especially in live stream where it's particularly difficult to execute.
Most content detection tools are designed to identify content at rest, whereas, live streaming content is constantly in motion.
So the various tools that exist to try to identify a child sexual abuse and exploitation within live streaming are currently significantly lacking in benchmarks. Yet, we have a number of organizations that are marketing technologies that claim to be able to identify child sexual exploitation and abuse content in live stream organize in video content. But we don't really have a good way to identify whether these products actually work or work sufficiently or whether there are sufficient safeguards in order to address potential errors in the content detection tools.
Multi-stakeholder engagement and transparency can help with both of those things. It can help us identify and improve these types of tools. And it can help us -- it can help us sort of understand how they work going forward. And help us deploy better transparency and accountability tools for tech companies themselves and for governments who are engaged in proper enforcement of their laws going forward.
So those are -- those are a couple things that I thought about. And I also wanted to turn back to Robbert's point about media literacy and ensuring that we're investing in people's understanding regarding how to engage in combating CSEA at the local and person-to-person and user level.
I think we definitely need -- we need significantly more engagement in that front. And we also need tech companies both in our cross-platform in a transparency way to talk to each other about how their reporting processes work.
One of the things that we see is that your enforcement efforts are only as good as your reporting efforts are. If it's difficult to find the button, the simple design feature of how to report harm and ensure that it's moving forward, that's another simple thing that platforms can share information about and can share transparency reporting about to make sure that we're consistently getting better in how we combat these harms while ensuring that we protect free expression and privacy in the process.
>> DHANARAJ THAKUR: Thank you, Kate. Thank you for raising those points or multi-stakeholder approaches that, as you put it, can be directly linked to improving transparency and accountability and help platforms address this content as well.
Okay. Very good.
So we're going to continue the pattern -- the format that Sabrina laid out that we can have a break for questions and interventions from the audience and, again, I'd ask for the same thing where people can line up at the mic.
So please line up and we'll take as many as we can. Yes, sir, please start and you can just say introduction as well before you share your intervention.
>> ANDREW: Surely. Good morning, everyone. My name's Andrew, I run a tech consultancy, and I'm also a trustee of the Internet watch Foundation.
Since we're talking about CSEA, no one's actually given a number. I think we need a number. It's estimated there's roughly 300 million victims of child sexual abuse and exploitation every year globally. That's about 14% of the world's children each year. Just to put some scale on this. This is a nontrivial problem.
To add to the list that Lisa started with back at the start, so couple of people have mentioned age estimation and verification. I just wanted to reiterate that, because a few sessions earlier in the week people have asserted that's simply a mechanism to let social media platforms get even more data about their users.
But as rightly has been said, there are preserving mechanisms do age verification and estimation, which are important tools to keep children from adult content, but importantly also adults off of child sites and from accessing child accounts on sites.
So we need to make better use of that. What hasn't really been said is that research shows us that encrypted messaging platforms are widely used to share child sexual abuse material once it's been captured, including video. And privacy is used as an excuse to not stop that, weaponize privacy.
I'd love to hear the panel's comment, they're well known privacy preserving techniques to block known CSEA on these platforms. Love to hear your views why they're not being used.
And then finally, just want to also mention since tech standards were covered I think by Sabrina, they are changing to the extent that a lot of existing parental controls and content filtering will stop working because metadata is increasingly being encrypted, that's a major problem not least of which because a lot of the policy community don't take part in the -- in the way that tech standards are defined.
So we need to find out a better way for getting multi-stakeholder engagement, otherwise we see the problem getting bigger, not smaller in the community.
And then finally, Kate talked about those tools not being known whether they work or not. The IWF has data we can test tools, so maybe let's get together perhaps with Sean's members afterwards and we can do some validation whether they actually are effective or not.
Thank you.
>> DHANARAJ THAKUR: Great. Thank you for that.
So maybe we can have some quick reactions to this. The first, okay, so first was the point about encryption. I don't know, Kate, if you had any quick thoughts about that. And then the point about tech standards as well, maybe Lisa or Sabine, you might want to jump in there.
Do you want to start with --
>> KATE RUANE: Sure, I would love to start with the end-to-end encryption question.
I think that oftentimes privacy and safety are placed intention with each other. I find that framing to be a little bit -- a little bit difficult because I think privacy and safety are very much in line with one another.
Folks often point to the distribution of child sexual abuse material through end-to-end encrypted platforms as a reason to create an encryption backdoor or a justification for no longer using or relying on encrypted technologies.
I think that that can obscure many of the benefits of encrypted technologies. Encrypted technologies are particularly salient for human rights defenders, for journalist, and for people who just want to keep their data private to both themselves and to the intended recipients and keep it outside of the view not just of governments and other bad actors, but also tech companies themselves.
As tech companies continue to hoover up so much more data about all of us that encrypted services are one of the few and potentially actually only place where they cannot see the content of the communication. And that becomes more and more important as we see the world changing in front of our eyes.
But I want to point to a particular research that has been done at Stanford where they looked at the effectiveness of the content oblivious versus content aware method of content moderation.
And what they found is for the most part, for the vast majority of harmful content, content oblivious methods of detecting these types of abuse are the -- are far more effective than content scanning or content aware methods of detecting content.
And so you know, when you put that in front of, you know, the fact that CSAM is a very particular type of harm this is specifically best protected by things like via content matching, putting it up against the value, putting the -- putting the ability to detect it on every single surface across the entire Internet against the value of having a safe and effective place to engage in communications for national security purposes, for the purposes of journalism, for the purposes of ensuring privacy from tech companies and from government that would otherwise harm people for so many other reasons, I think that there are many other ways to detect CSAM on user forums including [?] which has been an effective way to address that content.
We should have continued service and think about privacy and safety as things that complement one another add not necessarily at odds.
>> DHANARAJ THAKUR: Thank you.
Okay, purposes of time, and in the interest of time, I'm going to ask next two people in the line to maybe just very briefly offer your intervention or question and then I'll bring it back to the panel and then we'll move to the next round.
Please go ahead. And also just a reminder to introduce yourself.
>> Sure. [?] speaking from the experience of knowing someone, knowing a victim of financial sex extortion who ultimately took his life and what we've had go through in the follow-up to that, investigating reporting, coming from a developed nation I see a big gap between some of the things I'm hearing about on stages here and the reality that exist in the developing world.
Definitely reporting needs to be improved. Investigative methods need to be improved. If there's the ability for victims, the direct report to platforms, that would be good. Because we have material that we can put forward to the platforms and we need some outlet to give it to the authorities to -- competent authorities to teal with the matter.
I think the first speaker, first contributor, he talked about the law enforcement ability to do things. And I think that is something that needs to be developed further.
Assistance with respect to development -- developing capacity and law enforcement in developing states as well as general awareness within the general population, some of they line harms that exist out there, that needs developing -- in the developing world.
Sean brought up the issue of financial extortion. He also brought up the issue of there's an exit point [?] direct question to Sean would be, Lantern's project also contemplate signals from the financial transactions aspect some are you match organize searching within financial transactions to either fine perpetrator [?] per operators of [?] from victims.
Thank you.
>> DHANARAJ THAKUR: Thank you. And thank you also for sharing your experiences.
Next, yes please, go ahead.
>> Hi, I'm [?] I'm from Brazil and I come from a country with almost 200 million Internet users. One-third, 1 in 3, are children and teenagers below 18 years old.
And also live streaming is also become prevalent in Brazil. And we are seeing a growing number of reports not only in my country, but everywhere. If you look to the U.S., you can see the numbers. If you look globally, you can see -- you have the numbers.
So the numbers of reports, the number of cases is growing everywhere in the world.
On the other hand, the resources has declined. And my question is how to solve this dilemma, because we need resources to create technology. We need resources to create -- to develop [?] and prevention programs.
But when you look for the government resources, they are declining. Private resources industry resources, they are also -- if it's not declining, it's stable or very low level. And the problem is growing everywhere.
How to solve this dilemma. Thank you.
>> DHANARAJ THAKUR: Thank you. We'll have quick reactions from our speakers before moving to the next round.
So Sean, there's a question directed to you. Maybe you have some thoughts you can briefly share with us.
>> SEAN LITTON: Yeah, thank you. Yeah, there is a financial component to a lot of crimes against children online. With respect to Lantern specifically, we are piloting with two major global payment providers sharing signals on Lantern to determine the effectiveness of those signals.
They are only ingesting at this point signals from the social media, gaming, et cetera companies.
And we'll have a report out later this summer on whether it's effective. If it's effective, we'll scale it up and bring other companies on to the platform. You're right and I'm very sorry for what happened to your friend, and there have been a number of cases of suicides related to financial sex extortion. It's a difficult issue and law enforcement is a big challenge there because the perpetrators of abuse tend to thrive in countries where there's lower law enforcement capacity.
And then the victims may be in a different country. And so even if the report gets to that country where the origin of the crime or the abuse, law enforcement may not necessarily have the sufficient capacity to act on the report.
And so this leaves everyone in a bind. But anyway, we are piloting with financial companies and we hope to share those results later this summer.
Thank you.
>> DHANARAJ THAKUR: Thank you. Okay.
So we do have another round of questions, so I want to actually -- I know we've the good a lot of input and feedback from the audience persons, but I do have to turn it back to Sabrina so we can have another round. And maybe we can save some time for Q&A later on.
>> SABRINA VORBAU: Definitely. Stay in line, we will bring you in towards the end of the session. But in our final round we want to turn the attention to policy. It was brought up the need for more standard mechanisms and also the more stronger role of law enforcement as seasoned threats evolve. Also including the rise of self-generated content.
We hear global numbers and regional numbers and we will now look more into how should national and international policy frameworks respond.
So for the question to the speakers is how can national and international policy be strengthened to address emerging forms of abuse such as self-generated content and Lisa's already on screen, so we'll start this final round with you.
>> LISA ROBINSON: Thank you so much, Sabrina. And I -- I do, of course, have some thoughts in relation to the question. I just wanted to mention quickly two things relevant to the discussion we just had.
Firstly, not my part of the OECD, but there has been some research done by a part of the OECD action task force which has done some specific research in relation to disrupting financial flows relevant to live stream, sexual abuse, and sexual extortion. So I'll share that report with Sabrina so it can be shared with the group as well.
And the paper that I just mentioned in relation to CSEA, we developed an intensive list of services looking -- that facilitate CSEA and that really reflected what was said about cross-platforming and off-platforming about larger services to smaller services and need for scrutiny on those smaller services as well.
Which I think takes me into the policy question, so I wanted to mention those issues.
I think when we talk about national and international policies and how to strengthen them, certainly the position from the OECD and what we advocate is to take a tech neutral approach, a multilayered approach which can range from awareness rating with children, digital literacy through the industry action and then industry regulation and law enforcement.
That approaches a multi-stakeholder and we engage in good international collaboration so we don't end up with a fragmented space in this global area.
But I think what might be useful in relation to this conversation is to have a think about what are some sort of specific policy actions that maybe aren't getting scrutiny the same way that a broader, overarching policy actions are. A couple that have been touched on and are getting attention are transparency reporting, transparency reporting of practices and platforms in a number of areas. We've talked about transparency not just of what's happening on platform, but actually how companies are using tools.
And we've talked a lot about age assurance. I think there's two other areas that would be interesting to focus on from a targeted policy response when we're looking at -- at safety of children on -- from CSEA in a number of areas.
So not just in self-generated content online coercion, but in all the different manifestations that we have relative to CSEA. I'm really pleased that the two that I was concerned about and wanted to raise have already been an important part of the discussion today.
So the first is recidivism on platforms. I think it is been noted already that law enforcement is overwhelmed and I, again, as with my colleague on the panel express my sincere apologies and empathy with the gentleman who mentioned the terrible tragedy of his friend.
But recidivism on platforms, I'm not necessarily talking about a law enforcement recidivism, but where bad actors are ban from an account, but then are able to recreate an account without consequences or without scrutiny.
Now Sean has already mentioned the Lantern project and I obviously will let him speak to that. But we do know that there is problems with recidivism. Research from Australia's eSafety commissioner under its transparency programme shows that companies have very little safeguards in the way of recidivism and practices is limited across services.
We looked at material and found users and bad actor users were shut down an account and open other ones with very little oversight as to how that's managed across platforms and within platforms to stop new accounts being created. That is one area that could be focused on.
The second is complaint mechanisms. I'm really happy that that's been mentioned a lot today as well. And just to focus on children themselves in the capacity for children to make complaints, and I think Kate mentioned this and also Awo mentioned this today about really listening to kids and understanding complaints from a kids' perspective.
Again and again, which when we talk to kids about what they want, they want better mechanisms and understand what it means actually to file a complaint, what filing a report means, what are the consequences. And they want responses back to them to understand what happened with their complaint.
So I would -- I'm happy to mention on other things and also to mention the work that the OECD has done on transparency reporting and age assurance and more broad legislation and international cooperation. But I would pause at those two important areas where policy action could be focused.
>> SABRINA VORBAU: Thank you so much, Lisa. And you mentioned the multilayered approach which I think comes out very strongly of the discussion we're having today. And then also really crucial the strong collaboration that is needed with law enforcement. And I want to bring in Robbert here also based on the work that Off-Limits is doing and specifically the hotlines.
>> ROBBERT HOVING: When I look at policies, for instance, I think a lot is there. In the Netherlands a lot is there. With the DSA coming in into effect, a lot is there. We have policies how to deal with them. Sometimes I think authorities could maybe go a bit quicker when the party's not working, for instance, telegram, we had troubles with them in the Netherlands. You can also decide take them out the app store because we know there's a lot of CSAM, weapons being sold, et cetera.
And I think that that is a very good solution to directly go after these companies instead, for instance, looking at privacy rights, so to say.
Going to the buzzwords of IGF, this is my first IGF so maybe there are more buzzwords that I didn't hear. I heard multi-stakeholder approach, public-private cooperation. But I think that's what we need more. Also how we're sitting in this room what we're doing at IGF, it's like gold. You need to brush it and it starts to shine.
Because a dominate theme with online abuse is content. That to me means you should have an integral approach of how you deal with content online. Because another theme which will be dominate in new forms of abuse, it will start as harmful, but be lawful. It can still have tremendous effects of people offline, people of LGBTQ+ community, it can be very racist for instance with memes. But it is very sure that the dominant theme it's content and a lot of times it's lawful, it's allowed to post these things while we know that the people who are a victim need help.
I think by working that way in the public-private stakeholders working together with the public-private collaboration, you can pick up signals together. You can do triage. And from the signals you can say maybe we see a trend online that as society we don't accept anymore. And then you change the signals, out of those signals you decide we change the law.
It's something we did in the Netherlands with doxing. Spreading data has been criminalize the since the 1st of January 2024. Picking up the signals and working integral with the content online, that’s what we should do more.
I think we have the perfect example in the Netherlands, but I think how we did that in the Netherlands, the multi-stakeholder, public-private working together and actually building something and looking at, like, how should we approach this content and be curious, be curious as a company, be curious as education, also be curious as Civil Society and as, for instance, as ministries that that is much more the way forward. Because a lot of policies, legislation, in my opinion, is already there.
>> SABRINA VORBAU: Thank you. Thank you for sharing also that best practice examples. And if colleagues are interested, we are also here in the IGF village with the better Internet for kids and the safe booth and there's a lot more other best practices of such examples of the multi-stakeholder collaboration and connecting the different dots at national level if you are interested later on and to continue the conversation with us.
Before we will conclude this roundtable, I'd like to bring in two more colleagues and also turn now to Kate for intervening with her points.
>> KATE RUANE: Yes. Very quickly because I'd like to get to questions as well.
So I think on policy I can think of two specific things that I would like to see happen.
First, I'd like to see companies -- and I mentioned this already, but I'd like to see more transparency from companies regarding how they are identifying and removing child sexual exploitation and abuse.
The first is we don't actually know the prevalence of this type of abuse on many platforms because we do not have enough data from the platforms regarding how much they encounter.
Second, it would be helpful to have transparency into the effectiveness of the tools that they are using. Right now there are, generally speaking, three categories of tools being used to prevent, identify, and remove child sexual abuse material. And they are just kind of basically designed based features.
So for example, in order to live stream you need a certain amount of followers or an account that's existed for a certain amount of time. It would be good to know the degree to which those types of tools are reducing the number of live streams or reducing the amount of problematic content shared.
It would also be good to know the data on which content detection tools are trained. At this time, you know, content detection tools like photo DNA, we know those are trained on known CSAM, but they're specifically designed to identify particular images at rest.
If we are talking about content and live streaming, content that's moving, content that's live, it would be helpful to understand, A, the datasets that are being used to train these types of tools.
B, how the data is being sourced whether it is being done ethically.
And then C, whether there's been consent to the use of the training data, especially if it is being trained on existing child sexual abuse material.
Currently we are not aware or at least we're not sufficiently aware of how these tools are being created and trained and yet they are being marketed as tools to detect content in live streaming in and last thing we immediate know is how well are signals detection tools, how well are they working. Tools that as the Tech Coalition talked about, tools like okay we're looking at where are your IP addresses coming from. We're looking at content associated with the live stream to trying to figure out the likelihood that child sexual abuse is happening within specific content or within a live stream.
Data regarding how successful those are and how -- and how mitigation -- and what mitigation efforts look like when content is misidentified as being [?] I think would be helpful to engage from a policy perspective in and the last thing I wanted to talk about is law enforcement.
In the United States, one the biggest problems we have is underresourced law enforcement. So we're doing actually probably a relatively decent job of identifying CSAM particularly when it is at rest and reporting it in NICMIC. But what we don't know or what is underresourced is the ability to address that identified harm by law enforcement.
And another tool that is going to be necessary going forward and is going to get more necessary is the ability to separate out synthetic CSAM, AI-generated CSAM from real live CSAM that's been created using an actual child. Because that's going to be essential to helping law enforcement identify children that are in harm so that they can engage in enforcement efforts in a more efficient way.
>> SABRINA VORBAU: Thank you. Thank you for your points and also for concluding a bit on the transparency aspect.
And yeah, holding companies more accountable, as Robbert mentioned already in the EU, we have the Digital Services Act that came into force recently with a specific article on the protection of minors. And here this is a good example where for example our colleagues from the Safer Internet Centers are very active.
Also kind of transmitting this transparency in the education to the children, the young people, but also the parents and to schools to, yeah, make policy making in general also accessible.
One more intervention from our speakers for this final round and we have Sabine on the screen. Last points from your side. The floor is yours.
>> SABINE WITTING: Thanks so much. I think it's a question maybe from the legal angle. I think there are various areas of laws that still require strengthening, both on the international level, but also national level. And I think there is a bit of a misconception that child sexual exploitation is equally criminalized across the world and that's not the case.
More work need to be done many ensuring stronger policies. We still have a lot of countries that do not criminalize the mere accessing the child sexual abuse because the main law principle to the CRC was always on concession because that was the prevalent issue in the '90s when these conventions were drafted.
There's still quite a big of a push where upcoming issues such as live streaming need to be addressed in national criminal law.
The same issue is with self-generated content. That content is often considered a homogenous group of content that you need to criminalize. But that's a much more complicate the issue from a children's perspective. Because it's content that's produce and voluntarily consensually of adolescents above the age of consent of consensuality. And these need to be done in the care in the content that's produced.
I want to go back to the point which was mentioned a few times which was a question around law enforcement.
I would like to see the same effort from governments that they at the moment put into platform relation. They should put into law enforcement and strengthening the criminal justice framework. Because there is a bit of a -- I feel like an overfocus at the moment on the responsibility of platforms. While we know that if these cases then really reach the court system, most of them either, A, fall through the cracks or, B, the children that are forced to go through court system leave extremely traumatized.
And this is for various reasons. First of all, because we still have a lack of protective measures for children in the Criminal Justice System. For example, protection from cross-examination and there is insufficient court preparation for children. And the presiding magistrate and the also the prosecutor interview and examine children within the crediting the justice system are not done in a proper way and age appropriate.
I would like to see an equal effort to strengthen that system.
The same applies for example and especially for victims of technology facilitated child sexual abuse and expectation, a lot of measures used in a process for example the use of CCTV cameras can be quite traumatizing for a child that's a victim. Because you put the child again in front of the camera in front of the Criminal Justice System, even though the camera played a crucial role in the abuse and exploitation. Much more consideration needs to be paid around the use of toiling in the criminal justice sector and how that might impact children that have experienced technology facilitated abuse and exploitation.
My last thing I want to talk about is the context of live streaming of child abuse and exploitation, where it done in a commercial way. The financial sector can play a very crucial way in flagging suspicious payments.
I think mandatory reporting is really essential to make sure that financial and institutions report suspicious transactions that might be linked to live stream of child sexual abuse and exploitation, and one of ways do this is consider these kind of predicated offenses under anti money laundering laws and that would help institutions to file suspicious action reports.
As we know when it comes to organized crime, that includes a lot of cases of live stream or CSEA, follow the money is one of the most important leads for law enforcement, one of the most important starting points.
So I think also when we talk about industry responsibility, let's not leave out the financial sector, especially in the context of sexual exploitation and live stream. Thank you.
>> SABRINA VORBAU: Thank you for closing us up on a lot of points and we should not forget the well-being of the child which really needs to be and should be the centre of the action.
Thank you so much.
For the last couple of minutes, we would like again to give the audience here in the room and online possibilities for interventions and questions. And please take the microphone and briefly introduce yourself.
>> So my name is Jamison Cruz. I live in one of the biggest Brazilian cities elected in the middle of the Amazon Forest. And I have proudly represented the youth delegation of the Brazilian [?] Governor's training since 2022.
And during the Internet forum in Brazil, the largest IGF event in the world, I have proposed a workshop focused on the protection of children and adolescent in the online environment.
While addressing issues such as sexual exploitation is crucial, we must also emphasize the growing need to educate and communicate people about the danger of overexposure online.
This is a challenge that has intensified in recent years. Violence, discrimination, and sexualizing of minors have expanded beyond the physical and now [?] space.
The urgency to act is clear. We need platforms revelations, public policies, education, and digital literacy to ensure that the Internet is a safe space for all, especially for our youngest users.
Thank you.
>> SABRINA VORBAU: Thank you. Thank you so much. Wonderful to have you here and many other youth from all over the world. Especially the Global South to participate in the IGF.
And I think programs like the youth IGF are crucial and important. I think as you rightly said, there is the need to put also more youth voices in the different sessions within the IGF and definitely also put more spotlight on children's rights.
Thank you.
We'll turn to our next speaker here in the room.
>> Hi, I'm [?] and I work with the UK [?] centre for about six years.
Working on digital literacy and also policy. I think digital literacy is an incredible powerful tool and how we can frame the initiatives to be sensitive to the fact if we want to teach young people to be safe online around CSAM, it's important so they can understand it.
By wanted to quickly flag the idea that beyond the EU several countries are either showing apprehension to regulating AI or passing legislation that prevents its regulation at the moment.
And I just wondered with the rising role of AI in facilitating technology facilitated like gender-based violence and also more specifically CSAM, how we can navigate that. Because if we're outlawing regulation of AI and CSAM, that's outlawing that.
>> SABRINA VORBAU: I don't know if any of our speakers in the room would like to respond to the few points on the AI comment? Kate or Robbert?
>> KATE RUANE: Sure. I'm not -- I'm not aware of every single policy effort around the world right now, but the U.S. is currently considering a policy which would prevent U.S. states from regulating AI to some extent.
We -- our organization has identified this problematic for a must be of reasons. But it would prevent harms such as the generation of nonconsensual images or synthetic CSAM is one problem that might exist.
One thing to think about is whether existing laws actually already without naming AI specifically already cover that particular type of abuse. Hopefully there are around the world laws that currently exist that while they may not say AI, nonetheless, encompass the content that would be AI generated, child sexual abuse material.
Again, I don't know enough, but from our perspective, preventing the regulation of AI whole cloth without considering the human rights impacts of that type of an action is deeply problematic.
>> SABRINA VORBAU: Thank you. We have two last interventions and questions in the room and then we will close with Dhanaraj and some takeaways from this session. Maybe we can take both questions and then respond to it.
Please go ahead.
>> My name is Raoul Plommer I'm with the electronic frontier. My first IGF was in [?] I've been to quite a few of these. And this is the first time I actually have to take a little space to commend something [This portion has been removed from the record for violating the IGF Code of Conduct. Particularly, the stipulation to "Focus discussion or remarks on issues rather than on particular actors, whether they be individuals, groups, organizations, and refrain from personal or ad hominem attacks"].
This is a very polarized issue as it is. It's very tough and complicated. I totally appreciate saying that privacy and children's rights can be completely aligned. And -- but this kind of polarization [This portion has been removed from the record for violating the IGF Code of Conduct. Particularly, the stipulation to "Focus discussion or remarks on issues rather than on particular actors, whether they be individuals, groups, organizations, and refrain from personal or ad hominem attacks"]. Thanks.
>> SABRINA VORBAU: Thank you. We will give the opportunity for follow-up after the session.
Last speaker, please.
>> Yeah, I'm from the Finnish Green Party. My question is mostly relating to the issue that we have a lot of this -- I mean, even this week we've heard a lot from these different actors, international actors, organizations about the effort to take down this type of content.
But quite often the conversation ends up at washing their hands at, yeah, we took down this content, we don't have jurisdiction to go further. The investigation and other things need to be left up to law enforcement, which severely differs between different countries as well as legislation. And they don't have personnel and skills or understanding of current technologies to actually to anything about it.
And thus, we get those repeat offenders and recidivism on those platforms as well as larger crime syndicates which operate through this or earn some part of their income through perpetuating this material or disseminating it.
And it just seems still very confusing that this is such a long-lived issue but there is no, like, proper official body to unite these platforms and law enforcement.
I hope to see some response on this topic.
>> SABRINA VORBAU: Yes, thank you so much. I think we are all continue -- will continue trying our best to collaborate in the way that we have been doing here today. I think it's just taking a sort of glimpse of the conversation, opening up the conversation and hopefully continuing the conversation in mutual respect of everyone's opinion and the work we're doing.
We are up for time, but I would like to give a final minute to Dhanaraj to kind of take us a little bit through some of the takeaways from our session.
>> DHANARAJ THAKUR: Yes, thank you, Sabrina.
We are at time, so I'll be very brief just to say that we are having this very important conversation. But I think it's obvious that we recognise this is a serious problem that has significant impacts on children and their families and communities. So it's important to talk -- to highlight as the speaker said, as well as the audience members, highlight the important paths forward particularly in terms of different kinds of recommendations which included a point that was made at the very start, centering children in the design of solutions, in the -- in addressing the problem, starting from the design technologies right through to, like, the Criminal Justice System. Centering their views as well as their well-being.
Many participants speakers mentioned a multi-stakeholder, the relevance importance of multi-stakeholder approaches, but even more specifically talked about how that relate not just to say cross-platform approaches, public-private approaches, but also in standard setting and even coordination -- improving coordination with law enforcement, for example.
And we also talked -- we've talked about technical solutions, and where these technical solutions -- where we developing public policy require more information transparency around these. So transparency is a thing that came up several times as well both in the efficacy of technologies, but also in trying to decide, for example, emerging problems around synthetic CSAM and actual CSAM and so on.
I think there is a lot that was discussed and noting the time, we didn't get to every point. We will be sharing a report of this as well.
What I'd like to end on is just to thank all of our speakers. We really appreciate everyone who was able to join from many different places here on stage, Robbert and Kate, but also online with Pratishtha, Awo, Sabine, and Sean as well.
And thanks of course to Sabrina and Deborah, our co-organizers on this.
Thank you, everyone.
(Applause)
