IGF 2022 Day 2 WS #261 Perils and opportunities of data integration for security

The following are the outputs of the captioning taken during an IGF intervention. Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid, but should not be treated as an authoritative record.

***

 

>> MODERATOR:  I can start whenever.  Thank you so much.  If you can help me with the slides, that would be great.  That's it.  Okay.  Hello, everyone.  Thank you so much for attending today's session.  We're very excited to share today's space with a very diverse group of individuals who are both online and here with us physically, so I'm very excited about this.  But as we're very excited about sharing with an international audience, like the one IGF always offers.  For those who don't know me, my name is Eduardo, part of digital rights organization based in the south, particularly Paraguay.

So, we have developed this panel, so you can go to the next slide, I would really appreciate that.  So, we have developed this panel in partnership with the Privacy Brazil and Privacy International, both organizations we've been working with for quite some time and you'll hear from them very soon because we have speakers from both organizations and also the presence of Marina helping us with the online moderation.

Now, going back to the question and the panel, perils and opportunities of data integration for security, for the past 10 years, we have been working on the intersection of human rights and technology for quite some time, but one of the issues that we have been ‑‑ or that has been ‑‑ that we have been quite interested in for some time now is the digitalization of security policies, particularly the effect on rights to privacy, freedom of expression, and other associated human rights.

Now, in this interest that we have as an organization, there are three things that we have or that have particularly caught our attention in the debate in question.  The first one is the apparent understanding of the unescapable tradeoffs between security and privacy, that is a common notion right now among public security organizations across the different countries, and this doesn't differ between Global South countries.

The second is the lack of capacities from states, and this one is specific to Global South states, in implementing or to implement complex digital systems, and this is a direct effect since it brings a new actor to discuss to the table which is private sector companies mainly from the Global North but as countries like China that end up implementing such policies in our territories.

Lastly, and this is particularly important because we are adding an IGF, the role and responsibility of international cooperation are very much often overlooking Global South context.  Why?  Because most of the countries are dependent and need this kind of help, so it's very difficult to negotiate international cooperation agreements and at the same time for instance, contesting the kind of packages that are associated with those cooperations, and for instance, implement or develop human rights impact assessments or data protection impact assessments.  This is also, or this also functions backwards.  Why?  Because also international cooperation bodies don't necessarily and don't analyze the impact of digitalization in security policies, and they don't implement human rights impact assessment, or they don't analyze the impact of a given help they will give to a country from a human rights standpoint.

Now, transversal to the three points that have caught our attention as Civil Society organizations.  There are structural realities that are complex ‑‑ or further make complex the scenario that I am describing right now.  The effective multistakeholder model at the national level and intersection of technology, security and digitalization is also in general somewhat dead, and I do not want to sound pessimistic, but this let's say lack of effectiveness of the multistakeholder space as we speak, further complicates the different voices that can debate or generally evaluate the nuances of the tradeoffs, for instance, of privacy and security, which are indeed very well developed standards or well‑developed processes in the human rights field and also in the public admin literature.

Second, most of the countries of the majority of the world lack adequate data protection frameworks that could somewhat limit the potential adverse effects of surveillance systems and security programs in general.  They're not sufficient, but they are a starting point, and we don't have that.

And just to give an example, for instance, such effectiveness is reflected in laws that lack proper application or effective reparation mechanisms in our territories or even the lack of independent authorities to enforce the law in the first place.

Now, to reflect on all of these matters that we believe are urgent, and how we can establish networks moving forward because in the end this is a workshop that intends to create a new space or sort of like exchange knowledge about different spaces that are existing right now that are discussing these issues to ignite a multistakeholder collaboration on the topic from a rights respecting point of view, we have a great group of individuals with us, so I'm going to do a quick overview of who they are and then we can go straight into the questions.

The first one is Helena Secaf data privacy Brazil Research, producing research and advocacy actions before the justice system, legislative bodies and government.

Juan here on my left is coordinator of autonomy and dignity of charisma a Civil Society organization that seeks to protect and promote human rights and social justice in the design and use of digital technologies.  We have Eduardo Ferreyra also joining online so thank you so much to both of you because they are in Latin America so it's very early in the morning, I wouldn't even call it morning, to be honest.

It's the project leader of ADC a Civil Society organization based in Argentina that works to defend and promote civil and human rights in Argentina and Latin America.  We have Dorothy on my right chief executive officer of Unwanted Witness organization based in Uganda that seeks to create secure platforms for activists, bloggers, freelance journalists and vloggers to promote human rights through writing, informing, educating the citizenry who also utilize the platform for strengthening free expression and accountability.

I'm about to finish so bear with me.  It's a big group.  We also have the pleasure of having Dr. Lea WEOG that works with partners around the world and partnership of data and technologies.

And lastly and thank you very much Miguel Canada for joining on the online from Paraguay, serving Director of Human Rights Ministry of Foreign Affairs and past 10 years served as mission and Director of four strategic security of security of Ministry of Foreign Affairs and also a United Nations GE MAG member between 2017 and 2019 and LAC IGF program committee from 2019 to 2021.

Now just to give an overview of what's going to happen next, the panelists will bring different experiences across the security, privacy, broader human rights landscape, so they're going to bring reflections from their territories on how the deployment of these digital technologies have affected their territories in different ways, and also what are the different nuances that should and must be considered by policymakers, the private sector, vendors, and also international cooperation when deploying or financing these kind of initiatives, since they directly impact people's lives.  There are people who are affected by these technologies and it is also very important to consider that vulnerable communities such as socioeconomic disadvantaged groups or migrant community have a specific disproportionate effect when dealing with these technologies.

So now, Helena, I don't know if you hear us, I hope you do.  I want to start with you a bit.  If you can tell us a bit of what is happening right now in the triple‑border area, which is an area for those who don't know, an area shared by Brazil, Argentina, and Paraguay, so it's a very sensitive area in terms of a lot of interest happening and a lot of geo‑political interest as well, so of course there is a lot of public security debate on the region and for the region.

So, Helena what are the main things happening right now regarding technology right now in the area?

   >> HELENA SECAF:  Thanks.  Hello, everyone.  Thank you for the introduction.  It is really a pleasure to be here and to discuss this very important topic side by side with all of these other brilliant panelists.  But with no further ado because I have a very short time, let me start answering your question by giving a bit more of the contextualization of the triple‑border area.  As you mentioned located between Argentina, Paraguay, and Brazil.  It's strategic because it has one of the largest hydroelectric on the planet.  Thriving economy, high circulation of people, goods, services, and also the region presents a very difficult security context, so drug and weapon trafficking rates are high, and this border area facilitates smuggling, and there are correlated violence episodes and so on which also creates a feeling of insecurity in the region and a security challenge side by side with all of the great economy and so on.

So, this is the context in which the neighboring countries, they have been developing a series of strategies in the last years to seek to guarantee the areas security.  I will base my answer about the question of the trends on the explorative studies that Data Privacy Brazil and (?) has been working on for the last two years to investigate and bring awareness to digital security programs that were recently implemented in the triple‑border area.

So, the first study was conducted and finished in 2021.  It was a study of the integrated center of border operation, so a Brazilian public security program whose goal is to fight international trans‑organized crime by different public security agents by centralizing information under one roof.

The second study, a bit of a follow‑up study, we have been conducting since the beginning of the year and it's not finished yet, but our focus are the security programs of Mot (?) in Brazil and (?) in Paraguay.  M is a program that implementing a line of drones deployed in combination with security cameras to recognize lakes and provide technological support for the federal revenue, and this is part of this bigger program called (?) which has deployed 70 high‑definition cameras combined with artificial intelligence to identify patterns and generate data to improve the control of between Brazil and Paraguay.  And then the facial recognition technology system implemented in the main international airport of Paraguay to facilitate the services, so just to give context of what we have been studying.

I would like to highlight three main findings regarding the study, and the first one is the lack of transparency.  We already have a clue it would be hard to find the way data was collected and treated and how and who was involved and who has access to the data and so on, so we had a clue that it would be hard because the actual idea to start studying this project came from the lack of information available about it.  So, we used three sources for gathering information, documentary reviews, interviews with people related to the center, and for your requests the efficient question and answer with the government and data privacy and all of this focusing on the Brazilian side, Paraguayan side, and even with all of these sources, our final reports still have some gaps on how the center works, and this is a very relevant finding regarding this lack of transparency.  Right.

The second main finding I would like to highlight is the international involvement, so in this study the Paraguayan side identified an apparent interest of the European Union in promoting the adoption of integrated centers in the areas, and so for instance there was evidence that pointed to involvement of the main cooperation programs currently in place that have support from the European Union interested in promoting integrated centers.

The Brazilian side identified a significant U.S. presence in the sea of implementation, so although the U.S. has no part in the center's day‑to‑day operationalization, they encouraged Brazilian authorities that manifested interest in creating one center by conducting guided visits to the U.S. fusion centers, sharing strategy and logic and so on.

And this gives me a lead‑in to the third finding that I would like to share, which is a regard to due process.  Although the main inspirational source were the U.S. fusion centers and the ones that we managed to briefly take a look at during our study, they had their own privacy policy documents.  It does not have any type of guidelines addressing legal privacy issues, it does not have privacy policy document, it did not do impact assessment evaluation, and however the database can be accessed when solicited by public security organizations, regardless of them being members of the center, so there is a bit of confusion situation over there.  What I'm trying to say is that this regard to due process was explicitly highlighted during the study by some of the agents when they were talking about, for instance, the importance of having a close relationship with the authorities of the other countries in order to have quick responses, so they would say that sometimes those relationships are even informal ones because excessive formality in cases like this would stifle the work on both sides.

I chose to talk about these points because I believe they give us hints based on this empirical studies of the main trends regarding technology deployment for security purposes in the triple‑border area.  The project we're studding right now, (?); although, in different ways allow us to actually understand those trends as general trends and not isolated perceptions from it.

So finally, to answer the question, I would highlight three trends that I have noticed, and all of them have to do with the narrative revolving around the implementation of the security projects, so the first one is technology as a synonym for efficiency, so this idea that the more technology, the more digitalized it is, necessarily the better.  Like there is no space for thinking that maybe technology could represent a disadvantage.  This is not a mentality.  Like technology means efficiency.

The second trend is efficiency as a value opposed to rights, so it's not that rights are not important, but sometimes rights and the due process that assures them should make way for efficiency in security.  Like this is not something that was explicitly said in all the three programs, but it's a very easy‑to‑find and implicit mentality of a tradeoff between security and efficiency, and I know my time is up so just 30 second.

The third trend that I would like to highlight is this justification of the lack of transparency because of security.  So sometimes we had no answers from the government because those programs are a matter of national security and information cannot be disclosed, so which by itself is a fair justification, but when combined with the fact that there are no available documents, impact assessments, privacy guidelines and so on, this creates an environment of low accountability and no safeguards whatsoever, so I guess that's what I would have to contribute to this question, and I'm very eager to listen to what the other panelists have to say.  Thank you very much.

   >> CARRILLO EDUARDO:  Thank you, Helena.  8 minutes and 51 second.  We're okay.  So, yeah, I think that definitely I would like to highlight that the issue ‑‑ the difficulty of transparency for researchers and the public in general in accessing these sort of programs and what are the processes in place to implement them, are quite a challenge that needs to be having in mind when designing these kind of policies, what kind of transparency measures can be and should be in place although there is this narrative of security around the whole issue.

Now, Juan, going from the mass surveillance approach of the triple‑border area and policy that Helena was describing, I would like to hear a bit more about the Columbian landscape because you have an interesting ‑‑ well, a somewhat interesting implementation of technology ‑‑ implementation of digital technology for specific groups, particularly migrant groups, so you know what are the security concerns there?  And then how digital technologies have or are playing a role in all of this?

>> JUAN:  Well, the story I'm going to tell is not that different from what Helena already told us, but first of all, thank you all for attending and thank you for accommodating us on this panel, which is really nice.

As you already heard from Eduardo, I'm here from Charisma a Civil Society organization also from the majority of world, based in Colombia, and what I'm bringing on the topic of human mobility between Colombia and Venezuela as Helena was pointing to, and the deployment specifically of biometric systems in such context.

So, I'll start by giving a bit of general context around the border of Colombia and Venezuela and the situation on human mobility.  Similar to what Helena described is quite a complex border because there is a lot of trade historically there and there has been an extensive exchange between both countries in both directions in the 80s due to the Columbians moving to Venezuela and now days the other way around, a lot of Venezuela coming into Colombia.  2015 the border was closed a while and then reopened and then there was a huge wave of migrants.  We have the numbers for 2018 which are official numbers at least around 1 million 200 people living in Colombia and the number could be at least twice as high around 2.5 million Venezuela moving into Colombia.  That number is starting to decrease slowly but still quite large.  In that context the former Government of Colombia that just left office this year, they launched a program called the (?) for protection of Venezuela migrants and program aimed at granting the migrants the possibility of working and being legally in Colombia to have a more official way of relating to the state.

The issue of temporal protection pyramid, PPT, by Spanish acronym, and through that they can access services and so on.

And that in order it acquire that permit, the migrants are required to complete a survey, a unique registry for Venezuela migrant survey, and that survey is divided into three steps.  The first one is a pre-registry which usually takes place at the border, where you cross the border and then you go to the migration office, register, take a rudimentary data of you, and then after that you're missing two steps, which are a thorough, a very, very thorough (?) survey and biometric registry which is also quite thorough.

So, before we go into the biometric registry, I want to talk about the second point which is characterization survey.  This takes place once the migrants are already in Colombia and they have to some extent settled in the country.  Of course, they do not yet have a permit to be there, but they already have some sort of life there and they're not in the border, and they're already there.  And in that survey, they ask really a lot of questions.  They ask for a lot of information.  They ask about self‑recognition and belonging.  Of course, where you're asking for refugee status or not.  They ask you for your former identify documents and then that includes ethnic belonging and identity, and that includes sexual identity and sexual orientation, which is sort of gender identity and sexual orientation, which is why would you need that?

They ask about your family group, the rest of your family planning to move here, they ask about your living conditions, what do you have access to subsidies, and they ask whether you have children or teenagers that you're in charge of, they ask whether your children are enrolled in school, they ask about health, which is I mean it would be all right to ask about health in general, but they ask about very specific conditions, contagious diseases, infections, STDs, and which ones, whether you're on medication for STDs, whether you're pregnant, lactating, and they ask also for the motive or motivation for your migration and why did you decide to move, which is or can be very sensitive political information, of course.

As well, they ask for your integration perception, whether you feel that you're being well treated by the host country.  And then they ask whether you have been subject to any form of violence in Colombia.  The reason why I'm going through this detail in the survey is because all of this information will be later on intertwined with the biometric data that they're collecting as well, which is in my eyes, a huge issue.

So, I'm going to talk now about the abuse of biometrics in this context, and there are four main conclusions I want to draw from this.  The first is that the state collects more data in the case of migrants than in the case of the Columbian population, the biometric registry of Venezuela population in Colombia, includes for instance, the measuring of the iris which is not something we have in our national document of identity as Columbian citizens, so they're clearly using more biometrics for that sort of population.

The second one is the obvious experimental way in which the system is being deployed on this system, that the first reason to think it is experimental is the one that I just gave, they're using more biometric.  But as, the next thing I want to talk about is the possibility of migrant population to withhold consent.  It is experimental because they have a population which cannot withhold their consent on giving that information and on giving that biometric data to the government.  This is a reason I made such an emphasis on the fact that this population is already living in Colombia when they complete the survey, it's not like they can just turn around and go back to Venezuela because they're in most cases seeking refugee status, and even if they're not, they're at least fleeing some very poor living conditions.

That's a huge issue, and it's proof that the deployment of this is quite experimental.  And the last thing I want to think about in the context of biometrics is the fact that those biometric data points are connected to the survey and the survey data points.  You could technically split the information into separate categories in which one would be identification, and for that you need very little information.  And then you have a statistical significance measurement of whether people need certain services provided by the state.  They're not doing that currently.  They are still linking the information, the individual information to individuals via the biometrics, so there is no ‑‑ not really a good justification for that as far as I can see.

And to close, I want to leave you with some general questions.  I don't have much time left but I'll try to be brief.  The first one is about the narratives, the two conflicting narratives that the state has, as in the same case of Helena, not necessarily explicit but two conflicting narratives that the government gives for this.

The first one is that they aim at granting security through surveillance with the biometric identity and biometric data.  The second one is that this ample data collection is aimed not at surveillance but granting rights to the population, and those narratives are to some extent competing with one another.  And now I get the feeling that the security narrative is gaining space in that discussion.

So, open questions for the panel.  Do we only ‑‑ do we need all of these biometric data for that purpose?  What's the role of private companies in the case of Colombia, (?) is the one that provides all of the biometric technology, so do we need them here, what's the role?  And then is this feasible as a security strategy?  I'm not quite convinced, but I guess we'll have a chat about that now.  Thank you.

   >> CARRILLO EDUARDO:  Thank you so much, Juan.  That old friend.  (Laughing).  Okay.  In the interest of time, you gave great points but I'm going to move to Eduardo who is joining from Argentina from a Civil Society organization called ADC.  Eduardo, there has been an interesting debate in Argentina around the legality of facial recognition technologies and facial recognition cameras in particular in certain public spaces in Argentina and in the debate there is currently a court ruling that has ignited quite a debate internationally because it has banned these technologies in certain spaces, so did public security play a role in this discussion, and could you give us a snapshot on the situation?

   >> FERREYRA EDUARDO:  Thank you, Carrillo.  Thank you for the invitation to be part of this panel, and thank you to all of you for attending this session.  My name is Eduardo Ferreyra, and I'm the leader at organization for civil rights NGO based in Argentina and also from the global majority.  In my presentation, I want to talk about two things.  The first one will be the research that ADC is conducting about the landscape of technology operating in Argentina, and the second one is going to be about facial recognition, particularly.

ADC has been researching this ‑‑ the array of technology in Argentina for many, many years.  Based on your experience in the country, there are several obstacles that Civil Society organizations face when researching surveilling technologies, and the first one is the lack of information transparency.  It was really hard for us to see the whole picture of the surveillance technologies deployed in Argentina because this information is not available through public channels.  We tried to obtain information in official websites of state purchasing, but neither national nor local governments provided any kind of information about their agreements with technology providers.  Then we tried to submit per request, but most were never answered by the government, and in those cases that we do receive a response, the information provided was very insufficient.

And it was that, despite the fact that Argentina has access to information law that was passed like four years ago, it was a very modern law, but public safety or national interest or open sight by government justification for refusing to provide information.

Litigation is not a workable option since the judicial process in Argentina takes too much time to help us to get information that we need for our work.  We respect the private company, and the situation was very similar.  We tried to reach out to them via email, via LinkedIn, but in the vast majority of cases, we were not able to establish the direct communication with company representatives, and one additional obstacle is that surveillance technology in Argentina is acquired through local suppliers and not directly from manufacturers.  This is another way for surveillance companies to remain in the dark and avoid public scrutiny in addition to inquiries and failing to be communicative and transparent.

During this scenario, we had to rely on alternative strategies.  We look at official statement made by public authorities and company representatives, some of these technologies were public launch by governments and we used those opportunities for information.  Company websites were also a useful tool.  We noticed that companies used the deployment of this technology as marketing case studies and in this way, we can get a glimpse of their relationship with the public sector.

Major reports in recent by independent journalists are also an important source of information for us and they help us to shed light on the private/public partnership involved in surveillance deployment and the poor human record of the surveillance companies around the world, however, it's worth highlighting that most mainstream outlets in Argentina usually and critically portray the tools as solution to violence and crime.

Our experience researching surveillance and technologies has left us a number of lessons, and some of them are it is important to create coalition with other CSOs, public official and private company’s representatives already look to provide information and therefore it's key to organization working on surveillance technology to be in touch with each other in order to obtain information, share contacts, or distribute research.

Second lesson is to work more closely with like‑minded journalists and the media.  Journalists research can be very great asset in shedding light on agreement between companies and government, and journalists can also play a key role in creating awareness and debate about changing narrative about surveillance.

Last one is, try to engage international actors in what is happening in your country.  Due to public image, war with national governance, at least in Argentina, more likely to pay attention to certain concerns with when they're voiced by international human rights organizations or international human rights bodies or actors, rather than national ones only.  So, it was pretty important for us to create coalition to create with these external actors.  That's the first part of my presentation.

Then going back to specifically the situation in Argentina, our research showed that surveillance cameras with facial recognition software are the most widely used technology in Argentina, and this system has been implemented with the goal of detecting people accused of committing crimes and may be the most relevant case in the facial recognition system is the system deployed in the city, the capital of Argentina and the system is deployed in the subway station and it started in 2019.

The city system was 1.5‑million people every day and as soon as it started to work, there were many cases of false‑positives, people wrongly detained by the police because they saw that this person was a fugitive.

The government by international human rights organization, human rights watch, used in the system to track children wanted for alleged crimes.  Facial recognition is also deployed in other cities of Argentina, so there is like a growing trend in my country to use the kind of system.  And with that scenario, different organizations had to rely on different strategies to encompass that, possible litigation and started litigation of the government against the city to challenge the constitution and challenge the facial recognition system, and also they were other law suits submitted by other organizations, by other colleagues, and all of this ended up in the justice declaring that this system deploying in the capital city was unconstitutional because it was mass surveillance and affecting the right to privacy.

And this judicial ruling was appealed by the national government, by the government and also pending on the ruling by the Supreme Court.  Anyway, no matter what happens with the final result and we think this is very good step in challenging this kind of system because it hopes to raise awareness in Argentina and helps to expose how this system is not only used to prevent crime, but as are being used to surveil the whole population in Argentina.

Another strategy that we had to rely on is to like he said before, introduce human rights international bodies.  For instance, in 2019, the former United Nations Rapporteur on the right to privacy visited Argentina, and we met with him and expressed our concerns about the situation on right to privacy in the country.  He met with those from national government and local governments and then published a statement in which he said that the public officials from the City of Buenos Aires and that was because they haven't conducted any human rights and data privacy impact assessment on the topic, so that way they couldn't explain to the reporter on why this system is needed in Argentina or Buenos Aires to perform security functions, no.

So, one of the things that ‑‑ one of the lessons that we get from that is that it's important to push from this kind of evaluation, this kind of assessment because it helps to ‑‑ it forces the government to be more transparent and honest about the technology deployed and also what they are doing.

I will stop here because I know we have another round of comments and questions, but just to briefly summarize my intervention.  It has to be contested by many strategies, advocacy, research, litigation, and there are tools that we can use them, and we can share experiences in other countries because fighting this of the system is very difficult considering what was said, people in the mainstream media tend to be these tools as solution to crime and not as tool that affects human rights.  I will stop here and pass the floor.  Thank you.

   >> CARRILLO EDUARDO:  Thank you, Eduardo.  You raised a lot of interesting points.  Particularly the transparency one again is a common thread.  I'm glad you mentioned the public/private partnership of the program because this is something that Helena might touch on.  We've been so far focused on the Latin American region and most of the panelists from Latin America, o so there is a bias there, (Laughing).  We are very happy to have Dorothy Mukasa from Unwanted Witness to what's happening.  What are regarding surveillance, what are the actors playing a role?  Can you give a brief overview of this?

   >> DOROTHY MUKASA:  Thank you so much.  Taking a break from the Latin American perspective, but I want to plug in the question that was raised and the role of telecom technology companies.  But as add what the role of international bodies should be in terms of supporting the application of technology in human rights.  I want to say that in Africa we do not produce these technologies, definitely, these technologies are imported into Africa.  Okay.  That brings in a critical part of the international body, you know, in terms of regulating how the technology is imported into Africa.

In all ‑‑ in all scenarios, you see that, you know, there is a rush for African countries to adopt these technologies, but we also know that as it has been mentioned by Helena, that security ‑‑ the use of technology aim to having super security, you know, in our countries.  So, we need to question why these technologies are important, and this I think, the African authorities are not able to question that because they're rushing for these technologies with hope that they're going to be to secure their countries.

So, this is why the role of the international community is very important, you know, to question and put measures that really, you know, limit the abuse that we are continuously seeing.

As Unwanted Witness, we started the Smart City and we all know the Smart City projects that are being undertaken in different parts of Africa, and Uganda is part of the Smart City project where we see, you know, China importing technologies in the countries with a hope that we'll be able to ‑‑ that governments are able to really curb crime, you know, but we've also seen abuses of these technologies in return.  While we're seeing a lot of shrinking civic space with the use of these technology, clamping down on generalistic work, activist work, but as you know, targeting different political actors with these technologies and in these places.

There is also a question of investment, you know, we're seeing a lot of investment in these technologies, at the expense of the citizen's welfare.  We have ailing health care systems.  People are trapped in poverty.  Then the priority of our governments are focusing on investing in technology, you know, in a way of securing citizens.

We're also seeing in places where we have data protection laws, for example in Uganda we have a data protection law, but then the exemptions when it comes to national security, data collected for national security.  I think that gives a leeway for governments to collect as much data as they can.  I mean being covered in two national security causes, okay, and this is the case with the systems in Uganda.

So far, we have invested over 200‑million dollars for in the system project, you know, but if you look at and assess the value for money, you know, vis‑a‑vis the importance of the technology there is a big gap, you know.  But as we've seen these work ‑‑ being applied during election processes, you know trying to limit civic space again, where, you know, the opposition is not given an opportunity to assemble, simply because that divergent view is not in favor to the powers that are in place right now.  In West Africa, for instance, we're seeing the French security firm developing similar systems of mass biometric services aided to stop integration, but as facilitate the deportation of different groups and this is a threat and often types the systems are not really ‑‑ there is no human rights impact assessment being done prior to the deployment of the system just like what I've heard in Latin America.  You know, these systems are just deployed across borders without any due processes that are being done.

Again, in 2019, you know, Uganda also with French company to deploy automated biometric identification systems being used at borders, but as scan technologies being deployed still at borders and these have been used often times, we've heard critical voices and videos that try to come into the country being deported simply because the critical voices that speak around issues of human rights in the country.  Again, these systems like previous speakers have said, are deployed in a lot of secrecy, you know.  Both procurement and deployment is always done in secrecy.

In the research that we did around security systems, we realize that had parliament which is an oversight body, was often times not involved.  They don't even know the companies that are given or awarded the tenders.  They do not know how much money is spent on these systems simply because the requests to parliament are presented as classified expenditures and these classified expenditures are not questionable and not by parliament, so that power is withdrawn and you know taken over by the security agencies that procure these systems.

So, it also goes back to the lack of transparency when it comes to these technologies.

There is from private companies marketing these technologies to companies, and often ‑‑ or to governments, and often times it's these companies that reach out to our governments, you know, with proposals of great security to the citizens and also to empower them to be ‑‑ to acquire more power and control over citizens.  Okay.

We have a Russian company that as of recent presented a proposal to be able to provide automated number plates to the government so that the government is able to trace, you know, monitor citizens through ‑‑ monitor their mobility within the country, but as across the country.

So, this is where I think we need to question the involvement of private companies in the provision of security technologies that are supposed to provide security in our jurisdictions.  So, it's important that the multistakeholder approach is taken in terms of pushing back on the way how these technologies are procured by also how the technologies are deployed in our own jurisdictions.  Thank you.  Time is up.

   >> CARRILLO EDUARDO:  Thank you so much, Dorothy.  Two things that are very interesting about what Dorothy brought are first of all that even though in places where data protection laws are in place, they are not necessarily sufficient because they clash with the national security narrative and it's a balance that still needs to be discussed.

Then the other one is the market share interest and the revenue of the companies that have a huge interest in deploying systems across the world, and I think that so far with the four examples that we have seen, we've pretty much seen that they are the same technologies used over and over again in different contexts and exported for different reasons, but with the same methodology.

So, in that tone and going from global ‑‑ from local to global, we have Lea from Privacy International working with a lot with partnerships and also tracking companies in general and export of these technologies, so from a global perspective, what are these trends regarding surveillance technologies in the export of these technologies that perhaps PI is identifying.

What can you share about this with us?

   >> LLIA SIATISTA:  Thank you very much.  It is a real pleasure to join the conversation today.  It has been really ‑‑ I've been taking notes and it's great to see also how everything is coming together as you said.  And I will stick to the African continent, just to keep the balance of how it's the same trends all around, and also, I will start by bringing an additional driver of this balance.  Dorothy mentioned the question of movement of private companies, so it's a question of who pays as well for these private companies to be joining, and it was also mentioned in Colombia and at the border about who funds these projects and involvement of third countries and all of that.

Let me share a few examples of it from the continent, so the U.S. national security agency has provided Ethiopia with technology and training enhancing country surveillance capability with knowledge and advantageous location.

A number of Chinese companies with close ties to the Chinese government, beyond Uganda what is already mentioned, we have also reports they supplied the similar bob way government with technologies, and then finally, the EU has funded a 28‑million Euro program aiming to develop a universal ID system in Senegal as part of the migration management policy.

These are a few examples of how countries with the largest defense and security sectors are transferring technology and practices to governments and agencies around the globe, and this is something that I think comes out very strongly out of this conversation.  Privacy international's research on traps furs of surveillance from the U.S. to EU and also international organization, including the UN, has brought to light the policy agendas which underpin this huge investment being made in security agencies ‑‑ in security capabilities.

Funding from these bodies can take various forms from direct equipment or training from security agencies to financing of their operations and procurement to facilitating exports of surveillance technologies by specific companies which brings it back and promoting digitalization that enables surveillance and doesn't pay due regard to the human rights protections that need to be in place when ‑‑ when using such technologies.

And it has been already mentioned that there isn't much more to say about what's the problem with actually driving this surveillance equipment because we know very little about it, what tools do they have, where are they coming from, what legislation is regulating them and do we even need them?  Is that the priority we should be having?  And this is a key point because with many of these cases, they are actually facilitating not what they claim to do but actually area power, tear apart communities, and often even also close down spaces where we can live free from fear and aggression, and we've heard already about the different narratives used, securitization being a primary one.  Different narratives of security to be honest, and about how these are implemented, and bringing it back to the EU immigration policy, which to me sounded ‑‑ when I first saw it, I was like okay, that's a new way of extending the way we see security.

Basically, the EU has been pursuing a clear border externalization so how to push people without reaching the physical borders of the EU space and doing the controls before that in the countries, the countries the migrants could be coming from, and they are doing that by equipping non‑member states with surveillance to prevent immigration from Europe and this is particularly by EU trust fund for Africa for stability in addressing root causes of irregular immigration and displaced persons in Africa.  It does of course fund development borders like water sanitation, which is an important aspect, and also equips and trains security agencies to enhance surveillance capabilities and I already mentioned Senegal being one, and Cote d'Ivoire has a similar system, but as there is an 11‑million and 11.5 million project that is funding a surveillance tool, drones, cameras, software, and telecommunications in Nigeria border and there are so many areas and the fund has half a billion budget and what the implementation of this project conveniently does is also influences the development of legislation which criminalizes movement across borders and (?) borders and awards lucrative contracts for companies which was already mentioned and often well‑connected European security companies and will influence, and will be felt for decades by millions of people.  Thank you.

And while the European commission has been funding this project and we'll continue to do with a new fund that where it's going to be integrated to, they have been paying very little due regard to actually doing human rights impact assessments including data protection and impact assessments before making those transfers, and our research has shown through freedom for information and document reviews has shown a complete lack of any such impact assessments, and so fresh from the press, and not yet on the press, European ombudsman, monitoring body of UN compliance with human rights, at least with us, issued decision concluding that the European commission was not able to demonstrate at that time measures in place ensured coherent and structural approach to assessing human rights impacts of European trust fund for Africa projects, and recommended that they introduce impact assessments into the transfers.

Bringing it back to private states through major leap in a way but not.  I mean it comes the golden question of what can we do about it, what can we do all together, and there are multiple challenges in that work from lack of transparency, profit interests, lack of legal frameworks, and the multiple actors involved from the funders, to companies, to national government, to the company that are affected, to the Civil Society.  Just to name a few.

And yet there are many strategies one can follow, and one was what I describe about targeting the funders of surveillance and pushing them to ensure protections are in place before making the transfers.  Two, actually targeting the companies that are involved, and one way to do that as well is by actually pushing for the introduction of proper legislation around public/private partnerships, and this includes basically company, any kind of company/government collaboration that aims where the government is outsourcing some part of their surveillance powers and increasingly depending on these companies to conduct surveillance, which is ultimately the prerogative of the states and not private sector.  Thank you.  With very little to say that privacy international, we ‑‑ the experience we have gathered but as the support and input from our partners and experiences and including ADC and others, and we have developed a handbook for Civil Society to help challenge these partnerships and to assist with understanding multi‑level understanding of tech, the law, and the government mechanism that are involved in order to be able to push nationally and internationally for stronger regulation of this ‑‑ of these relationships.  Thank you very much.

   >> CARRILLO EDUARDO:  We look forward to reading the report as well.  So, just to give a bit of context.  We're one presentation away to finish the first part, and then we have planned 15 minutes for actual engagement, and we have some guiding questions to hear your feedback about.  Much of the conversation right now is very much concentrated on different avenues that Civil Society can take in order to contest or better understand how the system work, but we actually want to push it forward toward an agenda that also ‑‑ or that also includes other actors who are doing it type of work and who are interested in collaborating in different ways.

So, Miguel, I hope you're hearing us well.  Thank you so much for joining from Paraguay.  Miguel, as I said, is from the Ministry of Foreign Affairs in Paraguay and sort of like bringing also the government sensitivity into the matter, so Miguel, can you tell us a bit from the stateman perspective or point of view, how do you illustrate the perspective by which the technologies arrive in the country, how does the country deal when implementing these technologies and what are like minimum safeguards that could be put in place when dealing with them in.

   >> MIGUEL CANDIA IBARRA:  Hi.  Thank you very much for having me on the panel and good morning, everybody.  I am happy, actually, that it was my time to talk after all of you, because I got to hear everyone's opinion first and then I can react to that.

I had, you know, some information that I want to share with you, but being able to listen to every experience you have I'm going to change a little bit of the way I was going to approach this panel.

I'm going to be a bit of a voice because I have to tell you things from the state's perspective and that is always complicated.  I'm happy I have Eduardo and one of my House representatives so I cannot lie in any way, and exercising my diplomatic nature, I would say I'm pretty much in agreement with everything that you said so far, every single panelist.

I'm going to try to give you a few pointers on the general situation of the globe, of every single Nation State in the way we try to approach, of course, every country is different and we have scales and powers that are in place.  But one first element from the state position is necessary to take into account that probably it's not usually deeply taken on from any perspective is the sovereign issue.  Sovereignty for us is first and foremost goal, deal, and nature of the pressures of the state in any scenario, so whatever we to, it has to be within a sovereign capacity and with a sovereign security taken standpoint.

So for us, it's very an element that is embedded in the nature of what we do.  And it takes us to why we need technologies.  And for states, technologies are just but as a tool.  In this case, they are not an end goal, it's not the end game.  The end game is to provide the services we are there for.  Technologies in these case, this doesn't actually equal efficiency, but they need to provide and within that scenario you have different needs from states, so if you have a region like EU or power like U.S. or China, you have a different point of view on security, but in our case, as a state in Paraguay in particular in this case, we need to solve some issues with technology, and that is one of them is for example, the insecurity sense of the populations and in order to provide better security, you need ICTs and ICTs as an element of engaging properly the policies in place, and that is not always easy and because of the state of world that we are fully connect Adobe Connected and Paraguay has around 40% of Internet access and pretty much everybody in Paraguay is connected and everybody in Paraguay is pretty much at the same time vulnerable to ICT misuse from state or any other actor.  This engages us with the issue of security versus human rights and I'm using this phrase on purpose.  There is no versus issue.  Of course, every state, either more or less, are engaged in human rights.  In the case of Paraguay, we are members of pretty much every single instrument in the international system, in the international human rights system, inter-American and universal, so for us the implementation of the law comes with the human rights duty, and states aside from other actors, are bounded by due process, so when I hear the words about due process in implementation of ICT, first, I fully agree.  Second, I have to remind all panelists and everyone that states are the first one bound by due process because for example, if the military, the security segments, the executive branch, does an initiative investigation or judicial investigation and does not comply with the human rights obligations of the country, it will be taken down by the defense lawyers or the even the judiciary branch.

So, everything that you do will not give you the service, and we don't give you the result if you don't follow the due process.  This is the case everywhere, and it depends on how the state is structured, but it is pretty much the case everywhere.  And then you have dichotomy in what we see as defense.  Defense for us for states is a state‑to‑state issue.  It's different from the fight against crime or organized crime in particular that is multinational.  That is a national security issue and international security issue as well, so then you have the differences between states.  You have a sovereignty issue again, and then you have an international cooperation issue on security and against organized crime.  There you have different foreign policy, different technologies that are not interoperable between them, lack of confidence and trust‑building measures within and along different ways of other countries and there are countries that take into account different points of view on how to deploy the defense and tie organized crime issues.

That takes us ‑‑ I'm going to read this very quickly.  That takes us to the issue of transparency.  There was another thing that was in everybody's presentation and transparency is, it's a difficult issue when you put it aside or in the same ‑‑ in parallel ways with security.  You know, even in the international system, in the international human rights system, you have conflicting articles of privacy and access, you know.  Those, again, taken into account the needs of being able to fight organized crime that is very well equip with ICTs, makes it really hard to be open and transparent because you're already giving your advantage on the strategic field, and that of course, has an effect on the citizenship, on the human rights of other people and particularly when you have to talk about organized crime, you talk about migration.  These issues are well connected and they are taken within the states from different situations, and this is quite an issue because normally ‑‑

   >> CARRILLO EDUARDO:  One minute, so sorry Miguel.

   >> MIGUEL CANDIA IBARRA:  That's all right.  One minute I think is more than I need.  I want to close by saying this.  We have to take states as a whole and with it in mind and that is not usually the reality.  States in the case of (?) states for certain.  The states actually challenge themselves, different branches and powers and institutions look to each other and sometimes challenge each other and sometimes blame each other for different things, and you can see different politics and different views from within governments, and from within states, so this has an effect on how you implement both policies and ICTs in itself.  So, I just wanted to leave you guys with that.  This is our ‑‑ the complexity of how states have to deploy the guarantees of the rights of the people that we have to protect in every single way.  But human rights are first, and from that we can build on it.  That's all from me now.  Thank you very much for the time.

   >> CARRILLO EDUARDO:  Thank you, Miguel.  I hope you can stay for the last 15 minutes for the actual discussion with the participants.  Hopefully it will happen.  So, I don't know if you guys can help me again to put the slides up with the question.  I think that two things that resonate a lot with me with what Miguel just said is that first of all the issue, I think that in a way it's very reassuring the importance of sovereignty and of course state's value, but at the same time this issue that he was mentioning about the state not being a whole and being different institutions implementing different policies, could potentially have an effect on how sovereignty is understood, and then that acquires a new nuance when you talk about implementation of digital technologies when normally not much of a connection between technology and sovereignty and that for instance could be an interesting value of creating multistakeholder processes that can help those governments offices to deal with those ‑‑ with those nuances and help them to identify them.

So, basically, we just want to open the floor.  We're so glad that we have so many people participating both online and also here on the floor.  Some initial questions that we decided to put to sort of like ‑‑ sort of like start the discussion was suppose the digital technologies were to be deployed in the context of security, border, and public security.  Can we identify and agree upon the minimum safeguarding processes that should be in place for these technologies no not to undermine fundamental rights, or for instance what are the best practices or processes that we can start to strengthen for networking spaces to exist between Civil Society and academics from the Global South with policymakers and international cooperation that work at the intersection of the issues that we have discussed?

Lastly, more from an operational standpoint, I guess, what direct communication channels we could start thinking of for discussing these issues?

So, again, thank you so much for everyone joining, and I think that now is just an issue of hearing what are your thoughts and perhaps if you have other experiences to share that could also be very interesting to hear.

Also, I forgot, if you can please identify yourself so we can do a follow‑up, if that's possible, of course.

   >> AUDIENCE MEMBER:  Good morning, everyone.  My name is Yaga and I'm from Senegal, a French‑speaking country, so I'm going to speak French.

   >> CARRILLO EDUARDO:  I'm a bit limited but if someone else.

   >> AUDIENCE MEMBER:  On inclusion, you are excluded if you don't speak English.  All right.  Let me try to speak English.

   >> CARRILLO EDUARDO:  No, no, no, speak French.

   >> AUDIENCE MEMBER:  That's fine.  Let me try to speak English.  I think when it comes to security and data protection, we will never have neither security nor data protection as long as we keep using technology built elsewhere with other needs, most likely different from our needs.

We have to learn from American position about foreign technology.  Necker does not want to see Chinese 5G technology.  Why?  Tik Tok banned from America?  Why?  Because they don't want ‑‑ because they don't want ‑‑ what they used to do to other people is spying on them using technology, and Chinese do the same to them.

Unfortunately, we're in a world where algorithm is more powerful, more powerful than legal protection.  So why in Africa, then, want to make us believe that if we have strong data protection act and powerful data protection authority we can be protected against risk of digital technology.  No way!  Technology has to be built from the ground up.  No shortcut, no way from.  To do that, we need a strong political will, we don't have it in Africa.  Because most of our country we have authoritarian regime and they don't care about human rights or security.  The only thing they care about is how to stay in power.  So whatever technology you sell them they going to buy it as long as it's going to allow them to further the control of their own people to stay in power.  And among those countries moving from democracy to authoritarian regime, we have Senegal, Ivory Coast, and Benin.  Thereby anything about security, in front of the situation, we have weak Civil Society that lack human resources and finance and resources.

But the biggest uptick of human rights in this context is access to information.  In Senegal you don't have alone access to information, no government public anything about civilian and technology, nothing, and unfortunately, we don't have cooperation or collaboration between international human rights organizations and like privacy international, access international and local organization.  We are not aware of any collaboration between privacy international and local organizations in francophone area.  Access now, they show up only when something bad happens in a country, and otherwise you don't see them.  Amnesty international, human rights watch, Africa 19, they don't work with local organization in Africa, especially in the francophone area.  So, this Civil Society has no way to advocate for human right and transparency, so what we need now is to have a strong collaboration between these international organizations where we have accountable government that cooperates with local organization in order for them to get information for them to advocate more human right in Africa.  Thank you.

   >> AUDIENCE MEMBER:  Thanks for this panel.  It's been a fascinating conversation.  I have so many things on my mind right now.

   >> CARRILLO EDUARDO:  Could you identify yourself.

   >> AUDIENCE MEMBER:  My name is Shabnon with the international center for not‑for‑profit law.  So, just I really appreciated the perspective from Miguel, I think, the official from Paraguay, and I just wanted to reflect that like the privacy rights issue, it needs to be seen hand in hand with the human security issue.  If you exclude privacy rights from that, then you don't have full human security.  That's just my perspective on that.

The problem with lack of transparency, yes, due process, Paraguay and many other countries are valuing due process, but because of lack of transparency it's impossible to know when the rights are even violated.  Right.  A lot of these surveillance tech are being used without even ‑‑ with no intent to ever bring it to trial, like without any intent with using it as evidence in court, so it's never going to be challenged on those grounds, especially in the migration issues discussed by a couple of panelists, those are just used for security of the evidence and not for evidence and trial and how are you able to challenge through defense lawyers or through the judiciary, as was mentioned?

And then my questions are with Ilia you brought to light one of the things I've been thinking about is this whole ecosystem that feels very overwhelming to deal with, and I think you touched on an EU and Chinese cooperations and there is also FATF, the financial action task force recommendations, World Bank that's funding a lot of these things, ICAL at the UN which is with the airline biometric data collections standards, and these are in institutions that have conflicting bodies that are staying different things about the balance of rights and security and it just feels like an overwhelming problem to address especially with all the money that's involved, so yeah when the UNSR says something about the value of human rights and freedom of expression and privacy rights, sure, but it's going to be ‑‑ the money percent strings attached to addressing SIM card registration and digital ID laws, this is going to be enticing, especially with FATF recommending a lot of these measures.  What's the advocacy approach there?  Is there a plan?  You've done some interesting research, but what are ‑‑ what are actionable if he cans steps, and specifically are these for the NGOs in Uganda, Latin America, are these issues that you're thinking about and trying to address at the international norm's level?

And then, Dorothy, you mentioned procurement law which I think is quite fascinating, and you said there is like challenges there to actually have any transparency when it comes to procurement.  Do you think there are opportunities to reform existing laws to improve procedures and in other countries, are you dealing with or addressing procurement law or procurement law reform to deal with the tech and security sector?  I'm sorry if I took a lot of time.

   >> AUDIENCE MEMBER:  My name is Nema from Kenya ICT Action Network and we are an ICT regulatory company, and securely doing gender and digital rights program, and my question to you is are the opportunities of data integration security available for people with disabilities?

   >> CARRILLO EDUARDO:  That's the first round.  Do you want to address your first, Ilea?  There was someone else in the room, did you want to say something?  Are you okay?  Opportunity to address the first part?

   >> LLIA SIATISTA:  Yeah.  Happy to start.  No, it's an excellent point of the fact that it's an ecosystem, a multistakeholder ecosystem, and there isn't one advocacy strategy that will save it all.  It is coordinated efforts and concerted attempts to bring and different strategies ones used to actually challenge this.  Yes, you cannot bring it down but you can definitely improve it.  And so the one strategy we used was to go towards or against to search for information, to challenge the fact that they didn't do any impact assessment and try to establish an obligation they have to do that and that's what the European ombudsman is now as a tool, but I think it should also be brought to the national level as a card.

And then also to demand from the funders that they not only fund the surveillance but as fund trainings on human rights which we haven't seen, training introducing legislation that is also protecting, so this is one strategy.

Then another strategy would be to go towards the companies, telecos and big tech and ask them to introduce security protections that can mitigate some of the risk.

Another one would be to go and fight to better procurement procedures that you've mentioned.  So, there isn't one way that wins it all.  And as we were discussing first of all, it's a multistakeholder initiatives, and as they are, with fund strategies someone will be the target and aim.  Another one this same person or entity will be the ally and fighting together towards another goal.

I mean, I'll only ‑‑ I cannot speak on behalf of the entire international community, but I have, when we did the research on Senegal IDC system, this has been ‑‑ we have tried to reach out to Civil Society and we haven't been able to, and we also did freedom for information request, but actually right that did not work.  So.  It would be great to hear more after the panel.

   >> CARRILLO EDUARDO:  Do you want to say more about the procurement?  I think we are on time but can stretch for two or three minutes.

   >> DOROTHY MUKASA:  Let me quick.  Yes, I think there is need for reform, to reform the law, but as the laws, the traditional laws that are available can provide some basics, for instance provision of who is providing the technology, the bidding process provides transparency in the bidding process, but I think that this intentionally is vetted by governments to be able to operate in secrecy.  We have not seen in processes where these basics have been followed or even where they exist, but I think there is also ‑‑ it's also important to do some reforms to reflect the technological nature of the procurement processes.  Yes.

>> Thank you.  Yes.  (JUAN, the first issue I want to talk about is the issue of sovereignty quite at the center of this.  I think you can come from a classical perspective and understand sovereignty as many understand as with security and I'm thinking of course of the issue of borders.

But in our context, I think the fact that we're depending so much on foreign technology is in itself also a question of sovereignty, so those things cannot be at stake.  One has to take into consideration the fact that data also has to be dealt with to some extent, so that's very obvious in Colombia we have also biometric systems for elections and so on, and that, it's a real question of whether we're in charge of our own democracy or not.  So, I wouldn't ‑‑ I wouldn't go too deep on that argument from the state's perspective, perhaps.

The second thing I wanted to talk about is this very enriching idea of the state is not high of mind and not closed unit, but I think if you don't think of the state that way, the states are even higher for technology and for data recollection because if we think of the state as something which has many parts and it's shifting and moving, there is a chance that that information or technology can be used as misused as in Africa by somebody else, so you need even higher standards.

It's a very good idea of not thinking of it to construct a more structural and more complex image conceptually, but you also have to take into consideration the fact that those technologies can be misused by somebody else, by somebody who is not expecting to be misused by, and that's also a problem.

And lastly, I was thinking of how to move forward and whether part of taking this down and I think that's not necessarily the case.  I think we wouldn't disagree on the fact that we all form better security.  I think that's generally ‑‑ I mean not debatable, but the thing is do the systems work?  Do they actually do the job they're supposed to do?  I think we're saying that they don't, so let's get on the same page and let's all move in the same direction and try to achieve actual security through measures that work and that don't imply human rights violations.  That's important.

   >> CARRILLO EDUARDO:  Thank you so much, Juan.  I think that was a good wrap‑up to a lot of discussions that we've been having today.  Spanning from issues of transparency to issues of regulatory and issues of frameworks and more than anything I think the main idea or main purpose of the panel was to really see what other people are doing and what other people could potentially offer in a space where we can exchange information more than anything and get to a common ground in order to evaluate what one is saying, like are really these systems that effective?  Should the state really acquire them and spend a huge amount of money even though of course security has no price, but I think there are so many methodologies and processes in place, and I think, Juan, you mentioned one that hasn't been necessarily tackled is the issue of risk as is of the countries, that the U.S. or UK do in order to acquire certain technologies and that we don't necessarily in our context apply in the same way.

So, I think we're in the midst of this debate at the peak of its relevance, and I think that aside from the resistance strategies that we from Civil Society are indeed doing in different ways, like talking from the strategic litigation that Eduardo was mentioning or from the research that all of the organizations are doing and our, of course, dealing with difficulties when accessing the information but still finding its ways to get to what we need to get.  We still need more collaboration and still need to move forward also to a space where different actors can indeed convene in a meaningful and effective way, and for the just convene for the sake of and jointly address these topics.  I think we can wrap up now.  Thank you so much, everyone, for your time.  Thank you so much for everyone attending online.  Thank you so much to the technical team.  We really appreciate that everyone works smoothly.  Because I was very scared about that.  Thank you.  I hope we continue this conversation over coffee.  Thank you so much.