>> JAC SM KEE:  Good morning.  We're going to start at 9:50, because this morning is too awesome, and people are probably rolling around in the snow outside.  We will give them 15 minutes to roll, so if you want to roll in the snow, now is the time.

(Short break) 

>> JAC SM KEE:  Good morning, and thank you for being super early and super on time even though we're super late for the first session of the IGF, it's always a difficult one.  We anticipate registration and so on, and today is an additional fun challenge of the snow.  I'm guessing people will trickle in, but otherwise I think if we keep waiting, we will run out of time.  So welcome. 

Welcome to the session that has a very long name, which I actually don't even remember myself.  Let me see what it's called.  Body as Data: Dataveillance, the Informatization of the Body and Citizenship.  I'm Jac, and I'm with the Association for Network Organizations, and we're working on a range of issues on ICTs and social justice and we do a lot of work around gender and sexuality and the Internet.  We have with us here three discussions. 

This IGF ‑‑ this is probably going to be a recurring experience for you throughout this IGF.  Probably not just this IGF but many IGFs where speakers constantly change, actually.  So we apologize if you were here hoping to hear from Katarzyna, and Jeanette and Ralph didn't respond. 

We have three excellent discussions anyway who have done interesting research work in this area around data, around Big Data, around decision‑making based on Big Data and the indications and the intersection onto a body that is gendered and on the gender and sexuality and the body. 

We have with us Valentina Pellizzer, who heads EroTICs, which is a research network on sexuality and the Internet.  It began in 2008 and has expanded and also deepened in various area and now is looking at research on sexuality and the Internet particularly in South Asia. 

Then we have Bishakha Datta, who is really doing a whole range of work and really sort of pushing this envelope around thinking about the intersection between media, disability, sexuality, as well as Internet rights. 

And then we have (?) legal policy analyst and researcher and a huge geek on AI and AI ethics and is interested in this very interesting intersection as well within data law and how this impacts on social and political and cultural life. 

So let's ‑‑ so as a little bit of an introduction, I guess we are at the moment almost like living in an age of data where a lot ‑‑ we are driving towards more and more decision‑making that is being informed and driven by data.  It's almost like ‑‑ also, this is something which is a project by Civil Society.  We've always been asking for data‑driven policy, and we did not want policy that's done just based on assumptions and biases. 

However, this is now creating kind of a little bit of problematics to us in new and different ways, because the problems with data is, I suppose, three‑fold.  What is the issue of invisibility, so not being seen in datasets and counted and, therefore, not even sort of existing? 

The second is actually the problem of hypervisibility.  So you're too seen because your dataset or the representation of your particular group is somehow problematic and there were hypervisible and it's traditional for people that face discrimination in society, for example, because of their race, ethnicity, gender expressions or the sex workers they have or even populations of people like migrants.  So there's this problem of hypervisibility. 

Thirdly is, of course, this very familiar problem of bias where the dataset is simply somehow incorporating within it some level of pre‑existing bias that exists in society, whether this is in terms of who is asking the question and, therefore, to what perspective it is and to what end of value is this question? 

What are we trying to understand based on asking these questions?  How is the data collected?  Who is collecting the data?  How are particular dataset assembled to make bigger dataset and how and what decisions are made based on the datasets.  Traditionally we assume the how and who are people, policy makers who look at the data and sort of consider and make particular adjustments for biases and so on in the ideal situation. 

In the age of Artificial Intelligence and Machine Learning as well now, we're in a situation where maybe it's not like you can rely on this already problematic kind of agency of decision‑making by particular human beings.  So here we are in this moment, and what we discovered as well, having worked with primarily the Benjamin rites defenders and activists working on sexuality and reproductive rights as well as LBGTQ and queer communities, so it's difficult to do research in the area. 

It's difficult to do research in this area because those collecting huge amounts data are privatized entities and those who process it are privatized identities.  You enter into issues of algorithms and trade secrets.  Yesterday they had this wonderful hack in disco tech, and now the machines are so complicated they can't explain how they work.  So we have to hack them. 

So a lot of this hacking research is being done in some ways, sort of like reverse engineering.  You see the outcome and roll back, or you see datasets and you see vulnerabilities within that.  Yeah, within this kind of curious situation of being in the very, very data‑driven policy decision‑making age, the difficulty and challenges of doing research around this especially from Civil Society and your primary concern is looking at human rights impacted particularly at risk for marginalized communities and trying to orientate that this is what we need to pay attention to. 

This is where the panel aims to share with us some level of research done in this area, and also to invite you.  I think we're in this room for a reason, like we probably have an interesting curiosity in this, and invite you to share your thoughts, your work and your kind of like questions around this as well. 

Really, it's ‑‑ this room is very awkwardly organized.  It's like you versus us.  Hopefully we have a much more conversational thing happening for the remaining hour.  Consider this really an open invitation to really share your thoughts and your insights on this issue as well. 

So let me begin by inviting Valentina. 

>> VALENTINA PELLIZER:  Good morning.  This is a very linear presentation to help following when we talk about numbers, percentages and things.  It's based on the global surveys that we have done as part of the EroTICs. 

The presentation is aligned with the (?) and you have already heard what EroTICs is.  What's important is exploration in the network.  So these are real people.  People that come from different parts of the world, and that you want to understand, see and make change. 

So why all these bits of ourselves, of who we are, which are transformed and morphed and there is a touch to them but there's an attempt to try and secure them because they can be lost or re‑used.  So this idea of us, that is done by bits that then travel around and come back again in the form of any major attacks in the presentation of the story. 

As we have done a small issue paper called Big Data and gender surveillance that can be seen in the consulate and is available for our conversation.  One is about Big Data.  We don't talk about the little data each of us has, but we talk about the Big Data put together from the millions of people that interact.  That data contains everything, so I will not read again the definition.  That's just to set a common ground. 

So that contains social media, any kind of data, because we live in a world that surveils us all of the time.  We work on the machine.  If we pass a shop, there is a CC‑TV.  They're in the bank if we do a transaction.  It's just this body that works with all the data and those all come together. 

Then the meta data is invisible bits that people know are there that you can identify but not touch.  What is dataveillance?  That's when all the data comes together and can be used.  It's important to combine the two things, the data and surveillance, because surveillance very often is related with a specific mood as a tradition. 

So it's not so much about creating new jargon, but just to highlight and make invisible what is the means of surveillance.  Sometimes it's just the end of the surveillance itself.  It's how we package.  The package is important.  How things are put together is important, make a difference. 

All these institutions that collect and that come together, those four, let's say, this definition is the four main concepts around Big Data and data that's available to us.  So this survey.  The global surveys happened for a third time. 

The first two times the purpose is very simple.  It's to map the use of sexual rights activists, and this is a very wide spectrum.  It goes from women rights, gender rights, LBGTIQ population, and people invested in reproductive rights at large.  To document types of risk, content regulation, censorship. 

We usually don't stop at the dark side of the story.  We tell you how people react to it, and what is the joy on the Internet. 

The Internet at the beginning was a place of joy, and it's probably still a place of joy when people experiment about themselves, with pleasure, entertain.  Look at the policy question that this space also highlights. 

This survey was focusing on sexual expression, so what are the means of sexual expression?  What are the situations of surveillance, censorships and the rights with harassment and assistance strategies.  This surveillance is like giving you an experience. 

How people are on the Internet are specific people, activists, and what is their experience of this.  It will not tell you all the stories of the hidden layers, what connects the data and experience of the people and set on the ground.  It's important that we remember there are lived experiences because we can obstruct ourselves and we're real people suffering with consequences and enjoying victories. 

So I want the graphic.  We have 332 respondents, and 86% of the respondents define themselves as just women and just men.  We had a little number of trans and intersect populations.  We thought that wouldn't be fair to have them in percentages but just to highlight. 

This is a community that is presently aligned, but it's difficult to capture.  There is the issue of language and availability and how you can reach out and use the means to align the survey to respond and provide your data.  I will not pretend to know the specificity of the data. 

The majority of the respondents are LGBTQ, and the reason why it intersects people there is because there were a small number.  This is important for us, because this is a community we try to tell the story of or to talk together and to say ‑‑ and to see what is their experience and their strategy.  The age, the majority is 74% is between 18 and 39 years old.  Just 15% is between 40 and 49.  I just will go fast. 

81% of people suffer a kind of discrimination and the main form is related with social class and casts and then the sexual orientation.  There is the little word about intersection although.  It's important to understand this community we talk about is the experience while online. 

What is interesting is we asked why you are online and what you do online professionally and personally.  As you see, people are really consistent.  LGBTIQ activists are there to pursue their rights. 

They use the Internet as a space to raise their voice and to affirm their rights and to change our world.  The same is there for women's rights and sexual health.  Those are the main areas of engagement, I would say. 

And as you see, people are not there online just for surfing and looking.  They share information.  They search for information.  They create and do public action, look for dialogue, and also to network to work together.  As you see, just a small percent, 2% says the Internet is not good enough for what I do.  This is about the use. 

What are the tools?  As you see, there are many options.  Here I tried to make some of the aligns.  98% of people use social networks.  It's important to understand what social network means.  This specific public space that is a private space owned by someone. 

92% use instant messaging, and 89% website blogs and services.  What are these instant messaging, e‑mail and social network of people that mainly use?  WhatsApp is used by 90% of our respondents, and then 37% messages.  Both belong to the same corporation e‑mail, 87% is using (?) and 94% is on Facebook.  Facebook is the politics of their life.  From pleasure to politics, they put in there their specific place.  And the threat. 

75% of the respondents had experienced harassment, and then there are, of course, differing degrees.  63% intimidating online comments.  Whenever you say something that is different from what is perceived as the norm, this is the action. 

54% had experienced some blocked websites and filtering software, something that prevented them from accessing the information in looking for it.  If you remember, the majority of people are looking for information and sharing information. 

Africa is the region where this is most apparent.  The majority of our respondents are from Africa and Latin America.  We didn't have respondents in the main area of interest, north Europe or the U.N.  It's really a small percentage of voices but from what is called the Global South. 

This is the perception people have.  This is their experience, who has power over their sexual expression.  Who can influence, limit and model.  People say 6% from government and State, 64% from Internet providers and 40% from peers. 

Who gives peers so much power to see so much about us, so they can be so targeting and so specific and so expectative when they want to silence or intimidate?  And 43% of people say that they have ‑‑ they felt that social media, they had been followed and something had made them uncomfortable. 

People know sometimes exactly what, but this is between the bad experience or the uncomfortable experience and the affect.  This is what is the people experience. 

So someone is watching us.  And there's the strategy.  The majority of technical responses, which is mainly your privacy sector.  We show those individual responses, and a small community that empowers and tells each other how to protect or be more secured.  48% has no confrontation or solution. 

No confrontational goes from not saying anything and being silenced to just disappearing from the social network.  So just to delete your own account.  And 29% engaged in formal political social strategies to try and counter-react and create an alternative  narrative.  I'll stop here. 

This is just to give you a sense how LBGTIQ people and women's rights advocate, we receive and leave this sense of being surveilled, and there is this linkage like a submarine cable between the experience of the people and the data, which is what ‑‑ how to translate on the Internet or in the digit experience.  Our world becomes data. 

So what are the connections?  What makes this surveillance so effective, and how to change and be part of the conversation on what was said at the beginning.  How to move in this hyper visibility that is using twilight to be part of the conversation and then to make evident the bias and come back in a situation where rights are asserted. 

>> JAC SM KEE:  Thanks, Valentina.  You brought up interesting questions about impacts and value and the mission link.  Some realize there's some happening but not sure who, what, when, and how.  Just kind of knowing that it's ‑‑ what about the other 60 ‑‑ 57%?  Kind of like, okay, surveillance.  Oh, well.  So that's an interesting moment of existential living. 

Let's move to Bishakha Datta. 

>> BISHAKHA DATTA:  Hello.  I live in India in Mumbai, and how can I make this full screen?  Sorry.  Okay.  So I live in Mumbai, India, and our organization recently did a study, which is based on data. 

I'm going to take you through one small part of the study, because I think it throws up really interesting questions on the relationship between bodies and data.  So, first, just to introduce my talk, it's going to be called the pervert and the sexual freak, which are two of the categories that sort of came out in this dataset or that were produced by this dataset.  The research itself is a study that looks at one aspect of the Information Technology Act in India, which was an act that came in in 2000. 

Basically it deals with all Internet or digital‑related offenses.  We studied one section of the act called Section 67 or the provision that deals with what is called digital object sense knit.  Just very, very briefly, the language in the act is pretty much the same language in the act that deals with sort of physical obscenity in the Indian penal code, which came in in 1860 in colonial times when the court came in existence and frankly at a time when we didn't even have film and video, let alone the Internet, right? 

So it's a very archaic definition of on seven knit to begin with which talks about obscenity or something depraved or corrupt and is, of course, all these forward inherently subjects unveiled and leave a great deal of judicial discretion.  That is not the direction that I actually want to go in today. 

What I want to talk about is one piece of our research actually looked at one of the biggest datasets in India related to crime.  That is produced by a government agency called the national crime record bureau.  The national crime records bureau is essentially empowered through an act of parliament for the last 45 years or so to basically gather data on all crimes that take place in India. 

These are then collected every year and produced in a volume that is called Crime in India.  So the image that I have up here is for the cover of Crime in India 2015.  This is a volume that is essentially just statistics, so there are 20 chapters of statistics.  Crimes are subclassified and chapter 18 is the one that deals with what are called cybercrimes.  All crimes under the Information Technology Act. 

So one of the things that, you know, when we were doing this research, this is public data.  This is available on the national crime records bureau website every year.  This is actually used by journalists and media in India to extensively talk about factors in crime, right? 

So is sexual violence going up?  Are thefts going up?  Et cetera?  A lot of the data is taken at face value.  At the same time, what we are seeing is that academics and statisticians are really questioning the value of this kind of data simply because this data is collected from different states in India, and in that sense data is political. 

States also have incentives to underreport data, to show that the states are crime‑free or that they're doing a good job of governance, et cetera, et cetera.  So I think the first point I wanted to make is that data is political no matter how it's generated. 

We cannot take it at face value. 

I think this became really evident to us when we were looking at this particular section, which is section 67 and chapter 18 of the crimes in India things.  So one of the things that we found that is really interesting is there's a huge discrepancy in the way the national crime records bureau records data related to all other crimes and cybercrime. 

So, for example, out of the 20 chapters, in 19 chapters, which do not deal with Internet‑related offenses, there is just statistical data that is collected.  How many crimes were committed?  How many arrests were made?  How many cases are still pending convictions, et cetera?  When it comes to cybercrime, there was two additional categories that have been added on. 

One, for the first time there is a table called 18.7.  This actually tries to categorize crime according to the motives.  Now, if you just look at this, you'll see that just a very casual glance, you know, looking at things like some of the motives are personal, revenge, settling scores, emotional motives, greed, financial, gain, extortion, causing disrepute and insult to the modesty of women, which is archaic language for sexual harassment in India. 

We have a law under the Indian Penal Code which is sort of about insulting modesty of women rather than punishing sexual harassment. 

First of all, you see in the language itself the kind of biases underlying this. 

Secondly, I think one of the things that we felt is that if you actually look at the number of cases for which motives are attributed and the total number of cases, there's a huge mismatch.  So it's not very clear how these motives have actually been attributed.  It seems to us that basically somebody in the national crime records bureau has gone through this and has attributed motives sort of in a subjective fashion, right? 

There's no information or transparency on how it's undertaken, which makes the data less credible in my view because there's no information how it's derived.  The second issue is not clear whether a particular crime can have multiple motives or it's a singular motive.  Part of this is seeing this as well as the next one.  Look at the list of motives again. 

Sexual exploitation, political motives, inciting hate crimes against communities, inciting hate crimes against a country, disrupt public services, right?  So data classification, which is kind of a little bit apples and oranges.  It's not ‑‑ it's not comparing apples and apples.  It's actually comparing a wide basket of fruits, in my opinion. 

It's not very clear what is the motivation of the person or the agency that classified this data, and I think what is even more questionable is when they tallied up the numbers, we found that the biggest category of motives was called "other," which basically 50% of cybercrimes were classified as other motives. 

The question becomes how can you really treat as credible a system of data classification where the biggest element goes into the category which is actually supposed to be the smallest category, which is "other," right?  But I think the other things that I wanted to talk about is we also looked at apart from the table on motives, there was a new table that was introduced in 2014, which is for the first time the national crime records bureau tried to create a profile of persons arrested or accused in connection with cybercrime.

Again, I'm projecting that right now.  Again, the interesting point, this was not done for any other kind of crime.  So that itself, the fact that this kind of additional data is being created for Internet‑related crimes shows a certain underlying anxiety or a certain sort of underlying suspicion with relationship to the Internet. 

Again, if you look at some of the classifications here, it's, again, a very odd schema.  First of all, the people who are accused are divided into what are called foreigners and Indians.  It's not clear why this kind of classification is needed. 

Even more oddly, if you look at it, you find that under each head there are new categories that are created.  For example, column 4 has a category called a sexual freak.  The sexual freak is both like amusing and unethical and dangerous and highly biased, because here's the interesting thing.  Again, zero information. 

How do you arrive at this category?  We have no idea.  We've tried talking to the national crime records bureau, and we cannot get any answers.  What this category means is absolutely like opaque.  We have no idea, but does it mean ‑‑ the second thing is I haven't been able to reproduce the whole table here. 

What's interesting is if you add up ‑‑ suppose you sort of deduce that perhaps sexual freak is something that derives from people who have been arrested on digital obscenity charges.  You say, okay, how many people are under this category, and how many people have been arrested under this provision?  The numbers don't match at all.  There's no relationship. 

So, again, it seems like a very arbitrarily constructed category that criminalizes a certain kind of body, which is seen as dangerous in the mind of the State.  To this I would add the next column as well, which is cracker hacker, or you see the casualness with which people are just being labeled cyberterrorists, right?  These whole categories are just being created, and next to these I also like, again, equally subjective categories like employee, disgruntled employee. 

I want to say how can one even take this kind of data seriously?  You know, there's so many questions that it overrides the value of the data itself.  I'm showing you the same thing for Indian nationals, as it's called, but again you see there are categories like persons with psychological disorders. 

What does that mean?  Does that mean like what?  Somebody with a ‑‑ I mean, I don't know what to say, to be honest, because I could ask like 85 questions on this alone. 

Political person, what does that mean?  Religious person.  To me, what is really sort of scary about this whole piece is that it applies only to Internet‑related offenses and not to the rest of the statistics that are collected by the national crime records bureau.  Again, whether we looked at how many of these actually add up, we again found that the vast majority are under other, which shows that actually the people doing the classification had no idea where to put them, right? 

So another element of subjectivity, so actually to conclude, I just want to read out a couple of quotes, because we were looking at how, you know, data has been theorized.  Two of the really interesting things that we found was one is actually a book by two information statisticians.  One is Geoffrey Bowker and Susan Leigh Star, and they wrote a book “Sorting Things Out.”

They talk about how each standard and each category valuizing some point of view, and silence is another.  They say this is not inherently a bad thing. 

Indeed, it is inescapable, but it is an ethical choice, and as such it's dangerous and not bad, but dangerous.  To build on that, I want to actually quote another statistician and a radical librarian from the United States. 

His name is Sanford Berman, and in the 1960s he wrote a book called “Prejudices and Antipathies,” where he critiqued the Library of Congress catalog headings.  He actually wrote there that, you know, there was certain sort of categories.  For example, at that time lesbianism and homosexuality were placed under sexual perversion. 

He writes there again that, you know, the reference smears and blemishes the habits of gay and lesbian women may be different, but are not, therefore, more dangerous than those of the heterosexual majority.  He says the way in which the Library of Congress headings were done at that point in time was such that they dealt with people in cultures or humanity in such a way that it could only satisfy very, very parochial sort of points of view or world views, et cetera, and sort of very Orientalist kind of framing. 

The reason I end with these two quotes is just to say I think if we look more deeply as the assumptions behind the ways in which cybercrime or Internet‑related offenses under the Information Technology Act in India are collected, categorized, framed and presented to an uncritical media and perhaps public, I think we would find very much similar assumptions that point to a world view in which the inherent ‑‑ the Internet is considered inherently something that provokes panic, anxiety, suspicion and unease.  Thank you. 

>> JAC SM KEE:  Thanks very much.  Suddenly like you know the Excel sheets and headings themselves have become this very powerful way to create these deviant sexual subjects.  I can't believe sexual freaks is a category.  That's amazing.  What's a sexual freak?  That's like an ‑‑ computer geeks are right next to the sexual freak.  Spending too much time on the computer, you must be watching porn, sexual freak, end. 

I think the thing that sort of really is the most scary about that is actually what is it, and then what?  What else is it making?  What is this data that's supposed so neutral and well‑collected creating and causing? 

>> SPEAKER:  Hi, everyone.  Can you hear me okay?  Good morning.  I'm a legal researcher and, as Jac said, I'm a bit of geek.  I'm a lawyer that tries very hard to not be one.  I just stay within the law in the presentations. 

The previous two amazing presentations lead nicely onto mine.  Like every researcher, my interest in this area started from a place of confusion because surveillance and privacy and Big Data and increasing AI mean everything.  Hence, I feel they will start meaning nothing because there's very little precision in the kind of conversations that we have about these technologies and these concepts. 

So my general motive today is to talk about how do we begin thinking about all of these, you know, issues like surveillance and gender and Big Data and AI and every other technology that's going to safe us all and be the next big thing. 

How do we really break it down and try and understand it to a critical length?  So I'm going to offer one particular critical length and show how that maps onto issues of gender and data and surveillance. 

So when we talk about surveillance, I think it's important to think about how did we get here?  How did we move from a CC‑TV to Big Data and how do we break it down from doing research? 

I like to break it down into three different phases.  The first is an architectural phase of surveillance and the second is infrastructure and the third of conceptualization.  The first is synoptics is a synonym for surveillance.  He came up with this idea you're seen, but you can't see the one that sees you.  It's kind of an illusion of constantly being watched. 

It's the technology of discipline, which means I have self‑sensor myself in order to see myself in a certain way so the person watches me knows I'm not doing anything wrong. 

He took that and came up with a lot of interesting ideas.  The first one was, of course, (?) which is the type of power applied to individuals.  The more important ones and more relevant to this conversation is the idea of normation, where the norm is something to aspire to.  If my GPS data is normal, I'm safe from surveillance.  If I'm different from the norm or the ideal I should strive to, I'm inviting surveillance into my life. 

We create this situation where we're all striving for this ideal but don't question why we need to do that.  Often, we don't question why we do other things.  Why didn't you post that on Facebook even though you typed it out?  Maybe it would invite certain kinds of comments or engagement in your life you're not comfortable. 

Also using discipline as a technology.  I'm sorry.  The text is not very friendly for early morning reading.  We can roll with it. 

So in this first phase it's a very spatial concept.  It talks about being seen and not being able to see the one that sees you.  When we move away from this first phase of architectural theories of surveillance and move into infrastructure, we say, okay, it's not just about some prison guard looking at me.  We're moving away from governments into companies. 

It's not just a vertical relationship, but surveillance is a horizontal relationship.  Also, it's not just about what I am doing, you know, in my prison cell, but it's about all of the presentations of me I leave behind. 

So it's like my data point.  It's my status.  It's my location information.  So surveillance was first abstract.  I'm sorry. 

It was first very physical, and now it becomes an abstract concept because I can't see when I'm seen.  It's creating this layer where I'm not able to really engage with the person that's seeing me. 

Then we move on to post-synoptical surveillance.  This is a very ‑‑ you probably can't read it.  That's a visualization to show how the different forms of surveillance add to the system in India.  So you have your education and your school and your pension, which are all like small silos of daft at that that existed previously. 

Now in this surveillance, they are part of one and coherent whole.  It's moving away from an architecture to an infrastructure.  We're moving into a network of infrastructure that makes sense about me, which is the new vertical surveillance. 

We move on to that surveillance where we're inviting people.  We ask people to follow our YouTube channel and like the status.  I'm taking part in the surveillance.  I'm in Geneva and will put up a picture on Instagram and I want you to know I'm drinking this latte and inviting you to surveil me.  That becomes an incentive and not a problem. 

So we have different directions of surveillance and also different conceptualizations of surveillance.  I may not want a CC‑TV to see what I'm doing, but I want my friends and people that follow me on social media, to know what I'm doing.  As a researcher I found this problematic. 

How do we break down the different types of surveillance we think of?  On Facebook it's not ugly, it's great.  How do we begin thinking about this from the point of view of gender and sexuality and stuff like that? 

So just in case this was a bit too much in the beginning, you have phase one.  It's architecture and spatial.  It has mechanisms and it's about control.  We move to Phase 2 which moves from institutions to networks.  It increases the space between the person who is watching you and the person who is being watched.  So it's a different kind of idea that we have there. 

Also, we're moving on to disciplining people and controlling people.  So this is a very interesting shift from a legal and technical perspective for me.  Then we move on to (?) this is combining the digital and visual surveillance of the kind of thing we invite. 

It's peer‑to‑peer.  It's not governments but corporations and combinations of two which we don't always know.  With don't know which corporation is working with what government to understand what about me. 

So in the digital world, we have different layers of complexity, like I just mentioned.  With social media, especially, it's about many to many models of surveillance.  They can belong to different parts of the government and stuff like that. 

I wanted to share this video.  Okay.  It's on YouTube.  Before it starts, I just want to point out this is something that the ACLU came up with in 2003.  It was called ordering (?) 2015.  This was at a point where we weren't all super excited about Big Data and AI.  In fact, it wasn't as cool as it is today. 

Just to have some perspective of how things don't really change and also how much work we have going forward, I thought it would be good to share.  I don't know about the ‑‑ is the sound on?  I don't know ‑‑ I don't think the sound is on.  That's okay. 

You can just watch it.  It's called (?) of the future.  You can go into YouTube and watch it. 

>> JAC SM KEE:  We have a version of this with a biometric I.D. and it's like, can I order pizza in give me your IC number, and according to the number, you have issues and you can only have pizza, and then your credit rating is bad, so you can only pay by cash.  It goes on and on.  It explains how this different selection of datasets come together and that video identifies it quite well as well.  

>> SPEAKER:  I'm sorry about that.  I just wanted to make a point that these ideas have existed for a long time, and we need to think about it not as a fashionable new concept and it ties into all three phases I spoke about.  Because we're short on time, I want to actually move ahead. 

All right.  So then coming to this idea of surveillance and gender, how do we take the different phases and make sense of them in terms of gender?  I think that we also need to be critical of the different ways in which gender and surveillance play out.  It can be at the level of the body where you're going through, you know, apple security in your surveilled mode and someone that identifies a certain gender because of the technology we developed through a scanner sets at airports only recognizes two binary genders, right? 

Unless you fit into this idea of the norm, what people call normation and unless you strive to be what is normal, you invite surveillance into your life.  That brings up the question, what should we do when we build technology?  What are we trying to fight against?  Is every deviant a bad deviant? 

That's a question we need to ask at the level of building technology as opposed to when technology discriminates.  Then we make a fuss about it, as we should, but at that point we're too late.  So being aware of this impact on surveillance and gender at the time of building technologies to make them respectful by design, I think, is a really powerful concept.  The second and this is a really sad concept that I wish I didn't have, but I do. 

So the search query itself is Indian ma Salah.  That's like spices we use to cook, and we have spicy food.  Everybody knows about that.  If you look in Google, this is what you get.  It's like this idea of ‑‑ I don't want to say it, but like, you know, spicy women.  That's sad, because it's not even like machines picked this up on their own. 

Somebody is actually spending time and energy to label these kind of pictures as certain kinds of ‑‑ to correspond with certain searches.  In the age of Big Data, it's bad enough we collect a lot of data.  Especially in India we don't have a privacy law.  There's no framework to work on that. 

We feed problematic data into the system.  We expand energy into doing this.  At the point technology makes sense of data, they don't have context, right? 

In this way we exacerbate existing problematic ideas in society, but we're not critical at the time of building it.  I think as people are interested in this kind of work, we need to be more critical of it's not just about the law and it's not just about policies.  It's at the point when you build technology, what considerations do we take into account? 

And this is, again, very small.  I'm happy to share my slides after this.  Basically, what this is doing is talking about ‑‑ so in India the Mumbai phone is a form of control and power. 

It's a privilege.  It's not something everybody has.  In rural India women are not given cell phones.  In India we have this revolution where they're going to save your lives and give you credit and a bank account and money.  I'm sorry.  I'm almost done. 

So, you know, they're making these promises about we're going to give you credit when no one else can?  What's the basis for that?  Your data.  They look at your location.  Do you have a job and leave the house every day and go six times a week or once a week?  If you look at this in terms of women in India, they're usually homemakers and usually don't have that data to make. 

Being a data subject then becomes a privilege in India because most people don't have phones.  That becomes part of the system, but we have no context of the time in which the algorithm is deciding whether you get a loan or not. 

This is a deeply gender‑specific problem in India, but we're not thinking about it in terms of gender.  We think about it in terms of optimizing for money and this syntax revolution is great for everyone.  Is it?  It's only great for one particular gender and nobody else. 

I really don't have time, but there's also some research on safety apps.  I'm very happy to share it.  I just want to make us all think about the different considerations we need to have when we are studying the data, the body and gender.  I think I should stop there.  I'm happy to share these slides with anyone who wants them.  Thank you so much. 

>> JAC SM KEE:  I'm so sorry to rush you.  We have a lot of time because no one is here because they're late and to oh, my God, we have no time.  We have to have ten minutes for a bit of discussion.  I kind of want it.  I know Janice just joins us as well.  I think you will be good at this moment to maybe open it up to the floor. 

This is a huge universe, and I really liked what you said. 

Big Data, AI suddenly is so big that it starts to mean nothing.  I think that's part of our struggle, which is how to make this mean something.  Then there's this disparity between countries.  I think you mentioned it.  In developing countries, you sort of want to be counted. 

The call is count me in.  I want to be part of a dataset.  In developing countries, please don't count me.  I don't want to be part of the dataset.  You have a huge disparity of people that have access and control the technology resources overcounted and overdecided upon. 

You have biometric on the other hand as well to become reconditioned to access social rights and no privacy protection or very little to the other side, that is about stronger privacy collection and control over the technology and some say, don't count me.  I don't want to be counted. 

It's kind of strange global discrimination before you even go deep.  I think that's part of the problematics we're presented with today.  Any thoughts, questions or comments and so on? 

>> MARILYN CADE:  I really want to thank you for this.  I came late.  What I heard was fantastic.  I was asked to speak because I'm known for AI bias, and I'm not taking about that.  Speaking as someone who really does AI and knows about Machine Learning, the body is data.  It's literally nothing you can do. 

It's not about how you project.  We need to understand that the more data they have, the better models we build, and the less data we need about each person. 

So just these little snapshots are enough to make predictions about what you're going to do, which is a new concept of the self.  We aren't usually used to thinking about algorithmic we are so people make predictions for us.  It's huge for politics. 

I want to draw this in to the point, the issue ‑‑ I'm so sorry.  We know each other by Twitter, and I don't know how to see your name.  This thing you were saying about that you want to project yourself to your friends but not to the other people, and I really worry about this in particular when we shame academics for showing us that maybe they can make gaydar for their computer.  This is terrible.  How can you do that? 

It's completely misunderstanding categories of humans and unethical.  If you shame the academics, you just push it out into industry and into politics.  We see what happens with that.  We see IT with the Cambridge Analytica and Trump and Brexit. 

Okay.  To say something positive, too.  What we have to do about this is we do have to work with governments.  This is what we have to do.  We need to get to the point, you know, we know that the police and the Army can get into our houses, but still if someone comes into our houses, we have a hope of organizing around protecting our house.  We have to do the same thing with the data. 

The EU does this, and I used to think this was wishy‑washy nonsense.  They say, oh, if people use your data in ways you don't expect, that's wrong.  Even though it's wishy‑washy, this is the way by which the Brexit case is prosecuted right now.  If people used Facebook likes to identify and manipulate voters, that may have been a violation of EU data regulations and this may be one of the ways that the referendum will be challenged. 

So even if things that seem like that, so engaging with government and figuring out how to maintain the utility of government is critical. 

>> JAC SM KEE:  Thanks so much.  Gosh, five minutes.  Let's solve the problem of data governance, come on.  We'll give you three suggestions.  Any other thoughts or questions?  Like really just insights around this. 

I think this is such a big puzzle that we all probably hold a small piece of it, and any small piece, I think, will help to sort of map out the landscape we're talking about.  No?  There's a lot of puzzled faces.  How are you feeling?  Are you feeling okay? 

>> MARILYN CADE:  This was completely different than the other two I just made, so you have a third weird thing to worry about.  One concern is that some people are so used to reconstructing their identity they feel it's essential we're allowed to do this through AI.  So I really find it concerning that some members of our community, who are used to reconstructing their gender identity or other kinds of identities, say robots have as much rights as I do.  Unfortunately, this is undermining human rights. 

We need to remember AI is something that's made.  It's a corporate artifact, and it uploads our information, right?  So it just isn't the same as humans.  We need to stay human‑centered, and I hope the whole community can agree to that. 

>> JAC SM KEE:  Yeah.  It's kind of funny.  Now we're trying to see, like, okay, AI has legal subjectivity.  They are thinking and making like ‑‑ they're writing now Harry Potter chapters.  It's terrible but they're doing it. 

Therefore, we have the same kind of protections and legal rights, but at the same time we sense well maybe actually human rights does quickly apply to human datasets that is all problematic and so on. 

You look at me like you have something to say. 

>> SPEAKER:  No. 

>> BISHAKHA DATTA:  I wanted to say something.  We sort of got into this zone where we keep thinking about Big Data, Artificial Intelligence, et cetera, all of which is, of course, totally valid.  I'm really curious how we will treat little data going forward?  I want to give you an example that ties into the surveillance stuff you were talking about. 

So we did workshops in India with teenage girls from low‑income communities and therefore don't necessarily have smartphones.  Some do, and some don't.  What's interesting is if you ask them what's your experience of surveillance, guess what they say?  Each and every girl we spoke to says that every night when I come home, my parents check my phone. 

My parents check three things.  One is they check who has called me, so call logs, if we think of call logs at data.  Two, they check my messages, so SMS, if it's a feature phone, and then, you know, and they check WhatsApp. 

It's totally related to gender and sexuality, because the parents are, A, not checking their son's phone.  So if you have a brother and sister of exactly the same age or in the same age group, the brother's phone is not being checked, but the thing that the parents are trying to get at is does my daughter have a boyfriend?  Is she meeting guys?  Right? 

So it's so deeply connected actually with just sexuality in the most, basic everyday sense of romance and desire and stuff like that.

(Captioning for this session will end in a few minutes.)

>> BISHAKHA DATTA:  It's interesting to me because it's sort of little data.  We have a data paradigm going forward. 

>> JAC SM KEE:  Thank you for that.  Big Data, small data, efficient data.  Do you want to just have one little summary, and then I'll close?  Yeah.  You can say it, but be aware people might leave. 

>> SPEAKER:  I'll take two or three minutes.  It's really powerful and I'm glad you brought up the point of being a dataset.  In India we see women are especially adamant about getting in this system, which allows you to collect biometrics and surveillance in motion.  I can't get my biometrics out and you can use this to identify me in different ways.  The obsession is from this fact I'm not a data subject or a data file in your huge dataset.  If I'm starting tomorrow, I won't get help because you don't know anything about me. 

What's really powerful and important to keep in might not, when it comes to surveillance and having the power of saying, I don't want to be surveilled, there's only one gender that has that power.  Those are the men, because they leave the house and they're the breadwinners and like Bishakha said, they can.  We need to be more aware of cultural approaches to surveillance because they're different in concepts. 

One is to fit on like a blanket approach and will always build.  I also understand privacy and the surveillance because these are ethical questions that are different place to place.  We need to be mindful of that thinking about the body and data. 

>> JAC SM KEE:  Thank you.  That was super.  I'm continuing on the line of her, the technology, the way in which the data is collected is one thing to look at.  It comes from a specific place of power or a specific way of looking at it.  But then on the other side there are all these culture‑specific, all the individualized. 

So I think that this is so important when we talk about the rights, because if we don't contest, we'll never change.  The one ideology is the one that is solved through government, so there are also governments of the south that are implementing very important and big, overwhelming projects that are using a specific way of looking and seeing and calculating. 

So I think it's really important as ten years ago we started to talk about online violence as effective violence.  We really need to talk about data as a body, and does the human's body deserve rights?  We're not all equal.  Not all voices are represented. 

It's not just an issue of power but an issue of rights.  Otherwise, data will remain in this field.

(Session concluded at 3:30 a.m.)