The Automated Public Sphere
CC Attribution - ShareAlike 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Pasquale, Frank
Can consumer protection and competition authorities stop the worst effects of our increasingly automated public sphere? Or is a deeper cultural change in the way we relate to and consume media necessary?
 Well, thank you so much, Geraldine, and thank you to the media convention. It's fantastic to be here. And I'm just going to dive right in because I have a lot to cover today. And just to sort of put my talk into segments, the way I'm going to start is by discussing really troubling applications of big data, algorithms, and artificial intelligence to the public sphere, to politics and culture. After going through some of these problems, I'm next going to talk about seven regulatory approaches that could be deployed in order to address the problems caused. And finally, I'm going to reflect on some situations where perhaps regulation is too late or counterproductive. So I think with that type of presentation, we'll have a balance and we'll try to understand what can government do, can it do anything to respond to the evolving media landscape that in many areas is very troubling. I want to start with sort of four ways in which recent developments in the public sphere over say the past five years or so have reversed the common utopian story about the nature of media online. And I would summarize this by saying that we've moved from a wealth of networks to a black box society. How did that happen? Well, it first happened because we were told that, you know, the ability of almost anyone to set up a website to say their own piece online would end big media domination and would empower everyone. But what we've also found is that it's enabled a great deal of fragmentation and extremism. Secondly, we were told that the enabling of anonymity would give people an extraordinary courage and freedom to speak their mind. But what we're also seeing is that the most powerful plutocratic forces in societies can set up shell corporations. We saw that in the Panama Papers. They can set up other entities that essentially allow their influence to go unchecked and nearly unmonitored because it is so secretly funded and accomplished. A third example of the utopian version of the public sphere via the Internet was the idea that all would participate, all would have a right to speak. But we've seen the flip side of that is a hashtag spam where, yes, if everyone has the right to speak, then you certainly also have the right to say, flood the hashtag Black Lives Matter with white supremacist propaganda. And to the extent that this is permitted, it is an almost immediate effort to strangle in the cradle the emancipatory potentials of the public sphere that we had all hoped and dreamed of 10 years ago or so. And perhaps even to make it even more topical the last week or so, our original conception of WikiLeaks was this wonderful tool to discredit the surveillance state. Now it seems as though it's sort of really bent on discrediting anti-fascist candidates in elections. And so when we see this type of transformation, we have to worry, and I'm going to go into some particular examples of this. The proliferation of Twitter bots, I think, is not merely a reflection of technical evolution that people like Tim Hwang were writing about years ago. It's also a reflection of a commitment to what I call free expression fundamentalism, but the idea is that, say, if anybody can code up a program that wants to speak or tweet thousands of times online, that that is a natural extension of their right to speak. I think we really have to combat that because we can't just allow, say, the romantic appeal of Isaac Asimov's talking robots to influence our idea of how software and automated methods of speaking can influence the public sphere because, you know, I can never keep up with a bot that can tweet a thousand times an hour, neither can anyone here. And the logical endpoint of such a completely unregulated, unmediated public sphere is one where those who are the wealthiest with the best coders are able to drown out everyone else. I'd also say that fake content on Facebook is a huge fundamental challenge to the idea of a marketplace of ideas ultimately enabling good, true speech to drive out the bad speech. In fact, what we're seeing is dark ads and personalization are directly targeting certain types of content to those who are most susceptible to it, and those who would dispute it will never see this content because it's in the dark. It's one of the themes of my book, The Black Box Society, that no one will be able to see this and counteract the speech because it's simply been so targeted on such a personal level. And what's really scary, I believe, about this is that we have a problem of human subject experimentation in many social media. We have constant efforts at A-B testing to see what works, what doesn't, and this can just as easily be a way of accelerating propaganda as it is a way of accelerating truth. Now a lot of sophisticated observers will ask you, why does this matter? Who believes the fake news? Who actually believes that, you know, Hillary Clinton had five people killed during the election 2016, et cetera. Who would believe these other lies? Well, that does make some sense for a lot of the folks that run in, say, highly sophisticated, highly educated circles. But we also have to worry a great deal about floating voters, low information voters, voters that are marginally attached to the political process, being particularly susceptible to a lot of efforts at propaganda and online manipulation. And when we consider the possibility of low information voters being the key ones who decide an election, and I think you can think of this in your own experience, I mean, most people one knows, they know exactly what their political commitments are, but there are this sort of core of low information voters that could be really disproportionately influenced. I'd also say that, you know, the promise of Facebook was that of a unified national, supernational, even global public sphere where we could all talk and debate, and that still is part, say, of the 6,000 word Zuckerberg manifesto with respect to Facebook. But the problem is that a sort of radical egalitarianism as to source can lead to a total lack of educative effect or branding as to the validity of the content. So when you have a situation where automation homogenizes the presentation of information so that everything looks the same, something could be from the Denver Guardian, so-called, that was just cooked up by a couple of people in a few days, or a site like that was just cooked up in this way, could appear with the same level of authority as something, say, that was from a prize winning newspaper investigative team. And this again, this idea of sort of debased egalitarianism, a way of sort of making everything look the same, is very troubling. I also think that one of the biggest problems here is that a lot of the people that run the biggest online platforms and intermediaries, Google, Apple, Facebook, Amazon, Microsoft, et cetera, they come from the US. And the culture of the US really has culminated, at least with respect to free expression, in something that I call First Amendment fundamentalism. The idea behind this is, it's in a book called, Cording the Abyss, where the strange value system is that the worse the speech is, the more misleading it is, the prouder we are to allow it, because it shows how radically committed we are to let everybody say whatever they want. The culmination of this is in things like, there's a case called USV Alvarez, called the Stolen Valor case, where the Supreme Court says, the government doesn't even have a right to make it illegal for people to lie about whether they had military honors. There's another case from the Washington State Supreme Court that essentially guarantees political candidates a right to lie. It's really just that simple, that we're just going to allow the candidates to lie. And when you have a public sphere that has this type of cultural legal attitude toward free expression, of course you have viral stories that say lies about that there are 3 million illegal votes or what have you. We also have to look at the material foundations, not just the legal foundations of this automated public sphere. And I think these are really brilliantly investigated by Jodi Dean in her book, Blog Theory, where she describes the circuits of drive and the profit that is made by simply outraging people. Personal opinion formation and deliberation is not a profit center, rapid sharing and engagement is. And that's one reason, for example, why even dataist interventions like Ben Grocer's effort to randomly automate your reactions to Facebook posts, why these things are so popular. Because people have a sense that it's sort of an out of control public sphere that really is very hard to understand where it's going or what one can do to even influence it in a positive way. Public responses have been various to this problem. So we have a situation where you have a lot of people that are trying to figure out is there a role for regulators, is there a role for consumer protection authorities, for media authorities, for other entities. Unfortunately though, there's another aspect of US American centrism that I think is overly influencing the internet debate, which is a rhetorical commitment to something called one unified open internet or don't break the internet. And so often what happens is that when there are efforts at rational public interest regulation of internet media, a lot of those at trade authorities from the US and other entities say if you allow different rules in say Germany or France or Argentina or Japan than US centered rules, you will break the internet. And this I think is something to be resisted because we need culturally and nationally specific responses to some of the problems that I've just gone through. The problem also is that we have an attitude from the US Federal Communications Commission pioneered in the 1980s that television and now the internet under Ajit Pai, the current FCC chair, is little more than a toaster with pictures. And if we trust the market to give us the best toasters, we can trust it to give us the best internet, the best program online. Of course even in the case of toasters, that's not true. There are product safety commissions that actually try to ensure that your toaster won't explode in your face when you put bread in it. But nevertheless, this is one of the problems in which we have many, much rhetoric from the US saying just don't regulate, it'll all work out in the end. But this is very disingenuous because essentially deregulation is a lie. Once you deregulate by the state in a given area, you essentially see the power to massive corporations to be the de facto regulators. And in our case, especially the problems that we're discussing at this conference, this convention, Facebook and Google effectively are the regulators, but they're acting in regulatory ways without the normal modes of accountability, such as due process, public comment, public responsiveness that are the hallmark of a legitimate regulatory authority. And so as I go through seven different regulatory responses, part of the inspiration for this is trying to figure out a way in which the de facto regulatory power of online entities could be translated into something that is more legitimate and more responsive to public ideals. This is in a nutshell the seven items that I'll be going over over the next 15 minutes, which are to label and monitor and explain heat-driven search results, to ban certain particularly dangerous content, to audit the logs of data-fed and algorithmic systems, to allow some limited outside annotations to limit the predation possible by online intermediaries. Because I think that's a huge part of the story here is that as you see so much money, power and attention being sucked up like a vortex centripetally into these online intermediaries, there's so much less to go to the standard or our media forces. The right to be forgotten as one case study in this area of algorithmic accountability and finally a shout out for media literacy. So one example here is labeling and monitoring biased certain results. And an example that I'm sort of putting on this here on the slide is when involving the rise of advocacy by the outright to promote a revisionist account of the Holocaust and of history up to World War II. And what we see in the situation is that people searching on the term Hitler on a Holocaust on the 1930s in Germany, etc., all of these types of searches are leading to extreme prominence for renewed right wing results and or far right Nazi results. And this was revealed in a piece by Carol Cadwalad. A few weeks ago she did some empirical research, some anecdotal studies or stories of here. What's kind of ironic here is that we've actually moved backwards from where we were in 2004. I started writing about search engines and their arrangement of information back in about 2003 or so. And when I was writing some of my first articles like rankings, reductionism and responsibility, I found this example of Jew watch which was a Holocaust denial site that was in the top ten results for the query Jew. At the time what Google did is they were extremely worried by this. The anti-deformation league complained to them and they had an asterisk next to it. And at the bottom of the page they said we are very concerned about this result. We believe it's been manipulated but just know that we can't really change it but we would like you to bear in mind the potentially illegitimate origins of this result. What Cadwalad's reporting revealed in the past year or so is that there was backsliding from that commitment by the company. And it's rather ironic, right? Because if anything Google is ten times more powerful and more rich than it was in 2004, it had a backslid. Another problem that I think is really important and what demands some responses is autocomplete bias. So we have a problem where for example autocompletes about certain individuals are leading to clearly biased results. Now here's one because it's a public figure, I am less inclined to endorse any sort of intervention. But certainly when there are private figures who are having autocompletes going after them, that's clearly troubling. And secondly when we have whole racial groups having autocompletes that are derogatory, that are troubling, that are deriding them, that is something we have to address. And I'm really looking forward to this forthcoming book by Safiyah Noble who has explored in great detail exactly what is going on in many of these algorithmic biases that show up in autocompletes and has proposed some concrete ways in which large intermediaries can try to intervene to assure that there's not this kind of bias and influence. And certainly even governments that have anti-hate laws should seriously consider how far this could go, how far down the road we could go. I say this in particular because one of the most troubling terrorist incidents in the United States over the past 10 years or so was this man, Dylan Roof, went to a South Carolina predominantly African-American church and killed over 10 people. When asked why he did it, he said he had Googled black on white crime and saw all of these results that sort of led him to say that there was an epidemic, a race war happening, where essentially there was a white genocide possible because of all of the black on white crime. Of course these things are entirely spurious, they're all hate driven. And so the idea is how as society is committed to pluralism, to tolerance, and to diversity, we try to ensure that what we have committed to formally, what we commit to in education is showing up in our online media because to be frank, that's much more important to many people than many of the standard media-educated outlets. That's how people really are connected to each other and connecting to the web. Now another thing that I think is so important here is audit logs of the data fed into algorithmic systems. And I want to explain the significance of this with reference to a scandal that happened in the United States, or this bizarre situation, a story called Pizza Gate, where essentially there were rumors based on, ostensibly based on some of the WikiLeaks releases that there was a child sex ring at a popular pizzeria in Washington that was connected to the Democratic establishment. Again, entirely false. But one, it took some time to get to the bottom of it because the people who spread this news were savvy about sort of how they would disseminate it. If you had better audit logs so you could better understand exactly where certain stories that are quite suspect originate, this one originated in a white supremacist tweet, this type of public regulatory understanding of the black box of these algorithmic systems could be crucial to rapid responses. Responses that happened before say in this example, a gunman goes to the site and starts firing shots and says, I just want to see what's really going on here. This is where audit logs are really helpful. And they're also really helpful because we hear a lot from Silicon Valley lobbyists and other tech advocates that the current modes of ordering information on search engines and social networks are so complicated, lock and never catch up. There was actually one machine learning expert that was quoted in a recent story that said, trying to explain how Google or other sites arrange information for you, for me as a computer scientist to explain that to you as a layman would be like trying to explain Shakespeare to a dog. And I think it's very important that when we hear these metaphors, we push back immediately. And first we say, the story about Pizzagate, that's not Shakespeare. And then we secondly say, you cannot adopt the sort of condescending attitude that the computational sphere is itself automatically superior to modes of human self-interpretation and human understanding. But that is precisely the core normative value in most of the deregulatory efforts here. And one small pushback against this is to say, even if we can't necessarily understand or explain in a humanly relatable way how machine learning works on thousands of variables with thousands of parameters, we can at least have logs of data that are influencing certain results that we see on Facebook and Google, and we could try to isolate and try to find suspect sources of information. Those suspect sources of information could, for example, be part of the annotations on certain posts. So if you recall the Jew watch example I gave from the beginning about how white supremacists were gaming the search engine to lead to anti-semitic results, that could be a much faster process if we have data from audit logs that allow us to identify, say, suspect efforts to manipulate news feeds, to manipulate search engine results, and then immediately or very close to immediately allow annotation of those results. And I think this is a very important step forward in how we regulate the economy. Because we have already committed to the idea that we need to have labeling of, say, food and drugs. So we understand, say, how many calories, how much fat is in food. What we need to take as a second step in an information economy is we not only need to have information about goods and services, we need to label, we have to have information about the information we get. We have to be able to understand that there might be certain suspect or troubling influences on the data we're getting that helps us to figure out what shows we're going to watch, what news we're going to trust, what feed we're going to follow, et cetera. And that's a premise of Mark Patterson's book on antitrust law in the new economy. It's an effort to sort of step to a second level of awareness and to break out of our self-imposed tutelage about black boxes, to break out, to really enlighten the public sphere in these scenarios. I also think that part of this is inevitably going to involve restoring human editors. And when I say restoring human editors, I don't mean doing this in the classic Facebook style of creating a cast of untouchable journalists who are out in a trailer somewhere that are contract workers who can be immediately removed or vetoed by anyone who's an engineer in the company. We have to realize that this sort of caste system, very explicitly described in Kate Lossy's book about Facebook, is a reality about the way in which a lot of entities in Silicon Valley see the relationship between journalists and engineers. What we instead need is we need to really make sure that there is this robust and substantive commitment to respecting journalism as a profession, not merely a source of piecework, propaganda or PR, and to ensure that human editors can, for example, look at breaking news story and just knock out ones that clearly have no basis in reality. Google's fact-checking initiative is one small step in this direction. They have partnered with entities like Snopes.com and factcheck.org. They're trying to ensure that there are some rigid parameters to qualify those who serve as fact-checkers, to have some level of accountability and responsibility there. But I do think there are some issues. One is sort of a interface problem, so one problem here might be that even though Snopes.com is debunking the story that Pope Francis endorsed Donald Trump for president, we nevertheless have the headline is the Pope endorses Donald Trump for president. We have to be sensitive to a world of, as Hartmut Rosik characterizes it, a world of acceleration and social acceleration, that we have the ability of entities to respond to the possibility that people may only be reading the headlines. So maybe even headlines have to be altered to respond to the real activity of media reception by new audiences, fast audiences. Same with the story about Clinton campaign chairman involved in satanic spirit cooking. Again, false story. Let's perhaps not require people to click to the second page. Let's have a better sense of what accurate headline would look like. And I also want to say that the groups involved, first of all, I question, are they being adequately supported by the larger intermediaries? The second question I have is, have we really opened it up to all potential groups, right? Or do we have a very narrow concept of what fact checking could entail? And I think that this type of civil society initiative, trying to recognize its quasi-governmental nature, trying to recognize that there are questions of legitimacy of representation involved is very important and trying to be open about that, right? And I think that even to the Google response to the right to be forgotten decision, there was not quite enough pluralism in terms of how they established a board to try to respond to that. There could be more. In going further, what I want to say is that we have to, when I talk about funding these other entities, part of that funding probably has to come from intermediaries themselves. And we had a proposal back in 2006 or so by Harvard professor Terry Fisher to assure that content was getting some share of the profits that were being made from the growing connection and internet connections of other people around the world. And he would actually have a proposal that would allow lots of access to content in response in after having attacks on internet connections. Now of course, I think that is far too regressive attacks. I would make it a much more progressive attacks. I think that is more than line the vision of someone like Jerry Lanier, but he also has some other ideas about how to make sure that content is actually compensated. But part of the idea here is that we have to recognize that as more and more revenue goes to the intermediary, less and less to the source that actually is enabling the intermediary to be something of value, we have to respond to that. Another example I'll give number six of my examples is the right to be forgotten. And I think that the example here is one where I know it's controversial. I know there are certainly concerns in the press about potential. Censorship or collateral censorship would be a more accurate term. But I think that when we look at the demands of individuals say, and when you have a woman who says, whenever people search my name, the first result they see is the fact that my husband was murdered 20 years ago. I don't want that to be the first result everyone sees about my name for the rest of my life into perpetuity. It seems that a humane compassionate response to that really is for the intermediary to help that person say obscure the story. At least when it's a search on her name. It reminds me of a lot of credit report or a dossier, private investigator's report on somebody. They really need the ability to respond to that. And this I think is a first step toward assuring that we are not just sort of drifting into an artificially intelligent future, but we're drifting into a wiser future where there's some artificial wisdom that's informed by a compassionate response and a requirement under law that intermediaries be somewhat responsive to individual concerns. Can it be abused, of course? But the answer to this abuse I think is better, more legitimate operationalization rather than ending the right in itself. And I think that this is part of, you know, it fits in together with some of the ideas that I've already been talking about with respect to obscuring certain content, banning certain content, because it really is quite possible that we are on the cusp of a social media genocide. You know, if you have the ability of individuals to spread hatred, to spread damaging stereotypes or fake news about say a minority group in the community, we could be very close to a scenario where that group in itself could have accelerated hatred directed against it. And we need to have mechanisms in place to try to at least retard that or slow down the acceleration of damaging content. A final example that I would give, and this is steps back from that sort of disaster scenario that I was thinking about in the earlier slide, is educating users in media literacy. One of the concrete examples here that I think is most compelling is that there are lots of people that do not understand the difference between a sponsored story on Facebook and an organic content. They don't understand the difference between sponsored ads on Google and real content or content that is ostensibly organic. And just to have these basic levels of media literacy taught maybe in fourth, fifth, sixth grade, so the individuals know what are they being paid to see, what is something that is arising out of a less commercial process, although certainly a commercial process. This type of media literacy is really crucial. And it's also crucial even in the smallest settings. So for example, there are lots of scandals now in the U.S. about celebrities promoting, you know, not just the fire festival, but promoting drugs online. And we have this like Kim Kardashian was actually warned by the Food and Drug Administration not to promote a warning sickness drug without at least acknowledging this is an ad, right? Because people should have some level more of skepticism about advertising than about organic content. Ultimately, these seven ideas that I put out here, these seven modes of regulation, they are something that I think would enable us to move a bit closer to a better modulated, more representative, more legitimate online public sphere. I'm not saying there's the solution to everything. Certainly there are many problems with the legacy media that I've not even touched on here to this point. I'm going to in the last third of the talk. But they at least give some sense that there can be democratic will formation by a polity that is coherent and that wants to set certain rules of the road with respect to how information is disseminated and consumed online. But I also want to think about what happens in public spheres that are what I call post-liberative. And I have on this slide two social theorists who I think are really, there's almost a rivalry between their thought and whose thought will most influence the 21st century in politics. And you're going to have a master in Carl Schmidt. And when we think about you're going to have a master's work over a lifetime, he has put forward an idea of an ideal speech situation and a public sphere based on rational will formation on the unforced force of the better argument. And many people say, well, this is a pretty idealistic notion, right? I mean, and it's ironic when we think about the intellectual history of German social theory that, you know, he himself was almost acting to be a more realistic voice in response to the Frankfurt school. But I think it's very important to have these as regulative ideals for how we enter into political debate and dialogue. The alternative, of course, is the crown jurist of the Third Reich, Carl Schmidt's concept of politics, which is the politics is essentially a battle between friend and enemy. And we have to try to win. And we really have nothing to learn from the other. The other is the enemy. And why is this social theory relevant? Well, I think it's relevant because we should do a thought experiment about any reform that say is based on the concept of a filter bubble. So let's imagine that, you know, we're really concerned about Eleperizer's concept of filter bubble, and we ensure that everyone on their news feed, all people who are identified as left see some right-wing stuff on their news feed, and all people who are identified as right see some left-wing stuff on their news feed. If everyone is committed to a relatively Habermasian ideal of deliberative democratic politics, maybe this is going to lead us to moderation, right? Maybe this will lead us to consensus, to societal harmony. However, if you had a, if we can just imagine a society where, say, liberals are, say, very open-minded, you have, say, liberals are on the left, very open-minded, and you have those on the right who are completely closed-minded, then if we iterate over time, we could imagine a scenario where there's some folks on the left, liberals who say, yeah, you know, these sort of neoliberal policies with respect to immigration, with respect to taxation, et cetera, maybe they make some sense. Maybe I'll move a little bit closer to the right. And then you might have a right that says, we will never move. We are here. If you have this type of scenario, it's pretty easy to imagine what happens iteratively over time, right? And I don't think this is just a thought experiment, really. I think it's actually something that's coming somewhat close to reality in the United States of the time, at the present time. And this is where I want to draw a distinction between, say, being concerned about algorithms, being concerned about, say, legacy media. In a lot of the United States, we get sort of media that is presented. So for example, in this slide, the jobless rate for four presidents is compared, and we're meant to assume that essentially Trump is one of the best of the four, and Obama is the worst, because after three months, look, Trump's fixed all the problems. It gets even worse, where you have essentially 100-day job creation figures, and you have, say, Trump again looking as the best person, the other's not. If this is the media diet that is, say, being fed to one side of the political spectrum, and part of the filter bubble is a commitment to ensure that more and more people see this media diet, that to me would probably be a perversion of the fairness doctrine, right? Or at least one would not necessarily want to sign up for it. So I think we really have to think hard in the future about exactly what is the public sphere we're dealing with. Another example that I think is relatively telling, at least from the U.S. context here, if you look, for example, on free trade, on the left-hand side, on this slide, there was a huge Republican shift on free trade. So at 2009, you had a relatively stable support for free trade. Once you had a candidate who was against it, suddenly against it. It's even more dramatic if you look at the Syria strikes, where you have, among Democrats, a relatively stable opposition, roughly, to getting more Middle East adventures for the U.S. Whereas you have, among the GOP, 22% supporting the Democratic president, 86% supporting the Republican president. So again, the problem here is one of, if your model of public sphere regulation is one that presumes rational liberation among large portions of the polity, and it seems as though there might be, say, portions of the polity that really don't want it, that are committed to a certain way of viewing the world, you've got to think very deeply about how that is either improving the public sphere or potentially not improving it, potentially deforming it, and playing into the hands of one side or the other. My bottom line for this talk is that the U.S. and free expression advocates really should not try to stop well-ordered societies from imposing reasonable consumer protection competition regulation. I think that the seven items that I went over earlier, that those could be quite helpful in assuring that places that have a relatively well-ordered and deliberative political system can maintain that, and how they could be used to nip in the bud certain forms of extremism or ways of undermining the political common understanding, the imaginary of politics, to use Charles Taylor's term, that undergirds robust democracies. What I'd also say is that after a certain amount of time, in some public spheres, there's essentially a loss of this common understanding. In those public spheres, it's relatively easy to see the regulatory bodies being captured or regulation itself being put to ends that it was never intended for. This sort of cultural sensitivity to whether we are in a Habermasian or Schmidian public sphere will be central to how committed we are to the seven types of reform of the automated public sphere that I discussed earlier. With that, I just wanted to thank you so much, and I really look forward to questions and dialogue from this group. Thank you. Thank you so much, Frank. You've left some time for questions, and I'm pretty sure there are going to be some. We have our ladies with the yellow t-shirts to come around and give the microphones to you. If you can give me and Frank a sign of hand if you'd like to raise a question or comment. There's one over there here. Hey, first of all, thank you very much. I enjoyed it very much. I have a question. You had this idea about information about information, like labeling the information. I was thinking why you said that, like, how would you or what kind of thoughts have you about that people trust this label? Because there's so much mistrust. There's so much, like, conspiracy theories that people really believe in what you talk about. Like, what is your idea about, like, establishing trust for these kind of senders that label? That's a fantastic question. And I think that the problem of achieving some sort of self-reflexivity, self-reflection about the public sphere is a really critical one. I know that there are some, certainly the organization factcheck.org was widely distressed by a lot of people who felt that it had an extremely narrow approach to public financing, for example. And so that is something where enabling many groups to be part of the dialogue should be a first step toward allowing more legitimate forms of labeling information online. I think this is going to be part of a consultative process that should be a cooperative endeavor, say, between governmental authorities and the intermediaries that are sort of would be the most, have the most effect here. I just think that it can't be left entirely up to the intermediaries themselves because if it is, then I certainly am distrusting. And I want to see more involvement by civil society. Now part of that involvement could involve, say, looking at the past record applications from the group, setting up a commission to sort of go through the applications to figure out who would be, say, who's done the most work in this area, the most credible work, to allow challenges. So for example, in the rulemaking process, administratively, you have comment periods and then you allow, sometimes there's an allowance for comments on the comments. So that can be part of the issue as well. But I also have to acknowledge that you don't want to create a situation where it's simply the person that is most doggedly interested that wins because that sometimes you see in Wikipedia edit wars. And so you have to be careful to sort of make sure that you're not setting up a system where it's simply about whoever expends the most effort. There has to be a somewhat more closed process, but a process that is open at the beginning but then also allows some contestation and some renewal and some circulation too. You don't want the same organizations doing it over, say, past five or 10 years or something. But all of those things I think have been addressed by, say, intergovernmental advisory panels or other panels. And all that I'm really focusing on here is trying to ensure that these methods are recognized that something that is like a state action is recognized as in the identity of what intermediaries are doing, that they are acting like governments, that they need to act like it. And one last example is my colleague Danielle Citron has an article called Technological Due Process, Pathbreaking Article, where she says that when they make these decisions, there should also be due process, a right of contestation, a right to be heard. And that's where I think the worry about algorithms and artificial intelligence I think really bites is because so often in tech firms it's seen that anything that involves a human is a defect that indicates a failure of code. And what we need to do is to reverse that and say, in fact, humans are essential to governance and involving them is a sign of the legitimacy of the process, not of its deviation from a platonic ideal of an entirely automated public sphere. Thanks. Oh, there are more questions. You're going to have to wave and make yourself very visible over there. Thanks. Right there. Hello. Here. Over here. Sorry. You already have, OK, then you first piece and you after that. Thank you. All right. I wanted to ask about, you had this little scenario that you sketched, kind of the hard right that has a clear position and the beautifully liberal liberals that are so open. And that would kind of shift the whole society to the right. Now, I'm kind of part of a mixed tribal marriage. My wife's family is conservative, evangelical, Kansas. My own family comes from the liberal East Coast. I hear from both sides, especially after the election, I hear so much ignorant stuff on both sides. And I just don't quite buy the kind of beautiful, idealic picture that you painted of the liberals in the United States. Oh, sure. No, I mean, I would say certainly you could model it the other way. My point is basically the same about the filter bubble, right? You could model it the exact opposite way and you could say, yeah, the society could go completely to the left. Maybe that's Venezuela. I don't know. I mean, that could be the society that goes where there's one hardcore. But I just sort of thought that was a convenient slide because it was sort of an image that I think is quite compelling in terms of rebudding the filter model, the filter bubble model. The bottom line is that democracy is fragile and that if you don't have a real commitment to openness by all the major entities in a political spectrum, you really have to worry about that. And I think that is very troubling. So that's just the point. The point is that you have to have a general, beneath political commitments. You have to have sort of a Habermasian commitment, I think, to actually listening to the other. Over there. Yeah, hi. Good afternoon. My name is Craig Fagan. I'm with the World Wide Web Foundation, which was set up to basically keep the web and open and safe space for everyone. And obviously, algorithms are key. As you use the term social media genocide, it's affecting how this vision is kept alive. But in a lot of your remarks, you focused on the US, which obviously is a good example. How have you seen this play out in other countries? Because one of the things that we're very concerned about is it's very much just as you were talking about Silicon Valley driving the agenda in the US. It's Silicon Valley driving the agenda in Lagos, in Delhi, in other countries where there isn't that type of the concerns you talk about the filter bubble is on a totally different scale. And there's also a thing which I think you alluded to, which is around algorithmic harm, the idea that there is a social harm being created by algorithms, so we need to do something about it, which is similar to me in the concept of where the environmental debate, where the drive of the agenda in Germany or US is not going to be the same level of what's happening, again, in Nigeria, India. So I wanted to know how you're kind of looking at this broader spectrum and to bring it back to how you see these two things playing out in the debate in other countries. Thank you. Thank you. So I would say that one of the key messages of, I think one of the first part of the talk, the ending was that I am very worried that the US-centered view of the world is overly influencing international bodies that might misinterpret a culturally specific US idea of what free expression or free trade is in order to eliminate the possibility for many other countries to develop culturally, nationally specific responses. And just to turn the question around a bit, I guess I would say that perhaps my focus on US examples performatively embodies that attitude by indirectly confessing that that's what I'm familiar with, right? I don't read all of the German papers each day. I don't read all of the British papers each day. I don't read the other countries papers each day, but I do look quite scopsistically and look quite closely at what the US experience is. And I think that what it shows is that in a country with very advanced penetration and adoption of some of the most advanced automated public sphere methods that you have a lot of opportunities for distortion. So all that my bottom line would be is I'm trying to essentially use US parochialism against itself. I'm trying to say essentially that don't necessarily trust what is being advanced by largely, interest largely identified with culturally specific ideas of what free expression or free trade is, be open to other sort of experiences. So yeah, that would be my response. There's one more question in the front row here. And one over there. Hello. Annette Mürweberg. I'm a member of the Trade Union Valde. And my question is if it would be helpful to open the discussion more up, not only talking about filter bubble, but talking about the necessity of transparency of algorithm in all the spheres of democratic decision making, but also about security. I think we have to have much more transparency of algorithms in all the basic infrastructures. It's extremely intransparent, unfortunately, and therefore it's a security issue. But also, for example, in the trade union, we discuss we do have talking about different laws in different nations. In Germany, we have co-decision making, and we need to have transparency of algorithms or at least criteria what's going on with the software, the functioning, the decision making processes of software to be able to co-decide what kind of technology can be introduced. So I think that they're in general for democratic procedures and safe infrastructures. We do need much more transparency, and it's part of the filter bubble and part of other issues too. Maybe if we discuss it as a general problem of democracy and security, maybe then we have a little bit more strength in pushing it. Thank you. Oh, I would completely reiterate that point. I think that in particular, in my Black Box Society book, I was really focused on the data brokers, the media sector, and the finance sector. And I thought particularly if you look at the crash of 2008, the global financial crisis, much of that was down to the extreme opacity of algorithms. But what's so important is that we have to emphasize that this opacity was a social construction. A lot of firms say, oh, it's just too complicated for people to understand. In fact, it's because of trade secrecy requirements, et cetera. And one thing I'm very concerned about in the European context is that the right of explanation that is guaranteed under the GDPR, that that will be essentially cut down to nothing thanks to trade secrecy assertion by large algorithmically driven entities. So we really have to watch trade secrecy assertion. I'd also say, and I have to preview my next book, which is on robotics, that in the areas of medical robotics, education robotics, law enforcement, and the military, there is extreme opacity about the guts of robotic systems. And if you think it's bad when it's opaque as to how we're getting our news feeds and how we're getting search engine results, imagine when it's a robot deciding whether to arrest you or not, or whether to how it's going to be teaching or how it's going to be diagnosing illness. So as these algorithms power more and more parts of our lives, we're going to have to really recommit ourselves to exactly the type of commitments that you articulated. So thank you. And let's take one more question from the lady that read that. That's OK with the phone. Thanks. Well, thank you very much. First of all, this was a very inspiring talk and I look forward to following this up with your book. I'm not quite sure whether I really can put my finger on my question precisely now, but I would like you to connect what you just said to the notion of democracy, because what struck me is that a lot of what you suggested is actually very top down. You were talking about regulation. You were talking about the old notion of gatekeepers to information, about labeling information. Now, the concept of democracy says that people actually participate from the bottom up. That seems for me to be a bit at odds. And one of the good things that I think that the Internet has brought about, and especially also in the context of the Trump election, is that we've been seeing a lot of grassroots journalistic outfits, startups, et cetera, pay. So how do you put this together? Sorry. I just throw this at you. Yeah? No, it's an excellent question. So this approach that I'm putting forward in response set one with those seven proposals for regulation or for even self-regulation, it was in some of them, it really is a plea for structure. And here's how I would root this. And it actually comes from a James Q. Wilson book about bureaucracy. He's a great administrative law or political scientist and used administrative law. And he compares ideal types of how political struggle is done, and he says that in, and again, this is very crude, but I think it's illuminating. He says that in Europe, many decision procedures seem like they follow something like the Marcus of Queensbury rules or it's like a boxing match. There's a fight. People can test it. There's an argument. There's a decision on a winner. And people sort of are going along with it generally. He compared that to the American political process, which he analogized to a barroom brawl. He said that the fight there never ends. There might be a gang that goes out of the bar and tries to get somebody else to come in and beat up their enemies, et cetera. And he said that's essentially what the administrative process in the U.S. reminded him of was this sort of complete free-for-all. And I think what my argument is that I think that 10 years ago, say, and when we were most influenced by, say, a wealth of networks type of utopianism about the potential for networks online, that we thought that this would bring everybody in and that that would sort of automatically conduce to democracy, to free markets, et cetera. It was a very Hayekian vision of spontaneous order. I think that over the past 10 years, what we've seen is that what's really happening is that there are large intermediaries that have enormous power. So first of all, it's not like regulation is intervening on or interfering with a process that is now an open-ended and grassroots one. It's really one where the process right now, the default, is one of extreme centralized corporate power. And secondly, that the sort of grassroots vision, grassroots are very easily replaced by astroturf, by fake grass, right? And so to even be supportive of a grassroots group, you have to have some concept of what's grassroots and what is, like the Twitter bots I showed, simply purchased time from very good coders or from others to manipulate the public sphere. And without that type of basic differentiation between authentic civil society institutions and fake ones that are essentially made up by manipulators, we're essentially discrediting all of civil society. It would be the same as if we allowed counterfeit money in circulation, right? Gresham's law says that counterfeit money, the bad money drives out the good. In many respects, we would have bad civil society entities driving out the good, the fake ones driving out authentic ones, et cetera. How do you make that distinction? That's difficult. But at the very least, entities should be extremely transparent about their funding, about how they were formed, about their governance structures, et cetera. There are unfortunately, in many contexts, there's lots of think tanks that say, hey, we're just a civil society group, we're just trying to represent the grassroots. And it's obvious where the funding is coming from, but it's never noted on, say, reports, news media, et cetera. So this sort of self-protection of civil society demands some forms of structure. And without it, we're going to see that type of bar room, raw model of politics that really ends up in exactly where the US is right now.
 Well, thank you so much, Geraldine, and thank you to the media convention.  It's fantastic to be here.  And I'm just going to dive right in because I have a lot to cover today.  And just to sort of put my talk into segments, the way I'm going to start is by discussing  really troubling applications of big data, algorithms, and artificial intelligence to  the public sphere, to politics and culture.  After going through some of these problems, I'm next going to talk about seven regulatory  approaches that could be deployed in order to address the problems caused.  And finally, I'm going to reflect on some situations where perhaps regulation is too  late or counterproductive.  So I think with that type of presentation, we'll have a balance and we'll try to understand  what can government do, can it do anything to respond to the evolving media landscape  that in many areas is very troubling.  I want to start with sort of four ways in which recent developments in the public sphere  over say the past five years or so have reversed the common utopian story about the nature  of media online.  And I would summarize this by saying that we've moved from a wealth of networks to a  black box society.  How did that happen?  Well, it first happened because we were told that, you know, the ability of almost anyone  to set up a website to say their own piece online would end big media domination and  would empower everyone.  But what we've also found is that it's enabled a great deal of fragmentation and extremism.  Secondly, we were told that the enabling of anonymity would give people an extraordinary  courage and freedom to speak their mind.  But what we're also seeing is that the most powerful plutocratic forces in societies can  set up shell corporations.  We saw that in the Panama Papers.  They can set up other entities that essentially allow their influence to go unchecked and  nearly unmonitored because it is so secretly funded and accomplished.  A third example of the utopian version of the public sphere via the Internet was the  idea that all would participate, all would have a right to speak.  But we've seen the flip side of that is a hashtag spam where, yes, if everyone has the  right to speak, then you certainly also have the right to say, flood the hashtag Black  Lives Matter with white supremacist propaganda.  And to the extent that this is permitted, it is an almost immediate effort to strangle  in the cradle the emancipatory potentials of the public sphere that we had all hoped  and dreamed of 10 years ago or so.  And perhaps even to make it even more topical the last week or so, our original conception  of WikiLeaks was this wonderful tool to discredit the surveillance state.  Now it seems as though it's sort of really bent on discrediting anti-fascist candidates  in elections.  And so when we see this type of transformation, we have to worry, and I'm going to go into  some particular examples of this.  The proliferation of Twitter bots, I think, is not merely a reflection of technical evolution  that people like Tim Hwang were writing about years ago.  It's also a reflection of a commitment to what I call free expression fundamentalism,  but the idea is that, say, if anybody can code up a program that wants to speak or tweet  thousands of times online, that that is a natural extension of their right to speak.  I think we really have to combat that because we can't just allow, say, the romantic appeal  of Isaac Asimov's talking robots to influence our idea of how software and automated methods  of speaking can influence the public sphere because, you know, I can never keep up with  a bot that can tweet a thousand times an hour, neither can anyone here.  And the logical endpoint of such a completely unregulated, unmediated public sphere is one  where those who are the wealthiest with the best coders are able to drown out everyone  else.  I'd also say that fake content on Facebook is a huge fundamental challenge to the idea  of a marketplace of ideas ultimately enabling good, true speech to drive out the bad speech.  In fact, what we're seeing is dark ads and personalization are directly targeting certain  types of content to those who are most susceptible to it, and those who would dispute it will  never see this content because it's in the dark.  It's one of the themes of my book, The Black Box Society, that no one will be able to see  this and counteract the speech because it's simply been so targeted on such a personal  level.  And what's really scary, I believe, about this is that we have a problem of human subject  experimentation in many social media.  We have constant efforts at A-B testing to see what works, what doesn't, and this can  just as easily be a way of accelerating propaganda as it is a way of accelerating truth.  Now a lot of sophisticated observers will ask you, why does this matter?  Who believes the fake news?  Who actually believes that, you know, Hillary Clinton had five people killed during the  election 2016, et cetera.  Who would believe these other lies?  Well, that does make some sense for a lot of the folks that run in, say, highly sophisticated,  highly educated circles.  But we also have to worry a great deal about floating voters, low information voters, voters  that are marginally attached to the political process, being particularly susceptible to  a lot of efforts at propaganda and online manipulation.  And when we consider the possibility of low information voters being the key ones who  decide an election, and I think you can think of this in your own experience, I mean, most  people one knows, they know exactly what their political commitments are, but there are this  sort of core of low information voters that could be really disproportionately influenced.  I'd also say that, you know, the promise of Facebook was that of a unified national,  supernational, even global public sphere where we could all talk and debate, and that still  is part, say, of the 6,000 word Zuckerberg manifesto with respect to Facebook.  But the problem is that a sort of radical egalitarianism as to source can lead to a  total lack of educative effect or branding as to the validity of the content.  So when you have a situation where automation homogenizes the presentation of information  so that everything looks the same, something could be from the Denver Guardian, so-called,  that was just cooked up by a couple of people in a few days, or a site like that was just  cooked up in this way, could appear with the same level of authority as something, say,  that was from a prize winning newspaper investigative team.  And this again, this idea of sort of debased egalitarianism, a way of sort of making everything  look the same, is very troubling.  I also think that one of the biggest problems here is that a lot of the people that run  the biggest online platforms and intermediaries, Google, Apple, Facebook, Amazon, Microsoft,  et cetera, they come from the US.  And the culture of the US really has culminated, at least with respect to free expression,  in something that I call First Amendment fundamentalism.  The idea behind this is, it's in a book called, Cording the Abyss, where the strange value  system is that the worse the speech is, the more misleading it is, the prouder we are to  allow it, because it shows how radically committed we are to let everybody say whatever they  want.  The culmination of this is in things like, there's a case called USV Alvarez, called  the Stolen Valor case, where the Supreme Court says, the government doesn't even have a right  to make it illegal for people to lie about whether they had military honors.  There's another case from the Washington State Supreme Court that essentially guarantees  political candidates a right to lie.  It's really just that simple, that we're just going to allow the candidates to lie.  And when you have a public sphere that has this type of cultural legal attitude toward  free expression, of course you have viral stories that say lies about that there are  3 million illegal votes or what have you.  We also have to look at the material foundations, not just the legal foundations of this automated  public sphere.  And I think these are really brilliantly investigated by Jodi Dean in her book, Blog Theory, where  she describes the circuits of drive and the profit that is made by simply outraging people.  Personal opinion formation and deliberation is not a profit center, rapid sharing and  engagement is.  And that's one reason, for example, why even dataist interventions like Ben Grocer's effort  to randomly automate your reactions to Facebook posts, why these things are so popular.  Because people have a sense that it's sort of an out of control public sphere that really  is very hard to understand where it's going or what one can do to even influence it in  a positive way.  Public responses have been various to this problem.  So we have a situation where you have a lot of people that are trying to figure out is  there a role for regulators, is there a role for consumer protection authorities, for media  authorities, for other entities.  Unfortunately though, there's another aspect of US American centrism that I think is overly  influencing the internet debate, which is a rhetorical commitment to something called  one unified open internet or don't break the internet.  And so often what happens is that when there are efforts at rational public interest regulation  of internet media, a lot of those at trade authorities from the US and other entities  say if you allow different rules in say Germany or France or Argentina or Japan than US centered  rules, you will break the internet.  And this I think is something to be resisted because we need culturally and nationally  specific responses to some of the problems that I've just gone through.  The problem also is that we have an attitude from the US Federal Communications Commission  pioneered in the 1980s that television and now the internet under Ajit Pai, the current  FCC chair, is little more than a toaster with pictures.  And if we trust the market to give us the best toasters, we can trust it to give us  the best internet, the best program online.  Of course even in the case of toasters, that's not true.  There are product safety commissions that actually try to ensure that your toaster won't  explode in your face when you put bread in it.  But nevertheless, this is one of the problems in which we have many, much rhetoric from  the US saying just don't regulate, it'll all work out in the end.  But this is very disingenuous because essentially deregulation is a lie.  Once you deregulate by the state in a given area, you essentially see the power to massive  corporations to be the de facto regulators.  And in our case, especially the problems that we're discussing at this conference, this  convention, Facebook and Google effectively are the regulators, but they're acting in  regulatory ways without the normal modes of accountability, such as due process, public  comment, public responsiveness that are the hallmark of a legitimate regulatory authority.  And so as I go through seven different regulatory responses, part of the inspiration for this  is trying to figure out a way in which the de facto regulatory power of online entities  could be translated into something that is more legitimate and more responsive to public  ideals.  This is in a nutshell the seven items that I'll be going over over the next 15 minutes,  which are to label and monitor and explain heat-driven search results, to ban certain  particularly dangerous content, to audit the logs of data-fed and algorithmic systems,  to allow some limited outside annotations to limit the predation possible by online intermediaries.  Because I think that's a huge part of the story here is that as you see so much money,  power and attention being sucked up like a vortex centripetally into these online intermediaries,  there's so much less to go to the standard or our media forces.  The right to be forgotten as one case study in this area of algorithmic accountability  and finally a shout out for media literacy.  So one example here is labeling and monitoring biased certain results.  And an example that I'm sort of putting on this here on the slide is when involving the  rise of advocacy by the outright to promote a revisionist account of the Holocaust and  of history up to World War II.  And what we see in the situation is that people searching on the term Hitler on a Holocaust  on the 1930s in Germany, etc., all of these types of searches are leading to extreme prominence  for renewed right wing results and or far right Nazi results.  And this was revealed in a piece by Carol Cadwalad.  A few weeks ago she did some empirical research, some anecdotal studies or stories of here.  What's kind of ironic here is that we've actually moved backwards from where we were in 2004.  I started writing about search engines and their arrangement of information back in  about 2003 or so.  And when I was writing some of my first articles like rankings, reductionism and responsibility,  I found this example of Jew watch which was a Holocaust denial site that was in the top  ten results for the query Jew.  At the time what Google did is they were extremely worried by this.  The anti-deformation league complained to them and they had an asterisk next to it.  And at the bottom of the page they said we are very concerned about this result.  We believe it's been manipulated but just know that we can't really change it but we  would like you to bear in mind the potentially illegitimate origins of this result.  What Cadwalad's reporting revealed in the past year or so is that there was backsliding  from that commitment by the company.  And it's rather ironic, right?  Because if anything Google is ten times more powerful and more rich than it was in 2004,  it had a backslid.  Another problem that I think is really important and what demands some responses is autocomplete  bias.  So we have a problem where for example autocompletes about certain individuals are leading to clearly  biased results.  Now here's one because it's a public figure, I am less inclined to endorse any sort of  intervention.  But certainly when there are private figures who are having autocompletes going after them,  that's clearly troubling.  And secondly when we have whole racial groups having autocompletes that are derogatory, that  are troubling, that are deriding them, that is something we have to address.  And I'm really looking forward to this forthcoming book by Safiyah Noble who has explored in  great detail exactly what is going on in many of these algorithmic biases that show up in  autocompletes and has proposed some concrete ways in which large intermediaries can try  to intervene to assure that there's not this kind of bias and influence.  And certainly even governments that have anti-hate laws should seriously consider how far this  could go, how far down the road we could go.  I say this in particular because one of the most troubling terrorist incidents in the  United States over the past 10 years or so was this man, Dylan Roof, went to a South  Carolina predominantly African-American church and killed over 10 people.  When asked why he did it, he said he had Googled black on white crime and saw all of these  results that sort of led him to say that there was an epidemic, a race war happening, where  essentially there was a white genocide possible because of all of the black on white crime.  Of course these things are entirely spurious, they're all hate driven.  And so the idea is how as society is committed to pluralism, to tolerance, and to diversity,  we try to ensure that what we have committed to formally, what we commit to in education  is showing up in our online media because to be frank, that's much more important to  many people than many of the standard media-educated outlets.  That's how people really are connected to each other and connecting to the web.  Now another thing that I think is so important here is audit logs of the data fed into algorithmic  systems.  And I want to explain the significance of this with reference to a scandal that happened  in the United States, or this bizarre situation, a story called Pizza Gate, where essentially  there were rumors based on, ostensibly based on some of the WikiLeaks releases that there  was a child sex ring at a popular pizzeria in Washington that was connected to the Democratic  establishment.  Again, entirely false.  But one, it took some time to get to the bottom of it because the people who spread this  news were savvy about sort of how they would disseminate it.  If you had better audit logs so you could better understand exactly where certain stories  that are quite suspect originate, this one originated in a white supremacist tweet, this  type of public regulatory understanding of the black box of these algorithmic systems  could be crucial to rapid responses.  Responses that happened before say in this example, a gunman goes to the site and starts  firing shots and says, I just want to see what's really going on here.  This is where audit logs are really helpful.  And they're also really helpful because we hear a lot from Silicon Valley lobbyists and  other tech advocates that the current modes of ordering information on search engines  and social networks are so complicated, lock and never catch up.  There was actually one machine learning expert that was quoted in a recent story that said,  trying to explain how Google or other sites arrange information for you, for me as a computer  scientist to explain that to you as a layman would be like trying to explain Shakespeare  to a dog.  And I think it's very important that when we hear these metaphors, we push back immediately.  And first we say, the story about Pizzagate, that's not Shakespeare.  And then we secondly say, you cannot adopt the sort of condescending attitude that the  computational sphere is itself automatically superior to modes of human self-interpretation  and human understanding.  But that is precisely the core normative value in most of the deregulatory efforts here.  And one small pushback against this is to say, even if we can't necessarily understand  or explain in a humanly relatable way how machine learning works on thousands of variables  with thousands of parameters, we can at least have logs of data that are influencing certain  results that we see on Facebook and Google, and we could try to isolate and try to find  suspect sources of information.  Those suspect sources of information could, for example, be part of the annotations on  certain posts.  So if you recall the Jew watch example I gave from the beginning about how white supremacists  were gaming the search engine to lead to anti-semitic results, that could be a much faster process  if we have data from audit logs that allow us to identify, say, suspect efforts to manipulate  news feeds, to manipulate search engine results, and then immediately or very close to immediately  allow annotation of those results.  And I think this is a very important step forward in how we regulate the economy.  Because we have already committed to the idea that we need to have labeling of, say, food  and drugs.  So we understand, say, how many calories, how much fat is in food.  What we need to take as a second step in an information economy is we not only need to  have information about goods and services, we need to label, we have to have information  about the information we get.  We have to be able to understand that there might be certain suspect or troubling influences  on the data we're getting that helps us to figure out what shows we're going to watch,  what news we're going to trust, what feed we're going to follow, et cetera.  And that's a premise of Mark Patterson's book on antitrust law in the new economy.  It's an effort to sort of step to a second level of awareness and to break out of our  self-imposed tutelage about black boxes, to break out, to really enlighten the public  sphere in these scenarios.  I also think that part of this is inevitably going to involve restoring human editors.  And when I say restoring human editors, I don't mean doing this in the classic Facebook  style of creating a cast of untouchable journalists who are out in a trailer somewhere that are  contract workers who can be immediately removed or vetoed by anyone who's an engineer in the  company.  We have to realize that this sort of caste system, very explicitly described in Kate  Lossy's book about Facebook, is a reality about the way in which a lot of entities in  Silicon Valley see the relationship between journalists and engineers.  What we instead need is we need to really make sure that there is this robust and substantive  commitment to respecting journalism as a profession, not merely a source of piecework, propaganda  or PR, and to ensure that human editors can, for example, look at breaking news story and  just knock out ones that clearly have no basis in reality.  Google's fact-checking initiative is one small step in this direction.  They have partnered with entities like Snopes.com and factcheck.org.  They're trying to ensure that there are some rigid parameters to qualify those who serve  as fact-checkers, to have some level of accountability and responsibility there.  But I do think there are some issues.  One is sort of a interface problem, so one problem here might be that even though Snopes.com  is debunking the story that Pope Francis endorsed Donald Trump for president, we nevertheless  have the headline is the Pope endorses Donald Trump for president.  We have to be sensitive to a world of, as Hartmut Rosik characterizes it, a world of  acceleration and social acceleration, that we have the ability of entities to respond  to the possibility that people may only be reading the headlines.  So maybe even headlines have to be altered to respond to the real activity of media reception  by new audiences, fast audiences.  Same with the story about Clinton campaign chairman involved in satanic spirit cooking.  Again, false story.  Let's perhaps not require people to click to the second page.  Let's have a better sense of what accurate headline would look like.  And I also want to say that the groups involved, first of all, I question, are they being adequately  supported by the larger intermediaries?  The second question I have is, have we really opened it up to all potential groups, right?  Or do we have a very narrow concept of what fact checking could entail?  And I think that this type of civil society initiative, trying to recognize its quasi-governmental  nature, trying to recognize that there are questions of legitimacy of representation  involved is very important and trying to be open about that, right?  And I think that even to the Google response to the right to be forgotten decision, there  was not quite enough pluralism in terms of how they established a board to try to respond  to that.  There could be more.  In going further, what I want to say is that we have to, when I talk about funding these  other entities, part of that funding probably has to come from intermediaries themselves.  And we had a proposal back in 2006 or so by Harvard professor Terry Fisher to assure that  content was getting some share of the profits that were being made from the growing connection  and internet connections of other people around the world.  And he would actually have a proposal that would allow lots of access to content in response  in after having attacks on internet connections.  Now of course, I think that is far too regressive attacks.  I would make it a much more progressive attacks.  I think that is more than line the vision of someone like Jerry Lanier, but he also  has some other ideas about how to make sure that content is actually compensated.  But part of the idea here is that we have to recognize that as more and more revenue  goes to the intermediary, less and less to the source that actually is enabling the intermediary  to be something of value, we have to respond to that.  Another example I'll give number six of my examples is the right to be forgotten.  And I think that the example here is one where I know it's controversial.  I know there are certainly concerns in the press about potential.  Censorship or collateral censorship would be a more accurate term.  But I think that when we look at the demands of individuals say, and when you have a woman  who says, whenever people search my name, the first result they see is the fact that  my husband was murdered 20 years ago.  I don't want that to be the first result everyone sees about my name for the rest of my life  into perpetuity.  It seems that a humane compassionate response to that really is for the intermediary to help  that person say obscure the story.  At least when it's a search on her name.  It reminds me of a lot of credit report or a dossier, private investigator's report  on somebody.  They really need the ability to respond to that.  And this I think is a first step toward assuring that we are not just sort of drifting into  an artificially intelligent future, but we're drifting into a wiser future where there's  some artificial wisdom that's informed by a compassionate response and a requirement  under law that intermediaries be somewhat responsive to individual concerns.  Can it be abused, of course?  But the answer to this abuse I think is better, more legitimate operationalization rather  than ending the right in itself.  And I think that this is part of, you know, it fits in together with some of the ideas  that I've already been talking about with respect to obscuring certain content, banning  certain content, because it really is quite possible that we are on the cusp of a social  media genocide.  You know, if you have the ability of individuals to spread hatred, to spread damaging stereotypes  or fake news about say a minority group in the community, we could be very close to a  scenario where that group in itself could have accelerated hatred directed against it.  And we need to have mechanisms in place to try to at least retard that or slow down the  acceleration of damaging content.  A final example that I would give, and this is steps back from that sort of disaster scenario  that I was thinking about in the earlier slide, is educating users in media literacy.  One of the concrete examples here that I think is most compelling is that there are lots  of people that do not understand the difference between a sponsored story on Facebook and an  organic content.  They don't understand the difference between sponsored ads on Google and real content or  content that is ostensibly organic.  And just to have these basic levels of media literacy taught maybe in fourth, fifth, sixth  grade, so the individuals know what are they being paid to see, what is something that  is arising out of a less commercial process, although certainly a commercial process.  This type of media literacy is really crucial.  And it's also crucial even in the smallest settings.  So for example, there are lots of scandals now in the U.S. about celebrities promoting,  you know, not just the fire festival, but promoting drugs online.  And we have this like Kim Kardashian was actually warned by the Food and Drug Administration  not to promote a warning sickness drug without at least acknowledging this is an ad, right?  Because people should have some level more of skepticism about advertising than about  organic content.  Ultimately, these seven ideas that I put out here, these seven modes of regulation, they  are something that I think would enable us to move a bit closer to a better modulated,  more representative, more legitimate online public sphere.  I'm not saying there's the solution to everything.  Certainly there are many problems with the legacy media that I've not even touched on  here to this point.  I'm going to in the last third of the talk.  But they at least give some sense that there can be democratic will formation by a polity  that is coherent and that wants to set certain rules of the road with respect to how information  is disseminated and consumed online.  But I also want to think about what happens in public spheres that are what I call post-liberative.  And I have on this slide two social theorists who I think are really, there's almost a rivalry  between their thought and whose thought will most influence the 21st century in politics.  And you're going to have a master in Carl Schmidt.  And when we think about you're going to have a master's work over a lifetime, he has put  forward an idea of an ideal speech situation and a public sphere based on rational will  formation on the unforced force of the better argument.  And many people say, well, this is a pretty idealistic notion, right?  I mean, and it's ironic when we think about the intellectual history of German social  theory that, you know, he himself was almost acting to be a more realistic voice in response  to the Frankfurt school.  But I think it's very important to have these as regulative ideals for how we enter into  political debate and dialogue.  The alternative, of course, is the crown jurist of the Third Reich, Carl Schmidt's concept  of politics, which is the politics is essentially a battle between friend and enemy.  And we have to try to win.  And we really have nothing to learn from the other.  The other is the enemy.  And why is this social theory relevant?  Well, I think it's relevant because we should do a thought experiment about any reform that  say is based on the concept of a filter bubble.  So let's imagine that, you know, we're really concerned about Eleperizer's concept of filter  bubble, and we ensure that everyone on their news feed, all people who are identified as  left see some right-wing stuff on their news feed, and all people who are identified as  right see some left-wing stuff on their news feed.  If everyone is committed to a relatively Habermasian ideal of deliberative democratic politics,  maybe this is going to lead us to moderation, right?  Maybe this will lead us to consensus, to societal harmony.  However, if you had a, if we can just imagine a society where, say, liberals are, say, very  open-minded, you have, say, liberals are on the left, very open-minded, and you have those  on the right who are completely closed-minded, then if we iterate over time, we could imagine  a scenario where there's some folks on the left, liberals who say, yeah, you know, these  sort of neoliberal policies with respect to immigration, with respect to taxation, et  cetera, maybe they make some sense.  Maybe I'll move a little bit closer to the right.  And then you might have a right that says, we will never move.  We are here.  If you have this type of scenario, it's pretty easy to imagine what happens iteratively over  time, right?  And I don't think this is just a thought experiment, really.  I think it's actually something that's coming somewhat close to reality in the United States  of the time, at the present time.  And this is where I want to draw a distinction between, say, being concerned about algorithms,  being concerned about, say, legacy media.  In a lot of the United States, we get sort of media that is presented.  So for example, in this slide, the jobless rate for four presidents is compared, and  we're meant to assume that essentially Trump is one of the best of the four, and Obama  is the worst, because after three months, look, Trump's fixed all the problems.  It gets even worse, where you have essentially 100-day job creation figures, and you have,  say, Trump again looking as the best person, the other's not.  If this is the media diet that is, say, being fed to one side of the political spectrum,  and part of the filter bubble is a commitment to ensure that more and more people see this  media diet, that to me would probably be a perversion of the fairness doctrine, right?  Or at least one would not necessarily want to sign up for it.  So I think we really have to think hard in the future about exactly what is the public  sphere we're dealing with.  Another example that I think is relatively telling, at least from the U.S. context here,  if you look, for example, on free trade, on the left-hand side, on this slide, there was  a huge Republican shift on free trade.  So at 2009, you had a relatively stable support for free trade.  Once you had a candidate who was against it, suddenly against it.  It's even more dramatic if you look at the Syria strikes, where you have, among Democrats,  a relatively stable opposition, roughly, to getting more Middle East adventures for the  U.S.  Whereas you have, among the GOP, 22% supporting the Democratic president, 86% supporting the  Republican president.  So again, the problem here is one of, if your model of public sphere regulation is one that  presumes rational liberation among large portions of the polity, and it seems as though there  might be, say, portions of the polity that really don't want it, that are committed to  a certain way of viewing the world, you've got to think very deeply about how that is  either improving the public sphere or potentially not improving it, potentially deforming it,  and playing into the hands of one side or the other.  My bottom line for this talk is that the U.S. and free expression advocates really should  not try to stop well-ordered societies from imposing reasonable consumer protection competition  regulation.  I think that the seven items that I went over earlier, that those could be quite helpful  in assuring that places that have a relatively well-ordered and deliberative political system  can maintain that, and how they could be used to nip in the bud certain forms of extremism  or ways of undermining the political common understanding, the imaginary of politics,  to use Charles Taylor's term, that undergirds robust democracies.  What I'd also say is that after a certain amount of time, in some public spheres, there's  essentially a loss of this common understanding.  In those public spheres, it's relatively easy to see the regulatory bodies being captured  or regulation itself being put to ends that it was never intended for.  This sort of cultural sensitivity to whether we are in a Habermasian or Schmidian public  sphere will be central to how committed we are to the seven types of reform of the automated  public sphere that I discussed earlier.  With that, I just wanted to thank you so much, and I really look forward to questions and  dialogue from this group.  Thank you.  Thank you so much, Frank.  You've left some time for questions, and I'm pretty sure there are going to be some.  We have our ladies with the yellow t-shirts to come around and give the microphones to  you.  If you can give me and Frank a sign of hand if you'd like to raise a question or comment.  There's one over there here.  Hey, first of all, thank you very much.  I enjoyed it very much.  I have a question.  You had this idea about information about information, like labeling the information.  I was thinking why you said that, like, how would you or what kind of thoughts have you  about that people trust this label?  Because there's so much mistrust.  There's so much, like, conspiracy theories that people really believe in what you talk  about.  Like, what is your idea about, like, establishing trust for these kind of senders that label?  That's a fantastic question.  And I think that the problem of achieving some sort of self-reflexivity, self-reflection  about the public sphere is a really critical one.  I know that there are some, certainly the organization factcheck.org was widely distressed  by a lot of people who felt that it had an extremely narrow approach to public financing,  for example.  And so that is something where enabling many groups to be part of the dialogue should be  a first step toward allowing more legitimate forms of labeling information online.  I think this is going to be part of a consultative process that should be a cooperative endeavor,  say, between governmental authorities and the intermediaries that are sort of would  be the most, have the most effect here.  I just think that it can't be left entirely up to the intermediaries themselves because  if it is, then I certainly am distrusting.  And I want to see more involvement by civil society.  Now part of that involvement could involve, say, looking at the past record applications  from the group, setting up a commission to sort of go through the applications to figure  out who would be, say, who's done the most work in this area, the most credible work,  to allow challenges.  So for example, in the rulemaking process, administratively, you have comment periods  and then you allow, sometimes there's an allowance for comments on the comments.  So that can be part of the issue as well.  But I also have to acknowledge that you don't want to create a situation where it's simply  the person that is most doggedly interested that wins because that sometimes you see in  Wikipedia edit wars.  And so you have to be careful to sort of make sure that you're not setting up a system where  it's simply about whoever expends the most effort.  There has to be a somewhat more closed process, but a process that is open at the beginning  but then also allows some contestation and some renewal and some circulation too.  You don't want the same organizations doing it over, say, past five or 10 years or something.  But all of those things I think have been addressed by, say, intergovernmental advisory  panels or other panels.  And all that I'm really focusing on here is trying to ensure that these methods are recognized  that something that is like a state action is recognized as in the identity of what intermediaries  are doing, that they are acting like governments, that they need to act like it.  And one last example is my colleague Danielle Citron has an article called Technological  Due Process, Pathbreaking Article, where she says that when they make these decisions,  there should also be due process, a right of contestation, a right to be heard.  And that's where I think the worry about algorithms and artificial intelligence I think really  bites is because so often in tech firms it's seen that anything that involves a human is  a defect that indicates a failure of code.  And what we need to do is to reverse that and say, in fact, humans are essential to  governance and involving them is a sign of the legitimacy of the process, not of its  deviation from a platonic ideal of an entirely automated public sphere.  Thanks.  Oh, there are more questions.  You're going to have to wave and make yourself very visible over there.  Thanks.  Right there.  Hello.  Here.  Over here.  Sorry.  You already have, OK, then you first piece and you after that.  Thank you.  All right.  I wanted to ask about, you had this little scenario that you sketched, kind of the hard  right that has a clear position and the beautifully liberal liberals that are so open.  And that would kind of shift the whole society to the right.  Now, I'm kind of part of a mixed tribal marriage.  My wife's family is conservative, evangelical, Kansas.  My own family comes from the liberal East Coast.  I hear from both sides, especially after the election, I hear so much ignorant stuff on  both sides.  And I just don't quite buy the kind of beautiful, idealic picture that you painted of the liberals  in the United States.  Oh, sure.  No, I mean, I would say certainly you could model it the other way.  My point is basically the same about the filter bubble, right?  You could model it the exact opposite way and you could say, yeah, the society could  go completely to the left.  Maybe that's Venezuela.  I don't know.  I mean, that could be the society that goes where there's one hardcore.  But I just sort of thought that was a convenient slide because it was sort of an image that  I think is quite compelling in terms of rebudding the filter model, the filter bubble model.  The bottom line is that democracy is fragile and that if you don't have a real commitment  to openness by all the major entities in a political spectrum, you really have to worry  about that.  And I think that is very troubling.  So that's just the point.  The point is that you have to have a general, beneath political commitments.  You have to have sort of a Habermasian commitment, I think, to actually listening to the other.  Over there.  Yeah, hi.  Good afternoon.  My name is Craig Fagan.  I'm with the World Wide Web Foundation, which was set up to basically keep the web and open  and safe space for everyone.  And obviously, algorithms are key.  As you use the term social media genocide, it's affecting how this vision is kept alive.  But in a lot of your remarks, you focused on the US, which obviously is a good example.  How have you seen this play out in other countries?  Because one of the things that we're very concerned about is it's very much just as  you were talking about Silicon Valley driving the agenda in the US.  It's Silicon Valley driving the agenda in Lagos, in Delhi, in other countries where  there isn't that type of the concerns you talk about the filter bubble is on a totally  different scale.  And there's also a thing which I think you alluded to, which is around algorithmic harm,  the idea that there is a social harm being created by algorithms, so we need to do something  about it, which is similar to me in the concept of where the environmental debate, where the  drive of the agenda in Germany or US is not going to be the same level of what's happening,  again, in Nigeria, India.  So I wanted to know how you're kind of looking at this broader spectrum and to bring it back  to how you see these two things playing out in the debate in other countries.  Thank you.  Thank you.  So I would say that one of the key messages of, I think one of the first part of the talk,  the ending was that I am very worried that the US-centered view of the world is overly  influencing international bodies that might misinterpret a culturally specific US idea  of what free expression or free trade is in order to eliminate the possibility for many  other countries to develop culturally, nationally specific responses.  And just to turn the question around a bit, I guess I would say that perhaps my focus  on US examples performatively embodies that attitude by indirectly confessing that that's  what I'm familiar with, right?  I don't read all of the German papers each day.  I don't read all of the British papers each day.  I don't read the other countries papers each day, but I do look quite scopsistically and  look quite closely at what the US experience is.  And I think that what it shows is that in a country with very advanced penetration and  adoption of some of the most advanced automated public sphere methods that you have a lot  of opportunities for distortion.  So all that my bottom line would be is I'm trying to essentially use US parochialism  against itself.  I'm trying to say essentially that don't necessarily trust what is being advanced by largely,  interest largely identified with culturally specific ideas of what free expression or  free trade is, be open to other sort of experiences.  So yeah, that would be my response.  There's one more question in the front row here.  And one over there.  Hello.  Annette Mürweberg.  I'm a member of the Trade Union Valde.  And my question is if it would be helpful to open the discussion more up, not only talking  about filter bubble, but talking about the necessity of transparency of algorithm in  all the spheres of democratic decision making, but also about security.  I think we have to have much more transparency of algorithms in all the basic infrastructures.  It's extremely intransparent, unfortunately, and therefore it's a security issue.  But also, for example, in the trade union, we discuss we do have talking about different  laws in different nations.  In Germany, we have co-decision making, and we need to have transparency of algorithms  or at least criteria what's going on with the software, the functioning, the decision  making processes of software to be able to co-decide what kind of technology can be introduced.  So I think that they're in general for democratic procedures and safe infrastructures.  We do need much more transparency, and it's part of the filter bubble and part of other  issues too.  Maybe if we discuss it as a general problem of democracy and security, maybe then we  have a little bit more strength in pushing it.  Thank you.  Oh, I would completely reiterate that point.  I think that in particular, in my Black Box Society book, I was really focused on the  data brokers, the media sector, and the finance sector.  And I thought particularly if you look at the crash of 2008, the global financial crisis,  much of that was down to the extreme opacity of algorithms.  But what's so important is that we have to emphasize that this opacity was a social construction.  A lot of firms say, oh, it's just too complicated for people to understand.  In fact, it's because of trade secrecy requirements, et cetera.  And one thing I'm very concerned about in the European context is that the right of  explanation that is guaranteed under the GDPR, that that will be essentially cut down to  nothing thanks to trade secrecy assertion by large algorithmically driven entities.  So we really have to watch trade secrecy assertion.  I'd also say, and I have to preview my next book, which is on robotics, that in the areas  of medical robotics, education robotics, law enforcement, and the military, there is extreme  opacity about the guts of robotic systems.  And if you think it's bad when it's opaque as to how we're getting our news feeds and  how we're getting search engine results, imagine when it's a robot deciding whether to arrest  you or not, or whether to how it's going to be teaching or how it's going to be diagnosing  illness.  So as these algorithms power more and more parts of our lives, we're going to have to  really recommit ourselves to exactly the type of commitments that you articulated.  So thank you.  And let's take one more question from the lady that read that.  That's OK with the phone.  Thanks.  Well, thank you very much.  First of all, this was a very inspiring talk and I look forward to following this up with  your book.  I'm not quite sure whether I really can put my finger on my question precisely now, but  I would like you to connect what you just said to the notion of democracy, because what  struck me is that a lot of what you suggested is actually very top down.  You were talking about regulation.  You were talking about the old notion of gatekeepers to information, about labeling  information.  Now, the concept of democracy says that people actually participate from the bottom up.  That seems for me to be a bit at odds.  And one of the good things that I think that the Internet has brought about, and especially  also in the context of the Trump election, is that we've been seeing a lot of grassroots  journalistic outfits, startups, et cetera, pay.  So how do you put this together?  Sorry.  I just throw this at you.  Yeah?  No, it's an excellent question.  So this approach that I'm putting forward in response set one with those seven proposals  for regulation or for even self-regulation, it was in some of them, it really is a plea  for structure.  And here's how I would root this.  And it actually comes from a James Q. Wilson book about bureaucracy.  He's a great administrative law or political scientist and used administrative law.  And he compares ideal types of how political struggle is done, and he says that in, and  again, this is very crude, but I think it's illuminating.  He says that in Europe, many decision procedures seem like they follow something like the Marcus  of Queensbury rules or it's like a boxing match.  There's a fight.  People can test it.  There's an argument.  There's a decision on a winner.  And people sort of are going along with it generally.  He compared that to the American political process, which he analogized to a barroom  brawl.  He said that the fight there never ends.  There might be a gang that goes out of the bar and tries to get somebody else to come  in and beat up their enemies, et cetera.  And he said that's essentially what the administrative process in the U.S. reminded him of was this  sort of complete free-for-all.  And I think what my argument is that I think that 10 years ago, say, and when we were most  influenced by, say, a wealth of networks type of utopianism about the potential for networks  online, that we thought that this would bring everybody in and that that would sort of automatically  conduce to democracy, to free markets, et cetera.  It was a very Hayekian vision of spontaneous order.  I think that over the past 10 years, what we've seen is that what's really happening  is that there are large intermediaries that have enormous power.  So first of all, it's not like regulation is intervening on or interfering with a process  that is now an open-ended and grassroots one.  It's really one where the process right now, the default, is one of extreme centralized  corporate power.  And secondly, that the sort of grassroots vision, grassroots are very easily replaced  by astroturf, by fake grass, right?  And so to even be supportive of a grassroots group, you have to have some concept of what's  grassroots and what is, like the Twitter bots I showed, simply purchased time from very  good coders or from others to manipulate the public sphere.  And without that type of basic differentiation between authentic civil society institutions  and fake ones that are essentially made up by manipulators, we're essentially discrediting  all of civil society.  It would be the same as if we allowed counterfeit money in circulation, right?  Gresham's law says that counterfeit money, the bad money drives out the good.  In many respects, we would have bad civil society entities driving out the good, the fake  ones driving out authentic ones, et cetera.  How do you make that distinction?  That's difficult.  But at the very least, entities should be extremely transparent about their funding,  about how they were formed, about their governance structures, et cetera.  There are unfortunately, in many contexts, there's lots of think tanks that say, hey,  we're just a civil society group, we're just trying to represent the grassroots.  And it's obvious where the funding is coming from, but it's never noted on, say, reports,  news media, et cetera.  So this sort of self-protection of civil society demands some forms of structure.  And without it, we're going to see that type of bar room, raw model of politics that really  ends up in exactly where the US is right now.
re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_0_167.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_168_395.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_396_515.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_516_587.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_588_887.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_888_1667.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_1668_4739.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_4740_4883.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_4884_4943.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_4944_5015.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_5016_22643.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_22644_22883.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_22884_53471.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_53472_55187.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_55188_55307.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_55308_57083.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_57084_57179.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_57180_57335.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_57336_57455.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_57456_57575.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_57576_57839.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_57840_58115.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_58116_58715.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_58716_59267.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_59268_61463.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_61464_61775.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_61776_61823.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_61824_61919.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_61920_62111.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_62112_62495.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_62496_64271.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_64272_64751.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_64752_64871.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_64872_65063.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_65064_65171.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_65172_65219.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_65220_65447.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_65448_65555.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_65556_65603.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_65604_65699.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_65700_65867.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_65868_66035.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_66036_66671.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_66672_69179.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_69180_69359.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_69360_69407.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_69408_70547.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_70548_73595.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_73596_73763.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_73764_73967.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_73968_74039.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_74040_74099.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_74100_74699.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_74700_74747.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_74748_76391.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_76392_76571.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_76572_78023.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_78024_78071.png re_publica_2017_-_Prof__Dr__Frank_Pasquale__The_Automated_Public_Sphere_aa7ddud_klA_78072_78328.png