Open up online research
Alexander Halavais1
Nature volume 480, pages 174–175 (2011)Cite this article
Social media hold a treasure trove of information. But the secretive methods of ethics review boards are hindering their analysis, says Alexander Halavais.
The mass adoption of networked communication and an emerging culture of open sharing have provided a boon for social scientists. They have opened a window through which social scientists can observe the experiences of individuals and groups that have been all-too-easily ignored by history, and present an opportunity for researchers to examine not just a few, but thousands of eyewitness accounts of an event. My own work, for example, has examined protests on Twitter, how people use the Digg website to become part of an online community, how political campaigners reach voters online and how blogs can contribute to the spread of news. The US Library of Congress, which last year acquired the archive of all public tweets made since Twitter began in 2006, announced that it will open up access to the database, although it is not yet clear when or under what rules. Through social media, social science is entering an age of 'big science'.
Collecting public musings can create ethical dilemmas for researchers. The messages may be publicly available, but the individuals writing them might not realize just how public they are, and might be surprised when their words are preserved or placed in another context. When, in 2006, sociologists from Harvard University in Cambridge, Massachusetts, collected data from Facebook for a study on students' friendships and shared interests, some people questioned whether the researchers had sufficient permission to distribute the data, and whether they had made them sufficiently anonymous. Similarly, when America Online (AOL) released a data set of some 20 million search terms from 650,000 anonymized users for the purposes of research, also in 2006, journalists and others managed to link some individuals to strings of search terms. The scandal resulted in a class-action lawsuit and the resignation of AOL's chief technology officer.
Aspects of gaming cultures, such as players' conventions, are unfamiliar to many members of the ethics committees that must judge web-based privacy issues. Credit: J. JOHANNESSEN/PANOS
Working on large-scale public conversations is new ground for many researchers, and for most research ethics committees, also called institutional review boards (IRBs). What constitutes ethical conduct remains blurry. In the case of the Harvard Facebook study, the methodology passed ethics review, but still caused controversy. Equally troubling is the other side of the coin: many studies have been blocked by ethics review, even when they present minimal risk to the participants.
IRBs were initially created as a way to review medical research in the United States, protect against ethical misconduct and ensure that participants were made aware of the risks associated with medical experimentation. By the late 1960s, the boards were also overseeing privacy risks in social-science research, prompting the prominent US anthropologist Margaret Mead to argue to the National Institutes of Health that her field did not work with "subjects", but rather with "informants in an atmosphere of trust and mutual respect". By the mid-1990s, particularly in the United States, IRBs were overseeing groups that they used to ignore: historians, journalists and folklorists, for example. As Laurie Essig, a sociologist at Middlebury College in Vermont, put it in a blog post in August in the Chronicle of Higher Education: "IRBs have treated speaking with someone as equivalent to experimenting on them and have almost killed fieldwork in the process". The US model is becoming the global norm, as IRBs elsewhere begin to review studies being done in the social sciences and humanities.
The time and expense of intensive ethics review of online social science puts the brakes on such work: both slowing down research and restricting the sharing of data.
Steps are being taken to resolve these issues, but more needs to be done. Journals and funding agencies can, and should, help by making ethics reviews more transparent. This will demystify the ethics considerations behind such work so that researchers can learn from the successes of others. It will also encourage companies and the public to entrust their personal information to scientists. IRBs, for their part, should not apply the same standards used for studies that entail risks to health to those that may just have privacy issues, particularly when the societal perception of online privacy is in flux.
Working social scientists will agree that research conduct must be ethical, but few will extol the virtues of the IRB. War stories abound: a protocol held up because someone on the committee felt that the area of research was fruitless, or because of a spelling error, or because the research might have brought too much controversy to the campus. Even when the issues are more central to the protection of people, the standards of approval are often ambiguous and informal. High levels of scrutiny are clearly necessary for a drug trial. But scrutinizing whether gamers would be traumatized by being asked questions about dressing up as characters for conventions — to take an example from my students' research — is an issue best addressed by the researchers, who have had much more exposure to the participants and the culture being examined.
For those who research online interactions, it can be especially frustrating to have a board filled with members who have never used Facebook or played World of Warcraft. Although IRBs can, and sometimes do, bring in experts who can address the context of the research more directly, this happens more rarely than it should.
The decisions of IRBs seem to be idiosyncratic and, by extension, capricious.
Members of IRBs tend to be more comfortable with some methods than others: the hypothesis-driven experiments of the psychology lab rather than the inductive work done by ethnographers, for example. The decisions of IRBs seem to be idiosyncratic and, by extension, capricious, especially when multi-site research is approved by several boards, yet held up by others. In the case of a colleague, each of two review boards insisted on having the other approve a protocol first. This can easily lead to research gridlock, and has spawned a growing industry of professional ethics-review expediters.
Calls have continually been made to improve the ethics review process. The Institutional Review Blog (www.institutionalreviewblog.com) chronicles the overzealous restrictions of IRBs in the humanities and social sciences, for example. The largest organization of ethics review professionals, Public Responsibility in Medicine and Research, provides a venue for discussing ways to improve the process. Perhaps most promisingly, in July, the federal agency that oversees IRBs in the United States — the Office for Human Research Protections — announced its intention to revise the rules for research that poses only a minimal risk to people. The proposed changes should reduce many of the burdens currently imposed on researching open discourse on the web — such as studying blog posts or tweets. It is, however, unlikely to make standards of privacy clearer to researchers, students or those who host participatory websites.
After researchers have battled their way through the IRB process, the result, an IRB approval form, is usually tucked away in a drawer. Students are often introduced to this secretive process in a cursory overview of ethics in a methods course, and sometimes by being assigned to handle the IRB process for a project. Both good and bad models of ethical research are difficult to come by for students, particularly when the work is treading less trammelled ground: ethnographic research of virtual communities, for example.
Fixing the system
The solution is not to do away with the IRB, but rather to make amendments that render its dysfunctions less acute.
The first step is to agree on a reasonable threshold for when IRBs should become involved — although federal oversight bodies set a minimum standard, institutions have some discretion on their specific policies. Some universities, in an overabundance of caution, require work that incurs even the most minimal risk to be reviewed by an ethics committee before it can commence, often leading to months of delays. And not all committees consider research that involves text — such as tweets or blog entries — to be in need of ethics oversight. This lack of clarity and consensus results in unnecessary oversight, taxing boards that could spend their time better on work that presents a significant risk to vulnerable people. When risks are to adults' privacy only, for example, an IRB should not need to review the protocols.
A complementary important step is to ensure that upcoming students, in all fields, have adequate ethics training. This will help to make certain that research not requiring IRB approval still has ethical thought behind it, and discourage the notion that handing a proposal over to an IRB excuses researchers from considering the ethics themselves. Ethics thinking should begin when a study is first being designed and continue through to its completion and publication, rather than constituting a single bureaucratic hurdle.
The greatest problem faced by the ethics system is secrecy. Review boards must make decisions with limited access to previous cases. Their decisions are rarely available to other IRBs or to researchers who could make productive use of their precedents — particularly in new areas of research such as online social science, in which review boards tend to have little experience and would therefore benefit most from the experience of others. One solution would be to require IRBs to be transparent in their decision-making. This, however, seems unlikely to succeed. The IRB system is conservative by design; we must look elsewhere for change.
Most government funding agencies and private foundations prominently promote open data sharing and collaboration. Were they to make the open publication of IRB protocols or ethics reflections a requirement for receiving funding, it would provide a crack in an otherwise too-secretive process. Many journals expect social-science research to have been inspected by an IRB before submission; a few even require authors to sign a statement of IRB approval. None requires that the approved protocol be provided to the journal or published. They should. Although some information would need to be redacted from such documents — including anything that might violate the privacy of participants or of the researchers — they would open a new window on ethics considerations.
Enacting these changes would not only grease the wheels of social-science research, but also help to convince the public that the work is both important to their well-being and being done in a trustworthy way. That, in turn, might make companies more willing to share their data with science. At present, researchers find it difficult to study the large data sets from Facebook or Twitter, for example, without partnering up with someone within those companies; the sites' terms of use prevent outsiders from 'scraping' large data sets.
By moving the consideration of ethical conduct beyond the localized IRB to the wider research community, we can evolve the kinds of standards and best practices that can serve to instruct not just the scholarly world, but also the wider realms of government policy and corporate practice.
Alexander Halavais is president of the Association of Internet Researchers and associate professor of interactive media at Quinnipiac University, Hamden, Connecticut 06518-1908, USA.,
Alexander Halavais
Correspondence to Alexander Halavais.
Ess, C. Digital Media Ethics (Polity, 2009).
Schrag, Z. M. Ethical Imperialism: Institutional Review Boards and the Social Sciences, 1965–2009 (Johns Hopkins Univ. Press, 2010).
Stark, L. Behind Closed Doors: IRBs and the Making of Ethical Research (Univ. Chicago Press, 2011).
Halavais, A. Open up online research. Nature 480, 174–175 (2011). https://doi.org/10.1038/480174a
Issue Date: 08 December 2011
DOI: https://doi.org/10.1038/480174a