Ethics of using public and illegally obtained databases for academic purposes I'm on my senior year of college, and my professor and I have decided on an independent study where I will analyze and develop some method(s) for screening valid data sets for the 'selection' phase of data mining, where spam bots, dead accounts, and other irrelevant data sources are omitted from machine learning analysis.

The problem i'm running into is that I really want to use a data set that is from the real world, so I could truly discover something meaningful from the data that hasn't been discovered before.

My initial thought was to use the Ashley Madison hack dump from the summer of 2015, but I wanted to consider the ethical implications of using such sensitive data. Alternatively, I was thinking I could manipulate the data before analyzing it to provide some sense of anonymity to the victims of the hack (for instance, replacing all full emails with the first and last character of the name, as well as the @ and suffix).

My question is NOT whether you think these practices are ethical, but whether there has been some professional work done in the past anyone is aware of that can serve as a model for my current work.

For example, Facebook was caught manipulating the content of its users' Facebook feeds in order to measure the emotional reaction of the content on their future posts.
Relating to just the AshleyMaddison data, there is a plethora of articles on your exact question. I will summarize the main talking points and provide links at the end.

1. Can I download it?

The data is probably public-domain, but it depends on where you live. In some countries like the US, the data itself is publicly accessible and part of a wider conversation about personal rights to privacy, whilst in others like Canada, is has been explicitly decided that AshleyMaddison still holds the copyright and downloading the data is akin to acquiring stolen property (note, AM is owned by AvidLifeMedia, a Canadian company, and people have said that this decision was to offer some degree of protection to ALM, as hacked users cannot use the data dump as evidence in court). Other countries say that fundamentally nothing that can be downloaded can be stolen, so this doesn't even apply.

2. Can I share portions of the data with others?

If you are in a jurisdiction that allows you to download it, you can by default share it - and many (perfectly legal) sites exist for just that. You type your name or e-mail address in to see if you are part of the hack (or your spouse was). For better or for worse, there is no crime for making public-domain data easier to access.

3. Can I process the data and share summary statistics?

Oddly, unlike the above 2 issues, this is legal in every country which offers protection for journalists and researchers - including Canada. Many researchers, particularly those researching infidelity, have asked lawyers this question, and they all say the same thing - yes it's legal. In fact, top US lawyers have gone as far as to say that journalists are probably in the clear if they publish a list of names of celebrities who appear in the hack, for whatever little public good/interest there might be there.

Many articles also point out that there is a big difference between the legality of this, and the ethics of this. Both the law of the land and what is considered ethical behaviour changes over time, and they don't always have to be in sync. Some say that in using the data you are condoning and even encouraging the hack - which may lead to more hacking/data dumps in the future. Other's say that your research may be the only grain of good to come out of the whole debacle.

I will summarize by saying what I would personally do, which is use the data, get the outcomes, then way up the pros and cons of those outcomes with the example you will be setting for others in using this data. Legality is really not the issue here at all, because even if it is illegal, you are incredibly unlikely to find yourself going to court from either Ashley Madison or hacked users, as both parties would have a very poor case. Those aren't my words, those are the words of Jennifer Granick, a law professor at the Stanford Center for Internet and Society.

So if the real question here is the ethics, then this is something that is ultimately up to you to decide on. There may be all manner of real repercussions from your department - particularly if someone in the department turns up in the database... - but that's a very different question.

http://www.huffingtonpost.com/entry/ashley-madison-hack-creates-ethical-conundrum-for-researchers_us_55e4ac43e4b0b7a96339dfe9

http://www.al.com/news/index.ssf/2015/08/is_it_illegal_to_download_the.html

https://onlinejournalismblog.com/2015/07/20/ashley-madison-ethics-journalism-hacked-documents/

http://fortune.com/2015/08/19/ashley-madison-media/

Finally, there are many stories that people have posted online detailing their experience at the hands of the hack. Some say it wasn't really them, some say it was. Some are outraged, some are just numb. I would suggest reading one or two of the longer blog posts to really get a sense of what this data means to some people. It's more than just a resource. People have committed suicide due to the shame or discrimination they faced as a result - most notably people from the LGBT community - so it's really important to not shy away from that when deciding to proceed, or not, with your research.