Opening up data at the...
Opening up data at the European Medicines Agency
BMJ 2011; 342 doi: https://doi.org/10.1136/bmj.d2686 (Published 10 May 2011) Cite this as: BMJ 2011;342:d2686
Peter C Gøtzsche, professor,
Anders W Jørgensen, PhD student
1Nordic Cochrane Centre, Rigshospitalet and University of Copenhagen, Dept 3343, Blegdamsvej 9, DK-2100 Copenhagen Ø, Denmark
Correspondence to: P C Gøtzsche, pcg{at}cochrane.dk
Accepted 7 March 2011
Widespread selective reporting of research results means we don't know the true benefits and harms of prescribed drugs. Peter Gøtzsche and Anders Jørgensen describe their efforts to get access to unpublished trial reports from the European Medicines Agency
Doctors cannot choose the best treatments for their patients despite the existence of hundreds of thousands of randomised trials. The main reason is that research results are being reported selectively. Comparisons of published drug trials with unpublished data available at drug regulatory agencies have shown that the benefits of drugs have been much over-rated1 2 3 and the harms under-rated.4 Comparisons of trial protocols with published papers have also shown widespread selective reporting of favourable results.5 6
Selective reporting can have disastrous consequences. Rofecoxib (Vioxx) has probably caused about 100 000 unnecessary heart attacks in the United States alone,7 and class 1 antiarrhythmic drugs probably caused the premature death of about 50 000 Americans each year in the 1980s.8 An early trial found nine deaths among patients taking the antiarrhythmic drug and only one among those taking placebo, but it was never published because the company abandoned the drug for commercial reasons.9
Allowing researchers access to unpublished trial reports submitted to drug regulatory agencies is important for public health. Such reports are very detailed and provide more reliable data than published papers,1 2 3 4 but it has been virtually impossible to get access to them. We eventually succeeded in getting access to reports held by the European Medicines Agency (EMA) after three years of trying. Our case has set an important precedent, and we summarise here the process and the arguments.
Our application for access
On 29 June 2007 we applied for access to the clinical study reports and corresponding protocols for 15 placebo controlled trials of two anti-obesity drugs, rimonabant and orlistat. The manufacturers had submitted the reports to the EMA to obtain marketing approval in the European Union. We explained that we wanted to explore the robustness of the results by adjusting for the many missing data on weight loss and to study selective publication by comparing protocols and unpublished results with those in published reports.
The information was important for patients because anti-obesity pills are controversial. The effect on weight loss in the published trials is small,10 and the harms are substantial. People have died from cardiac and pulmonary complications11 or have experienced psychiatric disturbances, including suicidal events,12 and most of the drugs have been deregistered for safety reasons.
A basic principle in the European Union is to allow its citizens the widest possible access to the documents its agencies possess (box 1).13 But there are exemptions, and the EMA refuses access if disclosure would threaten commercial interests unless there is an over-riding public interest.14 We argued in our first letter to the EMA that secrecy was not in the best interests of the patients because biased reporting of drug trials is common.2 5 Furthermore, we hadn't found any information that could compromise commercial interests in 44 trial protocols of industry initiated trials we had reviewed previously.5
Box 1: Basic principles on citizens' access to EU documents13
"Any citizen of the Union, and any natural or legal person residing or having its registered office in a Member State, has a right of access to documents of the institutions, subject to the principles, conditions and limits defined in this Regulation."
"Openness enables citizens to participate more closely in the decision-making process and guarantees that the administration enjoys greater legitimacy and is more effective and more accountable to the citizen in a democratic system. Openness contributes to strengthening the principles of democracy and respect for fundamental rights as laid down in Article 6 of the EU Treaty and in the Charter of Fundamental Rights of the European Union."
Without any comment on our arguments, the EMA replied that the documents could not be released because it would undermine commercial interests. We appealed to the EMA's executive director, Thomas Lönngren, and asked him to explain why the EMA considered that the commercial interests of the drug industry should over-ride the welfare of patients. We argued that the EMA's attitude increased the risk of patients dying because their doctors prescribed drugs for them without knowing what the true benefits and harms were. He sent us a similar letter to the EMA's first letter, ignoring our request for clarification, and told us we could lodge a complaint with the European ombudsman, which we did.
Over the following three years the EMA put forward several arguments to avoid disclosing the documents: protection of commercial interests, no over-riding public interest, the administrative burden involved, or the worthlessness of the data to us after the EMA had redacted them (box 2). It also did not respond to the ombudsman's letters before his rather generous deadlines had run out.
Box 2: The path to the data
The delays on our part amounted to 130 days (11% of the time); we awaited replies for 1028 days.
29 Jun 2007: We asked the EMA to provide access to the clinical study reports and their corresponding protocols on rimonabant and sibutramine
20 Aug 2007: The EMA replied that the documents could not be released because they came under the exception of commercial interests
24 Aug 2007: We explained that the EMA's lack of transparency violated basic principles in the EU treaty and that it leads to suboptimal treatment of patients
17 Sept 2007: With no comment on our arguments, the EMA referred again to commercial interests and noted we could institute court proceedings against the EMA or complain to the European ombudsman
8 Oct 2007: We appealed to the ombudsman, noting that the published literature on drugs is flawed and arguing that protocols and study reports did not disclose anything that could undermine commercial interests
30 Jan 2008: The EMA replied to two letters from the ombudsman, referred to protection of commercial interests and mentioned that it could not identify any over-riding public interest that could justify disclosure of the requested documents
26 Feb 2008: We told the ombudsman that the EMA had failed to explain why commercial interests would be undermined
28 Apr 2008: The EMA replied to the ombudsman that it needed to protect the data against unfair commercial use; that evaluating the balance between benefits and risks of medicines is the EMA's job; and that redaction of personal data would cause disproportionate effort
17 Jun 2008: In our reply to the ombudsman, we argued against this and noted that if commercial success depends on withholding data that are important for rational decision making by doctors and patients, there is something fundamentally wrong with our priorities in healthcare
22 Jan 2009: The ombudsman proposes a friendly solution to the EMA and asks it to grant us access to the documents or provide a convincing explanation why such access cannot be granted
26 Feb 2009: The EMA restates the commercial interests; claims that we have not given evidence of an over-riding public interest; and refers to the workload involved in redacting the documents
10 Mar 2009: The ombudsman again proposes a friendly solution to the EMA and asks it to clarify its reasoning
7 Apr 2009: The EMA repeats its previous arguments.
19 May 2009: We again counter the EMA's arguments: the EMA has provided no evidence that the documents are commercially sensitive; many patients had been harmed by selective publication of trial data on COX 2 inhibitors; and redacting the documents should be quick and easy
31 Aug 2009: We tell the ombudsman that we have received trial data from the Danish Medical Agency on a third anti-obesity drug, sibutramine
6 Oct 2009: The ombudsman goes to the EMA to inspect the documents we had requested
19 May 2010: The ombudsman issues a draft recommendation that the EMA should grant us access to the documents or provide a convincing explanation as to why not
7 Jun 2010: In a press release the ombudsman accuses the EMA of maladministration because of its refusal to grant access
31 Aug 2010: The EMA informs the ombudsman that it will provide access
1 Feb 2011: We receive the data
Protection of commercial interests
Protection of commercial interests was the EMA's over-riding argument. It would undermine the protection of commercial interests to allow us access, it said, as the documents represented the full details of the clinical development programme and the most substantial part of the applicant's investment. Competitors could use them as a basis for developing the same or a similar drug and gather valuable information on the long term clinical development strategy of the company to their own economic advantage.
We explained that the clinical study reports and protocols are based on well known principles that can be applied to any drug trial; that the clinical study reports describe the clinical effects of drugs; and that nothing in the EMA's guidelines for preparation of such reports indicates that any information included in them can be considered a trade secret. The trial protocols are always sent to the clinical investigators, and it is unlikely that companies would have left in any information that could be of commercial value (such as a description of the drug synthesis). We also noted that the clinical study reports and trial protocols represent the last phase of drug development, which has been preceded by many years of preclinical development. Other companies could hardly use them as a basis for developing similar drugs. In fact, unpublished trial data are generally less positive than published ones,1 2 3 4 5 6 and competitors would therefore be less likely to start drug development if they had access to the unpublished results. Other companies are more likely to be interested in in vitro, animal, and early human studies, and drug companies have no problems with publishing such studies because the results may attract investors.
The European ombudsman, P Nikiforos Diamandouros, considered that commercial interests might be at stake but noted that the risk of an interest being undermined must be reasonably foreseeable and not purely hypothetical. He could not see that access would "specifically and actually" undermine commercial interests. He inspected the relevant reports and protocols at the EMA and concluded that the documents did not contain commercially confidential information. He therefore criticised the EMA's refusal to grant us access.
Over-riding public interest in disclosure
Even if commercial interests were undermined by disclosure, access would still have to be granted if there was an over-riding public interest. The EMA argued that it could not identify any over-riding public interest and remarked that the evaluation of safety and efficacy of drugs is its responsibility—the EMA constantly monitors drugs and updates its assessment reports and requires changes in product information as appropriate.
We considered this insufficient. Monitoring adverse effects reported by doctors to drug agencies would not have revealed that rofecoxib causes heart attacks. Few such events are reported, and heart attacks are common in people with arthritis. Postmarketing passive surveillance systems can therefore usually not detect whether a drug leads to more heart attacks than expected; randomised trials are needed for this.
We provided more evidence of the detrimental effects of selective publication but to no avail. The EMA continued to claim that we had not documented the existence of an over-riding public interest. We noted that we could not prove this in this specific case because we were denied access to the data, but we drew attention to the fact that the total number of patients in the main clinical studies of orlistat differed according to the source of the information: published reports, the EMA's website, and the website of the US Food and Drug Administration.
The ombudsman indicated that we had established an over-riding public interest, but he did not take a definitive stance on whether an over-riding public interest existed because this question needed answering only if disclosure undermined commercial interests. He asked the EMA to justify its position that there wasn't an over-riding public interest, but the EMA avoided replying by saying that we had not given evidence of the existence of such an interest. We believe that we had. Furthermore, the EMA's argument was irrelevant. A suspect asked for his alibi on the day of the crime doesn't get off the hook by asking for someone else's alibi.
Administrative burden
According to the EMA, the redaction of (unspecified) "personal data" would cause the EMA a disproportionate effort that would divert attention from its core business, as it would mean redacting 300 000-400 000 pages. This was surprising. The Danish Drug Agency had not seen the workload as a problem when it granted us access to the reports for the anti-obesity drug sibutramine, which was locally approved in Denmark. The 56 study reports we received comprised 14 309 pages in total, and we requested only 15 study reports from the EMA (the pivotal studies described in the European Public Assessment Reports (EPARs) on rimonabant and orlistat). The ombudsman declared that the EMA had overestimated the administrative burden involved.
Worthlessness of data after redaction
The EMA argued that, "as a result of the redaction exercise, the documents will be deprived of all the relevant information and the remaining parts of them will be worthless for the interest of the complainant."
From what we know of clinical trial reports and protocols it struck us as odd that they would contain so much personal data that the documents became worthless. The ombudsman noted that the requested documents do not identify patients by name but by their identification and test centre numbers, and he concluded that the only personal data are those identifying the study authors and principal investigators and to redact this information would be quick and easy.
The EMA also remarked that a possible future release of the assessment reports of the EMA's Committee for Medicinal Products for Human Use and the (co)rapporteur assessment reports "could satisfy the request of the complainants." These reports were not available and they would have been worthless to us because they are merely summaries used for regulatory decisions.
Maladministration
The EMA was completely resistant to our arguments and those from the ombudsman. However, after the ombudsman accused the EMA of maladministration in a press release on 7 June 2010,15 three years after our request, the EMA reversed its stance. The EMA now gave the impression that it had favoured disclosure all the time, agreed with the ombudsman's reasoning, and noted that the same principles would be applied for future requests for access but that it would consider the need to redact part of the documents.
The EMA's last letter was unclear: "The Agency will do its utmost to implement its decision as quickly as possible, in any case within the next 3 months at the latest. The Agency will keep the European Ombudsman promptly informed of the exact implementation date."
It was not clear whether the three months was the deadline for sending the reports to us, for implementing its new policy, or both. We received the data we requested from the EMA on 1 February 2011, which in some cases included individual patient data in anonymised format, identified by individual and test centre numbers.
According to the EMA's responses to the ombudsman, the EMA put protecting the profits of the drug companies ahead of protecting the lives and welfare of patients. Moreover the EMA's position is inconsistent because it resisted requests to give access to trial data on adult patients while providing access to data on paediatric trials, in accordance with EU legislation.16 The Declaration of Helsinki gives authors the duty to make publicly available the results of their research on humans.17 The declaration also says that, "Medical research involving human subjects must . . . be based on a thorough knowledge of the scientific literature." If the knowledge base is incomplete, patients may suffer and cannot give fully informed consent9 and research resources are wasted. The EMA should be promoting access to full information that will aid rational decision making, not impede it.
Our case sets an important precedent. On 30 November 2010 the EMA declared it would widen public access to documents, including trial reports and protocols.18 We recommend that the FDA and other drug regulatory agencies should follow suit. Access should be prompt—for example, within three months of the regulator's decision—and documents should be provided in a useful format. Drug agencies should get rid of the huge paper mountains and require electronic submissions from the drug companies, including the raw data, which should also be made publicly available.
Contributors and sources: PCG wrote the first draft, both authors commented on it and are guarantors.
Funding: Sygekassernes Helsefond and Rigshospitalet, University of Copenhagen.
Competing interests: All authors have completed the unified competing interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare no support from any organisation for the submitted work; no financial relationships with any organisation that might have an interest in the submitted work in the previous three years; and no other relationships or activities that could appear to have influenced the submitted work.
All documents in this case (133 pages) are available at www.cochrane.dk/research/EMA, together with a comprehensive 26 page report of the case including 54 references.
Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B. Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ2003;326:1171-3.
Rising K, Bacchetti P, Bero L. Reporting bias in drug trials submitted to the Food and Drug Administration: a review of publication and presentation. PLoS Med2008;5:e217.
Healy D. Let them eat Prozac. New York University Press, 2004.
Chan A-W, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA2004;291:2457-65.
Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med2009;361:1963-71.
Lenzer J. FDA is incapable of protecting US "against another Vioxx." BMJ2004;329:1253.
Moore TJ. Deadly medicine: why tens of thousands of heart patients died in America's worst drug disaster. Simon & Schuster, 1995.
Cowley AJ, Skene A, Stainer K, Hampton JR. The effect of lorcainide on arrhythmias and survival in patients with acute myocardial infarction: an example of publication bias. Int J Cardiol1993;40:161-6.
Padwal R, Li SK, Lau DCW. Long-term pharmacotherapy for obesity and overweight. Cochrane Database Syst Rev2003;4:CD004094 (updated 2009).
Mundy A. Dispensing with the truth. St Martin's Press, 2001.
Food and Drug Administration. FDA briefing document. NDA 21-888. Zimulti (rimonabant) tablets, 20 mg. 2007. www.fda.gov/ohrms/dockets/ac/07/briefing/2007-4306b1-fda-backgrounder.pdf.
Regulation (EC) No 1049/2001 of the European Parliament and of the Council of 30 May 2001 regarding public access to European Parliament, Council and Commission documents. Official Journal of the European Communities2001;L145:43-8. www.europarl.europa.eu/RegData/PDF/r1049_en.pdf.
Rules for the implementation of Regulation (EC) No 1049/2001on access to EMEA documents. EMEA/MB/203359/2006 Rev 1 Adopted. Management board meeting 19 December 2006. www.ema.europa.eu/docs/en_GB/document_library/Other/2010/02/WC500070829.pdf.
Diamandouros PN. Ombudsman: European Medicines Agency should disclose clinical reports on anti-obesity drugs. European ombudsman press release, 7 June 2010. www.ombudsman.europa.eu/press/release.faces/en/4940/html.bookmark.
Choonara I. Regulation of drugs for children in Europe. BMJ2007;335:1221-2.
World Medical Association. Declaration of Helsinki—ethical principles for medical research involving human subjects. 2008. www.wma.net/en/30publications/10policies/b3/index.html.
EMA. European Medicines Agency widens public access to documents. Press release, 30 November 2010. www.ema.europa.eu/ema/index.jsp?curl=pages/news_and_events/news/2010/11/news_detail_001158.jsp&mid=WC0b01ac058004d5c1&murl=menus/news_and_events/news_and_events.jsp.
Peter C Gøtzsche, Anders W Jørgensen
Gøtzsche Peter C, Jørgensen Anders W. Opening up data at the European Medicines Agency BMJ 2011; 342 :d2686
Articles by Peter C Gøtzsche
Articles by Anders W Jørgensen
You are going to email the following Opening up data at the European Medicines Agency
Medicines regulation