Do overly complex reporting guidelines remove the focus from good clinical trials?

The ever increasing emphasis on complex reporting guidelines is getting in the way of designing and conducting good clinical trials, sayJeremy Howick,Rebecca Webster, andJ André Knottnerus. ButDavid Moherargues that, while following the guidelines can be frustrating, such complexity remains necessary and is improving research, not impeding it

The original 1996 statement included a half page guide embedded in a three page explanatory document. 2 The updated 2010 CONSORT statement includes 25 items embedded in a 28 page paper, as well as a separate explanatory document. That's just the basic version. There are versions of CONSORT for trials of herbal treatments, orthodontic treatments, feasibility, and at least 25 other subtypes. 3 And that's just CONSORT; many other reporting guidelines have been developed since 1996.
Multiple guidelines are often required for a single study, each with a host of supplementary documents. Identifying, understanding, and implementing these guidelines takes time and effort. This would be justified if these efforts all improved study quality-or better still, patient outcomes. Such evidence is absent.
The fact is that guideline developers measure their success by checking to see how well researchers adhere to the guidance. 4 Such efforts often reveal that the guidelines are barely used. 5 Worse, checking whether guidelines are used doesn't tell us whether the guidelines (in their current form) are useful for improving research in the first place.

Perverse incentives
In addition, since successful (or successfully followed) guidelines lead to multiple publications and are widely cited, perverse incentives are likely to proliferate the problem of too many guidelines. Further, the proliferation of complex reporting guidelines can remove the focus away from designing and conducting good trials or from detecting poor trial design. This is supported by the following reasons.
First, generating, identifying, studying, and referencing the complex web of guidelines is a time consuming process that diverts scarce resources from designing clinically sound hypotheses and conducting good research. Second, compliance with reporting guidelines creates a veneer of detailed sound reporting that can distract from fundamental problems in a trial.
Trials of questionable value have continued to thrive despite CONSORT. In 2015, for example, researchers conducted a placebo controlled trial looking at the efficacy of a three month formulation of paliperidone palmitate in patients with schizophrenia, 6 finding that the treatment was superior to placebo. The trial was well reported in terms of adherence to the reporting guidance. However, previously published systematic reviews 7 had already established the efficacy of antipsychotics for reducing relapse in patients with schizophrenia. The authors did not explain why a placebo controlled trial of a new intervention was conducted instead of a trial of a new intervention versus a proven therapy. 8 Another well reported trial suggested that the diabetes drug rosiglitazone effectively lowered blood glucose (a surrogate outcome). It was subsequently found that the drug increased cardiovascular events (the outcome relevant to patients). 9 In another example, three of four well reported industry sponsored trials evaluating newer antihypertensive drugs used atenolol as the comparator, although a low dose thiazide diuretic had been found to be superior. 10 These examples show how a well reported trial with inappropriate comparators or outcomes is wasteful and may lead to the adoption of useless or even harmful interventions that lack benefits. They also show that the complex web of reporting guidelines the researchers meticulously followed failed to catch this, improve the trial design, or prevent such trials from happening in the first place.

The simpler the better
Counterintuitively, simpler guidelines may be more rigorous. 11 For example, Green and Mehr introduced a simple clinical guideline to determine whether a patient with chest pain should be sent to a coronary care unit. 12 This resulted in better sensitivity and specificity than a complex, 50 question instrument.
Moher argues that ". . . such as with pharmaceuticals, we should be more cautious about recommending the use of reporting guidelines without evidence of effectiveness." 13 We agree. In our view, the effectiveness of the current state of affairs-namely, many complex guidelines-must be formally compared with the use of vastly simplified, single guidelines.
The proliferation of increasingly complex guidelines needs to be tamed in the same way as the problem of too much medicine: with evidence based evaluation.

No-David Moher
Researchers have always struggled to report their randomised controlled trials (RCTs) completely and transparently. 14 This is why experts involved in the process, from journal editors to clinicians and methodologists, met as early as 1993 to find a solution. The resulting guidelines 15 16 include a 1994 publication by the Standards of Reporting Trials (SORT) group, aiming to provide prospective authors with guidance on reporting their trial design, methods, and results. They were not alone: Drummond Rennie, a deputy editor of JAMA, was a member of a separate initiative called Asilomar that met about six months later with a similar remit to SORT's. He brought both groups together, resulting in the original CONSORT statement in 1996.
CONSORT had two fundamental goals at its inception. Its longer term goal is to improve the quality of RCTs through better reporting, which admittedly remains distant. 17 But its immediate goal of improving the transparency and completeness of reporting RCTs is arguably being achieved, even if the implementation is complex.
For example, describing allocation concealment in reports of RCTs is more complete in journals that endorse the CONSORT guidance than in journals that do not. 5 The development of the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) standard for trial protocols in 2013 has brought further improvement in the transparency of protocols for RCTs. 18 If authors have adequately described the method used for allocation concealment in their protocol, and if there has been no deviation of this during the trial, it should be easy to include this information in the final report of the completed trial.

Clear problems, not so clear solutions
Undoubtedly, since CONSORT's birth the number of guidelines has proliferated: there are numerous extensions to CONSORT, while EQUATOR (Enhancing the QUAlity and Transparency Of health Research Network) contains a comprehensive open access library of more than 400 reporting guidelines.
Admittedly, this can be confusing for people reporting the trials, particularly if they are required to use multiple guidance. Journals also differ in their endorsement, often not recommending that peer reviewers use reporting guidelines when peer reviewing a manuscript, and most journals have not publicly indicated whether they monitor their endorsement of reporting guidelines. Some reporting guidelines have never been cited, 19 suggesting a lack of use.
But while the problems are clear, I think that the solutions are more complex. Even if guidance were made simpler, the evidence underpinning reporting recommendations takes time to accumulate. When CONSORT was created in 1996, there was little evidence about the consequences of outcome reporting biases or spin on the results and interpretation of RCTs. We know better today, and reporting guidelines need to be updated to reflect this knowledge.
While providing evidence will always be the main justification for reporting items in clinical trials, clarity and common sense are other important reasons. Authors who report their trial design, methods, and results completely and transparently may have an easier time disseminating their message. And, without an audit of the publication of RCT results, it's very difficult to continue to improve or change their practice for the better. Feedback to authors would help, but I am unaware of any journal or publisher that provides this type of audit and feedback on its website or to individual researchers.

Incentives
Just making guidelines simpler would not necessarily increase the number of people using them or improve the quality of reporting. A bigger problem is that authors are not incentivised or rewarded for using reporting guidelines in the first place. What if researchers were assessed as to whether they used a guideline when reporting their research, rather than the ubiquitous "publish or perish"? 20 The metadata are available to enable automated compilation of this information. Researchers could include this information in their curriculum vitae.
The problems of complexity are acknowledged, and efforts are under way to fix them. Journal editors have repeatedly urged the use of technology to help authors adhere to reporting guidelines. Authors can use Penelope (an EQUATOR wizard), and members of the newly merged SPIRIT and CONSORT executive are working to incorporate some of the key extensions into a single checklist.
While I understand why busy clinicians call for simpler guidance, I question the belief that this would improve the quality of clinical trial reporting. Conducting trials, even pragmatic real world trials, is complex and takes time. Shouldn't we have a similar philosophy for reporting them?