Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. Improving services for disadvantaged childbearing women. Secular changes in the quality of published randomized clinical trials in rheumatology. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Quality of reporting of randomized controlled trials in general endocrinology literature. Massage or music for pain relief in labour: a pilot randomised placebo controlled trial. Quality of reporting of randomized, controlled trials in cerebral palsy. Impact of study quality on outcome in placebo-controlled trials of homeopathy. What is the quality of reporting in weight loss intervention studies? A systematic review of randomized controlled trials. Standards of reporting of randomized controlled trials in general surgery: can we do better?
To explore whether reported methodologic quality affects estimated intervention effects in randomized trials and contributes to discrepancies between the results of large randomized trials and small randomized trials in meta-analyses.
                Meta-analyses of randomized trials that included at least one large trial (>/=1000 participants) were included, regardless of the therapeutic area. Eligible meta-analyses were identified through electronic searches and bibliographies of relevant articles.
                Full-length randomized trials.
                Methodologic quality was assessed according to reported randomization, double blinding, and follow-up as separate components and by using the Jadad composite scale.
                Fourteen meta-analyses involving 190 randomized trials from eight therapeutic areas were included. Compared with large trials, intervention effects were exaggerated in small trials with inadequate allocation sequence generation (ratio of odds ratios, 0.46 [95% CI, 0.25 to 0.83]; P = 0.011), inadequate allocation concealment (ratio of odds ratios, 0.49 [CI, 0.27 to 0.86]; P = 0.014), and no double blinding (ratio of odds ratios, 0.52 [CI, 0.28 to 0.96]; P = 0.01). Large trials did not differ significantly from small trials with adequate generation of the allocation sequence, adequate allocation concealment, or adequate double blinding. No association was seen between reported follow-up and intervention effects. The Jadad scale provided no additional information because the scale and the quality components overlapped substantially.
                Inadequate generation of the allocation sequence, allocation concealment, and double blinding lead to exaggerated estimates of intervention benefit and may contribute to discrepancies between the results of large randomized trials and small randomized trials in meta-analyses. To determine if inadequate approaches to randomized controlled trial design and execution are associated with evidence of bias in estimating treatment effects.
                An observational study in which we assessed the methodological quality of 250 controlled trials from 33 meta-analyses and then analyzed, using multiple logistic regression models, the associations between those assessments and estimated treatment effects.
                Meta-analyses from the Cochrane Pregnancy and Childbirth Database.
                The associations between estimates of treatment effects and inadequate allocation concealment, exclusions after randomization, and lack of double-blinding.
                Compared with trials in which authors reported adequately concealed treatment allocation, trials in which concealment was either inadequate or unclear (did not report or incompletely reported a concealment approach) yielded larger estimates of treatment effects (P < .001). Odds ratios were exaggerated by 41% for inadequately concealed trials and by 30% for unclearly concealed trials (adjusted for other aspects of quality). Trials in which participants had been excluded after randomization did not yield larger estimates of effects, but that lack of association may be due to incomplete reporting. Trials that were not double-blind also yielded larger estimates of effects (P = .01), with odds ratios being exaggerated by 17%.
                This study provides empirical evidence that inadequate methodological approaches in controlled trials, particularly those representing poor allocation concealment, are associated with bias. Readers of trial reports should be wary of these pitfalls, and investigators must improve their design, execution, and reporting of trials. Tackling health inequalities is high on the political agenda of the Labour government. The government wants to reduce inequalities through policies based on evidence, the health of childbearing women and their babies being priority areas for action. National Service Frameworks (NSFs) are being set up to ensure high quality services for all groups. The External Working Group looking into maternity services for the Children's NSF seeks evidence upon which to plan strategies for all women, while focusing attention on the most disadvantaged. Wide differentials are noted between high- and low-income groups in the outcomes of pregnancy and the health of babies. The worst outcomes appear to be concentrated in small subgroups of disadvantaged women.
                To report on a review of studies of interventions improving perinatal outcomes for disadvantaged women, to inform policy and practice around the organization and delivery of statutory services in the UK.
                We searched six relevant databases for reviews, intervention studies, audits and descriptions of services reporting outcomes for disadvantaged groups of women, published between 1990 and 2003. Synthesis was performed around what works and what does not work. Gaps in the evidence base were identified.
                We found limited evidence of effective and promising interventions for childbearing women from minority ethnic groups, women experiencing domestic violence, women with mental health illness and HIV infected women. Few studies were well-designed or powered to detect effectiveness. There were no studies of interventions for women prisoners, homeless women and travellers.
                Searching for intervention studies primarily by participant subgroup has brought in evidence from few well-designed studies on which to plan policy. Combining this approach with searching for intervention studies addressing behaviour like smoking, and needs like social support, may provide further evidence to tackle inequalities in the perinatal period. To assess the quality of published randomized clinical trials (RCTs) in rheumatology and to determine whether there has been improvement in quality between 2 time periods, 1987-1988 and 1997-1998.
                Using MEDLINE and a hand search of selected rheumatology journals, we identified RCTs of adult rheumatic diseases published in English in 1987-1988 or 1997-1998. We examined trial quality with an expanded version of the Jadad scale, which assesses the adequacy of reported random sequence generation, allocation concealment, blinding, and analysis. All trials were read by 1 reviewer, with prior standardization using a random sample read by 2 reviewers. We also evaluated "high"- versus "low"-impact journals based on citation index.
                Two hundred forty RCTs (1987-1988 119 RCTs, 1997-1998 121 RCTs) were assessed. Results showed improvement in the quality of the trials, but the rates of reported random sequence generation, allocation concealment, power, and intent-to-treat analyses were persistently low. Low rates of reports of random sequence generation, allocation concealment, and intent-to-treat analyses were present even in the high-impact journals.
                There has been improvement in the quality of reporting of RCTs in rheumatology between 1987-1988 and 1997-1998. However, methodologic problems such as lack of allocation concealment, inadequate random sequence generation, lack of reporting of power, and lack of intent-to-treat analyses remain common. Many of these problems are established sources of bias in RCTs and are easily rectifiable. Few meta-analyses of randomised trials assess the quality of the studies included. Yet there is increasing evidence that trial quality can affect estimates of intervention efficacy. We investigated whether different methods of quality assessment provide different estimates of intervention efficacy evaluated in randomised controlled trials (RCTs).
                We randomly selected 11 meta-analyses that involved 127 RCTs on the efficacy of interventions used for circulatory and digestive diseases, mental health, and pregnancy and childbirth. We replicated all the meta-analyses using published data from the primary studies. The quality of reporting of all 127 clinical trials was assessed by means of component and scale approaches. To explore the effects of quality on the quantitative results, we examined the effects of different methods of incorporating quality scores (sensitivity analysis and quality weights) on the results of the meta-analyses.
                The quality of trials was low. Masked assessments provided significantly higher scores than unmasked assessments (mean 2.74 [SD 1.10] vs 2.55 [1.20]). Low-quality trials (score < or = 2), compared with high-quality trials (score > 2), were associated with an increased estimate of benefit of 34% (ratio of odds ratios [ROR] 0.66 [95% CI 0.52-0.83]). Trials that used inadequate allocation concealment, compared with those that used adequate methods, were also associated with an increased estimate of benefit (37%; ROR=0.63 [0.45-0.88]). The average treatment benefit was 39% (odds ratio [OR] 0.61 [0.57-0.65]) for all trials, 52% (OR 0.48 [0.43-0.54]) for low-quality trials, and 29% (OR 0.71 [0.65-0.77]) for high-quality trials. Use of all the trial scores as quality weights reduced the effects to 35% (OR 0.65 [0.59-0.71]) and resulted in the least statistical heterogeneity.
                Studies of low methodological quality in which the estimate of quality is incorporated into the meta-analyses can alter the interpretation of the benefit of intervention, whether a scale or component approach is used in the assessment of trial quality. The reporting quality of randomized controlled trials (RCTs) is poor in general medicine and several areas of specialization but unknown in endocrinology.
                Our aim was to assess the reporting quality of RCTs in general endocrinology. A secondary objective was to identify predictors for better reporting quality.
                We systematically reviewed RCTs published in three general endocrinology journals between January 2005 and December 2006.
                We included parallel-design RCTs that addressed a question of treatment or prevention. Article selection and data abstraction were conducted by two reviewers independently, and disagreements were resolved by consensus.
                There were two main outcomes: 1) a 15-point overall reporting quality score (OQS) based on the Consolidated Standards for Reporting Trials (CONSORT); and 2) a 3-point key score, based on allocation concealment, blinding, and use of intention-to-treat analysis.
                Eighty nine RCTs were included. The median OQS was 10 (interquartile range = 2). Allocation concealment, blinding, and analysis by intention to treat were reported in 10, 20, and 16 of the 89 RCTs, respectively. A multivariable regression analysis showed that complete industrial funding [incidence rate ratio (IRR) = 1.014; 95% confidence interval (CI), 1.010-1.018], journal of publication (IRR = 1.068; 95% CI, 1.007-1.132), and sample size (IRR = 1.048; 95% CI, 1.026-1.070) were significantly associated with a slightly better OQS.
                The quality of RCT reporting in general endocrine literature is suboptimal. We discuss our results, highlight the areas where improvements are needed, and provide some recommendations. Research on massage therapy for maternal pain and anxiety in labour is currently limited to four small trials. Each used different massage techniques, at different frequencies and durations, and relaxation techniques were included in three trials. Given the need to investigate massage interventions that complement maternal neurophysiological adaptations to labour and birth pain(s), we designed a pilot randomised controlled trial (RCT) to test the effects of a massage programme practised during physiological changes in pain threshold, from late pregnancy to birth, on women's reported pain, measured by a visual analogue scale (VAS) at 90 min following birth. To control for the potential bias of the possible effects of support offered within preparation for the intervention group, the study included 3 arms--intervention (massage programme with relaxation techniques), placebo (music with relaxation techniques) and control (usual care). The placebo offered a non-pharmacological coping strategy, to ensure that use of massage was the only difference between intervention and placebo groups. There was a trend towards slightly lower mean pain scores in the intervention group but these differences were not statistically significant. No differences were found in use of pharmacological analgesia, need for augmentation or mode of delivery. There was a trend towards more positive views of labour preparedness and sense of control in the intervention and placebo groups, compared with the control group. These findings suggest that regular massage with relaxation techniques from late pregnancy to birth is an acceptable coping strategy that merits a large trial with sufficient power to detect differences in reported pain as a primary outcome measure. In conducting reviews on the effectiveness of physiotherapy interventions on children with cerebral palsy, the assessment of trials can be hampered by problems in reporting. Therefore, we set out to evaluate trial reporting by using the Consolidated Standards of Reporting Trials (CONSORT) statement recommendations.
                Randomized, controlled trials published in 1990 or later were identified in literature searches for reviews on the effectiveness of physiotherapy interventions on children with cerebral palsy. Two independent reviewers evaluated the trial reporting by using a modified 33-item Consolidated Standards of Reporting Trials checklist.
                We identified 15 randomized, controlled trials. Almost half (48%) of the applicable items were reported adequately. Inadequate reporting was found in the following items: outcome measures, sample-size determination, details of the sequence generation, allocation concealment and implementation of the randomization, success of assessor blinding, recruitment and follow-up dates, intention-to-treat analysis, precision of the effect size, co-interventions, and adverse events.
                Only a small number of sufficiently reported trials were found. Because nearly all items had been described in at least 1 article, high-quality reporting seems feasible. Assessment of trials depends on appropriate reporting, and poor reporting jeopardizes judgments on the clinical implications. Authors of randomized, controlled trials are encouraged to follow the Consolidated Standards of Reporting Trials criteria. There is a clear need to improve the quality of reporting of trials in this field. We investigated the influence of indicators of methodological quality on study outcome in a set of 89 placebo-controlled clinical trials of homoeopathy in three different ways: (1) The results of studies meeting single criteria (explicit statement of random allocation, allocation concealment, double-blinding, completeness of follow-up) of methodological quality were compared with those of studies not meeting the criteria in univariate and multivariate analyses; (2) The results of studies scoring above and below predefined scores in two quality assessment scales were compared; (3) Primary studies were consecutively entered into a cumulative meta-analysis according to the summary scores derived from the quality assessment scales. All analyses were performed using meta-regression methods. Studies that were explicitly randomized and were double-blind as well as studies scoring above the cut-points yielded significantly less positive results than studies not meeting the criteria. In the cumulative meta-analyses, there was a trend for increasing effect sizes when more studies with lower-quality scores were added. However, there was no linear relationship between quality scores and study outcome. We conclude that in the study set investigated, there was clear evidence that studies with better methodological quality tended to yield less positive results. Because summarizing disparate study features into a single score is problematic, meta-regression methods simultaneously investigating the influence of single study features seem the best method for investigating the impact of study quality on outcome. Despite the large number of randomized controlled trials (RCTs) assessing weight loss interventions, no study has assessed the quality of reporting in these trials.
                To assess the quality of reporting of RCTs of weight loss interventions and to identify predictors of reporting quality.
                The RCTs assessed were derived from a published systematic review of trials investigating the efficacy of weight loss interventions. For our study, two reviewers independently rated the quality of reporting in these trials, based on the Consolidated Standards for Reporting of Trials (CONSORT) Statement. We describe the quality of reporting using number (percent) of studies satisfying each of the 44 CONSORT criteria. We use generalized estimating equations (GEE) to fit a multivariable regression model to determine factors that are associated with the overall quality reporting score.
                We assessed 63 RCTs, of which 25 were dietary-lifestyle trials, 22 were pharmacological trials and 16 were behavior-cognitive, exercise-lifestyle, or surgical trials. Less than half (46%) of the trials defined the primary outcome of the study; about 10% provided the description of the method of allocation concealment. Multivariable GEE results showed that the sample size, type of intervention (non-pharmacologic trials having lower scores than pharmacologic trials), and publication time relative to the CONSORT Statement publication in 1996 (publications after 1996 having higher scores) were strong predictors of the quality reporting score. Reporting a statistically significant result on the primary outcome was not significantly associated with the quality score.
                While the overall quality in reporting seemed to have improved since the publication of the revised CONSORT Statement in 1996, the reporting of some key methodologic aspects, such as clear description of primary outcome and method of allocation concealment, still requires improvements. Factors that are significantly associated with the overall quality reporting score can be used as surrogates in the review of protocols to enhance the quality of the final reports. To evaluate the quality of reporting of surgical randomized controlled trials published in surgical and general medical journals using Jadad score, allocation concealment, and adherence to CONSORT guidelines and to identify factors associated with good quality.
                Randomized controlled trials (RCTs) provide the best evidence about the relative effectiveness of different interventions. Improper methodology and reporting of RCTs can lead to erroneous conclusions about treatment effects, which may mislead decision-making in health care at all levels.
                Information was obtained on RCTs published in 6 general surgical and 4 general medical journals in the year 2003. The quality of reporting of RCTs was assessed under masked conditions using allocation concealment, Jadad score, and a CONSORT checklist devised for the purpose.
                Of the 69 RCTs analyzed, only 37.7% had a Jadad score of > or =3, and only 13% of the trials clearly explained allocation concealment. The modified CONSORT score of surgical trials reported in medical journals was significantly higher than those reported in surgical journals (Mann-Whitney U test, P < 0.001). Overall, the modified CONSORT score was higher in studies with higher author numbers (P = 0.03), multicenter studies (P = 0.002), and studies with a declared funding source (P = 0.022).
                The overall quality of reporting of surgical RCTs was suboptimal. There is a need for improving awareness of the CONSORT statement among authors, reviewers, and editors of surgical journals and better quality control measures for trial reporting and methodology.
The included trial was a small-scaled study and there were serious limitations in the design and methodology (e.g. allocation concealment was unclear, blinding of outcome assessor(s) was not ensured, few quantitative outcome measures were used, and the results were not reported as planned). Therefore, the current evidence supporting the efficacy of EPG is not strong and there remains a need for high-quality randomised controlled trials to be undertaken in this area.