Scavengers of oxygen-derived free radicals prolong survival in advanced colonic cancer. A new approach. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy. Evidence of improving quality of reporting of randomized controlled trials in subfertility. Publication bias in presentations to the Annual Scientific Congress. Quality of reporting internal and external validity data from randomized controlled trials evaluating stents for percutaneous coronary intervention. Assessment of risk of bias among pediatric randomized controlled trials. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. Randomized controlled trial of electro-acupuncture for autism spectrum disorder. Association of industry sponsorship to published outcomes in gastrointestinal clinical research.
The influence of oxygen-derived free radicals on survival in advanced colonic cancer was assessed in a prospective randomized controlled double-blind trial using the radical scavengers dimethyl sulphoxide (DMSO) and allopurinol. Following palliative sigmoid colectomy for carcinoma at Dukes' stage D, 306 patients were randomized to the control group or to electrocoagulation of liver secondaries alone or with allopurinol (50 mg by mouth 4 times a day) or DMSO (500 mg by mouth 4 times a day). In 193 fully evaluable patients who were studied for 5 years, allopurinol and DMSO incurred a significant (p < 0.05) survival advantage over the whole period of study. The similarity in efficacy between allopurinol and DMSO and the fact that the only action they share is scavenging oxyradicals, suggest that these radicals mediate the detrimental effects of malignancy and that removing them incurs a survival advantage for patients with advanced colonic cancer. Selective reporting of outcomes within published studies based on the nature or direction of their results has been widely suspected, but direct evidence of such bias is currently limited to case reports.
                To study empirically the extent and nature of outcome reporting bias in a cohort of randomized trials.
                Cohort study using protocols and published reports of randomized trials approved by the Scientific-Ethical Committees for Copenhagen and Frederiksberg, Denmark, in 1994-1995. The number and characteristics of reported and unreported trial outcomes were recorded from protocols, journal articles, and a survey of trialists. An outcome was considered incompletely reported if insufficient data were presented in the published articles for meta-analysis. Odds ratios relating the completeness of outcome reporting to statistical significance were calculated for each trial and then pooled to provide an overall estimate of bias. Protocols and published articles were also compared to identify discrepancies in primary outcomes.
                Completeness of reporting of efficacy and harm outcomes and of statistically significant vs nonsignificant outcomes; consistency between primary outcomes defined in the most recent protocols and those defined in published articles.
                One hundred two trials with 122 published journal articles and 3736 outcomes were identified. Overall, 50% of efficacy and 65% of harm outcomes per trial were incompletely reported. Statistically significant outcomes had a higher odds of being fully reported compared with nonsignificant outcomes for both efficacy (pooled odds ratio, 2.4; 95% confidence interval [CI], 1.4-4.0) and harm (pooled odds ratio, 4.7; 95% CI, 1.8-12.0) data. In comparing published articles with protocols, 62% of trials had at least 1 primary outcome that was changed, introduced, or omitted. Eighty-six percent of survey responders (42/49) denied the existence of unreported outcomes despite clear evidence to the contrary.
                The reporting of trial outcomes is not only frequently incomplete but also biased and inconsistent with protocols. Published articles, as well as reviews that incorporate them, may therefore be unreliable and overestimate the benefits of an intervention. To ensure transparency, planned trials should be registered and protocols should be made publicly available prior to trial completion. To determine if inadequate approaches to randomized controlled trial design and execution are associated with evidence of bias in estimating treatment effects.
                An observational study in which we assessed the methodological quality of 250 controlled trials from 33 meta-analyses and then analyzed, using multiple logistic regression models, the associations between those assessments and estimated treatment effects.
                Meta-analyses from the Cochrane Pregnancy and Childbirth Database.
                The associations between estimates of treatment effects and inadequate allocation concealment, exclusions after randomization, and lack of double-blinding.
                Compared with trials in which authors reported adequately concealed treatment allocation, trials in which concealment was either inadequate or unclear (did not report or incompletely reported a concealment approach) yielded larger estimates of treatment effects (P < .001). Odds ratios were exaggerated by 41% for inadequately concealed trials and by 30% for unclearly concealed trials (adjusted for other aspects of quality). Trials in which participants had been excluded after randomization did not yield larger estimates of effects, but that lack of association may be due to incomplete reporting. Trials that were not double-blind also yielded larger estimates of effects (P = .01), with odds ratios being exaggerated by 17%.
                This study provides empirical evidence that inadequate methodological approaches in controlled trials, particularly those representing poor allocation concealment, are associated with bias. Readers of trial reports should be wary of these pitfalls, and investigators must improve their design, execution, and reporting of trials. Homoeopathy is widely used, but specific effects of homoeopathic remedies seem implausible. Bias in the conduct and reporting of trials is a possible explanation for positive findings of trials of both homoeopathy and conventional medicine. We analysed trials of homoeopathy and conventional medicine and estimated treatment effects in trials least likely to be affected by bias.
                Placebo-controlled trials of homoeopathy were identified by a comprehensive literature search, which covered 19 electronic databases, reference lists of relevant papers, and contacts with experts. Trials in conventional medicine matched to homoeopathy trials for disorder and type of outcome were randomly selected from the Cochrane Controlled Trials Register (issue 1, 2003). Data were extracted in duplicate and outcomes coded so that odds ratios below 1 indicated benefit. Trials described as double-blind, with adequate randomisation, were assumed to be of higher methodological quality. Bias effects were examined in funnel plots and meta-regression models.
                110 homoeopathy trials and 110 matched conventional-medicine trials were analysed. The median study size was 65 participants (range ten to 1573). 21 homoeopathy trials (19%) and nine (8%) conventional-medicine trials were of higher quality. In both groups, smaller trials and those of lower quality showed more beneficial treatment effects than larger and higher-quality trials. When the analysis was restricted to large trials of higher quality, the odds ratio was 0.88 (95% CI 0.65-1.19) for homoeopathy (eight trials) and 0.58 (0.39-0.85) for conventional medicine (six trials).
                Biases are present in placebo-controlled trials of both homoeopathy and conventional medicine. When account was taken for these biases in the analysis, there was weak evidence for a specific effect of homoeopathic remedies, but strong evidence for specific effects of conventional interventions. This finding is compatible with the notion that the clinical effects of homoeopathy are placebo effects. The quality of randomized controlled trials (RCTs) in subfertility and their suitability for inclusion in meta-analyses have been assessed in the past and found to be insufficient. Our aim was to assess whether this quality has improved over time, particularly since the publication of the Consolidated Standards of Reporting Trials (CONSORT) statement, and to assess what proportion of trials could be included in the meta-analyses of pregnancy outcomes such as those included in Cochrane Reviews.
                A selection of subfertility trials published in 1990, 1996 and 2002 was collected from the Cochrane Menstrual Disorder and Subfertility Group (MDSG) database. Only trials published in English as full journal articles, claiming to be randomized and reporting on pregnancy outcomes, were included.
                One hundred and sixty-four trials met our inclusion criteria. Twenty-four (15%) were found not to be randomized, despite claims, and only 10 trials (6%) provided adequate details on the methods of randomization and allocation concealment. Of these, only three had sufficient details extractable to allow for an intention-to-treat analysis of the outcome 'live birth'.
                Although an improvement in some subfertility-specific issues was observed, the quality of reporting of RCTs still needs to improve to make them suitable for inclusion in meta-analyses such as those in the Cochrane Library. Free papers presented to the Annual Scientific Congress (ASC) of the Royal Australasian College of Surgeons (RACS) were reviewed for the years 1994, 1995 and 1996. Reports were examined for evidence of publication bias.
                Suitable free papers were identified from the proceedings of the meetings and authors were contacted to obtain information about the research reported and any publications resulting from it.
                Responses were obtained from 302 of 576 presentations considered suitable. A total of 55% of responding authors reported publication of their paper. Basic science papers were most likely to be published. There was a significant bias in favour of publication of positive results (98 of 139 positive vs 76 of 159 inconclusive or negative reports; P < 0.01). Retrospective data were as likely to be published as prospective (51% and 57%, respectively). Reports describing studies of high-level evidence were more likely to be published in journals with a high impact factor.
                The ASC is a comprehensive meeting that attracts a wide range of free papers from most sections of the RACS. There appears to be no evidence of bias in selection of papers for inclusion in the meeting but there is bias in the subsequent publication, which favours positive reports. Stents are commonly used to treat patients with coronary artery disease. However, the quality of reporting internal and external validity data in published reports of randomised controlled trials (RCTs) of stents has never been assessed.The objective of our study was to evaluate the quality of reporting internal and external validity data in published reports of RCTs assessing the stents for percutaneous coronary interventions.
                A systematic literature review was conducted. Reports of RCTs assessing stents for percutaneous coronary interventions indexed in MEDLINE and the Cochrane Central Register of Controlled Trials and published between January 2003 and September 2008 were selected. A standardized abstraction form was used to extract data. All analyses were adjusted for the effect of clustering articles by journal.
                132 articles were analyzed. The generation of the allocation sequence was adequate in 58.3% of the reports; treatment allocation was concealed in 34.8%. Adequate blinding was reported in one-fifth of the reports. An intention-to-treat analysis was described in 79.5%. The main outcome was a surrogate angiographic endpoint in 47.0%. The volume of interventions per center was described in two reports. Operator expertise was described in five (3.8%) reports. The quality of reporting was better in journals with high impact factors and in journals endorsing the CONSORT statement.
                The current reporting of results of RCTs testing stents needs to be improved to allow readers to appraise the risk of bias and the applicability of the results. The goal was to assess the risk of bias among pediatric, randomized, controlled trials (RCTs) reported in 8 high-impact journals.
                We searched PubMed for all pediatric RCTs reported between July 1, 2007, and June 30, 2008, in 8 journals with high impact factors. Using Cochrane Collaboration methods for risk assessment, we evaluated all reports for risk of bias according to domain (ie, randomized sequence generation, allocation concealment, masking, incomplete outcome data, selective outcome reporting, and other). We used multiple logistic regression to test for associations between the presence of a high risk of bias according to domain and funding source, intervention type, trial registration, and multicenter status.
                Industry-funded RCTs were more likely to show a high risk of bias for sequence generation, compared with government-funded RCTs (adjusted odds ratio [aOR]: 6.1 [95% confidence interval [CI]: 1.70- 21.89]), and behavioral/educational trials were more likely to show a high risk of bias for sequence generation (aOR: 2.8 [95% CI: 1.06-7.36]) and allocation concealment (aOR: 4.09 [95% CI: 1.69-9.90]), compared with drug trials. Registered trials were less likely to have a high risk of bias for sequence generation, compared with nonregistered trials (aOR: 0.33 [95% CI: 0.15-0.71]).
                Overall, we found a large proportion of pediatric RCT reports with a high risk of bias for sequence generation and allocation concealment. Factors associated with a high risk of bias included industry funding and assessment of behavioral/educational interventions, whereas trial registration was associated with a lower risk of bias. The reporting of outcomes within published randomized trials has previously been shown to be incomplete, biased and inconsistent with study protocols. We sought to determine whether outcome reporting bias would be present in a cohort of government-funded trials subjected to rigorous peer review.
                We compared protocols for randomized trials approved for funding by the Canadian Institutes of Health Research (formerly the Medical Research Council of Canada) from 1990 to 1998 with subsequent reports of the trials identified in journal publications. Characteristics of reported and unreported outcomes were recorded from the protocols and publications. Incompletely reported outcomes were defined as those with insufficient data provided in publications for inclusion in meta-analyses. An overall odds ratio measuring the association between completeness of reporting and statistical significance was calculated stratified by trial. Finally, primary outcomes specified in trial protocols were compared with those reported in publications.
                We identified 48 trials with 68 publications and 1402 outcomes. The median number of participants per trial was 299, and 44% of the trials were published in general medical journals. A median of 31% (10th-90th percentile range 5%-67%) of outcomes measured to assess the efficacy of an intervention (efficacy outcomes) and 59% (0%-100%) of those measured to assess the harm of an intervention (harm outcomes) per trial were incompletely reported. Statistically significant efficacy outcomes had a higher odds than nonsignificant efficacy outcomes of being fully reported (odds ratio 2.7; 95% confidence interval 1.5-5.0). Primary outcomes differed between protocols and publications for 40% of the trials.
                Selective reporting of outcomes frequently occurs in publications of high-quality government-funded trials. To study the efficacy, safety, and compliance of short-term electro-acupuncture for children with autism spectrum disorder (ASD).
                Randomized, double-blind, sham-controlled, clinical trial.
                Children with ASD were randomly assigned to an electro-acupuncture (EA) group (n=30) or a sham electro-acupuncture (SEA) group (n=25) matched by age and severity of autism. The EA group received electro-acupuncture for selected acupoints while the SEA group received sham electro-acupuncture to sham acupoints. A total of 12 EA and SEA sessions over four weeks were given. Primary outcome measures included Functional Independence Measure for Children (WeeFIM), Pediatric Evaluation of Disability Inventory (PEDI), Leiter International Performance Scale-Revised (Leiter-R), and Clinical Global Impression-Improvement (CGI-I) scale. Secondary outcome measures consisted of Aberrant Behavior Checklist (ABC), Ritvo-Freeman Real Life Scale (RFRLS), Reynell Developmental Language Scale (RDLS), and a standardized parental report. Data were analyzed by the Mann-Whitney test.
                There were significant improvements in the language comprehension domain of WeeFIM (p=0.02), self-care caregiver assistant domain of PEDI (p=0.028), and CGI-I (p=0.003) in the EA group compared to the SEA group. As for the parental report, the EA group also showed significantly better social initiation (p=0.01), receptive language (p=0.006), motor skills (p=0.034), coordination (p=0.07), and attention span (p=0.003). More than 70 percent of children with ASD adapted to acupuncture easily, while eight percent had poor acupuncture compliance. Mild side effects of minor superficial bleeding or irritability during acupuncture were observed.
                A short, four-week (12 sessions) course of electro-acupuncture is useful to improve specific functions in children with ASD, especially for language comprehension and self-care ability. Recent years have seen an increase in industry sponsorship of clinical trials throughout medicine. We conducted a study to evaluate the association of industry sponsorship to published outcomes in gastrointestinal (GI) clinical research. Our aims were (1) to evaluate the trends in the source of funding for GI clinical research during the period from 1992 to 2002-2003, (2) to determine whether the source of study funding predicted the likelihood that a study would publish results that favor the drug or device being tested, and (3) to determine whether differences exist in the methodologic quality of the investigational study methods used in studies funded by private industry versus other sources.
                We selected all clinical studies evaluating a drug or device from 4 prominent GI journals (Gastroenterology, The American Journal of Gastroenterology, Hepatology, and Gastrointestinal Endoscopy). All studies were abstracted by using a standardized data abstraction form. We evaluated the trends in funding source for studies published during the years 1992, 2002, and 2003. All selected studies were scored for methodologic quality by using a previously validated scoring system. The percentage of studies that reported outcomes that favored the device or drug being tested against the standard therapy or placebo was determined for each funding source. A funnel plot was constructed to assess for the presence of negative publication bias.
                A total of 6326 studies were reviewed. For the 1992 trend data, 1860 studies were reviewed, and 135 were selected for inclusion in the study. Ninety-five studies were studies involving the investigation of drugs, and 40 studies involved the testing of a device. For the 2002-2003 data 4466 studies were reviewed, and 315 were selected for inclusion in the study. Two hundred twenty-two studies were clinical trials involving the investigation of drugs, and 93 were clinical trials involving devices. In comparing 1992 to 2002, the percentage of studies funded by industry sources more than doubled from 10% to greater than 28% of the total studies assessed. There was an associated decline in the proportion of studies with funding from non-industry sources during this period (62% to less than 48%). The percentage of studies that did not disclose a funding source fell modestly from 28% to 24%. We found that 86% of studies funded by private industry reported a result favorable to the study drug or device, and 83% of studies funded by academic sources reported a result favorable to the study drug or device (P=.572). On average, studies funded by private industry had a higher methodology score than studies funded by traditional academic sources (75 of 100 vs 65 of 100; P=.005). Analysis of the funnel plot did not reveal evidence of bias against the publication of small studies with insignificant results.
                The proportion of research funded by industry has more than doubled during the last decade and currently comprises almost half of the funding for GI clinical research. Industry-sponsored studies are, on average, of superior methodologic quality to studies funded by other sources. Industry-sponsored studies in leading GI journals were no more likely than other studies to publish results that favored the study sponsor, although an extremely high percentage of all studies in these journals reported positive results. There has been only a modest decline in studies not acknowledging a funding source.
On the basis of one randomised trial which did not describe its methodology in sufficient detail to assess risk of bias and quality, excluded 27% of patients after randomisation due to various reasons, and is probably not free from selective outcome reporting bias, there is insufficient evidence to conclude that in patients with colonic cancer liver metastases, electro-coagulation alone brings any significant benefit in terms of survival or recurrence compared with the control. In addition, there is insufficient evidence for the effectiveness of adding allopurinol or dimethyl sulphoxide to electro-coagulation. The probability for selective outcome reporting bias in the trial is high. More randomised trials are needed in order to sufficiently validate electro-coagulation with or without co-interventions.