Quality of reporting internal and external validity data from randomized controlled trials evaluating stents for percutaneous coronary intervention. Beyond Medline: reducing bias through extended systematic review search. Reporting randomised clinical trials of analgesics after traumatic or orthopaedic surgery is inadequate: a systematic review. Lessons for search strategies from a systematic review, in The Cochrane Library, of nutritional supplementation trials in patients after hip fracture. Identification and analysis of randomised controlled trials in nursing: a preliminary study. What is the quality of reporting in weight loss intervention studies? A systematic review of randomized controlled trials. Randomized controlled trials in pediatric surgery: could we do better? Assessing the quality of reports of randomized clinical trials: is blinding necessary? The reporting quality of randomised controlled trials in surgery: a systematic review. The quality of reporting of randomized controlled trials of traditional Chinese medicine: a survey of 13 randomly selected journals from mainland China. Systematic review of symptom control and quality of life in studies on chemotherapy for advanced non-small cell lung cancer: how CONSORTed are the data?
Stents are commonly used to treat patients with coronary artery disease. However, the quality of reporting internal and external validity data in published reports of randomised controlled trials (RCTs) of stents has never been assessed.The objective of our study was to evaluate the quality of reporting internal and external validity data in published reports of RCTs assessing the stents for percutaneous coronary interventions.
                A systematic literature review was conducted. Reports of RCTs assessing stents for percutaneous coronary interventions indexed in MEDLINE and the Cochrane Central Register of Controlled Trials and published between January 2003 and September 2008 were selected. A standardized abstraction form was used to extract data. All analyses were adjusted for the effect of clustering articles by journal.
                132 articles were analyzed. The generation of the allocation sequence was adequate in 58.3% of the reports; treatment allocation was concealed in 34.8%. Adequate blinding was reported in one-fifth of the reports. An intention-to-treat analysis was described in 79.5%. The main outcome was a surrogate angiographic endpoint in 47.0%. The volume of interventions per center was described in two reports. Operator expertise was described in five (3.8%) reports. The quality of reporting was better in journals with high impact factors and in journals endorsing the CONSORT statement.
                The current reporting of results of RCTs testing stents needs to be improved to allow readers to appraise the risk of bias and the applicability of the results. To evaluate the sensitivity and precision of various extended search methods in identifying randomized controlled trials (RCTs) for systematic reviews.
                Prospective analysis of extended search methods (specialized databases or trial registries, reference lists, hand-searching, personal communication, and Internet) used in two systematic reviews of RCTs. The gold standard was the total number of RCTs identified by major databases (MEDLINE, EMBASE, etc.) and extended search strategies combined. Sensitivity was the proportion of all known RCTs identified by any extended search method. Precision reflected the proportion of all items uncovered by any extended search method that actually were RCTs.
                The extended search identified 94 additional RCTs for the systematic reviews beyond those identified with the major databases. Specialized databases and trial registries had the highest sensitivity and precision for the lipid-lowering project (13.6% and 52.7%, respectively; p < .05) followed by scanning of reference lists (7.2% sensitivity and 41.9% precision; p <.05). Hand-searching was more effective than personal communication and Internet searching (1.7% sensitivity and 12.2% precision; p < .05). The acupuncture project had slightly different results, with the specialized databases and trial registries tied with the review of reference lists for highest sensitivity (14.2%). The precision followed the same trend as the lipid-lowering project (17.6% specialized databases; 8.3% reference lists; p < .05). A post-hoc analysis showed that 75 of the 94 RCTs were indexed in the major databases but missed by the major database search.
                Extended searching identified additional RCTs for the systematic reviews beyond those found in major databases. Specialized databases and trial registries were most effective. An important number of RCTs were missed by the major database search. Timing and accuracy of indexing may explain this finding. The definitive measure, whether there is an association between the method used to uncover RCTs, the quality of the items uncovered and their impact on systematic review results, is yet to be determined. Several randomised clinical trials (RCTs) of analgesics in postoperative pain after traumatic or orthopaedic surgery (TOS) have been published, but no studies have assessed the quality of these reports. We aimed to examine the quality of reporting RCTs on analgesics for postoperative pain after TOS.
                Reports of RCTs assessing analgesics in postoperative pain after TOS were systematically searched from electronic databases. The quality of reports was assessed using the CONSORT checklist (scoring range from 0 to 22). The quality was considered poor when scoring was 12 or lesser. The publication year and the impact factor of journals were recorded.
                A total of 92 reports of RCTs were identified and 69 (75%) scored 12 or lesser in CONSORT checklist (range 5-17). The mean (SD) CONSORT score of all reports was 10.6 (2.7). Missing CONSORT items included primary and secondary outcome measures (11%), the specific objectives and hypothesis definition (12%), the sample size calculation (12%), the dates defining the periods of recruitment (12%), the discussion of external validity of findings (14%), the allocation sequence generation (24%), and the interpretation of potential bias or imprecision of results (25%). There was a little improvement in CONSORT scores over time (r = 0.62; p < 0.001) and with impact factor of journals (r = 0.30; p < 0.001).
                Quality of reporting RCTs on analgesics after TOS is poor. Reporting of those RCTs should be improved according to methodological standard checklists in the next years. A key aim when conducting systematic reviews of randomized controlled trials (RCTs) is to include all of the evidence, if possible. Serious bias may result if trials are missed through inadequate search strategies.
                The objective was to evaluate the search plan for identifying RCTs in nutrition as part of a systematic review, in The Cochrane Library, of nutritional supplementation trials in patients after hip fracture.
                We identified potential studies by searching the electronic databases BIOSIS, CABNAR, CINAHL, EMBASE, HEALTHSTAR, and MEDLINE; reference lists in trial reports; and other relevant articles. We also contacted investigators and other experts for information and searched 4 nutrition journals by hand.
                We identified 15 RCTs that met the predefined inclusion criteria. The search plan identified 8 trials each in EMBASE, HEALTHSTAR, and MEDLINE and 7 in BIOSIS and CABNAR. BIOSIS was the only electronic database source of 2 trials. Eleven trials were identified by searching electronic databases and 2 unpublished trials were identified via experts in the field. We found one trial, published only as a conference abstract, by searching nutrition journals by hand. After publication of the protocol for the review in The Cochrane Library, we were informed of another unpublished trial.
                We found that a limited search plan based on only MEDLINE or one of the other commonly available databases would have failed to locate nearly one-half of the studies. To protect against bias, the search plan for a systematic review of nutritional interventions should be comprehensive. To describe preliminary work undertaken for development of a nursing contribution to the Cochrane Collaboration. To ascertain whether there are randomised controlled trials (RCTs) on nursing care which need to be identified for inclusion in systematic reviews of the effects of health care.
                Searches by Medline (1966-94) and by hand of 11 nursing research journals from inception to the end of 1994 to identify RCTs and systematic reviews; and a comparison of searches by hand and by Medline for three nursing research journals.
                Total number of RCTs identified and number of RCTs published in nursing journals; the sensitivity of Medline searches; and aspects of nursing care evaluated by RCT.
                The work is ongoing and 522 reports of RCTs and 20 systematic reviews of effectiveness have been identified so far. The sensitivity of Medline searches for RCTs in nursing journals is as low as 36% for one journal and the lack of reference to research design in the title or abstract was the main reason for the lack of sensitivity.
                There are RCTs that evaluate aspects of nursing care, and are published in nursing and non-nursing journals, and are largely undertaken by nurses. These must be reviewed in ongoing systematic reviews of the effects of health care (including those undertaken as part of the Cochrane Collaboration). Nursing journals must be hand searched to identify these studies as the lack of reference to study design in the titles and abstracts of nursing trials leads to poor indexing in electronic databases such as Medline. Despite the large number of randomized controlled trials (RCTs) assessing weight loss interventions, no study has assessed the quality of reporting in these trials.
                To assess the quality of reporting of RCTs of weight loss interventions and to identify predictors of reporting quality.
                The RCTs assessed were derived from a published systematic review of trials investigating the efficacy of weight loss interventions. For our study, two reviewers independently rated the quality of reporting in these trials, based on the Consolidated Standards for Reporting of Trials (CONSORT) Statement. We describe the quality of reporting using number (percent) of studies satisfying each of the 44 CONSORT criteria. We use generalized estimating equations (GEE) to fit a multivariable regression model to determine factors that are associated with the overall quality reporting score.
                We assessed 63 RCTs, of which 25 were dietary-lifestyle trials, 22 were pharmacological trials and 16 were behavior-cognitive, exercise-lifestyle, or surgical trials. Less than half (46%) of the trials defined the primary outcome of the study; about 10% provided the description of the method of allocation concealment. Multivariable GEE results showed that the sample size, type of intervention (non-pharmacologic trials having lower scores than pharmacologic trials), and publication time relative to the CONSORT Statement publication in 1996 (publications after 1996 having higher scores) were strong predictors of the quality reporting score. Reporting a statistically significant result on the primary outcome was not significantly associated with the quality score.
                While the overall quality in reporting seemed to have improved since the publication of the revised CONSORT Statement in 1996, the reporting of some key methodologic aspects, such as clear description of primary outcome and method of allocation concealment, still requires improvements. Factors that are significantly associated with the overall quality reporting score can be used as surrogates in the review of protocols to enhance the quality of the final reports. Randomized controlled trials (RCTs) are accepted as the gold standard for assessing the effectiveness of clinical interventions but are rarely reported in pediatric surgery. Have RCTs submitted to the British Association of Paediatric Surgeons (BAPS) Annual Congress during the last 5 years been adequately designed and large enough to produce a valid result?
                Abstracts accepted by the Annual BAPS Congress meetings between 1996 and 2000 were examined in collaboration with a senior health services researcher. The quality of the design, methodology, statistical analysis and conclusions, and the adequacy of the sample size were assessed for all identifiable clinical RCTs.
                From 760 accepted abstracts, there were only 9 RCTs (1%) of clinical interventions. In only 4 trials was the relevant primary end-point specified at the outset of the study, and none documented the method of randomization. Only one abstract mentioned blinding with respect to the intervention or outcome measure. Sample sizes were inadequate to detect even large clinical differences. To date, only one of these RCTs has been published in an English-language, peer-reviewed journal.
                Clear guidelines exist for the conduct of RCTs, yet compliance with these standards was rarely documented in abstracts of pediatric surgical RCTs presented at BAPS. Sample sizes were inadequate. RCTs in pediatric surgery are difficult to perform, but the specialty would benefit from well-designed, carefully conducted, multicentre, clinical RCTs to advance evidence-based practice.
                Copyright 2003, Elsevier Science (USA). All rights reserved. It has been suggested that the quality of clinical trials should be assessed by blinded raters to limit the risk of introducing bias into meta-analyses and systematic reviews, and into the peer-review process. There is very little evidence in the literature to substantiate this. This study describes the development of an instrument to assess the quality of reports of randomized clinical trials (RCTs) in pain research and its use to determine the effect of rater blinding on the assessments of quality. A multidisciplinary panel of six judges produced an initial version of the instrument. Fourteen raters from three different backgrounds assessed the quality of 36 research reports in pain research, selected from three different samples. Seven were allocated randomly to perform the assessments under blind conditions. The final version of the instrument included three items. These items were scored consistently by all the raters regardless of background and could discriminate between reports from the different samples. Blind assessments produced significantly lower and more consistent scores than open assessments. The implications of this finding for systematic reviews, meta-analytic research and the peer-review process are discussed. In order to examine the reporting quality of urological RCTs, the initial objectives of this study were to evaluate the degree to which RCTs involving urological surgical techniques (as the intervention) published in the years 2000-2003 complied with the CONSORT statement, and to assess trends and patterns of compliance. Following our initial findings in urology, we extended the methodology to a number of other specialties to assess whether our findings in urology could be generalised to other surgical disciplines.
                The Royal Society of Medicine (RSM) Library was commissioned to search the Medline and Cochrane databases for RCTs in compliance with the study inclusion criteria below. Additional analyses of five other specialties (non-urological trials: cardiovascular, gastrointestinal, hepatic, orthopaedic and vascular) were also made. For the non-urological trials, 15 English language trials (from each specialty) were randomly picked from a blast search conducted by the RSM for the year 2003. The RCTs were given a score out of 22, reflecting how many of the 22 CONSORT items were complied with (with each item being given equal weighting), this score was termed the "CONSORT score".
                Urology trials: In total 122 RCT abstracts were identified which met the inclusion criteria for this study. From these, 32 were excluded as they were follow-up studies, involved a virtual procedure or were a cost analysis, leaving 90 RCTs published across 35 different journals which were analysed (68.2%, 90/122). The average score of 11.1 for urological trials indicates that RCTs in our sample do not comply with the CONSORT statement. No trials reported how they implemented their randomisation process. Only 46% of RCTs stated that they had permission from an ethics review board, 20% had declared their sources of funding, 14% stated whether there were any conflicts of interest and only 1.1% stated their trial registry number. For non-urological studies, an average CONSORT score of 11.2 was obtained.
                Clinical research teams conducting RCTs in urology and other surgical disciplines demonstrate poor compliance with the CONSORT statement. We would recommend that trials should be registered at their outset and that urological and other surgical journals to consider supporting the CONSORT statement and to have compliance 'hard-wired' into their submission, editorial and peer-review processes. Since it seems the best researchers are unable to produce an RCT results which enable surgical techniques to be critically assessed, there is a need for education about the CONSORT statement and its importance at all levels of surgical training. We believe that an open debate is needed on the possible role of other research designs, such as tracker studies. Whether this study actually raises the question of how appropriate RCTs are to surgical techniques, we leave to the reader. The number of randomized controlled trials (RCTs) of traditional Chinese medicine (TCM) is increasing. However, there have been few systematic assessments of the quality of reporting of these trials. Objective: This study was undertaken to evaluate the quality of reporting of RCTs in TCM journals published in mainland China from 1999 to 2004.
                Thirteen TCM journals were randomly selected by stratified sampling of the approximately 100 TCM journals published in mainland China. All issues of the selected journals published from 1999 to 2004 were hand-searched according to guidelines from the Cochrane Centre. All reviewers underwent training in the evaluation of RCTs at the Chinese Centre of Evidence-based Medicine. A comprehensive quality assessment of each RCT was completed using a modified version of the Consolidated Standards of Reporting Trials (CONSORT) checklist (total of 30 items) and the Jadad scale. Disagreements were resolved by consensus.
                Seven thousand four hundred twenty-two RCTs were identified. The proportion of published RCTs relative to all types of published clinical trials increased significantly over the period studied, from 18.6% in 1999 to 35.9% in 2004 (P < 0.001). The mean (SD) Jadad score was 1.03 (0.61) overall. One RCT had a Jadad score of 5 points; 14 had a score of 4 points; and 102 had a score of 3 points. The mean (SD) Jadad score was 0.85 (0.53) in 1999 (746 RCTs) and 1.20 (0.62) in 2004 (1634 RCTs). Across all trials, 39.4% of the items on the modified CONSORT checklist were reported, which was equivalent to 11.82 (5.78) of the 30 items. Some important methodologic components of RCTs were incompletely reported, such as sample-size calculation (reported in 1.1% of RCTs), randomization sequence (7.9%), allocation concealment (0.3 %), implementation of the random-allocation sequence (0%), and analysis of intention to treat (0%).
                The findings of this study indicate that the quality of reporting of RCTs of TCM has improved, but remains poor. The effect of chemotherapy on survival of patients with advanced NSCLC is modest, therefore patient reported outcomes (PRO's) are of high interest in randomized controlled trials (RCTs). CONSORT (CONsolidated Standards On Reporting Trials) is a quality checklist of 22 items for the conduct and reporting of RCTs. The aim of this report was to analyse to what extent the different RCTs with information on PRO's adhere to the CONSORT statement.
                Systematic review of RCTs using PRO's either as primary or secondary endpoint. Compliance with the (revised) CONSORT statement was checked by 2 independent reviewers by making for each study the simple sum of the 22 CONSORT items, or a weighted score with a maximum rating of 31 points.
                The median weighted CONSORT score of the different RCTs was 25, with a remarkable difference from 12 till 30. There was no significant change over time, nor difference between academic and commercial studies, but a significant correlation between CONSORT agreement and journal type (P<0.0001). Adherence to CONSORT was similar for studies comparing chemotherapy with best supportive care alone, comparing different first-line chemotherapies with PRO either as primary or secondary endpoint, or studies looking at second-line chemotherapy. Benefit in PRO's was reported in all of these settings.
                The overall adherence of peer-reviewed RCTs to CONSORT is reasonable, with nonetheless major differences between journals, and with no clear sign of change over time. Apart from modest survival differences, benefits in PRO endpoints are present in all categories of studies we analysed.
This systematic review suggests that the use of external stents following pancreaticoduodenectomy may be beneficial. However, only a limited number of RCTs with rather small sample sizes were available. Further RCTs on the use of stents after pancreaticoduodenectomy are warranted.