To investigate whether alignment of strength of recommendations with quality of evidence differs in consensus based versus evidence based guidelines.Empirical analysis.Guidelines developed by the American College of Cardiology and the American Heart Association (ACC/AHA) and the American Society of Clinical Oncology (ASCO) up to 27 March 2021.Recommendations were clearly categorised as consensus or evidence based, were separated from the remainder of the text, and included both the quality of evidence and the strength of the recommendations.Paired authors independently extracted the recommendation characteristics, including type of recommendation (consensus or evidence based), grading system used for developing recommendations, strength of the recommendation, and quality of evidence. The study team also calculated the number of discordant recommendations  and inappropriate discordant recommendations .The study included 12 ACC/AHA guidelines that generated 1434 recommendations and 69 ASCO guidelines that generated 1094 recommendations. Of the 504 ACC/AHA recommendations based on low quality evidence, 200 (40%) proved to be consensus based versus 304 (60%) evidence based; of the 404 ASCO recommendations based on low quality evidence, 292 (72%) were consensus based versus 112 (28%) that were evidence based. In both ACC/AHA and ASCO guidelines, the consensus approach yielded more discordant recommendations  and inappropriate discordant recommendations  than the evidence based approach.Consensus based guidelines produce more recommendations violating the evidence based medicine principles than evidence based guidelines. Ensuring appropriate alignment of quality of evidence with the strength of recommendations is key to the development of “trustworthy” guidelines. Development of trustworthy guidelines—a collection of related recommendations for clinical practice—is key to improving physicians’ decision making and patients’ outcomes.Organisations adopt different grading systems to rate the quality of evidence and the strength of recommendations, of which GRADE The failure to ensure concordance between quality of evidence and strength of recommendations violates a key principle of evidence based medicine and risks misleading guidance.Box 1Example: In patients with life threatening disseminated blastomycosis, use of amphotericin which is more toxic , but might reduce mortality , in comparison with itraconazole. After considering the life threatening situation, the guideline panel made a strong recommendation to support the use of amphotericin.Example: Patients requiring surgery for symptomatic cataract can undergo either retrobulbar anaesthesia or topical anaesthesia. Retrobulbar anaesthesia might offer some uncertain benefit over topical anaesthesia, supported with low quality evidence . Moderate to high certainty evidence suggests, however, that retrobulbar anaesthesia is associated with substantial harm, including chemosis, periorbital haematoma and even, severe, life threatening complications. Because the harms are judged to outweigh benefits, the guideline committee issued a strong recommendation against retrobulbar anaesthesia rather than topical anaesthesia for patients undergoing cataract surgery.Helicobacter pylori bacteria, but historically, MALT lymphoma has been treated with radiation or gastrectomy. Low quality of evidence shows similar response (benefits) with antibiotics for H pylori but clearly less harm, morbidity, and cost . Based on the assessment that benefits of alternative treatments are similar, but the harms associated with antibiotics are plainly lower than those from gastrectomy or radiation, the guideline authors issued a strong recommendation in favour of eradication of H pylori bacteria with antibiotics rather than radiation therapy or gastrectomy in patients with MALT lymphoma.Example: Extranodal marginal zone lymphoma of mucosa associated lymphoid tissue ((MALT lymphoma) is linked to infection with Example: Whether patients with subclinical hypothyroidism benefit from thyroid hormones more than if they are left untreated, is uncertain. With or without treatment patient have similar benefits for quality of life or thyroid related symptoms, including depressive symptoms, fatigue, and body mass index . Low quality evidence suggests, however, that the administration of thyroid hormones is not harmless. In view of this, the guideline panel issued a strong recommendation against prescribing thyroid hormones for adults with subclinical hypothyroidism.Example: Oral anticoagulants and antiplatelet agents are often prescribed for patients with chronic coronary or other arterial diseases. Low quality evidence shows that the combination of oral anticoagulants and antiplatelet agents is associated with a high risk of serious bleeding. Hence the European Society of Cardiology made a strong recommendation against the use of a combination of oral anticoagulant and antiplatelet agents in patients with chronic coronary or other arterial disease and in favour of treatment with a single oral anticoagulant.In their approach to particular guidelines, some organisations, including ACC/AHA and ASCO, classify their guidelines as evidence based when much of the supporting evidence is deemed moderate or high quality. When most of the evidence is of low or very low quality, these organisations then often categorise their approach as consensus based.Both consensus and evidence based approaches require judicious consideration of the relevant evidence, expert interpretation of the evidence, and ultimately. panel consensus. In other words, guideline panels must always carefully consider the available evidence, regardless of quality, and must always rely on expert insights to arrive at consensus recommendationsIf this is the case, one would expect a greater number of discordant recommendations, particularly inappropriate discordant recommendations, in consensus based guidelines than in evidence based guidelines. To date, however, empirical support for this expected findinghttps://www.acc.org/guidelines#/doctype=Guidelines) and ASCO websites (https://www.asco.org/) on 5 May 2020 and updated the search on 27 March, 2021. Eligible guidelines adhered to the following criteria: (a) explicitly distinguished consensus from an evidence based approach to the development of guidelines; (b) included recommendations; (c) clearly categorised the recommendations as consensus versus evidence based and separated them from the remainder of the text; and (d) included both the quality of evidence and the strength of each recommendation. When several versions of the guidelines existed, we used the most recent.We searched the ACC/AHA .Both ACC/AHA and ASCO develop guidelines according to the following principle: when evidence is sufficient, the panels are instructed to use the evidence based approach to inform recommendations; when evidence is insufficient, the panels use a consensus based approach.Box 2Question: For patients with valvular heart disease for whom intervention is contemplated, should individual risks be assessed?Recommendation: For patients with valvular heart disease for whom intervention is contemplated, individual risks should be calculated for specific surgical or transcatheter procedures, using online tools when available, and discussed before the procedure as part of a shared decision making process.(Type: expert opinion (consensus based); evidence quality: level C ; strength of recommendation: 1 (strong))Question: In patients aged >65 years who require aortic valve replacement, should a bioprosthesis or a mechanical valve be offered?Recommendation: In patients aged >65 years who require aortic valve replacement, it is reasonable to choose a bioprosthesis over a mechanical valve.; strength of recommendation: 2a (moderate))Box 3Question: In treatment of cancer related anaemia, what special considerations apply to adult patients with non-myeloid haematologic malignancies who are receiving concurrent myelosuppressive chemotherapy?Recommendation: In patients with myeloma, non-Hodgkin lymphoma, or chronic lymphocytic leukaemia, clinicians should observe the haematologic response to cancer treatment before considering an erythropoiesis stimulating agent.Question: To reduce the need for red blood cell transfusions, should erythropoiesis stimulating agents be offered to patients who have chemotherapy associated anaemia?Recommendation: Depending on clinical circumstances, erythropoiesis stimulating agents can be offered to patients with chemotherapy associated anaemia whose cancer treatment is not curative in intent and whose haemoglobin has declined to 100 g/L. Transfusion of red blood cells is also an option, depending on the severity of the anaemia or clinical circumstances.For recommendations with low quality evidence, two reviewers (LY and MMA) independently assessed whether or not discordant recommendations were or were not inappropriate , and inappropriate discordant recommendation. Because recommendations (lower level units) were nested in guidelines (higher level units), we used multilevel mixed effect logistic regression,Owing to the nature of this work, we did not include patients and public.The search yielded 12 eligible ACC/AHA guidelines, published between 2013 and 2021 (appendix 3), with the median number of recommendations from each guideline 109 (interquartile range (74-139), totalling 1434 recommendations. Among the ASCO guidelines, 69 published between 2012 and 2021 (appendix 3) met the inclusion criteria; the median number of recommendations was 14 (interquartile range 3-34), with a total of 1094 recommendations. ACC/AHA guidelines included 504 recommendations with low quality evidence, of which 200 (40%) were consensus based and 304 (60%) were evidence based; for the ASCO guidelines, 404 recommendations were based on low quality evidence, of which 292 (72%) were based on consensus approach and 112 (28%) were evidence based . AppendiOf recommendations based on low quality evidence, using the consensus based approach ACC/AHA issued strong recommendations for 58% (n=115) versus 38% (n=117) using the evidence based approach . In ACC/AHA guidelines, 98% (n=113) versus 2% (n=2) discordant recommendations were in favour of, than against, interventions using the consensus method, but when relying on an evidence based approach, the proportions of discordant recommendations were 81% (n=95) versus 19% (n=22). ASCO guidelines generated 83% (n=76) versus 17% (n=16) discordant recommendations in favour of versus against interventions when using consensus based methods, but when using evidence based methods, the proportions of discordant recommendations were 60% (n=18) versus 40% .v 31%; odds ratio 2.6, 95% confidence interval 1.7 to 3.7, appendix 5 and v 9%; 5.1, 1.6 to 16.0, appendix 5 and v 25%, appendix 5 and We identified 26 discordant recommendations in ACC/AHA guidelines and 40 in ASCO guidelines that met the five characteristic situations in which issuing discordant recommendations might be justified (κ statistic for agreement between the reviewers was 0.74 for ACC/AHA guidelines and 0.81 for ASCO guidelines). The remaining 206 discordant recommendations in ACC/AHA guidelines and 82 in ASCO guidelines represent inappropriate discordant recommendations . The conv 21%) rather than evidence based methods (50% v 50%). In ACC/AHA guidelines, almost all consensus and evidence based recommendations were in favour of recommended interventions (appendix 6).ASCO guidelines generated more inappropriate discordant recommendations in favour of than against interventions using consensus based —with 2.1 times greater odds in ACC/AHA guidelines, and 2.9 times greater odds in ASCO guidelines—and 2.5 times greater odds of issuing inappropriate discordant recommendations than guidelines that were categorised as evidence based—with 2.6 times greater odds in ACC/AHA guidelines, and 5.1 times greater odds in ASCO guidelines. An additional important finding was the 31% frequency of discordant recommendations in the evidence based ACC/AHA guidelines .Strengths of our study include rigour of data abstraction and analysis. In particular, we used multilevel modelling to obtain an accurate effect when the data were nested.One limitation of our study is that we did not make our own detailed assessment of evidence quality to verify the authors’ rating of quality of evidence. Thus it remains possible that some of the discordant recommendations were not truly discordant—that is, that evidence of moderate or high quality was misclassified as low quality by the authors. Nevertheless, the guidelines panels made their strong recommendations with the understanding that underlying quality evidence was low. Other limitations are related to the use by ACC/AHA and ASCO guidelines of different grading systems to rate the quality of evidence and strength of recommendations, which creates problems in comparing results. Nonetheless, both organisations clearly distinguish the quality (certainty) of evidence from the strength of recommendations. Although reproducibility of rating of quality of evidence might differ between these two organisations, it is clear that when issuing their expert/consensus based recommendations, both organisations have instructed their panels to decouple quality of evidence from strength of recommendations.Some recommendations might have been classified as good practice statements (as characterised by GRADE).In this paper, we focus on strong recommendations based on low quality evidence. Analysis of weak recommendations based on high quality of evidence was beyond the scope of this study, but within the GRADE system would be appropriate whenever high quality evidence showed a close balance between desirable consequences of an intervention and a comparator. In such situations, when net benefit or harm is minimal, differences in patients’ values and preferences would lead to fully informed patients making different choices, and thus appropriate weak recommendations.ASCO guidelines generated a relatively small number of 10 (9%) inappropriate discordant recommendations. The multilevel analysis limits the power of the analysis. These two problems resulted in very wide 95% confidence intervals around the ASCO outcomes. A small number of discordant inappropriate recommendations also prevented full scale multilevel analysis; as a result, we reported only descriptive analyses for against versus in favour of intervention. This study was conducted because insights obtained from preliminary analyses informed the more sophisticated and detailed analyses that followed; therefore, the study was not preregistered as the protocol.Three previous studies by three different organisations identified discordant recommendations and calculated how many were inappropriate. A study of World Health Organization guidelines reported that of 302 recommendations supported by low or very low quality evidence, 160 (53%) were discordant and 73 (24%) were inappropriate.Because WHO, the Endocrine Society, and UpToDate issue only evidence based recommendations, the appropriate comparators in our study are evidence based recommendations from ASCO and ACC/AHA. Thus, the performance of ASCO for evidence based recommendations (9% inappropriate) falls between UpToDate and the other two organisations. In comparison, 31% of ACC/AHA inappropriate discordant recommendations were evidence based.No previous study has compared discordant recommendations in evidence based versus consensus based guidelines.Two leading professional organisations issue guidelines that they label as “consensus”—and recommendations categorised as “evidence based”—that include many inappropriate discordant recommendations, which raises serious concerns.It is further problematic that the guideline development organisations seem to have a deliberate policy of making strong recommendations when evidence is only of low quality. According to the ACC/AHA and ASCO guideline methodology,In doing so, these organisations run the risk of patients receiving treatment which, with shared decision making and acknowledging the uncertainty of evidence, they would not have otherwise chosen. The reason for this risk is that strong recommendations are intended as “just do it” guidance, in which a panel has concluded that all, or almost all, fully informed patients would make the same choice. These recommendations are therefore intended to help physicians, all of whom are time constrained, take the time for shared decision making when it is most important—including when low quality evidence provides a decision by considering the patient’s value and preference. Inappropriate strong recommendations might have other problematic consequences, including constraining future randomised trials that would generate higher quality evidence.Our results also show that classifying a guideline as evidence based provides only limited protection against inappropriate discordant recommendations. These occurred in 9% of ASCO evidence based guidelines informed by low quality evidence and 31% of corresponding ACC/AHA evidence based guidelines. This finding might indicate a conscious alternative perspective on guideline recommendations: those strong recommendations are appropriate even when one is uncertain about benefits and harms. Alternatively, it might indicate persistence of a tradition of strong recommendations based on the convictions of expert panellists.The consensus methodology employed by ACC/AHA and ASCO guidelines was associated with inappropriate discordant recommendations, which might lead to patients receiving interventions for which the benefits do not clearly outweigh the harms, and that might be inconsistent with their values and preferences. Our results support re-examining the wisdom of developing consensus based guidelines, and promoting enhanced rigour to ensure, whatever label the recommendations are given, appropriate alignment of quality of evidence with strength of recommendations.Many organisations explicitly classify their guidelines as evidence based when much of the supporting evidence is deemed moderate or high quality; when most of the evidence is of low quality or very low quality, these organisations often categorise their approach as expert or consensus basedA central principle of evidence based medicine is that the strength of recommendations should be consistent with the underlying quality of evidenceIt is not clear if consensus based guidelines violate this key evidence based medicine principle—that is, whether alignment of strength of recommendations with quality of evidence differs in consensus based versus evidence based guidelinesConsensus based guidelines generated more inappropriate strong recommendations than evidence based guidelinesFor both evidence based and consensus based guidelines, it is important to ensure appropriate alignment of quality of evidence with strength of recommendationsAmerican College of Cardiology/American Heart Association and the American Society of Clinical Oncology—two of the largest world organisations that develop guidelines for heart disease and cancer (leading causes of death), produce overall 41% and 20% inappropriate and probably harmful recommendations based on low quality evidence; when they use consensus rather than evidence based methods they have 2.6 and 5.1 times greater odds of issuing inappropriate discordant recommendations, respectively


We shall propose suggestions and reflections for future research, in order to provide a more detailed reference for future development of RAGs.Rapid Advice Guidelines (RAG) provide decision makers with guidance to respond to public health emergencies by developing evidence-based recommendations in a short period of time with a scientific and standardized approach. However, the experience from the development process of a RAG has so far not been systematically summarized. Therefore, our working group will take the experience of . Only two of the ten recommendations were fully supported by direct evidence for COVID-19, three recommendations were supported by indirect evidence only, and the proportion of COVID-19 studies among the body of evidence in the remaining five recommendations ranged between 10 and 83%. Six of the ten recommendations used COVID-19 preprints as evidence support, and up to 50% of the studies with direct evidence on COVID-19 were preprints.The development of the RAG by a group of 67 researchers from 11 countries took 50 days from the official commencement of the work  to submission . A total of 21 meetings were held with a total duration of 48 h (average 2.3 h per meeting) and an average of 16.5 participants attendingIn order to respond to public health emergencies, the development of RAG also requires a clear and transparent formulation process, usually using a large amount of indirect and non-peer-reviewed evidence to support the formation of recommendations. Strict following of the WHO RAG handbook does not only enhance the transparency and clarity of the guideline, but also can speed up the guideline development process, thereby saving time and labor costs.The online version contains supplementary material available at 10.1186/s12874-022-01545-5. According to the World Health Organization (WHO), standard clinical practice guidelines (hereafter referred to as "standard guidelines") must be developed clearly and transparently following a rigorous methodology to obtain unbiased and high-quality recommendations and minimize the influence of conflicts of interest. The development of standard guidelines however will require on average about six months to two years to complete , 2. Due Since December 2019, a novel infectious disease (COVID-19) caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection has swept through the world, posing significant challenges to public health safety and healthcare delivery systems worldwide . In suchThe WHO suggests to consider five factors when deciding to develop a RAG rather than standard guideline, or to delay the development of guideline. If all these five aspects are met, the development of a RAG is considered necessary. Therefore, prior to the official launch of the RAG for children with COVID-19, the core members  of the RAG working group assessed the need and rationale for the development of the RAG for children with COVID-19 based on these five aspects suggested by the WHO [There are currently more than 60 manuals for the development of standard guidelines, including manuals issued by the Institute of Medicine (IOM), Canadian Medical Association (CMA), Scottish Intercollegiate Guidelines Network (SIGN), World Health Organization (WHO), American College of Physicians (ACP), and National Institute for Health and Care Excellence (NICE) , 26–30. The RAG was initiated on January 28, 2020 by the National Clinical Research Center for Child Health and Disorders. On February 3, 2020, one of the CMs (Li W) sent invitation letters to selected experts in the relevant disciplines. Sixty-seven members from 11 countries were invited and indicated their willingness to participate in the development of the guideline. Three working groups were formed . The protocol was published on March 26 of 2020 .Four CMs  conducted a scoping review of the published literatures related to COVID-19, and developed the initial clinical questions related to children with COVID-19 . After tOn February 2, 2020, one CM (Li W) sent a clinical questionnaire to 33 GDG members by email. All experts scored each question from 1 to 7 according to its importance . Twenty-seven (81.8%) of the invited 33 GDG members completed the questionnaire. On February 7, 2020, two CMs  prioritized the clinical questions according to the questionnaire scores and modified the questions based on the feedback from the GDG. After considering the score ranking and discussing the feedback, six CMs  reached an agreement and included ten clinical questions , medRxiv (https://www.medrxiv.org/) and bioRxiv (https://www.biorxiv.org/) preprint platforms and Google Scholar, and tracked references of the included studies.The RRG systematically searched the following databases: Cochrane library, MEDLINE (via PubMed), Embase, Web of Science, CBM (China Biology Medicine), CNKI  and Wanfang Data. The search terms were COVID-19, 2019-CoV, Novel coronavirus, 2019-nCoV, Middle East Respiratory Syndrome Coronavirus, MERS, MERS-CoV, Severe Acute Respiratory Syndrome, SARS, SARS-CoV, and their synonyms, as well as terms related to specific the topic of each review. We developed the search strategy in cooperation with an experienced medical information retrieval specialist , 56. TheSince another high-quality rapid review on the same topic was published, the rapid review on quarantine to control COVID-19 was discontinued on 8 April 2020 , 57. FinBased on the ten clinical questions and supporting evidence from rapid reviews, six CMs  initially drafted ten recommendations. The relevant materials and questionnaires were sent to the GDG members by email in electronic format, and also to the two children’s guardians in paper versions. The preferences and values of the two children’s guardians and clinical experts were consistent. Two rounds of Delphi survey were conducted on February 24, 2020 and February 28, 2020, respectively, and we voted on the recommendations. The consensus criterion was that at least 70% of the invited panelists agreeing on the recommendation , 60. AftRecommendation #3: CT scan should not be used routinely in the diagnosis of COVID-19 in children, although it may be helpful in monitoring children who develop severe respiratory symptoms). Six CMs  revised the wording and rationale of the recommendations based on the first round of Delphi survey. Afterwards, a second round of Delphi survey was conducted for two recommendations: recommendation #3 for which consensus was not reached in the first round, and recommendation #6 . The recommendation was divided into two sub-recommendations  after feedback from the first round. Consensus were reached for both recommendations, resulting in a total of ten recommendations being finally included in the RAG. The panelists made a total of 181 comments during the two rounds of Delphi survey  reporting checklist, the RAG development process, clinical questions, recommendations, evidence summary and research gaps, two CMs  drafted the first version of the RAG , 61. FouIn summary, the development of the RAG by a group of 67 researchers from 11 countries took 50 days from the official commencement of the work  to submission . The process took another 50 days from submission to acceptance, resulting in a total duration of 100 days (Table http://www.guidelines-registry.org/news/141), the Guidelines International Network , and the Emergency Care Research Institute  databases. In addition, we contacted the editor-in-chief in advance and discussed publication strategies to ensure a rapid publication . Considering the differences in health policies and systems, resources, feasibility and equity across the countries, through above strategies, we will assist countries and regions to adopt or adapt the guidelines into their local context.We have translated this guideline into the following 20 languages: English, Chinese, Japanese, Russian, German, French, Italian, Vietnamese, Thai, Spanish, Arabic, Portuguese, Polish, Czech, Romanian, Burmese, Hungarian, Hebrew, Hindi, Turkish and Malay. The guideline is indexed in the International Practice Guidelines Registry Platform  According to the WHO guideline handbook ; 4) We wAs shown in Table Because COVID-19 is an emerging infectious disease that was poorly understood in children and even adults when this work was started, it was extremely challenging to select the top ten clinical questions at the beginning of the development of the RAG. First, there was a severe paucity of literature in children on COVID-19 at that time. Second, the limited number of pediatric cases were all contacts of infected adults, and the transmission routes and dynamics among the children themselves or through other sources (e.g. contaminated surfaces) were unknown. Third, the RAG development groups had no frontline clinicians with relevant clinical experience of the natural history of the disease. Fourth, as an international RAG, the clinical questions needed to be applicable to a wide and heterogeneous target population around the world.The health systems and policies of each country need to be taken into consideration in the process of identifying clinical issues, and the importance of clinical issues vary from country to country. For example, during the survey of clinical questions, the experts proposed the question "who should be tested for nucleic acid at the beginning of an outbreak?". Health policies in some countries have been very clear on this issue . But inThe RAG was supported by 13 rapid reviews that incorporated evidence from studies of COVID-19, SARS and MERS. Table The RAG aims to provide instructional recommendations in the short term, and therefore studies available as preprints were used to support the evidence. Table Engineering on April 1 [International Journal of Antimicrobial Agents on April 3rd due to not meeting the industry consensus criteria [COVID-19 is a global pandemic and the research evidence is growing rapidly. Some of the original studies included in the rapid reviews were retracted for various reasons, resulting in the data being updated again during the production of the rapid review. For example, two studies included in the rapid review for the clinical question #4 , lopinavir /ritonavir or oseltamivir be used to treat children with COVID-19?") were withdrawn, a controlled study on favipiravir was temporarily removed from  April 1 , and anocriteria . BecauseThe registration and publication of the protocol determines in advance the steps and methodology to be followed in developing the RAG according to the WHO handbook. Thorough development and documentation of the methodology, key issues and outcomes reduces the bias in the formulation of recommendations and minimizes the risk of inducing randomness into decision-making. In addition to the above, the registration of the guideline helps to increase the transparency of the development process, avoid duplication, enhance the credibility of the guidelines, and facilitate their dissemination and implementation.The collection and integration of parents’ preferences and values to form recommendations will help families fully understand the pros and cons of various treatment options, so that they have a stronger sense of acceptance and willingness to participate in clinical decision-making. Consideration of families’ preferences and values can thus improve patient compliance, maximize patient benefits, and improve their clinical outcomes . TherefoClinical questions are the starting point for guideline development that determine the scope of evidence to be retrieved and evaluated at a later stage. Clinical questions also determine the content of the final recommendations . Due to The development of guidelines must consider the best currently available research evidence to ensure that the recommendations are comprehensive and objective. Rapid reviews can synthesize the available research evidence on the same topic in a short duration of time, and thus reduce the risk of bias and ultimately improve the reliability and accuracy of decision making. Our guideline was accompanied by 13 rapid reviews to support the development of recommendations.It is well known that strict adherence to guideline development manuals and the protocol can enhance the transparency and clarity and ensure the quality and credibility of the guideline . In thisIn order to respond to public health emergencies, the development of a RAG requires a clear and transparent formulation process, and usually uses a large amount of indirect and non-peer-reviewed evidence to support the formation of recommendations. Strictly following the WHO RAG handbook does not only enhance the transparency and clarity of the guideline, but also speeds up the guideline development process, thereby saving time and resources.Additional file 1.Additional file 2.Additional file 3.


BRAKER is a suite of automatic pipelines, BRAKER1 and BRAKER2, for the accurate annotation of protein-coding genes in eukaryotic genomes. Each pipeline trains statistical models of protein-coding genes based on provided evidence and, then predicts protein-coding genes in genomic sequences using both the extrinsic evidence and statistical models. For training and prediction, BRAKER1 and BRAKER2 incorporate complementary extrinsic evidence: BRAKER1 uses only RNA-seq data while BRAKER2 uses only a database of cross-species proteins. The BRAKER suite has so far not been able to reliably exceed the accuracy of BRAKER1 and BRAKER2 when incorporating both types of evidence simultaneously. Currently, for a novel genome project where both RNA-seq and protein data are available, the best option is to run both pipelines independently, and to pick one, likely better output. Therefore, one or another type of the extrinsic evidence would remain unexploited.We present TSEBRA, a software that selects gene predictions (transcripts) from the sets generated by BRAKER1 and BRAKER2. TSEBRA uses a set of rules to compare scores of overlapping transcripts based on their support by RNA-seq and homologous protein evidence. We show in computational experiments on genomes of 11 species that TSEBRA achieves higher accuracy than either BRAKER1 or BRAKER2 running alone and that TSEBRA compares favorably with the combiner tool EVidenceModeler.TSEBRA is an easy-to-use and fast software tool. It can be used in concert with the BRAKER pipeline to generate a gene prediction set supported by both RNA-seq and homologous protein evidence.The online version contains supplementary material available at 10.1186/s12859-021-04482-0. Currently, the National Center for Biotechnology Information’s (NCBI) GenBank  hosts 79The most useful data to support accurate genome annotation are transcriptomic sequence data, e.g. RNA-sequencing (RNA-seq) data, from the same species and protein sequences from species that are sufficiently closely related to the target species in the tree of life. RNA-seq reads spliced aligned to a genomic region are used to infer likely intron intervals  in proteGenome annotation methods that use statistical models of gene structures such as splice site patterns in addition to the evidence from RNA-seq and homology, are arguably best suited for whole-genome annotation . BRAKER,combiner tool that takes various gene predictions as input with diverse sources of extrinsic evidence and constructs a genome annotation that is on average more accurate than any input genome annotation. Some previously developed combiner tools built their own gene structure model in the form of a graph and report a gene structure either based on the consensus of all available data, e.g. IPred [When heterogeneous extrinsic evidence sources are available, some genome annotation tools like MAKER2  and GeMog. IPred , or as tg. IPred , JIGSAW g. IPred , Evigan g. IPred , ExonHung. IPred . A promig. IPred . It usesg. IPred , 30.Transcript Selector for BRAKER (TSEBRA), a fast software tool for selecting gene predictions from the output of two branches of the BRAKER eukaryotic gene prediction suite based on all the heterogeneous extrinsic evidence. TSEBRA achieves high accuracy and is easy to use. We show that it delivers a significant increase in accuracy with respect to the input annotations generated by BRAKER1 and BRAKER2.In our approach, we first generate several sets of whole-genome gene predictions based on a single type of extrinsic evidence (i.e. by BRAKER1 and BRAKER2). We use a new combiner tool that scores and ranks these predictions (transcripts) based on heterogeneous evidence. Then, we select those with higher rank into a newly constructed genome annotation which is on average more accurate than any whole-genome annotation provided in the input. Up to now, the BRAKER suite has so far not been able to achieve a prediction accuracy that is reliably superior to either single-source evidence mode when using RNA-seq and proteins simultaneously , 32. NevTSEBRA uses a set of arbitrarily many gene prediction files in GTF format together with a set of files of heterogeneous extrinsic evidence to produce a combined output. From the whole set of transcripts contained in the gene predictions, TSEBRA must select those that are more reliably supported by a full complement of extrinsic evidence; these transcripts constitute the output. Less reliably supported transcripts are filtered out. The rational of TSEBRA’s approach is as follows. Taking a union of gene predictions generated by two or more gene finding tools makes a set of predictions with improved sensitivity but with lower specificity. A non-trivial task is to remove some predictions and increase specificity with little decrease of sensitivity. This task is tantamount to identification of likely false positives and filtering them out. TSEBRA solves exactly this problem.multiplicity – the number of alignments that support it, and ’src=’ determining its source, e.g., ’src=P’ for evidence from a protein alignment. The mult attribute is used to specify multiplicities larger than one.TSEBRA uses extrinsic evidence in the form of intron regions or start/stop codon positions to evaluate and filter transcripts from gene predictions. These must be provided in a GFF file that includes two attributes in the last column ’mult=’, a number specifying its transcript scores and the transcript score thresholds are used for comparing transcripts. The low evidence support thresholds consist of minimum fractions of intron or start/stop codon support. We recommend the application of the default hyperparameters provided in the TSEBRA configuration file to be used in a standard use case.TSEBRA takes three sets of different hyperparameters from a configuration file. More precisely, it takes a weight for any evidence source, four transcript score thresholds and two low evidence support thresholds. The weights are used to compute Take a union of transcripts predicted by BRAKER1 and BRAKER2 while merging identical transcripts.Compute vectors of support scores for all transcripts.Identify all pairs of transcripts with overlapping coding regions.Compare all pairs of overlapping transcripts by a transcript comparison rule using the extrinsic evidence and mark some of them for exclusion.Remove all transcripts marked for exclusion by the transcript comparison rule.Remove all transcripts with low evidence support.Combine the remaining transcripts into a final set of predictions with groups of overlapping transcripts making sets of alternative isoforms.The output of TSEBRA is the set of genes  in GTF format.The workflow of TSEBRA is as follows: In step 6, a transcript is removed if the fractions of introns and start/stop codons supported by extrinsic evidence are lower than the low evidence support thresholds. In step 7, genes are the single-linkage clusters of transcripts where two transcripts are in the same gene if they overlap . Two transcripts are considered to overlap if they share at least three adjacent protein-coding nucleotides on the same strand and in the same reading frame. Note that a transcript ’marked for exclusion’ in step 4 is still compared to all overlapping transcripts and may cause removal of another transcript. This filtering step is different from a simplistic approach that would first score transcripts and then apply a fixed threshold to their score. In our approach, the transcripts with strongest local support are kept, and those that are discarded can still have strong support in absolute terms if transcripts with even stronger support overlap.As a special case, TSEBRA may be used with a single gene prediction file to filter for the ones with the strongest evidence support. This may be useful for a genome annotation with many transcript isoforms per gene.features of a transcript, here introns (i) or start/stop-codons (s), by all extrinsic evidence E represented by hints. A hint h is either an intron region or start/stop codon position together with an identifier of its original source O is a set of original sources, e.g. h. A hint supports a transcript feature if all identifying characteristics match, i.e. sequence name, start/stop position, feature type, and strand.Four transcript scores nguished . MultiplF be the set of all of its features. Define F of type E and E.Consider a particular transcript and let f is the weighted sum of all supporting hints:A weight mentclass2pt{minimThe pairwise transcript comparison rule compares two transcripts with respect to their support of extrinsic evidence using the transcript scores. One or no transcript is marked for exclusion when comparing two overlapping transcripts, see Fig. The differences of all transcript scores (of the same type) are each compared to one of the score specific thresholds The TSEBRA suite includes a set of default hyperparameters, which are recommended for usage in a standard use case – to combine BRAKER1 and BRAKER2 – so that users are not required to set the hyperparameters themselves. Evidence sources in a standard BRAKER1 and BRAKER2 output are: protein database (P), EST database (E), combined EST/protein database (C), and manual anchored (M). The default weights for these are We compared the accuracy of TSEBRA in two experiments. First, we compared TSEBRA to BRAKER1 and BRAKER2 in their standard use modes, and second, we compared TSEBRA with EVM.t is considered as correctly predicted by transcript t and Specificity (Sp), sensitivity (Sn), and their harmonic mean - the F1-score - were the measures of gene prediction accuracy. Accuracy values were examined at the gene, transcript, and exon levels. A predicted gene is considered correct, if it is identical to at least one annotated alternative splicing isoform. A reference transcript A. thaliana, C. elegans, and D. melanogaster, we used proteins from three sets of species varied with respect to minimal evolutionary distance to the query species. Each protein set included proteins from a large clade, e.g. Plantae, Metazoa, and Arthropoda, of the query species. The three sets per species excluded either (i) proteins from the query species itself, (ii) all species of the same family or (iii) all species of the same order. The corresponding sets of proteins could provide more or less precise evidence for gene prediction depending on the degree of saturation by closely related species. The level (i) offers the largest number of close relatives while the level (iii) provides the least number of them and the least precise evidence for a query species. We used proteins from the corresponding sets of species selected at the level (iii) for the other eight species.Complete genome annotations generated independently by BRAKER1 and BRAKER2 (both BRAKER v.2.1.5) for 11 eukaryotic species  had a higher accuracy than either BRAKER1 or BRAKER2 across all 11 species and nearly all test settings, see Table The number of transcripts per gene selected by TSEBRA was on average 1.07 which is at the same level as BRAKER2, and lower than the average of 1.20 observed for BRAKER1.A. thaliana, C. elegans, and D. melanogaster.We also compared the accuracy of TSEBRA with the accuracy of EVidenceModeler  workingEVM takes extrinsic evidence in form of spliced alignments from assembled transcripts. This type of evidence is not produced by BRAKER1 utilizing mappings of unassembled reads. To make a comparison between EVM and TSEBRA on the same data, we produced new and comparable extrinsic evidence for EVM  and TSEBRA (i.e. intron or start/stop codon hints). We used the protein alignments generated by ProtHint during the BRAKER2 run and assembled spliced alignments from the RNA-seq reads sampled by VARUS. For each locus, we selected the protein alignment produced by ProtHint with the highest DIAMOND  score crEVM requires that a weight is assigned to each of the four input sources. For each test setting, we used 10% of the total number of partitions to decide on weights for EVM and to set hyperparameters of TSEBRA. A grid search with two iterations was employed to find good sets of parameters for each test setting. The grid had five values for each parameter of EVM and three values for each parameter of TSEBRA. We evaluated the accuracy of TSEBRA_EVM on the remaining 90% of the partitions for each test setting that were not used to set the hyperparameters. The values of the hyperparameters are available in Additional file We compared the accuracy of TSEBRA with EVM, one of the most cited combiner tools to date. EVM was previously used to combine BRAKER with other predictions , to combWe compared TSEBRA and EVM to address a question: which is the better method for combining BRAKER1 and BRAKER2 predictions? In a test setting with comparable extrinsic evidence, we evaluated them across three species with three different protein sets each. Both methods have successfully combined the BRAKER1 and BRAKER2 predictions into one set with increased F1-score, see Fig. We had to carefully select the test setting since the choice of partition size made a difference to EVM when it makes models of intergenic regions. An additional difficulty was that the tools used for creating the partitions enforced hard borders, even if they split a transcript. To neutralize this issue, an overlap between partitions was used.TSEBRA executes much faster than EVM, which constructs a complete gene model and evaluates all possible gene structures. TSEBRA, on the other hand, only evaluates the set of explicitly given transcripts as input. In our tests, we ran both methods separately in parallel on a 28 multi-core processor, the average runtime of EVM was 35.28 min and of TSEBRA_EVM 0.37 min.training statistical gene-finding models. A relative complementarity of the gene sets can be an advantage when they are combined subsequently. The ranking and selection of the final set of transcripts, however, does use both protein and RNA-seq evidence. This approach makes an effective use of both sources of extrinsic evidence for selection of most likely true positive transcripts from the set of candidates, the transcripts generated by BRAKER1 and BRAKER2 running in parallel.We presented TSEBRA, a tool that selects more reliable gene predictions (transcripts) from the sets of transcripts generated independently by BRAKER1 and BRAKER2. A novel approach to transcript selection was successfully implemented. In computational experiments made with genomes of 11 diverse eukaryotic species we have shown that the set of transcripts selected by TSEBRA matched annotated genes (believed to be the true ones) with higher accuracy than both BRAKER1 and BRAKER2. Note that the combined extrinsic evidence is not used at the step of generation of gene predictions. BRAKER1 and BRAKER2 use disjoint evidence sources also for Thus, TSEBRA makes a useful tool that with help of heterogeneous extrinsic evidence transforms the union of predictions of BRAKER1 and BRAKER2 into a set of gene predictions whose accuracy exceeds the accuracy of both BRAKER1 and BRAKER2 running separately.Additional file 1. Tables (S1–S6). The supplementary tables include a list of all genomes and annotation versions used in this paper, additional results, hyperparameters and statistical significance tests.


To determine the clinical effectiveness of common elective orthopaedic procedures compared with no treatment, placebo, or non-operative care and assess the impact on clinical guidelines.Umbrella review of meta-analyses of randomised controlled trials or other study designs in the absence of meta-analyses of randomised controlled trials.Ten of the most common elective orthopaedic procedures—arthroscopic anterior cruciate ligament reconstruction, arthroscopic meniscal repair of the knee, arthroscopic partial meniscectomy of the knee, arthroscopic rotator cuff repair, arthroscopic subacromial decompression, carpal tunnel decompression, lumbar spine decompression, lumbar spine fusion, total hip replacement, and total knee replacement—were studied. Medline, Embase, Cochrane Library, and bibliographies were searched until September 2020.Meta-analyses of randomised controlled trials  that compared the clinical effectiveness of any of the 10 orthopaedic procedures with no treatment, placebo, or non-operative care.Summary data were extracted by two independent investigators, and a consensus was reached with the involvement of a third. The methodological quality of each meta-analysis was assessed using the Assessment of Multiple Systematic Reviews instrument. The Jadad decision algorithm was used to ascertain which meta-analysis represented the best evidence. The National Institute for Health and Care Excellence Evidence search was used to check whether recommendations for each procedure reflected the body of evidence.Quality and quantity of evidence behind common elective orthopaedic interventions and comparisons with the strength of recommendations in relevant national clinical guidelines.Randomised controlled trial evidence supports the superiority of carpal tunnel decompression and total knee replacement over non-operative care. No randomised controlled trials specifically compared total hip replacement or meniscal repair with non-operative care. Trial evidence for the other six procedures showed no benefit over non-operative care.Although they may be effective overall or in certain subgroups, no strong, high quality evidence base shows that many commonly performed elective orthopaedic procedures are more effective than non-operative alternatives. Despite the lack of strong evidence, some of these procedures are still recommended by national guidelines in certain situations.PROSPERO CRD42018115917. About 200 musculoskeletal conditions have a substantial effect on the quality of life of millions of people in the UK and are associated with increased healthcare costs.Total joint replacement is one of the most common elective orthopaedic procedures performed worldwide for end stage osteoarthritis, the most common musculoskeletal condition.Whether common orthopaedic procedures for managing musculoskeletal conditions are accompanied by high quality evidence bases is unclear. In this context, using an umbrella review of systematic reviews/meta-analyses of randomised controlled trials or other study designs in the absence of meta-analyses of randomised controlled trials, we aimed to evaluate the clinical effectiveness of the 10 most common elective orthopaedic procedures, assess whether the evidence base on these 10 orthopaedic procedures has affected guidelines, and identify gaps in the existing evidence.We did an umbrella review—a systematic collection and critical evaluation of multiple systematic reviews/meta-analyses on a specific research topic.To identify the 10 most elective common orthopaedic procedures and their indications , we combWe searched Medline, Embase, and the Cochrane Library from inception to September 2020 for studies that compared the clinical effectiveness of each of the selected orthopaedic interventions with no treatment, placebo, or non-operative care. We did separate searches for each orthopaedic procedure. The computer based searches combined free and MeSH search terms and keywords related to the orthopaedic procedure  and population . We applied filters for systematic reviews/meta-analyses. We restricted the search to articles written in the English language because our study design was an umbrella review of systematic reviews/meta-analyses and we took into consideration the extent of the literature included by each contributing review during our selection of the best currently available evidence for a procedure. The detailed strategy for each orthopaedic procedure is reported in appendix 3. We screened all titles and abstracts of retrieved citations to assess suitability for inclusion. Two authors (SKK and RLD) independently evaluated the full text of articles potentially meeting eligibility criteria for study selection. When necessary, discrepancies were discussed and consensus reached with the involvement of a third author (MRW). To account for studies missed in the original search, we manually scanned reference lists of eligible articles.Studies were eligible if they were systematic reviews/meta-analyses including randomised controlled trials that compared the clinical effectiveness of any of the orthopaedic procedures mentioned above with no treatment, placebo, or non-operative care. In the absence of meta-analyses of randomised controlled trials, we sought individual randomised controlled trials followed by systematic reviews/meta-analyses of observational cohort studies, in that order, according to the hierarchy of evidence (appendix 4).2 values), any presented measure of publication bias, Grading of Recommendations Assessment, Development and Evaluation (GRADE) rating, sources of funding, and conflicts of interest. We explored whether the studies evaluated possible sources of heterogeneity across studies by using methods such as subgroup analyses and whether the authors formally conducted a sensitivity analysis.One author (SKK) initially extracted data by using a standardised data collection form. A second author (RLD) independently checked the extracted data against the original articles. In the event of discrepancies between authors, a third author (MRW) was consulted. We extracted information on the first author’s name, year and journal of publication, databases searched, number of component randomised controlled trials, outcomes and findings, type of effect model used in the meta-analysis (fixed or random), between study heterogeneity estimates  instrument,We used the Jadad decision algorithm  to proviIn the Musculoskeletal Research Unit of the University of Bristol and North Bristol NHS Trust, we have a dedicated patient and public involvement group, the Patient Experience Partnership in Research (PEP-R).The study selection process from retrieval of search results to inclusion in the umbrella review is illustrated in the PRISMA flow diagram , accordiOf 120 citations retrieved for anterior cruciate ligament reconstruction, three articles were eligible for the review , and theOf 26 retrieved articles, no reviews of randomised controlled trials comparing arthroscopic meniscal repair with no treatment, placebo, or non-operative care were identified . In the Of 82 retrieved articles on arthroscopic partial meniscectomy, six were eligible for evaluation . HoweverOf 117 retrieved articles on rotator cuff repair, three meta-analyses were eligible for the review . In the We retrieved 81 articles on arthroscopic subacromial decompression, and three fulfilled the eligibility criteria . Two metOf 50 retrieved citations on carpal tunnel decompression, nine meta-analyses of randomised controlled trials were eligible for the review , surgical treatment was shown to relieve symptoms significantly better than non-surgical treatment; however, surgical treatment was associated with more complications . No randFor the comparison between endoscopic versus open carpal tunnel decompression, the results were discordant across the eight reviews. However, general findings suggested that endoscopic and open release are about as effective as each other in relieving symptoms and improving functional status, although endoscopic release may have functional benefits over the open release with regards to return to work and improved grip strength. Some evidence suggested that endoscopic release increased the risk of nerve injury compared with open release. The GRADE quality of the evidence ranged from very low to low. On the basis of the fact that most reviews that investigated the same research question did not include the same trials and selection criteria were similar, the Jadad decision algorithm indicated that the best available evidence should be chosen according to search strategies and application of selection criteria . Hence, Of 137 retrieved citations for lumbar spine decompression, 12 meta-analyses of randomised controlled trials were eligible for the review . The GRAFor the comparisons among different surgical procedures, the results were generally consistent: no differences were seen in outcomes such as pain intensity, physical function or disability status, quality of life, recovery, perioperative blood loss, operation time, length of stay in hospital, and reoperation rates when the techniques were compared with each other. However, interspinous process device implantation seemed to be associated with higher reoperation rates than spinal decompression. No randomised controlled trials have compared lumbar spinal decompression with no treatment or placebo. In the single selected randomised controlled trial that represented the best body of evidence and compared segmental decompression with conservative treatment , both treatment groups showed improvement during follow-up with no difference in walking ability (appendices 8 and 9).Of 139 retrieved citations on lumbar spine fusion, 13 meta-analyses of randomised controlled trials were eligible for the review .For the comparison between lumbar spine fusion and total disc replacement, the results were variable in the earlier reviews. With the publication of newer randomised controlled trials, recent reviews showed that total disc replacement significantly improved pain and patient satisfaction, reduced reoperation rate and operation time, shortened duration of hospital admission, and decreased post-surgical complications compared with lumbar spine fusion in both the short term and the long term. The findings of the two reviews comparing minimally invasive versus open transforaminal lumbar interbody fusion for single level degenerative disease were discordant except for less blood loss and longer fluoroscopy time with the minimally invasive approach. On the basis of similarities in the research question and selection criteria, and differences in included trials, the Jadad decision algorithm indicated that the best available evidence should be chosen according to search strategies and application of selection criteria . The besOf 168 retrieved articles, no reviews of randomised controlled trials comparing total hip replacement with no treatment, placebo, or sham surgery for the treatment of end stage osteoarthritis were identified . The American Academy of Orthopaedic Surgeons (AAOS) recommends using arthroscopic anterior cruciate ligament reconstruction in certain subgroups of patients,Encouragingly, many of the guidelines from prominent national bodies closely reflect the current body of evidence. Consensus statements from guideline societies do not recommend arthroscopic partial meniscectomy in patients with knee pain and a meniscal tear,Although seven of the procedures  have been recommended for use by national guidelines, a high quality body of evidence on the clinical effectiveness to definitively support these recommendations does not exist for most of these procedures. This is mainly a consequence of the lack of randomised controlled trials that compare the procedure with non-operative care. The evidence base for the recommendations was mostly built on comparisons involving two or more different techniques of the same procedure , observational retrospective studies and case series with no control groups, and expert opinions. For example, the recommendations for arthroscopic meniscal repair were based on mostly indirect evidence and low quality observational studies,The strengths of this study include its novelty and the comprehensive search of well known databases as well as guidelines. Although umbrella reviews provide top level evidence and important insights on the clinical effectiveness of interventions, several inherent limitations to our review deserve consideration. Because of the absence of meta-analyses on a few procedures, due to the lack of randomised controlled trials, we used relevant study designs according to the hierarchy of evidence. We used the UK NICE guidelines evidence search for recommendations on each procedure, given that the study setting was the UK. The recommendations can be applied to other healthcare settings, however, as our search yielded guidelines from elsewhere in Europe and the US. Several meta-analyses did not use GRADE scores to appraise the quality of findings. Finally, several of the meta-analyses included in our review scored low when appraised with the AMSTAR tool, showing the need for future reviews to use accepted methods of reporting.Surgery is expensive and associated with considerable morbidity, increased risk of complications attributed to the surgical intervention, and excess mortality. Seeking high level evidence to support surgery is thus imperative. When high level evidence shows that non-operative care is equivalent, surgeons and patients should carefully consider what would be achieved by doing surgery. A cogent argument can be made for surgery being used as a second line treatment when non-surgical measures have failed or in certain subgroups of patients who have been identified as “responders” to surgical treatment. However, trials first need to be done to show the efficacy of surgery in these scenarios and to define the period of treatment with non-surgical interventions that is appropriate before surgery is undertaken. For an operation such as total hip replacement, the understandable assumption is made that all improvement is due to the surgery, but some may be due to the natural history including regression to the mean or treatment effect from non-replacement co-interventions. Given the current waiting lists for surgery due to the covid-19 pandemic, a randomised controlled trial comparing total hip replacement with best non-surgical care may be timely, especially if those patients randomised to non-operative care are subsequently offered surgery should they be dissatisfied with the outcome of non-operative care.Some of these interventions are just not clinically effective or may be effective only when used in specific circumstances. For example, although the use of arthroscopic partial meniscectomy in patients with knee pain and a meniscal tear is not recommended, especially in patients with significant or end stage osteoarthritis, guidelines suggest that the procedure can be used for a specific type of meniscal tear and should be used only in patients who have not responded to a period of non-surgical treatment. Despite the good body of evidence on the clinical ineffectiveness of arthroscopic subacromial decompression, national clinical guidelines recommend its use for patients with pure subacromial shoulder impingement whose symptoms fail to resolve with adequate non-operative treatment.The observation that most commonly used and recommended orthopaedic procedures had a limited and low quality evidence base relating to their effectiveness is concerning. Why clinicians and healthcare systems would choose to offer these procedures with limited evidence of their clinical effectiveness may not be entirely clear but is partly justifiable. Important reasons include the lack of randomised controlled trials that compare the procedure with no treatment or placebo. Properly conducted definitive trials are more difficult to run with orthopaedic interventions than with drugs and other interventions.The BMJ suggested that we should use “impactful, evidence based surgery alongside impactful, evidence based non-surgical care that may be faster, safer, and delivered at a lower cost.” Furthermore, with 10 million patients in England awaiting surgery, “now is an ideal time to invest in large platform trials.”With these findings, a need exists for an improved and more rigorous approach to the recommendation of procedures with limited evidence on their clinical effectiveness. Over the past few years, NHS England, in partnership with NHS Clinical Commissioners, the Academy of Royal Medical Colleges, and NICE, has put forward proposals to stop or reduce the commissioning of various interventions that have inconclusive evidence for their clinical effectiveness; the goal of this is to improve safety for patients and curb unnecessary use of time and resources.This umbrella review of the body of evidence on the 10 most common elective orthopaedic procedures suggests that most of these procedures recommended by national guidelines and used by surgeons have insufficient readily available, high quality evidence on their clinical effectiveness, which is mainly because of a lack of definitive trials. This forces patients and clinicians to make decisions based solely on observational evidence. When randomised controlled trials have been conducted, they sometimes support observational evidence and expert opinion, but not universally so. To optimise decision making, surgeons should apprise themselves of the evidence, make decisions based on the highest quality trials available, and, in the absence of these, base their judgment on observational evidence, acknowledging that this may be imperfect. Simultaneously, surgeons and research funding bodies should actively contribute to filling the key knowledge gaps by enabling and participating in well constructed pragmatic randomised controlled trials.The burden of musculoskeletal conditions on health services is large, with more than one million patients undergoing surgical interventions according to Hospital Episode StatisticsThe National Institute for Health and Care Excellence, NHS, and British Orthopaedic Association produce national clinical guidelines recommending common elective orthopaedic interventionsThese recommendations often lack strong supporting evidence, particularly in the form of randomised controlled trialsMost common elective orthopaedic interventions are not backed up by readily available high quality evidence, mostly owing to a lack of definitive randomised controlled trialsOf the procedures studied, carpal tunnel decompression and total knee replacement showed superiority over non-operative careAn urgent need exists to prioritise research into common elective orthopaedic interventions compared with no treatment, placebo, and non-operative treatment


The last decade has seen nearly 20 papers reviewing the totality of the data on saturated fats and cardiovascular outcomes, which, altogether, have demonstrated a lack of rigorous evidence to support continued recommendations either to limit the consumption of saturated fatty acids or to replace them with polyunsaturated fatty acids. These papers were unfortunately not considered by the process leading to the most recent U.S. Dietary Guidelines for Americans, the country’s national nutrition policy, which recently reconfirmed its recommendation to limit saturated fats to 10% or less of total energy intake, based on insufficient and inconsistent evidence. Continuation of a cap on saturated fat intake also fails to consider the important effects of the food matrix and the overall dietary pattern in which saturated fatty acids are consumed. Since the introduction of the U.S. Dietary Guidelines for Americans (DGA) in 1980, national nutrition policy has consistently advised limiting saturated fat consumption as a central strategy for reducing risk for atherosclerotic cardiovascular disease (CVD). Saturated fatty acids are defined as those molecules that are “saturated” by hydrogen, without any double bonds. They are found in all foods but are especially concentrated in dairy, red meat, and the so-called tropical oils. In 1990, the DGA added a specific cap limiting these fats to 10% of calories . The guiThe hypothesis that saturated fats cause CVD emerged in the late 1950s, when scientists observed that these fats tend to raise the concentration of total serum cholesterol, which in turn was considered a potent risk factor for heart disease. Ancel Keys, a physiologist at the University of Minnesota, postulated that saturated fats along with dietary cholesterol were the principal causes of cardiovascular disease, and his “diet-heart hypothesis” was adopted by leading groups, including the American Heart Association. At the time, the evidence supporting this advice consisted primarily of one observational study which compared the level of saturated fats intake with heart-disease outcomes in seven countries and involved 12,763 men in Europe, the U.S. and Japan   in the 1960s and 1970s. Altogether, these trials tested the diet-heart hypothesis in about 67,000 people [Remarkably, the diet-heart hypothesis gained widespread acceptance in the 1970s and 1980s, yet the results from the totality of these trials did not provide support for the hypothesis, as many critics at the time pointed out ,11,12. PDietary goals for the United States, widely known as the McGovern report [In 1977, the United States Senate Select Committee on Nutrition and Human Needs published the n report . Goal #5The lack of a response to the data on saturated fat stands in contrast to the DGA’s revised position in response to two other bodies of scientific evidence to more accurately reflect the data. The 1980–2000 DGA recommendation  to consuAccording to a narrative review of the relevant literature, there have to date been at least 10 reviews of the RCTs addressing the effects of dietary saturated fat on CVD, with varying conclusions . Those rAn updated systematic review by the Cochrane group in 2020  on saturObservational, or epidemiological, studies can demonstrate associations with disease outcomes, but may only be used to suggest cause-and-effect relationships when a number of criteria, such as consistency and strength of association, are met . ObservaReviews of this large body of evidence began in 2010 with a meta-analysis by Siri-Tarino et al., which concluded that available observational data provide “no significant evidence for concluding that dietary saturated fat is associated with an increased risk of CHD or CVD” . There hTwo of the meta-analyses used statistical modeling  to simulate replacement of SFAs with PUFAs and found an association with a lower risk of heart disease ,27. HoweAdditional evidence used to support the continued recommendation for limiting saturated fat consumption comes from the well-demonstrated ability of SFAs to raise LDL-cholesterol concentration, which becomes evident when SFAs are replaced by PUFAs . HoweverAnother explanation for why reductions in LDL-cholesterol concentration induced by saturated fat restriction have not been shown to reduce cardiovascular risk is that the decreased LDL-cholesterol primarily reflects reduced levels of large LDL particles . As reviMoreover, saturated fat intake increases the levels of high-density lipoprotein (HDL) cholesterol, improving the ratio of total to HDL cholesterol, which is a robust marker of CVD risk . Thus, dFor the 2020 DGA process, the USDA excluded from consideration all systematic reviews conducted outside the agency, choosing instead to rely on novel reviews performed by the USDA staff. Thus, the approximately 20 review papers on saturated fats mentioned above which were published during the last decade by “external” scientists were excluded from consideration. Outside experts attempted to introduce this evidence via written comments to the USDA ,34,35, pBy contrast, the 2015 DGAC had considered seven external review papers , yet theThe most recent review, by the 2020 DGAC, concluded that there is “strong” evidence for continuing caps on saturated fat intake, and for replacing SFAs with PUFAs in particular. This conclusion relied on RCT data showing that replacement of SFAs with PUFAs could lower total and LDL-cholesterol concentrations  Thus, the preponderance of evidence, as summarized here , does noThis statute also requires that the DGA be for the “general public”, yet the DGAC report notes that the studies on saturated fats reporting race or ethnicity were conducted mostly “in the U.S. or Scandinavia”, with “predominantly White or Caucasian participants” who were healthy middle-aged or older adults with overweight and without CVD at baseline  (pp. 41–12, riboflavin, niacin and pantothenic acid. These nutrients interact with each other, and one can play a role in the effective absorption of another. For instance, fat-soluble vitamins such as A and D require fat for absorption. Moreover, the lack of consistent relationships of foods to CVD risk based on their saturated fat content is likely due in part to variation in effects of the overall food matrix and varying content of specific saturated fatty acids in these foods, as well as the dietary patterns in which they are consumed [Saturated fats are increasingly being viewed as part of the food matrix and dietary patterns in which they appear naturally, rather than as an isolated nutrient ,48. Cheeconsumed .The matrix concept came to prominence in a 2010 international consensus statement . ImplemeIn addition to the food matrix, the overall dietary pattern, particularly the level of carbohydrate, has an important impact on the way saturated fat is metabolized ,50. For The U.S. Dietary Guidelines have also moved away from nutrient-based recommendations, specifically with regard to total fat, yet the DGA has not done so for saturated fat. The Committee did, however, “recognize the importance of the growing body of research on specific fatty acids, food matrix and sources of fats, explicitly saturated fat”  (Part D,Multiple reviews of the evidence have demonstrated that a recommendation to limit consumption of saturated fats to no more than 10% of total calories is not supported by rigorous scientific studies. Importantly, neither this guideline, nor that for replacing saturated fats with polyunsaturated fats, considers the central issue of the health effects of differing food sources of these fats. The 2020 DGAC review that recommends continuing these recommendations has, in our view, not met the standard of “the preponderance of the evidence” for this decision.


There are growing concerns that the public’s trust in science is eroding, including concerns that vested interests are corrupting what we know about our food. We know the food industry funds third-party ‘front groups’ to advance its positions and profits. Here we ask whether this is the case with International Food Information Council (IFIC) and its associated Foundation, exploring its motivations and the potential for industry influence on communications around nutritional science.We systematically searched the University of California San Francisco’s Food Industry Documents Archive, for all documents pertaining to IFIC, which were then thematically evaluated against a science-communication influence model.We identified 75 documents which evidence that prominent individuals with long careers in the food industry view IFIC as designed to: 1) advance industry public relations goals; 2) amplify the messages of industry-funded research organizations; and 3) place industry approved experts before the press and media, in ways that conceal industry input. We observed that there were in some cases efforts made to conceal and dilute industry links associated with IFIC from the public’s view.Instances suggesting IFIC communicates content produced by industry, and other industry-funded organisations like ILSI, give rise to concerns about vested interests going undetected in its outputs. IFIC’s deployment to take on so-called “hard-hitting issues” for industry, summating evidence, while countering evidence that industry opposes, give rise to concerns about IFIC’s purported neutrality. IFIC’s role in coordinating and placing industry allies in online and traditional press outlets, to overcome industry’s global scientific, legislative, regulatory and public relations challenges, leads also to concerns about it thwarting effective public health and safety measures.IFIC’s promotion of evidence for the food industry should be interpreted as marketing strategy for those funders. Effective science communication may be obfuscated by undeclared conflicts of interests. There are growing concerns that the public’s trust in science is eroding , 2. ThisWhat is particularly challenging about disinformation is that those who spread it often seek to conceal their involvement, taking a lead from the past activities of the tobacco industry. This industry used various groups at arm’s length to confuse the science in the public mind about the health impacts of tobacco, so as to thwart regulation and continue to sell products, over many decades . Indeed,foodinsights.org, reports that, consistent with its constitution as a non-profit body, it seeks to “effectively communicate science-based information on health, nutrition and food safety for the public good” . If my memory is good, they both have been one more of your good initiatives when aspartame first become under attack from activists and consequently the press. …  report conference call (see resulting media coverage and bulleted overview below). Participants included the Associated Press, Politico, WBEZ-Chicago, Capitol Press, and trade press as well as nutrition columnists and bloggers. The former DGAC panelists included Dr. Cheryl Achterberg, Dr. Joanne Lupton, Dr. Linda Van Horn, Dr. Theresa Nicklas, Dr. Connie Weaver and Dr. Roger Clemens. This hour-long webcast was recorded and has been posted to our website. We also live tweeted and heavily promoted our new Dietary Guidelines Infographic during the call. Today's combined Dietary Guidelines communication activities have resulted in more than 393,500 total impressions. In addition to the media call, we have compiled a list of 20+ experts with content-specific expertise in DGAC "hot-button" issues  who have agreed to be available for media inquiries . Following publication of this manual, sponsor workshops on risk communication, using the manual as a guide for industry, regulators, and possibly journalists. This would also be done in conjunction with IFIC.4. Monitor the activities of the Residue Committee with regard to the release of the National Academy of Sciences (NAS) report on pesticide exposure in the diets of infants and children to ensure that sufficient and appropriate action is planned before and at the time of its release. Ensure involvement of IFIC in the communications aspects of this effort.These minutes indicate that ILSI and IFIC work in collaboration, although as noted we could not verify whether or not these collaborations were realized as intended since such documents were not publicly available.However, the documents reveal that industry leaders viewed IFIC as playing a strategic role in media communications on science, as expressed in another email from Malaspina to John C Peters, a researcher at the University of Colorado, in which Malaspina summarizes the purpose of IFIC as follows:As to the generation of credible, consensus science on the issues hitting the industry - obesity and causative factors, sugar, low/no calories sweetener safety - in particular we have to use external organizations in addition to any work we directly commission (and that needs to be very carefully reviewed in fight of the BMY article I); examples are : ILSI…Scientific Societies… Medical Associations… National Academies of Science…EU/Gov’t, Research…  foolish [. foolish .Although there was no evidence in the document set of direct attempts use IFIC to influence policymakers, it is possible IFIC sought to achieve indirect influence by communicating media and evidence favourable to industry positions.IFIC’s “Form 990” filings to the Internal Revenue Service for both the parent entity and the Foundation suggest communicating with “government officials” is part of its core activities , 20. HowOur case-study of IFIC, including its associated Foundation, reveals evidence that suggests it is viewed by some former and current industry heads as holding a role in promoting and representing industry or other commercial bodies’ interests, and even as having the potential to conduct product or ingredient defence in the face of impending negative publicity. We make several important observations from the emails we identified. First, there was evidence that IFIC was intended to communicate evidence produced by other industry-funded organisations like ILSI , 31–33. Our research suggests at its heart IFIC is part of a network of actors that bring together industry with non-profits that it funds, alongside favourable experts from research institutions and universities, and the press. It acts in coalitions—with those networks—to mobilise product defence messaging or other messaging favourable to industry. Many of the individuals we observed discussing IFIC have themselves complex employment histories, working inside corporations and outside in non-profits and research. Our work, like Greenhalgh’s, observes that several core people exert influence, guiding coalitions and cooperation, including rather specifically “the MIT-trained food technology specialist Alex Malaspina (1931–), was concurrently vice president of TCCC ”, but also that many of the scientists engaged in emails traverse the public and private spaces, taking industry funding for specific research, and regularly sitting on industry-funded non-profit bodies as seemingly independent or neutral parties, while evidence suggests they act favourably toward in industry .Such convening of experts to further industry agendas mirrors observations made by Greenhalgh around ILSI, which as we saw, is considered a sister body to IFIC. Greenhalgh observes in her work that ILSI undertook a process that led to instances where five soda and food giants engaged the services of company-friendly scientists to promote a friendly public agenda . In her `It is also notable that we identified direct connections between GEBN and ILSI with IFIC., Greenhalgh’s recent work, which adds context to industry endorsement and promotion of “energy balance” frameworks, details “how physical activity became the priority solution” to address emerging concerns about the obesity epidemic, and how only small industry concessions about dietary intake featured in many approaches pursued across the 1990s and beyond. Greenhalgh observes a network of organisations acting with TCCC to devise and disseminate ‘a “science of energy balance” to buttress the case for the physical activity solution to obesity’.  She detaGreenhalgh also helpfully details how networks of bodies and experts are deployed to impact at “the early stage in the policy process to place industry-friendly researchers in important seats at the science and policy table” . She sugAs with all analyses of documentary archives, our analysis has several important limitations. First, as noted, we could only evaluate intent to influence or a person’s view on what IFIC could and was designed to achieve, and not actual incidences of influence. Relatedly, we could not assess the quality of the evidence produced by IFIC, which would be the scope of a future study. However, to demonstrate that IFIC may represent corporate backers, such evidence was not required. Second, the UCSF document archive is a pre-selected group of documents and does not represent the entire universe of documents available on IFIC. The archive contains documents that have been obtained by litigation, uploaded following receipt of batches via freedom of information request, or from documents uncovered by leaks. The communications therefore may be only parts of extended conversations and may be partial. It is likely that those documents most concerning from a public health standpoint were more likely to appear in the document archive. However, we also observed multiple instances where IFIC’s evidence was not linked to a clear expression of a position favourable or inimical to industry. Additionally, the archive contains communications by senior food industry figures over several decades, some of whom were speaking as former senior executives and others who held industry posts at the time of sending the communication. We note that we researched the background of each person writing the relevant document identified, to verify that they remained influential in the food industry at the time of sending the communication. Third, the documents in our sample failed to separate parent entities from their other arms and therefore distinguishing whether IFIC or the IFIC Foundation was being discussed was not possible in this study, with their roles often collapsed, reflecting the shared aims, objectives, activities, offices and staff.Despite the limitations of the evidence analysed, the email exchanges provide insight “in their own words” on senior level understandings, including from founders, and daily operations of IFIC and the IFIC Foundation. Other similar examples, including from different time periods and geographic locations, would help to confirm the extent of these activities, so further research should be conducted in the future.Finally, it is unclear the extent to which the observations about IFIC might be relevant to other industry funded communications organisations. There are suggestions, however, that other organisations adhere to a similar institutional model. For example, established in 1995, the European Food Information Council (EUFIC) similarly purports to be a neutral body with a mission “to produce science-based content to inspire and empower healthier and more sustainable diets and lifestyles” . Future ific.org email addresses throughout email chains. Our past research on ILSI exposed that it acted very clearly in the past on behalf of TCCC, among other food industry funders; a point ILSI has disputed [Our findings have important implications for IFIC as pertains to its standing as a leading nutrition communications organisation. We found evidence that some industry figures view IFIC as not only able to convene experts who are favourable to industry but does so on occasion to consolidate evidence in a way that could be product defence, especially when media or press coverage or academic studies are emerging that contain unfavourable claims around ingredients. While IFIC contends that its role is merely to promote scientific information, we found more than one instance discussed in the industry documents that suggest it acts to organize information into a “consensus science” on “hard-hitting issues” which is specifically favourable to companies that IFIC’s Form 990 drafts suggest fund either IFIC or its Foundation. We also found that IFIC plays a ‘rebuttal function’ in public debates, especially being called upon to galvanise a defence of ingredients or products when negative press has occurred. Leading industry players view IFIC as a ‘sister to ILSI’, in being central to promoting industry-favourable content in defence of products facing potentially negative press, such as aspartame, with widespread use of disputed , 35. In We argue that IFIC and its Foundation’s communications should be viewed as conducting marketing and public relations for the food industry. There is a clear need for better systems and transparency around science communications activity to manage complex veiled conflicts of interest, especially those from industry-funded entities like IFIC. As a start, following good governance practices, IFIC could publicly disclose on its website the funding it receives, including contributors, amounts, and purpose/s. Until then, we believe the starting assumption should be that IFIC acts as an agent of the food industry and be restricted from partnership with international health organisations until robust evidence is available to demonstrate that such conflicts of interest have been addressed.This case-study of IFIC adds a critical dimension to the literature on the commercial determinants of health, adding to evidence of how intermediary organizations work to obfuscate the industry role in their evidence summation and communications. Consistent with existing research on commercial determinants of health, we find that third-parties such as IFIC can be employed by underlying corporate sponsors to influence the perception and regulation of their products, as well as to muddy the water to lead to a perception that the evidence is not clear to support regulation; an activity that we know was well deployed by the tobacco industry , 37. Imp