Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses  as trustworthy.»A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.»Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field. Many authors do not recognize the complexity of such an endeavor and the many methodological challenges they may encounter. Failure to do so is likely to result in research and resource waste.Evidence syntheses are commonly regarded as the foundation of evidence-based medicine (EBM). They are widely accredited for providing reliable evidence and, as such, they have significantly influenced medical research and clinical practice. Despite their uptake throughout health care and ubiquity in contemporary medical literature, some important aspects of evidence syntheses are generally overlooked or not well recognized. Evidence syntheses are mostly retrospective exercises, they often depend on weak or irreparably flawed data, and they may use tools that have acknowledged or yet unrecognized limitations. They are complicated and time-consuming undertakings prone to bias and errors. Production of a good evidence synthesis requires careful preparation and high levels of organization in order to limit potential pitfalls2. Concerns about the trustworthiness of evidence syntheses are not recent developments. From the early years when EBM first began to gain traction until recent times when thousands of systematic reviews are published monthly3, the rigor of evidence syntheses has always varied. Many systematic reviews and meta-analyses had obvious deficiencies because original methods and processes had gaps, lacked precision, and/or were not widely known. The situation has improved with empirical research concerning which methods to use and standardization of appraisal tools. However, given the geometrical increase in the number of evidence syntheses being published, a relatively larger pool of unreliable evidence syntheses is being published today.Given their potential impact on people’s lives, it is crucial for evidence syntheses to correctly report on the current knowledge base. In order to be perceived as trustworthy, reliable demonstration of the accuracy of evidence syntheses is equally imperative6. Yet many clinical specialties report that alarming numbers of evidence syntheses fail on these assessments. The syntheses identified report on a broad range of common conditions including, but not limited to, cancer7, chronic obstructive pulmonary disease8, osteoporosis9, stroke10, cerebral palsy11, chronic low back pain12, refractive error13, major depression14, pain15, and obesity17. The situation is even more concerning with regard to evidence syntheses included in clinical practice guidelines (CPGs)20. Astonishingly, in a sample of CPGs published in 2017-18, more than half did not apply even basic systematic methods in the evidence syntheses used to inform their recommendations21.Publication of methodological studies that critically appraise the methods used in evidence syntheses is increasing at a fast pace. This reflects the availability of tools specifically developed for this purpose22. Similar concerns about the reliability of evidence syntheses have been expressed by proponents of EBM in highly circulated medical journals26. These publications have also raised awareness about redundancy, inadequate input of statistical expertise, and deficient reporting. These issues plague primary research as well; however, there is heightened concern for the impact of these deficiencies given the critical role of evidence syntheses in policy and clinical decision-making.These reports, while not widely acknowledged, suggest there are pervasive problems not limited to evidence syntheses that evaluate specific kinds of interventions or include primary research of a particular study design 27. These groups typically certify or commission systematic reviews that are published in exclusive databases  or are used to develop government or agency sponsored guidelines or health technology assessments . They offer developers of evidence syntheses various levels of methodological advice, technical and administrative support, and editorial assistance. Use of specific protocols and checklists are required for development teams within these groups, but their online methodological resources are accessible to any potential author.Several international consortiums of EBM experts and national health care organizations currently provide detailed guidance Table . They dr28. The World Health Organization requires Cochrane standards be used to develop evidence syntheses that inform their CPGs29. Authors investigating questions of intervention effectiveness in syntheses developed for Cochrane follow the Methodological Expectations of Cochrane Intervention Reviews30 and undergo multi-tiered peer review32. Several empirical evaluations have shown that Cochrane systematic reviews are of higher methodological quality compared to non-Cochrane reviews35. However, some of these assessments have biases: they may be conducted by Cochrane-affiliated authors, and they sometimes use scales and tools developed and used in the Cochrane environment and by its partners. In addition, evidence syntheses published in the Cochrane database are not subject to space or word restrictions, while non-Cochrane syntheses are often limited. As a result, information that may be relevant to the critical appraisal of non-Cochrane syntheses is often removed or is relegated to online-only supplements that may not be readily or fully accessible28.Notably, Cochrane is the largest single producer of evidence syntheses in biomedical research; however, these only account for 15% of the total28. Suboptimal methods described in in the literature may then be taken up by others. For example, the Newcastle-Ottawa Scale (NOS) is a commonly used tool for appraising non-randomized studies36. Many authors justify their selection of this tool with reference to a publication that describes the unreliability of the NOS and recommends against its use37. Obviously, the authors who cite this report for that purpose have not read it. Authors and peer reviewers have a responsibility to use reliable and accurate methods and not copycat previous citations or substandard work39. Similar cautions may potentially extend to automation tools. These have concentrated on evidence searching40 and selection given how demanding it is for humans to maintain truly up-to-date evidence41. Cochrane has deployed machine learning to identify randomized controlled trials (RCTs)2 and studies related to COVID-1942, but such tools are not yet commonly used43. The routine integration of automation tools in the development of future evidence syntheses should not displace the interpretive part of the process.Many authors are familiar with the evidence syntheses produced by the leading EBM organizations but can be intimidated by the time and effort necessary to apply their standards. Instead of following their guidance, authors may employ methods that are discouraged or outdated48. At this juncture, clarification of the extent to which each of these factors contribute remains speculative, but their impact is likely to be synergistic.Editorials about unreliable or misleading systematic reviews highlight several of the intertwining factors that may contribute to continued publication of unreliable evidence syntheses: shortcomings and inconsistencies of the peer review process, lack of endorsement of current standards on the part of journal editors, the incentive structure of academia, industry influences, publication bias, and the lure of “predatory” journals52. News and social media outlets also tend to reduce conclusions on a body of evidence and recommendations for treatment to binary choices  that may be assigned an actionable symbol .Over time, the generalized acceptance of the conclusions of systematic reviews as incontrovertible has affected trends in the dissemination and uptake of evidence. Reporting of the results of evidence syntheses and recommendations of CPGs has shifted beyond medical journals to press releases and news headlines and, more recently, to the realm of social media and influencers. The lay public and policy makers may depend on these outlets for interpreting evidence syntheses and CPGs. Unfortunately, communication to the general public often reflects intentional or non-intentional misrepresentation or “spin” of the research findings53. Informing them about research methodology could increase the likelihood they will apply rigorous methods45. We tackle this challenge, from both a theoretical and a practical perspective, by offering guidance applicable to any specialty. It is based on recent methodological research that is extensively referenced to promote self-study. However, the information presented is not intended to substitute for committed training in evidence synthesis methodology; instead, we hope to inspire our target audience to seek such training. We also hope to inform a broader audience of clinicians and guideline developers influenced by evidence syntheses. Notably, these communities often include the same members who serve in different capacities.Many authors and peer reviewers are volunteer health care professionals or trainees who lack formal training in evidence synthesisIn the following sections, we highlight methodological concepts and practices that may be unfamiliar, problematic, confusing, or controversial. In Part 2, we consider various types of evidence syntheses and the types of research evidence summarized by them. In Part 3, we examine some widely used (and misused) tools for the critical appraisal of systematic reviews and reporting guidelines for evidence syntheses. In Part 4, we discuss how to meet methodological conduct standards applicable to key components of systematic reviews. In Part 5, we describe the merits and caveats of rating the overall certainty of a body of evidence. Finally, in Part 6, we summarize suggested terminology, methods, and tools for development and evaluation of evidence syntheses that reflect current best practices.A good foundation for the development of evidence syntheses requires an appreciation of their various methodologies and the ability to correctly identify the types of research potentially available for inclusion in the synthesis.54. Systematic reviews with traditional components have become defined by the different topics they assess .Systematic reviews have historically focused on the benefits and harms of interventions; over time, various other types of systematic reviews have emerged to address the diverse information needs of clinicians, patients, and policy makersss Table . In addi59 and JBI60 provide methodologies for many types of evidence syntheses; they describe these with different terminology, but there is obvious overlap . A familiar example is the Design Tree endorsed by the Centre for Evidence-Based Medicine72. Problematic distinctions or decision points  and design labels  have been reported71. The variable application of ambiguous study design labels to non-randomized studies is common, making them especially prone to misclassification73. In addition, study labels do not denote the unique design features that make different types of non-randomized studies susceptible to different biases, including those related to how the data are obtained . Given this limitation, it is important to be aware that design labels preclude the accurate assignment of non-randomized studies to a “level of evidence” in traditional hierarchies74.Suboptimal levels of agreement and accuracy even among trained methodologists reflect challenges with the application of such tools76. Moreover, primary research reports often do not describe study design or do so incompletely or inaccurately; thus, indexing in PubMed and other databases does not address the potential for misclassification77. Yet proper identification of research evidence has implications for several key components of evidence syntheses. For example, search strategies limited by index terms using design labels or study selection based on labels applied by the authors of primary studies may cause inconsistent or unjustified study inclusions and/or exclusions77. In addition, because risk of bias (RoB) tools consider attributes specific to certain types of studies and study design features, results of these assessments may be invalidated if an inappropriate tool is used. Appropriate classification of studies is also relevant for the selection of a suitable method of synthesis and interpretation of those results.These concerns suggest that available tools and nomenclature used to distinguish types of research evidence may not uniformly apply to biomedical research and non-health fields that utilize evidence syntheses http://links.lww.com/JBJSREV/A979). It is important to conceptualize their methods as complementary as opposed to contrasting or hierarchical78; each offers advantages and disadvantages that determine their appropriateness for answering different kinds of research questions in an evidence synthesis.An alternative to these tools and nomenclature involves application of a few fundamental distinctions that encompass a wide range of research designs and contexts. While these distinctions are not novel, we integrate them into a practical scheme  designed to guide authors of evidence syntheses in the basic identification of research evidence. The initial distinction is between primary and secondary studies. Primary studies are then further distinguished by: 1) the type of data reported ; and 2) two defining design features (group or single-case and randomized or non-randomized). The different types of studies and study designs represented in the scheme are described in detail in Supplemental File 2B  are affected by a greater potential range of biases and thus vary more than RCTs in their ability to estimate a causal effect79. If data from NRSI are synthesized in meta-analyses, it is helpful to separately report their summary estimates74.When investigating an intervention’s effectiveness, it is important for authors to recognize the uncertainty of observed effects reported by studies with high RoB. Results of statistical analyses that include such studies need to be interpreted with caution in order to avoid misleading conclusions80. This process is facilitated by a study design feature checklist; guidance on using the checklist is included with developers’ description of the tool74. Authors collect information about these design features during data extraction and then consider it when making final study selection decisions and when performing RoB assessments of the included NRSI.Nonetheless, certain design features of NRSI  may help to distinguish stronger from weaker ones. Cochrane recommends that authors of a review including NRSI focus on relevant study design features when determining eligibility criteria instead of relying on non-informative study design labels81; in addition, some topics may be limited to a body of evidence that consists primarily of uncontrolled clinical observations. Murad and colleagues offer a framework for how to include case reports and series in an evidence synthesis82. Distinguishing between cohort studies and case series in these syntheses is important, especially for those that rely on evidence from NRSI. Additional data obtained from studies misclassified as case series can potentially increase the confidence in effect estimates. Mathes and Pieper provide authors of evidence syntheses with specific guidance on distinguishing between cohort studies and case series, but emphasize the increased workload involved77.Correctly identified case reports and case series can contribute evidence not well captured by other designsSynthesis of combined evidence from primary and secondary studies may provide a broad perspective on the entirety of available literature on a topic. This is, in fact, the recommended strategy for scoping reviews that may include a variety of sources of evidence . However, except for scoping reviews, the synthesis of data from primary and secondary studies is discouraged unless there are strong reasons to justify doing so.83. Assessments of RoB for primary and secondary studies are derived from conceptually different tools, thus obfuscating the ability to make an overall RoB assessment of a combination of these study types. In addition, authors who include primary and secondary studies must devise non-standardized methods for synthesis. Note this contrasts with well-established methods available for updating existing evidence syntheses with additional data from new primary studies86. However, a new review that synthesizes data from primary and secondary studies raises questions of validity and may unintentionally support a biased conclusion because no existing methodological guidance is currently available87.Combining primary and secondary sources of evidence is challenging for authors of other types of evidence syntheses for several reasonsWe suggest that journal editors require authors to identify which type of evidence synthesis they are submitting and reference the specific methodology used for its development. This will clarify the research question and methods for peer reviewers and potentially simplify the editorial process. Editors should announce this practice and include it in the instructions to authors. To decrease bias and apply correct methods, authors must also accurately identify the types of research evidence included in their syntheses.89. Systematic reviews rapidly became popular, but many were very poorly conceived, conducted, and reported. These problems remain highly prevalent23 despite development of guidelines and tools to standardize and improve the performance and reporting of evidence syntheses28. Table The need to develop criteria to assess the rigor of systematic reviews was recognized soon after the EBM movement began to gain international traction5. RoB assessments refer to systematic flaws or limitations in the design, conduct, or analysis of research that distort the findings of the review4. Reporting standards help systematic review authors describe the methodology they used and the results of their synthesis in sufficient detail92. It is essential to distinguish between these evaluations: a systematic review may be biased, it may fail to report sufficient information on essential features, or it may exhibit both problems. A thoroughly reported systematic evidence synthesis review may still be biased and flawed, while an otherwise unbiased one may suffer from deficient documentation.These tools are often interchangeably invoked when referring to the “quality” of an evidence synthesis. However, quality is a vague term that is frequently misused and misunderstood; more precisely, these tools specify different standards for evidence syntheses. Methodological standards address how well a systematic review was designed and performed97. We highlight the underlying constructs these tools were designed to assess, then describe their components and applications. Their known  uptake and impact and limitations are also discussed.We direct attention to the currently recommended tools listed in Table 5 was in use for a decade prior to the 2017 publication of AMSTAR-2; both provide a broad evaluation of methodological quality of intervention systematic reviews, including flaws arising through poor conduct of the review6. ROBIS, published in 2016, was developed to specifically assess RoB introduced by the conduct of the review; it is applicable to systematic reviews of interventions and several other types of reviews4. Both tools reflect a shift to a domain-based approach as opposed to generic quality checklists. There are a few items unique to each tool; however, similarities between items have been demonstrated99. AMSTAR-2 and ROBIS are recommended for use by: 1) authors of overviews or umbrella reviews and CPGs to evaluate systematic reviews considered as evidence; 2) authors of methodological research studies to appraise included systematic reviews; and 3) peer reviewers for appraisal of submitted systematic review manuscripts. For authors, these tools may function as teaching aids and inform conduct of their review during its development.AMSTAR6. AMSTAR-2 focuses on the rigor of a review’s methods irrespective of the specific subject matter. ROBIS places emphasis on a review’s results section. This suggests it may be optimally applied by appraisers with some knowledge of the review’s topic as they may be better equipped to determine if certain procedures (or lack thereof) would impact the validity of a review’s findings100. Reliability studies show AMSTAR-2 overall confidence ratings strongly correlate with the overall RoB ratings in ROBIS101.Systematic reviews that include randomized and/or non-randomized studies as evidence can be appraised with AMSTAR-2 and ROBIS. Other characteristics of AMSTAR-2 and ROBIS are summarized in Table 102 and ROBIS103 but neither tool has been shown to be superior in this regard105. Overall, variability in reliability for both tools has been reported across items, between pairs of raters, and between centers104. The effects of appraiser experience on the results of AMSTAR-2 and ROBIS requires further evaluation105. Updates to both tools should address items shown to be prone to individual appraisers’ subjective biases and opinions100; this may involve modifications of the current domains and signaling questions as well as incorporation of methods to make an appraiser’s judgments more explicit. Future revisions of these tools may also consider the addition of standards for aspects of systematic review development currently lacking  and removal of items that assess aspects of reporting that are thoroughly evaluated by PRISMA 2020.Interrater reliability has been shown to be acceptable for AMSTAR-2106. These methods may, to some extent, address the challenges associated with the diversity in methodological training, subject matter expertise, and experience using the tools that are likely to exist among appraisers.A good understanding of what is required to satisfy the standards of AMSTAR-2 and ROBIS involves study of the accompanying guidance documents written by the tools’ developers; these contain detailed descriptions of each item’s standards. In addition, accurate appraisal of a systematic review with either tool requires training. Most experts recommend independent assessment by at least two appraisers, with a process for resolving discrepancies as well as procedures to establish interrater reliability, such as pilot testing, a calibration phase or exercise, and development of predefined decision rules55 and JBI107 recommend AMSTAR-2 and ROBIS in their guidance for authors of overviews or umbrella reviews. However, ROBIS and AMSTAR-2 were released in 2016 and 2017, respectively; thus, to date, limited data have been reported about the uptake of these tools or which of the two may be preferred106. Currently, in relation to CPGs, AMSTAR-2 appears to be overwhelmingly popular compared to ROBIS. A Google Scholar search of this topic  found 12,700 hits for AMSTAR-2 and 1,280 for ROBIS. The apparent greater appeal of AMSTAR-2 may relate to its longer track record given the original version of the tool was in use for 10 years prior to its update in 2017.The standards of AMSTAR, AMSTAR-2, and ROBIS have been used in many methodological studies and epidemiological investigations. However, the increased publication of overviews or umbrella reviews and CPGs has likely been a greater influence on the widening acceptance of these tools. Critical appraisal of the secondary studies considered evidence is essential to the trustworthiness of both the recommendations of CPGs and the conclusions of overviews. Currently both Cochrane104. Reports from comparative studies available to date indicate that appraisers find AMSTAR-2 questions, responses, and guidance to be clearer and simpler compared with ROBIS105. This suggests that for appraisal of intervention systematic reviews, AMSTAR-2 may be a more practical tool than ROBIS, especially for novice appraisers105. The unique characteristics of each tool, as well as their potential advantages and disadvantages, should be taken into consideration when deciding which tool should be used for an appraisal of a systematic review. In addition, the choice of one or the other may depend on how the results of an appraisal will be used—for example, a peer reviewer’s appraisal of a single manuscript versus an appraisal of multiple systematic reviews in an overview or umbrella review, CPG, or systematic methodological study.Barriers to the uptake of AMSTAR-2 and ROBIS include the real or perceived time and resources necessary to complete the items they include and appraisers’ confidence in their own ratings108. When these judgments are explicit, the underlying rationale used when applying these tools can be assessed109.Authors of overviews and CPGs report results of AMSTAR-2 and ROBIS appraisals for each of the systematic reviews they include as evidence. Ideally, an independent judgment of their appraisals can be made by the end users of overviews and CPGs; however, most stakeholders, including clinicians, are unlikely to have a sophisticated understanding of these tools. Nevertheless, they should at least be aware that AMSTAR-2 and ROBIS ratings reported in overviews and CPGs may be inaccurate because the tools are not applied as intended by their developers. This can result from inadequate training of the overview or CPG authors who perform the appraisals, or to modifications of the appraisal tools imposed by them. The potential variability in overall confidence and RoB ratings highlights why appraisers applying these tools need to support their judgments with explicit documentation; this allows readers to judge for themselves whether they agree with the criteria used by appraisers109.Theoretically, we would expect an association of AMSTAR-2 with improved methodological rigor and an association of ROBIS with lower RoB in recent systematic reviews compared to those published before 2017. To our knowledge, this has not yet been demonstrated; however, like reports about the actual uptake of these tools, time will tell. Additional data on user experience is also needed to further elucidate the practical challenges and methodological nuances encountered with the application of these tools. This information could potentially inform the creation of unifying criteria to guide and standardize the appraisal of evidence syntheses92 with its accompanying explanation and elaboration document110. This guideline was designed to help authors prepare a complete and transparent report of their systematic review. In addition, adherence to PRISMA is often used to evaluate the thoroughness of reporting of published systematic reviews111. The updated version, PRISMA 202093, and its guidance document112 were published in 2021. Items on the original and updated versions of PRISMA are organized by the six basic review components they address . The PRISMA 2020 update is a considerably expanded version of the original; it includes standards and examples for the 27 original and 13 additional reporting items that capture methodological advances and may enhance the replicability of reviews113.Complete reporting is essential for users to establish the trustworthiness and applicability of a systematic review’s findings. Efforts to standardize and improve the reporting of systematic reviews resulted in the 2009 publication of the PRISMA statementThe original PRISMA statement fostered the development of various PRISMA extensions Table . These i92 for reporting have been widely endorsed by authors, journals, and EBM-related organizations. We anticipate the same for PRISMA 202093 given its co-publication in multiple high-impact journals. However, to date, there is a lack of strong evidence for an association between improved systematic review reporting and endorsement of PRISMA 2009 standards111. Most journals require a PRISMA checklist accompany submissions of systematic review manuscripts. However, the accuracy of information presented on these self-reported checklists is not necessarily verified. It remains unclear which strategies  might improve adherence to the PRISMA reporting standards; in addition, the feasibility of any potentially effective strategies must be taken into consideration given the structure and limitations of current research and publication practices124.The 2009 PRISMA standardsMisunderstanding of the roles of these tools and their misapplication may be widespread problems. PRISMA 2020 is a reporting guideline that is most beneficial if consulted when developing a review as opposed to merely completing a checklist when submitting to a journal; at that point, the review is finished, with good or bad methodological choices. However, PRISMA checklists evaluate how completely an element of review conduct was reported, but do not evaluate the caliber of conduct or performance of a review. Thus, review authors and readers should not think that a rigorous systematic review can be produced by simply following the PRISMA 2020 guidelines. Similarly, it is important to recognize that AMSTAR-2 and ROBIS are tools to evaluate the conduct of a review but do not substitute for conceptual methodological guidance. In addition, they are not intended to be simple checklists. In fact, they have the potential for misuse or abuse if applied as such; for example, by calculating a total score to make a judgment about a review’s overall confidence or RoB. Proper selection of a response for the individual items on AMSTAR-2 and ROBIS requires training or at least reference to their accompanying guidance documents.125. AMSTAR and ROBIS were not available when PRISMA 2009 was developed; however, they were considered in the development of PRISMA 2020113. Therefore, future studies may show a positive relationship between fulfillment of PRISMA 2020 standards for reporting and meeting the standards of tools evaluating methodological quality and RoB.Not surprisingly, it has been shown that compliance with the PRISMA checklist is not necessarily associated with satisfying the standards of ROBISChoice of an appropriate tool for the evaluation of a systematic review first involves identification of the underlying construct to be assessed. For systematic reviews of interventions, recommended tools include AMSTAR-2 and ROBIS for appraisal of conduct and PRISMA 2020 for completeness of reporting. All three tools were developed rigorously and provide easily accessible and detailed user guidance, which is necessary for their proper application and interpretation. When considering a manuscript for publication, training in these tools can sensitize peer reviewers and editors to major issues that may affect the review’s trustworthiness and completeness of reporting. Judgment of the overall certainty of a body of evidence and formulation of recommendations rely, in part, on AMSTAR-2 or ROBIS appraisals of systematic reviews. Therefore, training on the application of these tools is essential for authors of overviews and developers of CPGs. Peer reviewers and editors considering an overview or CPG for publication must hold their authors to a high standard of transparency regarding both the conduct and reporting of these appraisals.69 as well as primers with basic step-by-step instructions127. This section is intended to complement study of such resources by facilitating use of AMSTAR-2 and ROBIS, tools specifically developed to evaluate methodological rigor of systematic reviews. These tools are widely accepted by methodologists; however, in the general medical literature, they are not uniformly selected for the critical appraisal of systematic reviews96.Many authors, peer reviewers, and editors erroneously equate fulfillment of the items on the PRISMA checklist with superior methodological rigor. For direction on methodology, we refer them to available resources that provide comprehensive conceptual guidanceTo enable their uptake, Table 129; and/or 2) the review component is judged by standards of an AMSTAR-2 “critical” domain. These have the greatest implications for how a systematic review will be appraised: if standards for any one of these critical domains are not met, the review is rated as having “critically low confidence.”Issues involved in meeting the standards for seven review components , for example, only registers reviews that report on an outcome with direct relevance to human health. The PROSPERO record documents protocols for all types of reviews except literature and scoping reviews. Of note, PROSPERO requires authors register their review protocols prior to any data extraction138. The electronic records of most of these registry sites allow authors to update their protocols and facilitate transparent tracking of protocol changes, which are not unexpected during the progress of the review139.There are multiple options for the recording of protocols Table . Some jo126. This may allow comparison of results between contrasting study design types126. Certain study designs may be considered preferable depending on the type of review and nature of the research question. However, prevailing stereotypes about what each study design does best may not be accurate. For example, in systematic reviews of interventions, randomized designs are typically thought to answer highly specific questions while non-randomized designs often are expected to reveal greater information about harms or real-word evidence141. This may be a false distinction; randomized trials may be pragmatic142, they may offer important (and more unbiased) information on harms143, and data from nonrandomized trials may not necessarily be more real-world-oriented144.For most systematic reviews, broad inclusion of study designs is recommended145. When authors anticipate that limited evidence of any kind may be available to inform their research questions, a scoping review can be considered. Alternatively, decisions regarding inclusion of indirect as opposed to direct evidence can be addressed during protocol development146. Including indirect evidence at an early stage of intervention systematic review development allows authors to decide if such studies offer any additional and/or different understanding of treatment effects for their population or comparison of interest. Issues of indirectness of included studies are accounted for later in the process, during determination of the overall certainty of evidence (see Part 5 for details).Moreover, there may not be any available evidence reported by RCTs for certain research questions; in some cases, there may not be any RCTs or NRSI. When the available evidence is limited to case reports and case series, it is not possible to test hypotheses nor provide descriptive estimates or associations; however, a systematic review of these studies can still offer important insights126. There are many examples in which language bias (different results in studies published in different languages) has been documented148. This does not mean that all literature, in all languages, is equally trustworthy148; however, the only way to formally probe for the potential of such biases is to consider all languages in the initial search. The gray literature and a search of trials may also reveal important details about topics that would otherwise be missed151. Again, inclusiveness will allow review authors to investigate whether results differ in gray literature and trials153.Both AMSTAR-2 and ROBIS require systematic and comprehensive searches for evidence. This is essential for any systematic review. Both tools discourage search restrictions based on language and publication source. Given increasing globalism in health care, the practice of including English-only literature should be avoided1. If that is not possible, the search should be updated close to the time of completion154. Different research topics may warrant less of a delay, for example, in rapidly changing fields (as in the case of the COVID-19 pandemic), even one month may radically change the available evidence.Authors should make every attempt to complete their review within one year as that is the likely viable life of a searchAMSTAR-2 requires authors to provide references for any studies excluded at the full-text phase of study selection along with reasons for exclusion; this allows readers to feel confident that all relevant literature has been considered for inclusion and that exclusions are defensible.82 and single-case experimental designs156. Note the RoB tools selected must meet the standards of the appraisal tool used to judge the conduct of the review. For example, AMSTAR-2 identifies four sources of bias specific to RCTs and NRSI that must be addressed by the RoB tool(s) chosen by the review authors. The Cochrane RoB-2157 tool for RCTs and ROBINS-I158 for NRSI for RoB assessment meet the AMSTAR-2 standards. Appraisers on the review team should not modify any RoB tool without complete transparency and acknowledgment that they have invalidated the interpretation of the tool as intended by its developers159. Conduct of RoB assessments is not addressed in AMSTAR-2; to meet ROBIS standards, two independent reviewers should complete RoB assessments of included primary studies.The design of the studies included in a systematic review  should not be equated with appraisal of its RoB. To meet AMSTAR-2 and ROBIS standards, systematic review authors must examine RoB issues specific to the design of each primary study they include as evidence. It is unlikely that a single RoB appraisal tool will be suitable for all research designs. In addition to tools for randomized and non-randomized studies, specific tools are available for evaluation of RoB in case reports and case seriesImplications of the RoB assessments must be explicitly discussed and considered in the conclusions of the review. Discussion of the overall RoB of included studies may consider the weight of the studies at high RoB, the importance of the sources of bias in the studies being summarized, and if their importance differs in relationship to the outcomes reported. If a meta-analysis is performed, serious concerns for RoB of individual studies should be accounted for in these results as well. If the results of the meta-analysis for a specific outcome change when studies at high RoB are excluded, readers will have a more accurate understanding of this body of evidence. However, while investigating the potential impact of specific biases is a useful exercise, it is important to avoid over-interpretation, especially when there are sparse data.161.Syntheses of quantitative data reported by primary studies are broadly categorized as one of two types: meta-analysis, and synthesis without meta-analysis Table . Before 167.Systematic reviews that employ meta-analysis should not be referred to simply as “meta-analyses.” The term meta-analysis strictly refers to a specific statistical technique used when study effect estimates and their variances are available, yielding a quantitative summary of results. In general, methods for meta-analysis involve use of a weighted average of effect estimates from two or more studies. If considered carefully, meta-analysis increases the precision of the estimated magnitude of effect and can offer useful insights about heterogeneity and estimates of effects. We refer to standard references for a thorough introduction and formal training161. Forest plots visually present essential information about the individual studies and the overall pooled analysis .There are three common approaches to meta-analysis in current health care–related systematic reviews Table . Aggrega118. In IPD, the raw data on each participant from each eligible study are re-analyzed as opposed to the study-level data analyzed in aggregate data meta-analyses168. This may offer advantages, including the potential for limiting concerns about bias and allowing more robust analyses163. As suggested by the description on Table 169 or IPD170 for effect estimates from direct and indirect comparisons reported in two or more studies of three or more interventions. This makes it a potentially powerful statistical tool; while multiple interventions are typically available to treat a condition, few have been evaluated in head-to-head trials171. Both IPD and NMA facilitate a broader scope, and potentially provide more reliable and/or detailed results; however, compared to standard aggregate data meta-analyses, their methods are more complicated, time-consuming, and resource-intensive, and they have their own biases, so one needs sufficient funding, technical expertise, and preparation to employ them successfully173.Less familiar and more challenging meta-analytical approaches used in secondary research include individual participant data (IPD) and network meta-analyses (NMA); PRISMA extensions provide reporting guidelines for both174, MetaXL175, JBI SUMARI176) may facilitate exploration of these methods; however, such programs cannot substitute for the accurate interpretation of the results of meta-analyses, especially for more complex meta-analytical approaches.Several items in AMSTAR-2 and ROBIS address meta-analysis; thus, understanding the strengths, weaknesses, assumptions, and limitations of methods for meta-analyses is important. According to the standards of both tools, plans for a meta-analysis must be addressed in the review protocol, including reasoning, description of the type of quantitative data to be synthesized, and the methods planned for combining the data. This should not consist of stock statements describing conventional meta-analysis techniques; rather, authors are expected to anticipate issues specific to their research questions. Concern for the lack of training in meta-analysis methods among systematic review authors cannot be overstated. For those with training, the use of popular software . Unacceptable methods include vote counting by statistical significance or magnitude of effect or some subjective rule applied by the authors.Vote counting is commonly used in systematic reviews and involves a tally of studies reporting results that meet some threshold of importance applied by review authors. Until recently, it has not typically been identified as a method for synthesis without meta-analysis. Guidance on an acceptable vote counting method based on direction of effect is currently available180 and methodological guidance is available in the Cochrane Handbook181.AMSTAR-2 and ROBIS standards do not explicitly address conduct of syntheses without meta-analysis, although AMSTAR-2 items 13 and 14 might be considered relevant. Guidance for the complete reporting of syntheses without meta-analysis for systematic reviews of interventions is available in the Synthesis without Meta-analysis (SWiM) guidelineFamiliarity with AMSTAR-2 and ROBIS makes sense for authors of systematic reviews as these appraisal tools will be used to judge their work; however, training is necessary for authors to truly appreciate and apply methodological rigor. Moreover, judgment of the potential contribution of a systematic review to the current knowledge base goes beyond meeting the standards of AMSTAR-2 and ROBIS. These tools do not explicitly address some crucial concepts involved in the development of a systematic review; this further emphasizes the need for author training.130 and to stimulate careful contemplation of the scope of the review30. Authors’ training should also focus on justifiably choosing a formal method for the synthesis of quantitative and/or qualitative data from primary research; both types of data require specific expertise. For typical reviews that involve syntheses of quantitative data, statistical expertise is necessary, initially for decisions about appropriate methods161, and then to inform any meta-analyses167 or other statistical methods applied160.We recommend that systematic review authors incorporate specific practices or exercises when formulating a research question at the protocol stage, These should be designed to raise the review team’s awareness of how to prevent research and resource waste93. Systematic review authors are well acquainted with assessing RoB in individual primary studies, but much less familiar with assessment of overall certainty across an entire body of evidence. Yet a reliable way to evaluate this broader concept is now recognized as a vital part of interpreting the evidence.Report of an overall certainty of evidence assessment in a systematic review is an important new reporting standard of the updated PRISMA 2020 guidelines182. A critical appraisal of systems used by prominent health care organizations published in 2004 revealed limitations in sensibility, reproducibility, applicability to different questions, and usability to different end users183. Persistent use of hierarchical rating schemes to describe overall quality continues to complicate the interpretation of evidence. This is indicated by recent reports of poor interpretability of systematic review results by readers186 and misleading interpretations of the evidence related to the “spin” systematic review authors may put on their conclusions187.Historical systems for rating evidence are based on study design and usually involve hierarchical levels or classes of evidence that use numbers and/or letters to designate the level/class. These systems were endorsed by various EBM-related organizations; professional societies and regulatory groups then widely adopted them, often with modifications for application to the available primary research base in specific clinical areas. In 2002, a report issued by the AHRQ identified 40 systems to rate quality of a body of evidence183. These issues contributed to the development of the GRADE approach. An international working group, who continues to actively evaluate and refine it, first introduced GRADE in 2004188. Currently more than 110 organizations from 19 countries around the world have endorsed or are using GRADE189.Recognition of the shortcomings of hierarchical rating systems raised concerns that misleading clinical recommendations could result even if based on a rigorous systematic review. In addition, the number and variability of these systems were considered obstacles to quick and accurate interpretations of the evidence by clinicians, patients, and policymakers190. We briefly examine the GRADE approach but only as it applies to rating overall certainty of evidence in systematic reviews.GRADE offers a consistent and sensible approach for two separate processes: rating the overall certainty of a body of evidence and the strength of recommendations. The former is the expected conclusion of a systematic review, while the latter is pertinent to the development of CPGs. As such, GRADE provides a mechanism to bridge the gap from evidence synthesis to application of the evidence for informed clinical decision-making191. Certainty refers to the level of confidence systematic review authors have that, for each outcome, an effect estimate represents the true effect. The GRADE approach to rating confidence in estimates begins with identifying the study type (RCT or NRSI) and then systematically considers criteria to rate the certainty of evidence up or down . Thus, the proper application of GRADE requires systematic review authors to take an overall view of the body of evidence and explicitly describe the rationale for their final ratings.GRADE’s developers acknowledge some subjectivity is involved in this processwn Table  as well wn Table  reflect 191. These outcomes must inform the research questions  that are specified a priori in a systematic review protocol. Evidence for these outcomes is then investigated and each critical or important outcome is ultimately assigned a certainty of evidence as the end point of the review. Notably, limitations of the included studies have an impact at the outcome level. Ultimately, the certainty ratings for each outcome reported in a systematic review are considered by guideline panels. They use a different process to formulate recommendations that involves assessment of the evidence across outcomes201. It is beyond our scope to describe the GRADE process for formulating recommendations; however, it is critical to understand how these two outcome-centric concepts of certainty of evidence in the GRADE framework are related and distinguished. An in-depth illustration using examples from recently published evidence syntheses and CPGs is provided in Supplemental File 5A .Outcomes important to the individuals who experience the problem of interest maintain a prominent role throughout the GRADE process27. In fact, most interventions and outcomes in medicine have low or very low certainty of evidence based on GRADE and there seems to be no major improvement over time203. This is still a very important (even if sobering) realization for calibrating our understanding of medical evidence. A major appeal of the GRADE approach is that it offers a common framework that enables authors of evidence syntheses to make complex judgments about evidence certainty and to convey these with unambiguous terminology. This prevents some common mistakes made by review authors, including overstating results (or under-reporting harms)186 and making recommendations for treatment. This is illustrated in Table SF5A-2 , which compares the concluding statements made about overall certainty in a systematic review with and without application of the GRADE approach.The GRADE approach is applicable irrespective of whether the certainty of the primary research evidence is high or very low; in some circumstances, indirect evidence of higher certainty may be considered if direct evidence is unavailable or of low certainty204. However, others report variable agreement among those experienced in GRADE assessments of evidence certainty190. Like any other tool, GRADE requires training in order to be properly applied. The intricacies of the GRADE approach and the necessary subjectivity involved suggest that improving agreement may require strict rules for its application; alternatively, use of general guidance and consensus among review authors may result in less consistency but provide important information for the end user190.Theoretically, application of GRADE should improve consistency of judgments about certainty of evidence, both between authors and across systematic reviews. In one empirical evaluation conducted by the GRADE Working Group, interrater reliability of two individual raters assessing certainty of the evidence for a specific outcome increased from ∼0.3 without using GRADE to ∼0.7 by using GRADE191, which is different from rating overall certainty across outcomes considered in the formulation of recommendations205. Modifications of standard GRADE methods and terminology are discouraged as these may detract from GRADE’s objectives to minimize conceptual confusion and maximize clear communication206.Simply invoking “the GRADE approach” does not automatically ensure GRADE methods were employed by authors of a systematic review (or developers of a CPG). Table 208, which can distort a systematic review’s conclusions about the certainty of evidence. Systematic review authors without proper GRADE training are likely to misinterpret the terms “quality” and “grade” and to misunderstand the constructs assessed by GRADE versus other appraisal tools. For example, review authors may reference the standard GRADE certainty ratings  and included in Table Other caveats pertain to application of a GRADE certainty of evidence rating in various types of evidence syntheses. Current adaptations of GRADE are described in Supplemental File 5B  are not reliable. This relates in part to the fact that their authors, who are often clinicians, can be overwhelmed by the plethora of ways to evaluate evidence. They tend to resort to familiar but often inadequate, inappropriate, or obsolete methods and tools and, as a result, produce unreliable reviews. These manuscripts may not be recognized as such by peer reviewers and journal editors who may disregard current standards. When such a systematic review is published or included in a CPG, clinicians and stakeholders tend to believe that it is trustworthy. A vicious cycle in which inadequate methodology is rewarded and potentially misleading conclusions are accepted is thus supported. There is no quick or easy way to break this cycle; however, increasing awareness of best practices among all these stakeholder groups, who often have minimal (if any) training in methodology, may begin to mitigate it. This is the rationale for inclusion of Parts 2 through 5 in this guidance document. These sections present core concepts and important methodological developments that inform current standards and recommendations. We conclude by taking a direct and practical approach.93. In addition, we have identified several problematic expressions and nomenclature. On Table Inconsistent and imprecise terminology used in the context of development and evaluation of evidence syntheses is problematic for authors, peer reviewers and editors and may lead to the application of inappropriate methods and tools. In response, we endorse use of the basic terms Table  defined http://links.lww.com/JBJSREV/A983).We also propose a Concise Guide Table  that sumTo encourage uptake of best practices, journal editors may consider adopting or adapting the Concise Guide in their instructions to authors and peer reviewers of evidence syntheses. Given the evolving nature of evidence synthesis methodology, the suggested methods and tools are likely to require regular updates. Authors of evidence syntheses should monitor the literature to ensure they are employing current methods and tools. Some types of evidence syntheses  are not included in the Concise Guide; for these, authors are advised to obtain recommendations for acceptable methods by consulting with their target journal.We encourage the appropriate and informed use of the methods and tools discussed throughout this commentary and summarized in the Concise Guide Table . However24. Many intervention systematic reviews, including those developed by Cochrane203 and those applying GRADE202, ultimately find no evidence, or find the evidence to be inconclusive . This often reflects the primary research base; however, it is important to know what is known (or not known) about a topic when considering an intervention for patients and discussing treatment options with them.In line with our main objective, the suggested tools in the Concise Guide address the reliability of evidence syntheses; however, we recognize that the utility of systematic reviews is an equally important concern. An unbiased and thoroughly reported evidence synthesis may still not be highly informative if the evidence itself that is summarized is sparse, weak and/or biased183. It is becoming increasingly apparent that this approach may not be appropriate for all topics130. Development of the GRADE approach has facilitated a better understanding of significant factors (beyond effect size) that contribute to the overall certainty of evidence. Other notable responses include the development of integrative synthesis methods for the evaluation of complex interventions231, the incorporation of crowdsourcing and machine learning into systematic review workflows 2, the shift in paradigm to living systemic review and NMA platforms233, and the proposal of a new evidence ecosystem that fosters bidirectional collaborations and interactions among a global network of evidence synthesis stakeholders234. These evolutions in data sources and methods may ultimately make evidence syntheses more streamlined, less duplicative, and more importantly, they may be more useful for timely policy and clinical decision-making; however, that will only be the case if they are rigorously reported and conducted.Alternatively, the frequency of “empty” and inconclusive reviews published in the medical literature may relate to limitations of conventional methods that focus on hypothesis testing; these have emphasized the importance of statistical significance in primary research and effect sizes from aggregate meta-analysesWe look forward to others’ ideas and proposals for the advancement of methods for evidence syntheses. For now, we encourage dissemination and uptake of the currently accepted best tools and practices for their development and evaluation; at the same time, we stress that uptake of appraisal tools, checklists, and software programs cannot substitute for proper education in the methodology of evidence syntheses and meta-analysis. Authors, peer reviewers, and editors must strive to make accurate and reliable contributions to the present evidence knowledge base; online alerts, upcoming technology, and accessible education may make this more feasible than ever before. Our intention is to improve the trustworthiness of evidence syntheses across disciplines, topics, and types of evidence syntheses. All of us must continue to study, teach, and act cooperatively for that to happen.Michelle Oakman Hayes for her assistance with the graphics, Mike Clarke for his willingness to answer our seemingly arbitrary questions, and Bernard Dan for his encouragement of this project.The work of John Ioannidis has been supported by an unrestricted gift from Sue and Bob O’Donnell to Stanford University.All authors contributed to the development of the ideas, writing, and the final review of the submitted manuscript.jbjs.org. . This content was not copyedited or verified by JBJS.Supporting material provided by the authors is posted with the online version of this article as a data supplement at


We are grateful for the relevant comments by Hu et al.  for our In addition, we would like to address their thoughts on how to conduct a “[…] study to establish applicable guidelines and encourage standardized clinical practice”. As the authors write , the US It is common in medical practice to promote interventions based on evidence of their effectiveness and safety. At present, there are few effective therapies for CRF in breast cancer patients. Even though the evidence for the effectiveness of acupuncture is weak, this is also true for most other options. “Although evidence of the beneficial effects on fatigue outcomes is not particularly strong for interventions such as acupuncture, massage, or bright light, use of these therapies in clinical practice for fatigue management can be rationalized based on the fact that they are generally well tolerated and may be efficacious in particular fatigue contexts or may favorably affect symptoms that amplify fatigue, including anxiety, depression, sleep disturbance, and pain.” . As Hu a


Evidence-based medicine integrates results from randomized controlled trials (RCTs) and meta-analyses, combining the best external evidence with individual clinical expertise and patients’ preferences. However, RCTs of surgery differ from those of medicine in that surgical performance is often assumed to be consistent. Yet, evaluating whether each surgery is performed to the same standard is quite challenging. As a primary issue, the novelty of this review is to emphasize—with a focus on orthopedic trauma—the advantage of having complete intra-operative image documentation, allowing the direct evaluation of the quality of the intra-operative technical performance. The absence of complete intra-operative image documentation leads to the inhomogeneity of case series, yielding inconsistent results due to the impossibility of a secondary analysis. Thus, comparisons and the reproduction of studies are difficult. Access to complete intra-operative image data in surgical RCTs allows not only secondary analysis but also comparisons with similar cases. Such complete data can be included in electronic papers. Offering these data to peers—in an accessible link—when presenting papers facilitates the selection process and improves publications for readers. Additionally, having access to the full set of image data for all presented cases serves as a rich resource for learning. It enables the reader to sift through the information and pinpoint the details that are most relevant to their individual needs, allowing them to potentially incorporate this knowledge into daily practice. A broad use of the concept of complete intra-operative image documentation is pivotal for bridging the gap between clinical research findings and real-world applications. Enhancing the quality of surgical RCTs would facilitate the equalization of evidence acquisition in both internal medicine and surgery. Joint effort by surgeons, scientific societies, publishers, and healthcare authorities is needed to support the ideas, implement economic requirements, and overcome the mental obstacles to its realization. A review of a complex and disputed topic like Evidence-Based Surgery should include a detailed historical overview and a critical analysis of the most important available studies, synthesizing the resulting evidence in a usable form for a broad spectrum of readers ,2. NarroThe core principles of evidence-based medicine are established through many randomized controlled trials (RCTs) and meta-analyses. However, heterogeneity in surgical trials, such as surgeon-related factors , and methodological challenges , that there is significant room for improvement in the methodological reporting of orthopedic trials . RegardiAs the intervention-reporting quality determines the feasibility of reproducing a study, this situation needs urgent changes. Intervention modifications and intervention fidelity are the two items that are most often omitted. This gives rise to the potential for type I errors, where a non-significant outcome is incorrectly perceived as significant, and type II errors, where a significant outcome is mistakenly deemed non-significant. Modifications, like a change in the material used, must be documented. Intervention fidelity is equally important and should be analyzed. A failure to document such changes inevitably produces unacceptable inhomogeneity among the case groups, causing the whole set of results to become questionable.The difficulty in creating homogeneous groups when developing surgical randomized controlled trials has been identified as an important problem . In mediAmong the specific problems, the single factor of the skill-dependent quality of the intra-operative technical performance and its variations is not discussed seriously enough in the literature regarding Evidence-Based Surgery, despite its great importance regarding the outcomes of surgical treatment ,28. SkilAt present, data formats like those proposed by the ICUC working group ,33,34,35The construction of a comprehensive image dataset must adhere to the Health Insurance Portability and Accountability Act (HIPAA) standards for de-identification protocols and necessitates patient consent being obtained . With teThe advancements in information technology should now allow adapted publication formats to present the high data volume resulting from a complete documentation format by a link. Furthermore, it seems legitimate to assume that, in the near future, supervised machine learning and pattern recognition will facilitate the analysis of big image data. This would also allow us to better understand the details of surgical procedures. As a result, current paradigms must be challenged by the collaboration of scientists, surgeons, industry, and regulatory authorities. The clinical expertise of the individual surgeon and their intra-operative performance quality represent the most relevant components of Evidence-Based Surgery ,30.Along with the careful consideration of indications, clinical expertise includes the careful indication and a sufficient number of treated cases for a defined clinical problem. The correlation between the caseload and outcome quality is sound. However, the precise number needed to reach the expertise level is difficult to define and can vary from one surgeon to another .The transition to digital technology indicates that information is increasingly being sourced from alternative sources to textbooks, such as mobile phones. Yet, the data used for “guidelines” still suffer from the abovementioned “technical performance bias .” The reSeveral factors lead to the challenges in conducting surgical RCTs. Among the surgeon-related factors, a lack of corresponding basic training and knowledge of epidemiology and statistics might be relevant. This can easily be overcome by clinical trial units (CTUs), i.e., specialized biomedical research units that coordinate and help clinicians to design and analyze clinical trials and other studies . The advStandardizing a surgical procedure’s key steps is difficult when different surgeons or even multiple centers are involved. Rapid changes in technology, research, or institutional organization can render protocols rapidly obsolete before the end of a study. Even with the most accurate wording, including schematic drawings or images of single clinical examples, execution differences from one surgeon to another cannot be avoided. This “technical performance bias”  can onlyUnfortunately, such documentation is rarely available. Numerous studies display poor reporting and thus are at a high risk of bias . The surSpecialties that utilize endoscopy techniques have begun to compile comprehensive intra-operative documentation by recording the full video streams of the procedures. This practice has underscored the critical role that the quality of technical performance plays in determining the success of a surgical treatment ,45. The Comprehensive documentation of surgical interventions enhances the development of supervised machine learning algorithms. Advanced artificial intelligence (AI) technologies facilitate the automated post-processing of crucial still images and concise short video clips, ensuring rapid access and precision. AI-integrated surgical platforms have played a pivotal role in select endoscopic-assisted surgeries , indicatGiven that surgical protocols are pivotal elements of a study, without a possible secondary analysis of the abovementioned intra-operative performance documented by images, it will be exceedingly difficult to know how precisely a written protocol has been followed. Confounding variables such as different centers, surgical skills, and tacit knowledge between different surgeons will also influence the technical performance. It remains surprising that there is not more pressure to ask for complete intra-operative image documentation for surgical publications . SurgeonA significant issue is the sheer volume of guidelines and algorithms available in the current landscape. Professionals, regardless of their age, find it increasingly challenging to navigate this abundance while adhering to tight schedules and striving to maintain a healthier work–life balance compared to their more senior colleagues.Certain obstacles to Evidence-Based Surgery will remain, not the least of which is that double blinding will remain difficult, as well as considering the inclusion of placebo surgery ,50. FormSurgeons would appreciate to ability to base their daily decisions on evidence. Recognizing this, the American College of Surgeons produced a collection of electronic guideline books, “Evidence-Based Decisions in Surgery” . The avaIf an emergency procedure is needed in orthopedic trauma, time can be gained by following the concept of DCO, damage control orthopedics , which pFor elective cases, an additional trend is SDM, shared decision-making. Its adoption might be slowed by concerns about the potential legal consequences of its use . NeverthSurgeons might have preferences that can even be evidence-based, but mostly, there are alternatives, which might have equivalent evidence bases. The patient must be informed accordingly to respect their autonomy. If the alternative does not coincide with the surgeon’s expertise, the doctor should propose a colleague. SDM helps in such a situation, as the patient must know from the beginning that alternatives exist, and a choice will be needed. If confronted with uncertainty, a neutral information fashion is even more important.Comparable and reproducible data retrieval is an essential component of evidence production. The frequency of data recording in the OR is increasing, but guidelines do not yet exist, as surgical data science is a new field . The purInconclusive results in RCTs comparing different techniques or different results from different trials may result from having inhomogeneous groups. Missing details in the data presentation make any comparison impossible. Even the best-written protocols for a surgical procedure bear the risk of ambiguity and interpretation problems, even when drawings are added. Retrospectively, it is difficult to judge whether the protocol has been homogeneously followed. Therefore, by adding complete data, the scientific value of RCTs is significantly increased. It can help surgery to fill the gap to internal medicine.Improved intervention descriptions in surgical trials have been recognized to be necessary for almost two decades , but astFuture studies in orthopedics should explicitly address the concerns associated with the incomplete reporting of performance measures and the inadequate handling of missing data ,64. SurgModern high-quality research is very expensive. Funding bodies are indispensable, but conflicts of interest must be avoided by clear rules to guarantee complete transparency. Outcome studies should focus on factors relevant to patients (PROMs), and the outcome data should be provided by surgeons who are not directly involved in the surgery. Functional and pain level data should be obtained during every surgeon contact and not only at the end of the study. Protocols that allow telemedical data production should be preferred. Patients might be incentivized to participate in trials by incentives and people independent from the treatment team, but the risk of biases must be carefully considered. The CONSORT perfectly addresses ethical issues.The US government will guarantee open access to all research publications it funds. The policy will go into effect in 2026 and apply to everything that receives federal money. Similar decisions have been made in other countries, for example, the Swiss National Science Foundation in Switzerland.The “globalized” world with many different healthcare systems exposes impressive differences concerning access to facilities and technologies. Future “guidelines” should also consider “global real-world data” and not only data resulting from the special situations considered in randomized controlled trial protocols. By such measures, surgical decision-making for surgeons would become more evidence-based and no longer biased by tradition “on what works in my hands and/or what my mentor told me” ,67. To improve evidence in surgery, a joint effort is needed to guarantee good organizational support to ensure that adequate personnel, resources, and funding exist for those surgeons willing to improve Evidence-Based Surgery. More efforts are needed to educate young surgeons to define relevant clinical problems and analyze whether different techniques can solve the problems associated with a given surgery. “Shared decision-making” might also help with the management of complex clinical information combined with structured patient information ,68.New publishing formats ,69,70 wiTechnical progress will soon offer relatively simple, affordable head-mounted devices for the image documentation of daily surgical practice . Up to nIt will be interesting to see whether the commercial evolution and data protection legislation will allow these tools to enter the medical arena, i.e., the operation room, and produce relevant help to surgeons globally.Complete intra-operative imaging can offer readers and reviewers an assessment of the technical performance quality. Such an approach can also be integrated into RCTs, potentially minimizing the variability within surgical treatment groups. More stringent reporting should be encouraged by researchers, authors, and especially journal editors. The data should already be accessible to the experts for peer-reviewing. The detailed data should be separated from the main text and should be accessible by a link, creating a new lean publication format. The main text should be as short as possible to avoid redundant information. This allows surgeons to gain a more comprehensive understanding when interpreting the results of clinical trials or observational studies, which can be beneficial for advancing Evidence-Based Surgery.