The intensive global plastic production, use and associated plastic pollution have caused concern for the potential risks to human health and the environment. This has led to the adoption of numerous regulatory initiatives aiming to combat plastic pollution. Despite the considerable regulatory activity in the field of plastic, it appears that there is still debate about the actual risks of plastic to humans and the environment. This raises the question of to what extent the current plastic regulation is evidence-based, a declared ambition in the European Union. Therefore, the aim of this study was to investigate to what extent key policy initiatives targeting plastic pollution are based on scientific evidence. Selection of initiatives was based on expert elicitation accounting for the opinions of persons involved in the development of the policy initiatives, and a thorough assessment of the historical development of plastic pollution regulation, with focus on their importance both with respect to regulation of plastics as well as their historical importance as drivers for societal actions on plastic pollution. We find that scientific evidence appears to be generally present in the scientific foundation for the policy initiatives analysed in this study. All the initiatives are supported by scientific articles and reports about among others plastic sources, ecological impacts of plastic production and consumption patterns. Marine litter monitoring data was found to contribute to the evidence base for 4 out of the 6 policy initiatives and thereby appears to be one of the central scientific drivers behind the societal actions on plastic pollution. Other scientific tools applied when shaping the policy initiatives include risk assessment, impact assessment and life cycle assessment. Despite the prevalent consideration and application of scientific evidence, there seems to be a broad recognition in the preparatory work of the initiatives that there is still a lot of uncertainty related to determining the harm of plastic pollution. In these cases, taking precautionary actions seems however to be justified, recalling not least the precautionary principle. As the issue of plastic pollution is complex and still subject to uncertainty, it seems important both that policy initiatives allow for flexibility and continuing adjustment to the on-going knowledge generation and that the scientific community provides the needed research to continue the science-informed policy development. The widespread use of plastics globally and the associated plastic pollution have caused concern for potential risks to human health and the environment, and received increasing attention from citizens, media, policy makers and the scientific community . This haDespite the considerable regulatory activity in the field of plastics, it appears that there is still debate about the actual risks of plastics to humans and the environment , 15. NumThe Covid-19 pandemic has revitalized the use of single-use plastic products . Still s” which sets out a list of improvement proposals to EU law-making  and to provide an evidence base for the various actions needed to be implemented by decision-makers”. It mentions that there is uncertainty present at each stage of an environmental risk assessment. Important uncertainties summarized in the report include poor understanding of how evidence of effects from localized studies could be extrapolated to larger spatial scales; that it is not possible to link evidence of the substantial numbers of individuals affected by marine plastics litter to negative effects on populations for most affected species; and relative importance of plastics as a vector for transport of chemicals from sea water to the food web.The reports from the Ellen MacArthur Foundation , 73 do nIn 2019, at the fourteenth meeting of the Conference of the Parties to the Basel Convention (COP-14), a series of plastic waste amendments were adopted. These became effective as of 1 January 2021. The Convention was revised with the objectives of enhancing the control of the transboundary movements of plastic waste and clarifying the scope of the Convention. Specifically, Annexes II, VIII and IX  were revIn Annex VIII, plastic waste was inserted clarifying that it is presumed to be hazardous whereas Annex IX clarifies the types of plastic waste that are presumed not to be hazardous. The latter includes a group of cured resins, non-halogenated and fluorinated polymers, provided the waste is destined for recycling in an environmentally sound manner and almost free from contamination and other types of waste. It also includes mixtures of plastic waste consisting of polyethylene (PE), polypropylene (PP) or polyethylene terephthalate (PET). Annex VIII is subject to the Prior Informed Consent procedure (PIC) and Annex IX is not. Finally, a new entry Y48 was inserted in Annex II which covers plastic waste, including mixtures of such wastes unless these are hazardous or presumed not to be hazardous.movements…” [The amendments implemented to the Basel Convention were originally proposed by the Government of Norway see –76). The. The76])ements…” .Norway had already suggested the removal of solid plastic waste from Annex IX to the Basel Convention under COP-11 . In the Furthermore, it states that improving waste collection and management presents the most urgent solution to reduce plastic inputs, especially in developing economies. To support this, Norway cites the EC "A European Strategy for Plastics in a Circular Economy" for about half of the plastic waste collected is sent abroad where uncertainty remains over its treatment . FinallyThe report termed UNEP/EA.3/INF/5  providesThe “Report on possible options available under the Basel Convention to further address marine plasti litter and microplastics” (UNEP/CHW/OEWG.11/INF/22)  mentione” of 2018. These three legislations incorporate plastic materials in terms of product design, recycling and targets for local waste management, while the SUPD aims to build upon these efforts and focus on prevention of marine plastic litter by market restrictions and having the producers pay for cleaning up [In June, 2019, the EU Commission presented the “Directive (EU) 2019/904 of the European Parliament and of the Council of 5 June 2019 on the reduction of the impacts of certain plastic products on the environment” also known as the Single Use Plastics Directive (SUPD). The Directive originates from a broader approach in the EU to tackle plastic pollution in the environment combining the “European Strategy for Plastics in a Circular Economy” of 2018, the “Circular Economy Action Plan” of 2015 (since then the “New Circular Economy Action Plan” has been adopted in 2020) and the revised “Waste Framework Directiveaning up . Anotheraning up . Prior taning up  was publThe IA  consultsThroughout the IA the arguments for the state of the environment and stress of plastic pollution are backed by peer-reviewed sources which represent the current research at the time . HoweverAs part of the EU’s plastics strategy, aiming at changing how plastic products are designed, produced, used and recycled, the EC assessed the scientific evidence for taking regulatory measures to address releases of intentionally added microplastics for all types of consumer and professional use products in 2017 –88. As aTo establish a scientific knowledge-base, the EU funded a substantial amount of research projects related to microplastics in the environment years prior the restriction proposal. Of January 2016, as part of the Oceans Joint Programming Initiative, four research projects were launched with a total budget of €7.7 million . Thus, tBy observing the restriction proposal process, it is evident that the EC and the ECHA have taken several steps to account for the scientific state-of-the-art. As mentioned, the proposal was enabled after scrutiny of scientific evidence highlighting that microplastics constitute a potential risk to human health and the environment. The Annex XV restriction report prepared by the ECHA writes that it is “…setting out the main evidence justifying the proposed restriction…” clearly indicating that the restriction proposal builds upon “evidence” . E.g., bAs part of the restriction proposal process, the proposal was subject to scrutiny by RAC and SEAC. Especially, the work by RAC is deeply rooted in science, as risk assessment builds upon scientific evidence for hazards and exposure. Due to the complexity of the hazards associated with microplastics, the risk assessment of RAC is based on three different approaches: the classical (eco)toxicological risk assessment establishing evidence for the predicted no-effect concentration (PNEC) and the predicted environmental concentration (PEC) with a subsequent evaluation of the risk quotient (PEC/PNEC); a non-threshold approach based on a persistent, bioaccumulative, toxic or very persistent, very bioaccumulative (PBT/vPvB) perspective. This implies that no ‘safe’ environmental concentrations can be established; and a ‘case-by-case’ assessment addressing the extreme persistence of microplastics in the environment . The traThe restriction proposal report refers to the Registration, Evaluation, Authorisation and Restriction of Chemicals (REACH) annex I which outlines how to conduct chemical risk assessments. The three approaches are also implemented in the annex XV restriction report and indicate that multiple scientific approaches have been applied . AccordiThe ECHA considers the restriction of intentionally added microplastics to be both implementable and enforceable, even though harmonised analytical methods to detect microplastics in consumer products are not yet established. Also, methods for analysis of, and criteria for (bio)degradable plastics still needs further research. The argument behind, is that existing test methods are still applicable and can be readily applied to test for presence of microplastics in mixtures , which iThe work by SEAC entails an assessment of the socio-economic impacts of the proposal, including e.g., impacts on different sectors . SEAC concluded that the restriction proposal is the most appropriate measure to handle the risks concluded by RAC, considering the socio-economic benefits and costs.The restriction proposal report of intentionally added microplastics stresses that “risk assessment of microplastics is complicated by the current uncertainties apparent in relation to hazards, fate, exposure and risks” . It addrWhen diving into the preparatory work of the six policy initiatives targeting plastic pollution analyzed in this study, it appears that scientific evidence is generally present, and to a large extent for many initiatives. As it appears from Fig. Marine litter monitoring data was found to contribute to the evidence base for 4 out of the 6 policy initiatives. This includes among others the SUPD and the European Strategy for Plastics in a Circular Economy that both refer to the technical report by JRC on Top Marine Beach Litter Items . The OSPIn the 1990’s OSPAR adopts the so-called ecosystem-based approach as a way of protecting the marine environment through the integrated assessment and management of human activities. This approach is therefore also enshrined in the OSPAR Action Plan  and inteElements of uncertainty exist within most of the scientific evidence that underpin the policy initiatives investigated in this study – which is not surprising, as there is always uncertainty in science. Going back to the earliest initiative analysed, the OSPAR Action Plan, the knowledge base includes a list of gaps when it comes to assessing ecological impacts and sources of marine litter. These knowledge gaps are outlined in one of the main documents underpinning the action plan, namely , which aSince some of the policy initiatives analysed date years back, some of the knowledge gaps present at the time of implementation may have been fully or partly addressed since then due to ongoing research development. UNEP , one of As mentioned earlier, it is evident that monitoring data has played an important role as scientific foundation for initiatives targeting plastic pollution. Monitoring data constitutes the main basis for the selection of the SUP items included in the SUPD. This approach focuses on data for plastic item abundance and does not account for the environmental harm that the different items may cause. This is also highlighted in one of the documents  underpinSome of the policy initiatives examined are based on estimations that are also associated with some uncertainties. One of these is the Directive amendment regarding lightweight plastic carrier bags, where the underpinning LCA helds uncertainties when it comes to e.g., consumer behavior, landfill conditions and method of waste combustion . SimilarPrevent inputs of and significantly reduce marine litter, including microplastics, in the marine environment…” [Are plastic pollution policies evidence-driven? As mentioned in the introduction, the EC outlines in their Communication , 30 thatonment…” . Our stuIt should be noted that following the EU’s principles for ‘Better regulation’ when it comes to use of evidence, is not necessarily a proof that a policy initiative is more efficient or effective. Listorti et al.  investigIt is a declared ambition in the EU that policy development should be evidence-driven , 30, 38.Finally, it is important to recognize that the many different types of evidence used to support policy initiatives also provide room for scientific support for policy initiatives that might be conflicting. This can have consequences for the negotiations of the forthcoming United Nations plastic treaty, and even prevent a successful process. So, in order to ensure the best scientific evidence for these negotiations it is thus important that future studies aim at generating transparency about scientific foundations for ongoing and future policy initiatives around the globe, thereby providing the best scientific foundations for these important policy decisions.


In their study of manifestations of policy support organizations (PSOs), Al Sabahi et al found that PSOs are united in their goal to support evidence-informed policy-making (EIPM), albeit with differing approaches. Their article is an important contribution to the body of research on evidence utilization and implementation. The unprecedented evidence climate presented by coronavirus disease 2019 (COVID-19) provides a unique window to motivate EIPM implementation. Research such as Al Sabahi and colleagues must prompt a dialogue regarding how best to address some of the current shortcomings in the field of EIPM. Monitoring and evaluation of best practices in EIPM is scarce. EIPM uptake is unsatisfactory, and the scientific community needs to ask itself why that is and what can be done. And, we should strive to develop a gradient that discerns between the convenient and the essential so countries can evaluate and pursue the policies to best address their greatest pain points through evidence. While they found that PSOs are all united in their goal to support evidence-informed policy-making (EIPM), the approaches to achieve this goal varied by organization. Their article contributes to the way we understand the implementation of evidence utilization. Conducting policy-relevant research, packaging the evidence in appropriate formats, encouraging an accepting climate for research use, and implementing brokering mechanisms, such as PSOs that are described in the original article, are all examples of ways to support EIPM further. While the findings of this article are extremely important, we need to sometimes take a broader perspective and ask how we can push the field of EIPM to the next level. We need to ask: what more can we do to encourage EIPM uptake? How can we improve accepted practices for the evaluation of initiatives that support EIPM? And most pressingly, how do we advance evidence-informed policies beyond where they are convenient and relatively painless to where they are crucially and urgently needed? Among their findings, the authors concluded that the most common activities that PSOs undertake to pursue policy development include supporting learning about how to make evidence-informed decisions and synthesizing research evidence. It could be that these findings are a result of the types of approaches that have been encouraged heretofore. Synthesis of research evidence and methodologies are well established. However, we lack adequate monitoring and evaluation of current EIPM practices to know and fully understand what practices work and why. Despite this, we keep pushing forward with the currently accepted practices. Since there is so much to gain from improving and expanding the reach of EIPM, it is important to reconsider the most effective techniques for monitoring and evaluation and how best to improve these processes.2 Al Sabahi and colleagues’ article is a good evaluation of PSOs, and similar, additional work is necessary for the field of evidence implementation. However, while it is essential to increase our efforts to monitor and evaluate EIPM, it is these authors’ opinion that there is another component of EIPM that needs to be addressed, namely: are we doing enough? Perhaps the time has come to expand beyond evidence briefs and dialogues and implement more innovative approaches with which to encourage EIPM. In the past two decades, there has been an explosion of interest in EIPM. It started with an awareness that patients were receiving treatments of little-known effectiveness, and the push for evidence-based medicine was born in the late 1990s. Since then, we have seen huge interest and advancement in the field of evidence-based, which later became known as evidence-informed, policy. Scientists in the field of EIPM are studying and promoting different approaches to getting evidence into action, yet the evaluations of the approaches are falling short. It is reminiscent of the adage that the ‘shoemakers’ son always goes barefoot. Academics and scientists in EIPM are doing an excellent job of promoting evidence-informed decisions. However, the methods themselves are not 100% evidence-based and not necessarily fully evaluated. While there are challenges in evaluating and monitoring methods to promote EIPM, the reality is that evaluative tools do exist. WHO EVIPnet developed a series of questions to monitor and evaluate existing programs that ask whether a given program meets current needs, whether it produces the expected results, and whether it contributes to higher-level objectives. The EIPM crisis is two-fold. For progress to be achieved, we must expand our understanding of the structures and issues surrounding the implementation of evidence utilization and simultaneously explore how to promote its uptake in the first place. Al Sabahi and colleagues’ article makes an important contribution to the understanding of brokering organizations. Yet, despite all the existing knowledge and frameworks, EIPM uptake does not correspond with its importance, cost benefit, and potential benefits it can serve to patients’ health. EIPM and its value have been well established for over thirty years. Despite this, the reality is that EIPM has not been embraced as much as it should be. The pressing question that we need to ask ourselves as a community of researchers is why? And how can we advance this work so it will be integrated better? Al Sabahi and colleagues’ article, while furthering the understanding of current practices, stops short of addressing how best to challenge the current unsatisfactory reality. The coronavirus disease 2019 (COVID-19) pandemic has afforded us a rare opportunity. We have seen science and policy developed and implemented at previously unimaginable rates. As scholars of health policy, we should examine how various countries have made use of scientists and local science while managing the pandemic. As John Lavis commented in his 2021 WHO Global Evidence-to-Policy Summit podcast, due to COVID-19 and the increased interest in evidence it has inspired, we are experiencing “a once in a generation opportunity to build or rebuild evidence support systems.” The question is, how do we leverage this unprecedented opportunity? While these authors do not have the answers, some potential suggestions could be to look at other fields of expertise and see how they have succeeded. What have been the successes in fields such as behavioural economics, psychology, marketing, and the like, in getting individuals, organizations, and systems to change their behaviours and practices? We do not need to reinvent the wheel. We need to look outside of our EIPM world and outside of our healthcare bubble to learn from excellent practices in other disciplines. We should consider including these disciplines in PSOs. One thing to keep in mind, though, is that we cannot tackle this issue in every healthcare issue and policy. We need to be strategic.3 Political and economic factors, personal values and beliefs, culture, and practical barriers all stand in the way of EIPM.5 Due to the great challenges we face when implementing EIPM, we must ask ourselves which battles are most pressing to fight: those that are easy to win or those causing the most pain? In a perfect world, all policies would be evidence-informed. However, the reality is that the barriers are endless, and to a large extent, policy-making lies beyond the control of policy-makers.6 But when it comes to EIPM, are short-term wins enough? Or do we want to strive first and foremost to create EIPM in our most difficult, pressing issues? In John Kotter’s famous theory of change, he posits that to create change, we need to continuously generate short-term wins to justify our efforts and motivate continued success. Despite the impulse to push for evidence in all areas of health policy, it is prudent to consider that not all health issues are equally urgent, and not all policies are equally essential. When encouraging the uptake of EIPM, we need to develop a gradient. While some health concerns would be nice to solve through evidence-informed policy, others are simply paramount. Family violence, mental health, and managing a pandemic (to name a few) are issues that pose significant threats to our personal health and the health of our communities. When it comes to developing and implementing policy that is informed by evidence, these are issues on which we cannot afford to compromise. But how exactly do we ensure that our EIPM expands past the “merely beneficial” to undertake these (and other) paramount issues?7 The PSOs should focus on those areas. While it would be utopian to have EIPM in all areas of policy, and while the ‘quick wins’ are great, they do not always have the biggest impact or make the greatest change. There are pre-existing techniques that can assist in this process. For example, priority setting processes is a technique in which health options and alternatives are ranked systematically and explicitly according to a set of rules to prioritize health choices amidst scarce resources.8 Another technique is programme budgeting and marginal analyses, which lets decision-makers maximize the impact of available healthcare resources that most benefit the needs of their community.9 Such tools can help evaluate and pursue areas of policy that are most pressing and address the health issues that most significantly stand to benefit from policy rooted in evidence. Each country has its own burning issues, and therefore the gradient of issues should and will vary from country to country. For EIPM to be relevant and beneficial, it is crucial for communities of experts to decide on the top 10% of issues in their context, country, and or organization. Each country has its own interest, ideas, and institutions that they need to identify and, based on those values, decide which areas of policy are the most important for EIPM to occur. By prioritizing top health priorities, each region can focus its energy and resources on implementing research where it is most relevant. Some are leading this charge, such as the Global Commission on Evidence’s initiative to produce a report containing recommendations regarding how to better disseminate evidence to decision-makers in everyday policy-making as well as in times of crisis . Over the next decade, we must turn our focus to the areas in which policy-makers should work with academics to address the most burning issues. Likelihood of uptake should be in the areas of biggest pain, both financially and regarding human life. When it comes to health policy, going forward, it is imperative that we focus our efforts on those pain points rather than casting our net too wide. Yet we also should evaluate our initiatives that are meant to support EIPM. Only then can we truly understand which efforts are worth re-investing in and utilizing. We can only hope that as we succeed in addressing the critical issues of our day with relevant, constructive policy that is informed by evidence, policy-makers will begin to understand the need to use evidence to address lesser pain points and gradually let evidence lead the way in all policies. Not applicable. Authors declare that they have no competing interests. All authors contributed in the preparation of this manuscript.


Systematic reviews/meta-analyses (SRs/MAs) are still controversial on the effectiveness of Banxia Xiexin decoction (BXD) to treat gastroesophageal reflux disease (GERD). To assess the evidence reliability and inform the clinical use of BXD, we performed a meta-analysis from previous SRs/MAs to collate, critically appraise, and synthesize the effectiveness of BXD treatment in GERD.  SRs/MAs were collected by searching major medical databases. The included studies were evaluated in terms of methodological quality and quality of evidence using criteria from the Assessment of Multiple Systematic Reviews 2 (AMSTAR-2) tool, and the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) system, respectively.  Six SRs/MAs were included in this study. The methodological quality of SRs/MAs was generally unsatisfactory. Unregistered protocols, failure to provide a list of excluded trials, and lack of a comprehensive search strategy were the main limitations of previous SRs/MAs. No high-quality evidence was found to support the effect of BXD on GERD patients. The qualitative data synthesis relied on low-quality trials with a small sample size, which was the main factor for evidence degradation.  BXD seems to have promising efficacy to treat GERD patients. Although the quality of SRs/MAs was generally low and defects were frequent, our study highlights areas where methodologies need to be improved. The gastroesophageal reflux disease (GERD) is a functional gastrointestinal disorder characterized by heartburn, chest pain, dysphagia, and abdominal pain due to gastric acid entering the esophagus . The gloThe pathogenesis of GERD has not been fully elucidated. According to available evidence, the development of GERD involves a variety of potential mechanisms, such as transient lower esophageal sphincter relaxation, lower esophageal sphincter pressure, hiatus hernia, crural diaphragmatic dysfunction, and impaired esophageal clearance . PharmacSystematic reviews (SRs)/meta-analyses (MAs) are considered the highest level of evidence in evidence-based medicine, yet inconsistent results may interfere with evidence-based decision-making ,12. BXD http://www. crd. York.ac.uk/prospero), and the registration number was CRD42022287497. The methodology was performed following the criteria of the Cochrane Handbook [The protocol of this study was registered in PROSPERO  SRs/MAs studies; (b) patients diagnosed with GERD; (c) experimental group treated with BXD and control group treated with Western medicine (WM); and (d) outcomes included effective rate, recovery rate, efficacy under gastroscope, recurrence rate, acid regurgitation, heartburn, and adverse events.The comprehensive efficacy assessment criteria were as follows : (a) curExclusion criteria were as follows: (a) repeated publications; (b) graduate dissertation; and (c) meeting abstracts.Study identification and data extraction were carried out independently by two authors. Studies were first identified by screening titles and abstracts, after which the full text was read for papers that possibly met the criteria. Items of the included studies were extracted as follows: authors, sample size, interventions, outcomes, relative effect, and main findings.Methodological and evidence quality of the enrolled studies were carried out independently by two authors using the AMSTAR-2 tool and GRADE system, respectively. The methodological quality was ranked as high, moderate, low, or critically low . EvidencA narrative synthesis was performed in this overview. The characteristics and results of each SR/MA as well as the results of AMSTAR 2 were summarized by tabulation. The GRADE evidence profile and summary of findings table were generated by using the GRADE pro GDT online software.The literature search identified 202 relevant records, of which 191 records were removed after screening titles and abstracts. The remaining 11 papers were evaluated through full-text reading. Finally, five records were excluded, and the remaining six SRs/MAs  met the Features of included SRs/MAs are outlined in All included SRs/MAs failed to register protocols and to provide a list of excluded trials. Therefore, previous SRs/MAs were graded critically low for their methodological quality. Furthermore, search strategies, funding source, and conflicts of interest statement displayed different degrees of errors .Due to limitations of the enrolled trails, the evidence strength was weakened for all outcomes. Inconsistency, imprecision, and publication bias also limited the strength of evidence for some outcomes. Four outcomes were deemed of moderate evidence quality, ten outcomes were considered of low quality, and five were of critically low quality. Details are outlined in Two SRs/MAs , 23 compThe pooled results of adverse events were reported in only one study , and theSRs/MAs are considered the highest level of evidence in evidence-based medicine . HoweverAfter a systematic review and synthesis, it was found that the gap between evidence and its implementation in clinical practice stems from the low quality and uncertain characteristics of previous evidence. There are several notable findings from this study. First, all included SRs/MAs were published in recent years (2015 to 2020), suggesting that BXD is starting to gain attention as a complementary alternative therapy for GERD. Second, based on results from the AMSTAR-2 tool and the GRADE system, the methodological and evidence qualities of the included studies were limited, which means current evidence cannot provide a reliable basis for clinical decision-making. Third, almost all SRs/MAs yielded positive results. Of note, most authors did not tend to draw firm conclusions about the effects of BXD on GERD patients due to small sample size and low quality of randomized controlled trials.The AMSTAR-2 tool highlighted several challenges of SRs/MAs that should be addressed. First, studies did not register their protocols. The pre-registration protocol helps to improve transparency, minimize potential risk of bias, reduce duplicated work, and keep the study up to date. It is advocated and recommended that authors register protocols in public databases such as PROSPERO to avoid the risk of bias. Second, authors did not account for publication bias when selecting literature for their SRs/MAs. Therefore, SRs/MAs should provide a comprehensive search strategy for all databases, as a comprehensive and precise search helps to avoid the inclusion of ineligible studies and reduces the risk of publication bias. Furthermore, a list of excluded trials and exclusion criteria improves transparency and can be presented by direct reference or as a supplementary file. Finally, funding sources should be fully reported, as results from commercially funded research might be biased toward funders. Based on the GRADE system, the risk of bias of the enrolled trials was the most common downgrading factor, indicating that the main cause of the reduced quality of evidence came from the quality of the original trials. Only well-designed and rigorously conducted trials can reduce or avoid bias. Specific methods of randomization should be clearly described to reflect whether randomization has been successfully achieved. Moreover, the allocation concealment and blinding method should be fully reported, while large sample sizes and high-quality trials are the basis for high-quality evidence sources.Numerous SRs/MAs have evaluated the use of BXD to treat GERD, but the quality of these studies is limited and their results are not fully consistent, which is not conducive to the use their evidence. Compared with traditional SRs/MAs, an overview facilitates a comprehensive evaluation of current evidence on multiple identical topics, provides more focused high-quality evidence, and identifies key flaws in evidence use. To our knowledge, this is the first overview of SRs/MAs exploring the effect of BXD on GERD patients using AMSTAR-2 and GRADE. Results herein presented on the methodological and evidence qualities of previous SRs/MAs might help to inform evidence-based decision making and guide future high-quality studies , 28.Some limitations must be acknowledged. First, SRs/MAs lacked detailed characteristics of participants and observation time points. These can hinder the analysis and interpretation of data and reduce the translation of evidence into clinical practice. Second, only eight commonly used public databases were searched in this study. There might be eligible literature in other databases, which were not identified, thus limiting the comprehensiveness of this study. Furthermore, the on standards used in each included study were not fully reported, so it was not possible to determine if they were suitable to be reported together on effectiveness. Future studies are recommended to fully report the details of the inclusion criteria to minimize any potential risk of bias.BXD seems to have a promising efficacy in the treatment of GERD patients. Although the quality of included SRs/MAs was generally low and defects were frequent, our study highlights areas where methodologies need to be improved.


Hospitals continue to face challenges in reducing incorrect antibiotic use due to social and cultural factors at the level of the health system, the care facility, the provider, and the patient. The objective of this paper is to highlight the social and cultural drivers of antimicrobial use and resistance and targeted interventions for secondary and tertiary care settings in Canada and other OECD countries. This paper is an extension of the synthesis conducted for the Public Health Agency of Canada's 2019 Spotlight Report: Preserving Antibiotics Now and Into the Future. We conducted a systematic review with a few modifications to meet rapid timelines. We conducted a search in Ovid MEDLINE and McMaster University's evidence databases for systematic reviews and then for individual Canadian studies. To cast a wider net, we searched OECD organization websites and screened reference lists from systematic reviews. We synthesized the evidence narratively and categorized the evidence into macro-, meso-, and microlevel. A total of 70 studies were (a) from OCED countries and summarized evidence of potential sociocultural antimicrobial resistance and use barriers or facilitators and/or interventions addressing these challenges; (b) systematic reviews with 50% of included studies that are situated in secondary and tertiary settings; and (c) published in Canada's two official languages, English and French. We found that hospital structures and policies may influence antibiotic utilization and variations in antimicrobial management. Microlevel factors may sway inappropriate prescribing among clinicians. The amount and type of antibiotics used may affect resistance rates. Interventions were mainly comprised of antibiotic stewardship and training that modify clinician behavior and that educate patients and carers. This evidence synthesis illustrates the various drivers of, and interventions for, antimicrobial use and resistance at the macro-, meso-, and microlevel in secondary and tertiary settings. We demonstrate that upstream drivers may lead to downstream events that influence antimicrobial resistance. Inappropriate use of antimicrobials and the lack of new classes of antibiotics have reduced our ability to manage antimicrobial or antibiotic resistance (AMR or ABR) , 2. DespAppropriate patient care within hospitals relies on the use of effective antibiotics. When exposed to an antibiotic, bacteria become subject to “selective pressure,” which means that susceptible bacteria will be eradicated but bacteria that are resistant will have a competitive advantage and will be able to replicate and survive . AlthougStaphylococcus aureus and vancomycin-resistant Enterococcus [Much of the World Health Organization (WHO)'s critical and important AMR priority areas to support research and development of antibiotics relate to nosocomial bacteria that tend to increase length of stay and mortality within hospitals . Reducinrococcus . HoweverThe social determinants of health consider the way people live, work, and grow, as well as societal cultures, and how these influence health and well-being , 14. ManOrganizing health care systems into various strata (or levels)—“macro, meso, and micro”—may clarify the complexities of the drivers of AMU/ABU and AMR/ABR. The macrolevel refers to national/regional systems and policies, the mesolevel signifies the health care organization and hospitals/hospital wards, and the microlevel represents the individuals including patients, carers, and clinicians. Such a framework may inform and design interventions, programs, and initiatives that target these drivers and appreciate the interactions that exist within and across these levels .Interventions, including antibiotic stewardship programs (ASPs), have been used at the macrolevel and mesolevel to optimize and control AMU and downstream, at the individual level, among health care workers. The role of ASPs is mainly to monitor AMU, develop prescription targets and surveillance, promote infection prevention and control, and modify behaviors, as well as education and awareness of AMR , 17. DruThe objectives were to summarize evidence from secondary and tertiary health care settings regarding sociocultural drivers of, and targeted interventions for, antimicrobial use (including antibiotic use), as relevant to Canada and other OECD countries. An accompanying paper draws from the same rapid evidence synthesis; however, it is centred on primary and community care settings. We separated studies from these two settings to elucidate the varying drivers and interventions.Between February and April 2019, we undertook a rapid evidence synthesis at the Michael G. DeGroote Cochrane Canada and GRADE Centres, commissioned by the Public Health Agency of Canada. Our aim was to identify the social and cultural drivers of, and targeted interventions for, antimicrobial use and resistance. Although the Public Health Agency of Canada's report focused on primary care settings, our search included evidence on secondary and tertiary care settings. This paper synthesizes the evidence from secondary and tertiary care settings, which includes hospital wards, emergency departments, and hospital inpatient pharmacies. The protocol for this rapid evidence synthesis is registered in PROSPERO (registration #: CRD42019129669). This paper is reported according to the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) guidelines.We focused on studies from OECD countries that summarized evidence for potential sociocultural AMR/AMU barriers or facilitators and/or interventions addressing these challenges. We included systematic reviews if >50% of included studies were situated in secondary and tertiary settings: hospital wards, emergency departments, and hospital inpatient pharmacies. We placed language restrictions to strictly Canada's two official languages, English and French, given the Canadian focus. Given our rapid timelines, we selected articles that were published in 2009 or later for systematic reviews only. We also included individual Canadian studies to summarize the evidence from the Canadian context given the overall goal to inform Canadian public health regarding AMR and to potentially address these issues in Canada [We searched for evidence in Ovid MEDLINE  for existing systematic reviews and then for individual Canadian studies  using a hierarchical approach. We also searched McMaster University's evidence databases , OECD organizations websites, and references from systematic reviews for individual Canadian studies .Reviewers independently and in duplicate screened titles and abstracts and resolved disagreements by consensus. We undertook full-text screening and data extraction concurrently and included studies that could still be excluded at the data extraction or analysis stage if deemed ineligible by two reviewers. We worked closely with the Public Health Agency of Canada and crossed referenced our search results to ensure that we captured relevant studies.Reviewers independently extracted the data using a pilot tested and revised form, and a senior author confirmed the extraction. We recorded systematic review and study characteristics, results, and main conclusions. For grey literature, we extracted information on policies, guidelines, AMR/AMU goals, research programs, and their evaluations. Reviewers created summary of findings tables to organize evidence per outcome .We used the risk of bias assessment provided by the systematic reviews, if they were available. We did not assess the risk of bias of the systematic review or the original studies using standardized tools given our rapid timelines. All reviewers were trained in using the Risk of Bias in Systematic Reviews (ROBIS), the Cochrane risk of bias tools for RCTs, the Newcastle-Ottawa Scale (NOS), the Risk of Bias in Non-randomized Studies-of Interventions (ROBINS-I), and the Grading of Recommendations Assessment, Development and Evaluation (GRADE).Reviewers used a narrative synthesis to analyze the evidence from quantitative, qualitative, and mixed methods studies. We organized studies by outcome, from which two themes emerged: we identified outcomes that relate to antimicrobial use  and antimicrobial resistance . We used a framework , developAuthors independently, and in duplicate, used data extraction forms and developed the summary of findings tables to categorize each study into our two themes: antimicrobial use and antimicrobial resistance. We identified the most appropriate level  and outcome(s) reported for each study. Disagreements were resolved through discussion. For studies that fell into multiple levels and outcomes, the results were separated accordingly. Within each level and outcome, we summarized and grouped the common results and the themes wherever appropriate.Reviewers screened through a total of 1689 records, of which 177 studies were deemed eligible for full-text screening and data extraction. Of these, 70 studies met inclusion criteria and focused on secondary and tertiary care settings including emergency departments (ED), intensive care units (ICUs), teaching hospitals, and hospital laboratories: 44 studies were categorized under the theme of antimicrobial use and 26 studies were categorized under the theme of antimicrobial resistance . The stuWe identified studies that addressed AMU/ABU and AMR/ABR drivers at the mesolevel and microlevel only. Most of the studies focused on interventions, policies, and programs, which addressed AMU at mesolevel and microlevel only and AMR at macro-, meso-, and microlevels. Given the differences in the definitions of antimicrobials and antibiotics, we used the terms that were reported by the authors to ensure consistency with their findings. This means that there will be a mix of antimicrobial resistance/use and antibiotic resistance/use when describing the evidence. However, we found that the evidence is  exclusively for antibiotic resistance/use as we did not find any substantial relevant evidence for other antimicrobials.We identified no evidence addressing macrolevel drivers for antibiotic use.The authors assessed the appropriateness of antimicrobial use in a paediatric critical care unit using a standardized assessment and found that the most common ways of inappropriately using antimicrobials were using overly broad-spectrum antibiotics, prescribing the wrong dosage, unnecessarily using antimicrobials when ineffective, and unwarranted overlap of spectrum . Among pAMU, including the use of specific antibiotic classes and the prescribing of broad-spectrum antibiotics, varied across acute care hospitals. Notably, teaching hospitals had higher utilization rates . FurtherProvider-level intrapersonal factors, such as fear of adverse outcomes and complications, fear of complaints and lawsuits, and reputational risk, influenced physician antibiotic prescription behavior . ProfessWe identified no evidence addressing macrolevel drivers for antibiotic resistance.not associated with decreased antibiotic susceptibility [The amount and timing of AMU may influence resistance rates and adverse outcomes among patients. For example, higher-use nursing homes, compared to lower-use nursing homes, had higher rates of adverse outcomes . Among ptibility .Providers generally perceived the risk of AMR to be serious and optimizing the prescription process to be beneficial. Nevertheless, individual patient risks, patient expectations, and the belief that broad-spectrum antibiotics were effective and safer were barriers to conservative antibiotic prescribing .P. aeruginosa [Carbapenem, vancomycin and quinolone use, medical devices, ICU admission, underlying diseases, patient characteristics, and length of hospital stay were identified as the most significant risk factors for the presence of carbapenem-resistant ruginosa .In Canada, the Choosing Wisely Canada Campaign, Public Health Ontario's Antibiotic Stewardship Strategy: Empiric Antibiotic Prescribing Guidelines, and the ASP campaigns implemented at the provincial level represent current national/provincial interventions/policies that aim at improving AMU. Policies and campaigns mainly included physician guidelines, guiding principles to develop guidelines, providing evidence-based tools, and patient pamphlets and videos –37. UnitASPs may improve AMU, prescribing, costs, and length of hospital stay in various hospital settings. Significant decreases in consumption of antibacterial agents and antifungal agents, significant decreases in hospital costs, and improvements in patients' length of stay were seen after implementation of an ASP . SimilarThe duration and type of ASPs may also affect resistance rates. ASPs lasting longer than six months showed reductions in resistance. Interventions including formal antibiotic reassessment, deescalation protocol, computer-assisted decision support, or antibiotic practice guidelines were beneficial in improving use  .Rapid point-of-care tests were useful for diagnosing patients with influenza which may be helpful in guiding appropriate prescribing and deterring the inappropriate use of antibiotics for viral conditions  . We idenClostridium difficile infection management policy resulted in an overall increase in treatment concordance, a reduction in the duration of AMU, and a decrease in hospital stay and mortality [Use of, and adherence to, guidelines either implemented at high level or locally created de novo improved compliance to practice recommendations such as time to administration, prescription, and duration of antibiotics –47. Légaortality .Many antibiotic stewardship programs “fell short” of interventions including effective behavioral change techniques—self-monitoring, feedback, goal-setting, and action planning . FurtherProcalcitonin-guided therapy and C-reactive protein measurement reduced in the number of antibiotic prescriptions and use in the emergency department and the ICU  –58. For Other enablers and feedback mechanisms such as clinical surveillance software that identifies patients for prospective audit and feedback rounds resulted in significant reductions in broad-spectrum AMU and cost . More brClinician education on, and the implementation of, clinical practice guidelines reduced inappropriate management and increases in appropriate use  , 64. FurClostridium difficile infection with fidaxomicin as opposed to vancomycin—may limit the promotion of antibiotic-resistant bacteria, including vancomycin-resistant enterococci [Multimodal Infection Prevention and Control interventions implemented at national or subnational levels, instead of stand-alone interventions, demonstrated effectiveness  . Others,erococci .The WHO, the European Centre for Disease Control, the International Federation of Pharma Manufacturers and Associations, and the Centre for Disease Control and Prevention all developed approaches, guidelines, infographics, and frameworks to tackle AMR. Many aim at preventing the spread of AMR by providing guidance on Infection Prevention and Control practices, educational strategies, and action plans that can be adoptable/adaptable to specific contexts , 68–70. Antibiotic stewardship objectives, implementation of guidelines, deescalation of therapy, switching from intravenous to oral treatment, therapeutic drug monitoring, the use of a list of restricted antibiotics, and bedside consultation supported improvements in adverse events and resistance rates  . MultiplClostridium difficile infections—adherence to ASP interventions in hospitals provided an overall benefit [While most of the identified systematic reviews  –77 tende benefit .Enablers, such as antimicrobial deescalation strategies, and rapid molecular testing coupled with communication interventions may be helpful in guiding appropriate use in hospitalized patients initially treated with broad-spectrum antibiotics  , 80. RapThis rapid review highlights that there are various in-hospital interventions that may address drivers of AMU/AMR. A significant number of studies focused on interventions (about 75%), while less (about 25%) focused on drivers. Recently, a shift in the understanding of what causes inappropriate use and AMR has paved a new understanding of this issue: there appears to be a wide range of sociocultural, context-specific, systems level, and historical factors that, together, contribute to this public health threat . Our finAt the macrolevel and mesolevel, hospital structures and hospital policies, such as the size of the hospital and guidelines, may influence antibiotic utilization and variations in management. Intra- and interpersonal microlevel factors impact inappropriate prescribing among clinicians. These include fear of adverse outcomes or litigation, the perceived ability to communicate a decision, physician-patient interactions, relationships with colleagues, and work dynamics.Interestingly, we did not find studies that considered macrolevel drivers for both AMR and AMU. This may be due to the variations in the health care systems and governmental policies within OECD countries that make drivers of resistance and use at this level a challenging area to study. Studies in this area would provide insights into effective or ineffective health care systems and planning, which may influence resistance and help target potential flaws in the systems.As for the hospital level (mesolevel), the amount and type of antibiotics used may affect resistance rates. Similar to AMU, misunderstandings among clinicians and patient expectations may impact antibiotic prescribing and, in turn, resistance rates.Despite the lack of identified studies looking at macrolevel drivers for AMR, we did find one intervention study at this level. Infection prevention and control interventions, implemented on a national/subnational level, were effective at managing resistance. As for AMU, we did not find studies assessing interventions at the macrolevel. However, our grey literature search provided examples of national/international policies and interventions that have been implemented to improve both AMU and AMR.Interventions that improve both AMU and resistance tended to be consistent across the mesolevel and microlevel, which were mainly comprised of implemented ASP, enablers , and training programs that modified clinician behaviors. For patients/carers, microlevel interventions, including clinician education, delayed prescriptions, and providing information, may improve AMU and decrease resistance rates.This is the first evidence synthesis that illustrates the various drivers of inappropriate use and AMR at the macro-, meso-, and microlevels and interventions that may address these drivers in secondary and tertiary settings. The evidence was reviewed rapidly but systematically when possible.To our knowledge, we provide an initial start to a health care systems framework categorizing the drivers and interventions at the macro-, meso-, and microlevels. Within a short timeframe, we employed rigorous systematic review methods to synthesize a large body of evidence with a few modifications to the review process. We conducted a comprehensive search and, in duplicate, we screened titles and abstracts and performed data abstraction. We recognized the differences between primary and secondary/tertiary care settings, as well as subsequent challenges influencing the drivers of, and interventions to address, misuse of antibiotics. With this in mind, we separated the evidence to elucidate the drivers and interventions that are specific in each setting. We worked closely with the Public Health Agency of Canada and crossed referenced our search results to ensure that we captured the relevant studies. Further, we did not pursue the same level of detail as established frameworks that assess social, cultural, and other contextual factors given our timelines to complete this rapid review.We modified a few standard systematic review processes. Firstly, we did not conduct a risk of bias assessment and did not grade the overall certainty of the evidence of included studies. Secondly, instead of conducting full-text screening and data abstraction separately, we combined these two processes to meet timelines. Implementing these changes to the standard systematic review processes allowed us to meet rapid stakeholder deadlines.While a systematic and comprehensive search strategy resulted in identifying many key publications, it is possible that some eligible material has been missed, given our modifications to the standard systematic review process. This may have put our review at higher risk of selection bias compared to a standard systematic review, leading to publication bias . One solLearned lessons from the COVID-19 pandemic, and the potential impact of COVID-19 policies on AMR, are an opportunity to better prepare for the long-term effects of AMR . CountriFuture studies should consider the structure of health care systems; macro-, meso-, and microlevels; and the interactions across and within levels when identifying the drivers of incorrect use of antibiotics and targeted interventions. We provide an initial start to a framework to help organize drivers and interventions at each level .This rapid evidence synthesis provides insight into the various drivers and interventions that may influence AMU and AMR among OECD countries. Most remarkably, we highlight drivers and interventions at various levels of the health care system  and demonstrate that one may influence the other—upstream drivers may lead to downstream events including inappropriate use and may, in turn, influence AMR. Considering these drivers and the complex relationships between health care networks and variations within secondary and tertiary settings, the implementation and design of future interventions or policies may be enhanced. Future research into drivers of AMR/AMU and the collection of data is essential to manage AMR and to build effective policies and interventions.


The usefulness of evidence-based approaches for mechanism-based chemical risk assessments was stressed, particularly the potential contribution of the rigor and transparency inherent to such approaches in building stakeholders’ trust for implementation of NAM evidence and AOPs into chemical risk assessment.The workshop titled “Application of evidence-based methods to construct mechanism-driven chemical assessment frameworks” was co-organized by the Evidence-based Toxicology Collaboration and the European Food Safety Authority (EFSA) and hosted by EFSA at its headquarters in Parma, Italy on October 2 and 3, 2019. The goal was to explore integration of systematic review with mechanistic evidence evaluation. Participants were invited to work on concrete products to advance the exploration of how evidence-based approaches can support the development and application of adverse outcome pathways (AOP) in chemical risk assessment. The workshop discussions were centered around three related themes: 1) assessing certainty in AOPs, 2) literature-based AOP development, and 3) integrating certainty in AOPs and non-animal evidence into decision frameworks. Several challenges, mostly related to methodology, were identified and largely determined the workshop recommendations. The workshop recommendations included the comparison and potential alignment of processes used to develop AOP and systematic review methodology, including the translation of vocabulary of evidence-based methods to AOP and  Systematic approaches comprise evidence synthesis methods, such as systematic reviews (SR) and evidence maps  , a construct to structure biological events resulting in adverse effects, are two methodological approaches intended to advance toxicological research and decision-making in the 21nce maps , but alsnce maps . Such apnce maps . The AOPes (NAM) .The AOP framework is a promising approach to assemble mechanistic evidence for risk assessment;evidence-based methodology, including systematic reviews, has the potential to facilitate development of AOPs; andcommunication obstacles need to be overcome to optimize collaboration of the two fields.To further foster this collaboration, a follow-up workshop, summarized here, was organized, also by the EBTC, to deepen the discussions and work on concrete products to advance the field. The workshop was held at the European Food Safety Authority (EFSA), Parma, Italy, on October 2 and 3, 2019, hosted by EFSA and the EBTC at Johns Hopkins Bloomberg School of Public Health (EBTC).At the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group satellite workshop, held on June 12, 2019, in Hamilton, Canada, it was discussed how systematic review methods and AOP concepts can be combined to develop and use NAMs for predicting the toxicity of chemical substances to humans in an evidence-based manner, i.e., transparently, objectively, reproducibly, and consistently . This woThe workshop was opened with a welcome by Bernhard Url, the executive director of EFSA. He briefly introduced EFSA, stressing that its scope of work is risk assessment and that the risk is managed by the European Commission, Member State authorities and the European Parliament. He then made clear that the foundation of transparency has been the basis for the productive collaboration between EFSA and EBTC. Transparency and impartiality, and the lack and perceived lack thereof, has been a fundamental challenge for risk assessors, e.g., in EFSA since its foundation. Debates on these topics peaked over the years around several substances, most recently on bisphenol A and glyphosate, resulting in a loss of decision maker trust in the underlying science. EFSA considers interested experts as essential for their risk assessment tasks, as long as those interests do not constitute potential conflicts of interest in a given context. Therefore, conflicts of interest are understood as situation-specific and not expert-specific.Transparency of data and how data are appraised and integrated has become a focus in EFSA, as it provides the means for EFSA to be held accountable for their work. In particular, a logical framework to make processes such as weighting of evidence (WoE) fully transparent is essential. As an example of how EFSA could further improve its processes, he mentioned that instead of considering the quality of studies that comply with Good Laboratory Practices (GLP) as high by default, a detailed assessment of studies could be conducted. Ultimately, improved processes will allow us to make evidence-based decisions and, more importantly, to demonstrate impartiality. Behaving in such a trustworthy manner is aimed at increasing social trust. In addition, EFSA intends to create trust, even if assessment outcomes conflict with political values, by engaging with civil society. Scientists should look beyond evidence, which, in doubt, will not override societal values, and consider how they can contribute to the creation of societal trust, e.g., through educational activities. To achieve this goal of trustworthiness, evidence-based decision-making is needed, optimally based on a collaborative global risk assessment network with a common understanding of scientific methodology.Katya Tsaioun, director of the EBTC at Johns Hopkins Bloomberg School of Public Health, and Rob de Vries (EBTC) introduced the participants to the workshop topic. Starting with a joint colloquium of EFSA and EBTC in 2017 From these messages, three themes were selected for the workshop and are summarized here:Assessing certainty in AOPsLiterature-based AOP developmentIntegrating certainty in AOPs and NAM evidence into decision frameworksParticipants were assigned to one of the three themes at the start of the workshop. Per theme, two theme-specific presentations were given in plenary sessions. These were intended to stimulate discussion in the theme groups in break-out sessions, for which the groups were charged with specific questions. In addition to the theme-specific presentations, two more general plenary lectures were given.The key messages from that EBTC/GRADE workshop were:Rather than following the chronological sequence of the agenda, this report is structured around the topics addressed. Therefore, the two lectures are summarized first, followed by the three themes, for each of which the theme-specific presentations and break-out session results are presented.2In the first lecture, Juleen Lam introduced the audience to evidence-based methods and mechanistic evidence including AOPs. She started by stressing that decision-making to prevent harmful exposure in public health relies on high-quality scientific evidence. Robust methods to synthesize links between environmental exposures and harmful health effects are necessary to inform public policy and prevent harm. However, environmental health decisions have historically relied on expert-based narrative reviews to summarize the collection of scientific information . NarratiSystematic review methods, defined as a review of literature focused on a specific question that uses explicit, pre-specified methods to identify, select, assess, and synthesize scientific evidence , have beThe integration of systematic review methods within the field of environmental health is relatively recent but has been under development for the past decade. Several systematic review frameworks currently exist, including those that have been applied to several case studies in the environmental health field . Despitein vitro data . These outcomes .in vitro data in environmental chemical assessments  and AOP are organizational frameworks that describe chemical or biological KE that result from perturbations from a chemical stressor , suppression of transcription factors , activation of serine/threonine kinase 2, and inhibition of peroxisomal fatty acid beta-oxidation. All these MIEs trigger an array of effects, such as enhanced transcription of genes encoding mediators of cholesterol and lipid metabolism, including carbohydrate response element binding protein, sterol response element binding protein 1c, fatty acid synthase and stearoyl-coenzyme A desaturase 1. As a result, 3.3Liver fibrosis is a reversible wound healing response to either acute or chronic cellular injury that reflects a balance between liver repair and scar formation. A central event in liver fibrosis is the activation of hepatic stellate cells, which occurs in 2 phases, namely the initiation stage and the perpetuation stage. In the initiation phase, quiescent hepatic stellate cells become responsive to growth factors. This may be triggered by a variety of signals, including reactive oxygen species and apoptotic bodies originating from dying hepatocytes. In the perpetuation phase, the primed hepatic stellate cells undergo several changes related to proliferation, contractility, fibrogenesis, chemotaxis, extracellular matrix degradation, and retinoid loss, whereby they adopt a myofibroblast-like phenotype. Hepatic stellate cell activation may be counteracted in a resolution phase through apoptosis, senescence, or reversion to the quiescent phenotype. The most progressive form of fibrosis is cirrhosis, which, unlike fibrosis, is considered an irreversible event. At the time of the workshop, the AOP-KB contained one AOP on liver fibrosis in which protein alkylation is considered as the MIE. Different steps at the cellular and tissue level have been defined, including hepatocyte injury and cell death, activation of Kupffer cells, expression of transforming growth factor beta 1, activation of hepatic stellate cells, oxidative stress and chronic inflammation, collagen accumulation, and changes in hepatic extracellular matrix composition.4The main objective of Theme 1, led by Rebecca Morgan and Paul Whaley, was to discuss how certainty in AOPs should be assessed, i.e., how AOPs of higher certainty ought to be distinguished from AOPs of lower certainty. Two angles on this were taken: discussion of the use of the modified Bradford Hill considerations, also known as tailored Bradford Hill criteria, by AOP practitioners in assessing certainty in a putative AOP ; and expThe modified Bradford Hill considerations were chosen for discussion because the GRADE approach for assessing certainty in the evidence is an explicit adaptation of the original Bradford Hill considerations . Given pThe use of logic models in systematic reviews was chosen for discussion to determine what lessons there might be in the assessment of complex interventions for assessing certainty in AOPs. Since AOPs are simplifications of complex causal networks, but in a biological setting, it was considered that logic models for complex interventions may be informative in applying systematic methods from health research to the AOP context.4.1Bette Meek introduced the use of modified Bradford Hill considerations in AOP and MoA analysis. Descriptions of MoAs and AOPs facilitate systematic integration and assessment of mechanistic data in hazard assessment from a broad range of sources. Formalized description and analysis of the extent of supporting evidence for these pathway descriptions supports their use for various applications in testing and assessment.Selected Bradford Hill considerations form the basis for assessment of the extent of supporting evidence in formalized descriptions of AOPs. These considerations, modified somewhat from their initial characterization to assess causality in epidemiological studies and adopted in international frameworks in MoA analysis include biological plausibility, essentiality, and empirical support.in vitro or in silico. For AOPs in the AOP-wiki, the extent of confidence for these considerations in the KE, KER, and the AOP overall are also illustrated in a user-friendly summary graphical interface to guide appropriate application.The considerations, defined to address aspects critical in regulatory acceptance, are also rank ordered to reflect their relative importance in assessing the extent of supporting mechanistic data. Examples of the types of datasets associated with high, moderate, and low confidence accompany each of the modified considerations in program guidance . These dThe modified Bradford Hill considerations are “expert informed” and application driven, reflecting experience in regulatory application of MoA analysis and evolution in the OECD AOP development program through collaborative interface of the research and regulatory communities. They draw on accumulated experience to contribute to the delineation of the content of reporting templates for individual KE, KER, and the AOP overall.In addition to assessing confidence for each of the considerations as high, moderate, or low, developers provide text rationales to justify their designations based on the provided examples for components and the overall AOP. The OECD Extended Advisory Group on Molecular Screening and Toxicogenomics reviews WoE descriptions internally for compliance with program guidance. Descriptions are reviewed subsequently by a team of external scientific subject experts and approved by the OECD Working Group of the National Coordinators of the Test Guidelines Programme and the OECD Working Party on Hazard Assessment.Specific definition of the modified Bradford Hill considerations and provision of examples of the nature of supporting datasets giving rise to different levels of confidence and associated templates for description of the relevant features of the supporting datasets contributes to the consistency and transparency of AOP descriptions. Additionally, they increase common understanding among the research and regulatory communities of the elements and types of data or studies, contributing to confidence in AOPs/MoA for regulatory application.For hazard characterization, the focus in WoE assessment for AOPs/MoA on expected patterns across different levels of biological organization supported by different types of data from a range of sources addresses the synthesis or integration step of data assimilation and assessment. As such, it is one of a limited range of methods available to address integration across lines of evidence, arguably the stage most influential for application.The approach follows a number of previous requisite steps in considering data as summarized in Since the modified Bradford Hill considerations address critical considerations in interpretation of mechanistic data for regulatory application, WoE analysis for MoA/AOPs informs focus for efficient investment of resources to ensure transparency and defensibility in critical areas. This could include, for example, systematic reviews for specific questions, taking into account the specific nature of AOPs and MoA, which address how chemicals induce effects across levels of biological organization rather than associating cause and effect at one level only .As hypothesis-based integrating constructs based on prior biological knowledge, AOP/MoA development requires specialized, iterative literature search strategies. Early consideration in problem formulation of WoE considerations for integrating mechanistic constructs such as AOP/MoA is helpful, then, in identifying relevant patterns across studies and lines of evidence based on considerations relevant to regulatory priorities. It also facilitates the early integration of hazard and mechanistic data, considering patterns of relevant determinants across different levels of biological organization such as empirical support.4.2A priori, which are developed as close as possible to the onset of the systematic review, and which are not changed during the reviewStaged, which specify points at which new information from the review is expected to result in changes to the modelIterative, which can be modified at any time during the review, accounting, for example, for new learnings or societal changesThese types primarily differ in how they balance transparency and flexibility.Elizabeth Kristjanson introduced the application of logic models in systematic reviews. Logic models, also referred to as conceptual models, analytical models, concept maps, and theory of change, are usually graphical explanations of intervention and the mechanisms through which it has impacts. They have been applied in the context of systematic reviews since the early 2000s, where they are developed in the early stages of protocol development. They offer a solution to explicitly capture the steadily increasing complexity of systematic reviews , especia4.3There were three principal lessons from the break-out group discussions, which are discussed here in detail.4.3.1While the structured thinking being applied in AOPs is of high potential value for developing methods for assessing indirectness of evidence in systematic reviews, the differences between regulatory and research contexts should not be underestimated when adapting AOP approaches to the systematic review context.The development of AOPs is driven by the need to formalize biological knowledge in such a way that it can inform regulatory decisions. The fact that AOP development is an initiative of the OECD is a demonstration of its intent for regulatory use . It is iThe intent of AOPs is to organize elements of toxicity-related biology that predict apical outcomes to a degree that assays based on those elements may provide evidence of whether a chemical presents unacceptable health risks of a sufficient level of certainty for use in regulatory contexts. The function of the modified Bradford Hill considerations is to facilitate the assessment of whether a biological event, or a relationship between two events, is sufficiently certain that it can be described as a KER, and as such can be used in regulatory decision-making. The considerations also derive from past regulatory experience in MoA analysis. The context for application of the modified Bradford Hill considerations in developing an AOP is therefore quite different to assessing the certainty of a biological mechanism in a systematic review. That said, relating biological knowledge to an AOP can be viewed as a process amenable to systematic methods, on the condition that it is understood as a move from observed associations  to identifying those relationships in which we have enough certainty that they can be used to reliably predict health outcomes. The higher the certainty in a relationship between a biological event and a final outcome, the less the evidence would need to be rated down when using that event as an indirect endpoint in a systematic review. Whether the event is sufficiently predictive for regulatory use can be treated as a separate question with its own supplementary certainty criteria or by setting the bar for certainty in the relationship at the level that decision-makers determine to be appropriate.4.3.2Logic models are conceptual constructs that model complex causal relationships between risk factors and health outcomes . They arAs discussed at a previous workshop , GRADE mThe group concluded that the principle of applying GRADE to biological networks is, in theory, relatively straightforward: each event-event relationship in the network can be treated as if it were an exposure-outcome relationship; evidence relevant to assessing the causality of the relationship can therefore be systematically reviewed and assessed for certainty. If this is done for each event-event relationship in a biological network, then an understanding of the certainty of the overall network will emerge see . CertainThe group also concluded that one of the major challenges in applying systematic review methods to the context of AOP development is the volume of evidence being assessed. AOPs are also integrating constructs, where patterns across different levels of biological organization are highly influential in determining certainty in the biological network. Aspects of certainty for the elements are considered in the context of their contribution to an integrating construct across different levels of biological organization – hence, the modified Bradford Hill considerations. Handling such a large volume of evidence, consisting of multiple interdependent elements, in a systematic manner is a considerable challenge and may require rapid review methods or that evidence assessment be significantly supported by automated approaches.4.3.3vice versa) very challenging  and introduce additional concepts that may need to be accommodated in the GRADE approach. That SR methodologists and developers of AOPs come from different communities with, at this stage, only limited shared experience, makes the task of shaping SR methods for AOP development  populations  that offer flexibility under the rubric of current AOP development guidelines. These advantages include leveraging pathway-based systematic methods that add transparency, traceability, ease in updating, and reduced bias in selecting data and/or information mapped to AOP concepts.Therefore, the initial goal of this case study was to scope out the feasibility of a NAM that incorporates aspects of systematic methods (namely evidence mapping) guided by AOP concepts to prioritize and screen chemicals based on potential thyroid hormone pathway disruption in humans. Given that expert knowledge has identified several possible sites of chemical-molecular interaction as well as neurological and developmental outcomes related to thyroid pathway disruption , a pilotAcknowledging the desire to balance the advantages that could be gained using systematic methods with the downside of natural language variation, the pilot was retooled to leverage the AOP framework . CurrentRealizing that systematic review methodology is conducted in a step-wise structured manner that is amenable to technological innovation, strategies to address several advances and lingering challenges faced during application of computationally intelligent approaches are being explored.5.3A challenge in applying systematic review methods in AOP development is considering the depth of coverage. A single AOP is a broad topic that may include multiple hypotheses that could each be an independent SR. While SRs are well-suited to hypothesis-driven and focused questions that are confirmatory, it was recognized that SR could quickly become unwieldly in AOP development. Systematic evidence maps rely on the same principles as SR but are better suited to exploration . It is aFor example, evidence maps provide the same level of consistency, transparency, and documentation as an SR but present summaries of the evidence. Such summaries are useful for refining an AOP developer’s approach based on the available evidence and goals of AOP development. An additional challenge is developing the search strategy for finding data that will inform the evidence map, understanding that the broader the search, the more effort is required to screen the retrieved results. In the context of AOP, the initial literature search strategy may consider the intended application of final AOP, resources as well as time, and tailor the screening criteria accordingly. For example, an AOP developer may start with a broad search strategy and then refine the search results based on the available information or identify additional strategies needed to cover gaps in the evidence map. Search results may be refined in terms of taxonomy, MIE, KE, AO, test method, etc.To bound the search space, AOPs are defined as connecting a single MIE to a single adverse outcome (AO). In cases where the search is initiated from the MIE or AO, a decision must be made as the search is expanded regarding which MIE or AO will serve as the second anchor. For selecting an AO, the prevalence and severity of the disease are important factors. For selecting an MIE, the number of chemicals and relative toxicity of chemicals in the class would be factors for consideration. In cases where the starting point is an intermediate KE, a choice can be made to create a partial AOP with that KE as an anchor or to identify both an MIE and an AO as the search is expanded. All of these decisions should be driven by the planned use of the resulting AOP. If multiple interacting AOPs are desired, the overlapping KEs from the first defined AOP can be used as the initial search for the second and so on.Given the high correspondence between evidence mapping and the current processes used to define the overall structure of an AOP, the AOP development workflow shown in Once the KEs and their connectivity within an AOP have been defined, the next step in the AOP development process is evidence evaluation. Currently, three main elements are considered when evaluating the evidence in support of an AOP . The firSR approaches can improve the transparency, consistency, and accuracy of AOP development and include approaches for WoE evaluation and overall assessment of the AOP . The AOPWhile challenges remain with respect to applying systematic methodologies to AOP development, this goal is achievable and should be pursued. The AOP framework provides a structured way of representing mechanistic data that makes it straightforward for decision makers and stakeholders to review the evidence in support of a public health decision. The use of systematic methods to populate this framework will result in accuracy, consistency, and transparency in the collection and interpretation of the data underlying each AOP. In addition, advances in artificial intelligence (AI) such as natural language processing and machine learning may add efficiency to the process . These A6Theme 3, led by Ingrid Druwe and Sebastian Hoffmann, addressed the topic of how AOPs can inform the development of NAM, especially non-animal experimental studies. For this workshop, the topic was broadened to discuss how certainty in AOPs (see Theme 1) and in the evidence derived from NAMs used to inform an AOP interact and how this process can be systematically considered in frameworks used to derive chemical risk assessment conclusions and decisions.The first theme-specific presentation aimed at demonstrating commonalities and differences between chemical risk assessment frameworks to illustrate the extent of applicability of the topic to various contexts. It stressed that NAMs will increasingly provide more of the evidence to inform future toxicology paradigms, regardless of the framework. The second presentation introduced the GRADE approach for synthesizing evidence and developing recommendations in order to explore potential similarities with chemical risk assessment frameworks.6.1in vivo), and NAM, e.g., in vitro and computational approaches. While in most cases animal evidence is used to inform assessments, ethical and, more recently, political considerations have started to induce a shift towards NAM evidence.Ingrid L. Druwe and Sebastian Hoffmann presented commonalities and differences of chemical assessment frameworks. Various chemical assessment frameworks exist, which aid regulators and risk assessors to fulfil their mandates, particularly to support decision-making related to human health hazard identification and risk assessment. These frameworks serve multiple and related purposes such as: hazard assessment, classification, and labeling, risk assessment, prioritization, restriction, authorization, and support of other legal actions. They use relevant information of diverse origins, such as epidemiological, animal , the choice of framework(s) is determined based on the purpose of the assessment and the amount of information available to inform a chemical assessment. For example, Peer Provisional Reference Toxicity Value (PPRTV) assessments have limited data and are used to aid the Office of Land and Emergency Management (OLEM), while Integrated Risk Information System (IRIS) toxicological assessments, which identify and characterize health hazards of environmental chemicals, are data-rich and are used by a variety of stakeholders. PPRTVs integrate human-relevant information and use expert-driven read-across methodologies to fill toxicokinetic and toxicodynamic data gaps to derive a health risk value for a chemical of interest. In PPRTVs, a chemical framework such as the AOP framework may be used to anchor information from a given assay to a KE that leads to an AO of interest to regulators. In contrast, for IRIS toxicological assessments, a wealth of evidence from various origins is available, which is assessed and integrated to identify human health hazards and derive risk values. In the drafting of IRIS assessments, AOP frameworks are useful in organizing and visualizing the WoE in support of a given AO. The OECD has structured regulatory conclusion processes using diverse types of evidence by IATA. The generic IATA framework comprises three parts: 1) problem formulation and gathering of existing information, 2) WoE assessment and, if 2) is inconclusive, 3) iterative generation of additional information and WoE assessment, which is repeated until the information is adequate to draw a conclusion , as showIn summary, there are numerous commonalities of chemical frameworks, including structural problem formulation, and collecting and assessing of evidence from diverse data streams. As more mechanistic and NAM evidence becomes available, these chemical frameworks will continue to evolve to make best use of this evidence in decision-making. Differences in the use of chemical frameworks include methodological uses, e.g., uncertainty assessments, evidence integration, and legal requirements, as well as retrospective versus prospective elements.6.2Elena Parmelli and Andrew Rooney’s presentation was focused on the relation of the EtD framework to chemical assessment frameworks to explore potential similarities.Andrew Rooney explained how certainty is assessed in evidence evaluations of the National Institute of Environmental Health Sciences (NIEHS) Division of the National Toxicology Program (DNTP) Integrative Health Assessments Branch (IHAB), which serves as an environmental health resource for public and regulatory agencies . OHAT coBodies of evidence are then assessed for factors that can potentially increase confidence, such as effect size and dose-response, and factors that can potentially decrease confidence, such as risk-of-bias and indirectness. The final confidence is then determined by applying these assessment results to the initial confidence. Ultimately, the evidence, primarily human and animal studies, is integrated using a predefined approach based on the final confidence levels to obtain a conclusion, which is associated with an indicator .3 , which was co-funded by the European Commission  as a means to increase confidence in KER and b) as a way to provide points of departure (POD) useful in regulatory contexts. It was concluded that a glossary would be helpful to build a common understanding, e.g., by expanding the glossary of Another fundamental need to successfully implement NAM evidence and AOP in chemical risk assessment is stakeholder trust. This trust can only be built over time. The methodological rigor and transparency inherent to evidence-based and systematic approaches can contribute to trust-building and can help to objectively assess current approaches, e.g., in terms of reproducibility, as a point of reference. In this context, the potential of AOP, NAM, and IATA to provide mechanistic information required for population-specific assessments, especially when addressing vulnerable populations, which is hardly possible with current approaches, was stressed .It was further discussed that the gradual change to NAM and AOP-based chemical risk assessment would also require a fundamental change in chemical risk assessment methodology with its current focus on POD. If the concept of a POD were maintained, it would be a major challenge to obtain POD from integrating evidence from multiple NAM sources. While a NAM-based risk assessment framework with a POD appears to be applicable to some human health endpoints , this maThe discussion addressed how this could be done. Regarding the construction of an AOP, the importance of reflecting the dose- and time-dependence was stressed, which is a pivotal determinant of the progression of perturbations along the pathway. NAM data on multiple chemicals that allow observations of common responses can be helpful for the identification of early AOP events, particularly the MIEs. The later AOP KEs, which occur at higher levels of organization, can be informed by (histo-) pathological findings in animals and humans observed across multiple chemicals .in vitro data) or extrapolations from the KERs . The dose and time relationships should be examined for concordance. A low dose may produce effects on a MIE, which may or may not be followed by responses at higher levels. The progression in the AOP, with dose and time, can be confirmed by integrating the observed dose-response within the available information on the KERs. NAM may allow for direct measurements at the cellular and tissue levels and estimations on the likelihood of adverse effects for each point in the NAM dose-response relationships. Consequently, NAMs can provide actual evidence on effects up to cell/tissue level, including quantitative information on the KERs, identifying under which exposure conditions these KE responses are expected. At the levels of more complex biological organization, in particular the KERs connecting NAM-based observed effects with adverse endpoints, assumptions and extrapolations based on the available knowledge in a WoE approach are required. As part of the risk assessment process, the available evidence and the uncertainty related to those assumptions and extrapolations should be clearly described. This is indicated by the “?” in When an AOP is used for risk assessment, the information can be integrated based on actual measurements  and of varying quality. It needs to be kept in mind, however, that SR address narrow questions, while AOP are broad in scope.Evidence-based approaches can potentially be useful in the context of AOP. Backing up biological plausibility with systematically reviewed evidence has the potential to increase confidence. In particular, KERs could be addressed. As addressing the biological complexity of an AOP would require a number of targeted reviews, which appears to be impractical, (semi-)automated evidence maps are the more promising evidence-based tool. Automation would also facilitate updating AOPs with the latest evidence.An efficient discussion on the use of evidence-based methods in constructing and assessing mechanism-focused chemical assessment approaches requires mutual understanding of the terminology used. A controlled vocabulary, e.g., defined via an agreed glossary, would be extremely beneficial in this regard. In addition, controlled vocabularies that can be mapped onto biological ontologies would make information much more accessible, enabling the use of automated retrieval and assessment.The usefulness of the EtD framework in the context of chemical risk assessment and management should be explored in more detail.Kris Thayer focused in her resumé on how certainty can be assessed in evidence frameworks and how this relates to the (modified) Bradford-Hill considerations. Applying the concept from the GRADE Working Group approach to assessing the confidence in a body of evidence, initial confidence in evidence relevant for chemical risk assessment can basically be related to the experimental study design. Determination of the initial level of certainty is currently receiving considerable attention, particularly for the various types of NAM data. The level of certainty is then increased or decreased based on the systematic evaluation of eight (or more) factors. Factors potentially decreasing certainty are, e.g., risk-of-bias, indirectness, inconsistency, and publication bias. Factors potentially increasing certainty are, e.g., large effects and dose-response. This approach is very similar to the Bradford-Hill considerations, as demonstrated by Thomas Hartung and Kris Thayer briefly summarized the workshop from their individual perspectives. Thomas Hartung identified four main take-home messages:Relating biological knowledge to an AOP can be viewed as a process amenable to SR, moving from observed associations between biological events to identifying those event-event relationships for which we have enough certainty to incorporate them into human health risk assessments.Application of systematic review methods to AOP development is challenging due to different research contexts, vocabulary, and reasoning processes employed in the two approaches; considerable work is required to interpret and translate processes and vocabulary if the lessons from one are to be fully applied to the other.The rigor and transparency inherent to evidence-based and systematic approaches can contribute to building stakeholder trust for implementation of NAM evidence and AOP into chemical risk assessment.a priori.The iterative nature of AOP development requires a combination of evidence maps to define the KE and overall structure of the AOP and SR to perform the evidence evaluation for each KER. Methods for allowing semantic mapping and search expansion during the literature inventory or evidence mapping step(s) are essential because the full breadth of the literature needed is rarely known The time required for a full evidence map and SR covering all KERs within an AOP is prohibitive. The use of text-mining tools to facilitate the reviews is highly recommended. AOP developers should focus their efforts on those KERs that are most relevant for the current purpose. AOP documentation in a public knowledgebase allows the evaluation of additional KERs in a resource-dependent manner.Defining a PECO statement for AOP development is problematic, since a single AOP would include multiple SRs covering the individual KERs. Therefore, up-front problem formulation  is of utmost importance in developing and establishing an approach (including systematic methodologies) for interrogating any AOP pathway or its components.NAM and AOP-based chemical risk assessment will require a change in chemical risk assessment methodology, shifting the focus from AOs assessed via animal experiments to AOP-structured information on pathway perturbations.Harmonized and transparent assessment and measurement of the confidence in the AOP and the NAM populating it was identified as a challenge and a prerequisite for the integration of the evidence as a function of the various confidence-driving aspects.The potential application of the EtD framework to chemical risk assessment should be explored in more detail.In conclusion, several recommendations emerged from the workshop: