MUR) supplementation in broilers under Eimeria and/or Clostridium perfringens challenge. For this, 2 experiments were conducted. Experiment 1. A total of 256 one-day old male Cobb 500 chicks were placed in battery cages in a completely randomized design, with 5 treatment groups, 7 replicate cages per treatment and 8 birds per cage. The treatments were: nonchallenged control (NC), challenged control (CC), CC + MUR at 25,000 or 35,000 LSU(F)/kg, and CC + Enramycin at 10 ppm (positive control—PC). Challenge consisted of 15× the recommended dose of coccidiosis vaccine at placement, and Clostridium perfringens (108 CFU/bird) inoculation at 10, 11, and 12 d. Macro and microscopic evaluation, immunohistochemistry, and gene expression were evaluated at 7, 14, 21, and 28 d of age. Experiment 2. A total of 1,120 one-day old male Cobb 500 chicks were placed in floor pens with fresh litter in a completely randomized design, with 4 treatment groups, 8 replicate pens per treatment, and 35 birds per pen. The treatments were: Control, supplementation of MUR at 25,000 or 45,000 LSU(F)/kg, and a positive control . At 10, 11, and 12 d of the experiment all the birds were inoculated by oral gavage with a fresh broth culture of a field isolate Clostridium perfringens (0.5 mL containing 106 CFU/bird). It was observed that in Experiment 1 MUR supplementation reduced the infiltration of macrophages and CD8+ lymphocytes in the liver and ileum of infected birds, downregulated IL-8 and upregulated IL-10 expression. In Experiment 2, MUR linearly improved the growth performance of the birds, increased breast meat yield, and improved absorption capacity. MUR supplementation elicited an anti-inflammatory response in birds undergoing a NE challenge model that may explain the improved growth performance of supplemented birds.The objective of the present studies was to evaluate muramidase ( MUR) has shown to have positive impact on the growth performance of broiler chickens (PGN) that are present in the gastrointestinal tract (GIT) of animals, as part of the normal bacterial cell turnover , and coccidiosis infection has been considered one of the important predisposing factors of NE. It has been reported that birds supplemented with MUR under an Eimeria challenge showed higher plasmatic carotenoids, as indicator of good gastrointestinal functionality, compared to nonsupplemented animals and similar to Enramycin , challenged control (CC), CC + supplementation of MUR at 25,000 LSU(F)/kg, CC + supplementation of MUR at 35,000 LSU(F)/kg, and CC + Enramycin at 10 ppm (positive control—PC). The MUR used in the present study is same used by A total of 256 one-day old male Cobb 500 chicks were used in this trial. The birds were placed in battery cages in a completely randomized design. The experiment consisted of 5 treatment groups with 7 replicate cages per treatment and 8 birds per cage. Each cage contained sterilized litter, nipple drinkers, and automatic temperature control. The treatments were: nonchallenged control  and grower (14–28 d). All the diets contained phytase (RONOZYME HiPhos GT) at 1,000 FYT/kg of feed and 40 ppm of CAROPHLL yellow 10%. Anticoccidials were not included in the basal diets , and at 10, 11, and 12 d of the experiment challenged birds were inoculated by oral gavage with a fresh broth culture of a field isolate Clostridium perfringens  methodology  and alcian blue. For histopathological analysis, a total of 20 villi per bird were evaluated in 10× magnification  under optical microscope . The ISI microscopic methodology (patent: INPI BR 1020150036019) was used to measure histologic alterations on the intestine  are assigned to a predefined parameters observed through optical microscopy in the samples of ileum and liver. The intestinal assessment involved the scoring of lamina propria and epithelial thickness, enterocytes proliferation, inflammatory cell infiltration in the epithelium and in the lamina propria, congestion, and the presence of For the immunohistochemical analysis, ileum and liver samples collected in each age were fixed into zinc solution for at least 72 h. All the samples were dehydrated, infiltrated, and embedded into paraffin following common histological routine. The slides were then incubated with 500 µL of primary specific antibody for macrophages, CD4+, or CD8+ T Lymphocytes, followed by an incubation with HRP conjugated antibody specific for the primary antibody for 30 to 60 min. The peroxidase activity was blocked by using DAB kit for immunocytochemistry . The slides were then counterstained with hematoxylin solution. The labeled cells were counted under an optical microscope (400× magnification). Five fields per samples were evaluated.Ileum and liver samples collected on d 7, 14, 21, and 28 were stored in 1 mL tubes containing RNA later. Total RNA was isolated using Trizol reagent  following manufacture's recommendations. Turbo-DNAse kit  was sued to degrade any DNA present in the samples. RNA concentration was determined with NanoDrop Spectrophotometer  and the integrity by Experion Automated Electrophoresis System . One μg of mRNA was used to produce cDNA in a 20 µL reaction volume using iScript Reverse Transcription Supermix kit .The final qPCR reaction was performed using the iTAq Universal SYBR Green Supermix . The cycle conditions for all primers used were as follow: initial denaturation step of 60 s at 95°C, followed by 40 cycles of annealing and extension of 30 s at 60°C. The melting profile of each sample was analyzed after every qPCR to confirm the product specificity. The results were analyzed the delta-delta equation and using an endogenous control GAPDH; .Table 2FP ≤ 0.05. Parametric data were analyzed by 1-way ANOVA and Tukey's test was used for pairwise comparison between groups (P < 0.05). Nonparametric data (ISI scores) were submitted to Wilcoxon’ test and the means separated by Dunn's test. The gene expression analysis was performed by t-test with Welch's correction at 10% probability.All the analyzed parameters were considered significantly different at The experiment was conducted at the research center Mercolab, Cascavel, Parana, Brazil. All the procedures followed the guidelines and were approved by the internal Animal Care Committee, and the activities followed by a certified veterinarian.ad libitum access to water and feed during the entire experimental period that lasted 42 d. The MUR used in the present study is same used by A total of 1,120 one-day old male Cobb 500 chicks were used in this trial. The birds were placed in floor pens with fresh litter in a completely randomized design. The experiment consisted of 4 treatment groups with 8 replicate pens per treatment and 35 birds per pen. The treatments were: Control , basal diet plus supplementation of MUR at 25,000 LSU(F)/kg, basal diet plus supplementation of MUR at 45,000 LSU(F)/kg, and a positive control . The birds had The feed in mash form was produced at the feed mill located on Mercolab's research farm. The diets were formulated following commercial recommendations usually used in Brazil, based on corn and soybean meal to meet or exceed the requirements of the birds, according to each phase: Starter (0–21 d), grower (21–35 d), and finisher (35–42 d), with phytase (RONOZYME HiPhos GT) at 1,000 FYT/kg of feed. CAROPHLL yellow 10% at 40 ppm and monensin during starter and Salinomycin during the grower phase were also included in the basal feed as anticoccidials .Clostridium perfringens (0.5 mL containing 106 CFU/bird).At 10, 11, and 12 d of the experiment all the birds were inoculated by oral gavage with a fresh broth culture of a field isolate PEI) was calculated on d 42. The PEI was calculated with the following formula: PEI = )/(42*FCR at d 42)*100).The birds and feed from each were weighed on d 21, 35 and 42 to calculate BWG, FI, and FCR. Mortality and weight of the dead birds were also recorded. At 42 d, the FCR was also corrected by a common BW (2.7 kg) to obtain cFCR. At 42 d, the birds from each pen were individually weighed to evaluate flock uniformity within each pen. The production efficiency index  and breast weight used to calculate carcass and breast yield, respectively.At 21, 35, and 42 d, 2 birds per replicate were used to collect blood to measure total carotenoid concentration by using the iCheck CAROTENE device and using iEx CAROTENE reagent vials . The total carotenoid concentration is used as an indicator of intestinal integrity and absorption capacity .P < 0.05), and the means separated by Tukey's test. Additionally, in Experiment 2, the doses of MUR supplementation were submitted to polynomial regression analysis (linear and quadratic) to estimate the dose response by using JMP 16.0.Data were checked for normality and presence of outliers (2 ± SD). Growth performance data, carcass yield, and total carotenoid concentration were analyzed by ANOVA (P = 0.004). Feed intake was also affected by the treatment groups (P = 0.009) wherein the challenged birds supplemented with Enramycin showed lower FI than the nonchallenged control birds. Lastly, it was observed that the FCR was drastically impacted by the challenge (P = 0.02), but the supplementation of MUR at the 2 doses tested, and Enramycin partially counteracted this effect, wherein MUR improved the FCR by an average of 4.5 points compared to the challenged control birds.The growth performance results from 1 to 28 d are shown in P = 0.03) and internal (P = 0.009) scores, but the supplementation of MUR at 25,000 LSU(F)/kg of feed significantly reduced the respiratory score, and overall, MUR and Enramycin supplementation promoted an intermediary ISI Total Score (P = 0.03), although not different from the challenge control. At 14 d, the birds in the nonchallenged control group and the group supplemented with MUR at 35,000 LSU(F)/kg of feed presented a lower general intestinal score (P = 0.003), while the birds supplemented with Enramycin presented a higher score when compared to the nonchallenged group. Although not significant, MUR supplementation numerically reduced the total score (P = 0.02), while the group supplemented with Enramycin presented the worst score (P = 0.02). Lastly, at 28 d of age, the supplementation of MUR and Enramycin only numerically attenuated the general score of the birds (P = 0.03), the intestinal score (P = 0.04) and the total score (P = 0.05) without significant difference from the challenged control birds.The results for the ISI macroscopic evaluation at different ages are presented in st score . At 21 dP < 0.001). This effect, added by nonsignificant reductions in other liver parameters, has reduced the ISI Total Score of challenged birds supplemented MUR at 35,000 LSU(F)/kg of feed to the same level of the nonchallenged animals (P < 0.001). At 14 d of age, supplementation of MUR at 25,000 LSU(F)/kg of feed reduced the score of congestion compared to the nonchallenged control group (P < 0.001), Enramycin reduced the proliferation of bile ducts (P = 0.002), and 25,000 LSU(F)/kg of feed of MUR and Enramycin numerically reduced the ISI Total Score . At 21 d, MUR at both doses and enramycin supplementation reduced the cell vacuolation (P < 0.001) and the ISI Total Scores (P < 0.001) compared to the challenged control group. At 28 d, birds supplemented with 35,000 LSU(F)/kg of feed of MUR and Enramycin showed reduced congestion score (P < 0.001), and numerical reduction in the ISI Total Score (P = 0.008) when compared to the challenged control birds.The liver histopathology results are shown in P < 0.001). Moreover, animals supplemented with 35,000 LSU(F)/kg of feed of MUR exhibited numerical decrease in the score of epithelial thickness, proliferation of enterocytes inflammatory, cell infiltration of lamina propria, and goblet cells in comparison to the challenged birds , not significantly differing from the challenged control group.The ileal histopathology results are shown in P < 0.001), and animals supplemented with 35,000 LSU(F)/kg of feed of MUR exhibited numerical decrease of the goblet cell score (P < 0.001). Despite positive numerical improvement of 35,000 LSU(F)/kg of feed of MUR and Enramycin on the ileum ISI Total Score on both 21 and 28 d, none of the supplemented groups significantly differed from the challenged control group.It was observed that at 28 d, the supplementation of MUR and Enramycin reduced the presence of oocysts in the epithelium (P < 0.001) increased the infiltration of macrophages in the liver at all the ages. The supplementation of both doses of MUR and Enramycin significantly attenuated this effect on d 7, 14, and 21. However, the supplementation of 35,000 LSU(F)/kg of feed of MUR had a strong positive effect in reducing the macrophage infiltration in the liver to same level as the nonchallenged control birds at all the ages evaluated. Additionally, the challenge significantly increased the infiltration of CD8+ lymphocyte in the liver at all ages (P < 0.001). At 7 d, the supplementation of 25,000 LSU(F)/kg of feed of MUR reduced its infiltration to the same level as nonchallenged birds and as those supplemented with Enramycin. At 14 d, however, only the supplementation of 35,000 LSU(F)/kg of feed of MUR reduced the CD8+ lymphocyte infiltration to the same extent as the nonchallenged birds. At 21 d, all the supplemented groups reduced the CD8+ lymphocyte infiltration, and at 28 d, only the highest MUR dose and Enramycin reduced its infiltration to the same level as the nonchallenged group. Lastly, it was observed that at 7 d  and 14 d  the supplementation of 35,000 LSU(F)/kg of feed of MUR significantly reduced the CD4+ lymphocyte infiltration in the liver to the same level as nonchallenged birds, while Enramycin only reduced it at d 7. At 21 and 28 d, there was no clear separation between the groups and, in general, the infiltration was lower than at 7 and 14 d.The results presented in P < 0.001) the infiltration of macrophages in the ileum at all the ages evaluated. At 7, 14, and 21 d both doses of supplementation of MUR reduced macrophage infiltration compared to the challenged control birds, but at 7 d the supplementation of 35,000 LSU(F)/kg of feed of MUR and Enramycin showed the greater effects. At 28 d, only the supplementation of 35,000 LSU(F)/kg of feed of MUR reduced the macrophage infiltration to the same level as the nonchallenged control birds, while the lower inclusion level of MUR and Enramycin supplementation did not differ from the challenge control group. Furthermore, at 7 and 14 d, both doses of MUR and Enramycin reduced (P < 0.001) the infiltration of CD8+ lymphocytes; however, at 21 (P < 0.001) and 28 d (P = 0.009), the supplementation of 35,000 LSU(F)/kg of feed of MUR had the largest effect in reducing the CD8+ lymphocyte infiltration. Lastly, at 7 and 14 d, the supplementation 25,000 LSU(F)/kg of feed of MUR promoted the highest infiltration of CD4+ lymphocyte when compared to the other treatments (P < 0.001). At 21 d, the challenged control birds showed the highest CD4+ lymphocyte infiltration in the ileum, followed by the birds supplemented with 25,000 LSU(F)/kg of feed of MUR and Enramycin (P < 0.001). No significant difference between the treatments was observed at 28 d of age.The results are presented in P < 0.05), but the supplementation did not counteract this effect. However, at d 14, the supplementation of 35,000 LSU(F)/kg of feed of MUR and Enramycin prevented the upregulation of IL-8 in the liver which was not different from the nonchallenged control group. Additionally, the challenged upregulated the expression of IL-10 in the liver at d 14, but the supplementation of MUR at both doses and Enramycin counteracted this effect.The relative gene expression of IL-8 and IL-10 in the liver and ileum of chickens is shown in In the ileum, it was observed that the challenge upregulated the expression of IL-8 on d 7 and 21, and the supplementation of MUR at both doses and Enramycin counteracted this effect, but only at d 21. The challenge also increased the expression of IL-10 in the ileum at the 3 d evaluated, and only the supplementation of Enramycin, at d 21, reduced its expression to the same level as the nonchallenged control group.P < 0.05), when compared to the challenged control or supplemented with Enramycin, respectively. Considering the entire experimental period, from 1 to 42 d, a linear positive effect (P < 0.001) of the MUR doses was observed on the BWG. No significant differences were observed for FI. However, the body weight corrected FCR from 1 to 42 d was significantly improved by 5.6% with the supplementation of MUR (P = 0.02) when compared to the nonsupplemented birds. Additionally, a linear positive effect of MUR doses was observed on the FCR from 1 to 42 d (P = 0.008). The improved growth performance of MUR supplemented birds, also resulted in improved PEI when compared to the birds supplemented with Enramycin (P = 0.01); in addition, a linear positive effect of MUR doses was observed on the PEI at d 42 (P = 0.008). Viability and uniformity of the flock were not different between the treatment groups (P > 0.05).The growth performance results of the Experiment 2 are shown in P = 0.05). The total concentration of carotenoids was increased within the supplementation of 45,000 LSU(F)/kg of feed of MUR and Enramycin when compared to the nonsupplemented group (P < 0.001). It was also observed a linear positive effect of MUR doses on the blood carotenoids concentration at d 42 (P = 0.006).The carcass yield and total blood carotenoid results are shown in Eimeria and/or Clostridium perfringens challenge with focus on growth performance, histopathology of the liver and ileum, immune-related parameters, and blood carotenoids concentration. In Experiment 1, MUR supplementation had positive impact on several macro and microscopic aspects evaluated, and reduced the infiltration of inflammatory cells in both liver and ileum during all the ages evaluated and modulated the expression of immune-related genes in both organs. These results are of paramount importance to explain the anti-inflammatory effect of dietary MUR. In Experiment 2, targeting the evaluation of the growth performance of broilers, the immunoregulatory effect of MUR was translated into improved BWG (up to 95% of the breeder guidelines with MUR supplementation), FCR, breast yield, and absorption capacity, as measured by total blood concentration of carotenoids, especially at d 42 that allowed for the longest cumulative period of supplementation.In the studies presented herein we evaluated the impact of MUR supplementation in broiler chickens undergoing Regarding the macro and microscopic evaluations, it was observed that the severity of lesions observed herein agreed with previous published work by Several publications have shown the benefits in supplementing broiler chickens with the same MUR to improve the growth performance beyond their genetic potential . BortoluC. perfringens , and improved absorption capacity, as shown by the higher blood concentration of total carotenoids.


Results of randomized controlled trials (RCTs) provide valuable comparisons of 2 or more interventions to inform health care decision making; however, many more comparisons are required than available time and resources to conduct them. Moreover, RCTs have limited generalizability. Comparative effectiveness research (CER) using real-world evidence (RWE) can increase generalizability and is important for decision making, but use of nonrandomized designs makes their evaluation challenging. Several tools are available to assist.In this study, we comparatively characterize 5 tools used to evaluate RWE studies in the context of making health care adoption decision making: (1) Good Research for Comparative Effectiveness (GRACE) Checklist, (2) IMI GetReal RWE Navigator (Navigator), (3) Center for Medical Technology Policy (CMTP) RWE Decoder, (4) CER Collaborative tool, and (5) Real World Evidence Assessments and Needs Guidance (REAdi) tool. We describe each and then compare their features along 8 domains: (1) objective/user/context, (2) development/scope, (3) platform/presentation, (4) user design, (5) study-level internal/external validity of evidence, (6) summarizing body of evidence, (7) assisting in decision making, and (8) sharing results/making improvements.Our summary suggests that the GRACE Checklist aids stakeholders in evaluation of the quality and applicability of individual CER studies. Navigator is a collection of educational resources to guide demonstration of effectiveness, a guidance tool to support development of medicines, and a directory of authoritative resources for RWE. The CMTP RWE Decoder aids in the assessment of relevance and rigor of RWE. The CER Collaborative tool aids in the assessment of credibility and relevance. The REAdi tool aids in refinement of the research question, study retrieval, quality assessment, grading the body of evidence, and prompts with questions to facilitate coverage decisions.All tools specify a framework, were designed with stakeholder input, assess internal validity, are available online, and are easy to use. They vary in their complexity and comprehensiveness. The RWE Decoder, CER Collaborative tool, and REAdi tool synthesize evidence and were specifically designed to aid formulary decision making. This study adds clarity on what the tools provide so that the user can determine which best fits a given purpose. Tools are available to assist; however, they have not been compared.The limitations of traditional randomized controlled trials (RCTs) in informing health care decisions are well known, thus, the recent emphasis on comparative effectiveness research (CER).7 In 2017, the ISPOR-International Society for Pharmacoepidemiology (ISPE) Special Task Force on RWE in Health Care Decision Making established good procedural practice intended to enhance confidence by decision makers in RWE derived from RWD.8 In 2019, the RWE Transparency Initiative, a partnership among ISPOR, ISPE, the Duke-Margolis Center for Health Policy, and the National Pharmaceutical Council (NPC) released a draft white paper recommending widespread use of registries to improve transparency of RWE studies.9 The AMCP Format for Formulary Submissions, Version 4.1 has evolved to include RWE as part of clinical evidence.6 The Institute for Clinical and Economic Review (ICER) has described the opportunities and challenges of using RWE for coverage decisions and developed a framework to guide optimal development and use of RWE for this purpose.11 Governmental and quasi-governmental agencies have been leading similar efforts.18Early efforts to develop a framework to assist health care decision makers in using RWD were led by the International Society for Health Economics and Outcomes Research (ISPOR).21 This situation has led to a rise in the development of a number of tools to guide HTA decision making. Yet, in our own HTA work, we find that colleagues are still seeking clarity about how to evaluate RWE. This uncertainty led to creation of the Real World Evidence Assessments and Needs Guidance (REAdi) tool  and, eventually, to a comparison of the features of all 5 tools highlighted in this study.In the context of CER, HTA decision makers are seldom positioned to conduct RWE studies or to undertake comprehensive systematic reviews to inform timely decision making. Existing barriers have slowed the rate at which RWE is used in decision making.35In this report, we describe the 5 best practice tools identified by experts in CER and RWE: the Good Research for Comparative Effectiveness (GRACE) Checklist, the Innovative Medicines Initiative (IMI) GetReal RWE Navigator, the Center for Medical Technology and Policy (CMTP) RWE Decoder, the CER Collaborative tool, and the REAdi tool .22-351.The GRACE Checklist was derived from a set of principles that define the elements of good practice for the design, conduct, analysis, and reporting of observational CER studies. The original GRACE principles were developed by Outcome Sciences (now part of IQVIA) with funding from the NPC and have been endorsed by ISPE. The principles served as the foundation for the 11-item GRACE Checklist that aids stakeholders in the evaluation of the quality and applicability of CER studies. The validated checklist was developed from a review of published literature and was tested globally by volunteers.2.Launched in 2018, the GetReal Initiative is a public-private partnership of pharmaceutical companies, academia, HTA agencies, and regulators across the European Union. The goal is to increase the quality of RWE generation in medicines development and regulatory/HTA processes, optimizing and ensuring adoption and sustainability of the tools earlier developed under the “GetReal” Consortium. The online RWE Navigator includes educational resources to guide demonstration of effectiveness, a guidance tool to support development of medicines, and a directory of and links to authoritative resources for evaluation of RWE, including quality and credibility. Navigator maps the landscape of RWE to help users “navigate” to what they need to prioritize and make decisions.3.The online CMTP RWE Decoder was developed through a multistakeholder initiative for the purpose of making available an easy-to-use tool to help decision makers confidently and consistently assess RWE for their decision making needs. The tool is an Excel spreadsheet that facilitates user assessment of the relevance and rigor of existing evidence from RCTs and RWE. Finalized in 2017, RWE Decoder is composed of 3 modules. In Module 1, the user articulates the question of interest, framing the question in the PICOTS  format. Module 2a provides the framework for assessing the relevance of each identified study. Module 2b prompts for assessment of the rigor of each individual study, including the quality of the evidence, potential for bias, precision, and data integrity. Module 2c calls for the magnitude and direction of effect. In Module 3, an integrated summary of each of these assessments is presented in graphical format. RWE Decoder is available in the public domain.4.37 The CER Collaborative tool is online and interactive. Training modules and video tutorials are available. A 19-hour CER certificate program accredited by the American Council on Pharmaceutical Education was offered for a fee.38Developed by the CER Collaborative, a multistakeholder initiative of NPC, AMCP, and ISPOR, the CER Collaborative tool helps users synthesize RWE and assess its credibility and relevance of evidence. The goal is to provide greater uniformity and transparency in the evaluation of RWE to inform HTA decisions. The CER Collaborative tool facilitates the critical appraisal of 4 types of studies  and is composed of 2 parts. Part 1 involves critical appraisal of individual studies along 2 dimensions: (1) relevance, using the PICOTS framework, and (2) credibility, by critiquing study design, data sources, analyses, reporting, interpretation, and conflicts of interest. In Part 2, evidence from multiple studies of varying designs is synthesized and assessed for reliability along 2 dimensions: (1) magnitude of comparative net health benefit and (2) evidence certainty. A joint rating is generated and presented graphically using the Evidence Rating Matrix from ICER with an available export function for formulary monographs.5.46 Once completed, in Phase 3, the user is prompted to rate the strength of the body of evidence using GRADEPro .47 In Phase 4, the user assesses the applicability and sufficiency of the evidence for the intended purpose. In Phase 5, questions are posed to facilitate coverage decisions relevant to immediate payer decision need  Institute. Intended to provide guidance on the use of RWE for HTA decision making for drug and diagnostic interventions, the REAdi framework is comprehensive in leading the user through the decisionmaking process in 5 phases. In Phase 1, the user defines the research question in the PICOTS format. Once defined, the tool automatically synthesizes terms to create a PubMed search strategy; citations of relevant studies are returned for review. In Phase 2, the user reviews and quality-rates the RWE on a per-study basis, having been guided to an embedded qualityrating tool specific to each included study design.ion need . Constru49 In this article, we describe and compare those 3 tools and add comparisons of the GRACE Checklist and the IMI Navigator. In evaluating each tool, we have identified 27 features. For comparison, we informally organized these into 8 domains  at the AMCP Nexus meeting  in leading an invited workshop to compare, contrast, and offer an opportunity to use 3 of these tools, using a case study from the literature. domains . DomainsThe objectives of the GRACE Checklist and IMI Navigator are each unique—the GRACE Checklist provides a simple tool to review study quality and guide aspects of CER studies, while the Navigator aims to increase awareness and understanding of RWE and has multiple uses. The RWE Decoder, CER Collaborative tool, and REAdi tool are each intended to guide formulary decision making. While the target users of the Navigator are a broad group of stakeholders, the target users of the other 4 tools are health care decision makers. The GRACE Checklist and Navigator are designed at a basic level of complexity, whereas the RWE Decoder, CER Collaborative tool, and REAdi tool are increasingly complex, while still being relatively easy to use. Aligned with the objective, the context of use of the RWE Decoder, CER Collaborative tool, and REAdi tool are specified; this is not the case with the GRACE Checklist and Navigator.Three of the five tools are intended for use with RWE only, while the RWE Decoder and CER Collaborative tool allow for consideration of RCTs. Stakeholders were involved in development of all 5 tools, and a framework for decision making is specified for each. The RWE Decoder and REAdi tool have a U.S. focus, whereas the Navigator was developed for use in Europe, and the GRACE Checklist and CER Collaborative tool are intended for international use.All tools are available online, in PDF format (GRACE Checklist), webpages (Navigator and the CER Collaborative tool), Microsoft Excel (RWE Decoder), or R Shiny (REAdi). All are publicly available at no cost.All tools provide definitions of terms. Navigator is focused solely on pharmaceutical interventions; the others are not limited to intervention category. All 5 tools have adopted at least some elements of the PICOTS framework. All but Navigator allow specification of primary and secondary outcomes simultaneously. Neither the GRACE Checklist nor the RWE Decoder provide a list of study designs to which each tool can be applied. The Navigator and REAdi tool accommodate many study designs, while the CER Collaborative focuses on cohort studies, cost-effectiveness analyses, and (network) meta-analyses. The REAdi tool explicitly allows users to design and tailor research questions, guiding them to appropriate study designs, and assists users in constructing key words and search strategies that result in automatic execution of a PubMed search. As a checklist and collection of online resources, neither the GRACE Checklist nor Navigator are digitally interactive, while RWE Decoder, the CER Collaborative tool, and the REAdi tool are.All tools provide a systematic method to assess internal validity ; Navigator provides links to quality rating tools, while the REAdi tool embeds most of these. RWE Decoder provides 1 tool each for assessing the rigor of RCTs and non-RCTs, using the quality of the research questions, potential for bias, precision, and data integrity. The CER Collaborative tool assesses credibility using checklists corresponding to design, data, analysis, reporting, and interpretation. RWE Decoder, IMI Navigator, the CER Collaborative tool, and the REAdi tool assess relevance, that is, external validity, while the GRACE Checklist does not.The tools vary in their methods for summarizing the body of RWE. The GRACE Checklist is applicable only to individual studies. Navigator mentions the GRADE criteria, but GRADE is not a built-in feature. RWE Decoder uses a 3-dimensional graphic to summarize relevance, rigor, and effect size. The CER Collaborative tool has adopted ICER’s Evidence Rating Matrix that illustrates magnitude and certainty in 2 dimensions, which follows the GRADE methods. The REAdi tool has embedded the GRADEPro criteria.All 5 tools provide a structured framework for decision making, while the REAdi tool also provides recommendations for decision making. All except the Navigator provide documentation of tool usage, with the GRACE Checklist using PDF format and RWE Decoder and the CER Collaborative tool using Excel; the CER Collaborative tool also uses Word. The REAdi tool prints screen and saves projects.Each tool allows users to save their ratings and share with other users. Only the REAdi tool is designed to log, collect, and analyze user inputs to facilitate iterative improvements in features.We reviewed 5 online tools to synthesize RWE of CER, which vary in their objectives, complexity, and context for use. The simplest to use, the GRACE Checklist, provides a simple quality rating, while the Navigator provides education, guidance, and resources. The 3 remaining tools—RWE Decoder, the CER Collaborative tool, and the REAdi tool are similar to each other in that they integrate quality ratings, education, and guidance resources. They are therefore more complex and are intended to evaluate a body of RWE to enhance formulary decision making. The RWE Decoder and CER Collaborative tool are useful for evaluating already identified evidence, while the REAdi tool spans a broader set of decision-making tasks by beginning upstream in explicitly assisting the user in specifying the research questions and finishing downstream by offering recommendations for formulary decision making. With their varying features, breadth of tasks, and levels of complexity, the RWE Decoder, CER Collaborative tool, and REAdi tool synthesize evidence and were specifically designed to aid formulary decision making.21 Future research could also include a more in-depth comparison of these tools in the context of case studies to determine which features are of greatest value to decision makers. Best practices for tool use could then be developed and existing tools integrated. A discussion could then ensue about strategies to sustain the new tool. In the meantime, this study adds clarity on what the tools provide so that the user can determine which best fits a given purpose.This study characterizes 5 potentially useful tools for HTA decision making using RWE. Because use of RWE is low, research that explores awareness, usefulness, and barriers to use of these tools may result in their improvement, uptake in their use, and ultimately increased use of RWE for decision making.


Community-based organisations continue to take on a greater role in supporting children and young people in the UK with their mental health. However, little evidence exists on the capacity and capability of these settings to effectively implement evidence-based interventions (EBIs).To identify barriers and facilitators of the implementation of EBIs within community settings in the UK, for children and young people's mental health promotion, prevention and treatment.A PRISMA-guided, rapid scoping review was conducted, using predefined criteria and a relevant search strategy on eight databases: Ovid EMBASE, Ovid Medline, Ovid PsycINFO, Ovid Global Health: Scopus, Web of Science All Databases, EBSCO CINAHL and EBSCO ERIC. Study characteristics and data on barriers and facilitators were extracted, with results narratively synthesised.Five out of 4899 studies met the inclusion criteria, addressing the barriers and facilitators of community-based implementation of EBIs for children and young people's mental health promotion, prevention and treatment. All of the studies that were identified focused on school settings, but we identified no studies that included data on barriers or facilitators of implementing EBIs in other community-based or voluntary sector settings.There is a lack of available evidence on the capacity and capability of community settings in the UK to effectively implement EBIs and adhere to evidence-based practice. However, existing findings within schools have highlighted key barriers and facilitators to implementation, such as the importance of meaningful involvement of stakeholders throughout the research process, and greater allocation of resources to support evidence-based decision-making in these settings. Alongside this rising prevalence, there has been a subsequent increase in demand for clinical, voluntary and statutory services to support children and young people with their mental health. In an NHS Providers Survey carried out in May 2021, 100% of Mental Health Trust leaders said that the demand their systems were experiencing for services for children and young people had significantly (80%) or moderately (20%) increased compared with the previous 6 months.2 With the problem likely exacerbated by the COVID-19 pandemic and associated lockdowns, the need to increase provision for children and young people's mental health is now in sharp focus.Over the past decade, the UK has seen a significant increase in the prevalence of mental health problems in children and young people.3 Evidence-based interventions (EBIs) now exist for use across a wide range of clinical and non-clinical settings seeking to support children and young people. Those working within the voluntary and community sector (VCS) have been called upon by the UK Government in recent years to provide more visible and accessible support for children and young people.4 This includes voluntary and community groups, charities and third-sector organisations, all of which play a critical role for families and young people. Schools are similarly becoming recognised as community providers of mental health support, in what has been termed the ‘therapeutic turn’ in education.5 EBIs exist within these settings and have the potential to increase access to mental health guidance and support, particularly for those disadvantaged children and young people who otherwise would not receive help.6 Yet, despite a widespread call within the field to increase the role of community settings, such as schools and the VCS, in providing support for children and young people, little evidence exists on the capacity and capability of these settings to effectively implement EBIs and adhere to evidence-based practice.7A growing body of research is evolving aimed at supporting the promotion of mental health, prevention of emotional distress and development of treatment interventions to meet this need.8 despite reports that greater fidelity to intervention models is typically associated with better outcomes.9 In those studies that do report on implementation, several barriers and facilitators are emerging , but their specific relevance for community services supporting children and young people has yet to be determined.8Previous research shows that very few studies to evaluate EBIs within community settings adequately report on implementation, adaptation and fidelity within their findings,10 It is essential that implementers within community settings in the UK – be it youth workers, teachers, volunteers, parents or even young people themselves – receive practical and relevant guidance and suggestions on how to implement EBIs, with this context in mind.Youth services in the UK are facing a specific set of challenges, including significant cuts to funding beyond that of adult services.8 The increasing prevalence of mental health problems in children and young people, and the rising demand on schools and the VCS in the UK to provide support, adds further urgency to better understand how evidence informs practice in these settings and address gaps between evidence and implementation. Therefore, this rapid scoping review of the literature sought to explore factors influencing the implementation of EBIs within UK-based and youth-focused community settings. Although existing research has examined implementation within the VCS globally, supporting individuals across ages,7 this review will adopt a narrower focus and seek to understand how evidence informs practice in UK-based, youth-focused community services. A scoping review was selected as it is well-equipped to map the available evidence in this field, and to identify key barriers and facilitators to implementation within the literature.11 The focus of this review is on those EBIs specifically targeted at children and young people aged 0–18 years, intending to promote, prevent or treat mental health problems within community settings. This review focuses on studies from the UK only, because of its specific organisational landscape, with the aim of better understanding the subsequent contextual determinants of barriers and facilitators facing researchers working to implement EBIs in the VCS and encourage future research in these settings. Given its focus, this review aimed to provide suggestions for ‘implementers’ that were most relevant to this unique context and considered the most appropriate evidence.Effective implementation of EBIs at scale presents a challenge for those working to support children and young people, yet it is critical for the delivery of optimal and sustainable outcomes for families accessing services.The main objectives of this review were (a) to identify barriers and facilitators of the implementation of EBIs in the UK, within community settings, for children and young people's mental health promotion, prevention and treatment; and (b) to draw on the identified literature to provide practical guidance and suggestions for ‘implementers’ within community settings in the UK, be it teachers, volunteers, parents or even young people themselves.The focus of this rapid scoping review is captured in the following question: what are the barriers and facilitators of community-based implementation of EBIs in the UK, for children and young people's mental health promotion, prevention and treatment?A rapid scoping review was conducted according to the procedure and requirements described in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR). No protocol exists for this review.The search strategy was designed to be exhaustive in terms of identifying the current literature on studies investigating barriers to, and facilitators of, the delivery of EBIs outside of a clinical mental healthcare setting. A body of research addressing this topic was identified before the search through word of mouth, key reference lists and simple searches to check iterations of the strategy. The search was revised until it was sensitive enough to capture the preidentified studies. Eight databases were searched from 2011 to 30 July 2021: Ovid EMBASE, Ovid Medline, Ovid PsycINFO, Ovid Global Health: Scopus, Web of Science All Databases, EBSCO CINAHL and EBSCO ERIC.12 implementation terms were derived from Bach-Mortensen et al,7 mental health terms were derived from Uphoff et al13 and the Ovid EMBASE and Medline search strategies by Ayiku et al15 were used to retrieve research about the UK. These search strategies were then adapted for the other databases. Search results were limited to publications in the English language only. The full set of search strategies is available in Supplementary Appendix A available at https://doi.org/10.1192/bjo.2023.531.A search strategy was constructed in collaboration with an information scientist (E.H.) to search for the relevant thesaurus headings in the Ovid databases, and free-text terms were applied to the title or abstract fields  to search for synonyms for the following six concepts: children; evidence-based practice; mental health; community; implementation, barriers or facilitators; and the UK. The terms for evidence-based practice and barriers and facilitators were derived from Cahill et al,https://endnote.com/product-details), and duplicates were removed following the method described by Falconer,16 before being imported into Rayyan.ai online platform for further deduplication and screening.All references were exported to EndNote for MacOS Ventura 13.4.1 X9  were considered.7 The population of study included young people, practitioners, commissioners and policymakers from within schools or community-based settings. This includes education practitioners, regional education authorities, third-sector organisations, social services, welfare services, local authorities, community leaders, police and crime commissioners. ‘Community-based’ was understood as any organisation that operates outside of the National Health Service, or other public or private healthcare providers (i.e. primary care practitioners would not be included).In this review EBIs were defined as ‘interventions that have empirical support for improving mental health outcomes, which have been tested and validated according to the principles of evidence-based practice ’. This definition has been validated in previous reviews of EBIs in the VCS.10 and subsequent resourcing and capacity challenges. Overall, this review considers the evidence most relevant to this UK context.Studies were included if the data was collected within the UK between 2011 and 2021. As previously mentioned, although global literature can provide important insights, in the case of this review, the objectives were to inform national guidance and provide suggestions for ‘implementers’. Therefore, focusing this search within the UK in the past 10 years would likely offer a better overview of the specific landscape, current challenges and considerations faced by schools and VCS within this country. For example, in the UK, there have been significant funding cuts to the VCS since 2010,n = 1) were discussed, and a consensus was reached in line with the decision made by the primary screener.To identify eligible papers, two raters independently reviewed titles and abstracts against the inclusion criteria (A.T. and A.P.-C. or K.K.). There was strong agreement (95–96%) between the screener pairs. A.T. retrieved relevant articles for full-text screening, and a random 10% of the included studies were also cross-checked at the full-text screening stage by A.P.-C. The full-text screening allowed for the exclusion of those papers investigating interventions or programs that are not evidence-based or do not provide data on barriers or facilitators to implementation. Disagreements ; methods ; population ; the focus of EBIs implemented ; and results (barriers and facilitators in implementing EBIs).17 a basic descriptive analysis and narrative overview is provided in relation to the research question. School-based studies are separated from studies based in other community-based organisations at this stage, because of the organisational differences that exist within school settings.Relevant barriers and facilitators were mapped and presented within tables. In line with the aims of a scoping review,The literature search resulted in 4899 citations , of whic19 We did not identify any studies that included data on the barriers and facilitators of implementing EBIs in other community-based or voluntary sector settings.A total of five studies met the inclusion criteria and were included in the review, all of which collected data on EBIs delivered in schools . All stuEach included study reported on barriers  and faci21 Researchers believed a lack of fidelity to the intervention model occurred as a result of poor points of contact with schools delivering it, and noted that many school staff opted to deliver only specific components of the intervention that they felt were most helpful or felt most confident in.21 Indeed, in all of those studies included, those advocating for an intervention and its fidelity were rarely the same people who implemented it daily.Several intervention-specific barriers and facilitators to implementation were noted. Researchers highlighted the importance of integrating the research team in the delivery of the intervention or improving points of contact with the investigators, within the environment in which the intervention is being promoted and delivered.18 Similarly, interventions using tools that students were already familiar with, and could access when needed, were much more successful in their implementation than interventions using novel technology that was restricted by school staff.18 Actively engaging students, parents, carers and teachers in the intervention was identified as especially important for encouraging sustained implementation and adherence to evidence-based practice. For example, in those EBIs that relied on parental consent, where researchers failed to engage parents and carers, implementation of the EBI within the school community was hindered.18 Similarly, in those interventions which drew upon existing knowledge and skills present in schools already, championing of the interventions within the school community and long-term implementation were supported.19 Other factors that affected the adherence to evidence-based practice included the complexity of the intervention, with staff commenting that more sophisticated and complex procedures had a negative effect on the acceptability of the approach and their confidence in delivery.22 More complex interventions similarly required more training, which many school staff did not have the capacity for.22 Other studies noted the value of not wholly relying on teachers to enable the implementation of intervention tools because of limited teacher capacity to supervise students’ use of the tool, but rather to include student self-directed intervention use.18It appeared that interventions were more effectively implemented if they could be easily embedded into current routines, practices or tools; for example, by implementing an intervention at the start of the school year, rather than partway through when routines have been established.20 Further school-specific facilitators of implementation included the active support of (or lack of active obstruction from) the senior leadership within a school.20 The type of school within which the intervention was implemented was also highlighted.18 For example, the organisational complexity of a secondary school setting presented barriers to the consistency of delivery, reinforcement of intervention procedures, communication of intervention utility and, ultimately, adherence to the evidence base.19 Similarly, intervention delivery in alternative provisions presented extra challenges, as students required a greater number of support staff for delivery, which, for some interventions, was deemed to be unsustainable.18 Indeed, the availability of staff and resources within the school context was a challenge across all studies.Several barriers and facilitators at a school level were also identified. In particular, limitations in school-wide capacity, readiness and willingness were highlighted as potential barriers to implementation across all of the studies. Although each study faced similar systematic challenges with capacity and resourcing, differences in implementation occurred as a result of each school's vision and ethos concerning mental health and well-being, which, in some cases, encouraged resource allocation toward supporting the implementation of the intervention.Barriers and facilitators also occurred as a result of education-specific systemic challenges. The most frequently noted challenge across all studies included a need to prioritise the academic curriculum and resources away from the intervention as a result of steep targets and governmental pressure. All schools struggled to combat this barrier when implementing interventions in the long term and adhering to evidence-based practice.Five studies were identified that addressed the barriers and facilitators of community-based implementation of EBIs for children and young people's mental health promotion, prevention and treatment. All identified studies focused on school settings and, unfortunately, no studies were found that included data on the barriers or facilitators of implementing EBIs in wider community-based or voluntary sector settings.19 It is critical that researchers attempting to implement EBIs within a school setting actively engage and communicate with parents, carers and school staff, to ensure its acceptability within this context. Such a process might also aid fidelity to the intervention model if stakeholders and implementers feel able to communicate openly about the specific challenges they are facing, and the adaptations required to effectively implement the intervention.There were many similarities in the identified barriers and facilitators to implementation across the included studies. Perceived barriers and facilitators related to three domains: intervention-specific, organisational and systemic. With regard to intervention-specific barriers, the lack of active engagement and open communication with stakeholders was identified as a perceived barrier across all included studies. Engaging with the whole school community, including parents/carers, school staff and young people, was deemed to be paramount in ensuring the long-term implementation of an intervention within schools. By working closely with schools, understanding each school's ethos and the existing skills, knowledge and resources they have to offer, interventions are much more likely to be championed by stakeholders and within the school community. This could potentially aid evidence-based practice by encouraging resources and funding for mental health programmes, often overcoming many of the systemic barriers noted within schools.20 Although this was largely out of the researchers’ control, the recent ‘therapeutic turn’ in education5 may encourage more schools to allocate resources toward implementing universal EBIs for mental health and well-being. This may also improve fidelity to the intervention model in those instances where the required number of resources is high. It is also important to recognise the additional challenges that different types of schools face when implementing EBIs, including alternative provisions.18 Alternative provisions oversee some of the most vulnerable children and young people, many of whom seek support from both clinical and non-clinical settings.23 In these settings, students require a greater number of support staff for EBI delivery, which may be unsustainable in the long term with currently available resources.18 As such, researchers should be mindful of, and seek ways to, sustain the implementation of EBIs where a greater level of resource and support is required, to assist in effective implementation and evidence-based practice within settings such as alternative provisions.Indeed, the organisational context of schools presented a barrier to implementation in itself. Unsurprisingly, all of the included studies identified challenges with school-wide capacity, readiness and willingness to implement EBIs. However, for some studies, the impact of such challenges was reduced as a result of the school's positive ethos and culture surrounding mental health and well-being.4 the reality of funding cuts and governmental pressure to meet steep academic targets encouraged resource allocation away from EBIs. This presented a barrier to implementation that all studies and schools struggled to combat. Notably, those initiatives that were encouraged by the UK Government  faced similar challenges in prioritisation and implementation.19 To overcome this barrier it is key that interventions are developed with the involvement of the school community  who understand and work with these systemic challenges on a daily basis. Being willing to work closely with stakeholders as an intervention is designed and trialled could create solutions to potential barriers before they arise.Barriers and facilitators were also identified at a systemic level within the education system. Despite recent shifts and encouragement by the UK Government to offer more visible and accessible support for children and young people in the context of schools and the VCS,8 There were both within- and between-school variations for implementation quality. There was no difference in terms of the type of intervention being implemented . Notably, although all of those included studies commented on the barriers and facilitators of implementation within schools, none offered any pragmatic tools or guidance to optimise evidence-based practice in these settings. To support evidence-based practice in community-based services, future research should go beyond developing an understanding of the barriers and facilitators faced by implementers, and instead develop pragmatic implementation tools and frameworks for use in schools and the VCS.This review identified that the barriers and facilitators to the implementation of evidence-based practice for youth mental health in school settings are similar to those emerging within the wider literature, including resource challenges, level and quality of support available, and organisational structure.4 There is a clear need to develop research in this area and support new initiatives to implement evidence-based mental health practice within schools and community-based organisations. It is crucial to engage with stakeholders throughout these initiatives, and throughout the design, delivery, dissemination and implementation of an intervention.25Nevertheless, the findings of this review do highlight a number of challenges that UK implementers must be aware of when engaging with mental health interventions in their setting . They further offer some potential solutions to overcome some of these barriers , all of which one can be mindful of in the absence of any pragmatic tools or frameworks at this stage. This will be especially pertinent as these settings are increasingly called upon to support the mental health of children and young people in the community.It is important to highlight several limitations of this review. First, the lack of studies identified limited conclusions that could be drawn. There was a deliberate focus on studies from the UK only, given the likely contextual determinants of barriers and facilitators of implementation of EBIs. However, it is clear from these findings that there is not yet a sufficient evidence base within the UK, and research should turn toward the international evidence base in the meantime.Similarly, for pragmatic reasons and in line with the rapid scoping review methodology, grey literature was not considered in this review. The authors recognise that reports on EBIs in the UK VCS are likely to exist beyond the peer-reviewed evidence base, and had these reports been included in this review, such additional perspectives may have changed the overall conclusions drawn from this study. The lack of reports from the VCS that were identified in the peer-reviewed literature in this study reflects a disconnect between academic research(ers) and the VCS – an important finding to consider. To bridge that gap, future research should make greater efforts to consider perspectives from the VCS that exist outside of the peer-reviewed evidence base, integrating findings across a range of disciplines to better support young people with their mental health.26Similarly, in future follow-ups, researchers should consider the nature of community-based settings and how they operate. Interventions are often in the earlier stages of development than those implemented in more clinical settings, and therefore adopting less stringent criteria on what is deemed to be ‘evidence-based’ may be useful in fully capturing the types of interventions being implemented in these settings. For example, there is a publication bias toward professionally led interventions, whereas many small and successful community health projects operate under the radar of formal evaluations.27 More evidence needs to be based on the lived experiences of those most affected by health inequalities, including through research controlled and led by patients/participants. In this review, only two of the included papers collected data on students’ experiences of the EBI being implemented.20 Young people have commonly been excluded and their voices have been absent from the research process in the past.28 Although this is changing, in cases where involvement with children and young people exists, it has sometimes been tokenistic and lacking in effectiveness.28 Future research should amplify the voices of young people and other stakeholders throughout the implementation process.In addition, the dynamic nature of participatory methods being utilised in the VCS is not always reflected in academic literature, and often changes occur long after the evaluators have left.Finally, included studies were limited to shared characteristics, including the setting within which they took place (schools), the type of intervention  and the method of data collection . No studies were identified from the wider VCS. As such, the review is limited in its ability to address barriers and facilitators to the implementation of EBIs, beyond school-based universal interventions.7 Although there were a number of limitations, largely because of the rapid scoping review methodology, these findings (or lack thereof) represent a dearth of understanding or representation in the scientific literature of implementation and community-based approaches. They demonstrate a disconnect between academic researchers, practice and people.In conclusion, this review has highlighted a lack of available evidence on the capacity and capability of community settings in the UK to effectively implement EBIs and adhere to evidence-based practice.More attention is needed to understand how EBIs are transported, contextualised and implemented by community providers, to ultimately broaden access to mental health support and guidance for children and young people. Ultimately, as the demand for clinical services grows, community settings must be given the tools to support children and young people. As such, research is needed to promote the development and dissemination of practical and sustainable implementation tools and guidance to support these organisations and the professionals working within them. Improving efficient and effective implementation of EBIs within community settings will require meaningful involvement of stakeholders throughout the research process, greater allocation of resources and funding for research and development to support evidence-based decision-making in schools and other community-based organisations, as they continue to take on a greater role in supporting children and young people with their mental health.


Inequalities in access to and utilization of maternal and child health (MCH) care are hampering progress on the path to achieving the Sustainable Development Goals. In a number of Low- and Middle-Income Countries (LMICs) population subgroups at disproportionate risk of being left behind are the urban poor. Within this neglected group is the further neglected group of the homeless. Concomitantly, a number of interventions from the antenatal period onward have been piloted, tested, and scaled in these contexts. We carried out an overview of systematic reviews (SRs) to characterize the evidence around maternal and child health interventions relevant to urban poor homeless populations in LMICs.We searched Medline, Cochrane Library, Health Systems Evidence and EBSCOhost databases for SRs published between January 2009 and 2020 (with an updated search through November 2021). Our population of interest was women or children from urban poor settings in LMICs; interventions and outcomes corresponded with the World Health Organization’s (WHO) guidance document. Each SR was assessed by two reviewers using established standard critical appraisal checklists. The overview was registered in PROSPERO (ID: CRD42021229107).In a sample of 33 high quality SRs, we found no direct relevant evidence for pregnant and lactating homeless women (and children) in the reviewed literature. There was a lack of emphasis on evidence related to family planning, safe abortion care, and postpartum care of mothers. There was mixed quality evidence that the range of nutritional interventions had little, unclear or no effect on several child mortality and development outcomes. Interventions related to water, sanitation, and hygiene, ensuring acceptability of community health services and health promotion type programs could be regarded as beneficial, although location seemed to matter. Importantly, the risk of bias reporting in different reviews did not match, suggesting that greater attention to rigour in their conduct is needed.The generalizability of existing systematic reviews to our population of interest was poor. There is a clear need for rigorous primary research on MCH interventions among urban poor, and particularly homeless populations in LMICs, as it is as yet unclear whether the same, augmented, or altogether different interventions would be required.The online version contains supplementary material available at 10.1186/s12889-023-15410-7. Inequality in access to maternal healthcare services has consequences for Sustainable Development Goals (SDG) target 3.1 and 3.2, which relate to reduction of the global maternal mortality ratio, neonatal mortality and under-5 mortality, respectively [By the last year of SDG, i.e. 2030, it is estimated that 60 per cent of people will live in cities . The conDespite inequalities in health outcomes among the urban poor, issues relating to access persist. A study of 22 African cities showed disruptions in the maternal continuum of care, characterized drop offs in antenatal care, childbirth and postpartum care, with varying reliance on public and private sector use, and use of hospitals across cities . AnotherInterventions introduced from the antenatal period to the later childhood period can bring about a decrease in neonatal and later mortality . InterveMaternal and child health interventions have been a major focus of public health research for decades and are in a sense core to the discipline itself. Drawing on this evidence base, a multitude of systematic reviews have assessed a broad range of interventions, with varying primary aims and outcomes recognize the potential of interventions to improve maternal and child health outcomes –18. We sAn overview of SRs was conducted based on a pre-defined protocol, which was registered in PROSPERO .Systematic reviews involving women and children from urban poor settings in LMICs, including the homeless were included. Populations of interest were women  or children from urban poor settings in LMICs. We were looking for any interventions  addressing MCH were considered, guided by the WHO guidance document on packages of interventions for family planning, safe abortion care, maternal, newborn and child health . ComparaAll the SRs with or without meta-analysis of any study design were included. Non-English language reviews were considered for inclusion where English translated reviews were available. However, the overview did not find any SRs published in non-English languages. All the available SRs published from January 2009 till January 2020 were considered. An updated search was conducted from February 2020 till November 2021, utilising the same search strategies. Reviews that incorporated theoretical studies or text or opinion as the primary source of evidence were excluded. Reviews that included interventions conducted only in high-income countries (HICs) and those that had interventions in general women and child populations  conducted in LMICs were excluded.A comprehensive literature search was conducted in databases such as Medline (PubMed), Cochrane, Health System Evidence (HSE) and EBSCOhost platform. The search strategies are available in Supplementary File 1. Two reviewers  independently performed preliminary screening of titles and abstracts of the records with support from SM. Full-text screening of the selected records was done independently by SD and BM. Conflicts were resolved with mutual consensus and consultation with a third reviewer (DN). The review selection process is presented in the flow diagram adapted from the Preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines  checklist  and JoanThe data were independently extracted by SD and BM using a predesigned data extraction template. The following data were extracted: author/year, objectives, review characteristics, description of interventions and comparators, outcomes, and results. Extracted findings were summarised narratively by population and intervention type. Where SRs assessed certainty of evidence using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE)  and GRADBased on a comprehensive search performed, 203 reviews were identified. After removing duplicates, 177 records underwent title and abstract screening, of which 87 were excluded. Of the 93 records remaining, 54 records were excluded during the full-text screening of the articles. Thirty-three high-quality reviews were finally included for this review ; however, low quality evidence indicated that the interventions may reduce stunting in infants and children below five years of age in locations outside slum areas . The revA qualitative review of factors influencing antenatal care (ANC) visits found that pregnancy was seen as a healthy condition by women in a slum in Dhaka that rendered ANC visits unnecessary, and this finding was based on high confidence in the evidence .Among the included SRs, 14 reviews were on nutritional interventions, 15 on community health services  and health promotion, while three reviews were on water sanitation and hygiene (WASH) and one on slum upgrading. Findings from these reviews are summarized below.Most reviews reported on nutrient supplementation for improving MCH. Some interventions assessed supplementation , others looked at fortification, and the remaining examined supplementary feeding.Moderate quality evidence indicated that lipid-based nutrient supplements (LNS) (given to pregnant women) might be of slight benefit to babies who are born small, as well as on newborn weight and length compared to iron-folic acid (IFA) . In anotVery low quality evidence indicated that providing additional food to children aged three months to five years may lead to small gains in weight (0.24 kg a year) and height (0.54 cm a year) . SpecifiModerate quality evidence suggested that IFA and MMN likely resulted in a decrease in maternal anemia compared to LNS . Low quaModerate quality evidence indicated that LNS likely led to a clinically significant benefit in the number of children recovering from malnutrition compared with blended foods . Kramer Moderate quality evidence suggested that vitamin A supplementation (VAS) given to infants in the 1–6 months age group likely did not reduce mortality or morbidity . In a reCommunity health services and health promotion interventions carried out in a wide range of contexts have shown some impact on a range of disease conditions.Moderate quality evidence indicated that the administration of anthelminthic (or co-interventions) for soil-transmitted Helminth (STH) infections during the second or third trimester of pregnancy likely resulted in little to no difference on preterm births or perinatal deaths . ModeratLow quality evidence showed that Integrated Management of Childhood Illness (IMCI) strategies including post-natal home visits may lead to lower neonatal and infant mortality . Low quaHigh quality evidence showed that single community-based interventions and health systems interventions (including home visits for pregnant women by CHWs) did not reduce LBW . AlthougLow quality evidence suggested that nutritional education interventions given to pregnant women may slightly improve LBW compared to standard care or no intervention . AnotherModerate quality evidence showed that IMCI strategies including post-natal home visits had little to no impact on measles vaccine coverage . In anotLow quality evidence suggested that monthly administration of sulphadoxine-pyrimethamine (SP) was found to decrease maternal parasitemia and placental parasitemia at the time of delivery in HIV-positive pregnant women in their first or second pregnancy, living in malaria-endemic areas . Low quaIntegration of HIV/AIDS and MNCHN-FP  services likely had positive effects on contraceptive use, HIV testing, initiation of antiretroviral therapy in pregnancy; however, the quality of the evidence was low . Low to Low to very low quality evidence indicated that education and hygiene promotion interventions with messages on disposal of child faeces might decrease diarrhea incidence by nearly 30% but did not affect diarrhea prevalence . EvidencLow quality evidence based on short-term studies indicated that WASH interventions  showed a marginal benefit on linear growth of children aged under five years . It was This overview examined interventions addressing MCH among urban poor populations, including homeless people in LMICs. It must first be mentioned that given the variation in definition and operationalization of populations, interventions and outcomes of interest, direct comparisons are likely to be misleading. We found no direct relevant evidence for pregnant and lactating homeless women (and children) in the reviewed literature. Most SRs that evaluated interventions relating to pregnant and lactating homeless women were conducted in HICs, which indicates a need to conduct more research in this domain in LMICs.The results showed that there was mixed evidence of effect of slum upgrade, sanitation education and nutritional interventions on improving outcomes in urban poor pregnant and lactating women (and children) in LMICs , 37, 39.Supplementary feeding had little or negligible impact on the growth of children under five years of age , 42, 66.There was no evidence of a reduced risk of mortality due to neonatal VAS on children less than one-year-old , 50, 68.Vitamin D supplementation had no benefit on the incidence of pneumonia,  and in cThree or more doses of sulphadoxine-pyrimethamine given to HIV positive women may have a marginal effect on the prevalence of maternal anemia and the number of LBW babies,  and IPT There was mixed evidence on the effect of mass media campaigns and education on the uptake of ANC services . SimilarThe evidence on the impact of educational interventions on growth outcomes was mixed and generally of low quality, which indicated that educational interventions slightly improved immunization uptake and that redesigned reminder cards may enhance immunization uptake. In comparison, an SR suggested that educational interventions significantly increased childhood immunization uptake in LMICs, compared to HICs where the intervention was not consistently effective . EvidencThe effect of handwashing on diarrheal episodes differed based on location (school/community), specifically incidence. Evidence from less developed countries showed that handwashing reduced diarrhea illness , anotherThe varying reportage of risk of bias for overlapping studies in different SRs was noted in this overview. It highlighted the need for greater rigour and consistency in the quality appraisal of reviews. The main limitation of our overview was the non-use of search terms related to individual LMIC countries. This may have resulted in the omission of some SRs. Further, given the relative dearth of literature, we are now persuaded to attempt a review rather than an overview for our population of interest, urban poor women and children as the literature is not as deep as we had anticipated when we began the overview. We had constraints of time in this exercise and moreover, were interested in the diversity of populations covered in existing reviews of MCH interventions (which we found to be quite poor!) .There are some key policy implications of these findings. For one, the review suggests that even as MCH interventions have been the mainstay of public health interventions for decades; rather little is known about their impact on urban poor and homeless populations. Even as there remains work to be done with MCH interventions overall - in neglected areas like family planning, safe abortion care, and postpartum care of mothers- there is a need to generate primary evidence on what interventions work for urban poor and particularly homeless families.This review cast a wide net to try to see what interventions work for the MCH needs of the urban poor and homeless. Overall, the review did not find directly relevant information for the homeless. As regards the general urban poor population, there was a lack of emphasis on evidence related to family planning, safe abortion care, and postpartum care of mothers. There was mixed quality evidence that the range of nutritional interventions had little, unclear or no effect on several child mortality and development outcomes. From a policy perspective, this suggests that more research would be needed before promoting such interventions for urban homeless people. Interventions such as WASH, ensuring acceptability of community health services and health promotion type programs could be regarded as beneficial, although location seemed to matter. Overall, the generalizability of findings was poor, and there is a clear need for rigorous primary research on MCH interventions among the urban poor and homeless.Below is the link to the electronic supplementary material.Supplementary Material 1


In studies of eHealth literacy, researchers have primarily assessed participants’ perceived eHealth literacy using a short self-report instrument, for which ample research has shown little to no association with actual performed eHealth-related skills. Performance-based measures of eHealth literacy may be more effective at assessing actual eHealth skills, yet such measures seem to be scarcer in the literature.The primary purpose of this study was to identify tools that currently exist to measure eHealth literacy based on objective performance. A secondary purpose of this study was to characterize the prevalence of performance-based measurement of eHealth literacy in the literature compared with subjective measurement.We conducted a systematic scoping review of the literature, aligning with the PRISMA-ScR  checklist, in 3 stages: conducting the search, screening articles, and extracting data into a summary table. The summary table includes terminology for eHealth literacy, description of participants, instrument design, health topics used, and a brief note on the evidence of validity for each performance-based measurement tool. A total of 1444 unique articles retrieved from 6 relevant databases  were considered for inclusion, of which 313 (21.68%) included a measure of eHealth literacy.Among the 313 articles that included a measure of eHealth literacy, we identified 33 (10.5%) that reported on 29 unique performance-based eHealth literacy measurement tools. The types of tools ranged from having participants answer health-related questions using the internet, having participants engage in simulated internet tasks, and having participants evaluate website quality to quizzing participants on their knowledge of health and the web-based health information–seeking process. In addition, among the 313 articles, we identified 280 (89.5%) that measured eHealth literacy using only a self-rating tool.This study is the first research synthesis looking specifically at performance-based measures of eHealth literacy and may direct researchers toward existing performance-based measurement tools to be applied in future projects. We discuss some of the key benefits and drawbacks of different approaches to performance-based measurement of eHealth literacy. Researchers with an interest in gauging participants’ actual eHealth literacy  should make efforts to incorporate tools such as those identified in this systematic scoping review. Individuals form beliefs, make decisions, and enact behavior based on what they perceive to be high-quality and relevant information . Ideas aAlthough the affordances of the internet as a health information resource are plentiful, locating high-quality health information on the internet can be challenging. The volume of health information on the web is massive and perpetually increasing; for example, a Google search for “sore throat treatment” returns nearly 2 billion results. Within this plethora of information, users’ search results are often clouded with misleading, inaccurate, or unsubstantiated information ; for exaThe construct of eHealth literacy was initially derived from 6 underlying foundational literacies: traditional literacy, information literacy, media literacy, health literacy, computer literacy, and scientific literacy . NorgaarConcurrent with their publication coining eHealth literacy, Norman and Skinner  publisher=0.29) relationship between self-reported ability and performance-based measurement. Several psychological mechanisms may contribute to the considerable gap in estimates of ability and actual ability, many of which have been supported by research across a variety of cognitive skills [Although self-report measurements have functioned as invaluable research tools in several settings, the efficacy of using self-report to evaluate one’s own skills, abilities, or proficiency in many contexts has been criticized for decades ,13. In te skills ; for exae skills , meaninge skills ,16. TheiRelating to web-based information seeking, ample literature indicates that people have a tendency to overestimate their ability with computers ,18 as weAlthough research syntheses have been undertaken related to eHealth literacy measurement instruments and their properties ,27, no sThe purposes of this study were 2-fold. The first purpose was to identify tools that currently exist to measure eHealth literacy based on objective performance. Within this purpose, we were interested in the design of these tools, their intended population, and whether their authors included some evidence of validity. The second purpose of this study was to characterize the prevalence of performance-based eHealth literacy measurement tools in the literature broadly compared with subjective eHealth literacy measurement tools.We conducted a systematic scoping review of the literature. A scoping review was deemed appropriate for the purpose of this study, given its broad focus on how research involving eHealth literacy measurement is conducted . This scIn the first stage, we devised an electronic search protocol that was completed by 2 researchers. We used the following search string, modified from the one used in a systematic review conducted by Karnoe and Kayser : OR ) AND We applied this string to search 6 relevant databases  selected in consultation with a university librarian. All searches were conducted in June 2021. Across these 6 databases, 1735 search results were obtained, which we reduced to 1444 (83.23%) unique articles after removing duplicates.In the second stage, we conducted 3 screening phases to narrow down the search results. In each screening phase, articles were sought that met the following inclusion criteria: (1) written in English; (2) published in peer-reviewed journals; and (3) created, revised, or used an eHealth literacy assessment tool  were retained for inclusion in this scoping review; 31 (10%) of these 311 articles contained a performance-based measure of eHealth literacy. In cases of disagreement between the first and second authors during the third screening phase, the third author was consulted, and discussion ensued until consensus was reached. Throughout the entire screening process, all authors only excluded articles that very evidently did not meet the inclusion criteria. At this point, the first author read the bibliographies of the articles containing a performance-based measure of eHealth literacy (n=31) and highlighted titles that could meet the inclusion criteria of this scoping review. After discarding studies that had already been considered in the screening process, new studies were selected (n=6) and read in full by the first and second authors. Of these 6 new studies, 2 (33%) were included in the scoping review, and both contained a performance-based measure of eHealth literacy. Thus, the total number of articles included in this scoping review is 313, of which 33 (10.5%) contain a performance-based measure of eHealth literacy. The scoping review process we undertook is summarized in In the third stage, we extracted data from each of the articles containing a performance-based measure and summarized them into a table. The first and second authors each reread the articles containing a performance-based measure of eHealth literacy (n=33) and input information from each into a comprehensive table devised by all 3 authors. This table was later simplified into appropriate headings, selected in the interest of providing a concise and relevant summary of the findings in this paper. In judging evidence of validity, we considered 3 relevant types of validity: ecological validity, criterion validity, and construct validity. Ecological validity describes the extent to which a measure represents or predicts behavior in real-world settings . In the Our scoping review identified 33 peer-reviewed studies using a tool to measure eHealth literacy that incorporated a performance-based aspect. Within these studies, there were just 2 measures used more than once. In total, 3 (9%) of the 33 articles used the eHealth literacy assessment toolkit (eHLA) created by Karnoe et al , and anoWe categorized the 29 unique performance-based measurement tools identified in this scoping review into 5 broad categories according to the structure of their main measurement technique. It should be noted that several of the measurement tools have components that fall into ≥2 of these categories; for instance, the Research Readiness Self-Assessment-health (RRSA-h) ,55 measuOne of the most common types of performance-based measurements identified in this scoping review involved participants answering health-related questions using the internet; these were questions for which responses could be scored according to correctness, completeness, and specificity. In the majority of the studies using this assessment method participants were allowed open access to the internet to inform their answers. An exception was the study by Freund et al  in whichcorrect (1 point) or incorrect (0 points); however, Agree et al [An advantage of eHealth literacy measurements where participants use the internet to answer health-related questions is that participants can be assessed without the need for researcher observation and video recording, except in cases where completion time is additionally factored into the scores. With these measurement tools, participants’ responses to questions were graded by ≥1 researchers according to a predetermined rubric. The majority of measurement tools in this category coded responses as simply ee et al  also exaee et al  gave paree et al  gauged pcorrectness that the participants exemplified throughout each simulated task; for example, in the study by van der Vaart et al [successful or unsuccessful, depending on whether they accomplished the end result of the task , whereas in the study conducted by Camiling [Another common type of performance-based eHealth literacy assessment identified in this scoping review involved web-based information–seeking simulations, where participants’ web-based behavior was observed  and assessed for proficiency by researchers. Proficiency was determined by assessing whether participants were able to complete a set of tasks; many researchers went further by also assessing the degree of efficiency and rt et al , particiCamiling , a rubriCamiling  viewed rCamiling  who had Health-related questions using the internet methods discussed previously.eHealth literacy assessments based on simulated internet tasks tended to be the most time-consuming for participants: the 15 simulation tasks in the study by Neter and Brainin  took appA third prominent type of performance-based eHealth literacy assessment identified in this scoping review involved participants evaluating the quality of health-related websites. In the studies conducted by Kalichman et al , Schulz Deviating from the structure of these studies but still related to evaluating website quality, Mitsuhashi  had partThe rating of the quality of real health-related websites by participants significant ecological validity in terms of participants’ knowledge of how to recognize signifiers of credible health information on the web; however, in contrast to the simulated internet task measurement tools, these tools do not require participants to form queries or extract information to apply to health-related problems. Although evaluating the credibility of a web-based source is undoubtedly a pivotal component of eHealth literacy, it does not encompass all components of eHealth literacy as defined by Norman and Skinner .Distinct from measurement tools where participants demonstrate eHealth literacy skills through task completion or by correctly answering health-related questions using the internet, other tools tested participants on their knowledge of procedural internet skills and information-seeking strategies to gauge their eHealth literacy. The most used tool in this category is the RRSA-h ,55. The In contrast to this measure that takes relatively long to complete, Hanna et al  used jusQuizzing participants on their knowledge related to eHealth literacy does not require recording or observing participants’ web-based activity, and nor does it require necessarily providing participants with internet access, which makes this method more accessible for some research settings in terms of resources and complexity than other tools mentioned previously. As participants are not actively carrying out web-based health information–seeking tasks with these measurement tools, the tools may not carry the same degree of ecological validity as those mentioned previously. However, basing eHealth literacy assessment on participants’ knowledge of proficient web-based information–seeking skills still avoids many of the pitfalls related to subjective self-report measures of eHealth literacy.We identified 2 measurement tools that examined participants’ baseline knowledge of health and their ability to apply that knowledge as an indicator of eHealth literacy. The most prominent tool is the eHLA , which iLiu et al  also creIn these 2 eHealth literacy measurement tools, there are no components that directly assess computer literacy in a performance-based manner nor do they contain performance-based components related to actively seeking health information . We have included them in this scoping review because the authors themselves define these as measures of eHealth literacy . However, judging their performance-based components solely by the definition of eHealth literacy proposed by Norman and Skinner , these mOf the 313 studies included in this scoping review, 33 (10.5%) used a performance-based measurement tool, and 280 (89.5%) used measures of eHealth literacy that only incorporated subjective or self-rating aspects. Furthermore, it is notable that 210 (67.1%) of the 313 studies reported using eHEALS in either its original form or translated into another language. Our findings indicate that current research on eHealth literacy has a strong tendency to rely on self-rated perceptions of eHealth literacy rather than assessing actual ability.The primary purpose of this scoping review was to identify and describe tools that measure eHealth literacy based on objective performance (as opposed to subjective self-rating). We identified 29 such measurement tools, of which only 2 had been used in >1 peer-reviewed study as of the date of our search. These measurement tools were the RRSA-h , which iThe second purpose of this scoping review was to characterize the prevalence of performance-based eHealth literacy measurement tools in the literature in contrast to subjective measurement tools. Our findings indicate that the vast majority of current research on eHealth literacy relies on self-rated perceptions of eHealth literacy rather than assessing actual ability. This is concerning, considering the limited utility of this short self-report measure for predicting performed eHealth literacy as indicated by a substantial body of literature ,22,24-26correct answers to health-related questions may need to be updated over time; for example, dietary guidelines have changed substantially in the past few decades because they have been updated based on our growing scientific knowledge [Another notable finding of this scoping review is that of the 29 unique eHealth literacy measurement tools with performance-based components identified, only 2 had been used in >1 peer-reviewed study at the time of this research. This comes in stark contrast to the prevalence of eHEALS, which was used by 210 (67.1%) of the 313 studies in various contexts and languages. This could be due in part to several of the studies  not including full versions of their performance-based instruments, making it challenging for other researchers to replicate these measures in other projects. Another implicit challenge to creating a performance-based eHealth literacy measure that may be adopted for widespread use is the changing state of scientific consensus on health-related topics, meaning that nowledge ,62. In anowledge  designedJudging these performance-based measurement tools by the definition of eHealth literacy proposed by Norman and Skinner , the tooOne glaring absence across most performance-based measures of eHealth literacy is any mention of social media. In discussing the modern utility of eHEALS, Norman  noted thThis study includes a few notable limitations. This scoping review only considered peer-reviewed journal articles published in English, which may have caused bias in our findings. Furthermore, it is possible that we missed including studies that evaluated skills under the umbrella of eHealth literacy but did not describe this using any of the terminology in our search protocol. We did locate studies using terms such as “health information evaluation skills”  and “infHealth-related questions using the internet, Simulated internet tasks, or Website evaluation tasks categories of this scoping review may offer ecologically valid options for assessing eHealth literacy. In contexts where providing such equipment may not be feasible, measures from the Knowledge of the web-based health information–seeking process category may offer a simpler performance-based eHealth literacy assessment tool that still reduces strict reliance on participants’ ability to gauge their own skill level.This is the first literature review that specifically identifies objective performance-based measurement tools of eHealth literacy in contrast to subjective self-report measurement tools. We identified 29 unique measurement tools of eHealth literacy with performance-based components used in various populations and covering various health-related topics. To better establish the utility and validity of performance-based measurement tools of eHealth literacy, scholars looking to incorporate performance-based measurement of eHealth literacy into their research should look to use and build from some of these existing measurement techniques rather than producing their own. In contexts where research participants can be provided a computer with internet access, measures from the In addition, to the best of our knowledge, this is the first literature review that quantifies the approximate prevalence of performance-based measurement versus subjective measurement of eHealth literacy in the literature broadly. In peer-reviewed scholarship, eHealth literacy assessment is predominantly conducted using subjective self-report measurement techniques. Performance-based measures of eHealth literacy are likely to provide a far better picture of how proficiently people find, evaluate, and use web-based health information. As such, researchers should consider using more objective measures of eHealth literacy such as those identified in this scoping review.