Evidence-based policies should underpin successful implementation of innovations within child health care. The EU-funded Models of Child Health Appraised project enabled research into effective methods to communicate research evidence. The objective of this study was to identify and categorize methods to communicate evidence-based research recommendations and means to tailor this to stakeholder audiences.We conducted an online survey among national stakeholders in child health. Analysis of the most effective strategies to communicate research evidence and reach the target audience was carried out in order to ensure implementation of optimal child health care models at a national level.Representatives of stakeholders from 21 of the then 30 EU MS and EEA countries responded to the questionnaire. Three main approaches in defining the strategies for effective communication of research recommendations were observed, namely: dissemination of information, involvement of stakeholders and active attitude towards change expressed in actions. The target audience for communicating recommendations was divided into two layers: proximal, which includes those who are remaining in close contact with the child, and distal, which contains those who are institutionally responsible for high quality of child health services. They should be recipients of evidence-based results communicated by different formats, such as scientific, administrative, popular and personal.Influential stakeholders impact the process of effective research dissemination and guide necessary actions to strengthen the process of effective communication of recommendations. Communication of evidence-based results should be targeted to each audience’s profile, both professional and non-professionals, by adjusting appropriate communication formats. Child oriented health policies are universally important. Moreover, one of the priorities of the Universal Health Coverage strategy of the World Health Organization (WHO) is Primary Health Care, which includes actions on improvement of maternal, newborn, child and adolescent health.Evidence-based policymaking has been defined as ‘a set of rules and institutional arrangements designed to encourage transparent and balanced use of evidence in public policymaking’,Implementation of evidence into new policies is seldom a stand-alone activity—policymakers frequently and wisely look to see what has been done in similar neighbouring countries, and a previous Horizon 2020 project—the Research Inventory of Child Health Europe—specifically focussed on cataloguing such evidence in Europe.There is also a distinction to be made between top-down policy implementation by instruction, and enthusiastic adoption of the practices at the delivery level, and this depends on making the underpinning evidence accessible and credible. This requires effective knowledge communication relevant to specific recipients.The goal of our research was to identify effective methods of communicating evidence to facilitate effective policy implementation, including identification of key audiences, drawing from practical experience in European Union (EU) and European Economic Area (EEA) countries.This study was part of the EU-funded MOCHA project, which intended to assess various models of primary child health care across Europe,The types of stakeholder and relevant topics of inquiry were developed from consultation with fellow MOCHA researchers who were asked to identify key elements for the future of primary child health care. They focussed on domains, such as prevention, mental health, chronic care and complex care.The stakeholder selection process was achieved via the MOCHA Country Agents (CAs), who were national experts from the study countries who were recruited for the project in order to provide country-specific information. CAs were asked to identify, and supply contact details of, at least three stakeholders in their country who would be willing to complete a questionnaire about three broad areas of primary care, and three broad age groups of children as users of primary care. We highlighted that these stakeholders might be policymakers, physicians, school health doctors, paediatricians, nurses or others, but they needed to be knowledgeable about the healthcare system in the country. We asked them to include at least one policymaker in the field of primary child health care on a national level. In addition, European Union for School and University Health and Medicine (EUSUHM) congress members provided the names of relevant national stakeholders in their countries. The stakeholders were asked to respond to the questionnaire based on their expert knowledge and experience and expertise, not their personal opinions.Stakeholders were asked to complete a digital questionnaire about communication modes to ensure implementation of evidence-based solutions in their countries:the most effective strategy for communicating recommendations, to ensure implementation of optimal models,the most effective target audience for promoting implementation of optimal models andthe most effective format for communicating policy evidence.In line with the MOCHA project’s established methodology, the questionnaire was designed by the topic researchers, approved by the project coordination team and validated by the project’s External Advisory Board, comprising members nominated by European medical, paediatric and policy bodies, WHO European Regional Office, UNICEF Innocenti Research Centre and civil society groups, as published.The data collection was carried out between March and May 2018. Out of the 30 EU/EEA countries, the MOCHA CAs and EUSUHM congress members of 22 countries provided names of 161 stakeholders.Our questions were open ended, and were analyzed by using thematic content analysis. The collected responses were coded by highlighting relevant parts of the answers. This facilitated further categorization, which led to emergence of umbrella themes characterizing strategies to implement evidence-based research recommendations and means to tailor this to the audiences. In order to identify, analyze and report patterns (themes) within the data, the approach proposed by Brown and ClarkeThe analytical process led to identification of clusters of strategies of effective communication of evidence-based data, format of recommendations and target audiences on which the stakeholders participating in the survey showed a convergence.The study was reviewed and approved by the ethical committee of the Faculty of Behavioural, Management and Social Sciences of the University of Twente under file number BCE17614, on 19 September 2017.In total, 99 (61.5%) of 161 nominated stakeholders started the questionnaire, 90 (55.9%) completed it—they were from 21 countries comprising all EU Member States except Belgium, Cyprus, Estonia, France, Lithuania, Luxembourg, Malta, Slovenia and the UK, plus Norway and Iceland from the EEA .Most respondents were experts in prevention of communicable diseases (vaccination as a tracer), and recognition of mental health problems in adolescents. The least numerous group was experts in treatment and monitoring of a chronic condition .A total of 62 out of 90 respondents answered the questions about most effective strategy, target audience and format for communicating policy recommendations. They represented three types of MOCHA Primary Health Care system as identified by the MOCHA project: GP-lead (31.2%), Paediatrician-lead (33.3%) and Combined (31.7%) along with others (4.8%) . Some ofWe identified three over-arching approaches to effective communication of policies :influential stakeholders’ impact on communication processes regarding evidence-based recommendations for policies (36.2% responses),dissemination of information in order to provide effective communication of evidence-based recommendations for policies (42.6% responses) andnecessary actions in order to strengthen the process of effective communication of evidence-based recommendations for policies (21.3% responses).Consistently, dissemination played the most important role in each of the three groups of countries classified by system type .The most influential stakeholders in disseminating the recommendations influencing optimal models of child health were authorities and policymakers, as those who are also responsible for further adoption and implementation of innovations. The significant role of health professionals and other associations  was indicated as well . The GerStrategic tools for dissemination of information were identified regarding ‘hard’ law  and ‘soft’ recommendations .A Spanish respondent claimed that ‘legislation, policies, standards, advice and guidance are necessary to provide the framework for addressing critical issues such as the provision of care of high quality, the improvement of access to care, the protection of rights’ . It was stressed that strategies should be ‘suited to the target audience's profile’ (respondent 19), while media often determine what is visible for the public and politicians . The media has power and can lead to the mobilization of societal action that creates the conditions and place for health issues on the national public agenda and can catalyse action at the national and local levels .In the respondents’ opinion actions should be based on the implementation of long-term strategies or legislative changes, with the involvement of users at professional and non-professional level. Promoting the model by spreading a positive message is key in the process of increasing awareness .The regular renewal of the existing action plan and program of health care measures was said to be important . In order to obtain the broad scope of the recipients who are aware of new child healthcare evidence recommendations, discussion among stakeholders about pros and cons of a new model and cost-benefit analysis of this model is recommended . Importance was also given to the meeting and personal encounters with authorities directly responsible for child health services . The optimal strategy should be ‘through well planned, sufficiently funded implementation work that targets service providers directly with content that appears to be useful in their everyday work’ , emphasizing the targeting to the particular needs of each stakeholder in their work context. The Austrian expert in problem recognition and early diagnosis proposed a strategy, which was education based, with activities oriented to those who are working with the child in the field, to children and to parents .In order to identify the most appropriate recipients who should be informed about the development of a new model, we asked respondents to identify the most effective target audience for communicating recommendations, to ensure successful implementation of optimal models in their countries. Experts recognized the significant importance of both patients and their environment at micro level as well as decision/policymakers, and professional associations and organizations at macro level. Many of them stressed that both the format of the recommendations, as well as the strategy, should be suited to the target audience.Observing the data, we divided the reported target audience for communicating recommendations into two layers:audiences in the proximal environment of the child/patient (42.2% responses) andaudiences in the distal environment of the child/patient (57.8% responses).We noticed that experts from all three groups of MOCHA systems were choosing the distal audience as most relevant .The proximal target audience consists of children/patients, families, parents, people supporting parents, self-help groups, health care workers, teachers and health professionals . This grThe importance of health professionals who have the power to change the system was stressed . The Spanish respondent highlighted that parents should be informed by primary care professionals about new evidence-based solutions .The audience of the distal environment of the child includes: decision makers, professional organizations and associations, politicians, child advocacy groups, patient associations, administration—civil servants, health insurances, governmental institutions, opinion leaders, authorities , knowledge centres, general public/service users, journalists and health mediators . This grThe Latvian stakeholder stressed that ‘politicians in particular in municipalities do not have a high level of health literacy, so they are definitely one target audience’ . Also, governmental institutions should be included in the group of primary target audiences . Policymakers and other stakeholders need to have the expertise to examine state-level data and differentiate specific risk sub-populations .Based on the data collected, we could observe that there are four types of effective formats of communication :scientific format (33.1% responses),administrative format (27.4% responses),popular format (28.3% responses),personal format (13.2% responses).On the one side, the scientific approach is still relevant, popular and expected , togetheWe also observed that the countries that are representing the combined and paediatrician-lead MOCHA system were mostly choosing the administrative format of recommendation as most relevant whereas GP-lead countries were preferring the scientific format. However, the differences were minimal .The answers given by the respondents confirm that the format of advice ‘should be suited to the target audience's profile, either individual or priority groups, i.e. peer-reviewed journal and/or seminar for stakeholders and professionals’ (respondent 19). The Norwegian expert highlighted that ‘reports, scientific publications, seminars and news items are either useless or make a temporary change. The format must appear useful for the person receiving it, and it must be followed up regularly to ensure actual implementation’ . A Croatian respondent stressed that ‘health professionals will like a peer-reviewed journal, politicians and decision makers would prefer EU report, and parents will react to the popular media’ .The most relevant and effective format for patients is media because ‘patients should know what is possible and might be better’ . Public discussions with doctors should be facilitated by famous persons/celebrities (particularly in terms of immunization), supported by educational shots in media, whereas scientific publications should be directed to medical professionals . Reports published in mass media and social networks, as well as innovative approaches, technology-based, and peer-led approaches, may increase awareness amongst patients and also general population .A Latvian expert stressed that ‘evidence-based scientific publication in a peer-reviewed journal is good for scientists and writers but not for wider society’. He suggested that permanent and positive information in popular media and advocacy from the authorities could have the greatest benefit . Others claimed that strong scientific evidence should be disseminated by social media .It is important to recognize that evidence-based policy, in order to be effective, needs to rely on appropriate strategies of dissemination of scientific results. Based on the analysis of stakeholders’ views, we characterized the strategies to communicate evidence-based research recommendations and means to tailor this to the audiences and we set it in the wider context of the recognized policy cycleIn our study, we identified three essential aspects that need to be taken into account while planning introduction of evidence-based innovation (model).,Firstly, stakeholders representing medical and patient environment should be considered as crucial component in the process of the dissemination of information. It is compatible with a definition of a stakeholder as ‘any group or individual who can affect or is affected by the developed (…) system’.Disseminated information about innovative solutions can take various forms and go through various channels. In particular media impact was highlighted, and that is consistent with the opinion of Van de Goor et al.These findings support communicating evidence in a way that appeals to specific complementary audiences. They make the difference between mechanical acceptance of a policy and enthusiastic adoption and successful implementation. Effective dissemination requires active circulation of the evidence/research and leads to positive local innovation. It fits the view of Greenhalgh et al.Successful transfer of the innovation requires tailoring the message to appropriate audiences. We identified two layers of the target audience; the proximal audience include those who are remaining in close contact with the child and who indirectly influence the policymaking processes, and the distal audience of those who are institutionally responsible child health services or play an advocacy role. This division is compatible with a classification identified by the MOCHA project where two groups of children’s agents were identified, agents of proximal and distal child environment.Eventually, this study presented a cascade view of the most effective format of recommendations  that is funded by the European Commission through the Horizon 2020 Framework (grant agreement number: 634201).None declared.The MOCHA data contains no patient information, but may contain other personal or institutional data, such as source of a commentary. The MOCHA project has therefore resolved that source data will be curated on the MOCHA web site, and be accessible via the Principal or other Partners through a curator function, so that data relevant to any enquiry can be supplied, and redaction effected, but also contextualization given.There are three main approaches in defining the strategies of effective communication of recommendations, namely: dissemination of information, stakeholder impact and actions.Stakeholders are communicating the information in order to undertake significant national actions.The format of recommendations, as well as the strategy, should be tailored to the target audience.The audience of proximal environment includes those who are remaining in close  contact with the child whereas the audience of distal environment contains those who are institutionally responsible for high quality of child health services in their countries.The baseline for the process of circulating the information is the evidence-based data of a scientific character as it is the main source of the administrative recommendations and reports, which later appear in broadly understood media, the channel, which reaches the recipient at personal level.ckaa234_Supplementary_DataClick here for additional data file.


A new CaCO3 RM, USGS44, has been prepared to alleviate this situation.The stable carbon isotopic  isotope‐ratio mass spectrometry (IRMS) of NBS 19 CaCO3 (δ13CVPDB = +1.95 milliurey (mUr) exactly, where mUr = 0.001 = 1‰), and LSVEC Li2CO3 (δ13CVPDB = −46.6 mUr exactly), and normalized to the two‐anchor δ13CVPDB‐LSVEC isotope‐delta scale. The second set of values was obtained by dual‐inlet (DI)‐IRMS of CO2 evolved by reaction of H3PO4 with carbonates, corrected for cross contamination, and normalized to the single‐anchor δ13CVPDB scale.USGS44 was prepared from 8 kg of Merck high‐purity CaCOδ13CVPDB‐LSVEC value of −42.21 ± 0.05 mUr. Single‐anchor δ13CVPDB values of −42.08 ± 0.01 and −41.99 ± 0.02 mUr were determined by DI‐IRMS with corrections for cross contamination.USGS44 is stable and isotopically homogeneous to within 0.02 mUr in 100‐μg amounts. It has a δ13CVPDB‐LSVEC value of −42.21 ± 0.05 mUr for both EA/IRMS and DI‐IRMS measurements. As a carbonate relatively depleted in 13C, it is intended for daily use as a secondary isotopic reference material to normalize stable carbon isotope delta measurements to the δ13CVPDB‐LSVEC scale. It is useful in quantifying drift with time, determining mass‐dependent isotopic fractionation (linearity correction), and adjusting isotope‐ratio‐scale contraction. Due to its fine grain size , it is not suitable as a δ18O reference material. A δ13CVPDB‐LSVEC value of −29.99 ± 0.05 mUr was determined for NBS 22 oil.The new high‐purity, well‐homogenized calcium carbonate isotopic reference material USGS44 is stable and has a  To achieve high‐quality δ13C analysis, isotopic reference materials (RMs) are required. In the past several decades, the international isotopic RMs NBS 18, NBS 19, NBS 22, LSVEC, IAEA‐CO‐1, IAEA‐CO‐8, IAEA‐CO‐9, and IAEA‐603 have been gradually introduced to the isotope community and used for the determination of δ13C values of carbon‐bearing materials.δ13C scale be established with NBS 19 carbonate assigned the value of +1.95 milliurey (mUr) exactly as its single anchor, where 1 mUr = 0.001 = 1‰.δ13C measurements.δ2H and δ18O scales substantially improved agreement among laboratories,δ13C scale. Based on high‐precision isotope‐ratio mass spectrometry (IRMS),δ13C values of all carbon‐bearing materials should be measured and expressed relative to VPDB on a scale normalized by assigning consensus values of −46.6 mUr to LSVEC lithium carbonate and +1.95 mUr to NBS 19 calcium carbonate.δ13C RMs significantly compared with previously assessed uncertainties, as demonstrated in Figure 1 of Coplen et al.δ13C values of most new secondary RMs for forensic, environmental, paleontological, and atmospheric applicationsδ13C values of RMs on the NBS 19‐LSVEC scale.High accuracy measurements of stable carbon isotope ratios  due to contamination with atmospheric CO2, and a similar observation was made of the LSVEC material stored at the National Institute of Standards and Technology .δ13C RM, particularly as a scale anchor, and IUPAC has advised against its use as a δ13C RM.δ13C instability of LSVEC demonstrates the need to develop and characterize a new secondary isotopic RM with the potential to replace LSVEC as the second anchor of the VPDB scale.In 2015, careful laboratory analyses performed at the IAEA, Seibersdorf, Austria, demonstrated that the δ13C RM. This calcium carbonate RM is named USGS44. Due to its fine grain size (<63 μm), it is not suitable as a δ18O RM because its oxygen can exchange with atmospheric water, changing its δ18O value. Nevertheless, δ18O data are reported herein because (1) users may be interested in a nominal δ18O value, (2) these data demonstrate the high precision that can be achieved by dual‐inlet IRMS, and (3) these data support the isotopic homogeneity of USGS44. The assessments of δ13C values of the material were performed by the RSIL, the BGC‐IsoLab, the Centre for Isotope Research University of Groningen, Groningen, Netherlands (CIO), and the Centre de recherche, Geotop, Université du Québec à Montréal, Canada.The Reston Stable Isotope Laboratory  at the U.S. Geological Survey (USGS) and the stable isotope laboratory at the Max Planck Institute for Biogeochemistry, Jena, Germany (BGC‐IsoLab) surveyed many commercial calcium carbonate reagents and identified one material from Merck that could serve as a secondary 22.1Sixteen bottles of high‐purity calcium carbonate powder with a total mass of 8 kg were purchased from Merck . To ensure isotopic homogeneity of the RM, the following steps were carried out  technique. The observation has not found a satisfactory explanation so far and warrants further experimental investigations. NBS 22 oil, which was anchored to LSVEC, is used as an anchor in this study.δ13C values of USGS44 directly on an EA connected to an isotope‐ratio mass spectrometer following the principle of identical treatment.The analytical methods used to measure 2.3δ13C analysis are similar to the procedures and techniques used previously for determination of δ13C values of secondary δ13C RMs.δ13CVPDB values of USGS44 were normalized to NBS 19 calcium carbonate (δ13C = +1.95 mUr exactly) and LSVEC (δ13C = −46.6 mUr exactly).At the RSIL, the methods used for online δ13C EA‐IRMS analyses largely followed the described procedures and techniques from previous publications.plus isotope‐ratio mass spectrometer (Thermo Fisher Scientific) coupled to a 1100 CE EA analyzer  via a ConFlo III open‐split interface (Thermo Fisher Sceintific). In most cases, the samples were analyzed in dilution mode to reduce systematic errors associated with blanks. Measurement sequences and post measurement blank, linearity, and drift corrections were performed according to Werner and Brand.δ13CVPDB values of USGS44 were normalized by assignment of IAEA‐603 calcium carbonate δ13C = +2.46 mUrδ13C = +1.95 mUr) and LSVEC δ13C = −46.60 mUr.At BGC‐IsoLab, the measurement procedures for 3 were weighed into tin cups to obtain the same amount of CO2 for all samples and RMs, hence eliminating potential linearity issues. The samples were then analyzed with an Isoprime 100 isotope‐ratio mass spectrometer  coupled to a Vario MicroCube elemental analyzer  in continuous‐flow mode. The samples were analyzed in dilution mode. Blank corrections were performed according to Werner and Brand.δ13CVPDB values of USGS44 were normalized by assignment of NBS 19 calcium carbonate δ13C = +1.95 mUr and LSVEC δ13C = −46.6 mUr exactly. All reference materials were stored under vacuum.At Geotop, 2.8 ± 0.1 mg of CaCO2.4η (eta), and it is a property of a specific dual‐inlet isotope‐ratio mass spectrometer during a specified time.δ13CVPDB = +2.46 mUr and δ18OVPDB of solid IAEA‐603 = −2.37 mUr.Cross contamination between the fraction of reference gas that contaminates the sample, and vice versa, must be accounted for during dual‐inlet measurements. The symbol of the dimensionless quantity used to express cross contamination is 2 from the USGS44, IAEA‐603, and NBS 19 calcium carbonates using its “Acid Reaction and Mixing System” (ARAMIS). We refer the reader to the relevant publications for details on ARAMIS and the reaction procedure.2 was evolved from four aliquots of USGS44 (and other RMs) at a reaction temperature of 25 ± 0.1°C, frozen into 300‐mL sample vials, and analyzed on a MAT 253 isotope‐ratio mass spectrometer (Thermo Fisher Scientific) equipped with a dual‐inlet system. Because no second‐scale anchor is available for such measurements, it is paramount to avoid scale contraction effects. Thus, ion source settings were chosen to minimize the value of η, and idle time experiments were conducted in 2018 and 2019 to evaluate η and its stability over time 17O correction. Offline data evaluation included daily drift correction and normalization to the δ13CVPDB scale on the MAT 252 IRMS measurements. In addition to those corrections, an offline η correction was necessary for the MAT 253 measurements.BGC‐IsoLab evolved CO2 evolved by treatment of USGS44, NBS 19, and IAEA‐603 calcium carbonates with high‐purity phosphoric acid at 25.0 ± 0.1°C was analyzed with a dual‐inlet MM10 isotope‐ratio mass spectrometer . Measurements and determination of the value of η were performed as described in Meijer et alη. Offline data evaluation included daily drift correction and normalization to the δ13CVPDB scale. IUPAC‐recommended 17O correction parameters 17O correction. A varying number of samples of USGS44 were extracted during the same time period as at BGC‐IsoLab . Each separate acid reaction corresponds to one sample. In addition, the 2017 and 2018 sample sets contained LSVEC, and the 2018 and 2019 sets contained IAEA‐603. All samples were calibrated using CO2 evolved from NBS 19 or IAEA‐603, of which the same number of aliquots were produced and measured.At CIO, CO2.53 were weighed and loaded into 12‐mL round‐bottomed borosilicate vials  capped with Labco butyl rubber septa. The vials were flushed for 10 min on an in‐house multiple‐port purging line with grade 5.0 helium at a flow rate of 55 mL/min so that no air contamination was observed during our experiments. Ten drops of 102% H3PO4 were injected into each vial to react with the calcium carbonate. The Exetainers were placed in the aluminum block of the GasBench II and heated to 25 ± 0.1°C overnight to ensure quantitative conversion into CO2.At the RSIL, a GasBench II gas preparation and introduction system  – equipped with a PAL autosampler  coupled to a ConFlo IV interface and a Delta XP isotope‐ratio mass spectrometer (both Thermo Fisher Scientific) – was used to check the isotopic homogeneity of USGS44 at microgram masses. The method used was modified from Breitenbach and Bernasconi.ρ = 1.92 g/cm3) were delivered under vacuum. The resulting CO2 was trapped in a cold finger at −180°C (liquid nitrogen) for 15 min. A water trap (−70°C) was used to condense any moisture between the vial and the cold finger. The “dry” CO2 was then heated at −60°C and “focused” in a second cold finger at −160°C for 5 min. The resulting gas was released in a fixed volume and the pressure of the “monitoring gas” was equilibrated with that of the sample. The “monitoring gas” is a Jackson Dome CO2 with an approximate δ13C value of −3 mUr. Because the original design of the measurement sequence included LSVEC, no cross‐contamination test was performed. IUPAC‐recommended 17O correction parameters 17O correction.At Geotop, between 100 and 120 μg of NBS 19, IAEA‐603, LSVEC, and USGS44 were weighed into glass micro crucibles. The samples were then transferred into glass conical‐bottomed vials closed with septum caps. The samples were inserted in a 90°C heated rack. After a minimum of 1 h of heating, samples were analyzed with an Isoprime isotope‐ratio mass spectrometer coupled to a Isoprime MultiCarb preparation system in dual‐inlet mode. For each sample, three drops of ortho‐phosphoric acid  were used to weigh NBS 22 oil. Prior to weighing, the glass filters were baked at 475°C for 2 h. The baked filters were cut into 1.5 × 1.5‐mm pieces. A small piece of filter was placed on an unfolded 5 × 3.5‐mm tin capsule. A clean, thin stainless‐steel wire was used to transfer a tiny drop of oil onto the glass filter. The weighed oil on the filter was wrapped into a tin capsule.33.1δ13C using two different EAs and a GasBench at the RSIL and a MultiCarb system at Geotop with different masses of USGS44 as specified above. At the RSIL, an IsoLink EA was used, and three aliquots containing 84 μg of carbon  from each of 27 fractions were analyzed to confirm δ13C homogeneity. Homogeneity tests were carried out in four separate analytical sequences. The first fraction of USGS44 was designated as a quality control (QC) sample and was analyzed at the beginning, middle, and end of each of the four sequences. The measured δ13C values along with associated 1‐σ standard deviations of USGS44 are summarized in Table δ13C values in this table were normalized to the QC sample by assigning it a δ13C value of −42.21 mUr. The uncertainty from nine bottles of USGS44 was 0.03 mUr, which indicates that the USGS44 is well homogenized at amounts of 0.70 mg. The overall standard deviation from 78 individual analyses is 0.05 mUr. In a second RSIL test, a Costech EA, as specified above, was used. One aliquot containing 12 μg carbon  from each of 27 fractions was analyzed. In this sequence, NBS 19 was analyzed at the beginning, middle, and end as a control sample, and IAEA‐603 was also analyzed. The measured δ13C values of the 0.10‐mg samples of USGS44 are presented in Table δ13C value of −40.08 mUr from USGS44 in Table At the RSIL, the homogenized USGS44 was divided and stored in nine glass containers. Three aliquots of USGS44 were sampled from the top, middle, and bottom of each container, making a total of 27 samples for isotopic homogeneity evaluation. The homogeneity test was carried out by measuring δ13C analysis to normalize small samples, such as in the analysis of foraminiferaδ13C analyses,δ13C value of −41.94 mUr from USGS44 in Table Considering the need for high‐precision 2  were introduced into sample vials when purging the samples, which affects the δ13C value of USGS44 (δ13C = −42.21 mUr) more than that of NBS 19 (+1.95 mUr).The isotopic homogeneity of USGS44 was also evaluated in comparison with NBS 19 using the data from routine sample analysis with the GasBench at the RSIL. Figure 3.2δ13C value does not change when the material is exposed to a humid environment,2 equilibration test3 samples (with different lot numbers) were selected for this test. One was a 2‐g vial of CaCO3, and another was a large bottle containing 500 g of CaCO3, which was the candidate material for USGS44. Three aliquots of about 1‐g of each material were loaded into an 8‐L glass desiccator. For comparison, a set of LSVEC samples was also placed in the CO2 equilibration desiccator along with CaCO3. A vial of water was also placed inside the desiccator to ensure a humid environment. The desiccator was evacuated and approximately 300 μmol of CO2 (δ13C = −4 mUr) was introduced into the desiccator. After 7 days at ambient temperature, the samples were removed from the desiccator and dried in a vacuum oven at 40°C for 5 h. Comparison measurements between original samples and samples that had been equilibrated with CO2 were made in the same analytical sequence. The measured δ13C values are shown in Table 2 and its δ13C value increased by 1.01 mUr. This confirms previous observations that the δ13C value of LSVEC is not stable.2 and the two Merck CaCO3 materials, which demonstrates that USGS44 is stable and acceptable for use as a δ13C RM.To ensure that USGS44 calcium carbonate is a stable material and that its 3.3l‐glutamic acid. Six to eight 5 × 3.5‐mm tin capsules were folded together to act as one sample to produce a substantial CO2 peak so that the carbon blank and the δ13C value of the blank could be determined accurately. A δ13CVPDB value of −26.0 mUr, normalized to USGS40, was obtained for the blank of the tin capsules. The carbon blank in each capsule was about 1 μg. The carbon blank is thought to be a byproduct of mineral oil used in the production of the tin capsules, causing the blanks to be similar within the same batch of capsules. Using the same method, six 1.5 × 1.5‐mm baked glass filters were combined to act as one sample. The CO2 peak from the glass filters was too small to integrate as a peak. Therefore, the δ13C values were only corrected for the carbon blank introduced from use of tin capsules. A similar carbon blank evaluation was carried out at BGC‐IsoLab. The smaller, but heavier smooth‐wall tin capsules for liquid samples, and the larger, but lighter, tin capsules for standard solid samples were investigated by combining 10 tin capsules into one larger sample and burning that to obtain a satisfactorily sized blank peak. In both cases a value of approximately −27 mUr was obtained for the δ13CVPDB of the carbon blank. The amount of blank was so small that the blank‐corrected USGS44 value deviated by less than 0.02 mUr.At the RSIL, the carbon blanks from both the tin capsule and the glass filter were carefully evaluated against USGS40 3.4δ13CVPDB value of UGSS44 is traceable to the VPDB scale, the primary stable carbon isotopic RMs NBS 19 (δ13CVPDB = +1.95 mUr exactly)δ13CVPDB = +2.46 ± 0.01 mUr)δ13CVPDB value of USGS44, NBS 22 oil was used as the second anchor point with an assumed δ13CVPDB value of −30.03 mUr. Because oil and calcite are different chemical matrices, the quantitative conversion of carbon from these two materials was thoroughly investigated. The quantitative conversion of LSVEC Li2CO3 had been evaluated in the study of the determination of the δ13CVPDB value of USGS41aδ13C values from NBS 19, NBS 22, and USGS44, and the normalized δ13CVPDB values are summarized in Table δ13CVPDB results to those weighed in tin capsules as long as the sample amounts were identical, which also indicates that the carbon blank from tiny pieces of glass filter used for oil weighing is negligible. The sample amount does not affect the final δ13CVPDB values of USGS44 if all samples and standards contain the same amount of carbon, thereby minimizing linearity issues.To ensure that the determination of the δ13C values of NBS 22 and USGS44 drifted in the same direction, 0.19 mUr and 0.24 mUr, respectively, when the sample mass was changed from 0.85 mg to 4.6 mg . Dataset 2, the δ13CVPDB value of USGS44 was obtained directly against LSVEC with a δ13CVPDB value of −46.6 mUr exactly . The computation of the final δ13CVPDB value and uncertainty presented in Table δ13CVPDB‐LSVEC values of the three laboratories were combined to obtain a consensus using a multivariate Gaussian meta‐analysis modelδ13CVPDB‐LSVEC value of USGS44 is −42.210 with a combined standard uncertainty of 0.048 mUr and a 95% uncertainty interval of . A consistent average value was observed from both two‐point normalization and one‐point normalization. This indicates that the scale correction is insignificant (<0.01 mUr), whether using two‐point normalization or one‐point normalization when the unknown sample (USGS44) has a δ13C value very close to that of the anchor RM (LSVEC). The high‐precision measurements carried out by three laboratories yielded a consistent difference in the δ13CVPDB value of −4.390 ± 0.071 mUr between LSVEC and USGS44 from data in Table δ13CVPDB‐LSVEC values that were obtained from analysis of USGS44, NBS 19, and NBS 22 by EA/IRMS and normalized to a NBS 22 δ13CVPDB value of −30.03 mUr . Surprisingly, the average δ13CVPDB value of −42.268 ± 0.069 mUr for USGS44 from column 5 of Table δ13CVPDB value = −30.03 mUr) was used as an anchor point, does not agree with the value of −42.210 ± 0.048 mUr from column 4 of Table δ13CVPDB, a difference of 0.058 mUr between different runs is acceptable, based on 0.06 mUr acceptance criterion for repeatability of δ13C measurements of modern IRMS instruments. However, for this work, substantial effort was made to achieve a higher precision. This 0.058 mUr discrepancy deserves some discussion: (1) Appropriate choice of RMs is crucial as discussed by Meier‐Augenstein and Schimmelmann.δ value range covered by the RMs, the less accurate the resulting normalized measurements of samples with δ values outside that bracket. However, we do not believe that this is the case in this study because stringent carbon blank corrections were applied in all measurements in the three laboratories, and the quantitative conversion was carefully evaluated, and these are the two most dominant factors that contribute to less accurate normalized δ values outside that RM's bracket, (2) Could the issue be either that the value of −30.03 mUr for NBS 22 was incorrect in 2006 or that the LSVEC used in that work had been compromised? If the latter were true, the USGS44 value from Table δ13CVPDB‐LSVEC value of NBS 22 is –29.988 mUr with a combined standard uncertainty of 0.054 mUr and a 95% uncertainty interval of . The computation of the final δ13CVPDB‐LSVEC value of NBS 22 and the uncertainty presented in Table Three sets of EA data were produced in this study. Dataset 1: the δ18O value of the reference injection gas in Isodat arose during the project. Does it make any difference whether the reference injection gas is assigned as 0 mUr or +23 mUr (or −23 mUr)? We confirmed that as long as one normalizes the δ13C measurements with two anchors, the impact upon the normalized δ13CVPDB value of the assigned δ18O of reference injection CO2 is insignificant (<0.01 mUr).A concern about the impact of incorrectly assigning the 3.6δ13CVPDB determination of USGS44 should have been carried out with two‐point normalization. However, a second scale anchor that is independent of LSVEC currently does not exist. To overcome this deficiency, the best method to obtain an accurate δ13C measurement with only one RM, NBS 19 or IAEA‐603, is that described by Meijer et al2 from four aliquots of USGS44 was extracted during three time periods between August 2016 and April 2018  is less than 0.006 mUr. The USGS44 CO2 gases produced in 2016 and 2017 were analyzed on the MAT 252 isotope‐ratio mass spectrometer at BGC‐IsoLab that is known to have minimal cross contaminationη correction on the MAT 253 system. Standardization was achieved using the eight CO2 gases evolved from NBS 19 syntheses from 2016. The average δ13CVPDB and δ18OVPDB values and standard deviations of the MAT252 measurements were −42.08 ± 0.01 mUr and −15.75 ± 0.07 mUr, respectively. Within analytical uncertainty, the MAT 253 and MAT 252 isotope‐ratio mass spectrometers produce identical δ13Cvalues for CO2 evolved from USGS44, thus supporting our contention that the applied corrections to the MAT 253 measurements are valid.Ideally, an accurate 18 Table . A total2 was evolved from three USGS44 samples in 2016, four in 2017, eight in 2018, and ten in 2019  truly represents a milliurey, but we are confronted with a difference that we deem beyond our estimated scale realization uncertainty. In Figure The cross‐contamination‐corrected, mean‐weighted ed Table . Rather,19 Table . At BGC‐3.7δ13CVPDB value of −42.21 ± 0.05 mUr and DI‐IRMS values of −42.08 ± 0.01 mUr from BGC‐IsoLab and −41.99 ± 0.02 mUr from CIO. This relatively large 0.13‐mUr (or 0.22‐mUr) difference between the two techniques, and the difference of 0.09 mUr within DI‐IRMS values, merit discussion. We identify three possible causes for the observed difference between the EA/ and DI‐IRMS measurement results: (a) the EA/IRMS measurements are faulty, (b) the DI‐IRMS measurements are problematic, or (c) the scaling of the δ13CVPDB‐LSVEC scale is incorrect. The EA/IRMS measurements were conducted at three different laboratories using different setups and yet provide USGS44 δ13CVPDB‐LSVEC values with a 95% uncertainty interval of 0.05 mUr  in columns 4 and 5 with previous measurements. The normalized δ13CVPDB values of NBS 22 and LSVEC using the BGC‐IsoLab value of −42.08 mUr for USGS44 are −29.90 mUr and −46.46 mUr, respectively. These two values are in excellent agreement with the δ13CVPDB value of Qi et alδ13CVPDB value of −46.48 mUr. Likewise, the normalized δ13CVPDB value of −26.23 mUr of USGS40 l‐glutamic acid with the BGC‐IsoLab USGS44 value agree very well with the value of −26.24 mUr of USGS40 from Qi et al.δ13CVPDB values of −41.99 mUr for USGS44, values of −29.83 mUr and −46.36 mUr were obtained for NBS 22 and LSVEC, respectively. These values are in fair agreement with the δ13CVPDB value of Qi et alδ13CVPDB value of LSVEC of −46.48 mUr. The normalized value of −46.36 mUr for LSVEC is between the δ13CVPDB values of LSVEC (−46.25 and −46.84 mUr) reported by Verkouteren and Klinedinst.δ13CVPDB values of USGS44 between the DI‐IRMS and EA measurements suggests that: (a) the EA/IRMS measurements are not faulty and (b) the discrepancy in δ13CVPDB values between the EA and DI‐IRMS measurements is caused by scaling due to an incorrect value of −46.6 mUr assigned to LSVEC. If a δ13CVPDB value of −46.46 mUr , with this scale realized using secondary RMs such as NBS 22 or USGS44, in an identical fashion to realization of the LSVEC is replaced as the second anchor by another RM with adoption of a new VPDB_202X scale.δ13CVPDB scale has been accepted by the CIAAW and IAEA experts' panel.For now, we recommend the continued use of the VPDB‐LSVEC scale until USGS44 or another suitable second scale anchor for the k) of 2. Therefore, it is unadvisable to combine them to recommend an average or weighted average DI‐IRMS δ13CVPDB value for USGS44. Additional DI‐IRMS δ13CVPDB measurements are needed to solve this conundrum. With the knowledge that we have now, the value of −42.08 mUr for USGS44 is preferred because: (1) the δ13CVPDB value of −42.08 mUr from BGC‐IsoLab was determined on two different instruments over a period of several years that both provided the same value, and its uncertainty of 0.01 mUr is better than that of −41.99 ± 0.02 mUr from CIO; (2) the re‐normalized δ13CVPDB value of −46.46 mUr for LSVEC using the value of −42.08 from BGC‐Isolab agrees well with the value in Table δ13CVPDB value of Qi et alδ13CVPDB value of LSVEC of −46.48 mUr. Likewise, if one re‐normalizes the NBS 22 value of −29.99 ± 0.07 mUr published in Table 2 of Qi et alδ13CVPDB value of LSVEC of −46.48 mUr, the re‐normalized value is −29.91 ± 0.07 mUr, which is in excellent agreement with the re‐normalized value of −29.90 ± 0.05 mUr.The two high‐accuracy DI‐IRMS measurements reported in this study by BGC‐IsoLab of −42.08 ± 0.01 mUr (combined standard uncertainty) and CIO of −41.99 ± 0.02 mUr (combined standard uncertainty) do not agree even by expanding their uncertainties with a coverage factor  obtained by EA/IRMS, a value that is recommended for the δ13C measurements both by EA/IRMS and by DI‐IRMS. A δ13CVPDB‐LSVEC value of −29.99 ± 0.05 mUr of NBS 22 oil is recommended. Both the above δ13CVPDB‐LSVEC values are henceforth the recommended values of these secondary isotopic reference materials on the VPDB‐LSVEC scale having an assigned δ13C value for NBS 19 of +1.95 mUr exactly and a δ13C value of −46.6 mUr exactly for LSVEC. A single‐anchor DI‐IRMS measurement of USGS44 yielded a δ13CVPDB value of −42.08 ± 0.01 mUr (combined standard uncertainty) as measured on two different mass spectrometers over an interval of several years by the Max Planck Institute for Biogeochemistry, Jena, Germany (BGC‐IsoLab). A value of −41.99 ± 0.02 mUr (combined standard uncertainty) was measured by the Centre for Isotope Research (CIO), University of Groningen, Groningen, The Netherlands. These high accuracy DI‐IRMS measurements are not in agreement even with a coverage factor (k) of 2. Additional investigations are needed to solve this conundrum. Therefore, we recommend continued use of the two‐anchor VPDB‐LSVEC scale, but with realization of the δ13CVPDB‐LSVEC scale using RMs other than LSVEC Li2CO3, whose use is to be deprecated for δ13C measurements. The discrepancy in δ13CVPDB values between EA measurement and DI‐IRMS is caused by scaling due to an incorrect value of −46.6 mUr assigned to LSVEC. If a δ13CVPDB value of −46.46 mUr were used for normalizing EA/IRMS data, a δ13CVPDB‐LSVEC value of −42.08 mUr for USGS44 would have been obtained, identical to the BGC‐IsoLab DI‐IRMS value. If the two‐anchor δ13CVPDB‐LSVEC scale is replaced by another scale, we recommend a δ13CVPDB value of −42.08 mUr for USGS44.The new high‐purity, well‐homogenized calcium carbonate δ13C measurements can achieve high precision and high accuracy and EA values are comparable with the measurements by DI‐IRMS if two‐point normalization is applied using two appropriate reference materials with correctly assigned δ13CVPDB‐LSVEC values. As a carbonate relatively depleted in 13C, USGS44 is intended for daily use as a secondary isotopic reference material to normalize the δ13CVPDB‐LSVEC scale. It should be useful in quantifying drift with time,evaluating mass‐dependent isotopic fractionation (linearity correction), and adjusting isotope‐ratio‐scale contraction. Due to its fine grain size , it is not suitable as a δ18O reference material.EA/IRMS http://isotopes.usgs.gov/lab/referencematerials.html.USGS44 is available in units of 0.5 g in a glass vial; each vial is vacuum sealed in a plastic pouch. There is no limit on purchasing USGS44. It is available at 5https://publons.com/publon/10.1002/rcm.9006.The peer review history for this article is available at Data S1 Supporting InformationClick here for additional data file.


Confusion has persisted for decades in the United States (U.S.) over how much plain water to drink, despite national water intake recommendations which are based on high quality scientific evidence. This editorial summarizes the definition, alignment and coordination of evidence that informs the current U.S. adequate intake (AI) recommendations for water. It highlights gaps in the evidence that perpetuate confusion and opportunity to address the gaps through strategic planning. Many data are not included in NAM reviews, such as the 2005 report , because


While selecting predictive tools for implementation in clinical practice or for recommendation in clinical guidelines, clinicians and health care professionals are challenged with an overwhelming number of tools. Many of these tools have never been implemented or evaluated for comparative effectiveness. To overcome this challenge, the authors developed and validated an evidence-based framework for grading and assessment of predictive tools (the GRASP framework). This framework was based on the critical appraisal of the published evidence on such tools.The aim of the study was to examine the impact of using the GRASP framework on clinicians’ and health care professionals’ decisions in selecting clinical predictive tools.A controlled experiment was conducted through a web-based survey. Participants were randomized to either review the derivation publications, such as studies describing the development of the predictive tools, on common traumatic brain injury predictive tools (control group) or to review an evidence-based summary, where each tool had been graded and assessed using the GRASP framework (intervention group). Participants in both groups were asked to select the best tool based on the greatest validation or implementation. A wide group of international clinicians and health care professionals were invited to participate in the survey. Task completion time, rate of correct decisions, rate of objective versus subjective decisions, and level of decisional conflict were measured.193t=8.53; P<.001); increased objective decision making by 32%, from 62% (3.11/5) to 82% ; decreased subjective decision making based on guessing by 20%, from 49% (2.48/5) to 39% ; and decreased prior knowledge or experience by 8%, from 71% (3.55/5) to 65% . Using GRASP significantly decreased decisional conflict and increased the confidence and satisfaction of participants with their decisions by 11%, from 71% (3.55/5) to 79% , and by 13%, from 70% (3.54/5) to 79% , respectively. Using GRASP decreased the task completion time, on the 90th percentile, by 52%, from 12.4 to 6.4 min . The average System Usability Scale of the GRASP framework was very good: 72.5% and 88% (108/122) of the participants found the GRASP useful.We received a total of 194 valid responses. In comparison with not using GRASP, using the framework significantly increased correct decisions by 64%, from 53.7% to 88.1% (88.1/53.7=1.64; Using GRASP has positively supported and significantly improved evidence-based decision making. It has increased the accuracy and efficiency of selecting predictive tools. GRASP is not meant to be prescriptive; it represents a high-level approach and an effective, evidence-based, and comprehensive yet simple and feasible method to evaluate, compare, and select clinical predictive tools. Clinical decision support (CDS) systems have been discussed to enhance evidence-based practice and support cost-effectiveness -10. On tprofessionals in this paper, involved in decision making are challenged with an overwhelming and ever-growing number of tools. Many of these tools have never been implemented or evaluated for comparative effectiveness [When selecting predictive tools for implementation in clinical practice or for recommendation in clinical guidelines, clinicians and health care professionals, referred to as tiveness -19. By dtiveness . Professtiveness ,22. Whentiveness ,24. In ativeness . Furthertiveness .To overcome this major challenge, the authors developed and published a new evidence-based framework for grading and assessment of predictive tools (the GRASP framework) . The autPredictive tools are assigned the letters A, B, or C based on the highest phase of evaluation: before implementation, during planning for implementation, or after implementation respectively. If a tool’s predictive performance, as reported in the literature, has been tested retrospectively for validity using observational data, it is assigned phase C. If a tool’s usability or potential effect have been tested prospectively using small pilots, which may or may not reflect routine practice, it is assigned phase B. Potential effect of a tool is the expected, estimated, or calculated impact of using the tool, assuming it has been successfully implemented and used in clinical practice. Finally, if a tool has been implemented in clinical practice and there is published evidence evaluating its achieved postimplementation impact prospectively, it is assigned phase A.A numerical score within each phase is assigned based on the level of evidence associated with each tool. A tool is assigned grade C1 if it has been tested for external validity multiple times, grade C2 if it has been tested for external validity only once, and grade C3 if it has been tested only for internal validity. Grade C0 means that the tool did not show sufficient internal validity to be used in clinical practice. Grade B1 is assigned to a predictive tool that has been evaluated during the planning for implementation, for both of its potential effect, on clinical effectiveness, patient safety, or health care efficiency, and for its usability. Grade B2 is assigned to a predictive tool that has been evaluated only for its potential effect, while if it has been studied only for its usability, it is assigned grade B3. Finally, if a predictive tool had been implemented and evaluated for its postimplementation impact on clinical effectiveness, patient safety, or health care efficiency, then it is assigned grade A1 if there is at least one experimental study of good quality evaluating its postimplementation impact, grade A2 if there are observational studies evaluating its impact, and grade A3 if the postimplementation impact has been evaluated only through subjective studies, such as expert panel reports.For each phase and level of evidence, a direction of evidence is assigned based on the collective conclusions reported in the studies. The evidence is considered positive if all studies about a predictive tool reported positive conclusions and negative if all studies reported negative or equivocal conclusions. The evidence is considered mixed if some studies reported positive results, while others reported either negative or equivocal conclusions. To determine the overall direction of evidence, a protocol is used to sort the mixed evidence to support an overall positive or negative conclusion. The protocol is based on 2 main criteria: (1) degree of matching between the evaluation study conditions and the original tool specifications and (2) quality of the evaluation study. Studies evaluating tools in closely matching conditions to the tools’ specifications and providing high-quality evidence are considered first for their conclusions in deciding the overall direction of evidence.The final grade assigned to a predictive tool is based on the highest phase of evaluation, supported by the highest level of positive evidence, or mixed evidence that supports a positive conclusion. More details on the process of critical appraisal of published evidence, summarizing the evidence, and assigning grades to predictive tools are discussed in a published study that describes the development of the GRASP framework . The GRAThe aim of this study was to evaluate the impact of using GRASP on the decisions made by professionals in selecting predictive tools for CDS. The objective was to explore whether the GRASP framework positively supports professionals’ evidence-based decision making and improves their accuracy and efficiency in selecting clinical predictive tools. To explore this impact, a group of hypotheses have been proposed including that using the GRASP framework by professionals is going to (1) make their decisions more accurate, that is, selecting the best predictive tools; (2) make their decisions more objective, informed, and evidence-based, that is, decisions are based on the information provided by the framework; (3) make their decisions less subjective, that is, decisions are less based on guessing, prior knowledge, or experience; (4) make their decisions more efficient, that is, decisions are made in less time; and (5) make them face less decisional conflict, that is, become more confident in their decisions and more satisfied with them. We also proposed that using GRASP can move professionals who have less knowledge, less experience, and are less familiar with predictive tools to an equal or even higher accuracy of decision making than professionals who have more knowledge, have more experience, and are more familiar with tools when they do not use GRASP.This study was based on experimental methods. It aimed to examine the performance and outcomes of professionals’ decisions in selecting predictive tools with and without using the GRASP framework. Through a web-based survey, the experiment involved asking participants to select the best predictive tool for implementation in clinical practice or for recommendation in clinical practice guidelines from a group of 5 similar tools doing the same predictive task, one time with and another time without using the GRASP framework. In addition, participants were asked a few questions regarding the process of making their decisions through the 2 scenarios. Participants were also requested to provide their feedback on the perceived usability and usefulness of the evidence-based summary of the GRASP framework. This experiment does not include registration of the randomized controlled trial as it does not involve any patients, medications, or treatments.The emergency department (ED) is among the top health care specialties that are increasingly utilizing predictive tools, especially in the area of managing traumatic brain injury (TBI), which is the leading cause of death and disability among trauma patients -33. Two The authors recruited a wide group of international professionals to participate in this experiment through a web-based survey. To identify potential participants who work at the ED and those who have knowledge or experience about CDS tools, published studies were used to retrieve the authors’ emails and invite them. To retrieve studies on CDS systems, tools, models, algorithms, and pathways or rules used in the ED, emergency service, or emergency medicine published over the last 5 years by professionals who work in the EDs or services of their health care organizations or those who conducted emergency medicine, EDs, or emergency services research, 4 databases were used: MEDLINE, EMBASE, CINAHL, and Google Scholar. The authors expected a response rate of approximately 10%. Before the deployment of the survey, a pilot test was conducted by 10 expert professionals. The feedback of the pilot test was used to improve the survey. Professionals who participated in the pilot test were excluded from participation in the final survey. An invitation email, introducing details about the study objectives, the GRASP framework, the experiment task, the survey completion time, which was estimated at 20 min, and a participation consent was submitted to the identified potential participants with the link to the web-based survey. A reminder email, in 2 weeks, was sent to the potential participants who did not respond or complete the survey. The web-based survey was developed using the Qualtrics Experience Management Solutions Platform . The surIn this section, participants are asked to assume that they are the heads of busy EDs and are responsible for selecting the best tool, the most validated in the literature or implemented in clinical practice, out of the 5 diagnostic head injury predictive tools. The PECARN is the correct answer among the 5 pediatric tools, and both the CCHR and the NOC are correct answers among the 5 adult tools. On a 5-point Likert scale, participants were asked to show how much they agreed to the following: (1) they made their decisions based on guessing, (2) they made their decisions based on prior knowledge or experience, (3) they made their decisions based on the information provided in the survey, (4) they were confident in their decisions, and (5) they were satisfied with their decisions. The third section includes asking participants to provide their feedback on the usability of the evidence-based summary of the GRASP framework through a standard set of System Usability Scale (SUS) questions. The SUS is a simple, 10-item attitude Likert scale that provides a global view of subjective assessments of usability. It was developed by John Brooke in 1986 as a tool to be used in the engineering of electronic systems. It is now widely accepted as a tool for the evaluation of system usability ,53. Partt test with a significance level of 0.05 and power of 0.95. Calculations were conducted using G*Power software [As an initial estimate of the required sample size for this study, and based on similar studies, evaluating the impact of using information systems on professionals’ objective, informed, and evidence-based decisions, the authors aimed to recruit a sample of 40 to 60 participants -56. Moresoftware .t test to .007 instead of .05. The sample size was re-estimated to 96 participants. In addition, the SUS was calculated for the average rate and distribution of scores. The perceived usefulness and free-text feedback were analyzed. The demographic variables were analyzed for possible correlations or differences.To test the 5 proposed hypotheses, the study was designed to compare the 2 scenarios, making decisions with and without using the GRASP framework, based on a group of 7 measures: (1) time needed for tool selection decision making; (2) accuracy of tool selection decisions; (3) making decisions subjectively based on guessing; (4) making decisions subjectively based on prior knowledge or experience; (5) making decisions objectively based on the information and evidence provided; (6) levels of participants’ confidence in their decisions; and (7) levels of participants’ satisfaction with their decisions. The accuracy of making decisions, with and without GRASP, was also compared along with knowledge, experience, and familiarity with predictive tools. As this randomized controlled experiment was conducted via a web-based survey of clinicians and health care professions, Out of 5857 relevant publications retrieved, 3282 professionals were identified and invited to participate in the survey. Over the survey duration of 6 weeks, from March 11 to April 21, 2019, we received a total of 194 valid responses, with a response rate of 5.9%. Valid responses were identified as those who completed the survey until the end and answered questions in all the survey sections, with no missing sections. Six participants missed answering one or more questions in one or more of the survey sections, 5 participants did not provide their demographics, and 57 participants did not wish to be acknowledged in the study. The detailed distributions of participants based on gender, age group, years of experience, clinical and health care role, clinical specialty, familiarity with head injury predictive tools, and their countries are illustrated in P<.001). On a 5-point Likert scale, where strongly agree is considered equal to 5 and strongly disagree is considered equal to 1, the participants reported that they made their tools’ selection decisions based on guessing with an average of 1.98 (SD 1.22), when they used GRASP, compared with an average of 2.48 (SD 1.37), when they did not use GRASP. This shows a statistically significant reduction of 20% (P<.001). Participants reported that they made their tools’ selection decisions based on their prior knowledge or experience with an average of 3.27 (SD 1.44) when they used GRASP, compared with an average of 3.55 (SD 1.31), when they did not use GRASP. This shows a statistically significant reduction of 8% (P=.004).Using the GRASP framework, an evidence-based summary of predictive tools and a detailed report on each predictive tool, along with allowing participants to search the internet for further information, made them select the correct tools 88.1% of the time. Without GRASP, that is, using the basic information about the predictive tools, the full text of the studies describing each tool, along with allowing participants to search the internet for further information, they selected the correct tools 53.7% of the time. This shows a statistically significant improvement of 64% . Participants reported that they were confident in their decisions, with an average of 3.96 (SD 0.87), when they used GRASP, compared with an average of 3.55 (SD 1.15), when they did not use GRASP. This shows a statistically significant increase of 11% (P<.001). Participants reported that they were satisfied with their decisions with an average of 3.99 (SD 0.89), when they used GRASP, compared with an average of 3.54 (SD 1.20), when they did not use GRASP. This shows a statistically significant increase of 13% (P<.001). The duration of completing the task of selecting predictive tools showed high variability, with many statistical outliers. In addition to the average, the authors used the percentiles to avoid the effect of extreme outliers. The average duration of making the selection decisions showed a statistically insignificant reduction of 52% from 14.5 7 min (P=.39). There is also a reduction of 18.9% from 2.2 to 1.8 min on the 50th percentile, 37.3% from 5.3 to 3.3 min on the 75th percentile, 48% from 12.4 to 6.4 min on the 90th percentile, and 30.6% from 19.2 to 13.3 min on the 95th percentile. Participants reported that they made their tools’ selection decisions based on the information provided in the survey with an average of 4.10 (SD 1.10) when they used GRASP, compared with an average of 3.11 (SD 1.42), when they did not use GRASP. This shows a statistically significant increase of 32% . This indicates that participants who were less familiar with predictive tools thought that the GRASP framework was easy to use more than participants who were more familiar with the tools.The overall SUS rate of the GRASP framework and evidence-based summary, considering the responses of all 194 participants, was 72.5%, which represents a very good level of usability ,60. ExamAmong the 194 valid responses of participants, almost two-third (122) provided free-text feedback on the GRASP evidence-based summary usefulness and explained their feedback. The qualitative analysis of the open-ended question was conducted using the NVivo Version 12.3 software package . Most reThe use of GRASP has positively supported, and significantly improved, evidence-based decision making and increased the accuracy and efficiency of selecting predictive tools. Using the GRASP framework has significantly increased correct decisions and objective decision making, and significantly decreased subjective decision making based on guessing, prior knowledge, or experience. Moreover, using the GRASP framework significantly decreased decisional conflict, increasing the confidence and satisfaction of participants with their decisions. Furthermore, using the GRASP framework decreased the task completion time for selecting predictive tools. In addition, the average SUS of the GRASP framework was very good, and most participants found the GRASP framework useful.It is a challenging task for most health care professionals to critically evaluate a growing number of predictive tools, proposed in the literature, to select effective tools for implementation in clinical practice or for recommendation in clinical guidelines, to be used by other professionals. Although most of these predictive tools have been assessed for predictive performance, only a few have been implemented and evaluated for comparative effectiveness or postimplementation impact. Professionals need an evidence-based approach to provide them with standardized objective information on predictive tools to support their search for and selection of effective tools for clinical tasks. On the basis of the critical appraisal of the published evidence, the GRASP framework uses 3 dimensions to grade predictive tools: (1) phase of evaluation, (2) level of evidence, and (3) direction of evidence. The final grade assigned to a tool is based on the highest phase of evaluation, supported by the highest level of positive evidence, or mixed evidence that supports a positive conclusion. In this study, we present an evaluation of the impact of the GRASP framework on professionals’ decisions in selecting predictive tools for CDS.The GRASP framework provides a systematic and transparent approach for professionals to make objective, well-informed, and evidence-based decisions regarding the selection of predictive tools. This is very similar to the findings of using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework in evaluating the quality of evidence and strength of recommendations regarding treatment methods and decisions endorsed in clinical guidelines ,63. The Providing professionals with GRASP evidence-based information also enabled them to minimize subjective decision making, such as guessing, prior knowledge, or previous experience. This has been discussed in other studies investigating the role of utilizing evidence-based resources in decreasing subjective bias in making clinical, population-related, and health policy decisions ,68. EvidUsing GRASP made nurses and other professionals make more accurate decisions than physicians when they are not using GRASP. Using GRASP, clinicians of specialties other than emergency medicine make better decisions than emergency medicine clinicians without GRASP. Furthermore, using GRASP, professionals who were not familiar with head injury predictive tools made better decisions than professionals who were familiar with the tools without GRASP. Furthermore, the use of GRASP made decisions more efficient. Accordingly, using GRASP has moved professionals with less knowledge, less experience, and less familiarity with predictive tools to higher accuracy, higher efficiency, and better decision-making levels than professionals who had more knowledge, had more experience, and were more familiar with tools, but did not use GRASP.The usability of systems is an important foundation for successful implementation and utilization . UsabiliThrough this study, the GRASP framework is presented as an effective evidence-based approach to support professionals’ decisions when selecting predictive tools for implementation in clinical practice or for recommendation in clinical practice guidelines. Using the GRASP framework and the evidence-based summary improved the accuracy of selecting the best predictive tools, with an increased objective, informed, and evidence-based decision making and decreased subjective decision making based on guessing, prior knowledge, or experience. Using GRASP also decreased the decisional conflict faced by professionals by improving their confidence and satisfaction with their decisions. Using GRASP has also improved the efficiency of professionals in making their selection decisions by decreasing the time needed to complete the decision-making task.The GRASP framework represents a high-level approach to provide professionals with an evidence-based and comprehensive, yet simple and feasible method to evaluate and select predictive tools. However, when professionals need further information, the detailed framework report provides them with the required details to support their decision making. The GRASP framework is designed for 2 levels of users:(1) Expert users, such as health care researchers, experienced in evidence-based evaluation methods. They will use the framework to critically evaluate published evidence, assign grades to predictive tools, and report their details.(2) End users, such as clinicians and health care professionals, responsible for selecting tools for implementation in clinical practice or for recommendation in clinical guidelines. They will use the GRASP framework detailed reports on tools and their assigned grades, produced by expert users, to compare existing predictive tools and select the most suitable tools .The GRASP framework is not meant to be absolutely prescriptive. A lower grade tool could be preferred by a health care professional to improve clinical outcomes that are not supported by a higher grade one. For example, a practicing clinician may prefer an A2 tool showing improved patient safety in 2 observational studies rather than an A1 tool showing reduced health care costs in three experimental studies because they are now trying to improve patient safety to avoid reducing health care costs. It all depends on the objectives and priorities that the clinicians and health care professionals are trying to achieve. In addition, sometimes, more than one predictive tool should be endorsed in clinical practice guidelines, each supported by its requirements for application, conditions of use, and recommended for its most prominent outcomes of predictive performance or postimplementation impact on health care and clinical outcomes. Furthermore, even when GRASP assigns high grades to predictive tools, some of these tools may not be simply recommended for use in a different country or population than the ones that were used to develop and validate the tools in the first place. This might happen because of the population-related differences in the risks associated with the incidence of certain medical conditions, outcomes, or prognoses. This necessitates adjustment of the tools to the local context, thereby producing new versions of the tools, which requires re-evaluation by GRASP.Although the GRASP framework has been developed to assess and grade predictive tools and other similar CDS systems, the application of the framework concept of grading tools and systems based on the published evidence is not limited to predictive tools or CDS systems. The GRASP framework concept can be applied to assess and grade many other types of clinical tools, systems, and methods.Although we received a large and sufficient number of 194 valid responses, the very low response rate of 5.9% could have been improved if potential participants were motivated by some incentives. They could have also been motivated if more support was provided through their organizations, which need more resources to synchronize such efforts. For the sake of keeping the survey feasible, for most busy professionals, the number of questions was kept limited and the time required to complete the survey was kept in the range of 20 min. However, some of the participants showed their willingness to provide more detailed feedback, which could have been done through interviews, for example, but this was out of the scope of the study and was not initially possible with the huge number of invited participants. The reduction in the decision-making duration of selecting predictive tools, while using GRASP, was statistically insignificant, because of the high variability and extreme statistical outliers, with and without GRASP. This could be explained by the fact that the Qualtrics platform of the survey measures the task completion duration by subtracting the time of loading the page from the time of pushing the Next button after completing the task and not the actual time the participants spent active on the page, which is currently under development .To enable a wider global audience of clinicians, health care professionals, and clinical guideline developers to access detailed information, reported evidence, and assigned grades of different predictive tools, it is essential to implement the GRASP framework into a web-based platform. However, maintaining such a grading system up to date is a challenging task, as this requires continuous updating of the predictive tools grading and assessments when newly published evidence becomes available. In addition, the entire process is currently conducted manually, which represents a large burden on assessing and grading the huge number of existing predictive tools and those continuously emerging. Accordingly, it is essential to use automated or semiautomated methods for searching and processing new information to keep the GRASP framework information, grades, and assessments updated. Finally, we recommend that the GRASP framework be utilized by working groups of professional organizations to grade predictive tools to provide consistent results and increase reliability and credibility for end users. These professional organizations should also support disseminating such evidence-based information on predictive tools, similar to announcing and disseminating new updates of clinical practice guidelines.


In sub-Saharan Africa (SSA), malnutrition remains a major public health challenge, particularly among children under 5 years of age. Despite nutritional screening tools being developed and available to detect early malnutrition in under five-year-old children, malnutrition continues to be a health concern. However, the level of evidence on nutritional screening tools for predicting early malnutrition at the community level in a high disease burden setting is unclear. The objective of this scoping review is to systematically map the evidence on malnutrition screening tools for children under 5 years in sub-Saharan Africa (SSA) and to identify knowledge gaps.et al. 2010 recommendations, and the 2015 Joanna Briggs Institute guidelines. We will conduct a systematic search of relevant imperial sources of evidence from the following databases: CINAHL with full text, Academic search complete via EBSCOhost, Google Scholar, Science Direct, and PubMed. We will search for grey literature from the following humanitarian and aid organization websites: World Health Organization (WHO), The United Nations International Children’s Emergency Fund (UNICEF), and governmental departments. Following the database searches and title screening, eligible sources of evidence will be exported to an EndNote X9 reference library. Thereafter, duplicate articles will be removed in preparation for abstract and full article screenings. Data from the included sources of evidence will be extracted, and the emerging themes will be analyzed. The relationship between the emerging themes and the research questions will be critically examined. The quality of the included sources of evidence will be determined by using the Mixed Method Appraisal Tool (MMAT) version 2018. The search results will be presented in adapted Preferred Reporting Items for Systematic Reviews and Meta-Analysis: Extension for Scoping Reviews chart (PRISMA-ScR).The proposed study will be guided by an improved Arksey and O’Malley’s framework, Levac We anticipate finding relevant literature on malnutrition screening tools for children under 5 years in SSA. This study is likely to reveal research gaps, which could guide future research on malnutrition screening tools. Malnutrition remains a major public health challenge, particularly among children under 5 years of age . GloballMalnutrition screening tools (MST) are instruments used for early detection of patients who are nutritionally at risk and those who are already malnourished . Since 1An initial scoping search of the literature to determine whether systematic reviews and recommendations on our research questions that have been published has found these manuscripts more in HIC. Suitable screening tools for children are scarce with no consensus on the best method to assess their risk of malnutrition despite several recommendations on the importance of its early identification –16. MoreHence, as this is a broad topic, a scoping review was found to be most useful over a systematic review to map a range of literature that exists and would aid in focusing the research questions by charting existing research findings and identifying research gaps , 22. It The objective of this scoping review is to systematically map the evidence on malnutrition screening tools for children under the age of five in sub-Saharan Africa (SSA) and to identify knowledge gaps.et al. 2010 recommendations, and the 2015 Joanna Briggs Institute guidelines to guide the methodology of this scoping review [A scoping review method was selected as it outlines different types of evidence in the area of interest and highlights gaps for further research. Based on this, the current scoping review uses an enhanced Arksey and O’Malley’s framework, Levac g review –25. The g review . This prg review , 29.The research questions are based on the research objective that was formulated using the Population-Concept-Context (PCC) framework designed by the Joanna Briggs Institute . It was The primary research question is what evidence exists on malnutrition screening tools for children under 5 years living in sub-Saharan Africa?What malnutrition screening tools are used for children under 5 years in sub-Saharan Africa?What evidence exists regarding the performance of malnutrition screening tools for children under 5 years in sub-Saharan Africa?The secondary research questions are:The review will include empirical literature and grey literature that present evidence on malnutrition screening tools for children under 5 years in SSA. We will search for relevant empirical literature from the following electronic databases: CINAHL with full text, Academic search complete via EBSCOhost, Google Scholar, Science Direct, and PubMed. To improve the quality and reduce errors of the electronic search, the search strategies were peer reviewed using the PRESS 2015 Evidence-Based Checklist . The keyTo fine-tune the selection of the sources of evidence process and to improve consistency, a pilot test will be conducted prior to the review process commencing. There will be no language restrictions applied to the literature search. About 60 publications will be used for this procedure. To reduce any selection bias, screening of study titles and abstracts from the databases listed above will be conducted by two investigators (TPM and EO) independently. The relevant sources of evidence will be identified with the guidance of the inclusion and exclusion criteria. The eligible sources of evidence will then be exported to an Endnote X9 library created specifically for this review. All duplicates identified will be deleted before sharing the Endnote library with the two reviewers. An abstract screening form with questions will be developed based on our eligibility criteria. Discrepancies between reviewers at the title and abstract stage will be resolved through discussions until a consensus is reached. Following the title and abstract screening, full articles will be screened by the two reviewers (DMM and TPM) independently in parallel. Discrepancies between reviewers at the full article stage will be resolved by a third screener (DK). To ensure the reproducibility of the study, the references of excluded sources of evidence and the rationale for exclusion will be provided in an additional file of the completed review. Figure PopulationWe will include articles presenting evidence of children under 5 years. Under-five in this study refers to children who are less than 5 years old.We will exclude articles presenting evidence on children who are under-five but with developmental delays . This is because the tools that are used to screen children at risk of malnutrition are different from the ones used in children without delays.2.ConceptArticles reporting evidence on malnutrition screening tools will be included in this study. Malnutrition screening tools refer to tools used to screen patients who are at risk of malnutrition .Articles and studies that did not include the specificity of malnutrition screening tools will be excluded.Articles published between 2010 and 2019 will be considered in order to obtain the most recent information on our research topic.3.ContextWe will include articles reporting evidence from SSA. Sub-Saharan Africa refers to 46 African countries that are fully or partially located south of the Sahara .To ensure that relevant sources of evidence are selected for this review, the study selection process will be guided by the eligibility criteria as specified under the inclusion/exclusion criteria. The sources of evidence will include information from empirical literature and grey literature that present evidence on malnutrition screening tools for children under 5 years in SSA.A data charting form will be created using a Google form where all the necessary extracted data from the included sources of evidence can be populated Table . The staWe will present a narrative account of the findings from the included sources of evidence and present themes. Thematic content analysis will be employed to extract the themes, which will be critically examined in relation to the study research question, the aim of the study, literature, gaps for future research, and MST for under 5 years in SSA countries. The implications of the study results for future research, policy, and practice will be examined and reported on.We will utilize the Mixed Method Appraisal Tool (MMAT) version 2018 to appraise the methodological quality of included sources of evidence . The criAll differences between the protocol and the final research study will be reported together with the rationale for these changes. The consequences of these modifications on the magnitude, direction, and validity of the outcomes will also be presented .Timely treatment of malnutrition in children of five years of age at primary healthcare facilities could prevent 500,000 deaths annually . Hence, The limitations of this research study include the following: (1) it may omit sources of evidence that include participants older than five and may result in the exclusion of important sources of evidence and (2) including sources of evidence published between 2010 and 2019 could introduce the risk of publication bias. However, this period was chosen because it will represent the most recent information on our research topic. The study will focus on children under the age of 5 years in SSA as literature has shown that this region has the highest under-five mortality rate in the world . MoreoveThe current scoping review strength is that it addresses objectives that are important for patients, clinicians, and policymakers. We expect that the scoping review results will provide a comprehensive overview of the evidence on the topic and highlight areas where evidence is controversial or missing. Additionally, it will provide key information to policymakers and health professionals interested in planning, funding, and delivering evidence-based effective interventions aimed at preventing malnutrition in children under 5 years. Moreover, identified research gaps could inform future studies and guide policy decisions to enhance healthcare outcomes in SSA. We plan to disseminate the study’s findings in peer-reviewed journals and at conference proceedings that focus on nutritional and disease screening. Additionally, the study findings will be disseminated to professionals and stakeholders involved in malnutrition prevention and treatment.Additional file 1. PRISMA-P  2015 checklist: recommended items to address in a systematic review protocol*.