The determination of nutritional knowledge and nutrition literacy among nursing students will enable nursing departments to establish the needs and solutions to enhance nutrition education in their education programs. Therefore, this study is aimed to evaluate the nutrition literacy and nutrition knowledge level of nursing students.The study data were collected with ‘Information Form’, ‘Anthropometric Measurements’, ‘Nutrition Knowledge Level Scale for Adults’, and ‘Evaluation Instrument of Nutrition Literacy on Adults’. Analyzes were performed using descriptive and nonparametric tests.p > 0.05). There was a statistically significant positive correlation between the nutrition knowledge score and the nutrition literacy total score and the nutrition literacy sub-sections scores (p < 0.05).The score of nutrition knowledge is 56.6 ± 6.8 and 50.5% of them have a good nutrition knowledge level. The total nutrition literacy score is 28.6 ± 4.4 and 91.6% of them have a sufficient nutrition literacy level. It was no significant difference between students’ characteristic features and nutrition knowledge score and nutrition literacy total score (It has been determined that the nutrition knowledge and nutrition literacy levels of nursing students correlated with each other. To improve students’ nutrition knowledge levels, as well as to improve their nutrition literacy and prevent non-communicable diseases nutrition lessons should be included in the curriculum. TherefoNutrition literacy and nutrition knowledge have emerged as critical components in promoting and maintaining healthy eating behaviors , 7. HealNurses play an important role in protecting and improving health and increasing the quality of life . It is aIn Turkey, studies that examine nutrition literacy and nutrition knowledge among nursing students are limited. Considering the nursing students, who are defined as the young population, it is very important to have a high level of nutrition knowledge in this age group for the society of the future to be healthy. In this context, this is aimed to examine the nutrition literacy and nutrition knowledge level of nursing students.What is the nutrition literacy level of the nursing students?What is the nutrition knowledge level of the nursing students?Is there a relationship between nutrition literacy and the nutrition knowledge level of nursing students?This cross-sectional and descriptive study design was used in this study. The study was conducted with 309 nursing students between January-March 2022 at the Faculty of Health Sciences, Department of Nursing.The sample comprised 690 nursing students from the Faculty of Health Sciences, Department of Nursing at a university in Turkey. The G*Power software (Version 3.1.9.6) was used to analyze the sample's size . BecauseThe inclusion criteria of the study were being nursing students, being able to speak Turkish, volunteering to participate in the research. The exclusion criteria were filling in the data in completely and wanting to leave the research at any stage of the research.Ethical approval with the decision number 25/23 dated 27.12.2021 from Trakya University Faculty of Medicine Dean's Office of Ethics Committee for Non-Invasive Scientific Research before data collection. The researchers wrote to the nursing students the aims and methods of this research in an online google survey, explaining that confidentiality was protected and participation in the research was completely voluntary. Informed consent was obtained from all participants included in the study. Individuals who ticked the "I consent to participate in this study voluntarily" tab at the beginning of the web-based questionnaire were included in the study. In addition, the students were informed that they could leave the research at any time without giving any reason. All procedures in the study were carried out following the Declaration of Helsinki.The study data were collected with ‘Information Form’, ‘Anthropometric Measurements’, ‘Nutrition Knowledge Level Scale for Adults (NKLSA)’ and ‘'Evaluation Instrument of Nutrition Literacy on Adults (EINLA)’. Researchers create data collection tools through google surveys. With the help of the academic staff in the nursing department, necessary explanations were made and shared with the students in the WhatsApp groups of each course. This data collection method has been preferred in order not to exchange materials and maintain social distance in accordance with the pandemic rules.The form was created by the researchers according to the literature , 22, 23.2, individuals are classified as underweight, between 18.50–24.99 kg/m2 as normal , between 25.0–29.99 kg/m2 as overweight, and 30.0 kg/m2 or higher as obese [Anthropometric measurements (body weight and height) of the individuals were questioned based on the statement. Individuals were informed about how to take anthropometric measurements in the questionnaire form. Body mass index (BMI) was calculated by dividing the body weight by the square of the height . If the The first part of the 'Nutrition Knowledge Level Scale for Adults' was used to evaluate the nutritional knowledge level. The scale was developed by Batmaz and Güneş  in 2018 The nutrition literacy status of individuals was determined with the 'Evaluation Instrument of Nutrition Literacy on Adults’. The scale was developed by Cesur et al.  and its There are 10 questions about ‘general nutrition information. Nutrition literacy level is scored between 0–3 points as 'insufficient', between 4–7 points as ‘borderline’ and between 8–10 points as 'sufficient'.There are 6 questions about ‘reading comprehension and interpretation’. Nutrition literacy level is scored between 0–2 points as 'insufficient', between 3–4 points as ‘borderline’ and between 5–6 points as 'sufficient'.There are 10 questions about ‘food groups’. Nutrition literacy level is scored between 0–3 points as 'insufficient', between 4–7 points as ‘borderline’ and between 8–10 points as 'sufficient'.There are 3 questions about ‘serving sizes’. Nutrition literacy level is scored 0–1 point as 'insufficient', 2 points as ‘borderline’ and 3 points as 'sufficient'.There are 6 questions about ‘reading food labels and basic mathematics’. Nutrition literacy level is scored between 0–2 points as 'insufficient', between 3–4 points as ‘borderline’ and between 5–6 points as 'sufficient' .The analyses eliminated cases with missing data for the primary research variables. The Statistical Package for the Social Sciences (version 22.0) software was used for analyses. Data were evaluated with descriptive statistics such as mean, standard deviation, number and percentage. Distribution analysis of the data was performed using the histogram, coefficient of variation ratio, Skewness, Kurtosis and Kolmogorov–Smirnov tests. Mann–Whitney U test and Kruskal Wallis test were used in independent groups for comparison. The relationship between numerical variables was evaluated with the Spearman correlation coefficient. A p-value of less than 0.05 was considered to be statistically significant.2. 69.9% of the students have a healthy body weight. 27.2% of the students participating in the study had taken a course on nutrition at the university. The number of main meals consumed is 2.4 ± 0.5 and the number of snacks is 1.5 ± 1.0 by the students.The general characteristics of the nursing students are given in Table The nutrition knowledge levels of individuals are given in Table The nutrition literacy status of individuals is given in Table p > 0.05).The nutrition knowledge level and nutrition literacy status of individuals according to some parameters are given in Table The relationship between nutrition knowledge level and nutrition literacy status is given in Table p > 0.05) and nutrition knowledge score (p > 0.05). (data not shown in the table).In addition, the relationship between BMI and nutritional literacy and nutritional knowledge level was evaluated, and no statistically significant difference was found between BMI and nutritional literacy total score  have sufficient nutrition literacy. However, only 13.6% of students have adequate nutrition literacy in portion sizes and 52.8% in food label reading and basic math. Mehri et al. , stated It was emphasized that nutrition knowledge and nutrition literacy correlated with each other . In thisThis research reports the findings of a study that evaluated nursing students' nutrition knowledge levels, as nurses are in a superior position to guide as nutrition instructors and advisors by interacting with their patients in discussions that enhance their knowledge regarding disease prevention and treatment via nutrition. It has been determined that the nutrition knowledge and nutrition literacy levels of nursing students are related to each other, therefore, clinical and general nutrition should be given more place in the curriculum of nurses. The findings of this study show that nursing students need nutrition-related courses in nursing curricula in order to improve their nutritional literacy levels, as well as improve their nutritional knowledge and prevent NCDs. The implications of this study are significant for the nursing curriculum.This study has some limitations. Firstly, the study included a small sample size. The response rate was also approximately 45% which to an extent may not give a wider representation of the study participants. Secondly, we aimed to evaluate the relationship between nursing students' nutritional knowledge and nutritional literacy. For this reason, we did not add the factors affecting the nutritional knowledge and nutritional literacy levels of nursing students to the questionnaire. It is recommended that future studies examine the factors affecting the nutritional knowledge and nutritional literacy levels of nursing students.


There is conflicting evidence on the effectiveness of acupuncture in the treatment of postpartum depression (PPD). This study aimed to assess previous systematic reviews/meta-analyses (SRs/MAs) on the effectiveness of acupuncture to treat PPD.SRs/MAs regarding the use of acupuncture for PPD were identified from the establishment of digital databases to November 2021. The Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2) was applied to evaluate the methodological quality of included SRs/MAs. The Grades of Recommendations, Assessment, Development and Evaluation (GRADE) was utilized to evaluate the evidence quality for outcomes of interest.Six studies that conducted quantitative syntheses were included. According to AMSTAR-2, the methodological quality of these SRs/MAs was critically low owing to limitations of items 2, 4, and 7. According to GRADE, no study included high-quality evidence and most studies included low-quality evidence.Acupuncture m be beneficial for PPD, however, due to limitations of current evidence and inconsistent findings, further studies are needed to provide stronger evidence to draw definitive conclusions. Postpartum depression (PPD) is a mood disorder associated with childbirth, since its onset begins between the first day and 4 months after delivery . TypicalTreatment of PPD includes pharmacotherapy, psychotherapy, or both, which is consistent with the treatment recommended in guidelines for major depression . HoweverThis study followed the methodology of the Cochrane Handbook and high-quality studies –16.The following eligibility criteria were used to screen studies: (a) SRs/MAs based on randomized controlled trials (RCTs) on the use of acupuncture to treat PPD; (b) participants diagnosed with PPD by a recognized guideline; (c) interventions included acupuncture therapy or acupuncture plus conventional medication (CM), while the control group was treated with CM, CM plus acupuncture, sham acupuncture, or other non-pharmacological therapy; (d) outcomes included the Hamilton Rating Scale for Depression (HAMD), Edinburgh Postnatal Depression Scale (EPDS), effective rate and estradiol levels. Repeated publications or studies lacking complete data were removed.Embase, PubMed, Web of Science, Cochrane Library, CNKI, CBM, Wanfang, and VIP were searched for studies published between database creation and November 2021. The following search terms were applied: postpartum depression, acupuncture, meta-analysis, and systematic review. Two independent evaluators screened abstracts and titles, and then assessed potentially eligible full texts for final inclusion. Disagreements were resolved through discussion with a third independent reviewer. The following data were extracted from included studies: first author, year of publication, country, sample size, interventions, outcomes, quality assessment methods, and summary estimates of effect.Two independent evaluators assessed the methodological quality of SR/MA using the Assessment of Methodological Quality of Systematic Evaluation 2 (AMSTAR-2) . AMSTAR-Two independent evaluators used the Grade of Recommendation, Assessment, Development and Evaluation (GRADE)  to assesA narrative synthesis was used in this overview. The characteristics and results of each SR/MA as well as results from AMSTAR 2 were summarized by tabulation. The GRADE evidence profile and summary of findings table were generated using the GRADE pro GDT online software.The literature search identified 114 articles, of which 40 duplicates were removed. Titles and abstracts of 74 articles were screened, and 63 articles were subsequently excluded. The full text of the remaining 11 articles was read and five articles were excluded. Therefore, six papers were included in our analyses –25. The All included studies were conducted in China and published within the last 5 years. All studies were MAs with significant differences in sample size . The interventions in the experimental group were acupuncture, or a combination of acupuncture and CM, while the control interventions were CM and/or sham acupuncture. Details on study characteristics are reported in All studies were rated as very low quality according to the AMSTAR-2 criteria. The key factors contributing to lower methodological quality were item 2 (only one review registered a protocol), item 4 , and item 7 . Detailed assessment results of AMSTAR-2 are shown in A total of 19 outcome indicators were assessed. No indicator was deemed high, while two were moderate, 12 were low and five were very low quality of evidence. Risk of bias was the most common reason for downgrading the evidence, followed by inconsistency, imprecision, publication bias, and indirectness. Details are shown in All studies used the HAMD to assess the severity of depression, and one review  concludeAcupuncture is routinely used in clinical therapy for PPD in China as a way to improve therapeutic effectiveness. Numerous SRs/MAs have evaluated the effectiveness of acupuncture for PPD, however, inconsistent results have been reported. In this context, a critical evaluation of different SRs/MAs and a summary of the scientific nature of the evidence is necessary . FurtherA total of six SR/MAs were included in this study, all of which were published in the past 5 years, suggesting that more researchers are beginning to study acupuncture as an alternative therapy for PPD. Nineteen outcome measures on the effectiveness of acupuncture to treat PPD were evaluated, and although most indicators suggested positive results, these were inconsistent. Furthermore, although most of the included studies suggested that acupuncture was effective as a treatment for PPD, most authors did draw firm conclusions due to the low methodological quality of evidence or the small size of included trials. Indeed, all reviews were considered to be of very low quality according to AMSTAR-2 criteria. Therefore, our analysis concluded that acupuncture might be an effective treatment for PPD, but such conclusion must be treated with caution due to limitations of the current evidence.Over recent years, AMSTAR-2 has become the most widely used tool to evaluate the methodological quality of SRs/MAs. All included studies had more than one critical flaw, so that there is very low confidence in their results. The key factors contributing to this setting were item 2 (only one review registered a protocol), item 4 , and item 7 . It was found that study protocols contribute to increased transparency of the methodology used and improve the overall methodological quality of SRs/MAs . The absIn this study, authors of the included SRs/MAs did not draw definitive conclusions. Indeed, after rating the evidence using the GRADE system, we found that the certainty of evidence was unsatisfactory, indicating that findings of the included SRs/MAs are uncertain. Although all SRs/MAs evaluated only RCTs, the certainty of evidence was limited owing to the risk of bias , inconsistency, imprecision, or publication bias. The results of the methodological quality evaluation of RCTs showed that there is room for addressing random, distributed hidden or blind biases. Nevertheless, we must acknowledge that there are specificities of acupuncture therapy (inability to blind physicians and patients) that make the implementation of RCTs challenging. Improved standardization and precision of acupuncture techniques and procedures are urgently needed, as only a rigorously designed and implemented RCT can reduce the risk of bias and therefore assess the effectiveness of interventions .To our knowledge, this is the first overview of SRs/MAs summarizing the current evidence on the use of acupuncture to treat PPD. The methodological and evidence qualities of the included SRs/MAs may help to inform evidence-based decision-making and guide future high-quality studies. However, our study presents some limitations. First, the quality analysis demonstrated numerous methodological flaws in the performance of SRs/MAs, and the evidence quality was not satisfactory, making it impossible to draw firm conclusions about the use acupuncture for PPD. Second, the rapid growth in the number of SRs/MAs highlights challenges faced by healthcare decision-makers and researchers in keeping up with the evidence. This overview found that there were typically a large number of low-quality SRs/MAs. To help evidence-based practice, there is an urgent need for high-quality SRs/MAs that do not overlap and are up to date. Furthermore, widely used AMSTAR-2 tool and GRADE system are subjective evaluation tools, therefore the accuracy of assessments can vary. To mitigate this limitation, quality assessments were performed by two independent authors.Acupuncture might be beneficial for PPD. However, due to limitations of the current evidence and inconsistent findings, further studies are needed to provide strong evidence to draw definitive conclusions.The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding authors.XH and QF conceived the study and drafted the manuscript. LM, RJ, RG, and XZ help with the implementation of research. FQ and LZ provided guidance on the overview methodology. LM revised the manuscript. All authors read, critically reviewed, and approved the final manuscript as submitted.This work was funded by Jiangxi Provincial Department of Science and Technology Key Research and Development Program General Project (No. 20192BBGL70037) and Changshu Science and Technology Development Plan Project (No. CS202136).The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


The authors, journal, date and country of publication, patient group studied, study type, relevant outcomes and results of these papers are tabulated. The RIVER and ENAVLE trials showed non-inferiority of rivaroxaban  and edoxaban  when compared with vitamin K antagonist. These studies include a low number of patients within 3 months of index surgery and overall low statistical power regarding this particular subgroup of patients. Data derived from lower evidence studies are compatible with the aforementioned findings. The available evidence suggests that non-vitamin K antagonist anticoagulants are as safe and as efficient as vitamin K antagonist regarding thromboembolic prophylaxis and bleeding event rates in patients with surgical bioprosthesis and atrial fibrillation within 3 months of bioprosthesis implantation. However, this evidence is derived from a limited number of studies with important methodological limitations. Expanding non-vitamin K antagonist anticoagulant recommendation to the early postoperative period warrants more confirmatory research.A best evidence topic in cardiac surgery was written according to a structured protocol. The question addressed was: ‘Are NOACs as safe and efficient as vitamin K antagonist regarding thromboembolic prophylaxis and major bleeding in patients with surgical bioprosthesis and atrial fibrillation within 3 months of surgery?’ Altogether more than  A best evidence topic was constructed according to a structured protocol. ICVTS [A best evidence topic was constructed according to a structured protocol. This is fully described in the ICVTS .In [patients with surgical bioprosthesis and atrial fibrillation] is [non-vitamin K antagonist anticoagulant (NOAC) similar to vitamin K antagonist (VKA)] in terms of [safety (major bleeding) and efficacy (thromboembolic prophylaxis)] within 3 months of bioprosthesis implantation.A 62-year-old aeroplane pilot with symptomatic aortic stenosis is proposed to surgical valve replacement. Because of his erratic schedules, the patient chooses a bioprosthesis in order to avoid therapeutic drug monitoring. However, in the postoperative period, the patient develops atrial fibrillation refractory to rhythm control attempts. You are tempted to use an NOAC to manage this patient postoperatively, however, your senior consultant reminds you that according to 2020 American College of Cardiology/American Heart Association (ACC/AHA) guidelines VKA should be considered for the first 3 months after bioprosthesis implantation. A review of the current evidence regarding this topic is warranted to validate either approach.A comprehensive literature review was performed searching Medline from inception to February 2021 using the PubMed interface. The following strategy was used: [ OR NOAC OR apixaban OR rivaroxaban OR edoxaban OR dabigatran) AND prosthesis]. References of selected papers were screened for additional relevant papers. The eligible papers were in English.In total, 324 papers were found using the reported search. From these, 6 papers were identified that provided the best evidence to answer the question. These are presented in Table post hoc analysis of the ARISTOTLE , as well as the primary safety outcome (major bleeding) . The subgroup analysis of patients with AF is consistent with the main results regarding the primary efficacy endpoint  and the primary safety outcome ; but no P for interaction of this subgroup is presented. This is a single-centre analysis including a small heterogeneous population and relatively large non-inferiority boundaries (8%) with limited statistical power, which precludes any definite conclusion without further research.The results of the ENAVLE trial were published by Shim et al. . ENAVLE et al. [P = 0.22) or thrombosis/stroke  rates were reported. Even though CHA2DS2 VASC and HAS-BLED scores were similar between the two groups this remains an unmatched retrospective analysis with inherent bias and small sample size.Further lower evidence observational studies corroborated the aforementioned findings: Pasciolla et al.  conducteet al. [P = 0.64), rehospitalization rates for stroke or transitory ischaemic attack  or rehospitalization for major bleeding complications . The subgroup analysis of valvular patients revealed non-significant P for interaction for the 3 aforementioned outcomes , thus, rendering the overall results applicable to the subgroup of valvular patients. This is an observational non-randomized study with inherent bias and multiple exclusion criteria including preoperative AF and preoperative anticoagulation. Additionally, the reported outcomes are limited to a 30-day follow-up period and events that required rehospitalization.Nauffal et al.  publisheThe available evidence suggests that NOAC are as safe and as efficient as VKA regarding thromboembolic prophylaxis and bleeding event rates in patients with surgical bioprosthesis and AF within 3 months of index surgery. However, there is a significant lack of evidence regarding this specific period, as the available data may be influenced by the heterogeneity of pharmacological agents and bioprosthesis position/manufacturer used, timing of anticoagulation initiation, small and heterogeneous cohorts, and overall limited statistical power. Expanding NOAC recommendation to the early postoperative period calls for more confirmatory evidence, therefore, further research is warranted.Conflict of interest: none declared. Interactive CardioVascular and Thoracic Surgery thanks Milan Milojevic and the other anonymous reviewers for their contribution to the peer review process of this article.


The present work aimed to delineate (i) a revised protocol according to recent methodological developments in evidence generation, to (ii) describe its interpretation, the assessment of the overall certainty of evidence and to (iii) outline an Evidence to Decision framework for deriving an evidence-based guideline on quantitative and qualitative aspects of dietary protein intake.A methodological protocol to systematically investigate the association between dietary protein intake and several health outcomes and for deriving dietary protein intake recommendations for the primary prevention of various non-communicable diseases in the general adult population was developed.The developed methodological protocol relies on umbrella reviews including systematic reviews with or without meta-analyses. Systematic literature searches in three databases will be performed for each health-related outcome. The methodological quality of all selected systematic reviews will be evaluated using a modified version of AMSTAR 2, and the outcome-specific certainty of evidence for systematic reviews with or without meta-analysis will be assessed with NutriGrade. The general outline of the Evidence to Decision framework foresees that recommendations in the derived guideline will be given based on the overall certainty of evidence as well as on additional criteria such as sustainability.The methodological protocol permits a systematic evaluation of published systematic reviews on dietary protein intake and its association with selected health-related outcomes. An Evidence to Decision framework will be the basis for the overall conclusions and the resulting recommendations for dietary protein intake.The online version contains supplementary material available at 10.1007/s00394-021-02789-5. Traditionally, nutrition research has primarily focused on the health impact of dietary carbohydrates and dietary fats. Accordingly, previous guidelines developed by the German Nutrition Society  addressed health-related recommendations for the consumption of dietary fat , 2 and dA guideline panel consisting of DGE staff members, DGE scientific board members and selected experts was constituted in 2016 to develop the evidence-based guideline on dietary protein intake. This panel selected a panel coordinator (BW) and reached a consensus-based decision on the outcomes to be addressed in the guideline, taking aspects, such as disease burden and expected relevance of protein intake, into account. Since the amount of protein reported in observational studies or used in intervention studies varied notably, the panel refrained from a strict unifying definition for high or low protein intake (such as above or below 0.8 g/kg body weight per day). Instead, within each chapter, the link of higher and lower protein intake as reported or used in the included studies to the specific endpoint will be evaluated. A methodology task force  was set up to delineate the procedures to be followed for establishing the guideline. A working group  was commissioned to perform the literature search and selection, data extraction and assessment of methodological quality, and outcome-specific certainty of evidence. Conflict of interest statements were obtained from all members of the guideline panel and published (Supplement 1).In consideration of the increased availability of SRs with or without MA, it was primarily decided to develop an approach based on systematic searches for these study types and to perform umbrella reviews for each of the nine selected health-related outcomes. A continuous publication of each successively completed umbrella review, from now on referred to as guideline chapter, and a final overall recommendation based on these guideline chapter conclusions, were set out to be the backbone for the protein guideline. The evidence-based guideline on dietary protein intake was registered as an umbrella review in PROSPERO (CRD42018082395) .Like the previous approaches to derive the guidelines of the DGE , a multiExplicit research questions for each guideline chapter are defined. To support the formulation of explicit research questions, the PICO scheme  or its modification, the PECO scheme , was applied . Table 1For the systematic literature searches, the databases PubMed, Cochrane Database of Systematic Reviews, and Embase are used. Due to restricted resources, the outcome-specific searches are performed successively, covering a 10-year period. Details on search periods and search updates will be presented in the respective outcome-specific guideline chapters. Specific search terms were selected, and search strings were formulated. Table Literature search is restricted to articles published in English or German within the last ten years. The search period is extended if update searches are performed. SRs with or without MA of prospective studies , prospective cohort studies, case-cohort studies, or nested case–control studies) are eligible. Those study designs are considered the most reliable in nutrition research . SRs alsSelection of SRs is performed by two authors independently according to the predefined inclusion and exclusion criteria Table . Any disFlow diagrams in each guideline chapter will outline the study selection process, including the number of records identified, the number of records considered as potentially relevant after title/abstract screening, and finally the number of records included and excluded, respectively, with reasons for excluding records following full-text screening.p values, heterogeneity estimates and subgroup analyses. Any discrepancies between the two authors are resolved through discussion, if necessary, with consultation of the methodology task force. In addition, an overview of all primary sources included in the SRs on the respective health outcome is provided to highlight overlaps.Two authors independently extract the following data from each included SR using a standardised form: first author of the SR, year of publication, study type of relevant primary studies, study period of relevant primary studies, study population of relevant primary studies, dietary assessment method of relevant primary studies, range of protein intake if provided, intervention/exposure(s) of primary studies, outcome(s) investigated by primary studies, effect estimates including 95% CI, A Measurement Tool to Assess Systematic Reviews 2” tool (AMSTAR 2) [For the assessment of methodological quality of the retrieved SRs, a modified version of the “MSTAR 2)  is used The NutriGrade scoring tool is used to assess the outcome-specific certainty of evidence for each SR with at least one MA of RCTs and/or prospective cohort studies   or the majority  of the results must be consistent. Biological plausibility must be given in any case (direct or inverse association). In the final step, the results of the NutriGrade and AMSTAR 2 ratings are considered. Depending on the level of evidence, the SRs must have achieved a certain rating in both tools. If no SR is identified or if the majority of SR reached a very low outcome-specific certainty of evidence and/or a low methodological quality, the overall certainty of evidence is considered insufficient. The detailed underlying criteria are outlined in Table The suggestions of the overall certainty of evidence rating, and the outcome-specific conclusions will be finalised after discussing them with all guideline panel members to reach an overall expert decision.The finalised guideline chapters, e.g. the outcome-specific umbrella reviews, will be submitted for publication to an international journal with peer review. In case of insufficient overall evidence regarding a selected outcome, an umbrella review might not be possible and is therefore not submitted for publication.Once all guideline chapters will be finalised, the guideline panel will reconvene to summarise all results and conclude comprehensive recommendations. To that end, the guideline panel will adapt and modify the GRADE EtD frameworks  as a sysOverall certainty of evidence: as described above .Problem priority: weighting of outcomes  to prioritise potentially contradictory outcome-specific results .Weighting of benefits and harms: e.g. for some outcomes increased intake of protein might be beneficial but for others rather harmful. Therefore, it will be necessary to weight both sides.Dietary intake levels and preferences: dietary habits, cultural preferences in food intake, and current protein intake levels of the target populations .Ecological sustainability: environmental impact  and economic considerations  , 16.Acceptability and feasibility of recommendation: estimate of possible implementation impact, including potential barriers .Potential impacts on health equity: considering differential effects on disadvantaged populations or specific population subgroups.To derive these recommendations, the panel will judge and consider the following criteria, supported by the best available evidence. More details on the following aspects 2 to 7 will be provided in a separate publication:A draft version of the protein guideline will be published online for public consultation. Comments will be accepted for a period of two months. After careful consideration of all the remarks by the entire guideline panel, the final guideline will be published.Today, an overwhelming body of nutrition research study data is available which can hardly be critically evaluated and made applicable by a single user, be it a research scientist or a practitioner. The systematically developed and evidence-informed dietary guidelines of the DGE should provide a basis to assist nutrition experts as well as other experts to make appropriate professional decisions in their respective fields. Similar endeavours have been described by methodologists, pointing out the need to provide trustworthy nutrition guidelines by adhering to internationally accepted and transparently described literature review standards .The upcoming guideline for dietary protein will reflect the current knowledge on quantity and quality of dietary protein intake and the risk association with several health-related outcomes . Importantly, in addition to the overall certainty of evidence, several additional aspects will have been considered in the overall recommendations on dietary protein intake, including considerations on problem priority, benefit and harms, dietary intake levels and preferences, ecological sustainability of a certain protein intake level and financial costs of recommendations, acceptability, feasibility, and potential impacts on health equity. Finally, the results of an open consultation will be handled before the final guideline on protein intake is published. To that end, this publication described in detail the overview on the procedures to assess the overall certainty of evidence, and outlined cornerstone aspects of an EtD framework that is to be adapted to the current GRADE EtD frameworks . The GRAThe multi-step procedure followed in previous guidelines of the DGE . Adhering to these standards will ensure that the derived recommendations will be based on a high quality, novel overview of systematic literature reviews. Nevertheless, we are aware that including solely SRs with or without MA involves the potential risk of overlooking evidence from (recently published) primary studies. When setting up the methodology, we assumed that for the majority of guideline chapters the amount and nature of SRs will likely be sufficient to conduct umbrella reviews and that they will be sufficiently up to date.The dietary recommendations derived in the guideline of the DGE on protein intake will be put forward by a group of experts in nutritional physiology, public health and clinical nutrition, nutritional epidemiology, and evidence synthesis methods. Respective details from each member of the guideline working group are made available (Supplement 1), thereby disclosing potential conflicts of interest.The described methodological procedure has various limitations. The most relevant general aspect to be discussed refers to the predominantly available non-randomised studies, e.g. prospective observational studies, which have been repeatedly criticised to be prone to various biases and  confounding . To addrFurthermore, both SRs and umbrella reviews as such hold limitations. Although SRs potentially provide stronger evidence than individual studies alone, methodological and statistical heterogeneity of the included studies represent, among others, difficult to solve challenges. In terms of fundamental approaches to umbrella reviews, instead of searching for the various SRs available, only the best or latest could be considered. Given the considerable overlap of SRs on a specific topic, this could be an option. However, since we aim for a comprehensive summary of the current body of SR evidence, it is preferred to include all SRs regardless of overlap. This decision is supported by the recently revised chapter of the Cochrane handbook “Overview of Reviews” .A comprehensive protocol for systematic literature reviews relying on umbrella reviews and cornerstones of an EtD framework has been developed for the derivation of the upcoming evidence-based guideline for dietary protein intake of the DGE. Once the recommendations of the guideline are derived and published, nutrition and health experts as well as public health policymakers may profit from these guidelines; its implications include a better understanding of the desirable and undesirable associations/effects of protein consumption and provide consequently the option to promote a better-informed decision-making by the general population.Supplementary file1 (XLSX 14 KB)Supplementary file2 (PDF 405 KB)Supplementary file3 (PDF 267 KB)Supplementary file4 (PDF 316 KB)Supplementary file5 (PDF 293 KB)Below is the link to the electronic supplementary material.


SLEEP, Dr. Pengo and colleagues offer a framework for management that contextualizes current evidence [Medical decisions are best made in consultation with high-quality evidence, but, where such evidence is incomplete, clinicians must counsel patients using available knowledge. This is the case for patients with obstructive sleep apnea (OSA) who lack excessive daytime sleepiness (EDS) and established cardiovascular disease (CVD). In the current issue of evidence . The autIn its first step, the OSCAR algorithm advises referring patients with overweight or obesity to weight loss clinics for behavioral interventions aimed at achieving 10% weight loss. We agree with this approach, although implementation will be challenging. Despite longstanding recommendations that patients with OSA are counseled about weight loss and referred for services, few patients receive evidence-based weight loss interventions . Even inThe OSCAR algorithm then suggests that clinicians reassess symptoms. The authors raise two distinct issues: (1) reliance on the Epworth Sleepiness Scale (ESS) score may underdiagnose EDS, and (2) additional symptoms beyond EDS may be worth treating. Given the strong evidence that CPAP improves EDS, it is important that we do not miss this symptom. We fully endorse the authors’ recommendation to include family members in assessing symptoms and to use a holistic approach in evaluating EDS. While the ESS is the instrument most often used to assess sleepiness , it has Among patients without symptoms, the OSCAR algorithm then considers cardiovascular risk. The authors propose evaluating OSA-induced cardiovascular risk by employing ambulatory blood pressure monitoring and prescribing CPAP to patients with hypertension, particularly if there is a non-dipping pattern. This decision rests on evidence that CPAP reduces blood pressure and the association of hypertension and non-dipping patterns with cardiovascular risk . Among aAs we consider the potential impact of the OSCAR algorithm on CVD, we must consider the challenge posed by primary prevention. The OSCAR algorithm focuses on primary prevention because previous trials among patients with preexisting CVD and low symptom burden failed to find a cardiovascular benefit with CPAP . HoweverA further complication relates to adherence. Many attribute the negative results for CPAP in secondary prevention trials to low adherence , 16. Sindo have high-quality evidence of benefit (e.g. excessive daytime sleepiness). Furthermore, evidence suggests that those currently least likely to receive effective OSA care are those marginalized by society—individuals who are poor, with low health literacy, and from minority backgrounds [Finally, one must consider the potential consequences of deploying this algorithm within a health system. In designing the OSCAR algorithm, the authors focused on decisions that are justifiable at the individual patient level and the authors appropriately acknowledge the principle “primum non nocere”—first do no harm. However, it is important to look beyond the individual when considering harm. Sleep medicine services are a scarce resource , 18, andkgrounds . As a fiDespite its potential limitations, the OSCAR algorithm is a welcome addition to the literature. As the authors clearly point out, it is a proposal that needs to be rigorously tested. By providing a clear and logical framework, the OSCAR algorithm identifies specific areas of uncertainty in the clinical management of OSA in those without EDS or CVD. OSCAR also serves as an excellent starting place for future trial protocols to resolve uncertainties around how to manage this important patient population.


This paper investigates using multi-label deep learning approach to extending the understanding of cognitive presence in MOOC discussions. Previous studies demonstrate the challenges of subjectivity in manual categorisation methods. Training automatic single-label classifiers may preserve this subjectivity. Using a triangulation approach, we developed a multi-label, fine-tuning BERT classifier to analyse cognitive presence to enrich results with state-of-the-art, single-label classifiers. We trained the multi-label classifiers on the MOOC discussion messages that were categorised into the same phase of cognitive presence by the expert coders, and tested the best-performing classifiers on the messages that the coders categorised into different phases. The results suggest that multi-label classifiers slightly outperformed the single-label classifiers, and the multi-label classifiers predicted the discussion messages as either one category or two adjacent categories of cognitive presence. No messages were tagged as non-adjacent categories by the multi-label classifier. This is an improvement compared to manual categorisation by our expert coders, who obtained non-adjacent categories and even three categories of cognitive presence in one message. In addition to the fully correct prediction, parts of messages were partially correctly predicted by the multi-label classifier. We report an in-depth quantitative and qualitative analysis of these messages in the paper. The automatic categorisation results suggest that the multi-label classifiers have the potential to help educators and researchers identify research subjectivity and tolerate the multiplicity in cognitive presence categorisation. This study contributes to extending the literature on understanding cognitive presence in MOOC discussions. This paper explores using the multi-label deep learning approach to automatically analyse the phases of cognitive presence, as a triangulation approach to enriching the understanding of cognitive presence in the discussion messages in Massive Open Online Courses (MOOCs). As a type of open education, MOOCs have an absence of barriers to entry and a wider range of learner profiles, diverse learning objectives, and motivations. Tracing the first MOOC from 2008 Siemens, , MOOCs aAsynchronous online discussion forums are a vital component of MOOCs, offering a virtual place for learners and instructors to interact and communicate together. Previous studies found that learners who actively engaged in discussion forums tend to achieve better performance in MOOCs : To what extent can our multi-label classifier accurately predict the cognitive presence categories in online discussion messages from a target MOOC? Can the multi-label deep learning classifier outperform the state-of-the-art, single-label classifier in this study?Sub-question 2 (SQ2): Are there any MOOC discussion messages, which have been manually labelled into the same category by coders, be tagged into multiple labels by the multi-label classifier? In what proportion? Are there any common patterns?Sub-question3 (SQ3): Are there any MOOC discussion messages, which have been manually labelled into different categories by coders, be tagged into one single label by the multi-label classifier? In what proportion? Are there any common patterns?The multi-label classification (prediction) can allow a text associated with more than one label simultaneously, which has been used in the semantic scene classification construction and problem-solving processes  Exploration, in which the learners exchange information to investigate the answers or solutions of the problems proposed in the previous phases but not able to reach a coherent conclusion; 3) Integration, in which the learners synthesise coherent conclusions to the problems proposed in the previous phases with sufficient support; 4) Resolution, in which the learners apply, test or argue the solutions or conclusions suggested in the previous phase, shaping the new meaning or construct. We acknowledge the richness and complexity of message content in the MOOC discussion data. Since this study focuses on the cognitive presence, messages that were not fitted into any of the above phases were classified as the Other category. For example, messages that only indicate social presence  were classified into Other. The other two presences of the CoI  will be explored in our future work. We applied an adapted coding rubric of cognitive presence  a large number of features constructed a high-dimensional space that could easily raise over-fitting problems; 2) the resulting classifiers were domain-specific; thus, the classifiers were difficult to generalise to other domains. To address these issues, Kovanović et al.  developeDeep learning algorithms have been shown outstanding performance in many fields, especially in computer vision fields , the Transformer aims to solve the problems where the sequential input information is required to pass to the output sequences, such as speech recognition and machine translation tasks  MOOC on the FutureLearn platform.Three expert coders classified the sample data into four phases of cognitive presence and the Other (the fifth category), according to an adapted coding rubric for the MOOC discussions  by the different trials of oversampling ratios in the training process. In the best case, we created 400 synthetic messages for the fewest categories , and 250 for the second-fewest categories , and maintained the original size of the largest category  as shown in Fig. F1 scores are common evaluation metrics for single-label classification tasks   and the predicted labels for each test instance  separately, and then average over all test instances in the test set  asfully incorrect prediction if neither of the two predicted labels is equivalent to the manual label; thus, we defined the fully incorrect ratio (IR) asApart from the Sorower, , we defiIR is equivalent to the error rate in the single-label classifications as defined in Tharwat’s  approach as a baseline compared to the multi-label classifiers. Only the single-label classification approaches have been attempted in the literature of automatic classification of cognitive presence in online discussions. The RF approach and its extensive use was regarded as the state-of-the-art single-label classifier for analysing cognitive presence in online discussions  and ntree . After fine-tuning, mtry of 118 and ntree of 700 were reported as the optimal parameters in the baselines. A final tenfold CV method was performed to repeat the training-test loop ten times with the optimal RF classifier to maintain consistency with the multi-label classifiers. We compared the classifier performance with and without the SMOTE exact method to the multi-label classifiers with and without the data augmentation methods, respectively, in Sect. 4.1.Followed the previous studies, we adopted 199 classification features extracted from the sample data by two computational linguistics tools, Coh-Metrix  of the multi-label classifiers with and without the Data Augmentation (DA) method to compare with the single-label baselines (RF classifiers). The multi-label classifiers achieved the lower fully incorrect ratios but slightly higher example-based F1 scores than the single-label baselines. Instead of fully correct or incorrect, several messages fell into the partially correct group by the multi-label classifiers. Also, the use of the DA method enlarged the proportion of the partially correct group from 12.16% to 39.86%.Table macro-averaged F1 scores with the SDs of the multi-label and single-label classifiers by every category of cognitive presence. The multi-label classifiers reached better F1 scores in most of the categories. In Exploration, all the classifiers performed the highest F1 scores, in which the multi-label classifiers were almost the same as the baselines ; however, the F1 score decreased in Integration after using the DA method in the multi-label classifier.Table example-based metrics of the multi-label classifiers with and without the DA method. The exact match ratio and the F1 score were improved by using the DA method. Compared to the results of the AgreementSet in Table F1 score reached the similar value in the test with the DA method, but the exact match ratio and the fully incorrect ratio largely declined. Interestingly, the messages that were labelled partially correct accounted for beyond half of the entire DisagreementSet. This outcome was opposite to the results in the testing AgreementSet .Some relevance between the classification decisions of expert coders and the multi-label classifiers was reflected in the outcomes of the tests on the DisagreementSet. Compared to the results of the tests on the AgreementSet Table , the pares Table  largely F1 scores. Using the DA method might increase the risk of deepening the confusion  between Exploration and Integration since most of the disagreements between the expert coders appeared between these two categories  by building on the conclusion of other group members, which should be a “convergence with supporting ideas” in Integration. The classifier also predicted a slight probability of 0.39 in Resolution. The seventh message proposed a divergent opinion from another learner’s conclusion with a concrete example. Also, it offered deeper thoughts based on the previous conclusion, which should be in Resolution. The classifier tagged it a low probability of 0.34 in Resolution but a higher probability of 0.67 in Integration. The classifier tends to regard these two messages as similar patterns. Objectively, these two messages might not have clear boundaries if we rethink the definition of ‘new’ or deeper constructions as the paramount indicators of Resolution in the Garrison et al.  and Hu esecond sub-question (SQ2) that the best-performing multi-label classifier tagged almost 40% messages of the testing set into two adjacent labels in the cognitive presence categories, in which one of them was the same as their manual labels. We also found that the messages with the manual label of the start (the Other) or the end (Resolution) of a cognitive presence cycle had only one alternatively predicted label other than the manual label. The message with the manual label that occurred at the middle of the cycle  had two alternatively predicted labels, either before or after the manual one. This distribution was similar to the disagreements between the three coders  Therefore…”. The structure can be closely associated with the definitions and indicators of Exploration and Integration in the coding rubric  Therefore…”. However, this expression seldom occurred in the messages from the latter group.The group of messages that contained the two predicted labels (Exploration and Integration) accounted for 62% in the messages that had the same two manual labels in the third row in Fig. The multi-label classifiers predicted 62% of the messages that had the manual label of both Integration and Resolution  that the best-performing multi-label classifier predicted 51.1% of the DisagreementSet messages as one single category of cognitive presence, and 96% of these single-label predictions was the same as one of the expert coders’ decisions. Meanwhile, all the predictions of the DisagreementSet messages were either one single phase or two adjacent phases of cognitive presence, even the messages that were categorised to two non-adjacent phases or three phases , we conclude that the multi-label classifiers slightly outperformed the state-of-the-art single-label classifiers, and they predicted the messages that had been categorised differently by the expert coders into one single category or two adjacent categories of cognitive presence. We envisage that overlaps exist between the definitions the adjacent phases of cognitive presence, and the multi-label deep learning method has the potential to identify the subjectivity of cognitive presence categorisation in the discussion messages from the target MOOC.To summary, in response to the main research question  address the class imbalance problem before the model training process by investigating effective methods (e.g., active machine learning (Rubens et al.,


First and foremost, I would like to thank an anonymous Associate Editor and Prof. Nigel Stallard for arranging a fascinating discussion around my article, “Target estimands for population‐adjusted indirect comparisons.”This rejoinder discusses the potential development of an estimands framework in the context of evidence synthesis and health technology assessment (HTA). I consider the following base‐case scenario. An evidence synthesis  is required for HTA. The evidence synthesis combines the results of multiple randomized controlled trials (RCTs).Two challenges arise. First, different policy decisions will require different research questions; the estimands targeted for regulatory approval will likely not align with those targeted for HTA. Second, within the evidence synthesis, there may be misalignment between the estimands targeted by individual studies. Finally, I reassert that marginal estimands should be preferred in the context of the original discussion.1For RCTs in the regulatory setting, the use of estimands has been stimulated by the recent publication of the E9 (R1) Addendum, issued by the International Council of Harmonisation (ICH) and recognized by agencies such as the United States Food and Drug Administration.estimand is rarely used. This does not imply a lack of interest in clarifying the research question prior to the analysis. In HTA, the PICO framework is typically used to translate policy questions into research questions. PICO consists of four components: (1) population; (2) intervention; (3) comparator(s); and (4) outcome. To obtain reimbursement for a given treatment, the manufacturer must submit an evidence dossier that addresses the research question(s) included in the scope. Current best‐practice guidelines recommend prespecifying the relevant PICO question(s) in the HTA scoping process.In evidence synthesis, and more generally in HTA reporting, the term The PICO framework is a valuable tool. Nevertheless, it needs refinement to more precisely characterize the inferential target in HTA; that is, to map the HTA research question into a formal estimand. The current framework does not discuss the population‐level measure used to summarize the treatment effect. It also lacks a strategy for intercurrent events, which as stated by Van Lancker et al,My original commentaryFundamentally, the estimands required for regulatory and HTA decision‐making are expected to diverge. Regulatory decisions are made prior to HTA decisions. Individual studies will likely be completed at the time of undertaking the evidence synthesis, and estimand definitions for the former will precede estimand definitions for the latter.Moreover, regulatory and HTA decision‐makers have different perspectives. The HTA paradigm places emphasis on evaluating “real‐world” effectiveness, whereas regulators typically prioritize internal over external validity when assessing clinical efficacy. In terms of intercurrent events, estimands based on the “treatment policy” and the “composite” strategies seem more relevant for HTA purposes. Nevertheless, regulators may prefer other strategies in certain scenarios. In terms of comparator(s), HTA demands comparing the intervention under study versus all alternative treatments for a specific disease state. Conversely, regulators typically require a head‐to‐head comparison against placebo or standard of care.In addition, there is a multiplicity of decision‐makers within HTA. Different agencies will target different populations and, therefore, different estimands. HTA policy decisions typically consider external validity, but external validity depends on the population that is targeted by the payer. Given the large number of regulatory bodies and national and regional HTA agencies, I coincide with Schiel: we should avoid focusing exclusively on the viewpoint of a single stakeholder when designing studies.2Apart from mismatches in estimands between decision‐making contexts, there may also be inconsistencies between the trial‐specific estimands combined in an evidence synthesis. As expressed by Russek‐Cohen, “individual studies were not planned with similar estimands nor they were necessarily planned in anticipation of a meta‐analysis.”The availability of standardized outcome and treatment definitions depends on the clinical indication and the interventions being evaluated. Differences in endpoints may arise if outcomes are measured using different methods, at different times, or if the trials have different duration. When a treatment is assessed in multiple studies, there may be dissimilarities between the versions of the intervention being administered, for example, in dosing formulation, delivery mechanism, co‐treatment regimens, or concomitant background medications. Clinical input is critical to assess the potential impact of these sources of misalignment on the conclusions of an evidence synthesis.population in the estimands framework, and in formulating research questions in evidence synthesis and HTA.The studies being synthesized may target populations that differ in terms of their baseline characteristics, due to different inclusion/exclusion criteria. Even if inclusion/exclusion criteria are identical, the covariate distribution of the sampled patients will likely vary from study to study. These differences may give rise to heterogeneous treatment effect sizes across studies, which threaten the validity of the evidence synthesis. Random effects models have traditionally been used to account for effect heterogeneity.Here lies the value of covariate‐adjusted indirect comparisonsConflicts in the summary effect measures that are pooled in an evidence synthesis are also a cause for concern. When the effect measure is non‐collapsible, combining marginal and conditional estimates will produce bias.3To conclude, I wish to examine some of the considerations made by SennThe target population of interest is not necessarily broad. Often, it is highly selected and takes into account specific patient characteristics. For instance, the final scope of a health technology appraisal for abemaciclib (TA810), recently published by the National Institute for Health and Care Excellence, describes the target population as “adults with hormone receptor‐positive, HER2‐negative, node‐positive early breast cancer after definitive surgery of the primary breast tumor at high‐risk of recurrence.”A preference for marginal estimands does not necessarily imply that an “all or nothing” decision is required or that “homogeneity” is assumed. When subsets of the target population are considered to benefit differently from the reimbursement of a drug, relevant sub‐populations are defined during the scoping process, in which case separate PICO questions can be formulated for each. In this case, the covariates used to define the subsets are binary or categorical, and the distinction between marginal and conditional estimands is blurred. A conditional estimand at the population level can be viewed as a marginal estimand at the sub‐population level.Nevertheless, the specific context—covariate‐adjusted indirect comparisons in HTA—is central to the debate. Consider the situation described by Phillippo et al in their original simulation study:However, the estimands framework demands that the estimand is articulated first, prior to any estimation. Here, the definition of a conditional estimand would depend on statistical modeling assumptions. Conversely, the marginal estimand can be unambiguously interpreted without reference to a particular adjustment model. I believe that this point is implicit throughout Van Lancker et al's discussion.


The World Health Organization develops global normative guidelines using a comprehensive and transparent process involving critical evidence appraisals. Basing these appraisals on direct evidence alone may delay the introduction of new interventions, particularly for areas where indirect evidence exists from similar interventions or related health conditions.This study presents an example of application of indirect evidence from HIV self-testing to inform a recommendation for hepatitis C virus self-testing.This example supports pragmatic approaches for rapid development of global guidelines, such as situations where relevant and sufficient indirect evidence can facilitate accelerated introduction of new interventions to achieve public health impact.Evidence appraisal is a critical component of global health normative guidelines development by the World Health Organization (WHO). Since 2007, WHO’s handbook for guidelines development has outlined a structured and transparent process to ensure guidelines are informed by the latest evidence on intervention effectiveness, feasibility of implementation and considerations on other important elements such as values and preferences of users and providers, resource use, equity and human rights.4Self-testing is a process whereby an individual collects their own specimen, performs a simple rapid test and interprets their own result. WHO first recommended HIVST in 2016.To support the rapid development of WHO guidelines on HCVST, complementary sources of information were used, including: (1) a systematic review on the effectiveness of HCVST; (2) an update to a previous systematic review on the effectiveness of HIVST that informed the 2019 WHO recommendation on HIVST; (3) community-based HCVST usability and acceptability studies and community values and preferences; and (4) HCVST cost-effectiveness modelling. Below we outline the process and use of evidence sources with a focus on application of indirect evidence of intervention effectiveness in WHO guidelines development process.We conducted a systematic review of the published and grey literature from 1 January 2010 to 16 December 2020. The review protocol was prospectively registered with PROSPERO (CRD42021235825). We included studies that compared HCVST to standard HCV testing services and measured one or more key outcomes, including: testing uptake; antibody positivity; linkage to additional testing following a reactive HCVST result; linkage to clinical assessment and/or treatment initiation among those diagnosed; proportion cured among those diagnosed; HCVST-related social harms, adverse events or misuse. We found no articles that met the inclusion criteria, thus no direct evidence on the effectiveness of HCVST was identified. Results of this review are detailed in an annex of the WHO HCVST guidelines.4a priori decision made in partnership with the guidelines group to include evidence on HIVST effectiveness given the similarity in intervention, diseases and affected populations. We used a living systematic review on HIVST for evidence,There was an WHO collaborated with the Foundation for Innovative Diagnostics to conduct HCVST usability studies among men who have sex with men, people who inject drugs and general populations across several countries. These studies showed that majority of participants were able to accurately perform and interpret their results using a prototype HCVST kit.13A cost-effectiveness modelling study was conducted to identify drivers of cost of testing or HCV cure with introduction of HCVST across four countries.14The systematic review identified no direct evidence on HCVST effectiveness. However, extensive indirect evidence on HIVST found the intervention to be safe, effective, acceptable and feasible across a range of populations and settings. Extensive implementation experiences now exist across the globe that can inform routine implementation. Additional direct, supportive evidence showed HCVST is perceived as beneficial and acceptable by potential users and providers and may identify additional infections. Taken the available sources of evidence together, the guidelines group, by consensus, determined that HCVST should be offered as an additional approach to HCV testing as a WHO recommendation.4This novel approach can expedite the guideline development for new interventions where evidence from similar interventions exists with sufficient overlap in epidemic contexts , with important caveats. We purposefully have not further defined these criteria as each requires careful consideration and specification according to the intervention under consideration. For example, defining what level of population overlap and within which characteristics will be a case-specific determination. Additionally, there are some instances where sole use of indirect evidence would never be appropriate, such as for drugs or vaccines that would require specific safety and efficacy evaluations among the populations being considered. The specific example presented in this commentary—HCVST—relates to a testing approach for an infectious disease—a decidedly different type of intervention as compared with a pharmacological treatment or vaccine. Given the rise of differentiated service delivery models and self-care approaches within a growing number of health areas , one could envision taking an approach to guideline development using indirect evidence within these intervention types when feasible. For HIV, hepatitis, tuberculosis and sexually transmitted infections, integrated service delivery models are already being implemented or considered.Additionally, when using indirect evidence, it is important to note differences that exist across the intervention under consideration and the comparator. For example, for the HCVST guidelines, no HIVST trial was identified among people who inject drugs—a key priority population for HCV. Thus, this population was purposefully prioritised for usability studies. There are also differences in HCV and HIV testing . Additionally, no evidence on HCV cure rates was available as it is not applicable to HIV. An a priori decision was thus taken to use linkage to HIV treatment or care among those diagnosed HIV-positive as a proxy for potential cure rates. The guidelines group convened by WHO noted sufficient similarity between HIVST and HCVST in terms of interventions, programming and priority populations and deemed that indirect HIVST evidence would be applicable to HCVST. The guidelines group also identified research gaps and priorities that would inform future implementation considerations, including the need for more HCVST research among people who inject drugs, innovative service delivery models and efficient referral pathways.In this commentary, we described a pragmatic guideline development process that occurred with available indirect and supportive evidence in the absence of direct evidence. As evidence from RCTs for new interventions can take several years to generate, this process accelerated the recommendation of HCVST, thus potentially accelerating improved health outcomes among affected populations. For other urgent public health problems, such processes for guideline development can potentially expedite availability of interventions that have proved effective in similar contexts. Developing more specific, standardised procedures and criteria for determining when the application of indirect evidence is appropriate are needed, and we welcome engagement to further refine this.Basing recommendations on mostly indirect evidence necessitates close monitoring of implementation and highlights a need to address research gaps, such as understanding the feasibility of implementation among certain populations, within certain contexts, or using certain delivery approaches that were not included in the evidence used to inform the initial recommendation.Results from the HCVST guidelines process provide a potential roadmap for navigating this process in situations where direct evidence is lacking but indirect evidence is available and appropriate to use. In such instances, the guideline development process can be made more efficient through use of accessible resources and information sharing among guideline developers. Additionally, obtaining feedback from stakeholders across the relevant interventions is important to ensure consensus with using indirect evidence to inform policy. WHO plans to replicate this process for other similar conditions such as syphilis self-testing.


Clinical research broadly aims to influence decision-making in order to promote appropriate healthcare. Funding agencies should prioritise research projects according to needed research topics, methodological and cost-effectiveness considerations, and expected social value. In Chile, there is no local diagnosis regarding recent clinical research that might inform prioritisation for future research funding. This research aims to comprehensively identify and classify Chilean health research studies, elaborating evidence gap maps for the most burdensome local conditions.We will search in electronic databases  and perform hand searches to retrieve, identify and classify health research studies conducted in Chile or by authors whose affiliations are based in Chile, from 2000 onwards. We will elaborate evidence matrices for the 20 conditions with the highest burden in Chile  selected from those defined under the General Regime of the Health Guarantees Act. To elaborate the evidence gap maps, we will consider prioritised interventions and core outcome sets. To identify knowledge gaps and estimate redundant research, we will contrast these gap maps with the available international evidence of high or moderate certainty of evidence, for each specific clinical question. For this purpose, we will search systematic reviews using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach.No ethical approval is required to conduct this project. We will submit our results in both peer-reviewed journals and scientific conferences. We will aim to disseminate our findings through different academic platforms, social media, local press, among others. The final results will be communicated to local funding agencies and government stakeholders.We aim to provide an accurate and up-to-date picture of the research gaps—to be filled by new future findings—and the identification of redundant research, which will constitute relevant information for local decision-makers. The protocol for this scoping review includes a comprehensive electronic and hand search strategy.We will be able to accurately elucidate local evidence gaps by first constructing evidence grids and then populating them with the results of our search.Considering that we used the estimations from the Global Burden of Disease 2019 for prioritising the most important diseases in Chile, we might be neglecting some important clinical entities, such as multicomponent diseases.We will develop gap maps only for the 20 conditions with the highest burden in Chile.Primary clinical research and synthesised evidence are the basis for knowledge translation, decision support and implementation in clinical settings, which should ultimately guide the generation of primary research .1 2Good quality clinical research, which must be set on priority questions,17 18Although some frameworks have been proposed to evaluate research applications,20The lack of a prioritised agenda in developing countries may generate a source of inequity, by letting third-party interests  inhabit regional research planning,This is a protocol for a government-funded project with the following objectives:To identify and classify health research studies conducted in Chile or by authors whose affiliations are based in Chile.To elaborate evidence maps for relevant health conditions in Chile, considering prioritised interventions and core outcome sets.To populate the maps with the primary and secondary evidence and their risk of bias and to identify knowledge gaps and redundant research incorporating international evidence.This section describes the methods for each of the objectives.We will conduct a scoping review that will be reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews (PRISMA-ScR).27We will consider any clinical study design, either conducted in Chile or at least one affiliation based in Chile. We define clinical studies as those focused on a clinical health topic describing, measuring or exploring a health-related outcome in humans. Considering that our focus in the following objectives will be the highest priority conditions of the last Chilean health reform, we will consider studies published from 2000 onwards.28We will consider any descriptive, observational or experimental primary study designs, including case reports, case series, cross-sectional studies, case–control studies, cohort studies, quasiexperimental studies, diagnostic or prognostic studies, mixed methods studies, and clinical trials. We will also include different formats of evidence synthesis, including systematic reviews, scoping reviews, evidence maps and overviews.We will perform a comprehensive search in electronic databases, without restriction of language or publication status.We will search the following databases:MEDLINE via Ovid SP.Embase via Elsevier.PsycINFO via ProQuest SP.Cumulative Index to Nursing and Allied Health Literature (CINAHL) via EBSCOhost.Latin American and Caribbean Literature in Health Sciences (LILACS).Web of Science (WoS) via Clarivate.3210.1136/bmjopen-2021-057555.supp1Supplementary dataWe will conduct complimentary hand searches with the same criteria reported in the electronic search. We identified seven information resources considered relevant to identify all the evidence that has not been indexed in the databases described in the electronic search and additional four information resources necessary to retrieve trials, reviews or other types of evidence, published peer reviewed research, that meet the eligibility criteria , gender and type of authorship .Diseases or health conditions addressed by each study according to the taxonomy developed by the Global Burden of Disease.36Funding: we will classify the type of funding of each article, public or private, competitive funding, pharmaceutical or medical device industry sponsoring, or international grants.Conflicts of interest: we will extract the authors’ statements from each report, considering descriptions for each author if available.We will enter the data into a data extraction form .We do not intend to perform a risk-of-bias assessment at this stage (see Objective 3). We will summarise the findings of each category . Moreover, we will illustrate the trends for each study design and disease category.We will elaborate evidence matrices for prioritised health conditions selected from those defined under the General Regime of the Health Guarantees Act ) that was stated in 2004 in Chile. This Regime commanded public and private health providers to guarantee access, opportunity, quality and financial protection for the most relevant programmes, diseases or conditions.38We extracted the list of health conditions and excluded those defined as programmes  for people aged 65 years and older’ which aims to improve independence in the elderly). These conditions were initially prioritised by the Chilean government based on a local study of the GBD in 2007.We will then conduct the following steps to complete the matrices that will be populated with evidence in the following objective.The definition of main outcomes for each health condition is a complex and difficult task, considering that reported outcomes are prone to bias, and that many studies tend to report outcomes with positive or statistically significant findings.41http://www.comet-initiative.org/). The COS includes outcomes that are most relevant to clinicians, decision-makers, patients and carers. If COS are not available for an individual condition, we will build a set of outcomes based on consensus with the input of multidisciplinary experts and patients, considering the existing outcomes embedded in current GES clinical practice guidelines.For that reason, we will consider agreed standardised sets of outcomes, known as core outcome sets (COS) for identifying the main outcomes. The COS represents a non-restrictive minimum set of outcomes to be assessed and reported in studies for every health condition, and they are permanently updated and revised by the Core Outcome Measures in Effectiveness Trials (COMET) initiative  in the rows, and the main outcomes (defined in Step 2) in the columns. These grids will be populated in each intersection with the included studies, as we detail in Objective 3 (see Objective 3 below). We will use evimappr,After the completion of the tasks proposed in Objective 1, we will select all the studies that might provide evidence for decision-making regarding interventions, to populate each of the matrices elaborated as described in Objective 2. Whenever an evidence matrix is populated with studies for a certain outcome and intervention, we will refer to it as an evidence map.From the complete set of research articles included in Objective 1, we will separate all those studies with a design relevant for decision-making regarding interventions:Randomised controlled trials.Non-randomised primary studies, including controlled clinical trials, quasiexperimental designs  and observational studies .Synthesised evidence relevant to interventions .Descriptive studies, qualitative or mixed methods studies, diagnostic accuracy studies, narrative reviews, scoping reviews and evidence maps will not be considered for this objective as they do not provide evidence supporting the efficacy or effectiveness of interventions.To assess risk of bias of the included studies, we will use standardised tools for each methodological design: (1) Cochrane Risk-of-Bias tool for randomised clinical trials,We will populate each node of the previously elaborated grids or matrices with the selected studies, considering the intervention and outcome on that node. We will represent the populated nodes with visual symbols (bubbles), with different colours according to the type of study and risk of bias, and different sizes according to the number of studies available. Once a matrix is populated with Chilean studies, we will refer to it as a ‘local evidence map’.We will develop a second map for each prioritised condition populated with rigorous international research (defined as the availability of high or moderate certainty of evidence), for each specific clinical question, according to the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach,Once Step 4 is concluded, we will have two maps for each condition: (1) a local evidence map  and 2) a global certainty of the evidence map . In order to identify knowledge gaps and estimate redundant research, we will contrast these two maps. Once we cross these two maps, we will classify each populated node of the local map as ‘adequate research’ or ‘redundant research’, while each empty node will be classified as ‘evidence gap’, according to the criteria defined in We will consider the absence of local evidence as a gap. If this gap is established in the context of low or very low certainty of evidence in the global map, or if there is no global evidence, we will consider it as a true gap. If the gap is established in the context of high or moderate certainty of evidence in the global gap, we will consider it in general as a false evidence gap.If there is local research in the context of a low, very low certainty or no evidence in the global map, we will consider the local research as adequate. If there is local research in the context of a high or moderate certainty of evidence in the global map, we will consider the research as adequate if it was published before the review that informed the certainty of evidence, or as redundant research if it was published after this review.No patients were involved in the development of this protocol.No ethical approval is required to conduct this project. Our findings will be submitted to peer-reviewed journals and scientific conferences. To amplify the impact of our work and to accomplish knowledge translation objectives, we will disseminate the results through different platforms, including academic social media, blogs, local press, among others. At the same time, we will aim to communicate the final results to local funding agencies and government stakeholders, in order to inform the elaboration of future national research agendas.After the completion of this proposed study, we will deliver an overall picture of the clinical research conducted in Chile since 2000, and we will elaborate evidence gap maps for the main health conditions guaranteed in Chile, with a detailed description of the most relevant interventions and outcomes and the type of clinical research in each node.We will also study the evidence gaps for the 10 health conditions with the highest burden in Chile and compare them with the available research globally; as a way to establish the true gaps of clinical research, and the amount of redundant findings addressed in Chile or by authors affiliated to Chilean institutions.A limitation of our protocol might be the use of the GBD 2019 estimations to prioritise the most important diseases or conditions in Chile, according to GES. As GES is based on policy and local definitions, and not totally on an epidemiological rationale, our conceptualisation may neglect some important clinical entities (mainly multicomponent diseases). Nevertheless, we have added A main strength of our protocol is the comprehensiveness of our search strategy. The development of a peer-reviewed search strategy in major electronic databases complemented with an exhaustive hand search will allow us to probably identify the whole body of clinical evidence conducted locally. The hand search is crucial and constitutes an important source of information, considering that research in the Latin American Region might not be adequately indexed in the electronic databases. The process of development of evidence maps constitute another major strength of our proposed research. We will first develop the evidence grids identifying main outcomes and interventions independently of the results of our search and screening process (Objective 2), and we will populate those grids afterwards with the results of our selection process (Objective 3). Besides identifying the research being conducted in Chile, with this approach, we will be able to accurately elucidate the evidence gaps, since it is possible that particular interventions in the grid remain unpopulated, as the construction of the grid will not be guided by the results of our search.This accurate and up-to-date picture of the research gaps—to be filled by new future findings—and the identification of redundant research will constitute relevant information for local decision-makers and, at the same time, an interesting methodological proposal to visualise clinical research trends and gaps in other countries or regions.