Social presence is an important concept for understanding psychosocial processes in learning scenarios that make extensive use of mediated communication like online distance learning. Despite this centrality, a coherent and nuanced theory of social presence is yet to emerge from the literature. Past research has shown associations with desirable affective variables like satisfaction and perceived learning, yet our knowledge as to when and for whom these effects are expected is still very limited. By introducing two contextual explanatory variables, we provide the means toward a more mature theory of social presence. The first variable, social presence divergence, relates students experiences to their preferences, yielding three distinct scenarios: too little, too much, and just the right amount of social presence. The second variable, interaction integration, considers the centrality of social interaction in the learning scenario, suggesting that this functions as a moderator. In a sample of teacher education students (N = 305), we find evidence that these variables interact with social presence and affective dependent variables as expected. These results add nuance and context to the discussion about the practical relevance of social presence. The implications of these findings as well as limitations of this study are discussed. Given the increasing importance of online-based teaching and learning methods, mediated communication has become near-ubiquitous in many learning experiences. Where physically co-located learning offers a rich social context and many ways to interact with peers, online and distance learning relies on technology for mediation , the how (predictors and mechanisms), and the why , a mature theory should also be able to explain the who, where, and when, thus allowing for predictions with a certain degree of nuance , social interaction among students (I), perceptions of social presence among students (P), contributing to a sound social space (S), hence the name SIPS model. The model has since found support in a replication of its overall structure . Interaction integration is such a contextual condition that is implied by the social presence literature but has not been explicitly investigated. For this reason, our second research question is:What is the role of interaction integration for understanding the relevance of social presence divergence with respect to outcome variables?RQ2: How did students perceive interaction integration in the course?What is the role of interaction integration in moderating the relationship between social presence divergence and outcome variables?RQ2.2: RQ2.1: Survey data was collected in a compulsory course on instructional design within in a teacher education program at a Turkish higher education institution. Pre-service teachers take this course to learn about information technologies in education, theories of educational technology, new developments in teaching with technology, and evaluation of technology-enhanced learning experiences. As such, the course is designed to cover an array of theoretical aspects but, at the same time, to be applicable to classroom practice. As data collection took place during the spring semester of 2021, the course was taught via emergency remote teaching, due to the Covid-19 pandemic. To ensure interaction and quality learning experiences, several communication channels and learning activities were in place to provide a rich and varied learning experience. Students and teachers communicated via synchronous as well as asynchronous channels, as there were regular online meetings and message board discussions that took place via institutions’ Learning Management System. Students were informed about the research project during a teaching module concerned with gamification and flipped learning. They were asked for participation in data collection at the end of a 2-hour synchronous session via Zoom, in which they were presented with learning content, the opportunity to assess their knowledge through a quiz, and a learning activity in breakout rooms, where they were tasked with analyzing and synthesizing of learning content. A total of N = 305 students participated. Of these, 213 (69.8%) indicated being women, with the remaining indicating being men. No further demographic information was collected.would have liked, and (5) interaction integration. For satisfaction, perceived learning, and perceived social presence, we use verbatim established scales  satisfaction with the learning experience and (2) perceived learning. In terms of predictors, we measure (3) the degree to which social presence was experienced during the learning activity, (4) the degree of social presence students  satisfaces Table .Scales for measuring our constructs of interest were translated into Turkish by three translators with different relevant expertise, one being an expert in educational technology whose mother tongue is Turkish but also speaks English fluently. A second translator comes from the area of measurement and evaluation, also speaking Turkish at first-language and English fluently. Finally, a linguist with very high proficiency in both Turkish and English. After translating independently, they discussed and resolved remaining discrepancies and agreed on the final item wording.X2/df ratio (good fit: ≤ 3), RMSEA (good fit: < 0.08), NFI (good fit: ≥ 0.95), and CFI (good fit: : ≥ 0.95). Results of CFA suggested well-fitting models for our constructs of interest with most indices clearing predefined thresholds  was conducted on scales. To this end, a separate sample of 193 pre-service teachers were recruited and asked to respond to our translated scales. Using the well-established benchmarks of fit indices by Hu & Bentler  as well Social presence divergence was calculated by subtracting perceived social presence from preferred social presence. Thus, the variable has a negative value if perceived social presence is less than what students would have liked in this learning scenario. Conversely, a positive value indicates that perceived social presence was higher than what students wanted. A value of zero indicates full convergence.To understand students’ perceptions of social presence divergence and interaction integration (RQ1 & RQ2), we first explored these variables on a descriptive basis by inspecting, among others, central tendencies, comparing subgroups, and producing graphs. To account for the potential existence of gender differences with regards to social presence divergence and interaction integration, statistical comparison of gender was done via a Mann-Whitney-U test as these subgroups were unequal in size, making equality of variance unlikely.To assess the relevance of social presence divergence with respect to outcome variables, we then first calculated bivariate correlations (H2.1) between them. Although some variables were negatively skewed , only desired social presence exceeded the value of -1 (skewness = -1.1). Thus, all variables remained below common cutoffs; i.e., between − 1.5 and 1.5  from considering social presence divergence alongside perceived social presence (H2.2), a hierarchical regression was conducted. This analysis was run twice to account for both dependent variables, satisfaction and perceived learning. For these analyses, assumptions of non-collinearity and heteroskedasticity were fully met. However, for the model predicting satisfaction it appeared that normality of residuals was not present. As our sample size was by order-of-magnitude larger than what is usually considered the threshold for the central limit theorem (n = 30), we considered this issue negligible for our purposes.Then, to assess added explanatory power , we ran a linear regression with the two independent variables social presence divergence and interaction integration, as well as their product as interaction term. Further simple slopes analyses with interaction integration as moderator yielded information about moderator levels  at which associations are augmented or attenuated. These analyses were also run twice to account for both dependent variables.We report the results of all analyses that were run twice in light of a manual Bonferroni correction, with an adjusted alpha level of 0.05/2 = 0.025.2 = 0.036, which corresponded to a small effect size according to Cohen  than men (M = − 0.91). This difference is statistically significant, Mann-Whitney U = 7629, p = .002. However, gender did not account for differences in perceived social presence, Mann-Whitney U = 9133, p = .347. This implies that gender difference in social presence divergence are driven mainly by preferred social presence, as reflected in a statistically significant mean difference, Mann-Whitney U = 7903.5, p = .007, with women preferring higher degrees of social presence (M = 4.3) than men (M = 4.02).How is social presence divergence associated with outcome variables?To probe associations of social presence divergence with affective outcome variables, we calculated bivariate correlations using Pearson’s r . To avoid confusion, we do not introduce the social presence divergence variable itself into the model but instead preferred social presence. This is because social presence divergence is directly derived from perceived social presence and preferred social presence and, thus, holds no further informational value. In other words, in terms of estimation, it does not matter whether we introduce social presence divergence or preferred social presence in addition to perceived social presence in model 2 as both yield the same results.R2 = 0.19 and R2 = 0.43, respectively  and a median of 3.4  and men (M = 3.54).Turning to interaction integration, we found students reporting relatively high values of this variable, with a mean of 3.49 , we centered our predictors in order to be able to report interpretable intercepts and estimates (VIF of interaction term = 1.02). Results suggested a statistically significant interaction effect  a divergence where perceived social presence was insufficient, (2) a divergence with an excess of perceived social presence, and (3) a convergence of perception with preference. Notably, only a small group of students perceived a perfect convergence (n = 19). However, if we define social presence convergence more leniently by allowing slight deviations, we find that a substantial number of students – about 30% – perceived a rough social presence match (n = 87). As this interval  is similar to the size of one standard deviation of perceived social presence (SD = 0.98) and preferred social presence (SD = 0.8), we judged this to be an appropriate margin to conceptualize social presence convergence in this sample.Regarding the question more-is-better view, as implied by the broader social presence literature and synthesized in Richardson et al.,  are highly significant, and thus robust to Bonferroni correction. This aligns well with our prediction that social presence divergence may be relatively inconsequential in learning scenarios that are less dependent on successful student-student interaction. If, on the other hand, students perceive this to be central to the learning experience, social presence divergence becomes a more important consideration and closing this discrepancy may be a priority goal for learning designers.Although these dependencies may appear rather intuitive or straightforward, they have hardly been discussed nor investigated in the social presence literature. One example of this notion may be found in Oyarzun et al.,  with theIn the CSCL literature, we can also find traces of implicit assumptions that interaction integration is consequential for understanding effects of social presence. For example, when Poquet et al.,  explaineAs a practical implication of these findings, we suggest that educators consider the degree of interaction integration in their learning scenarios. Although direct measurement of perceived interaction integration may be the ideal approach, it may not always be feasible in practice, especially ahead of time. For this reason, an expertise-based judgement on the interaction integration in the instructional design may be a good starting point to think about specific relevance of social presence --particularly social presence divergence– in their learning design. Based on this judgement, investments into enhancing social presence, for example by prioritizing instructional strategies to enhance social presence , we add nuance to the discussion and point to an area worthy of further investigation. At the same time, there has been a chasm between social presence research focused on CSCL scenarios versus research focused on “classic” online distance learning. With interaction integration, we introduce a concept that bridges this chasm and allows for conceiving of these disparate learning scenarios on a dimension that appears to be relevant for social presence research. We hope that these findings further stimulate the scholarly discussion of when, how, and to which extent social presence is relevant for delivering high-quality online distance learning experiences.


Recommendations for using evidence-based practices have become increasingly common in services for individuals diagnosed with autistic spectrum disorder (ASD). The aim of this study was to conduct a narrative literature review to identify differences and similarities in evidence-evaluation criteria for group and single-subject designs that empirically support interventions for people with ASD. Data sources used in this analysis were reports and articles elaborated by different clearinghouses . The criteria for evaluating evidence, as defined by these documents, contained specific components or quality indicators for each type of study design. The different criteria for evaluating evidence and for classifying the interventions  were identified and described. This manuscript discusses the need for (a) expanding the analysis beyond the evidence identified by different researchers and organizations such as the clearinghouses, (b) proposing interventions that are based not only on scientific evidence but also on social validity — which is directed by client idiosyncrasies, and (c) attention to the fact that EBPs should not be seen as static information regarding interventions with empirical support: evidence-based practices are the result of constant analysis of the intervention implementation data added to professional training and client values and context. Some additional issues and the study limitations are also presented. The diagnostic classification autism spectrum disorder (ASD)   which, up to this day, is based on the assumption that medical conducts should be implemented using the best scientific evidence that is available at a given moment.The EBM movement grew stronger with Sackett et al. ’s work iBesides medicine, other fields have been broadening their interest in the evidence-based practices (EBPs) paradigm. Among these fields are nursing  expanding the analysis beyond the evidence identified by different researchers and organizations such as the clearinghouses, (b) proposing interventions that are based not only on scientific evidence but also on social validity — which is oriented by client idiosyncrasies, and (c) attention to the fact that EBPs should not be seen as static information regarding interventions with empirical support; evidence-based practices are the result of constant analysis of the intervention implementation data added to professional training, client values, and context. Additional issues and the study’s limitations are presented in the conclusions. Below, in the main text, we describe the clearinghouses, as well as a summary of their review and evidence classification process. We organized their presentation in chronological order.The National Autism Center (NAC) is a nonprofit organization, dedicated to disseminating evidence-based information about ASD. The clearinghouse was created in 2006 and is considered May Institute’s Center for the Promotion of Evidence-based Practice. In general, NAC proposes to offer comprehensive and reliable resources for families, professionals, and various members of the community interested in ASD-related services.In 2009, NAC published the National Standards Project (NSP) — a synthesis report that describes phase 1 of its first systematic review with the aim of evaluating the body of evidence produced by experimental intervention research with individuals with ASD. They selected studies on focused and comprehensive interventions. Articles from peer-reviewed journals whose implementation environment was school, home, community, vocational, or clinical settings were included. Participants were children with ASD with no significant comorbid conditions. The reviewers identified 11 practices as “established treatments,” 22 as “emerging treatments” , and 5 that without enough evidence to guide the decision-making of professionals. Finally, the review did not identify any study that has proved ineffective or harmful to participants.class or strategy. The panel of reviewers regarded an intervention class as a combination of strategies that bared the main common features. An intervention strategy, in turn, consisted of therapeutic techniques that can be implemented in isolation.Importantly, NAC’s synthesis report (2009) presents its own definition of the term “treatment.” For them, the term could represent an intervention In 2010, a second review was published, which investigated the effectiveness of ASD interventions, authored by the  Odom et al. (https://afirm.fpg.unc.edu/).The findings of NPDC’s first review  to continue NPDC’s work.In 2015, NAC published the findings of the second phase of their search and assessment process of EBPs conducted in 2009. They aimed to perform a data update and verify whether any emerging interventions met the criteria to be considered established. They covered a longer period in years and expanded the age range of participants with ASD who took part in the primary studies to include those over 22 years old. Finally, the reviewers proposed an important term change to refer to interventions. In phase 2, instead of “treatment” or “strategies,” they began using the term “evidence-based interventions” as a label for evidence-based practices to be adopted by professionals in the applied field. The authors warned that although this terminology may differ from other similar research studies, this distinction is important to elucidate the difference between an evidence-based intervention and the broader structure of professionals’ decision-making processes when providing evidence-based practice services , 14 interventions for children and young adults with ASD were identified as EBPs. In addition, they described 18 practices as emerging and 13 with unestablished effects. For adults, they found one intervention that meets the EBP criteria, one identified as emerging, and four with unestablished effects.In 2020, the National Clearinghouse Autism Evidence Practice (NCAEP) completed a new systematic review that followed those performed by NPDC . In addition, it is important to note that NCAEP was assigned to replace and continue the work of NPDC, maintaining most of the criteria and modus operandi developed by the organization that preceded it. These results indicate that, to date, only the Frank Porter Graham Child Development Institute (FPG)  and the NAC have performed extensive analyses of intervention practices focused on children and young adults with ASD; only NAC analyzed comprehensive interventions.We must consider the differences regarding the period covered by different systematic reviews and the total interventions labelled as effective by each one. Table Another information that can be observed in Table Table The data published by NAC failed to specify the types of group designs used by primary studies. Only NCAEP (2020) identified a study that used the SMART design, although it had already been accepted by both previous NPDC reviews.Table Table Although NAC’s , 2015 anAccording to NCAEP’s (2020) report, multiple baseline is the most used among single-subject designs, followed by multiple probe, and a combination of designs . Randomized clinical trials were the most used among group designs. Also, NCAEP’s (2020) report shows a considerable increase in the use of alternated treatments, multiple probe, and RCT designs when compared to NPDC’s  report.To assess the experimental designs and their effects on primary studies, NAC developed their own assessment tools and strategies, which included three systems: (a) the Scientific Merit Rating Scale (SMRS), (b) the Intervention Effects Ratings, and (c) the Strength of Evidence Classification System.The analysis established by the criteria of SMRS generated a score from 0 to 5 points for each primary study. The scale evaluated five key evidence components, namely research design, measurements of the dependent variable (DV), measurement of the independent variable (IV), participant ascertainment, and generalization and maintenance effects.To assess the effect of interventions on participants’ target behaviors, NAC ; 2015 usThe Strength of Evidence Classification System describes the quality, quantity, and consistency of the body of evidence directed towards ASD. This was the last evaluation system, and it sorted the effectiveness of interventions into “established,” “emerging,” and “unestablished.” Scores 3, 4, or 5 on the SMRS plus beneficial effects for the target behaviors signal sufficient scientific rigor by the study. Thus, it is possible to draw reliable conclusions about its effects and to infer that future studies would obtain similar results. Score 2 signals initial evidence of the effects of the intervention with beneficial effects on some target behaviors, thereby indicating a need for further, higher quality research. Scores 0 or 1 signals insufficient scientific rigor, making it impossible to label an intervention as beneficial, ineffective, or harmful.Unlike NAC, to establish the review criteria for experimental designs and intervention effects, NPDC  and NCAEP (2020) used analysis protocols explicitly derived from the existing literature. Among the references used to create the protocols were the studies by Nathan and Gorman , Rogers Based on the different types of research designs, different criteria were used to evaluate and score primary studies. Table The main methodological parameters evaluated are related to (i) the expected minimum number of groups per study, (ii) the way participants are organized among the groups, (iii) the minimum number of participants, and (iv) the tolerance regarding data loss. Higher scores are attributed to the studies with greater methodological rigor on the various key components of the scale. Also, maximum scores (5) are attributed to randomized group studies with no data loss. Some data loss is tolerated even in studies that receive a score 4. Lastly, the minimum number of participants is only specified for studies with scores 4 and 5 and not even reported for lower scores. These results indicate that attrition bias and the demand for a high total number of participants seem to be less relevant methodological parameters when it comes to investigations on interventions for individuals with ASD.Table Table Table Other important data contained in Table Changes made to the assessment form on the latest systematic review , highlighted in bold in Table Table Clearinghouses define their classification categories by demonstrating the positive effects of interventions on participants’ repertoires. Interventions must have sufficient scientific evidence to demonstrate either their effectiveness or ineffectiveness. Interventions with little to no evidence are called “other practices with some support” by NPDC , “emerging” or “unestablished” by NAC , 2015, aTable The data on evidence classification and its criteria must be highlighted because, although all reviews fundamentally aimed at identifying evidence-based interventions for individuals with ASD, their classification is based on different definitions and criteria. Such differences must be considered by practitioners, who will need to analyze these definitions and criteria in a more technical way to know what each category refers to.Such a thorough analysis demands technical knowledge regarding the process of scientific production and its language which can constitute a barrier to its dissemination to consumers of services offered to individuals with ASD. On the other hand, a superficial analysis or the insensible consumption of information regarding EBP’s could generate misguided practices or, even worse, mislead the clinical practice in the field.In recent years, summary reviews have been published to describe interventions that show evidence of positive effects in children and young people with ASD. Such publications can be considered a product of an expanding search for professional activities compatible with the EBP paradigm. The purpose of the present narrative review was to identify differences and similarities between parameters used to evaluate studies using group and single-subject design. Such parameters are described in clearinghouses summary reports regarding interventions for people with ASD.Notably, science produces evidence in different ways, and different types of studies produce different levels of strength of evidence . The criteria used by NAC in both phases of their systematic review project were developed by the organization itself, and the report, despite citing classic studies . The tables presented here can assist the analysis of evidence by people with little or no training in reading scientific articles.At this point, we find it important to revisit the discussion on the function of systematic reviews when it comes to interventions for people with ASD. Several factors have influenced the emergence of the search for EBPs in this field among. One of these factors is the interests of different communities such as the scientific and professional communities and the stakeholders, including individuals with ASD. Systematic reviews should play an important role in the search for information that supports decision-making regarding interventions for this population. The results of such reviews should aid the translation of research findings into clinical practice , the effects obtained are likely to differ from those described in the literature  and articles such Carr and Nosik’s  and WeisAlthough a broad discussion about the point raised in this paragraph falls outside the scope of the present manuscript, we would like to emphasize that the use of EBPs by behavior analysts can be understood as an ethical duty for responsible professionals who work only within their training/experience limits and, therefore, consistently with the scientific principles that underpin our practice. However, as discussed earlier, aligning the provision of scientific evidence-based services with the client’s idiosyncrasies can be a challenging task. In the field of ASD, several pseudoscientific interventions as well as those with a body of emerging evidence are recurrently implemented by family members who seek to improve the life quality of a loved one. Behavior analysts may refuse to adopt practices that have no evidence and/or no support in behavior analysis. Interventions that are proven harmful  and chelation therapy), illegal , or go against fundamental therapists’ values should not be accepted or adopted. Nevertheless, as discussed by Broadhead , RosenbeWe must remember that in the decision-making process of EBPs, choosing an intervention with empirical support is only one of the components that constitute the paradigm in question (e.g., McGrew et al., It is important to discuss that an evidence-based clinical practice goes beyond the identification of scientific evidence available to support interventions (e.g., McGrew et al., In addition, it is important that researchers, implementation professionals, and stakeholders are able to critically evaluate the criteria for analyzing evidence-based practices (e.g., McGrew et al., Due to the substantial amount of data presented in the synthesis documents and the endless possibilities of analysis, it should be noted that the topics discussed here were chosen under the authors’ bias. Further analyses and discussions can and should be performed from different viewpoints. To this limitation, we add the issue of an absence of specific and replicable criteria for choosing the data analyzed here. In addition, we note that the discussion and syntheses provided here are more qualitative than quantitative. Finally, it is important to highlight that both primary studies as well as literature reviews and meta-analyzes need to consider the social validity of the knowledge produced, as it only becomes relevant if the evidence is relevant to practice (e.g., Green,


Purpose: To assess the quality of clinical practice guidelines (CPGs) related to drug therapy for prevention and control of ventilator-associated pneumonia (VAP) and compare the differences and similarities between recommendations.Methods: Electronic databases , guideline development organizations, and professional societies were searched to identify CPGs for VAP from 20 January 2012 to 20 January 2022. The Appraisal of Guidelines Research & Evaluation (AGREE) II instrument was used to evaluate the quality of the guidelines. The recommendations on drug therapy for prevention and treatment for each guideline were extracted, and then a descriptive synthesis was performed to analyze the scope/topic, and consistency of the recommendations.Results: Thirteen CPGs were included. The median score and interquartile range (IQR) in each domain are shown below: scope and purpose 72.22% ; stakeholder involvement 44.44% ; rigor of development 43.75% ; clarity and presentation 94.44% ; applicability 20.83  and editorial independence 50% . We extracted 21 recommendations on drug therapy for prevention of VAP and 51 recommendations on drugs used for treatment. Some controversies remained among the included guidelines.Conclusion: There is considerable variability in the development processes and reporting of VAP guidelines. Despite many similarities, the recommendations still had some inconsistencies in the details. For the prevention and treatment of VAP, local microbial epidemiology and antibiotic sensitivity must be considered, and recommendations should be regularly revised as new evidence emerges. Ventilator-associated pneumonia (VAP) is a special type of nosocomial infection typified by pulmonary parenchymal inflammation, which usually occurs 48 h after artificial airway or mechanical ventilation . VAP is Escherichia coli has high resistance to the third generation cephalosporins and fluoroquinolones, and Klebsiella pneumoniae has strong and widespread resistance to the third generation cephalosporins, Methicillin resistant Staphylococcus aureus (MRSA) was present in up to 90% of patients in some parts of the region. In Europe, Klebsiella pneumoniae is highly resistant to the third generation cephalosporins, MRSA was present in up to 60% of patients in some parts of the region  in April 2014 . It is re region . At presarly 60% . In 2001arly 60% . The UK arly 60% . In 2016arly 60% .At the same time, countries have implemented clinical guidelines to further standardize the medication use by professionals and the public. The National Institute for health and Care Excellence (NICE), a British Government institution, has formulated evidence-based clinical medication guidelines for antibiotics to guide the rational use of antibiotics and increase the clinical management of antibiotic use. At the same time, the Advisory Committee on Antimicrobial Resistance and Healthcare Associated Infection (ARHAI) in British released the clinical guidelines with the theme of “Start Smart-then Focus” . China hClinical practice guidelines (CPGs) are systematically constructed recommendations formulated to aid decision-making among medical professionals, which provide evidence-based recommendations for clinical practitioners and other healthcare professionals about the management of patients with diseases or other clinical conditions . They heTo date, there is still uncertainty regarding VAP management . Many CPRelevant guidelines were identified through computerized searches of PubMed, Cochrane library, Embase, Web of Science using a combination of text free terms and their corresponding Mesh terms, as well as three major Chinese academic databases. The search strategy is showed in All guidelines related to drug therapy for prevention or treatment of VAP published in English or Chinese from 20 January 2012 to 20 January 2022 were included. Documents were considered guidelines if they met the following criteria: : A guideExclusion criteria: Single-author overviews, editorials, letters to the editor, textbook-like publications, short summaries, documents without clear recommendations, and secondary publications (including versions translated from other languages) were excluded. If a guideline only applied to children, patients with immunodeficiency or COVID-19, it was also excluded.CPGs were evaluated independently by four assessors from different backgrounds, including one ICU expert (H-SW), two pharmacists , and one methodologist (X-CH). All assessors have extensive experience in evaluating CPGs using the AGREE II instrument. AGREE II consists of 23 key items organized into six domains . The scoWe developed a draft data extraction form which included document characteristics  and methodological features of the guideline . Consistent with the scope of this study, we also tabulated the information on drug therapy for prevention and treatment of VAP, which we used to determine if the recommendations only applied to VAP, as is the case of some recommendations, or if they were pertinent to VAP, however, all evidence supporting the recommendations came from VAP-related research.A descriptive analysis was performed by calculating each domain score and scaled domain score. Agreement among the four assessors was calculated by the intraclass correlation coefficient (ICC) with 95% confidence interval (CI) for each domain. According to the scale proposed by Fleiss, the degree of agreement between 0.00 and 0.40 was deemed poor, 0.41 to 0.75 was fair to good, and 0.75 to 1.00 was excellent . StatistA total of 3572 relevant documents were obtained in the initial examination, and 13 guidelines were finally included according to the inclusion and exclusion criteria . The floFor each guideline finally included, we systematically collected all relevant information and data. All guidelines were evidence-based. Five guidelines were updated versions . The genFour assessors independently assessed the 13 guidelines with an ICC value of 0.82 (95% CI = 0.73–0.87), which indicated a high level of reliability among assessors. The quality of the guidelines varied greatly, from fulfilling most of the AGREE criteria to only fulfilling an unsatisfactory number of items. Across all guidelines, none of them had high scores for all domains, and the assessors assigned the highest score to the domain of “clarity of presentation” and the lowest score to “applicability”. The ERS 2017 guideline  ranked hThe median score and interquartile range (IQR) of this domain was 72.22% , The highest score in this domain was 83.33% , and theThe median score and IQR of the stakeholder involvement domain was 44.44% , The highest score in this domain was 77.78% , and theThe median score and IQR of the rigor of development domain was 43.75% . The highest score in this domain was 79.17% , and theThe median score and IQR of the clarity of the presentation domain was 94.44% . The highest score in this domain was 94.44% , and theThe median score and IQR of the clarity of the applicability domain was 20.83% , which is the lowest score of all items. The highest score in this domain was 50.00% , and theThe median score and IQR of the editorial independence domain was 50% . The highest score in this domain was 83.33% , and thep < 0.05 was considered a statistically significant difference. The results showed that there was no significant difference in the six domains of AGREE II. See All guidelines reported explicit grading for the strength of the recommendations. Nine guidelines reported that they used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) tool for evaluating the quality of evidence and forming the final recommendations . Two guiWe extracted 21 recommendations regarding drug therapy for prevention of VAP from 13 guidelines, including 5 strong recommendations, 5 moderate recommendations, 5 weak recommendation, 3 special approaches, 2 definitively not recommended, and one no formal recommendation. VAP has specific risk factors and pathogenesis. The recommendations that resulted from interpretation of the evidence varied among guidelines. There were 3 recommendations related to enteral nutrition for the prevention of VAP. Two Chinese guidelines  recommen5 guidelines  made the7 guidelines  related 3 guidelines discussed the use of prophylactic probiotics. Two Chinese guidelines  do not rTwo guidelines  have recThe CMA 2013 guideline  recommenWe extracted 51 recommendations regarding drug treatment for VAP from 8 guidelines , includiWe extracted 21 recommendations regarding empiric therapy for VAP from 5 guidelines . For empIf the pathogen is considered empirically to be multidrug resistant bacteria, antibiotic combination treatment can be selected. The IDSA 2016 guideline  gave indin vitro drug sensitivity tests. The IDSA 2016 guideline  should be formulated with reference to the results of uideline , CMA 201uideline  and IDSTuideline  give detAcinetobacter Baumann was found to be susceptible to sulbactam (SBT) and ampicillin (ABPC) which was recommended as a first-choice drug for respiratory infectious diseases  in the JAID (Acinetobacter Baumann infection (extensively-drug resistant or pan drug resistant), a combined regimen  should be used. The IDSA 2016 guideline cautioned against the use of tigecycline in patients with hospital-acquired pneumonia (HAP) or VAP caused by Acinetobacter species, however they did not recommend any specific drug for Acinetobacter Baumann. This was based on evidence synthesis which indicated that the dose currently recommended on the label of tigecycline worsened clinical outcomes compared with several other therapies. The panel’s strong caution against tigecycline, despite low-quality evidence, was intended to emphasize the importance of avoiding potentially harmful therapies, particularly when alternative choices exist. This is in sharp contrast to the Chinese guidelines.the JAID guidelineFive guidelines  coveringFive guidelines  relate tSince October 2016, the National Health Commission of the People’s Republic of China has initiated work to construct a national clinical practice guideline database aiming to promote CPG development, dissemination, and implementation in China. CPGs aim to formulate specific, explicit recommendations that, if properly adopted in clinical settings, will produce better outcomes for patients, and promote cost-effective practices . The preThe methodologist plays a key role in guideline development meetings by helping the guideline development group formulate recommendations informed by the evidence in a transparent and explicit manner . Over thThe low score in the field of applicability indicates that the guideline expert group regards the development and implementation of the guideline as a separate issue and does not pay enough attention to the potential obstacles in its promotion and dissemination . When reGuideline recommendations should be based on the balance between the estimated costs of the interventions or services and their expected benefits compared with an alternative . AlthougAt present, VAP prevention measures focus on the pathogenesis to reduce the occurrence of VAP and improve the prognosis of patients. The recommendations for preventing VAP for these guidelines include non-drug prevention and drug prevention. Non-drug prevention mainly includes semi-recumbent position; use of new endotracheal intubation and subglottic secretion drainage; reducing the use of invasive ventilation, shortening the time of invasive ventilation, and limiting the use of narcotic drugs .For drug prevention of VAP, we found that some disparities remain among the included guidelines. When there are substantial differences in major recommendations of guidelines, patients and clinical practitioners may question the validity which may then lead to poor adherence and implementation . VAP druAt present, there are still many disputes about the specific scheme of selective purification and its clinical application in areas with different antibiotic resistance levels . AlthougRecent studies have also questioned the efficacy and safety of oral chlorhexidine. Although there is evidence that its use as a gargle may help to reduce the risk of VAP, but this is unclear in the non-cardiothoracic ICU population . Some stThe application of probiotics in patients with mechanical ventilation is still controversial. Two recent meta-analysis have shown that the application of probiotics helped to prevent VAP without impacting the length of ICU stays or mortality . HoweverThe treatment of VAP includes two aspects: first, empirical treatment, which needs to consider the severity of the patient’s disease, suspected pathogens and MDR risk factors; second, etiological treatment. Indeed, some evidence used by those guidelines was derived from the same trials or systematic reviews, meaning that they are based on shared evidence. Variations could be the result of differences in data interpretation or the indication of available resources, while actual recommendations could still be tailored to local and cultural contexts. Nonetheless, one needs to be cautious when considering guidelines for local use and should make sure that the clinical data and evidence are in harmony with the clinical judgment.acinetobacter baumannii in the VAP pathogen spectrum was as high as 35.7–50.0%, followed by Pseudomonas aeruginosa and Staphylococcus aureus, and the isolation rate of Acinetobacter Baumann has been reportedly increasing year by year (Acinetobacter Baumann to various antibiotics was at a serious level (S. aureus (approximately 20–30% of isolates), P. aeruginosa (approximately 10–20% of isolates), enteric gram-negative bacilli (approximately 20–40% of isolates), and Acinetobacter Bahmani (approximately 5–10% of isolates) (The pathogens found in VAP patients are mainly Gram-negative bacteria . Antibio by year . By the us level . Howeversolates) . UnderstAntimicrobial stewardship is an important issue; however, the approach to manage it differs considerably in different countries. Currently more and more attention has been paid to the deterioration of VAP-associated death rates in China. The use of antibiotics in China faces severe challenges, and in recent years the National Health Commission of the People’s Republic of China has continued to issue notices on further strengthening the management of the clinical application of antimicrobial drugs to curb bacterial resistance . The aimVAP will continue to be the main infection in ICU in the next decade . This paOur study possesses several strengths. First, our research team consisted of methodologists with full experience in the development and assessment of CPGs and clinical experts and obtained consensus from all appraisers to ensure the reliability of our conclusions. Second, we conducted a thorough systematic literature search. Third, we extracted and compared recommendations for drug use for VAP.Nevertheless, several limitations should be noted. First, we only assessed guidelines published in the English and Chinese languages and on some important professional society websites, which may not represent all the guidelines for VAP. Guidelines published in other ways  may have been omitted, which may have introduced bias into our assessment. Second, we attached relatively more weight to the quality of guideline development than to whether the recommendations were feasible in our specific practice environments or matched a particular clinical practice We extracted recommendations on drug therapy for VAP prevention and treatment; however, many guidelines were developed for both HAP and VAP, and their recommendations did not differentiate between the two mostly because some guidelines considered VAP to be a special type of HAP, and primary studies included a mix of HAP and VAP samples. In this study, we recognized that patients with HAP and VAP belong to 2 distinct groups and extracted the recommendations that only applied to VAP or recommendations that were described as pertinent to VAP/HAP with evidence indicating the recommendation came from VAP-related research.In conclusion, there is considerable variability in the development process and reporting of VAP guidelines, although the principles for guideline development have been described. The experience of the organization and experts in assessing evidence and developing guidelines may explain higher scores for some items. There were substantial differences in some recommendations of VAP guidelines. For the prevention and treatment of VAP, local microbial epidemiology and antibiotic sensitivity must be considered, along with economic issues. The most effective clinical practice guidelines should incorporate the current best evidence and place these in the context of local patterns of drug resistance. Recommendations should be regularly revised as new evidence emerges.


Cultural adaptation of psychological interventions has been discussed controversially in literature. On the one hand, culturally diverse groups are underrepresented in psychological trials, and evidence on acceptability and efficacy of interventions cannot necessarily be transferred from one cultural group to another . On the There is also considerable debate about what to adapt, and the effect of such adaptations. Empirical evidence on substantial modifications is scarce, and cultural adaptation methods are often insufficiently reported in literature . This doIn this special issue, experiences and empirical evidence on the cultural adaptation of psychological interventions for refugee populations are brought together. In 2016, the German Federal Ministry of Education and Research (FMER) launched a call for research proposals covering the ‘mental health of refugee populations’. Seven research projects were funded. One exclusively focuses on diagnostics, and the other six projects test evidence-based psychological interventions. Each of those six projects consists of three or more sub-projects, which are testing diagnostic tools, the efficacy and cost-effectiveness of interventions, and implementation methods. A total of eleven randomised controlled trials (RCTs) are implemented to test different kinds of psychological interventions among a diversity of target groups, i.e., age groups, specific disorders, or unspecific psychological distress.In the context of the FMER call, a “task force for cultural adaptions of mental health interventions for refugees” was launched. It pursued two major goals: First, it aimed to develop a common understanding and methodology for documenting and monitoring cultural adaptations in clinical trials. Second, it aimed to integrate the findings of the first step and compile criteria on how cultural adaptations in clinical trials could be reported. The conceptual framework for cultural adaptation by This special issue features experiences, empirical evidence and recommendations resulting from this task force, as well as the final composition of the reporting criteria. The paper by The process of adapting an already culturally-sensitive, transdiagnostic treatment to also include substance use disorders is described in the paper by Reporting Cultural Adaptation in Psychological Trials (RECAPT) criteria. The RECAPT criteria were developed jointly by the above-described task force. To achieve a broader consent on the RECAPT criteria, an online survey was conducted among eleven international experts in the field of global mental health and psychological interventions for refugee populations.Finally, In summary, this special issue features the experience of a variety of studies in which a diversity of psychological interventions were culturally adapted and tested among refugee populations in Germany. The task force provided a unique opportunity for exchange and discussions, with the aim of advancing the emerging field of cultural adaptations in mental health interventions. The RECAPT criteria (


To better understand and plan health systems featuring multiple levels and complex causal elements, there have been increasing attempts to incorporate tools arising from complexity science to inform decisions. The utilization of new planning approaches can have important implications for the types of evidence that inform health policymaking and the mechanisms through which they do so. This paper presents an empirical analysis of the application of one such tool—system dynamics modelling (SDM)—within a tuberculosis control programme in South Africa in order to explore how SDM was utilized, and to reflect on the implications for evidence-informed health policymaking. We observed group model building workshops that served to develop the SDM process and undertook 19 qualitative interviews with policymakers and practitioners who partook in these workshops. We analysed the relationship between the SDM process and the use of evidence for policymaking through four conceptual perspectives: (1) a rationalist knowledge-translation view that considers how previously-generated research can be taken up into policy; (2) a programmatic approach that considers existing goals and tasks of decision-makers, and how evidence might address them; (3) a social constructivist lens exploring how the process of using an evidentiary planning tool like SDM can shape the understanding of problems and their solutions; and (4) a normative perspective that recognizes that stakeholders may have different priorities, and thus considers which groups are included and represented in the process. Each perspective can provide useful insights into the SDM process and the political nature of evidence use. In particular, SDM can provide technical information to solve problems, potentially leave out other concerns and influence how problems are conceptualized by formalizing the boundaries of the policy problem and delineating particular solution sets. Undertaking the process further involves choices on stakeholder inclusion affecting whose interests may be served as evidence to inform decisions. Key messagesSystem dynamics modelling (SDM) is increasingly being used to inform health policy and systems decision-makingThe formal implementation of SDM can serve as a knowledge translation and evidence provision tool for policymakingThere are multiple ways to consider how SDM might affect evidence-informed policymaking, including: as a direct provision of information, as a way to identify needed evidence, or as a process by which possible problems and their solutions can be formalized.Planning health system activities can be a challenging process, requiring information about a range of variables and relationships that can shape service delivery and ultimately clinical outcomes for populations. To assist health sector decision-makers with such issues, there have been increasing efforts to engage with complexity sciences and the tools this field offers . As a toThere has been some work reviewing the application of SDM to characterize health systems . HoweverEnd TB Strategy outlines three strategic pillars to help achieve this—one of which is integrated patient-centred care and prevention, encompassing TB infection prevention and control (IPC)  .IPC measures seek to prevent patients and healthcare workers from avoidable infection . For TB Within SDM literature, this set of problems could be referred to as ‘messy problems’ given their ‘tangled’ nature and lack of straightforward definition or solution , p. 71. SDM is recognized as having a clearly defined methodology for problem-solving and has therefore become increasingly applied to understand policy problems and guide policy decision-making in several fields . The metAt a group level, GMB’s goals include: (1) learning by helping people gain more insight into the structure and behaviour of the system; (2) creating agreements/consensus about policy decisions; and (3) generating commitment with a decision . GMBs arIn this section, we outline four distinct conceptual perspectives which exist in the broader literature theorizing the use of evidence for policymaking. Each of these will be taken to reflect on SDM’s use in relation to evidence and policymaking within the GMB intervention in our specific case study. A second perspective is to take a so-called ‘programmatic approach’ to evidence utilization  project (translation ‘good air’ from isiZulu) which was undertaken from 2017 to 2020 with an aim to identify the diverse drivers of nosocomial transmission of DR-TB in South African primary care facilities. As TB IPC represents a complex intervention that is delivered within a dynamic context shaped by myriad factors such as policy guidelines, organizational culture and resource availability, the UO project used a ‘whole systems approach’ to examine social, biological and infrastructure-related transmission dynamics in two South African provinces to inform health systems interventions . This included data on clinic infrastructure, space and resources; on the policies, norms and values governing processes of care and IPC for staff and patients; and on the prevalence of TB among clinic attendees. Once primary data had been collected, UO facilitated a GMB exercise consisting of two workshops with high-level national and provincial policymakers, public health professionals and patient advocates engaged in DR-TB service design and delivery. An initial list of participants was drawn up by researchers to capture a range of diverse insights into the complex problem of TB IPC in South Africa. To capture policy insights, participants responsible for designing clinics and infrastructure and TB- and HIV-related policymakers were purposively targeted. To capture clinic-based perspectives, nurses, district health managers and occupational health representatives were invited. Convenience sampling was used where participants from the original list could not attend. Overall, nine policymakers, 15 practitioners and nine researchers participated in each of the two workshops. While participants were informed of the study’s purpose and research activities before the GMB workshops, most were not familiar with the SDM process.The first workshop held in person in August 2019 focused on the development of causal loop diagrams (CLDs). On Day 1, policy- and decision-makers at the national and provincial level focused on identifying distal and broader macro-level health system and policy influences on TB prevention and care. As primary data was not distributed ahead of the workshops, a variable elicitation exercise was also conducted to identify key events and factors influencing TB burden in South Africa. On Day 2, practitioners focused on identifying proximal factors related to TB care delivery and nosocomial transmission which could be linked to Day 1’s identified factors. For each participant group, a researcher and expert modeller then guided the development of an initial CLD and assisted with the identification of points of fragility and intervention .The second workshop was conducted virtually due to COVID-19 restrictions in October 2020 and reviewed both the model generated as well as the preliminary mathematical and economic modelling results. UO researchers were embedded in the GMB to be able to critically feed in insights from the collected data. Participants then discussed core intervention mechanisms and researchers further elaborated the mechanisms and, at follow-up calls, repeatedly checked in with participants to validate.For this paper, a sub-study was conducted around the GMB workshops. Individual interviews with participants involved in these workshops were undertaken face to face after the first workshop, and via phone/video call after the second. We interviewed eight policymakers and three practitioners after the first workshop and five policymakers and three practitioners after the second workshop see . As manyObservational notes from the first GMB workshop, video recordings of the second workshop and post-workshop interview data were analysed using thematic analysis. After interviews had been transcribed, they were imported into DEDOOSE software for coding by SP and JP. Interviews were coded using a framework based on programmatic and constructivist concepts as detailed in the conceptualization section above and were refined throughout the data analysis stage. An iterative approach was taken to identify the main themes and sub-themes as they link back to the four perspectives, with the assistance of co-authors  shown in . RelevanIn this section, we present the key findings of this sub-study according to the conceptual concerns detailed above. First, in line with understanding SDM as a KT tool, we consider how its use is linked to evidence use from a rationalist perspective. In relation to the programmatic perspective on SDM, the second section explores participants’ views on what they considered relevant evidence in relation to programmatic tasks and goals. Third, in line with the post-structuralist concerns with problem-structuring, we explore how the research objectives and GMB process set boundaries around the evidence utilized in the process. And finally, in the fourth section, we address concerns over representation by exploring which stakeholders were accounted for within the GMB process.From a KT perspective, the problem of interest is often specified in advance. From this starting point, SDM can provide decision-makers with ways to apply data developed by researchers, but also, through its participatory engagement via GMB activities, clarify for decision-makers where key informational gaps might lie and how to fill them through research efforts for those pre-defined problems. We identified two ways in which this occurs: (1) SDM can serve as a scoping/mapping tool, and (2) assist with gap-identification.During the GMB workshops, researchers provided evidence from the broader UO project to guide participants in the development of a preliminary CLD presenting broader factors influencing TB care systems. This diagram was then used to reflect on areas for intervention to improve TB IPC. One key way this enabled the application of useful evidence was by providing a framework for understanding how information fits into the problems and solutions being discussed. For example, as one policymaker explained:I think it’s [SDM] a good tool because it throws all the elements in the system, you know, because they are things that … kind of human resource dependent, and things that are dependent on management or maybe enforcement of policies that exist. But the things that … kind of related to patients themselves … the things that relate to more of infrastructure and logistics, that should be there in our facility. So I feel it kind of dissects it so that you can see what is the gap or what is not working well, so that at least your intervention can be kind of focused, and kind of helps to see where you can get more fields. Because it doesn’t help how to implement something that’s so costly, that will move maybe the needle by 2%. So you rather … allocate your effort and your resources knowing that if I do this, and they do this correctly, at least it can help me in kind of improving so much. This excerpt reveals the nature of the ‘messy’ problem of TB IPC in South Africa, and one gets a sense of the many sectors and resources required to address the issue. As a KT tool, SDM helped to expose the disconnect between problems, solutions and current resource allocation, as well as providing an opportunity to ‘see’ the issue more clearly with all component pieces placed together, using the visuals and data provided. Similarly, another policymaker stated:[SDM is] like a scoping project as to where to source your various sources of information. source the information they require to solve the specific problems discussed. Additionally, it can provide information from a research team, while also providing a way to map out data or informational needs, and potentially helping to think through where sources of useful data might be found to solve the problem.While SDM primarily serves to address strategic problems, it could also help to highlight where stakeholders can In addition to helping identify sources of information, the process of mapping out the complexities of systems was also seen to help identify policy-relevant information gaps. For example, in one interview, a practitioner discussed the discrepancy between guidelines (and the research that informs these guidelines)—and what happens ‘on the ground’ within clinical settings. They explained:I really do wish that [the] gap between policymakers and people who think that they know what is in the best interest of a facility and for clinicians and patients … could be addressed. The GMB workshops provide an opportunity for stakeholders to raise their concerns about gaps such as these and to link them to evidence-informed solutions. Several of the workshop participants also suggested that the SDM process was helpful in identifying lacunae in information by hearing other stakeholders’ viewpoints. One policymaker noted:What [GMB] does, is it gets around the table … [things] that you didn’t even consider … it takes you on a journey or process, where it opens up other doors that normally we [policymakers] [would] just walk past. The GMB workshops were able to facilitate communication and engagement between participants, but created room to feed back evidence needs to researchers, creating a dynamic, iterative space for information-sharing. One practitioner, for instance, stated that ‘community engagement’ was missing and ‘essential’ to add to the model, which could then be addressed by researchers almost immediately .Overall, the SDM process served KT functions in a number of ways. Firstly, system complexities can be visually mapped providing a simple, broad representation of the various components that influence complex issues, generating shared understanding between different stakeholders about where pieces of data would fit into their needs. Secondly, it identified gaps in research knowledge and provided decision-makers the opportunity to feedback thoughts on those gaps or evidence needs to researchers. Lastly, it can provide a mechanism for the direct provision of research results to decision-makers, but also help to identify where information to address problems can be sourced.In contrast to the KT approach which starts from a goal of getting uptake of previously gathered data and evidence, a programmatic perspective starts from the recognition that decision-makers may have multiple tasks and goals, each of which may have implications for which evidence is considered policy-relevant. We thus also investigated which programmatic goals the SDM appeared to provide useful information on from the perspective of participants. Within our interviews, we specifically asked several questions to allow policymakers and practitioners to identify their various roles and to reflect on evidence in relation to them.Participants held multiple roles and carried out a variety of different tasks, however most were involved in some form of programme management. As one policymaker explained, their focus was to:prevent or control the spread of TB, make sure those that have [TB] are on treatment and … make sure that those [who] don’t have it, don’t get it, especially in our health facilities. For both policymaker and practitioner categories, programme management included tasks related to ‘facility management or clinical governance’, the provision of training and the organization of human resources, in particular ‘recruitment and selection … induction and orientation’ activities . Programme management also involved budget management, that is:[going] over specific percentages for the month then [projecting] an overspending. If we are not overspending, we can project an underspending which means we can push money from one line item to another line item. Other participants were tasked with focused monitoring and evaluation activities. For one policymaker, this meant coordinating the introduction of an integrated information system which aims to ‘improve patient management and care, clinical governance, as well as improve data quality’ . This led to quite specific conceptions of what relevant data or evidence was. For example, one practitioner utilized a tool called the Ideal Clinic checklist for their monitoring and evaluation task . As a weThis variety of tasks and goals of participants, and the relevant evidence seen to assist in achieving them, has implications when reflecting on the role of the SDM process. Two tasks that SDM was seen to be able to facilitate were monitoring and evaluation and programme management. As one policymaker involved in conducting monitoring and evaluation activities explained, ‘If you use system modelling, you identify challenges, and then it also guides you [on] how to address each of those challenges’ .Another policymaker engaged in monitoring and evaluation echoed this, saying that SDM could help ‘analyse the problem’, find ‘new ways of doing things’ and ‘fill gaps and possibilities’ in implementation . Similarly, speaking with reference to how SDM can assist with programme management, one policymaker said that ‘[SDM] assists us to plan ahead’ and explained that SDM, unlike other tools that might utilize a more linear approach, allows for ‘cyclical planning’ .Despite the widespread perception among participants that SDM was able to facilitate a variety of different tasks, some felt that it was not always the most effective approach for the types of decision-making they had to undertake. One reason for this was that the SDM process was not seen as being especially practical for the context. As one practitioner explained:I think there were some things that are a little bit far-fetched and very theoretical, that I think wouldn’t work in our context. Another practitioner touched on the issue of SDM’s incompatibility with this ‘context’ in further detail:I think your biggest challenge is to get people to buy in … Not everyone, including myself, [was] exposed to systems thinking in school … So, the one challenge is everyone is not exposed to … thinking in that way. Two, not everyone understands it. I think number three, within the systems model thinking, it’s heavily linked to digital … like utilizing of a computer … and a lot of our staff are not computer literate. Others echoed this sentiment by saying that it was ‘not user friendly’  and that it would require more training workshops to get all members involved in the GMB workshops on the same page . Thus, from a programmatic perspective, SDM was seen as providing potentially useful information for some but not all practitioners’ goals, yet it might not provide the form of evidence most conducive to the teams involved.The third approach to reflect on evidence use is to recognize that the process of evidence utilization itself can serve to construct or formalize the bounds of the policy problem itself, establish conceptual understandings and delineate particular solution sets for consideration.One way to see this unfold is in how the SDM language used during the GMB workshops could lead to participants adopting certain terminology to think through problems and solutions. For example, one practitioner stated:[SDM] would be a tool for me to build specific processes and systems. [If] I’ve volunteered to use the systems dynamics tool, then I will utilize it to build the specific processes and systems and flows to achieve what has to be achieved … And then taking from that, everything else will then flow and become the system that we envision … you know, we built, for example, double loop systems and a double loop system is … if a client complained about receiving expired medication, we want to do a root cause analysis as part of the whole system dynamics … and then once we understand the problem, we innovate, and it’s called double loop because now we understand the problem, we’ve innovated and we can prevent it from happening again. Here, the participant uses SDM-type language such as ‘processes’, ‘flows’ and ‘double loop systems’ to think through how SDM could be an integrated process embedded in the organization and functioning of the health facility that they manage. Providing knowledge of the SDM process also gives people a new framework with which to see how existing systems function—or where the gaps are—in a novel way.However, while participants in the GMB workshops brought their own understandings of the problem of TB transmission, as well as of how they thought it could be addressed, the objectives of the research project itself played an important role in shaping how the problem was framed and solutions constructed. For instance, the second GMB workshop presented information on three central areas where specific dynamics were found to affect nosocomial transmission: (1) actual drivers of transmission at the clinic level; (2) the clinic working environment; and (3) the broader policy environment which shapes the clinic environment and the types of programmes implemented at the clinic level. The workshop also included the presentation of results from the mathematical modelling of various interventions (clinic and community), and SDM was presented as being useful for costing in order to facilitate a priority-setting exercise to: (1) allocate scarce resources efficiently to maximize health benefits; (2) adapt to local needs ; and (3) rank and optimize across multiple interventions.These elements, however, represent only a sub-sample of all the possible pieces of information that might be considered policy-relevant evidence, depending on which tasks, goals and construction of the problem are being considered. For example, Ultraviolet Germicidal Irradiation (UVGI) was one intervention presented by researchers and discussed during the second GMB workshop. However, one participant acknowledged that they did not have enough knowledge about UVGI to fully comprehend its benefits:For me, the discussion around UV especially in the context of what other benefits it brings … my knowledge on this needs to improve … besides TB what other possible advantages [does] it bring? This example of the limitations of knowledge regarding the interventions developed during the SDM process, informed by evidence generated by the researchers, may have implications for which ideas and approaches to infection control became realized in the discussions.Finally, once there is recognition that different policy needs exist, and that problems and policy-relevant evidence can be constructed in different ways, it follows that it may be particularly important to consider which groups or individuals are involved in the processes leading to such constructions—to consider if the values or priorities being embedded into these constructions are representative of the population or the wider community for whom the policies are meant to serve.The SDM process enables various stakeholders to participate in a process of multi-disciplinary teamwork during GMB workshops to identify problems and solutions. Indeed, the inclusion of multiple stakeholder perspectives was emphasized by some of the interviewees as being a major benefit of using SDM for policymaking. As one participant explained:What interested me with that modelling [GMB exercise], as complicated as it is … is that it includes many stakeholders. Because, for example, every workshop, they make sure that government is involved, labour or professional associations are involved, victims or people who were infected were part of that … representatives from communities were involved in that … so they took all the inputs of everyone and [made] sure that [the] model tried to accommodate each level of understanding. This multidisciplinary involvement during the GMB workshops was seen to be beneficial, not only because various stakeholder views were represented, but also because it enabled some participants to feel as although they were working together towards a common objective:[The] benefit of [SDM] is that we need to work as a team. We can’t work alone, because as a team … you need engineers, we need a patient advocate … Because all of us in health … are focusing on health … whether you are a healthcare worker, an information person, an infrastructure person … We look at those dynamics … it assists us to work as a team and achieve our objective as a team, together. While GMB may have assisted participants to ‘work as a team’ towards a common objective, there could be differences in perspectives shaping perceptions of relevant evidence. For instance, one policymaker spoke of the ‘policy perspective’ when discussing the application of relevant evidence to refer to all the pieces of information that might inform policy decision-making . Evidence in the form of ‘utilization patterns, cost-effectiveness and survey acceptance’ all seemed to illustrate a ‘top-down’, ‘government perspective’ to the application of relevant evidence . Other policymakers also referred to the need for ‘the most recent evidence of models that work’ , training manuals informed from evidence ‘based in different countries’  and evidence generated from monitoring and evaluations checklists such as the Ideal Clinic checklist .This perspective contrasted with some practitioners, however, who referred to the application of relevant evidence that was more facility-specific. For example, as one practitioner explained:facilities are now capturing their own TB data … immediate data that’s at the facility level and subdistrict level makes it a bit easier for us to plan. We have facilities we want to roll out our TB services to, so obviously we need to take into account what the community level of TB infection is … and depending on that, we … will staff appropriately, get things in order and then protect our staff as well. A similar sentiment was expressed by another practitioner who felt relevant evidence was ‘good-quality data’ that could be used to develop ‘a strategic plan for the facility, as well as an operational plan’ . From both accounts, we see the distinction in the types of evidence deemed relevant by policymakers and practitioners. As such, the idea that SDM can facilitate gap identification and that it occurs through a process of multidisciplinary teamwork could be made more effective by providing standard information to all stakeholders preceding GMB activities to ensure all participants are on a level playing field.As a methodology, SDM is progressively being used in policy spaces as a framework to delineate problems and plan their associated solutions as well as to identify pieces of information deemed policy-relevant. In doing so, there are a number of implications in relation to key considerations raised by scholars concerned with the use of evidence for policy-making more broadly. While many authors have previously recognized that the GMB approach serves as an effective communication tool and develops a shared understanding of ‘messy’ problems  specific problems and solutions rather than ones that could have potentially been seen if an alternative approach were taken . This is not to judge in one way or another which is a better approach. Pre-defined categories and problem constructions allow clarity and prioritization of certain variables and considerations, which the SDM process can potentially serve. But these results help to provide an example of how particular constructions subsequently direct problem definition and evidentiary considerations in one or another direction—and how they could exclude alternatives as well.The growth in the application of planning tools deriving from complexity science to inform health and broader public policy decision-making is likely to continue. Yet with the health sector’s well-established embrace of the idea that policies should be informed by evidence, this study has aimed to explore how the process of undertaking an SDM process can function in relation to evidence provision in a specific health system context. This is done in recognition of a range of critical considerations that policy scholars have raised in the policy sciences in relation to evidence utilization for public policy.In our case, SDM was clearly a useful tool and process to help facilitate the understanding and utilization of research evidence in many ways. Yet as the conceptual discussion and analysis point out, there are other considerations around evidence use beyond KT that can also be considered. A programmatic approach to evidence use starts by reflecting on the multiple goals and tasks pursued by decision-makers, to consider which evidence is most relevant in relation to those. Furthermore, constructivist scholars raise questions about how the use of particular forms of evidence can serve to frame and construct policy problems and their solutions; while those concerned with the propitiation of competing social values raise questions about representation amongst groups involved in evidence use. Each of these lenses can provide insights into ways that the SDM process relates to evidence use, and the political nature of that use in practice. Such critical perspectives will be important to consider as planning tools such as SDM and GMB continue to increase in popularity and in their application as a policy-informing tool—both in the health sector and beyond.


Inadequate health and nutritional literacy is a common problem among adults, associated with poor health outcomes. Therefore, this study aimed to investigate the relationship between health literacy and nutritional literacy to sun exposure behaviour.We conducted a cross-sectional study on 261 adults (18–65 years) in Iran. Data was collected on knowledge, motivation, health literacy, nutritional literacy, and sun exposure behaviour using an interview-assisted questionnaire. Using the information–motivation–behavioural skills model and structural equation modeling, we tested whether health and nutritional literacy were associated with the relationships between knowledge of vitamin D, attitudes toward sun exposure, and sun exposure behaviour. Different models using structural equation modeling were performed to analyze the data.β = 0.29, p < 0.001) and nutritional literacy  was directly associated with sunlight exposure. Indirect relationships also existed between knowledge and sunlight exposure through health literacy  and nutritional literacy . The model had good fit . There was no significant relationship between health literacy and motivation , nutritional literacy and motivation  and motivation and sun exposure .The finding showed that health literacy (The findings showed that individuals with sufficient health literacy and nutritional literacy were more likely to have exposure to sunlight. Health and nutritional literacy should be considered when educating adults about vitamin D supplements and sunlight exposure. In recent decades, numerous investigations have reported the prevalence of vitamin D deficiency worldwide –3. VitamNutritional literacy is the skill and the ability to access, and understand information about healthy nutrition and use it to have a healthy diet and lifestyle . PreviouHealth literacy is a personal and social capacity for access to understanding, assessment, information, and health services, and optimal utilization for the promotion and improvement of health . Based oHealth literacy and nutritional literacy may play a role in the relationship between knowledge, motivation, and sun exposure behavior. Since there has not been a study on the relationship between health literacy, nutritional literacy and sunlight exposure, the present cross-sectional study aimed to investigate such relationships using structural equation modeling (SEM) .This cross-sectional study was carried out on a sample of Iranian adults who referred to health centres in Tehran, Iran from February to July 2018. The inclusion criteria were: being adult men and women aged 18 to 65 years, having reading and writing abilities, and being able to participate in social activities. Participants were selected using two-stage cluster sampling from existing health centres in Tehran. Health centres were divided into five regions: North, South, East, West, and Central. Then a list of health centres in each region was provided, and proportional to the number of health centres in each area, 25 health centres were selected randomly. In all, 300 individuals were approached. Of these, thirty-nine participants dropped out of the study. A total of 8 participants were older than 65 years, 6 were illiterate, and 25 completed the questionnaires incompletely. This sample size for SEM analysis seemed to be enough because the minimum sample size required for such an analysis is 200 . The stuDemographic, anthropometric and physical activity: Demographic questionnaire included information on age, sex, marital status, education level, occupation, smoking status, body mass index, and physical activity. Age was recorded on a continuous scale, and education level was recorded as primary, secondary, and higher. Weight was measured with light clothing and without shoes using a digital scale (Seca 808) and recorded to the nearest 100 g. The height was measured in a standing position without shoes, using the standard Seca stadiometer, recorded with an accuracy of 0.1 cm. BMI was calculated as weight (kg)/height2 (m). Level of physical activity was assessed with the International Physical Activity Questionnaire (IPAQ). Data from the IPAQ were used to estimate compliance with guidelines for physical activity presented as low, moderate, and high levels of activity expressed in (MET-h per week) [er week) .Health literacy: We used the Health Literacy Instrument for Adults (HELIA) for data collection [llection . The queNutritional literacy: The Nutrition Literacy Scale (NLS) consisted of 28 items [28 items . In geneAttitude toward sunlight exposure: Sunlight exposure was measured using six items: (1) ‘‘I like sunlight’’; (2) ‘‘I use sunhat when exposed to sunlight”; and (3) ‘‘I use sunscreen products containing SPF ≥ 15 when exposed to sunlight’’ (4) ‘‘I like outdoor activities’’ (5) ‘‘Usually I spend most of my time outdoors’’(6) ‘‘The time I expose myself to sunlight is enough’’ [enough’’ .Knowledge of vitamin D: Knowledge of vitamin D was measured by five items: (1) ‘‘I have ever heard about vitamin D’’;; (2) ‘‘Vitamin D is good for bone health’’; (3) ‘‘Vitamin D supports calcium absorption’’; (4) ‘‘Vitamin D can be supplemented by sunlight exposure’’; (5) ‘‘The minimum time needed for sunlight exposure is 30 min if we want our body to develop a sufficient amount of vitamin D’’ [amin D’’ . Scores amin D’’ .Sunlight exposure behaviour: Sun exposure duration was used to calculate the hours of daily sun exposure over the previous week [ous week . There w2/df), normed fit index (NFI), comparative fit index (CFI) and root-mean-squared error associated (RMSEA). Smaller relative chi-square values indicate a better fit, and an insignificant relative chi-square is desirable. Relative chi-square is thought to be less dependent on sample size, and values greater than 1 and below 2 are considered a good fit [P-value of < 0.05 was considered statistically significant.Data were analyzed using R version 3.4.4. A model was developed for the hypothesized relationships among health literacy, nutritional literacy, knowledge of vitamin D, attitudes toward sunlight, and sunlight exposure behavior. In fact, sun exposure behavior was considered as an outcome measure, and the remaining variables in the model were considered as covariates including, knowledge of vitamin D (indicated by five questions), attitudes toward sunlight exposure (indicated by six questions), health literacy, and nutritional literacy. The model’s goodness of fit was determined using four measures of fit: relative chi-square , overweight (42.1%), married (73.2%), employed (51.7%), and had a higher education qualification (49.4%). The Participants’ characteristics are shown in Table In total, a convenient sample of 261 individuals participated in the study. The mean age of participants was 38.8 years (SD = 11.09). The majority of participants were female (This study has indicated a high level of vitamin D Knowledge. The majority of participants (83.9%) had heard about vitamin D, 83.1% agreed that vitamin D is good for bone health and 72.4% knew that vitamin D is necessary for supporting calcium absorption, and 69% knew that vitamin D could be supplemented by sunlight exposure. Moreover, 64% of them had information on the minimum time needed to spend outdoors to get enough Vitamin D  responded that they like sunlight, 64.4% indicated that they used sunscreen products with a sun protection factor (SPF) ≥ 15. Moreover, 80.0% said they used a parasol to shade themselves from the sun; 67.8% reported that they like outdoor activities, and 21.1% reported that they spend most of the time outdoors. Overall, 31.8% of the participants felt that they had sufficient sunlight exposure (Table % respondn = 212) had an adequate level of health literacy . In addition, attitude toward sun exposure was not directly associated with sunlight exposure behaviour  , suggest the model has an acceptable predictive ability or fit. The relationship between knowledge and sunlight exposure behavior was insignificant  and knowledge and health literacy  were statistically significant. Thus, health literacy had a direct effect on sunlight exposure. In addition, results confirmed the expected indirect effect of knowledge on sunlight exposure through health literacy. The path between attitude and health literacy  was statistically insignificant  indicated a relatively satisfactory model fit to the data. The paths between health literacy and sunlight exposure , and positive effect of nutritional literacy on sunlight exposure . This result revealed that the relationship between knowledge and sunlight exposure might be nutritional literacy dependent. The path between attitudes, and nutritional literacy  was not statistically significant . The results showed a significant direct path between knowledge, nutritional literacy  and knowledge and sun exposure behavior and nutritional literacy . Indeed, these results indicate that the relationship between knowledge and sun exposure might be health literacy and nutritional literacy dependent. There was not a significant relationship between attitude and sun exposure even engaging health literacy  and nutritional literacy .The final model shows the relationship between knowledge, attitude, and sun exposure behavior and the effect of health and nutritional literacy on this relationship Fig. . Resultsβ = -0.07, P = 0.47) and attitude and sun exposure behavior . The results of the present study also showed that some paths such as using sun hat and attitudes  and using sunscreen and attitudes  were statistically significant. There was not significant relationship between enjoy outdoor activities , spending time outdoor  and feel have sufficient sun exposure  and attitudes. Likewise, the relationship between knowledge and three items: bone health , calcium absorption  and supplement vitamin D from sun exposure  were significant, whereas the path of between the minimum time for sun exposure and knowledge  was not statistically significant. The final model showed adequate goodness of fit for the data .Examination of the path coefficients in the final model showed that there is no direct relationship between knowledge and sun exposure behavior  would be appropriate solution to achieve the study objective. The SEM allowed this study to examine the complexity of the relationship between health literacy, and nutritional literacy and influence vitamin D knowledge, motivation toward sunlight and sun exposure behavior.The findings showed that knowledge of vitamin D was associated with sun exposure via health literacy and nutritional literacy. In addition, we found that health literacy and nutritional literacy were positively associated with sun exposure behaviour. In this context, improving health literacy and nutritional literacy might be a useful way to increase the sun exposure behaviour in adults. Despite the lack of enough vitamin D intake , there iThe results also revealed no significant association between motivation and sun exposure. Additionally, health and nutritional literacy did not influence the relationship between motivation and sun exposure. The results showed that individuals’ motivation was not as good as their knowledge of vitamin D and was moderately low. The study showed that less than half of the individuals had a positive attitude toward sun exposure. Similar to our study, others from China  and VietThis study had several strengths. The current study was the first to examine the relationship between health literacy, nutritional literacy, knowledge, motivation, and sun exposure among adults using SEM. Also, the study included a large sample size with a variety of ages, professions, and educational backgrounds. However, we should also consider a few limitations. First, causality cannot be inferred by the cross-sectional nature of this study. Longitudinal and experimental approaches are needed to further explore the relationship between health literacy, nutritional literacy, knowledge, motivation, and sun exposure. Second, the use of vitamin D supplements should be further investigated because vitamin D supplements may be a negative motivation for sun exposure. Third, since this study has been conducted in Iran, whose lifestyle and cultural context may be different from those of other countries, the current findings may differ from those that do not match the rest of the world. Finally, we did not look at the reasons for sun exposure avoidance.The findings of the current study showed that health literacy and nutritional literacy were significantly associated with knowledge but not significantly associated with attitudes toward sunlight exposure. Rather, health literacy and nutritional literacy mediated the relationships between knowledge and sunlight exposure. The findings also suggest that health professionals should consider the level of health literacy and nutritional literacy of individuals when conducting health education on sun exposure as a way to receive vitamin D.