Evidence-Based Medicine (EBM) skills have been included in general practice curricula and competency frameworks. However, GPs experience numerous barriers to developing and maintaining EBM skills, and some GPs feel the EBM movement misunderstands, and threatens their traditional role. We therefore need a new approach that acknowledges the constraints encountered in real-world general practice. The aim of this study was to synthesise from empirical research a real-world EBM competency framework for general practice, which could be applied in training, in the individual pursuit of continuing professional development, and in routine care. We sought to integrate evidence from the literature with evidence derived from the opinions of experts in the fields of general practice and EBM.We synthesised two sets of themes describing the meaning of EBM in general practice. One set of themes was derived from a mixed-methods systematic review of the literature; the other set was derived from the further development of those themes using a Delphi process among a panel of EBM and general practice experts. From these two sets of themes we constructed a real-world EBM competency framework for general practice.A simple competency framework was constructed, that acknowledges the constraints of real-world general practice: (1) mindfulness - in one’s approach towards EBM itself, and to the influences on decision-making; (2) pragmatism – in one’s approach to finding and evaluating evidence; and (3) knowledge of the patient – as the most useful resource in effective communication of evidence. We present a clinical scenario to illustrate how a GP might demonstrate these competencies in their routine daily work.We have proposed a real-world EBM competency framework for general practice, derived from empirical research, which acknowledges the constraints encountered in modern general practice. Further validation of these competencies is required, both as an educational resource and as a strategy for actual practice.The online version of this article (doi:10.1186/s12909-017-0916-1) contains supplementary material, which is available to authorized users. Translation of uncertainty to an answerable questionSystematic retrieval of best evidence availableCritical appraisal of evidence for validity, clinical relevance, and applicabilityApplication of results in practiceEvaluation of performanceEvidence-Based Medicine (EBM) has become established as a prominent paradigm in the field of healthcare,  and is dThese five sequential steps are included in several general practice curricula, such as those published by the Royal College of General Practitioners (RCGP) in the UK,  and the what themes underpin the meaning of EBM competence in general practice?’ This resulted in a series of statements such as ‘GPs are reluctant to challenge expert opinion.’ The statements were themselves organised into a set of overarching ‘analytic’ themes. We organised quantitative survey outcome data under descriptive headings. Outcomes were synthesised with qualitative data if they were derived from surveys with low or moderate risk of bias, and either consistent across several surveys, or from a single survey with external validity. Each analytic theme derived from the qualitative synthesis was matched with quantitative survey outcome data under relevant descriptive headings. This allowed triangulation of analytic themes with survey data, and provided additional useful insights to several of the analytic themes. We carried out sensitivity analyses to test the effect of including and excluding studies of poorer quality, using to the method described by Thomas and Harden: first we reviewed the quality of studies contributing data to each of the analytic themes derived from the qualitative synthesis; then for each analytic theme, a decision was made regarding the extent to which it depended upon data from less robust studies [We identified empirical research, published in English between 1991 (when EBM first appeared in the literature ) and Aug studies . Themes  studies  to generThe systematic review yielded 3444 articles, of which 38 were considered suitable for inclusion Fig. . These cBelgium, Sweden and Germany. The 38 included studies comprised a combined sum of over 3500 GP participants Table . The mosN = 31) were considered to be of moderate quality  attitude towards EBM; (2) major influences on decision making; (3) use of pre-appraised sources of evidence; (4) integration of evidence and clinical expertise; (5) communication skills; (6) skill in searching for primary sources of best evidence; and (7) skill in critical appraisal of evidence. The findings relating to each theme are described in more detail below.‘One sees it at times as dogma... what I mean is that not all patients are the same and many times you are in front of a patient that EBM says has this or that, and you don’t see it’ [‘The last month I got two patients back... look, your patient does not fulfil our criteria and so he does not have to come. And I think: ‘Is that the kind of medicine I will be forced to do? That person comes with his complaints, whether he is fulfilling my criteria or not. But that will be the future task of the GP: helping the people who do not fit the criteria of the specialists’ [There was a widespread perception that EBM fails to acknowledge the complexity and constraints of general practice:  see it’ . This wa see it’ , 26–42.  see it’ , 14, 43  see it’ , 43, 44. see it’ . Time co see it’ –15 as we see it’ –14 and a see it’ , 14, 16. see it’ . Some GP see it’ , 38, 45.ialists’  restrictialists’ , 38 and ialists’ . Some GPialists’ . There wThe doctor-patient relationship exerts a powerful influence on decisions. The desire to preserve it can outweigh considerations of evidence , 41. In ‘I think EBM is predicated upon there being well-supported and financed independent reviewers who are doing the meta-analyses and the broader views which have become the key to evidence-based medicine and then selecting which ones are quality enough to include. It's just not practical for the family practitioner to be able to do that, even if you do have the tools, the time just isn't going to be there’ [A preference emerged among GPs for acquiring clinical information passively, through the post or electronically (sometimes described as information ‘push’ ) rather e there’ . Surveyse there’ , 10 and e there’ –11. Surve there’ , 13, 50 e there’ , 13. GPse there’  or othere there’  than to e there’ , 14 may ‘The guidelines will only take you so far and they're a useful stepping off point, but each individual case has so many factors at play beyond what the guidelines cover and that's where the art and the pleasure of medicine comes in using your clinical judgement to realize that what the protocol says doesn't apply to that person or it applies in a different way. That takes a good understanding of what the meaning is behind the protocol so I don't think you're ever just a technician’ [barrier to EBM, rather than a necessary component [There is a belief that evidence must be integrated with experience, and tailored for individual patients. This was among the strongest themes to emerge from qualitative studies, with numerous expressions of the principle that evidence should be integrated with both knowledge gained from experience, and considerations relating to the individual patient. , 45, 47.hnician’ . In one hnician’ . The stuhnician’ , 38. Finhnician’ . This promponent , 12–14.. This theme arose within a single qualitative study, [‘So you need to get a rapport, and the trust of the patient, there’s no good enforcing treatment’ [‘We have to know, aside from diagnosing the patient, what he or she feels, and empathise with the person’ [There is a perceived need for competence in explaining complex information, including the risk and benefit of proposed interventionse study,  in whiche study, . In a sue study, . Other se study, –11, 13. eatment’ . ‘We hav person’ . The ing person’ . Support person’ . The GP  person’ . GPs par person’ . The surSome GPs believe that they should be able to conduct searches for research evidence. This emerged as a theme from four qualitative studies. , 30, 49 Some GPs perceive a need for competence in critical appraisal of research evidence. However, this theme arose from only one qualitative study, conducted among GPs practicing in Australia: three quarters of the 50 GPs interviewed agreed that GPs need some ability to critically appraise medical literature . One surAfter mapping the seven themes derived from the systematic review to the core principles of adult learning theory,  the follThe EBM competent GP reflects on instances in which risk and benefit were difficult to explain to a patient, and develops a concise, clear explanation for future use.The EBM competent GP demonstrates an ability to identify decisions they found challenging, and to reflect upon their reasoning in terms of ‘4Es’: Evidence , Experience (their own accumulated on-the-job knowledge), Empathy  and Environment .The EBM competent GP demonstrates a commitment to checking newly acquired knowledge from colleagues, media, pharmaceutical companies and other sources against the best available evidence, using secondary sources such as guidelines, evidence-based summaries and systematic reviews.The EBM competent GP demonstrates a commitment to efficiently addressing clinical uncertainty, using secondary sources of evidence, and the best available information technology.The EBM competent GP works with colleagues to share the tasks of identifying and prioritising clinical questions, finding, appraising and presenting evidence, and refreshing knowledge of EBM concepts and resources.The EBM competent GP demonstrates advocacy and leadership in identifying and addressing barriers to EBM within their practice.Decisions are influenced by information drawn from a complex array of sources.There are social, cognitive, informational and organisational barriers to the notion of ideal EBM.EBM involves tailoring decisions to the needs and perspective of the individual patient.A sense of pragmatism must underpin the approach to seeking and critically appraising evidence, and to the development of competence in EBM, given the constraints implicit in general practice.EBM requires commitment.Reflection is a continuous background process, and should be objective.EBM is difficult to demonstrate using currently available performance appraisal tools.The variance for mean overall ratings between Round 1 and 2 of the Delphi process reduced for five out of six competencies, suggesting a degree of consensus. There was little concordance of rankings between panelists, beyond that which would be expected by random chance, as indicated by a Kendall’s W statistic of 0.132. Thematic analysis resulted in seven themes, representing an integration of evidence from the literature, with evidence from a consensus of opinions from experts in general practice and EBM:Mindfulness in one’s approach towards EBM itself, and to the influences on decision-making.Pragmatism in one’s approach to finding and evaluating evidence.Knowledge of the patient, as the most useful resource in effective communication of evidence.Among the themes derived from the mixed-methods systematic review, two were considered to lack credibility – (1) skill in searching for primary sources of best evidence, and (2) skill in critical appraisal. In each case, survey findings failed to support the enthusiasm expressed for these attributes in qualitative studies. The remaining five themes were compared with those derived from the Delphi process. Much meaning was shared between the two sets of themes, and they were condensed into a simpler final framework of three competencies:mindfulness in one’s approach towards EBM, and the influences on decision-making; (2) pragmatism in one’s approach to finding and evaluating evidence; and (3) application of one’s knowledge of the patient in communication of evidence. Mindfulness originated in Buddhist practice and is described as the process of developing an open and unbroken awareness of present-moment cognitive-effective and sensory experience [A systematic review provided a literature-based, thematic exploration of what EBM means in general practice; a Delphi process developed those themes further in light of the consensus opinion of experts in EBM and general practice. Integration of both sets of data resulted in a simple real-world competency framework with potential application in training, and in the individual pursuit of continuing professional development, and which could guide routine practice: (1) perience . It has perience . It has perience . We are perience . His properience  reflectsperience . It entaThere are similarities between our real-world framework and the other EBM competency frameworks and curricula we have reviewed. Mindfulness, and knowledge of the patient as a resource for evidence communication are both echoed in several instances Table .Table 3EAll the frameworks and curricula have noticeably moved away from the stepwise, sequential nature of the traditional model, perhaps recognising the difficulty of integrating such a process with routine work. There are however, fundamental differences. The other frameworks cleave more closely to the competencies of the traditional EBM model. Our systematic review findings do not support this approach. We found consistent evidence that the traditional model has failed to gain traction in general practice: direct observation of GPs has revealed no engagement in the traditional five steps of EBM . Even seIt is clear from our data that the traditional model of EBM has encountered considerable challenges in general practice. The real-world approach we propose here offers two advantages: simplicity, and flexibility. We propose that a simpler, more streamlined framework of only three competencies might be easier to integrate into routine work. It is more flexible in that it allows some room for manoeuvre in the imperfect, real world. It assumes the position that EBM is about individual GPs and individual patients making the best possible decisions in the prevailing circumstances. To illustrate how a GP might demonstrate the real-world competencies in practice, we describe a clinical scenario.Fiona’s last patient on Friday afternoon is Reg, a 70-year-old retired army officer. He’s well turned out, having put on a tie to see the doctor. He tells Fiona he hasn’t been sleeping in the last six weeks since his wife died. Fiona’s colleague had prescribed a sleeping tablet, and that had helped. He moves on quickly in response to Fiona’s enquiries about his mood, and says all he needs is some more of those sleeping tablets. Fiona recalls a complaint made by Reg some years ago, when she suggested a referral for cognitive behavioural therapy (CBT) for a bout of anxiety. She’s about to type ‘Temazepam 10 mg’ when she remembers insomnia had been discussed at a Practice Based Small Group Learning (PBSGL) session she attended a few weeks ago. She considers whether she might have been a little rash with her decision to prescribe. Perhaps it was because it’s Friday afternoon and she’s tired – or maybe because she didn’t want to upset Reg again (mindfulness). She asks Reg if he would mind waiting a second while she looks for some useful information on the computer. She opens the TRIP database (www.tripdatabase.com) she had used in the PBSGL meeting (pragmatism).She types ‘insomnia’ in the search box, and opens the first item listed, which is an evidence-based synopsis from BMJ Clinical Evidence . Casting her eyes quickly down the page, she reads that ‘the evidence for effectiveness of CBT for insomnia is convincing, whereas the evidence for exercise programmes and timed exposure to bright light is less clear. She reasons that a suggestion of exercise might however be more acceptable to Reg than a course of CBT or bereavement counseling (knowledge of the patient). She prints off an information sheet on insomnia fromwww.patient.co.ukand uses a hypnogram to explain sleep cycles. She also explains how sleeping tablets only work at the start of sleep, and that they can soon produce dependence. She explores the various options with Reg, broaching CBT very tentatively (knowledge of the patient). She’s not surprised when he seems more inclined to try a programme of exercise, and she manages to secure his agreement to try that, and report back in a week. After Reg leaves, she wonders whether 20 min was a bit long for such a consultation (mindfulness). She decides to print off a batch of information sheets and a hard copy of the evidence synopsis to keep in her desk drawer.To our knowledge, this methodological approach to the development of a competency framework is novel. We were unable to identify a clear consensus regarding the best approach within the field of medical education. Myriad techniques have been used, including practice profiling, task analysis,  focus grThere were a number of limitations to the mixed-methods systematic review. The search strategy included studies that addressed engagement with EBM as an overarching approach to patient care, and excluded those that addressed individual steps in the traditional model of EBM as published in the Sicily Statement . It was The Delphi process was also subject to limitations. Because of the limited availability of experts willing to give precious time to a relatively time-consuming process, the size of the expert panel was at the lower end of those reported as acceptable . A smallCurrent curricula and competencies for EBM do not acknowledge the constraints and competing demands of real-world general practice. Given low levels of engagement with EBM in general practice, future development and refinements should take into account the available evidence from empirical research regarding the knowledge, skills, attitudes and behaviour of GPs in this context. We synthesised this evidence, and proposed an EBM competency framework for real-world general practice. We have illustrated how this approach might be applied in practice, and considered its potential as a learning tool. It will require further validation as an educational and on-the-job resource.


Ustilago maydis and the ascomycete Aspergillus nidulans reveals a crucial role of early endosomes (EEs) in the dynamic behavior of POs required for their cellular distribution and interaction. This article summarizes the main findings, which provided unexpected insight into the mechanism by which fungal cells organize themselves.Peroxisomes (POs) are an essential part of the fungal cell’s inventory. They are involved in cellular lipid homeostasis, reactive oxygen metabolism and in the synthesis of secondary metabolites. These functions are thought to require frequent organelle-organelle interactions and the even-distribution of POs in fungal hypha. Recent work in the basidiomycete  Thus the molecular mechanism of this organelle-organelle interaction remains to be elucidated. Interestingly, the interaction between EEs and POs, as well as EEs and LDs in U. maydis was transient. In other words, POs and LDs bind to EEs, move a certain distance and then fall off the moving organelles. Such binding and dissociation was previously described for polysomes (ribosomes and mRNA), which also hitchhike on moving EEs in U. maydis  and EEs (green) in the basidiomycete fungus Hok1-GFP  moves toIt is widely-accepted that motor proteins bind directly to their cargo to move it along the fibers of the cytoskeleton. Studies in s to POs . Howevers to POs , suggests to POs . Subsequs to POs , suggestnidulans . This sunidulans . Studiesnidulans . PxdA or. maydis . Thus, t3U. maydis demonstrated that random motion of POs and LDs requires enzymatic activity , as it is slower and non-directed. In fact, EEs motility is likely to cause turbulences within the crowded cytoplasm, which indirectly enhance the diffusion of POs. Mathematical modeling of this process revealed that active diffusion and directed transport co-operate in two ways  EEs bind to POs via a specific adapter and drag POs throughout the cytoplasm, (2) moving EEs cause turbulences in the cytoplasm , which enhances diffusion of POs. Note an PO adapter was identified in ns PxdA,  and has The majority of fungal POs and LDs do not undergo directed motility, but instead undergo short-range and undirected motions ; Video 1activity . This isactivity . In U. mtwo ways . Firstlytwo ways . Currentomycetes , which s4U. maydis showed that POs, but also LDs, transiently interact with moving EEs, thereby “hitchhiking” for long-range directed motility in the hyphal cell (A. nidulans, where a specific adapter links POs to EEs (U. maydis (In this focused Video article, I have reviewed recent work that demonstrated that EEs mediate directed transport of POs in fungi. Surprisingly, work in hal cell . This mes to EEs . In addi. maydis . These r. maydis  and dist. maydis . Thus, f


The study systematizes the lessons learned by the NHD group at WHO.Over the past decade, the World Health Organization (WHO) has implemented a standardized, evidence-informed guideline development process to assure technically sound and policy-relevant guidelines. This study is an independent evaluation of the n = 12) and was analyzed through axial coding. Guideline adaptability was assessed quantitatively using two standardized instruments completed by key stakeholders. The Appraisal Guideline for Research and Evaluation questionnaire, version II was used to assess guideline quality (n = 6), while implementability was assessed with the electronic version of the GuideLine Implementability Appraisal (n = 7).We used a mixed methods approach to determine the adaptability of the nutrition guidelines. Adaptability was defined as having two components; methodological quality and implementability of guidelines. Additionally, we gathered recommendations to improve future guideline development in nutrition actions for health and development. Data sources for this evaluation were official documentation and feedback  from key stakeholders involved in the development of nutrition guidelines. The qualitative data was collected through a desk review and two waves of semi-structured interviews (The nutrition evidence-informed guideline development process has several strengths, among them are the appropriate management of conflicts of interest of guideline developers and the systematic use of high-quality evidence to inform the recommendations. These features contribute to increase the methodological quality of the guidelines. The key areas for improvement are the limited implementability of the recommendations, the lack of explicit and precise implementation advice in the guidelines and challenges related to collaborative work within interdisciplinary groups.Overall, our study found that the nutrition evidence-informed guidelines are of good methodological quality but that the implementability requires improvement. The recommendations to improve guideline adaptability address the guideline content, the dynamics shaping interdisciplinary work, and actions for implementation feasibility. As WHO relies heavily on a standardized procedure to develop guidelines, the lessons learned may be applicable to guideline development across the organization and to other groups developing guidelines. Over the past decade, the World Health Organization (WHO) has implemented a standardized process to assure technically sound and country-relevant guidelines. In the past, WHO’s guideline development process relied heavily on expert opinion, an approach that raised criticism from the scientific community. In 2005, the organization launched a comprehensive undertaking to guarantee guideline quality. This included devising a new process to ensure that WHO recommendations were informed by the best available evidence . This prThe Evidence and Programme Guidance, a unit with normative functions at the Department of Nutrition for Health and Development (NHD) at WHO, has followed the Handbook closely to develop over 20 guidelines on nutrition-specific or nutrition-sensitive interventions in collaboration with several internal partners. The group has taken some additional measures to ensure a steady increase in the methodological rigor of the guidelines. However, guidelines need not only to provide robust, reliable, and independent advise but also advise that is feasible and easily implemented in the real world . While tInnovations have been funneled by the group’s rigorous approach to guideline development, entailing the continuous monitoring, evaluation, and revision of the process. The present article is the product of an independent evaluation of the nutrition evidence-informed guideline development approach and systematizes the lessons learned by the NHD group at WHO. The results of the evaluation should further help to ensure that the guidelines meet their purpose and are adapted by Member States and partners.adaptability of the nutrition guidelines. Assessing guideline adaptability was deemed to be the best strategy because it allowed the evaluation to focus on two key aspects of the guidelines; methodological quality and implementability /[maximum possible score – minimum possible score].The methodological quality was assessed through the GREE II)  instrumeGREE II) , 20. ResGREE II) . Domain The user’s manual does not specify cutoff points, nor does it set minimum domain scores or patterns of scores across domains to differentiate between high-quality and poor-quality guidelines. We were not interested in carrying out a between-guideline comparison, but rather, we wanted to evaluate the quality of the guidelines as a whole. For this reason, the interpretation of the results in this evaluation was done based on the average score obtained by domain across all guidelines.Implementability was measured by using eGlia, a validated questionnaire for guideline implementability assessment . The eGlThe evaluation used the online versions of the AGREE II and eGlia questionnaires . The ninIt is important to consider that neither the AGREE II nor the eGlia questionnaires have been validated for use in the field of nutrition. However, Agree II has demonstrated pertinence  and valin = 8) of semi-structured interviews complemented the data collected from the desk review and questionnaires. Wave 2 (n = 4) probed the emerging themes from wave 1 and further asked respondents to formulate recommendations to improve adaptability.A total of twelve semi-structured interviews were conducted for this study across two waves. The interviews explored the key characteristics of the evidence-informed guideline development process, the adaptability of the nutrition guidelines, and suggestions for improvement; dimensions corresponding to the research questions are shown in Table The main selection criterion for the interviewees was heterogeneity; the evaluators actively sought to interview individuals with different areas of expertise and who worked in different WHO regions. Half of the interviewees were technical specialists, and the other half implementation experts; all had substantial experience in guideline development and program implementation. Technical experts were members of the Guideline Development Group “GDG” whose area of expertise is the technical evaluation of the available evidence and can provide feedback on the methodological quality of a guideline. Implementation experts were members of the GDG whose area of expertise is the science of implementation and can provide key insights into guideline feasibility. The sample was comprised of an equal number of males and females. The combined experience of the participants covered the Americas, South-East Asia, Eastern Mediterranean, Western-Pacific, and the European region. All participants had attended at least one GDG meeting in the year previous to the evaluation. The GDG is multidisciplinary and composed of individuals from all WHO regions likely to use the guideline with a membership that is balanced in terms of gender and geography . This grBoth waves of interviews were closed after an interim assessment, which indicated that saturation had been reached . The anaVarious strategies were used to assure the reliability and validity of data. The first was data triangulation, which compared findings from different sources to ensure consistency of results . The resSaturation is present when no new information emerges from additional data . TherefoTwo evaluators, external to WHO, conducted this evaluation. Both had experience in qualitative research in the social and health sciences. This background included a combined experience in other programmatic evaluations as well as knowledge in qualitative interviewing techniques and data analysis. This research project was conducted in compliance with the principles delineated by the United Nations Evaluation Group .The main aim of the evaluation was to assess the adaptability of the nutrition guidelines produced by the NHD group at WHO. In this study, adaptability is defined as having two components: methodological quality and implementability . This seThe guidelines’ methodological quality was assessed using the AGREE-II questionnaire. The results were triangulated with data from the semi-structured interviews and the desk review, which focused on identifying the key characteristics of the guideline development process. Table The evaluation results indicate that the nutrition guideline development process has improved over the years, resulting in guidelines of higher quality. Guidelines were rated by the study participants as rigorously developed; evidence-based, clearly scoped, and unbiased Table . The AGROne area where WHO nutrition technical staff performs exceptionally well is in the management of conflicts of interest (COI) (step 3) , which assures transparency in the conduct of the meetings and the decision-making process for developing recommendations. This happens through careful screening of the previous publications and projects of prospective GDG members, an analysis of potential real or perceived conflicts of interest, and declaration of interests  by all external contributors to guideline development. Individuals who declare one or more relevant interests that are deemed relatively minor in significance may be allowed to participate in the meetings because of their expertise; however, they are asked to not participate in deliberations or even to be present in closed sessions of the meeting when decisions are being made on recommendations related to their declared interests. An adequate management of the COI is paramount for guideline quality, as it assures that the guideline development is independent from its funding bodies , 34.A second area of strong performance is the evidence retrieval, summary and assessment, and systematic reviews  (step 5). The methods to search, select, and assess the evidence are clearly described. Further, the Department of Nutrition for Health and Development has built capacity to produce high-quality systematic reviews when a high-quality summary of the evidence is not available. More recently, the Department has requested that ongoing and unpublished trials are documented as part of the systematic process , throughNonetheless, the desk review and interviews indicated that the step involving the formulation of recommendations (step 6) can be improved . This pertains specifically to the process used by the GDG to establish the strength of the recommendation. The strength of a recommendation conveys the degree to which the GDG is confident in the balance between the desirable and undesirable consequences of implementing a recommendation . A recomLastly, the scoping of the question and purpose of the guidelines (AGREE-II domain) (step 4) is an area of good performance. The guidelines’ objectives and research questions were considered adequate and unambiguous. In addition, the population to whom the guideline applies is adequately specified.Guideline implementability was assessed through the eGlia questionnaire. The results of this assessment are shown in Table Indications about the need to improve guideline implementability were a consistent finding throughout the data sources used in this study. Notably, the interviewees clearly and repeatedly identified the applicability and executability of the guidelines to be a weakness  . First, there is a need for specific strategies and tools to implement the guideline’s recommendations. Second, recommendations need to state more explicitly the actions to be taken. While there are different perspectives within WHO on how directive the guidelines should be, there was ample consensus in that guidelines need to assess implementation barriers and consider costs, resources, and potential impact on health systems . Results also indicate that the guidelines do not permit much interpretation and that they don’t allow for alternatives in execution, which is a barrier to implementation. Lastly, the recommendations tend to require novel and unconventional behaviors for clinicians or patients, which further challenges their implementability .Lastly, the desk review indicated that the nutrition guideline development process should improve the dissemination, adaptation, and implementation of the nutrition guidelines (step 8) . These limitations had been informally captured by the technical staff, and more focus has been given to these considerations in the past few years. Lastly, the desk review indicated that the department of Nutrition for Health and Development needs to put in place a strategy to evaluate the impact of the guidelines (step 9), as it lacked one at the time of this evaluation.The semi-structured interviews produced valuable information on specific aspects of the guideline development process that impact guideline adaptability and that were not in the initial framework of the evaluation. The interview results are shown in Table group diversity, the decision-making process, and the group meetings. These issues deserve attention for their potential to negatively impact guideline adaptability. The diversity of the GDG members in terms of expertise potentially enables debates and the integration of varied ideas and perspectives. Yet, academic knowledge tends to be privileged over implementation knowledge; thus creating a dynamic where the implementers refrain from debating, thereby undermining the very purpose of their participation in the GDG. Thus, it is necessary to devise strategies to integrate the diversity in knowledge that the GDG members can provide.This emerging information allowed the identification of three aspects of the guideline development process that merit improvement. These pertain group dynamics operating in the GDG that are shaped to some extent by the institutional framework provided by the Nutrition for Health and Development group. Because these pertain issues resulting from interactions, it comes to no surprise that these aspects only emerged in the interviews. These are the management of the decision-making process was described as straightforward and successful in fostering evidence-informed recommendations. However, it is negatively affected by the way evidence is presented at the GDG meetings. In most cases, no information is sent prior to the meeting, which requires participants to ponder considerable amounts of information on the spot. The very full agenda limits the time available to review large volumes of complex evidence. More recently, at guideline development group meetings, WHO nutrition technical staff have started to present the evidence with a full description of the reasons for excluding specific studies. This helps achieve transparency and avoids distractions from discussions of potentially missed studies at meetings where time is critical.The One of the aims of the evaluation was to elicit recommendations coming from the study participants to understand how guideline adaptability can be improved. The present evaluation collected a sizeable number of recommendations through the semi-structured interviews. The recommendations pertained the same areas discussed above and focused on “methodological quality,” “organizational dynamics,” and “implementability.” The recommendations are presented in Table n = 252, Table The topic of implementability elicited the largest number of recommendations  determine the level of adaptability of its guidelines , (2) identify the key characteristics of the evidence-informed guideline development process, and (3) produce recommendations to improve future guideline development in nutrition actions for health and development. The most salient strengths of the current process and methods are the management of conflicts of interest and the systematic use of high-quality evidence to inform the recommendations. The key areas for improvement are the limited implementability of recommendations and the challenges related to collaborative work within interdisciplinary groups.Recent research indicates that some of the difficulties identified in this evaluation, specifically, those pertaining the determination of the recommendation strength and the decision-making process, may be partially due to limitations of the GRADE guidance itself when assessing the quality of the evidence. For instance, the decision-making process can be hindered because the GRADE guidance has too few categories to capture different levels of evidence other than randomized controlled trials and because it is highly time-consuming, which may thus not make it suitable for a grading of evidence under time pressure . NonetheTo continue improving the process, WHO needs to conduct periodic evaluations and act on results. Notably, the Department of Nutrition for Health and Development has started to implement some of the recommendations formulated in this evaluation. The latest guidelines have started to provide implementation guidance in the form of considerations related to equity, implementation, ethics, and regulatory aspects. While this remains an important step in the right direction, a recent study of the implementation guidance provided by the WHO guidelines shows an institutional tendency to emphasize policy-based implementation techniques as opposed to evidence-based active techniques, which demonstrates the need to build stronger guidance for implementation into the guidelines. Active implementation techniques include follow-up and personal interaction with the implementers, audit and feedback, educational outreach, and use of opinion leaders . Recent Another recent change is the ongoing development of guideline packages to be published by mid 2017 . With this change, WHO expects to increase the relevance of the nutrition guidelines because these will now fit better into the dynamics of health systems, as decision makers generally address broad problems such as “nutrition in antenatal care” or “anemia” or focus on target populations such as “infants 6–23 months of age.”The Handbook , 5 recomChanges are currently underway to improve other issues identified in this evaluation. Among these, WHO has started to monitor guideline implementation in different countries. This is a positive step, as this will generate evidence on what works and under which circumstances  which caThe present evaluation also has limitations. It drew primarily from individuals who have collaborated with WHO to develop evidence-informed guidelines on micronutrients and nutrition actions; the data could therefore be biased as a result of social desirability (respondents may be reluctant to provide answers seen as undesirable). To help to overcome such potential biases, anonymity was assured, allowing frank and in-depth discussions of the guideline development process. Further, the qualitative approach of this project benefits from collecting information from the key informants, who are highly engaged in guideline development and who can provide a critical assessment of its strengths and problems. To further strengthen data validity and credibility, the information was triangulated from different sources.Commissioned by the unit of Evidence and Programme Guidance, Department of Nutrition for Health and Development, the present evaluation identified approaches to improve the adaptability  of future guidelines. The evaluation also identified strengths and weaknesses, as well as the catalyzers and barriers of its current guideline development process.This study focuses on the guideline development process at WHO led by the Evidence and Programme, Guidance, at the NHD department. Our research shows that the current process has several strengths, among them are the appropriate management of conflicts of interest and the systematic use of high-quality evidence to inform the recommendations. The key areas for improvement are the limited implementability of recommendations, the lack of explicit and precise implementation advice in the guidelines, and challenges related to collaborative work within interdisciplinary groups. As WHO relies heavily on a standardized procedure to develop guidelines, the lessons learned may be applicable to guideline development across the Organization and to other groups developing guidelines.


VetCompanion is an evidence-based, point-of-care resource for small animal (canine and feline) veterinarians, veterinary students, and veterinary technicians. It is one of the few veterinary resources specifically designed for point-of-care use. VetCompanion was co-founded in 2014 by Matthew P. Mauer, a doctor of osteopathy who served as a medical director at the New York State Department of Health and currently works as a medical editor at Elsevier, and Jennifer Garcia, a board-certified veterinary internist, certified continuing medical education professional, medical writer, and editor [Registration and payment for VetCompanion are straightforward and can be completed from the product website. Each user has a unique, password-protected account. Subscriptions that accommodate institutional credentials or Internet protocol (IP) address recognition were not offered at the time of this review (August 2016). VetCompanion is a web-based resource; no mobile app is currently available. The site does not provide options for downloading, emailing, or saving content for offline use, which may limit its utility in veterinary practices that lack wireless service or for ambulatory veterinarians with sporadic access to wireless service. No access problems were experienced on any of the devices  or browsers  on which the site was tested.The VetCompanion interface is clean, intuitive, and mobile-responsive. Upon logging in to VetCompanion, users are presented with a home page that offers a search box as well as the ability to browse all topics alphabetically. A menu bar at the top of each page provides links to: Topics, which is an alphabetical list of all topics; Specialties, where topics are organized according to clinical specialty; Recently Viewed; News, which displays information about the site, such as new topics or updates to existing topics; and Suggestions, where users can submit suggestions about the site’s content or design. One topic, Conversions—which provides formulas for converting weight, temperature, and other measures—does not fit into the Specialties organization scheme, making it difficult to find. A dedicated link in the menu bar at the top of each page would make this topic easily accessible from any page.Queries for diseases ; clinical signs ; and procedures (thoracocentesis) were used to test the functionality of VetCompanion’s search. According to VetCompanion’s CEO, the site’s search matches the words in a query to topic titles, summaries, and author- or editor-applied tags . These tTopics are selected for inclusion in VetCompanion based on consultation with practicing veterinarians and members of the editorial board; review of recent guidelines, consensus statements, and research; perusal of frequently asked questions on veterinary websites; and user recommendations . A preciThe title of the topic and the species to which the topic is applicable  appear at the top of each monograph, and date of last update appears at the bottom. Content is updated as new information becomes available. The authors of the monograph are not listed. Each monograph is arranged in a series of expandable topic page headings and subheadings: Summary, Causes and Risk Factors, Differential Diagnosis, Diagnosis and Screening, Prevention, Treatment, Follow-up, Prognosis, and Evidence .This arrangement mirrors how a clinician works through a case, with a few exceptions. For example, the Diagnosis and Screening heading is positioned after the Differential Diagnosis heading, yet differential diagnosis typically takes place after initial evaluation of the patient. A more intuitive arrangement could be achieved by placing the subheadings Signs and History and Physical Exam under a separate heading called Clinical Presentation. This new heading should be positioned before the Diagnosis and Screening and Differential Diagnosis headings. Unfortunately, the monograph pages do not have a table of contents with links to each heading and subheading, which makes navigating to a particular section inefficient, particularly on a mobile device.Information in VetCompanion is presented in brief, bulleted sentences. After reviewing several monographs, this reviewer felt that the content was not detailed enough to be clinically useful. Also, the monographs examined for this review had a few omissions and unusual recommendations. For example, the monograph for congestive heart failure did not mention echocardiography for diagnostic follow-up. In the pancreatitis monograph, methadone was one of the drugs recommended for pain control. While a valid recommendation, methadone is rarely used in small animal practice in the United States.Saunders Handbook of Veterinary Drugs would be added to the site [Drug dose recommendations, culled from multiple sources, are provided in the topic monographs. VetCompanion’s terms of use encourage users to consult product information sheets and “an appropriate drug information database”  to confithe site . InitialFew monographs include images, and the images that are included are not always useful. For example, the feline diabetes monograph has an image of a glucometer, syringe, and pill bottle. Fortunately, for topics where a visual component is particularly useful, such as dermatological conditions, images are provided.Specific studies are mentioned in the text of some monographs. Unfortunately, in-text citations are not included, so the user cannot connect the information in the monograph to the references in the Evidence section. In many monographs, studies are not mentioned at all. For example, the pancreatitis monograph states that canine pancreatic specific lipase (cPLI), a blood test that can aid in the diagnosis of pancreatitis, has a high sensitivity and specificity but does not include either the percentages or the reference for this information. If VetCompanion is truly designed to help clinicians make evidence-based decisions, then more detailed information and in-text citations with links to the full reference in the Evidence section need to be included.References in the Evidence section are organized under subheadings: Guidelines and Consensus Statements, Systematic Reviews/Meta-Analyses, Randomized Controlled Trials, Other Studies or Reviews, and Additional Readings. According to VetCompanion’s website, “All pertinent guidelines, consensus statements, systematic reviews, meta-analyses, randomized, controlled trials, and observational studies are included for each clinical topic” . SourcesUsing a system derived from the Strength of Recommendation Taxonomy (SORT), VetCompanion assigns a level 1, 2, or 3 to each study and A, B, or C to each guideline or consensus statement in the Evidence section. The levels of evidence (LOE) criteria are described on the Methodology page of VetCompanion’s website. This information is not available within the resource itself, either as a link from the menu bar or adjacent to the reference. The link to the abstract, located at the end of many references in the Evidence section, is equally problematic. Links to abstracts are not provided for all references, and, for journal articles, the destination of the links varies, with some going to the PubMed record and others to the article on the journal’s website. According to VetCompanion’s CEO, an editorial decision was made to only provide links to abstracts for guidelines, consensus statements, and studies with an evidence level of 1 or 2 [Veterinary Information Network (VIN), a professional networking and continuing education site, was one of the few online resources for veterinarians. VIN does not offer evidence-based summaries of diseases and is not designed, nor was it intended to be used, as a point-of-care resource. Therefore, despite its longevity and popularity among veterinarians, it cannot be considered a direct competitor of VetCompanion.For many years, Vetstream, a British company offering digital solutions for veterinarians, launched Vetlexicon, a peer-reviewed, point-of care resource for veterinarians. This resource consists of four separate “services” or databases: Canis (canine), Felis (feline), Lapis (rabbits), and Equis (equine). A fifth service for cattle, Bovis, will be launched at the end of 2017. Compared to VetCompanion, Vetlexicon covers more species and topics, provides a broader array of content, and has a more extensive list of contributors and reviewers. For example, Canis covers 1,035 diseases; 362 diagnostic and surgical techniques; 543 formulary items; and has 4,359 images, videos, and sounds; 175 owner factsheets; and over 500 contributors and reviewers from around the world [In 1996, This reviewer was left with the impression that VetCompanion is a work-in-progress without a well-defined audience. The small number of board-certified contributors and reviewers, narrow scope of content, and inconsistency with how evidence is presented limit the usefulness of the current iteration of VetCompanion for veterinary students, technicians and clinicians, and librarians who support these populations.


Implementing treatments and interventions with demonstrated effectiveness is critical for improving patient health outcomes at a reduced cost. When an evidence-based intervention (EBI) is implemented with fidelity in a setting that is very similar to the setting wherein it was previously found to be effective, it is reasonable to anticipate similar benefits of that EBI. However, one goal of implementation science is to expand the use of EBIs as broadly as is feasible and appropriate in order to foster the greatest public health impact. When implementing an EBI in a novel setting, or targeting novel populations, one must consider whether there is sufficient justification that the EBI would have similar benefits to those found in earlier trials.In this paper, we introduce a new concept for implementation called “scaling-out” when EBIs are adapted either to new populations or new delivery systems, or both. Using existing external validity theories and multilevel mediation modeling, we provide a logical framework for determining what new empirical evidence is required for an intervention to retain its evidence-based standard in this new context. The motivating questions are whether scale-out can reasonably be expected to produce population-level effectiveness as found in previous studies, and what additional empirical evaluations would be necessary to test for this short of an entirely new effectiveness trial. We present evaluation options for assessing whether scaling-out results in the ultimate health outcome of interest.In scaling to health or service delivery systems or population/community contexts that are different from the setting where the EBI was originally tested, there are situations where a shorter timeframe of translation is possible. We argue that implementation of an EBI in a moderately different setting or with a different population can sometimes “borrow strength” from evidence of impact in a prior effectiveness trial. The collection of additional empirical data is deemed necessary by the nature and degree of adaptations to the EBI and the context. Our argument in this paper is conceptual, and we propose formal empirical tests of mediational equivalence in a follow-up paper. One goal of implementation science is to expand the use of evidence-based interventions (EBIs) appropriately and as broadly as feasible in order to foster the greatest public health impact . This goWhen an EBI is implemented with fidelity in a setting that is identical to or very similar to where it was previously tested and found to be effective, it is reasonable to anticipate that the EBI would provide similar benefits to those found earlier. However, every EBI implementation raises two critical questions: (1) is there sufficient empirical evidence or justification from prior evidence that this EBI would impact health as expected, and (2) whether system, organization, and/or EBI adaptations are necessary, sufficient, and culturally and organizationally appropriate to make it feasible, practical, and acceptable in the new context. We argue first that EBI implementation in a moderately different setting or with a different population can sometimes “borrow strength” from evidence of impact in a prior effectiveness trial with additional empirical data deemed necessary by the nature and degree of adaptations. This strategy of testing precise elements in a mediation model can be seen as an extension of Cook’s five pragmatic principles for justifying generalized causal inferences to different target populations and settings . We arguWe define the approach to adapting and delivering EBIs across health and allied health service systems and organizations and/or across different target populations as scaling-out. Scaling-out is the deliberate use of strategies to implement, test, improve, and sustain EBIs as they are delivered in novel circumstances distinct from, but closely related to, previous implementations. Although we propose an approach that identifies three types of scaling-out, we focus on two major types of scaling-out in this paper, (1) one that involves delivery of an intervention to the same target population as previously tested, but through different settings or delivery systems, and (2) one that involves delivering an intervention to a different population than previously tested, but through similar settings or delivery systems. The motivating questions are whether scaling-out can reasonably be expected to produce population-level effectiveness as found in previous studies, and what additional empirical evaluations would be necessary to test for this short of an entirely new effectiveness trial. If testing of this new scale-out requires the full empirical evaluation that would be required for establishing an EBI, this would be exceptionally costly, time consuming, and would delay implementation, especially to populations underrepresented in scientific trials or in settings where its delivery could reasonably produce benefit. Indeed, if we can legitimately borrow strength from previous studies and a modest amount of empirical evidence, this could accelerate and expand benefit to populations that have experienced health disparities that might never be included in a rigorous randomized effectiveness trial .The words “scaling,” “scale-up,” or “scaling-up” have clear meaning and importance in implementation science. In scaling-up, an EBI designed for one setting  is expanded to other health delivery units within the same or very similar settings under which it has been tested . An expectation of beneficial impact when scaling-up relies upon Cook’s principle of proximal similarity  because In contrast to this perspective on scaling-up, we use the term scaling-out to refer to specific variants in implementing an EBI, policy, or set of programs that are evidence-based. As shown in Table We present a logical argument regarding the degree of empirical evidence needed for scaling-out that extends beyond the trials that established the original evidence of impact. Though we introduce three types of scaling-out, we focus on types I and II scaling-out in this paper, wherein either the target population or delivery system remains fixed, respectively. We note that the logic behind borrowing strength from previous studies requires that we justify that key elements of the intervention still exist and are delivered with fidelity, that the delivery system retains critical components of the implementation strategy, and that broader ecological systems are still supportive of the delivery and sustainment of this intervention. We describe four levels of evidence and recommend that evidence of the effectiveness of scaling-out can be supported if we establish that mediational pathways have equivalent strength as they did in the original trials.In one sense, the concept of scaling-out of an EBI is analogous to off-label use of a pharmaceutical that has been approved for patients having a specific indication. For example, the US Food and Drug Administration’s (FDA) and other regulatory agencies’ limited approval of medications to be used for specific conditions is designed to give strong assurance of what patients and physicians can expect under these limited settings. Except for specified contraindications, it is not illegal to prescribe FDA-approved drugs outside these settings  or at different dosages than originally approved. However, no assurance is given for their effectiveness upon doing so; there are no assurances of their safety, or that iatrogenic effects will be absent. Similarly, EBIs that are tested for effectiveness in one setting have some assurance of impact when scaled-up to health units and subjects in very similar settings. But we would have less confidence about the potential health impact of an EBI when scaled-out to different settings or populations. Intuitively one would expect that the more similar the delivery system, the broader the contextual setting, the intervention, and the population, the more we should be able to rely on previous evidence, and the less new empirical evidence should be needed, to anticipate a successful scaling-out.But exactly what is the degree of “borrowing strength” from previous research studies that we would want to rely on? When is it justified to expect the same or similar impact of an EBI in this new setting? What new effectiveness testing should be required when an EBI is moved to different setting or population? What similarities exist in the mechanisms of action in the scaling-out compared to that previously found? When is it legitimate from a causal inference perspective to combine previously collected effectiveness and mechanistic evaluation data with new evaluation data on the scaled-out version? When would an entirely new effectiveness trial be required for a scaling-out to re-establish the existing standard of evidence?To answer these questions, we begin with Cook’s conceptual principles of proximal similarity and heterogeneous irrelevancies . ProximaIn our view, one must either establish or be willing to accept the following similarities between scaling-out and previous research. First, even as the EBI is adapted to new settings or populations, it still must retain its core elements . Second,Here we emphasize two distinct types of scaling-out, “Type I: population fixed, different delivery system” and “Type II: delivery system fixed, different population” that we argue have potential for retaining their evidence-based status provided they also satisfy what we call “mediational equivalence” (discussed in greater detail below). In the discussion, we contrast these with both a typical scaling-up, wherein both the population and delivery system remain fixed, and the more complex scale-out, type III, wherein both the population and delivery system are different than originally tested.type I: population fixed, different delivery system scaling-out as implementation where an EBI is delivered through a different delivery system to the same population where it has previously been tested. This type of scaling-out pursues an alternative avenue to reach its target population. As an example, a number of evidence-based parent training programs have been tested and found to be effective in universal, selective, and indicated prevention trials [We formally define n trials –17. Mostn trials , mental n trials  but may n trials . In scaln trials –23. Famin trials , 25, andn trials , there in trials , 28. Thun trials . If we cSuccessful implementation of these parenting interventions within primary care would require a reorganization to screen and refer families as well as to integrate, co-locate, or establish formal agreements between service systems, agencies, or community-based organizations that can deliver such programs effectively. In addition, type I scale-out may require important changes in how the intervention is delivered. For example, the Familias Unidas training program has been delivered by facilitators in small parent group meetings as well as in the home with individual families. SafeCare is delivered in the home where parents can learn and practice skills in vivo. A parenting program that is initiated by primary care may need to replace such group and home delivery modalities. This may be facilitated through the support of more logistically efficient technologies and content, such as interactive content about parenting viewable through tablets in primary care waiting rooms and virtual groups, having mock home environments within the primary care setting, or use of technology for parents to practice in their homes with coaches to work with them remotely. These sessions would support parents in practicing skills related to child safety, health, and parent-child interactions.A final example of type I scaling-out is the delivery of pre-exposure prophylaxis (PrEP) to populations at high risk of HIV infection. In international and domestic clinical trials, adherence to PrEP medication has been demonstrated to reduce the risk of HIV infection by close to 90% among adult sero-negative men and women whose partners were infected with HIV and at-risk adult men who have sex with men, populations for whom federal guidelines released by the Centers for Disease Control and Prevention (CDC) recommend use of PrEP . In ordeWe define type II: delivery system fixed, different population scaling-out as implementation that extends the reach of an existing intervention to a novel population within a similar service system. Type II scaling-out uses the same delivery system but aims to reach a different population of individuals, groups, or families for which the intervention has not yet been tested. For example, a smoking cessation approach originally tested with behavioral health patients may be implemented for those with diabetes in the same managed care health system. However, populations may also vary by race or ethnicity, cultural heritage, and considerations would have to determine what core elements must be retained and what might be adapted.A large literature exists on cultural adaptation of evidence-based interventions , 33; manPrEP delivery for adolescents is an example of a type II: delivery system fixed, different population scaling-out implementation. The CDC PrEP guidelines  did not When scaling-out, having metrics to test specifically whether expected outcomes improve are important; however, we suggest that there are instances where it may not be necessary to include all, or even any of the health outcomes that might be included in an efficacy or effectiveness trial. To assess whether a scaled-out version has impact on the ultimate health outcome or distal target of interest, we propose an efficient evaluation that tests a limited set of means and relationships, and combines new data with evidence from previous trials. Typically, a more limited evaluation would collect a small amount of—or even no—health outcome data, but a substantial amount of implementation process and output data, what others have called implementation outcomes . It woulAs shown in Table To examine when such a limited evaluation design for scaling-out would be sufficient to judge whether the EBI retains effectiveness, we consider a simplified sequential mediational model presented in Fig. Cook’s principle of causal explanation  requiresFor all types of scaling-out, it may be possible to use the identical EBI . However, we more often need to adapt the intervention to fit the new population and/or new delivery system. To further describe the adaptations that need to occur during scaling-out, we build on commonalities that are present in three implementation frameworks, the Exploration, Preparation, Implementation, Sustainment (EPIS) framework , the DynConcern regarding the effectiveness of an intervention when adapted has been addressed for a number of specific EBIs, including child maltreatment interventions , substanOne of the first steps in scaling-out is to delineate the core elements of the EBI to be implemented. We define core EBI elements as activities or components of an intervention that are necessary in order to obtain the clinical or public health outcomes , 64–66. In this paper, we introduced and defined a new concept for implementation called scaling-out. Scaling-out provides an opportunity to use a strategic approach to improve the efficiency of moving an EBI from one setting to another and/or from one population to another. When scaling-out an EBI in a moderately different setting or with a different population, we suggest it is sometimes possible to “borrow strength” from evidence of impact in a prior effectiveness trial with additional empirical data deemed necessary by the nature and degree of adaptations. We take a mechanistic approach in suggesting that by testing underlying mechanisms or mediators of effects, the efficiency of testing EBIs for new populations or new service systems can be streamlined through a greater understanding of and use of prior data to borrow strength from previous effectiveness studies.We have focused primarily on two types of scaling-out in this paper for a single EBI and recommended different levels of empirical tests involving key mediators that would provide some assurance that the scaled-out implementation would produce its intended effects. Similar scaling-out approaches are possible when implementing other types of interventions including evidence-based decision support systems, such as Communities that Care –72, whicThe notion of level of similarity of a population or delivery system is an important consideration. Take type I scale-out first, wherein the population remains fixed and the delivery system is different. Type I scale-out will require implementers to develop ways in which population differences may be minimized. Inclusion/exclusion criteria for those to receive a given EBI could be utilized to clearly (as much as possible) define the service population. For example, adolescents with juvenile justice system involvement, a substance use disorder, and of a similar cultural background could be identified. For type II scale-out, wherein the delivery system is fixed, though the target population is different, there also could be multiple approaches. For example, San Diego County Mental Health and Los Angeles county Mental Health are both large public sector service systems operating within the same state in the US. They both provide services to a large extent through some direct service provision, but primarily provide services through procurement and contracting with community-based organizations to provide direct services to children and adolescents. While these systems do share some similarities, there are also some differences in regard to their procurement and contracting processes.For different types of scaling-out, we propose a logical framework that researchers and policy-makers can use to assert conditions under which an established sequence of mediational elements described in Fig. Turning such guidance on the levels of new knowledge required, as presented above, into a rigorous system of empirical evaluations for non-inferiority compared to previous findings, will require additional development and explication of statistical methods that will be presented in a follow-up paper. At the heart of the scale-out approach is borrowing strength through the use of mediational modeling. Mediational analysis has an extensive history , 75 but This proposed framework for implementation research regarding scaling-out still leaves unanswered details regarding how best to assess changes in the complex implementation systems that we construct to deliver EBIs. Major implementation strategy adaptations often occur, and indeed are sometimes required to support implementation in delivery systems with widely different system and/or organizational cultures and climates, readiness, and resources , to popuTo close this discussion of a framework for asserting that a scaling-out is expected to share the original version’s impact on health outcomes by relying on previous studies and new empirical data, we note that more rapid implementation is especially important to the delivery of effective interventions to minorities and other populations experiencing health or health service disparities that would otherwise not benefit from the extensive research required to demonstrate effectiveness , 91.


In this paper, a photosensor-based latency measurement system for head-mounted displays (HMDs) is proposed. The motion-to-photon latency is the greatest reason for motion sickness and dizziness felt by users when wearing an HMD system. Therefore, a measurement system is required to accurately measure and analyze the latency to reduce these problems. The existing measurement system does not consider the actual physical movement in humans, and its accuracy is also very low. However, the proposed system considers the physical head movement and is highly accurate. Specifically, it consists of a head position model-based rotary platform, pixel luminance change detector, and signal analysis and calculation modules. Using these modules, the proposed system can exactly measure the latency, which is the time difference between the physical movement for a user and the luminance change of an output image. In the experiment using a commercial HMD, the latency was measured to be up to 47.05 ms. In addition, the measured latency increased up to 381.17 ms when increasing the rendering workload in the HMD. Nowadays, movies and game contents using virtual reality (VR) have taken a center stage because they provide greater immersion and realism. Thus, the VR market is expected to expand rapidly . EspeciaReducing the latency requires an accurate measurement system that can consider the human physical movement . The folAs a conventional and commercial measurement device, the Oculus latency tester  is the oHowever, the existing measurement system has three problems. First, it does not consider a physical head movement and simply measures the luminance change between the input and the output signals in the screen. The second one is compatibility. The Oculus latency tester is exclusively dependent on the Oculus Rift DK1 hardware. Thus, it cannot be used for other HMD devices, nor can it be used as a reference measurement system for measuring the latency. The third one is the low accuracy. The conventional method starts to measure the latency by pressing the button on top of the tester device. Using the signal activated from the tester device, the PC renders the specific patterns and sends them to the display of the HMD system. Therefore, it does not consider the physical movement of the HMD system . In addition, it is impossible to measure the latency change by changing the workload of the rendered image.In a previous research , a new mIn this paper, a novel latency measurement system with high accuracy for solving these problems is proposed. The proposed system can generate an accurate movement of the HMD and measure the movement using high-accuracy encoders and motors. Then, a photodetector system detects the luminance of the changed image for multiple movement directions and calculates the time difference between two events. Therefore, it has high accuracy and reliability and can accurately measure the motion-to-photon latency, which is a critical performance indicator of an HMD system. In addition, the proposed method is easy to apply to various kinds of HMD systems because it is developed considering the characteristics of common HMDs. -Consideration of the physical head movement-Compatibility between various HMD devices-High measurement accuracy-Easy applicability to various kinds of HMD systems.In summary, this paper offers the following contributions: The control PC controls each module and analyzes the measured signals, and the rotary platform is a physical device for modeling a head movement and for measuring the movement determined by the high-accuracy encoders and motors. The pixel luminance change detector measures the luminance change in the display of the HMD system and converts it into a voltage value. The oscilloscope measures and displays the voltage values of the measured signals. In this case, the typical range for the head movement is the rotation of the yaw direction, which is up to ±50°, and the rotation of the pitch direction, which is up to ±40°. The maximum angular velocity of the head movement is up to 780°/s in the yaw direction and up to 380°/s in the pitch direction . The pron-3)-th frame to the (n)-th frame. The changed luminance is converted into a voltage in the photosensor, and this voltage is measured using the oscilloscope. This permits the luminance change in the screen to be measured.The operation process of the pixel luminance change detection is as follows. The output image of the HMD panel is changed according to the HMD movement, as shown in 3DoutI and 2DoutI denote the luminance in the 3D virtual space and 2D mapped space in the display, respectively. sqx and sqy denote the horizontal and vertical indexes, respectively, which are sampled and quantized. x and y denote the horizontal and vertical indices in the 2D space, respectively.In this case, the luminance change in the virtual space should be considered instead of the pixel luminance change in the display panel because a change occurs in the image of the virtual space based on the head movement. In some cases, the luminance change could not be measured because the pixel luminance change is not sufficient enough to be detected by the photosensor. To solve this problem, a virtual lens technique is proposed as shown in p denotes the magnification of the virtual lens and foutI denotes the final output luminance. The proposed system uses p = 5, which was experimentally selected. If p is higher, the output luminance is also higher, and it is changeable.Using (1), the proposed method using the virtual lens is defined as follows:inV denotes an input signal at time t and N denotes the total number of input signals during a specific period. mV denotes an average voltage of the input signals with noise. cN denotes a candidate noise at time t, and eN denotes an estimated noise, which is the largest absolute value among candidate noises. It is set to the threshold value, and the proposed method only selects the voltage beyond this value. Finally, the time difference between two signals, namely, a pulse from the encoder and a pulse from the photosensor measuring the luminance change in the display of the HMD system, is measured. It is defined as follows:photot denotes the time point when the luminance changes in the HMD display, encodert denotes the time point when the physical movement of the HMD occurs, and Δt denotes the final motion-to-photon latency.The signal analysis and calculation modules finally compute the motion-to-photon latency using two signals, namely, a pulse from the encoder for the physical movement and a pulse from the photosensor measuring the luminance change. In this case, it is important to separate exactly an original signal from a noise signal by considering the sensitivity of the signal change. The proposed system uses a thresholding technique to remove noise. It is based on the change in measured signals to set the optimal threshold value. Specifically, it measures the number of n signals without changing the signal during a specific period as follows:Two different experiments for the latency measurement were performed using the proposed system. First, the motion-to-photon latency for the commercial HMD was measured. Second, this latency was evaluated by changing the graphic rendering workload for the HMD system.First, motion-to-photon latencies were measured by rotating in the yaw and pitch directions. Second, the change in motion-to-photon latency was measured according to the change in the image rendering workload. Generally, the HMD system requires a high-performance GPU because it needs to generate 3D-rendered contents with high resolution and a variety of visual effects. However, the rendering process generates a latency, and this latency is changeable according to the workload for rendering an output image. For example, if an output image is complex in the same hardware resource, the latency would increase. Therefore, in the following experiments, the change in latency was measured when varying the rendering workload in the HMD. The proposed measurement system used a model with a high number of polygons as a basic unit in the Unity game engine to set the rendering workload, as shown in This paper proposed a new measurement system that can accurately measure the motion-to-photon latency. The system, which uses high-accuracy DC motors and encoders with high resolution, could replicate the head movement, and it is possible to measure the rotation angles of the platform in realtime. In addition, the luminance change in the display could be detected using a photosensor. Specifically, the system is composed of a head position model-based rotary platform, pixel luminance change detector, and signal analysis and calculation modules. The proposed system can exactly measure the motion-to-photon latency. The experimental results showed that the latency was measured to be up to 47.05 ms. Additionally, the measured latency increased up to 381.17 ms when increasing the rendering workload in the HMD.The future work will focus on how to measure the motion-to-photon latency in successive frames using this developed measurement instrument. Specifically, the total system including the graphic user interface will be developed for tracking and expressing the head orientation by changing the intensity level of the image displayed in a HMD device.


This study aimed to explore an evidence-based nursing practice model of CRF management in hospitalized adult patients using the PARIHS evidence-implementation framework as the theoretical structure to provide guidance for similar nursing practices. The implementation of guideline evidence into clinical practice was conducted on the oncology and radiotherapy wards of a university-affiliated hospital. The process of integrating the guideline into the symptom management system of cancer patients was described. The impact of the evidence implementation was evaluated from three aspects: organizational innovations and outcome measures associated with nurses and with patients pre- and post-evidence implementation. During the implementation of evidence into practice on the wards, a nursing process, health education, a quality control sheet and CRF training courses were established. Through this implementation, compliance with evidence related to CRF increased significantly on the two wards, with that of ward B being higher than that of ward A. Regarding nursing outcomes, nursing knowledge, attitude and behavior scores with respect to CRF nursing care increased substantially after its application on the two wards, and the ward B nurses’ scoring was higher than that of the ward A nurses. Qualitative analysis concerning the nurses suggested that leadership, patient concern about CRF management, and the need for professional development were the main motivators of the application, whereas the shortage and mobility of nursing human resources and insufficient communication between doctors and nurses were the main barriers. Additionally, most nurses felt more professional and confident about their work. Regarding patient outcomes, patient knowledge, attitude and behavior scores regarding CRF self-management increased significantly. Patients’ post-implementation CRF was alleviated compared with the pre-implementation treatment cycle. The PARIHS framework may provide instructive guidance for the incorporation of evidence into practice, and the process-oriented framework might provide greater operational utility of the application. Cancer-related fatigue (CRF) has been reportedly experienced by 80%-90% of cancer patients undergoing chemotherapy or/and radiotherapy  and is cOur goal was to explore an evidence-based nursing practice model of CRF management in hospitalized adult patients during the clinical application of the “Clinical Practice Guideline: Nursing Care of Cancer-Related Fatigue in Adults with Cancer” (hereinafter referred to as the CRF nursing guideline) using the PARIHS framework of evidence implementation as the theoretical construct to provide guidance for similar nursing practices.This study acquired the ethics approval from the medical ethics committee of the First Affiliated Hospital of Soochow University on 25th March 2014, and the ID is 2014–147. All participants signed the informed consent.This evidence application project in the departments of medical oncology and radiotherapy of a university-affiliated adult hospital in Suzhou adopted a pre- and post-test design. Although most of the patients from the two units experienced clinically significant CRF, CRF had not been consistently screened for, assessed and treated as a dependent symptom . BaselinThe steering group was composed of six directors from the nursing department, department of medical oncology, department of radiotherapy, department of Traditional Chinese Medicine, and department of psychiatry and two researchers from the Evidence-Based Nursing (EBN) Collaborating Center of the Joanna Briggs Institute, Fudan University.The characteristics of the context were assessed using the observational method, including nursing routines or procedures, current regulations and rules related to the proposed evidence to be implemented, nursing workload, interdisciplinary support, organization culture of the research site, degree of support, and emphasis from managers.The evidence in the CRF nursing guideline derived from foreign professional institutions' guidelines , systemaEvidence application facilitations and barriers were evaluated and defined using focus group discussions. These factors were organized, analyzed and discussed to develop countermeasures.Context, Evidence and procedure, Facilitation during evidence application) was held to define the available resources , formulate proposed changes, and determine a CRF evidence-based nursing practice program.A steering group symposium focused on the baseline survey assessment results  using a “nursing quality checklist of inpatient CRF management” to evaluate the changes in CRF management on the pilot units.At the end of the clinical application, maximum difference sampling was used to select nurses to assess the nurses’ experiences and insights regarding this EBP project using semi-structured interviews. The included nurses were those who (1) had worked in the pilot unit for at least one year; (2) were fully engaged in this EBP project and (3) were willing to participate in the interview. COREQ criteria were used for the reporting of this qualitative research . After sThe pre- and post-implementation changes in nurses’ knowledge, attitudes and behaviors regarding CRF (NKAB-F) were surveyed using the 35-item scale developed based on the clinical guideline of CRF nursing care. This 3-point scale was reviewed by experts in evidence-based practice and oncology nursing, and its content validity index (CVI) was 0.96, indicating that this scale had good content validity .Patients’ outcome measures included two aspects: CRF self-management ability  and the changes in CRF scaling. The former was assessed using the 3-point (ranging from 1 = to no extent to 3 = to great extent) and 16-item “CRF self-management scale” . Self-management ability was also assessed using the 6-item “self-efficacy questionnaire for CRF management (SQFM)”, which was adapted from the “Self-efficacy Questionnaire for Cancer Patients, SQCP” ; its CVIt tests, the CRF scales and patients’ CRF self-management scales were compared between the two pilot units. Qualitative content analysis was conducted on verbatim transcripts of the semi-structured interviews. First, researchers read the transcripts carefully, extracted and marked the meaningful statements according to the interview outline, coded and classified these statements, and refined the themes using generic analysis methods  and unit B . Regarding CRF self-management ability, significant differences were found for patient knowledge, attitude and behavior scores between the baseline and final evaluation in both unit A and unit B .t test, the most severe CRF scores  during the treatment cycle that included the EBP project were lower than those of the prior treatment cycle . Before implementation, most cancer patients rested to alleviate their CRF, and after the EBP project, patients who had received CRF nursing education (which was developed based on the CRF nursing guideline evidence) had adopted more effective interventions for their CRF. According to their diaries, we found that the most frequent interventions to relieve CRF adopted by the patients during the treatment interval included exercise , Traditional Chinese Medicine (n = 11), taking drugs to improve appetite (n = 6), and massage (n = 6). Through these interventions, their CRF levels were effectively alleviated.During the implementation, a total of 30 patients completed the CRF diary. Using a paired From this EBP project, we confirmed the feasibility and effect of the EBN scheme based on the CRF clinical guideline and identified the following critical factors that influenced the SI of evidence into practice through a comprehensive examination and analysis of the entire application process.First, formulating simple and effective strategies and interventions of the innovation can guarantee the SI of evidence into practice. Because “high” evidence is the only basic element in the inception phase of the implementation in clinical practice ,42, the :, it also included the “opinion leader” in the study setting, whose opinions were often repeated and accepted by other nurses and whose behaviors were often imitated by other nurses. Particularly during the inception stage of the implementation when most of the staff nurses rejected the innovation, we identified the “opinion leader” in the study setting [Second, “high” context contributes to the SI of evidence into practice. Regarding “contexts conducive to change” ,42, we s setting , who triThird, the conversion of “external causes” into “internal causes” was fundamental to ensure the sustainability of the SI of evidence into practice . The difFinally, we examined the EBP practice based on the PARIHS framework. We divided the entire evidence application process into four stages: the preparation stage, inception stage, transitional stage and sustainable stage. During the preparation stage, the primary facilitators were the investigators, and the importance of the clinical problems to be resolved and the necessity of implementing the clinical innovations were emphasized. The leading factors were the evidence quality level and characteristics  (E) . During The implementation structure incorporating CRF guidelines into clinical practice using the PARIHS framework demonstrated some feasibility, appropriateness and effectiveness and might be used as a reference for the clinical utilization of other similar guidelines. The PARIHS framework may provide instructive guidance for incorporating evidence into practice, and the process-oriented framework might provide greater operational utility of the application.Due to the limitations of time and funds, the EBP was only conducted on two units, and the clinical practice implementation of evidence only continued for two months. The observation of patient outcomes only continued for one treatment cycle (one month). Additionally, this EBP project encountered a relative shortage of human resources in the setting, which might weaken other influential factors. In future research, a multi-center evidence application, a longer implementation process and outcome observations should be performed to examine the long-term application effects of the EBP project.S1 Table(DOCX)Click here for additional data file.S2 Table(DOCX)Click here for additional data file.S3 Table(DOCX)Click here for additional data file.S4 Table(DOCX)Click here for additional data file.S5 Table(DOCX)Click here for additional data file.S6 Table(DOCX)Click here for additional data file.S7 Table(DOCX)Click here for additional data file.S8 Table(DOCX)Click here for additional data file.