Equity-focused health impact assessment (EFHIA) can function as a framework and tool that supports users to collate data, information, and evidence related to health equity in order to identify and mitigate the impact of a current or proposed initiative on health inequities. Despite education efforts in both the clinical and public health settings, practitioners have found implementation and the use of evidence in completing equity focussed assessment tools to be challenging.We conducted a realist evaluation of evidence use in EFHIA in three phases: 1) developing propositions informed by a literature scan, existing theoretical frameworks, and stakeholder engagement; 2) data collection at four case study sites using online surveys, semi-structured interviews, document analysis, and observation; and 3) a realist analysis and identification of context-mechanism-outcome patterns and demi-regularities.We identified limited use of academic evidence in EFHIA with two explanatory demi-regularities: 1) participants were unable to “identify with” academic sources, acknowledging that evidence based practice and use of academic literature was valued in their organization, but seen as less likely to provide answers needed for practice and 2) use of academic evidence was not associated with a perceived “positive return on investment” of participant energy and time. However, we found that knowledge brokering at the local site can facilitate evidence familiarity and manageability, increase user confidence in using evidence, and increase the likelihood of evidence use in future work.The findings of this study provide a realist perspective on evidence use in practice, specifically for EFHIA. These findings can inform ongoing development and refinement of various knowledge translation interventions, particularly for practitioners delivering front-line public health services. Health inequities are defined as systematic and potentially remediable differences in one or more aspects of health across socially, demographically, or geographically defined populations or population subgroups . These dDespite dissemination, training, and efforts at application in both the clinical and public health settings, practitioners have found implementation and the use of evidence in completing EFHIA tools to be challenging, demonstrating an important knowledge-to-action gap , 5. BaseIn this paper, we report on a realist evaluation of EFHIA completion conducted in four public health practice sites using a mixed methods case study approach. Realist science seeks to generate explanations through observing patterns in the data that recur often enough to support hypothesized mechanisms of action . These eRealistic evaluation is an emerging methodological approach , as indiTheory and proposition developmentObservations through multi-method data collectionAnalysis and identification of CMO configurations and demi-regularitiesThe realist evaluation cycle consists of three main phases :Theory aDeveloping propositions was informed by a scan of the literature, stakeholder engagement, three existing theoretical frameworks, and abductive reasoning.A total of 986 relevant abstracts were found through a librarian-guided search strategy, hand searching and expert advice. Articles covered realist evaluation, EFHIA evaluation, and/or evidence informed decision making in public health.We conducted informal interviews with five knowledge users from each case study  site and ten research team collaborators. Interviews sought to understand individual perspectives on the use of evidence in the EFHIA, the facilitators and barriers that practitioners experience in engaging with health equity evidence, and their definition of success in the use of evidence.We were informed by three existing evidence-to-practice theoretical frameworks that best informed our research question. First, we drew on a KT review conducted by the Institut National de Santé Publique du Quebec (INSPQ) . The INSJagosh et al.  case study sites already known to be implementing EFHIAs were approached to participate in the study. Four sites were recruited, and at each of these sites, a team of individuals actively involved in completing EFHIA formed the unit of our analysis. We refer to these as EFHIA teams. Each team used the EFHIA tool being implanted at their site, with some variation between tools. Case study sites included rural-urban and urban settings in two Canadian provinces (BC and Ontario). Topics covered by the EFHIA teams that were recruited to participate included oral health, food safety, sexual health, and child injury and prevention. The teams varied in size from two to five members, including front line staff members, managers, and policy analysts in varying combinations at each site. The time period that each team took to complete their health equity assessment tools ranged between four weeks and six months. Table Separate from the members of the EFHIA team, key informants (KI) were also identified and recruited at each participating site. These were individuals involved in EFHIA implementation at the site, including key health unit leaders, community members, and staff who support health equity and evidence use in the organization. These individuals were identified as critical to understanding the EFHIA implementation and context specific to the case study site including the organization’s evidence culture and health equity culture.Data collection methods included: 1) document review, 2) semi-structured interviews, 3) online surveys, and 4) observation  to provide additional context and outcome information. Relevant information was extracted into case study reports.Key informants participated in two semi-structured interviews. They were interviewed at baseline to better understand organizational contexts, including equity and evidence culture, and policy landscapes. They were also interviewed at end of the study for information on the EFHIA use, outcomes and impact within the organization, as well as their experience with the quality of evidence use in the final products.All individual EFHIA team members were interviewed at the midpoint in their EFHIA completion process and at end of study. Midpoint interview questions asked about team members’ experience with the tool completion process, with using and interpreting evidence, with any barriers and facilitators encountered, and about any anticipated outcomes from the tool. End of study interviews focused on the individual’s experience with completing an EFHIA, anticipated outcomes from EFHIA completion and any change in healthy equity knowledge or attitude.All interviews were analyzed using NVivo 9 software. An initial codebook was created with subsequent thematic coding of interview data into categories. All coding was completed by researchers JW, TP and IT. The initial codebook was updated as new categories and subcategories emerged during analysis and until saturation was reached.Surveys were completed by EFHIA team members at baseline and end of study. Pre and post surveys included open and close-ended questions to assess EFHIA team members’ attitudes, values, motivations, levels of experience, educational and professional qualifications, previous experience with completing an EFHIA, mandate to support EFHIA, and perceptions of leadership attitudes towards EFHIA. The end of study survey contained follow-up questions to the baseline survey to enable comparison, as well as questions related to evidence use and outcomes.Surveys were administered through SurveyMonkey and completed at the team members’ convenience within a specified timeframe. Online survey responses were exported into Excel and subsequently de-identified. For close-ended question responses , we used basic descriptive analysis to count the responses. Open-ended responses were transferred to NVivo and coded thematically.Lastly, we observed EFHIA teams during tool completion by observing meetings and reviewing relevant emails. The research team observed the EFHIA work processes, the KT activities taking place, barriers and facilitators encountered and any change in attitude that appeared to have occurred. Along with document review, team observation notes and email text were used to verify or interpret interview and survey data through the process described below.Overall we had 14 EFHIA team participants complete the study over the four case study sites. A total of 11 and 9 of 14 EFHIA team members responded to the baseline and final surveys, respectively. We interviewed 15 key informants at baseline and 7 key informants at end of study  and observed 15 team meetings.Data were categorized according to context (C), mechanism (M) and outcome (O). Using case study analysis, processes of pattern matching, retroduction, and iteration for each of the outcomes were identified for each case study site. We then identified the mechanisms that were triggered to generate these outcomes, and the contexts within which these mechanisms were triggered. As the CMO configurations were developed, we sought confirmatory and contradictory evidence from all the site specific data available . This CMOnce the CMOs operating at each site were identified, we tested our initial propositions through a process of pattern matching, , 29 whicAfter identifying patterns within and across study sites, we took a retroductive approach, hypothesizing possible mechanisms capable of generating the observed outcomes. Retroduction has been described as “abduction with a specific question in mind”  or “logiThe realist method of analysis is not linear, and involved an iterative analysis process with the sets of data outlined above . We usedWe coded 1500 items over 300 categories in 15 themes. Not all case studies contributed to all categories. Each coding category had an average of two sites contributing data.Data from all sources were used to generate comprehensive accounts of EFHIA completion and evidence use in four different health unit contexts. Document review and team member observations were used to help interpret and triangulate data analysis and interpretation. Using all data sources, we generated case study reports for each site with observed study outcomes, and the CMO connections. These individual case study site reports were shared with research participants to ensure accuracy and resonance with their experiences. Participants were also contacted several months post study to participate in a data interpretation meeting over teleconference to verify if our demi-regularity interpretations of the data collected were accurately represented.We present here the key findings from the CMO evidence for propositions and the two emerging demi-regularities.contextual categories, which included public health unit (PHU) description, team description, KT strategies, evidence and equity experience, attitudes towards evidence and equity, and EFHIA tool implementation. Mechanism categories included conceptual, practical and organizational factors. In total 14 potential mechanisms and 12 potential outcomes were identified from the data. Outcome categories included overall experience completing the tool, changes in attitude and use of evidence, engagement in the completion process, barriers and facilitators to evidence use, and actual evidence and equity engagement outcomes of tool completion. Not all context, mechanism or outcome categories identified could be matched to CMO relationships.Table Proposition 1: Knowledge brokering at the local site can facilitate evidence familiarity and manageability, and increase user confidence in using evidence“In our case a big thing was having [name of knowledge broker] there …and we were more comfortable… in the fact that she could just help to shed a little bit of light, and then she was able to say, here’s a couple of places you might want to look for some information.” PHUA.We found evidence supporting our initial proposition that knowledge brokering at the local site can facilitate evidence familiarity and manageability, and increase confidence in using the evidence.Few respondents participating in the baseline survey had significant experience identifying (4/11), assessing (3/11), and incorporating (6/11) evidence directly into their decision making. Six of eleven practitioners (55%) responding to the baseline survey indicated a lack of confidence in incorporating public health evidence in their work.“We’ll definitely be bringing it to the team …in future planning that we need to [be] taking into account adding some research aspect or literatures into any of the upcoming campaigns or projects that our team does.” PHUA.While we did not see increased evidence use in this study, increased self-confidence encouraged participants’ intention to use more evidence in the future.Proposition 2: Evidence sources aligned with user needs increases acceptance and use of the informationProposition 3: Adapting the knowledge to match user characteristics can encourage evidence use because there is increased understanding of the knowledge and consonance with the contentProposition 4: Correspondence between knowledge produced and the problem to be solved can facilitate evidence use because the users will perceive the knowledge as applicableWe were not able to corroborate our initial propositions as written; however we observed important trends in evidence use by type of information that correlated to our initial propositions 2, 3 and 4.“… the sources that I found useful have been conference presentations and poster sessions you will never see [in] a journal, right? So, you know, it really does kind of suggest that if you kind of want to find out what’s working in, local implementation of interventions to address inequity, you’re probably going to really have to make some connections with people at that level” PHUC.The most commonly accessed source of evidence used was local, practical evidence . This was the only evidence source majority of participants (10/11) stated during the baseline survey that they used “often” or “very often”. These sources were seen as accessible, available and applicable, and therefore more likely to be used. They were also considered to correspond better to user needs, providing practical information that could be applied directly to their work. In addition, we found that participants had more consonance, or identification with, sources that were familiar, including local surveillance data, PHU based data, and data from individual or others’ experience:“we can relate to this and using their examples really helped us…just spark ideas or even go about filling [the tool] out, kind of got the ball rolling.”PHUB.Another team member described referring to a health equity assessment tool completed by a team with a similar clinical background and feeling drawn to this evidence because There was one PHU team that demonstrated the opposite trend. The EFHIA team in PHUD “often” (2/2) used evidence from academic sources and rated the source highly. In this case, both team members were policy analysts as opposed to clinical staff or front line managers.Proposition 5: Knowledge brokering during the knowledge production process can help build relationships with users, establish trust and familiarity in the producer, and facilitate evidence useProposition 6: Knowledge brokering at the reception site can facilitate evidence use because users have timely access to knowledge, reducing the perception of barriers“…the biggest help was having [knowledge broker] as kind of our navigator, … it was a little bit like a light bulb went off, and I thought, okay, I get this now. And then it started to be really rewarding, because we were coming up with ideas that we hadn’t thought of before…” PHUA.We observed that participants’ trust in the knowledge broker changed their attitude towards evidence and increased intention to use evidence in the future.“I’ve relied on more of the committee, what has been brought to the table as compared to doing a ton of research on my own because I’ve got a number of other projects that are on my plate. So it’s sort of like I do read the stuff that comes in but I’m not going out and looking at it or evaluating it as I would assume that’s being done by those that are bringing it to the table.” PHUC.However, at one site we observed a different relationship with knowledge brokers, one of deference or “acquiescence”. In this case, participants were most likely to use evidence brought forward by the knowledge broker and simply accepted or deferred to sources that were easily available or brought to them rather than looking for evidence. Due to this acquiescence, other evidence sources were not explored. This occurred in a context of unclear mandate or expectations related to EFHIA.As described above, ‘demi-regularities’, are semi-predictable patterns where outcomes are linked to context through mechanisms , 34. OurDemi-regularity 1: Practitioners are less likely to incorporate information that they do not understand or do not “identify with” “…I feel like the literature review was probably the most challenging, partly because it’s very foreign to us. As we’ve mentioned before, we don’t typically do literature reviews in our program, we’re very clinical people and it’s kind of out there for us....” PHUA.Practitioners were unable to identify with academic sources, even while acknowledging that evidence based practice and use of academic literature was valued in their organization. Front line practitioners in particular acknowledged the importance of academic evidence “in theory” and evidence based practice as an organizational priority; however they had challenges appreciating their role in the evidence informed decision making process.“I must’ve read that article three times and I still didn’t really know if it was relevant at the end of it so I just went with it wasn’t relevant, right because… I didn’t really know what it was saying.” PHUB.Additionally, as noted in the propositions above, EFHIA team members encountered challenges when interpreting academic literature, even when there was knowledge brokering and capacity building support available.Demi-regularity 2: Practitioners will access information for decision making which is easily accessible, immediately available and directly applicable, for which they perceive a “positive return on investment”“I think that the population data and our internal statistics are really helpful ‘cause they say, hey, this is what we’re doing and this is who we’re meeting [the needs of] and who we’re not meeting [the needs of].” PHUC.We identified across all health units that local evidence from practice  was associated with a perception of a “positive return on investment” .“We did do a literature review as well but didn’t find a whole lot.” PHUA.“…my opinion at the beginning [of] it would be that oh my gosh, there’s not enough time for all of this work and all of this collection of data to just make one decision.” PHUB.In contrast, academic evidence was not seen as a good return on investment.“…so it was kind of using our own sort of personal…clinical knowledge or clinical experiences that was the easiest to, kind of navigate through.” PHUA.Team members felt a sense of comfort using local sources knowing that its relevance and applicability were sound.We came to some important conclusions about the use of evidence in EFHIA in public health unit (PHU) case study sites in Canada. First and foremost, we did not observe much documented use of academic evidence, thereby highlighting the importance of knowledge translation (KT) research to support evidence based public health practice, particularly in the area of front-line health equity practice.One of the significant observations found was differential use of evidence based on practitioner type. Most of our EFHIA case studies were assigned by local PHUs to front line practitioners for completion. While this makes sense, as they may be the most knowledgeable of their program, many of these staff did not feel that they had the experience to use formal evidence sources. While some had recent training in the area, these individuals still did not see this as an integral part of their job in which they should be updating their skills and competence. This included both staff and managers. In contrast, where we had policy level staff complete the tool, more evidence was used. This should lead us to question who is best positioned to complete the EFHIA tool within the public health structure – and identify other ways in which front line staff can best contribute to inform EFHIA completion.We found the most common types of evidence used were surveillance data and personal practice experience. This is consistent with our initial proposition that consonance with the type and content of evidence and correspondence between knowledge produced and the problem to be solved are important mechanisms for evidence use. Again noting that the predominant practitioner type in our study was front line staff, it begs to reason that they were most comfortable with information that related directly to their practice experience and work, including local data, personal experience and grey literature/experience from nearby communities. Martin et al.  noted thMcMahon  defined Knowledge brokers were able to help with evidence access and use. Many of our sites did not have formal knowledge brokering roles, and informal knowledge brokers emerged. There was often a high level of trust in the knowledge broker. Many EFHIA team participants seemed relieved to take the information presented to them by the knowledge broker and apply that to the program. Therefore it is imperative that if local health units were to implement a formal knowledge broker model that these individuals are well versed in critical appraisal, evidence interpretation, and options analysis, as their suggested sources or conclusions were not commonly questioned. There are some models for this that already exist, for example, National Collaborating Centre for Methods and Tools local knowledge broker mentoring program . LauncheThe conclusions we reached are similar to findings from a qualitative secondary analysis conducted by Martin et al.  on evideFurther research is needed to explore how to make research evidence more accessible to front line staff. In addition, capacity building efforts for front line staff should include information on how to extrapolate findings from the literature to local context. By increasing the use of evidence in EFHIA, this study may support a normative change in the inclusion of equity within daily programmatic goals, through the development of EFHIA decision-support material. The learnings from this evaluation will enable health practitioners to respond to the World Health Organization call to action to reduce health inequities, as it has the potential to facilitate broader adoption of EFHIA across other Canadian provinces.There were limitations in the data collected making it difficult to address all of our research questions regarding equity focused evidence use. While we hypothesized five initial propositions to test from our phase I research, our data points to only a few direct connections between mechanism and outcome, and they did not manifest exactly the way we hypothesized and defined the initial propositions. Our data only captured a small sample and may not reflect the entire organization’s perspectives. Staff changes on the EFHIA team also occurred near the end of the study, so not all the key informants and EFHIA team members interviewed at baseline and midpoint, respectively, were interviewed again at end of study. Because of this inconsistency, we were not able to capture all of the team members’ final perspectives. Another limitation is the varying amount of data collected from each study participant on each case study team. Furthermore, we found many interesting mechanisms occurring in single case study contexts, but were unable to extrapolate into semi-predictable patterns or demi-regularities. While we identified specific KT strategies that helped team members use evidence in the EFHIA tool, we were unable to make concrete connections between KT and evidence use. We had data that described specific evidence outcomes as well as the mechanisms that led to those outcomes, but had limited data that clearly illustrated the influence of KT strategies on mechanisms that triggered evidence use. Finally, information regarding organizational culture and policy was collected in 2016, and this context may have since changed and not be entirely reflective of the current health unit.The findings of this study provide a realist perspective on knowledge translation and evidence use in the implementation of EFHIA in public health practice. We identified important mechanisms, including evidence users’ intuitive appraisal of “return on investment” when using evidence and their preference to “identify with” evidence sources – leading to a preference for local and experience based data. Academic sources were less likely to have correspondence with users’ needs or consonance with the content. These findings can inform ongoing development and refinement of various knowledge translation interventions knowledge brokers could be used to mitigate these responses, with some limited success.


In the recent past, mental health care and psychiatric service organization conceptually and structurally changed. The introduction of antipsychotic drugs in the 1950s substantially changed the treatment of schizophrenia and other psychotic disorders .In recent years, treatment of severe mental illness (SMI) shifted from management and stabilization of symptoms, to the much broader and more ambitious goal of achieving functional recovery. Despite advances in pharmacological treatment of people with SMI, it has become clear that medications alone are not sufficient to achieve a full symptom remission and functional recovery –5.The effectiveness of drug treatments is further impaired by the total or partial non-adherence, affecting more than half of patients with SMI , 7. In tPsychiatric rehabilitation has a bidirectional focus, seeking to influence both the individuals' strengths and challenges related to these goals, and the community contexts in which the persons will live them out , 9. The In recent years, however, psychiatric rehabilitation has been better defined and its starting assumptions and paradigms have been refined and consolidated by evidence-based research. Now psychiatric rehabilitation offers structured interventions, defined by approved procedures and accurate assessment tools and measures, with clear target, in order to achieve specific outcomes in patients with SMI .In the last few years, several research groups in different countries identified a wide and increasing body of effective and effectiveness psychosocial rehabilitation practices , 12, 13.The term EBP takes into account those interventions shown to be effective in improving illness course and outcome, based on rigorous studies methodologically well-designed and conducted . There iOn the other hand, other psychosocial interventions are diffused and widely applied, although not yet evidence-based. Despite few studies or reviews are now available, some data have shown that additional use of the so-called expressive therapies, in particular art therapies, seems to reduce negative symptoms among schizophrenia patients, being able to reach an improvement in self-knowledge, awareness, as well as affective and relationship skills in this population –30. MoreCurrently, the two major challenges of mental health in increasing the quality of psychiatric services are: (i) redesigning services on the principles of recovery and (ii) implementing services that deliver interventions supported by scientific evidence. The two strategies are complementary and should be integrated with each other. At present, there is the risk of not of neither a proven effective intervention, nor a really person-centered treatment approach. Despite the growing scientific literature on this topic, the most serious problem is that evidence-based rehabilitative interventions are not widely available in real-world practice. The science-to-service gap—that is, the gap between practices knowledge effective and that available and provided in mental health services—is one of the most relevant problems in the public mental health system .Although evidence-based psychosocial interventions led to clearly promising findings, there are still some doubts and uncertainties regarding their usefulness and feasibility in the daily clinical practice of mental health services. For many years, high-income European countries continued to invest funds and resources in old and too expensive not-evidence-based, not-recovery-oriented and not-personalized care services. The major investments still concern psychiatric acute care, hospitalization and residency, day centers and resocialization activities carried out within psychiatric services . Even toAnother weakness in that psychiatric rehabilitation interventions lack of enough evidence about their precise indications, predictors of response and the presence of any contraindications and/or adverse effects. Not least, it should be taken into careful consideration the clinical studies methodology, such as the choice of outcome measures and the presence of confounding factors, which may give rise to different and/or wrong interpretations . EvidencThe situation is not so different in the USA; although the USA was among the first countries in the world to address the issue of evidence-based psychiatric rehabilitation services implementation, the quality of rehabilitative services offered in the USA seems to be even behind the other high-income countries .Moreover, mental health costs comprehend consistent funding for hospital and residential treatment and psychotropic drugs, significantly unbalanced with the funding allocated for psychosocial, recovery-oriented interventions .It is now important to emphasize how the implementation of evidence-based and recovery-oriented interventions takes a long time to be acquired by mental health workers and to overcome their resistance to change. Moreover, the achievement of these objectives also requires a clear leadership direction and a constant commitment to teach the staff the new techniques and to learn from their experimentation. Leadership should be strongly committed to providing resources and supports, giving relevance to continuing education on work-place, developing quality measures and recovery indicators that include the use of new practices. Furthermore, leadership should have the courage to eliminate inefficient practices often defended by mental health worker, families, politicians and other stakeholders. It often happens that the new learned techniques are neither transmitted to the working group and are nor they applied in the real-world setting.In this context, the public resources invested in staff training remain unused assets of a few professionals and are far from the everyday clinical practice. Research in the field of EBP demonstrated that the learning of theory alone does not significantly influence professional behavior and that the introduction of innovative treatments in the real practice occurs only if it responds to the mental health workers specific needs. According to Williamson et al.  there isImplementing EBP and recovery in mental health are complex processes that require not only a real transformation of paradigm and working modalities, but also a cultural change, updating the knowledge and information baggage applied for years. The latter represents one of the greatest obstacles and one of the most deep-rooted resistances to change in a mental health service. It is therefore not enough that just a few learn the skills, but rather the entire working group, or better the whole care system, should acquire all the strategies and tools that allow homogeneous levels of performance, with the purpose to apply the new methods and techniques learned and to assess their impact at all the mental health services levels.The existing literature on psychiatric rehabilitation delivery is still scarce all over the world. In particular, the operational translation of the psychiatric rehabilitation evidences and its theoretical concepts has never been the object of a careful analysis in order to verify which rehabilitation activities and techniques are actually offered in the mental health delivery system.Current psychosocial rehabilitation practices are highly variable in terms of methodology and contents, with relevant differences from one country to another, and also within the same country, according to the specific orientation and tradition characterizing each Department of Mental Health.There is the need to increase the knowledge and awareness about the state of the art of different systems of management and funding of psychosocial rehabilitation in the “real-world” settings. It is also crucial to reveal commonalities and divergences as well as strengths and limits of the various assets in the different countries. Furthermore, specific pathways should be designed and implemented with the aim to overcome all the personal, social, organizational and political barriers that deny evidence-based treatments for patients and their families.Moreover, it is now pivotal to increase our knowledge on: (i) different types of interventions, (ii) modalities of delivery of interventions, such as intensity and duration of treatments, (iii) assessment tools, (iv) outcome measures, (v) predictors of treatment response, (vi) persistence of intervention efficacy, and (vii) integration with pharmacological therapies and with other psychosocial interventions. The new acquired knowledge should be framed inside a personalized intervention, also taking into account patient's preference.Frontiers in Psychiatry journal aims at publishing original studies that advance the evidence and understanding of psychosocial treatment and recovery of people with SMI , consistent with the principles and values of psychiatric rehabilitation and person-centered care. Frontiers in Psychiatry—Social Psychiatry and Psychiatric Rehabilitation will describe the psychiatric rehabilitation process and aims to differentiate psychosocial interventions that can be classified as evidence-based psychiatric interventions from other not or not yet evidence-based. The new Frontiers section will examine the main psychiatric rehabilitation interventions within the framework of the psychiatric rehabilitation process, taking into account the evidence of their efficacy, effectiveness and efficiency. Psychiatric rehabilitation interventions are currently a mixture of evidence-based practices, promising practices and emerging methods that can be effectively tied together, providing a broad strategy to achieve personal functional recovery.In this context, scientific societies in different countries could play an important role in defining the best strategies to promote and disseminate among the stakeholders the principles underlying a patient-centered and evidence-based psychosocial rehabilitation. The same goal could also be achieved by sharing and disseminating scientific research in the field of psychosocial rehabilitation. In this regard, the Social Psychiatry and Psychiatric Rehabilitation section in the AV and SB participated in the writing process of the first draft of the manuscript, revised and approved the final version of the manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Rates of emergency hospitalisations are increasing in many countries, leading to disruption in the quality of care and increases in cost. Therefore, identifying strategies to reduce emergency admission rates is a key priority. There have been large-scale evidence reviews to address this issue; however, there have been no reviews of medication therapies, which have the potential to reduce the use of emergency health-care services. The objectives of this study were to review systematically the evidence to identify medications that affect emergency hospital admissions and prioritise therapies for quality measurement and improvement.This was a systematic review of systematic reviews. We searched MEDLINE, PubMed, the Cochrane Database of Systematic Reviews & Database of Abstracts of Reviews of Effects, Google Scholar and the websites of ten major funding agencies and health charities, using broad search criteria. We included systematic reviews of randomised controlled trials that examined the effect of any medication on emergency hospital admissions among adults. We assessed the quality of reviews using AMSTAR. To prioritise therapies, we assessed the quality of trial evidence underpinning meta-analysed effect estimates and cross-referenced the evidence with clinical guidelines.We identified 140 systematic reviews, which included 1968 unique randomised controlled trials and 925,364 patients. Reviews contained 100 medications tested in 47 populations. We identified high-to moderate-quality evidence for 28 medications that reduced admissions. Of these medications, 11 were supported by clinical guidelines in the United States, the United Kingdom and Europe. These 11 therapies were for patients with heart failure , stable coronary artery disease (intensive statin therapy), asthma exacerbations , chronic obstructive pulmonary disease (long-acting muscarinic antagonists and long-acting beta-2 adrenoceptor agonists) and schizophrenia (second-generation antipsychotics and depot/maintenance antipsychotics).We identified 11 medications supported by strong evidence and clinical guidelines that could be considered in quality monitoring and improvement strategies to help reduce emergency hospital admission rates. The findings are relevant to health systems with a large burden of chronic disease and those managing increasing pressures on acute health-care services.The online version of this article (10.1186/s12916-018-1104-9) contains supplementary material, which is available to authorized users. Emergency hospital admissions place a major burden on patients and health-care systems. Large increases in emergency admissions can cause delays and cancellations of elective procedures, prolong emergency department waiting times and increase the risk of hospital-acquired infections –5. EmergIdentifying interventions that reduce emergency hospital admissions is, therefore, a priority for health services globally, and there have been large-scale evidence reviews to address this , 11. HowThe objectives of this study were to review systematically the evidence to identify medications that affect hospital admissions and prioritise therapies for quality improvement by assessing the quality of evidence and cross-referencing the findings with clinical guidelines.Cochrane Handbook of Systematic Reviews of Interventions [We developed the methods using guidance on systematic reviews and overviews described in the ventions . The proventions  and publventions . In our We included systematic reviews of RCTs published in English that examined the effect of any medication on emergency hospital admissions in adults (16 years or older). We included reviews that searched two or more electronic databases and assessed and reported the quality of included studies. We defined a medication as any administered chemical or biological product. We included only reviews that reported at least one meta-analysed effect estimate for emergency hospital admissions that was not part of a composite measure. We defined an emergency hospital admission as an unanticipated admission or readmission to hospital that occurred at short notice because of a perceived need for immediate health care [We searched MEDLINE (OvidSP), PubMed, the Cochrane Database of Systematic Reviews & Database of Abstracts of Reviews of Effects, Google Scholar and the websites of ten national funding agencies and health charities, using broad search criteria from inception to February 2016. The search strategy was developed by a library and information scientist (NR). The websites of national funding agencies and health charities were identified using Google searches and by our academic-clinician co-authors. We also contacted experts in emergency admissions and reviewed the reference lists of included reviews to identify additional studies. Details of the search can be found in Additional file In pairs, we independently extracted information in duplicate on the characteristics of the reviews and assessed their quality using the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) tool . We haveWe assessed the quality of meta-analysed effect estimates showing statistically significant effects on admissions. We used criteria from the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group . Using tA minimal important threshold in effect had to be defined to utilise the GRADE method, specifically, the imprecision domain . To our I2 ≥ 50% or chi-squared P < 0.1) [The subgroup estimate was calculated to explain the presence of substantial heterogeneity in the summary estimate (P < 0.1) .The subgroup estimate was calculated from a subset of trials at low risk of bias .The subgroup estimate showed a significant reduction or increase in hospital admissions, while the summary estimate found no effect.We assessed the quality of subgroup effect estimates only if the subgroup analyses were pre-specified and met one or more of the following criteria:We considered high and moderate GRADE ratings to represent strong evidence, since the effect estimates are unlikely to change if additional studies are conducted , 21.To prioritise the therapies further, we cross-referenced the list of medications supported by high- or moderate-quality evidence with clinical guidelines. We conducted this analysis to ensure that the overall balance of benefit to harm for the therapies had been judged acceptable by key health-care stakeholders. We first cross-referenced the list with National Institute for Health and Care Excellence (NICE) clinical guidelines . NICE isTo standardise reporting, we used international classification systems. We described patient populations using the World Health Organization’s International Statistical Classification of Diseases and Related Health Problems 10th Revision (WHO ICD-10) . We clasFor significant effect estimates that underwent quality assessment, we calculated the number needed to treat to avoid one hospitalisation and the number needed to treat to cause one hospitalisation. For each estimate, we used the median control-group event rate from the RCTs in the meta-analysis , 38–40. We used Excel for data extraction and management. For the quantitative data analysis, we used STATA 14 . The resWe screened 11,442 titles and abstracts and 1563 full text articles , of which 690 RCTs reported hospital admission outcomes for 577,604 patients. The number of RCTs underpinning the treatment comparisons ranged from 1 to 184 (median 3), with patient sample sizes ranging from 18 to 88,367 (median 1116). The reviews contained data on 100 unique medications tested in 47 patient populations  inhibitors in patients for whom non-steroidal anti-inflammatory drugs (NSAIDs) were indicated, intermittent antipsychotic drug therapy in patients with schizophrenia and fluticasone in patients with COPD Table . EvidencWe examined 140 reviews of 100 medications, and identified high- to moderate-quality evidence for 28 therapies that significantly reduced admissions in 15 different populations. Eleven of these therapies were supported by major clinical guidelines from the United States, the United Kingdom or Europe. We also identified high- and moderate-quality evidence for three medications that increased admissions.Previous reviews of interventions to reduce hospital admissions have focused on non-pharmacological initiatives , 43–45. Our results complement a growing body of evidence about drug-related hospital admissions. Several systematic reviews of observational studies have shown that approximately 3% of all emergency hospitalisations are related to suspected adverse drug reactions and drug–drug interactions –51. DrugPolicymakers and commissioners may use these results to prioritise quality measurement and improvement efforts. We have systematically identified a list of 11 evidence- and guideline-based treatments for five major chronic diseases: heart failure, stable coronary artery disease, COPD, asthma and schizophrenia. These diseases cause millions of hospital admissions each year globally –56 and aThe results of this study may be fed into existing mechanisms for tracking and improving clinical practices in health systems. Prescribing data for some of the 11 evidence- and guideline-based medications are currently monitored in several health systems, for example, use of ACE inhibitors, angiotensin II receptor blockers and beta blockers is currently measured as part of the UK Quality and Outcomes Framework, a pay-for-performance incentive structure that has demonstrably reduced prescribing gaps for these medications and improved prescribing efficiency for other medications –65. All For some of the therapies in this review, bridging treatment gaps and improving prescribing may result in immediate benefits and may rely only on a small number of stakeholders. For example, we found evidence that treating patients with acute exacerbations of asthma in the emergency department with inhaled corticosteroids or anticholinergics helped to avoid hospital admissions. Optimal utilisation of these medications could be achieved through direct efforts by emergency department physicians and hospital pharmacists.However, many of the therapies in this review form part of ongoing chronic disease management, and beneficial effects occur after months of use. Optimal utilisation of these medications would require coordination of multiple stakeholders, including physicians in different specialties, home care or case management nurses, community pharmacists and patients.We can estimate potential reductions in hospital admissions by combining the results of our study with data on existing treatment gaps and disease prevalence. For example, in studies of prescribing for heart failure in the United States and Europe, 13% of patients with reduced left ventricular ejection fractions who are eligible for treatment do not receive the first-line therapy , 66. It Our results also reinforce the dangers of prescribing COX2 inhibitors, particularly for patients at risk of cardiovascular disease, inhaled corticosteroids in patients with moderate to severe stable COPD, and intermittent antipsychotic drug therapy for patients with schizophrenia. It is, therefore, reassuring that the harms associated with the use of these drugs and steps to ensure that they are used appropriately, if at all, have been reported widely in clinical and government prescribing guidelines –73.In addition to high- and moderate-quality evidence, this study identified 28 medications with low- and very low-quality evidence for reducing admissions. According to the GRADE working group, these estimates for reducing admissions are likely to change if additional research were conducted . TherefoOnly 1% of the reviews examined the effect of medications in patients with multi-morbidity. Given the challenges of effective clinical management and high hospitalisation risk for patients with multiple diseases , 74, we Low- and very low-quality evidence indicates the need for high-quality research to increase confidence in the reliability of effect estimates. Some of the medications in this overview were supported by low- and very low-quality evidence, suggesting a need for additional high-quality research. Hospitalisations, however, are only one important patient and health system outcome. A larger set of core outcomes that reflect patient and health system priorities should be considered when establishing research priorities, including assessment of mortality, adverse events, quality of life and cost.Similarly, there were 17 drugs with high- and moderate-quality evidence that were not supported by clinical guidelines. We did not record whether the medications had been evaluated. In formulating guidelines, the effect of an intervention on reducing emergency admissions would form only one of many criteria considered by multi-disciplinary panels of stakeholders. We would not recommend that drugs be included in guidelines or considered for inclusion solely because they reduce hospital admissions.Researchers should consider reporting rates of hospital admissions, as opposed to ratio measures; 476 of the 517 (92%) effect estimates we reviewed were reported as odds or risk ratios. These are crude measures of hospitalisation, as they assess admission as a binary outcome: present or absent. These effect measures equate a patient who has had one admission during follow-up with a patient who has had five admissions. In patients with chronic diseases, such as heart failure or COPD, for which hospital admissions are common, rate-based measures may have greater utility in evaluating the effectiveness of interventions.This study has three key strengths. First, it was comprehensive. We identified, analysed and synthesised information on nearly one million patients to identify the highest quality evidence for medications that affect emergency hospital admissions. Secondly, we minimised the impact of duplicate RCT evidence between reviews by excluding outdated reviews; every review we included has a unique set of hospital admission data. Thirdly, we classified all patient populations using ICD-10 and therapies using WHO ATC. This helped to homogenise and simplify the data, which was heterogeneously reported in the systematic reviews. When possible, we also converted quantitative data to comparable measures and units (i.e. risk ratios for estimates and months for follow-up duration). This will enable users of our review to navigate and interpret this large body of evidence.This study has some limitations. First, although we extracted and reported all effect estimates, we have conducted only quality assessments on significant estimates. While it may be useful for decision makers to know the quality of evidence for all tested interventions, our aim was to support decision-making by identifying and prioritising therapies for which an effect has been demonstrated. All the estimates are listed in the online database Additional file 2:Database 1. (XLSX 130 kb)


Clinical guidelines are important and influential; they can improve processes involved in patient care, thereby also improving patient outcomes . They arAccordingly, many organisations and professional societies issuing clinical guidelines have formalised their development process. This has several advantages, but perhaps most significantly, it permits methodological rigour to be embedded into the guideline development process in a consistent fashion. The organisation is able to stipulate, in advance, minimum standards for literature searching, evidence synthesis, construction and ratification of guideline statements; all of which would otherwise be at the whim of individual guideline development committees or groups. Yet, what makes a high-quality guideline? Fortunately, this question has been the subject of many years’ work by several organisations, notably the AGREE consortium, whose most recent iteration (AGREE II) encompasses 23 items that can be used to assess the quality of a particular clinical guideline . As a byhttps://www.esgar.org/guidelines-publications/research-committee/).The European Society of Gastrointestinal and Abdominal Radiology (ESGAR) has developed its own guideline development process, aiming to standardise and improve the quality and transparency of guidelines issued under its aegis. This was initiated in the mid-2017, and guidelines published from October 2018 onwards will adhere to its structure. The key components of this process, which adhere to the principles of AGREE II, are summarised in Fig. Potential topics for guidelines will be sought from the ESGAR membership, members and leaders of other societies and members of the public and journal editors. A form is available on the ESGAR website to assist this (at the same URL as listed above). Topics will be prioritised based on specific criteria, namely the burden of the relevant disease , the impact that this has on radiological services, the extent of uncertainty in current clinical practice  and the availability of published evidence upon which a guideline could be based. The feasibility of developing a guideline using the resources and expertise of the ESGAR membership will also be considered. These criteria will be assessed by the ESGAR research committee before a new guideline is commissioned.The guideline development group will construct and circulate an initial questionnaire in order to identify the scope and aims of the guideline, and prioritise topics for detailed literature review. Thereafter, where possible these topics will be converted into specific clinical questions adopting the PICO format . These clinical questions will then serve as the basis for the literature search. The literature search, including the databases used, search terms, inclusion dates and language restrictions, will be documented clearly and used uniformly by the guideline group. Relevant articles will be retrieved and summarised into evidence tables with accompanying explanatory text, including an assessment of the literature quality.Using the evidence synthesis tables and explanatory text, the guideline group will construct draft consensus statements for the final guideline. Where the evidence is contradictory or incomplete, relevant statements can still be constructed, but may need to be based on the group’s opinion with anonymous voting, using a modified Delphi process. Each voting member will be asked regarding their agreement with a given statement on a five-point scale . For a statement to achieve consensus, 80% of guideline group members must agree or strongly agree with that statement. Statements not achieving consensus initially may be modified and subjected to repeat voting; a maximum of three iterations is suggested as the upper limit before a statement is defined as failing to reach consensus. Statements that do not achieve consensus may be included in the Discussion of the final guideline document, but should not be recommended by the group.The final output of the process will be a guideline document, which will include its background, target audience and endorsing societies. The consensus statements themselves, with an associated level of evidence quality and strength of recommendation, will be provided, with the evidence summary tables and explanatory text made available as Well-constructed clinical guidelines have the potential to substantially improve patient care across multiple institutions in different countries. To maximise their benefit, they must be prioritised, developed, disseminated and implemented using methodologically sound and transparent techniques. The ESGAR is committed to ensuring consistency and excellence in its guidelines, and all future ESGAR guidance will follow the principles outlined here.ESM 1Full ESGAR guideline development process document (PDF 1031 kb)ESM 2Example search strategy for guideline evidence synthesis (PDF 445 kb)ESM 3Example evidence summary tables (PDF 1115 kb)


Group A rotaviruses are the major causative agents of pediatric gastroenteritis worldwide. Several studies have reported the predominance of G4P[6] rotavirus genotypes in Korean neonates, which is uncommon in other countries. Therefore, the purposes of this study were to determine the genotype constellations of complete genomes of G4P[6] rotavirus strains isolated from Korean neonates using next-generation sequencing, to compare these sequences with other G4P[6] strains in other countries, and to determine the reason for the predominance of G4P[6] genotypes in Korean neonates.Twenty rotavirus G4P[6] strains, isolated from January 2013 to January 2016, were selected for whole-genome sequencing. Eleven rotavirus genes were amplified using specific primer sets, and sequencing was carried out using an Ion S5 XL next-generation sequencing platform. Genotypes of each gene were determined, and phylogenetic analyses were performed to investigate genetic distances between genes of rotaviruses in this study and those of other rotavirus G4P[6] strains whose whole-genome sequences were previously published. All 20 rotavirus strains in this study had the same genotype: G4-P[6]-I1-R1-C1-M1-A1-N1-T1-E1-H1, representing the Wa-like genotype constellation. BLAST searches of 20 G4P[6] rotavirus strains revealed that all G4 sequences in this study showed the highest nucleotide identity to G4 sequences of G4P[6] rotavirus strains isolated in Korea in 2008 (GenBank accession number: FJ603447). Additionally, P[6] gene sequences in this study showed the highest nucleotide identity to P[6] sequences of G4P[6] strains detected in Korea in 2002 (AY158093). Phylogenetic and nucleotide sequence analyses showed that G4P[6] strains in this study and previously reported G4P[6] strains in Korea were mostly detected in neonates and had similar G4 and P[6] sequences compared with other G4P[6] strains detected in other countries.This study showed that the whole-genome constellation of rotavirus G4P[6] strains from Korean neonates resembled a Wa-like genotype constellation. Additionally, rotavirus genotypes detected in Korean neonates had unique P[6] sequences, which may be the cause of Korean neonatal rotavirus infection.The online version of this article (10.1186/s13099-019-0318-5) contains supplementary material, which is available to authorized users. Group A rotaviruses are the most important pathogens causing pediatric gastroenteritis worldwide. The virus contains a triple-layered capsid surrounding a genome of 11 double-stranded RNA segments . G1–G4 aIn Korea, G1P[8] is the most frequent genotype in children, and G4P[6] is the most frequent genotype in neonates , 9–12. TNext-generation sequencing (NGS) technology has recently been applied to viral genome research and human genome research . NGS canIn this study, we attempted to determine the genotype constellation of the complete genome of G4P[6] rotavirus strains characteristically isolated from Korean neonates using NGS and to compare the sequences of Korean G4P[6] strains with G4P[6] strains in other countries. In addition, we tried to determine the reason for the predominance of G4P[6] genotypes in Korean neonates.Genotype constellations of the rotaviruses isolated from 20 neonates in this study were all the same . BLAST searches of 20 G4P[6] rotavirus strains revealed that all G4 sequences in this study showed the highest nucleotide identity to G4 sequences of G4P[6] rotavirus strains isolated in Korea in 2008  and high genetic similarities suggested the possibility of persistent infection with the same rotavirus strain over 3 years in one hospital. However, 11 of the 20 G4P[6] rotavirus cases were detected on the first admission day and were transferred from other hospitals or clinics, indicating the occurrence of outside infection because rotavirus infection requires an incubation period for at least 2 days. Additionally, G4P[6] rotavirus infection in Korean neonates has been reported in several studies in other cities in Korea since 1999 , suggestThe various genotype constellations of the G4P[6] strain are thought to originate from reassortment events of the I5, A8, and T7 genotypes in the original G4-P[6]-I1-R1-C1-M1-A1-N1-T1-E1-H1 genotype constellation Table . The reaBoth genotypes of G8P[6] and G4P[6] were frequently detected in Korean neonates, and sequence similarities were observed between P[6]s in G8P[6] strains and P[6]s in G4P[6] strains, whereas differences were found in P[6] sequences from G4P[6] strains detected in other countries. These findings suggested that selective infection by rotaviruses with these unique P[6] sequences occurred in Korean neonates. Moreover, previous reports have demonstrated that the VP8 portion of VP4 attaches to the human blood group antigen (HBGA) in the intestinal epithelium and that there is an association between the antigenicity of VP4 (VP8) and HBGA . TherefoIn summary, G4P[6] strains isolated from Korean neonates in 2013–2016 had the same genotype constellation, G4-P[6]-I1-R1-C1-M1-A1-N1-T1-E1-H1 (a Wa-like constellation). Korean G4P[6] and G8P[6] strains have been shown to easily infect Korean neonates, and the common Korean P[6] sequences in G4P[6] and G8P[6] strains have unique nucleotide sequences compared with G4P[6] strains detected in other countries. This may be the cause of the association between P[6] and Korean neonatal rotavirus infection. Further studies are needed to determine the mechanisms through which P[6] genotypes easily infect Korean neonates.Rotavirus-positive stool samples were collected from neonates younger than 1 month of age in a 650-bed hospital from January 2013 to January 2016. Twenty G4P[6] rotavirus-positive samples were successfully genotyped for whole-gene genotyping using NGS . During this period, 270 rotavirus antigen-positive samples from neonates with symptomatic diarrhea were collected, and 56 samples were arbitrarily selected for this G4P[6] whole-genome sequencing study. Forty-nine samples from these 56 samples (87.5%) were genotyped as G4P[6] strains using G and P typing (seven samples were non-G4P[6] strains). Of 49 G4P[6] strains, 20 samples were successfully amplified for all 11 rotavirus genes evaluated in whole-genome sequencing. Clinical data, including age and sex, were collected from patient medical records. Eleven (55.0%) samples were collected from males, and the overall median age of the donors was 11 days (range 5–28 days). This study was approved by the Institutional Review Board of Hallym University Dongtan Sacred Heart Hospital .http://www.clcbio.com/). Sequenced reads were trimmed and mapped to the rotavirus reference sequence (ASM265499v1 or ASM268153v1), and consensus sequences of each gene were obtained. Because we could not obtain the sequences of VP7 genes by NGS, VP7 genotyping was carried out using RT-PCR and Sanger sequencing with another specific primer set  and NGS. Viral RNA was extracted from fecal suspensions using a QIAamp Viral RNA Mini kit  and the QIAcube platform (Qiagen). The RNA was denatured and reverse transcribed using the SuperScript III First-Strand Synthesis System . Eleven rotavirus genes were amplified from the double-stranded RNA genome using specific primer sets described in Additional file The genotypes of gene sequences were obtained using the Rota C v2.0 online automated genotyping tool , and whohttps://www.ncbi.nlm.nih.gov/genome/viruses/variation/). MEGA software version 7 was used for phylogenetic analysis [Phylogenetic and nucleotide sequence analyses were performed to investigate genetic distances among rotavirus G4P[6] strains in this study and comparative G4P[6] strains having whole-genome sequences, including RVA/Human tc/GBR/ST3/1975/G4P2A[6], RVA/Human wt/CHN/E931/2008/G4P[6], RVA/Human wt/CHN/GX54/2010/G4P[6], RVA/Human wt/CHN/GX77/2010/G4P[6], RVA/Human wt/CHN/GX82/2010/G4P[6], RVA/Human wt/CHN/R479/2004/G4P[6], RVA/Human wt/CHN/R1954/2013/G4P[6], RVA/Human wt/THA/CMH-N014-11/2011/G4P[6], RVA/Human wt/THA/CMH-N016-10/2010/G4P[6], RVA/Human-wt/LKA/R1207/2009/G4P[6], RVA/Human wt/IND/mani 362/2007/G4P[6], RVA/Human wt/ARG/Arg4605/2006/G4P[6], RVA/Human wt/ARG/Arg4671/2006/G4P[6], RVA/Human wt/HUN/BP271/2000/G4P[6], RVA/Human wt/HUN/BP1125/2004/G4P[6], RVA/Human wt/HUN/BP1227/2002/G4P[6], RVA/Human wt/HUN/BP1231/2002/G4P[6], RVA/Human wt/HUN/BP1490/1994/G4P[6], RVA/Human wt/HUN/BP1547/2005/G4P[6], RVA/Human wt/HUN/BP1792/2004/G4P[6], RVA/Human wt/HUN/BP1901/1991/G4P[6], RVA/Human wt/COD/KisB332/2008/G4P[6], RVA/Human/NCA/OL/2010/G4P[6], RVA/Human wt/PRY/1809SR/2009/G4P[6], RVA/Human-wt/ZMB/MRC-DPRU1752/XXXX/G4P[6], RVA/Human wt/KOR/CAU 195/2006/G12P[6], and RVA/Human wt/KOR/CAU 214/2006/G12P[6]. The previously reported G4 and P[6] sequences of rotavirus G4P[6] strains detected in Korea (KF650074–650095)  strains in this study and other G4P[6] rotavirus strains with full genome sequences. Black circles indicate the G4P[6] strains isolated from neonates in this study, and black diamonds indicate the Korean G4P[6] strains from GenBank.Additional file 2: Fig. S2. Phylogenetic tree of VP1 (R1) sequences of rotavirus G4P[6] strains in this study and other G4P[6] rotavirus strains with full genome sequences. Black circles indicate the G4P[6] strains isolated from neonates in this study, and black diamonds indicate the Korean G4P[6] strains from GenBank.Additional file 3: Fig. S3. Phylogenetic tree of VP2 (C1) sequences of rotavirus G4P[6] strains in this study and other G4P[6] rotavirus strains with full genome sequences. Black circles indicate the G4P[6] strains isolated from neonates in this study, and black diamonds indicate the Korean G4P[6] strains from GenBank.Additional file 4: Fig. S4. Phylogenetic tree of VP3 (M1) sequences of rotavirus G4P[6] strains in this study and other G4P[6] rotavirus strains with full genome sequences. Black circles indicate the G4P[6] strains isolated from neonates in this study, and black diamonds indicate the Korean G4P[6] strains from GenBank.Additional file 5: Fig. S5. Phylogenetic tree of NSP1 (A1) sequences of rotavirus G4P[6] strains in this study and other G4P[6] rotavirus strains with full genome sequences. Black circles indicate the G4P[6] strains isolated from neonates in this study, and black diamonds indicate the Korean G4P[6] strains from GenBank.Additional file 6: Fig. S6. Phylogenetic tree of NSP2 (N1) sequences of rotavirus G4P[6] strains in this study and other G4P[6] rotavirus strains with full genome sequences. Black circles indicate the G4P[6] strains isolated from neonates in this study, and black diamonds indicate the Korean G4P[6] strains from GenBank.Additional file 7: Fig. S7. Phylogenetic tree of NSP3 (T1) sequences of rotavirus G4P[6] strains in this study and other G4P[6] rotavirus strains with full genome sequences. Black circles indicate the G4P[6] strains isolated from neonates in this study, and black diamonds indicate the Korean G4P[6] strains from GenBank.Additional file 8: Fig. S8. Phylogenetic tree of NSP4 (E1) sequences of rotavirus G4P[6] strains in this study and other G4P[6] rotavirus strains with full genome sequences. Black circles indicate the G4P[6] strains isolated from neonates in this study, and black diamonds indicate the Korean G4P[6] strains from GenBank.Additional file 9: Fig. S9. Phylogenetic tree of NSP5/6 (H1) sequences of rotavirus G4P[6] strains in this study and other G4P[6] rotavirus strains with full genome sequences. Black circles indicate the G4P[6] strains isolated from neonates in this study, and black diamonds indicate the Korean G4P[6] strains from GenBank.Additional file 10: Table S1. Primers used in this study [is study .


Due to the increasing amount of available published evidence and the continual need to apply and update evidence in practice, we propose a shift in the way evidence generated by learning health systems can be integrated into more traditional evidence reviews. This paper discusses two main mechanisms to close the evidence-to-practice gap: (1) integrating Learning Health System (LHS) results with existing systematic review evidence and (2) providing this combined evidence in a standardized, computable data format. We believe these efforts will better inform practice, thereby improving individual and population health. Investments in “Big Data” in the absence of concomitant investments in appraising, synthesizing, and translating that evidence into practice will not by themselves realize the greatly anticipated effects on individual and population health. Closing the evidence-to-practice gap will require two key elements: (1) the integration of locally and rapidly generated evidence—what we are calling “Learning Health System evidence”—with cumulative and comprehensive systematic review evidence from the peer-reviewed literature and (2) making this combined evidence available in standardized computable forms so it can be efficiently and effectively assimilated to inform practice.Patients expect that the care provided to them will be based on the best available evidence. However, it is widely recognized that, absent directed efforts to bring evidence to practice, many years will elapse before new, validated evidence will actually be integrated into practice.Crossing the Quality Chasm (2001) and re-expressed in 2007,2 described the generation of evidence as a by-product of care delivery and application of that evidence to support continuous improvement, evidence-based care delivery, and population management. As such, the LHS concept requires that evidence generation not be an end in itself; efforts to generate evidence must be accompanied by equally emphasized efforts to apply it to improve health. Currently, there are no pathways for harvesting new evidence, produced by LHSs or any other methods, besides publication in peer-reviewed journals. In this report, we argue for the need to integrate all sources of high-quality evidence addressing a particular question—published or not. We also define a LHS broadly as any entity that routinely and continuously seeks to generate and learn from data, for purposes of improving individual and population health. LHS’s can and increasingly do exist in multiple forms and at multiple levels of scale: single-delivery systems such as the Mayo Clinic,3 collaboratives such as the High Value Healthcare Collaborative , and national-scale practice-based networks such as PCORnet .4The Learning Health System (LHS), first envisioned by the Institute of Medicine in 5While locally generated LHS evidence is clearly important, it is not sufficient. To safely guide practice, local results must combine with the larger body of what is already known about a health problem. What is needed is an expanded cycle of learning that allows “external” evidence from trials, studies, and reviews to inform practice within Learning Health Systems, and conversely, allows data from practice to feed back into the overarching evidence base. In Figure 7 The Agency for Healthcare Research and Quality’s Evidence-based Practice Centers (EPC) program has a long history of engaging patients and other stakeholders and has been engaging health systems through projects to better understand the formats and critical elements that health systems need for evidence that can change practice. The EPC program and the Cochrane Collaboration are also experimenting with methods to improve efficiency of the production of evidence reviews without compromising scientific rigor so that evidence can be continuously updated and available. This sets the stage for integration of external knowledge into LHS activities, as illustrated by the gold arrow in Figure Alongside increasing ability for LHSs to generate and analyze their own data—and share the results—producers of systematic reviews are working to tailor both the review process and presentation of results to facilitate bringing evidence to practice.As illustrated in the dotted green line in Figure 8New infrastructure will be necessary to support this shift toward near real-time evidence sharing and the routine application of evidence to change practice and improve care through the evolution of LHSs. The initial step in this process is a recognition of the importance of putting knowledge into standardized, computable representations, augmenting varied representations in text, tables, and figures. The importance of this change forms a pillar of the National Library of Medicine’s 2018 Long Range Plan.Biomedical evidence has traditionally been represented in human readable form—in text, tables, and figures—and published in books and journals. However, knowledge in this format cannot readily promote the mass action required to support the rapid and routine translation of new knowledge to practice, especially given the accelerated pace of new evidence generation in the expanding LHS environment, thus, the urgency for addressing this issue now. Driving improvement with knowledge in human readable form will be limited by access, time, and resources needed to fully inform priority areas for improvement. This current state approach cannot possibly succeed. There is too much entirely new evidence and existing evidence is changing too rapidly.9 several software products  and standards10 that support creation of DKOs and their management in digital libraries already exist. These resources, which will continue to mature, combine to make this vision an achievable reality.To enable mass action as we enter the era of LHS, evidence of all types must be available in standardized, computable formats to support the curation, representation, dissemination, and application of evidence. Evidence of all types  can be packaged as digital knowledge objects (DKOs) using a standardized format. Metadata attached to each DKO will describe the knowledge held in the object and will provide essential information about the analytic results used to generate the knowledge, including indicators of the strength of the evidence. These DKOs can be curated and managed in digital libraries and, through mechanisms completely analogous to the traditional function of libraries, made available to a range of users. Taking inspiration from pioneering work in enterprise knowledge management originally performed within Partners Healthcare,Most important, the modularization of knowledge into DKOs makes possible the integration of locally produced LHS research results and externally published evidence, as envisioned in the previous section. In most straightforward scenario, a health system can generate advice from LHS evidence and published evidence side-by-side, and determine if the advice generated from the two sources conflict or coincide. In more sophisticated scenarios, health systems can create compound DKOs each of which synthesizes the evidence from multiple models. The ability to amass evidence more broadly is important in having sufficient power to support evidence needed to ensure safe and effective care delivery.To complement the efforts self-organizing, private collaboratives like HVHC, incentivizing a computable knowledge ecosystem will likely require government action such as participation requirements for federally funded projects, organizational priorities for large-scale learning, and training to facilitate clinical use of the most up-to-date indications gleaned for evidence-based medicine. As this ecosystem evolves, the most immediate implication for authors contributing to journals like JGIM will be the extension of the publication pipeline. For papers that result in statistical models, rules, recommendations, or guidelines, publication in human readable form will be followed by representation of this same evidence in a computable form. The details of this approach will evolve over time. As it does, research authors and clinician users of that published research will have important roles to play in designing the pipeline that makes computable knowledge accessible at scale.We stand on the brink of a transformation in health evidence generation and application, catalyzed by the growth of LHS practices, technological advances, shifts in culture, revised work processes, and a burning “need to know” now. Achieving the greatest benefit from this change begins with recognition that new evidence is necessary but not sufficient for better health and that moving evidence into practice, at present and in the future, will require an ecosystem supporting mass action at an accelerated pace. We must continually mind the knowledge-to-practice gap and take positive steps to bridge it.


In a Policy Forum, Muin Khoury and colleagues discuss research on the clinical application of genome sequencing data. There is currently insufficient scientific evidence to support routine nondiagnostic use of germline genome sequencing in healthcare settings and population screening, but an increasing number of health systems are piloting genomic sequencing projects for clinical care.In principle, numerous diagnostic or prognostic tests based on genes or variants could be used for different purposes across the life span, and an evidence-based approach is urgently needed to evaluate their possible clinical utility and facilitate appropriate implementation.We discuss a translational research framework that features collaboration among multiple health systems with already available genome sequencing data, intervention information, and clinical outcomes. The framework is based on evaluating the impact of genetic information on improving health outcomes with study designs that depend on the evolving level of evidence for specific intended clinical uses.In addition to observational studies, randomized controlled trials will be needed to assess health benefits, harms, and costs based on returning or not returning the results of selected genes/variants to patients, providers, or both, for specific clinical scenarios.The proposed approach will allow learning health systems to collect clinical utility evidence in a research environment and develop the necessary capacity for appropriate integration of sequencing alongside other medical services.A vision for genomic medicine is that germline genome sequencing will be routinely conducted in health systems to provide healthcare and preventive services tailored to each individual . For theCurrently, there is limited direct evidence of clinical utility of germline genome sequencing to guide health service delivery and disease prevention in the population . While wYet, several health systems in the United States and other countries  are begiTo address this challenge, some have advocated to focus sequencing efforts only on selected genes with evidence of clinical utility to guide practice . Many geThe early adoption of germline genome sequencing in practice may be viewed as an uncontrolled experiment, which could lead to both benefits and harms. However, it also provides a unique opportunity to develop an evidence-based process to accelerate evaluation and appropriate implementation. A fundamental question is whether or not we can use a genome sequencing platform embedded into learning health systems that can accelerate simultaneous evaluation of multiple testing scenarios with differing levels of evidence.The topic of integrating genome sequencing in large health systems was discussed at a National Academy of Medicine workshop  that feaAll of Us Research Program [However, most work in this area is fragmented, based on small sample sizes, conducted in one health system at a time, or completely outside the healthcare systems by commercial and/or direct-to-consumer entities. The 100,000 Genomes Project in the United Kingdom  and the  Program  , risk factor information, intervention information, and clinical outcomes. The proposed framework calls for collaboration among several organizations, to recruit adequate numbers of individuals. This is crucial in order to understand the consequences of knowledge of multiple risk variants for multiple clinical scenarios. In addition to observational studies, randomized controlled trials (RCTs) can be designed to assess individual, family, system, and population outcomes based on returning versus not returning the results of selected genes/variants for specific clinical scenarios, depending on the existing level of evidence. This implies that the return of the results of genome sequencing to patients and providers will occur in a research controlled fashion, specified based on pre-agreed study protocols. The proposed framework will allow learning health systems to evolve the necessary capacity for appropriate integration of genomics alongside other medical services, such as screening and treatment, as well as to prepare the healthcare workforce and the public.The framework has five components, with a three-prong research agenda  and two 1Ongoing knowledge integration of the human genomeAn ongoing knowledge integration process will be needed to drive genomic medicine translation research. As proposed by Berg and colleagues , knowledAn example of an ongoing effort in evaluating clinical validity is the Clinical Genome Resource (ClinGen) . ClinGenTier 1 genes and variants that have synthesized evidence with an evidence-based guideline that supports implementation in clinical practice. These genes are ready for an accelerated implementation science agenda (see below).Tier 2 genes and variants that have strong clinical validity information  but limited or no evidence for clinical utility  to support implementation in practice. These genes/variants should be targeted for accelerated test/drug development and research evaluation of clinical utility. In addition, hybrid effectiveness/implementation studies could be designed to assess how these genes and variants may be incorporated into recommendations for clinical practice.Tier 3 genes and variants include the majority of genes and variants with insufficient evidence on clinical validity or utility. These genes and variants can be targeted for accelerated evaluation of both clinical validity and utility. Tier 3 also contains variants that have guidelines against their use. These would not be addressed further and could, as evaluations mature, allow ineffective technologies to be removed from use (de-implemented).2Accelerated implementation science agenda for tier 1 genes/variantsImplementation of evidence-based tier 1 genes/variants will require enhanced services associated with the management of sequencing results . Implementation science is a well-established field that is increasingly applied to genomic medicine research , which eImplementation science is typically used when high levels of evidence of benefit from the use of a new technology exist. As an example, we discuss the 10 genes relevant to three autosomal dominant high-penetrance conditions that seem immediately appropriate for implementation science research: hereditary breast and ovarian cancer risk associated with BRCA 1 and BRCA2 mutations, Lynch syndrome  associated with mutations in mismatch repair genes, and familial hypercholesterolemia (FH) associated with dysfunctional cholesterol processing. Each of these inherited conditions has evidence-based recommendations that are currently underutilized in most healthcare settings. The adoption of these recommendations will reduce the burden of heart disease or cancer . In aggrBRCA1, BRCA2, and other genes.For example, current USPSTF BRCA guidelines recommend counseling and testing based on high-risk family history and ethnicity . StudyinFor FH, cascade screening can prevent early heart disease in affected relatives . Cascade3Accelerated clinical utility research for tier 2 genes and variantsA collaborative translational research agenda can also accelerate the evaluation of and implementation studies for genetic variants that have robust and validated associations and have the potential for clinical utility. This category includes many monogenic risk variants as well as selected pharmacogenetic traits that are prevalent in the population, with potential relevance to clinical management of several commonly used drugs . Collaborative groups such as the Clinical Pharmacogenetics Implementation Consortium (CPIC) and the Pharmacogenomics Knowledgebase (PharmGKB) evaluate emerging evidence of pharmacogenomics and publish recommendations to inform clinical practice . Such efAdditionally, when healthy people undergo genome sequencing, results of pharmacogenomic tier 2 genes/variants will be available to be evaluated for clinical utility if patients need certain medications. The value of preemptive pharmacogenomic testing has been suggested but not validated . Many ph4Accelerated evaluation of validity and utility of tier 3 genes and variantsMost human genes and their variants currently reside in a tier 3 bin, for which additional research will be needed to establish both clinical validity and utility. Collaborative multi-institution genome-wide association studies (GWAS) have provided a firm foundation for establishing gene–disease associations for numerous diseases . So far,A collaborative effort across multiple institutions that are already generating data on low effect size variations in large numbers of individuals will offer immediate opportunities to accelerate joint analyses to evaluate the validity and utility for selected tier 3 genes/variants included in genetic risk scores. One example is the use of genetic risk scores in practice . GeneticFor example, in the context of heart disease, Knowles and Ashley recently discussed the emerging promise of genetic risk scores for a field in which the use of risk factors for clinical decision-making has had a long history in medicine . They di5Enhanced development of workforce, tools, and resourcesAn accelerated translational research agenda for evaluating genome sequencing in healthcare and population screening will require several “drivers” . For thiWe have described essential elements of a collaborative translational research agenda to evaluate the potential use of germline genome sequencing in primary care and population screening. There is some urgency to developing this agenda, as genome sequencing is becoming more reliable, less expensive, and widely used inside and outside healthcare systems, without a thorough investigation of its clinical utility.The emphasis on binning genes/variants is based on independent evaluation of evidence for clinical utility and may not readily take into account multiple domains of genetic information over a lifetime of healthcare decisions. Also, aggregating genome data across populations and over time creates a challenge for evidence acquisition that requires an ongoing collaboration. In addition, there will be a need for additional research studies for other domains of genomic information, such as studying healthy newborns and children, and for carrier testing and reproductive purposes. Ongoing studies such as the MedSeq project  are provThe proposed approach is focused on healthcare and population screening settings that require integration of genome sequencing with health services. As such, it requires a high evidence bar. Clearly, there is growing interest in genetics by the public, as exemplified by the growth of direct-to-consumer genetic testing. The popular interest in genomic information should be factored into the evaluation of impact and outcomes of various studies suggested as part of this framework.A major challenge in implementing a research framework such as this is to motivate participation from various stakeholders. Also, clear lines of demarcation will have to be drawn between the “research” and “clinical practice” arms of such a collaboration ,39. For A robust large-scale translational collaborative effort is now needed to understand the health benefits and potential harms and costs of genome sequencing, by studying the implementation of what we know can work, evaluating the possible utility of promising genes and variants, and critically assessing the validity of emerging genomic information for improving health and preventing disease. Although details on how to accomplish this task are beyond the scope of this piece, we plan to assemble a transdisciplinary, multi-sectoral group of experts to identify compelling questions, examine the available data, explore critical challenges and opportunities, and develop specific recommendations to implement a collaborative research framework for accelerating the evaluation of human genome sequencing in clinical practice.


In 2016, we wrote an article discussing the relationship between evidenced-based medicine (EBM) and evidence-based policy, informed by the particular evidence supposed to constitute the basis for decision-making , FretheiWe identified and organised the official publications from NOKC to an empirical corpus of typical ‘top-of-the-line’ evidence represented by SRs adhering to EBM standards. From a total of 151 SRs published by the NOKC from 2004 and 2013, a purposive subsample  was examined, addressing their potential as policy decision tools.Supported by theoretical perspectives from rhetoric of health and medicine, we assessed and interpreted the persuasive power of the conclusions mediated by the concepts used, taking terms indicating positions of certainty or reluctance as our point of departure . AnalogoIn 2012, 57,368 studies were screened, identifying 351 that were included for synthesis in 14 SRs. For each SR, the average number of hits was 4098  and on average 25 (range 3–91) studies were included. Caution in various forms was advocated as the major rhetorical pattern in the SR conclusions. Some of the SRs stated that no certain conclusions could be drawn, while others used very cautious terminology in their conclusions, such as ‘probably increases’, ‘possibly increases’, ‘increases perhaps’, ‘may reduce’, ‘uncertain’, or ‘difficult to conclude’. These reservations were apparently reflecting the GRADE rating system […we assessed the persuasive power of the conclusions mediated by the language used, especially with regard to terms indicating positions of certainty or reluctance” [Rhetorical analysis is an interpretative methodology exploring interaction between arguments, actors and policy. Assessment of how a statement may function as an argument can be conducted in different ways, depending on purpose and context. We read Fretheim’s comments as a fundamental disagreement upon paradigms. Highly skilled in statistical meta-analysis, Fretheim dismisses the assumption of subjective judgements and theoretical reflection as essential elements of interpretative research methods. Assessing validity by trying to replicate analysis and expecting identical findings makes no sense, since there is no single correct answer to the questions asked in this kind of studies. This does not mean that the results are casual outcomes of cherry picking, but rather that different preconceptions, theoretical frameworks, background, positioning and methodological approaches have an impact on the interpretations and explain different findings. Referring to Segal , we decluctance” .In Fretheim’s judgement, 4 of the 14 reports included findings without major reservations about uncertainty. Our interpretation – with a different aim, a different method for analysis and a different philosophical foundation – implied that the conclusions of 12 of the reports were characterised by major caution regarding their potential as policy decision tools. We do not argue that Fretheim’s analysis is wrong. In fact, Fretheim’s judgement supports our suggestion that most of the reports advised major caution. The minor discrepancies between our assessments of the reports are easily explained by different interpretation strategies. Given Fretheim’s methodological position, it is no surprise that his understanding of the conceptual validity of the cautious terminology is dissimilar from ours. His institution has been pivotal in the development of the GRADE system, which offers a tool for systematic assessment of the evidence quality and strength of recommendations . Fretheiexcluded” [Although we argued in the article that the SR methodology is better suited for synthesis of medication efficacy studies than for complex public health interventions, we did not assume or claim that any SR about medication efficacy studies would serve as an adequate policy decision tool. Discussing the potential of SRs concerning medication efficacy studies, we believe that Fretheim’s expression “xcluded”  refers tWriting this article, we intended to offer critical reflection upon the tools used for development of evidence. Although our analysis has demonstrated some limitations for SRs, we do not argue that they have no place in policy-making processes in general. However, several of the SRs in our sample dealt with the synthesis of randomised controlled trials from complex and contextually dependent interventions, which are neither easily conducted nor standardised , 7. We aIt is possible to dismiss the contemporary wave of science scepticism without believing that research knowledge is a universal resolution to any problem. The ‘knowledge translation’ metaphor represents the ‘know-do’ gap to be bridged between scientific facts and policy-making as a simple pipeline model , 9, ofte