In omic research, such as genome wide association studies, researchers seek to repeat their results in other datasets to reduce false positive findings and thus provide evidence for the existence of true associations. Unfortunately this standard validation approach cannot completely eliminate false positive conclusions, and it can also mask many true associations that might otherwise advance our understanding of pathology. These issues beg the question: How can we increase the amount of knowledge gained from high throughput genetic data? To address this challenge, we present an approach that complements standard statistical validation methods by drawing attention to both potential false negative and false positive conclusions, as well as providing broad information for directing future research. The Diverse Convergent Evidence approach (DiCE) we propose integrates information from multiple sources  to estimate the strength of the available corroborating evidence supporting a given association. This process is designed to yield an evidence metric that has utility when etiologic heterogeneity, variable risk factor frequencies, and a variety of observational data imperfections might lead to false conclusions. We provide proof of principle examples in which DiCE identified strong evidence for associations that have established biological importance, when standard validation methods alone did not provide support. If used as an adjunct to standard validation methods this approach can leverage multiple distinct data types to improve genetic risk factor discovery/validation, promote effective science communication, and guide future research directions. Since other associations in the set of comparisons may have no bearing on the one in question, the upshot is that irrelevant information from the data can diminish the informativeness of an association of possible interest.”It should also be noted here that the rationale behind multiple testing adjustments and their use in certain contexts has been exposed to important criticisms in the last 25 years . In 1990“ . . . in exploratory studies without prespecified hypotheses there is typically no clear structure in the multiple tests, so an appropriate multiple test adjustment is difficult or even impossible. Hence we prefer that data of exploratory studies are analyzed without multiplicity adjustment . . . To confirm these results, the corresponding hypotheses have to be tested in confirmatory studies.”With the advent of omic research designs, and the development of new options for multiple testing adjustment, Rothmans’s analysis has become even more important, and a number of authors have extended his comments, including Bender and Lange : “ . . .“We argue that when examining an array of nominally positive findings, statistical stringency alone does not permit us to determine which findings are by chance and which are not, and therefore, setting too stringent cutoff for Type I error criterion for association decreases power to find real associations.”Recently, Williams and Haines revisited and extended these lines of thought . They emDiCE further extends these ideas by emphasizing that the processes of validation should be dependent on diverse evidence, because this better addresses the type I/type II error problem, as well as other recognized and unrecognized weaknesses of omic analyses. With this method we have not abandoned our concern for type I error. We still value the evidence that comes from small p-values and rely on independent confirmation, but we now deliberately acknowledge the importance of type II errors and proactively attempt to reduce them.To the extent that our approach streamlines the consideration of diverse convergent evidence, it can speed up the progression from omic findings to interventions.DiCE is a semiformal, dynamic heuristic that reflects the strength of available diverse convergent evidence, and it is designed to supplement standard statistical validation procedures. This makes it different from most statistical analytic approaches, but there are some useful comparisons to be made with other methods. In particular, discussing Meta-analysis, Inter-Rater Reliability, and Inference Ranking techniques should help to contextualize the role of DiCE in high throughput genetic research.In genetic research meta-analysis typically utilizes fixed effect models to integrate information from multiple observational studies to estimate a single association magnitude  for a given SNP . Thus, iThe fixed effect meta-analyses typically used in discovery genomics settings obtain a single omnibus effect estimate by assuming the heterogeneity of effect size is due to random error . HoweverConceptually the DiCE score is similar to an ordinal inter-rater reliability (IRR) metric  in a setIn this paper we present a new method (DiCE) for improving the detection and validation of relevant biological signals in omic data by proactively considering diverse evidence. This approach provides a chance to strengthen our validation strategies and advance the progress of research. We argue that DiCE, when properly implemented, should leverage multidisciplinary information to reduce rates of both false positive and false negative conclusions. Standard validation protocols implicitly assume that there is one truth  and it will be discoverable no matter what the contextual background . Furthermore, these validation procedures, when used in isolation, can lead to incorrect conclusions when there is a consistent bias in the observational studies. Therefore, many causal factors will go unnoticed and some meaningless “hits” may be over-interpreted without the development of additional validation approaches, such as DiCE.The utility of gathering diverse classes of evidence in the context of complex disease research is not a new idea , but in An attempt to assess the consistency of association by trying to repeat the results in an independent sample from the original population with the same analytic approach .An attempt to assess the consistency and generalizability of association by trying to repeat the results in an independent sample from a different population using either the same analytic method or a different approach .A component cause or causal cofactor . Component causes are factors that are insufficient to cause disease by themselves but can help cause disease when they occur with other component causes. For more details see .A variable that may impact the estimated association between the variable of primary interest and the outcome  A covariate may have this impact through causal or non-causal  relationships. If not properly considered in the analysis covariates may generate bias in the estimated association between the variable of primary interest and the outcome. Cofactors are covariates that may influence estimated associations through causal mechanisms.GWAS: Genome-wide association studies.The authors declare that they have no competing interest.This paper evolved out of discussions at the EDGE 2013 conference organized by MDR and JHM. TC drafted the manuscript and proposed the scoring methodology which developed as a product of exchanges with the co-authors. SP and MW drafted the figures. NK and SMW provided editorial and intellectual content review. All co-authors provided important input or feedback. All authors read and approved the final manuscript.Implementation examples for the Diverse Convergent Evidence (DiCE) Scoring System.Click here for file


Policy and decision making should be based on evidence, but translating evidence into policy and practice is often sporadic and slow. It is recognised that the relationship between research and policy uptake is complex and that dissemination of research findings is necessary, but insufficient, for policy uptake. Political, social, and economic context, use of (credible) data and dialogues between and across networks of researchers and policymakers play important roles in evidence uptake. Advocacy is the process of mobilising political and public opinions to achieve specific aims and its role is crucial in mobilising key actors to push for policy uptake. Advocacy and research groups (i.e. those who would like to see research evidence used by policymakers) may use different approaches and tools to stimulate the diffusion of research findings. The use of mass- and social media, communication with study participants, and the involvement of stakeholders at the early stages of research development are examples of the approaches that can be employed to stimulate diffusion of evidence and increase evidence uptake. The Research and Advocacy Fund (RAF) for Maternal and Newborn Health (MNH) worked within the health system context in Pakistan with the aim of espousing the principles of evidence, advocacy, and dissemination to improve MNH outcomes. The articles included in this special issue are outputs of RAF and highlight where RAF’s approaches contributed to MNH policy reforms. The papers discuss critical health system issues facing Pakistan, including service delivery components, demand creation, equitable access, transportation interventions for improved referrals, availability of medicines and equipment, and health workforce needs. In addition to these tangible elements, the health system ‘software’, i.e. the power and the political and social contexts, is also represented in the collection. These articles highlight three considerations for the future: the growing importance of implementation research, the crucial need for participation and ownership, and the recognition that policymaking can be ‘informed’ by rather than ‘based-on’ evidence. The future challenge will be to continue the momentum RAF has created and to welcome a new era of health, wealth, and growth for Pakistan. Policy- and decision-making should be based on evidence , yet, trstakeholders are aware of and use research evidence to inform their health and healthcare decision-making”. Contributions from social or political science fields suggest that three factors play important roles in influencing policy uptake: (1) political, social, and economic contexts; (2) utilisation of (credible) data; and (3) researcher and policymaker networks [The relationship between research and policy uptake is complex; few people would now defend the simplistic model of linear causality between the two and realisation of its complexity is growing. According to Grimshaw et al. , translanetworks . A multiAdvocacy is the process of mobilising political and public opinions to achieve specific aims. Its role is crucial in mobilising key actors to push for evidence uptake and greater accountability. Advocacy groups or organisations include wide ranging groups of people with differing motives and purposes. For example, patient support groups may be instrumental in highlighting the gaps in treatment, care, and support to patients  which reDissemination and diffusion of research findings is necessary for the transfer of evidence to practice and to influence policymaking . Yet, diguide actions and underpin the relationships amongst system actors and elements are also critical to overall health system performance”.Once called a neglected tragedy as epitomised by Rosenfield and Maine’s seminal article , maternaThe publication of this special issue has two main aims: first, to highlight where our approaches have been able to contribute to policy-uptake in Pakistan, and second, to further contribute to the dissemination of evidence from Pakistan, stimulating MNH policy and practice reforms. The nine research papers and the third commentary included in this issue stem from projects funded by the Maternal and Newborn Health Programme Research and Advocacy Fund (RAF), a programme implemented within the health system context in Pakistan and which aimed to espouse the principles of evidence, advocacy, and dissemination described above.RAF was set up as a grant fund in 2008 to work with researchers and civil society organisations to influence pro-poor policy and practice reform on MNH in Pakistan. In addition to providing grants, RAF provided capacity building support to researchers and civil society organisations, for example, technical advice during the proposal development phase on research methods, advocacy strategies, and gender and inclusion, and communication approaches. The programme also held consultations and reviewed gaps in evidence to identify topics for advocacy. Overall, 56 research and advocacy projects were funded, which provided a range of evidence on critical MNH issues, influencing over 30 different policy and practice issues. In addition to policy reforms documented in the special issue, we witnessed RAF-funded projects contributing to changes in health worker curriculum, legislation on breastfeeding and nutrition being passed in Sindh, Punjab, Balochistan, Khyber Pakhtunkhwa, and Azad Jammu and Kashmir, and increased government commitments to family planning targets in line with Family Planning 2020.In the articles included in the special issue, critical health system issues compromising MNH outcomes in Pakistan are discussed, for example, service delivery components, demand creation, equitable access, transportation interventions for improved referrals, availability of medicines and equipment, and health workforce needs. In addition to these tangible elements, the ‘software’ of the health system, i.e. the power and the political and social contexts, is also represented in the collection.The first two commentaries from Ghaffar et al.  and BhutPakistan’s Community Midwives (CMW) programme has introduced a new cadre of skilled birth attendants in the community. Similar to Indonesia’s Village Midwives, who had the right to establish private practices, Pakistan’s CMWs were expected to establish private practices after 2 years of government financial support. Many Indonesian village midwives abandoned public service in favour of the more lucrative private practice and the maternal mortality ratio stagnated; three papers examine this high profile initiative. Mumtaz et al.  used a pContracting out is another method of quickly expanding service availability. Zaidi et al.  used a mInequity in access and the barriers to Emergency Obstetric and Newborn Care services in Sindh province are discussed by Ansari et al. . In addiThe issue of caste-based discrimination and social exclusion is further discussed by Aziz et al. , who criPivotal to the functioning of a good maternity healthcare system is a referral system. Mian et al.  reviewedHuman resources, another important component of a health system, are examined by Mir et al.  for thisWe have included a short article by Majrooh et al.  to demonWe began this article by considering the essential and interlacing influences of research, dissemination, and advocacy for policy uptake. Understanding of how these groups of activities and actors can and should work together has grown considerably in the last decade , 36. SimThe articles in this supplement highlight three considerations for the future: the growing importance of implementation research, the crucial need for participation and ownership, and the recognition that policymaking can be ‘informed’ by rather than ‘based-on’ evidence.powerful tool for capturing and analysing information in real time, allowing for the assessment of performance, for example, and facilitating health systems strengthening” . hening” . In terms of methodologies, implementation research is a relatively new field. There is no consensus on optimal methodology for implementation research and interdisciplinary/multidisciplinary approaches are needed, drawing on quantitative, qualitative, and mixed methods. Discipline-specific questions (e.g. ) are alsIn resource-constrained settings, there is a special need to prioritise initiatives for research and advocacy on contextually-relevant priorities. To do so, wide and inclusive participation of stakeholders is necessary , 47. ParWe also acknowledge the limitations and challenges associated with stakeholder participation and ownership. When engaging with stakeholders, not everyone’s view can be represented, as ‘stakeholders’ are non-homogenous groups of people with differing motives and purposes. Participation is a highly political process where existing power relations may be reinforced . Though Our paper began with the idea of evidence-based policymaking. However, it is well recognised that policy is influenced by more than just evidence, with politics, lobby groups, media, and chance playing a part ,50. The There will be no quick fixes in tackling such complex issues in the future of Pakistan. The building of robust healthcare systems is an area where many low- and middle-income countries have been underperforming . These s


Knowledge translation and evidence-based practice have relied on research derived from clinical trials, which are considered to be methodologically rigorous. The result is practice recommendations based on a narrow view of evidence. We discuss how, within a practice environment, in fact individuals adopt and apply new evidence derived from multiple sources through ongoing, iterative learning cycles.The discussion is presented in four sections. After elaborating on the multiple forms of evidence used in practice, in section 2 we argue that the practitioner derives contextualized knowledge through reflective practice. Then, in section 3, the focus shifts from the individual to the team with consideration of social learning and theories of practice. In section 4 we discuss the implications of integrative and negotiated knowledge exchange and generation within the practice environment. Namely, how can we promote the use of research within a team-based, contextualized knowledge environment? We suggest support for: 1) collaborative learning environments for active learning and reflection, 2) engaged scholarship approaches so that practice can inform research in a collaborative manner and 3) leveraging authoritative opinion leaders for their clinical expertise during the shared negotiation of knowledge and research. Our approach also points to implications for studying evidence-informed practice: the identification of practice change (as an outcome) ought to be supplemented with understandings of how and when social negotiation processes occur to achieve integrated knowledge.This article discusses practice knowledge as dependent on the practice context and on social learning processes, and suggests how research knowledge uptake might be supported from this vantage point. Over the years, the Evidence-Based Practice (EBP) movement has gained widespread acceptance attributable, in part, to the increasing emphasis on the need for public accountability and transparency in decision-making at the patient, organization and policy levels within the healthcare sector , 2. By sa dynamic and iterative process that includes synthesis, dissemination, exchange and ethically-sound application of knowledge to improve the health of Canadians, provide more effective health services and products and strengthen the health care system’ . To determine which knowledge should be ‘translated’ or promoted for application in practice, KT, like EBP, has relied on the use of the traditional evidence hierarchy in which specific forms of scientific endeavour are assigned greater value on the basis of methodological rigour, while other forms of inquiry and other sources of knowledge or ways of knowing are discounted or rejected [Knowledge translation (KT) seeks to narrow the perceived gap between knowledge and practice , 7. KT hrejected , 11. As rejected , 8, 12.If one rejects the assumptions associated with the traditional, linear, EBP-linked view of knowledge translation, is it possible to describe a process of social learning in which knowledge is both emergent, and pluralistic, and in which practice and knowledge are not separated? The purpose of this paper is to discuss how individuals can adopt and apply research-based evidence in practice through ongoing, iterative cycles of learning that include processes of reflection and collective negotiation of shared practices. If the ongoing, learning process through which research-based evidence is selected and integrated by practitioners is better understood, then it may be possible to capitalize on this process by identifying potential points of intervention through which improved research uptake can be facilitated in the future.We begin, in Section 1.0, by suggesting that, despite the emphasis on a narrowly-defined sample of research-based evidence selected for ‘translation’ into practice settings, multiple forms of evidence are applied at the point of care delivery. Reflective practice, discussed below in section 2.0, is contrasted with evidence-based practice to describe a process that produces contextualized knowledge drawing on a variety of knowledge sources for practice decisions. In section 3.0, the idea of team-based contextualized knowledge developing from individual theories of practice is highlighted. Examples of how social learning communities can negotiate across knowledge sources to derive a communal understanding of practice norms are presented. Implications for practice, related to measuring KT efforts and possible points of intervention to support the uptake of research findings within social learning communities, are discussed in section 4.0.According to Sackett, the practice of evidence-based medicine represents the conscientious and explicit integration between the best available evidence derived from systematic research with clinical expertise in decision-making . The proAdherence to traditional hierarchy or levels of evidence mechanisms discourages the use of research from a variety of systematic modes of inquiry designated as weak or considered insufficiently empiric . EmpiricResearch findings are defined, identified, selected, distilled, discussed, and shaped into guidelines that are provided to healthcare practitioners to provide them with direction regarding what to do to achieve a practice standard based on the best available evidence. The use of a hierarchy or levels-of-evidence-based selection process for identifying which research findings are worthy of this type of linear translation results in the selection of a decontextualized and internally valid evidence base that is less well-suited to real world applications than to controlled, experimental environments , 18, 19.The notion of evidence as an object to be transferred or translated from researcher to practitioners , 21, 22 Practice environments themselves are complex and uncertain, and bear little resemblance to the strictly controlled experimental environment . ContextIt is generally acknowledged that a variety of knowledge sources inform practice and service delivery. In the context of the real-world practice environment, evidence can be defined, in its broadest terms, as that which informs effective judgement or decision-making . In the According to Lam , human kIf knowledge-in-practice is emergent, synthesized from different sources within social contexts, the production of more and better clinical guidelines or best practice recommendations is unlikely to narrow the perceived gap between research and practice in a significant way. Research findings, such as those selected for translation via products such as clinical guidelines, represent only one among multiple potential sources of knowledge used in the negotiation of a social and professional understanding of evidence that can be applied in practice , 8, 34. acontextual, generalized, unbiased and predictive type of knowledge” (p 132), reflective practice is perceived as being associated with knowledge that is subjective and bound by context [There has been an underlying, and perhaps persistent, assumption that evidence-based practice and reflective practice represent two distinct and incompatible frameworks . While E context . Reflect context . However context  and refl context .instrumental problem-solving made rigorous through application of scientific theory and technique” (p. 21) [In 1983, Schön suggested that the dominant epistemology of practice, which he called ‘technical rationality’, was one in which practice consisted of “ (p. 21) . Profess (p. 21) ). OveralLike the creation of a ‘best evidence’ knowledge product or guideline that relies on traditional evidence hierarchies and is separate from, but used to inform action in context, ‘technical rationality’ is an insufficient model to describe clinical practice or practice change , 36, 37.Practices have been defined as combinations of 1) bodily and mental activities, 2) artefacts and their use and 3) a background, or tacit knowledge which both organizes and gives meaning to the practice . PracticIn the course of daily practice, individual healthcare practitioners develop a repertoire of techniques, expectations, images, perceptions, and so on. Over time, their personal repertoire becomes increasingly spontaneous, automatic and tacit . This exKnowledge creation or learning within a practice environment is not strictly an individual process. Learning is social, and is as much about social culture, context and lived experience as it is about the acquisition of specific facts or technical skills , 40, 43.Externalized tacit knowledge, such as might be surfaced from a process of critical reflection, becomes more mobile and can be shared more easily with others. Individual practitioners might seek out other members of their practice community with whom they can exchange experience-based information as they evaluate and re-evaluate their own evolving theories of practice , 22, 43.Processes of comparison and evaluation are not limited to other team members or professional colleagues who have common contextual understandings. Practitioners also examine their know-how against other available sources of knowledge including formal, research-based evidence , 16. HowThe process of evaluating and reviewing existing personal practice theory in light of knowledge from other sources and then reconciling and merging new information with it has been described as one of combination . The comclinical mindlines”, which are internalized, collectively-reinforced, often tacit, and informed by multiple knowledge sources including the practice-based experience of other healthcare practitioners in addition to current clinical practice guidelines, recommendations or practice policy [Gabbay & LeMay theorised that, similar to personal theories of practice, practitioners develop “e policy . Accordie policy . Mindline policy . In addie policy .Individual mindlines share a reciprocal relationship with collective mindlines; each contributes to the development of the other . In theiA community of practice is defined to some extent by engagement in the processes of social practice or creating collectively-negotiated, shared repertoires and shared purpose. Engagement of sufficient density and duration fosters both a sense of community, identity and of belonging . Social If knowledge ‘translation’ is conceived of as a naturally-occurring, ongoing and iterative process of critical reflection and social learning in which practitioners, both individually and collectively, check, re-check and re-negotiate both personal and shared theories of practice, then the potential to support opportunities for research uptake in concert with this process should be identified.Learning is not constrained to a structured classroom, but is a fundamentally social phenomenon occurring outside of situations one might identify as a formal educational opportunity , 43, 46.In learning supportive environments, ample opportunity should be provided to facilitate participation in both formal and informal activities associated with learning, including the negotiation of shared repertoire through engagement in discussion, debate or other forms of storytelling, as well as individual engagement in active learning in practice , 46, 47.collaborative learning environments” [Active learning is a process of engagement with experience via critical reflection, learning from practice, and evaluation . Activitonments” . A focusonments” . The adoonments” .In healthcare settings, joint or team working can be about creating new ways of thinking and communicating across professional boundaries that includes negotiating shared knowledge and ways of doing things that are accepted in context , 57. Of Facilitating ‘evidence-informed practice’ is often based on a techno-rational, problem-solving approach rather than a social, collaborative and active learning approach. Didactic programs offer training to promote evidence-based skills such as searching the literature or evaluating research. While these skills are not unimportant, the success of active learning approaches relies on the availability of leaders or facilitators with the appropriate skills and abilities to lead and sustain active learning and practice development based on collaborative or social learning perspectives. Key facilitators/leaders should be identified and appropriate training and leadership development opportunities should be provided to enable leaders to develop skills in supporting active learning, critical reflection on their own leadership style and promoting sustainability of a learning culture , 62. SimIn their recent scoping review, Thomas & Law describe several characteristics of collaborative learning environments that support the uptake of research-based evidence and support evidence-based practices . In addiClinical or professional experience, including tacit know-how, is a highly valued, and frequently consulted source of knowledge , 67–69. Within practice environments, trusted authorities or opinion leaders valued because of their clinical experience influence whether (and to what extent) research-based knowledge or knowledge products are used to inform practice . If respRevised conceptualizations of the ways in which evidence is integrated into practice requires revisions to the ways in which evidence-informed practice is studied and evaluated. In an evidence-informed practice, like the one described by Gabbay and LeMay , knowledIt has been suggested that ‘reflective capacity’ is an essential characteristic for professional competency insomuch as the ability to engage in reflective practice provides the practitioner with the means to engage with new evidence and knowledge to improve practice , 71–73. Traditional models of knowledge translation have relied heavily on a linear, techno-rational based model in which selections of research-based information are used to inform practice guidelines that are disseminated for application. It is suggested that knowledge used in practice is collaboratively constructed, drawing upon information from a variety of sources -- not just a selection of research-based evidence, as informed by use of a traditional hierarchy of evidence . This co


Demographers warn us of the “gray tsunami” approaching our global doorstep . ResearcThis Research Topic on evidence-based programing for older adults reflects decades of progress by researchers, practitioners, aging service providers, and policy makers working together to understand how to help older adults achieve optimal health and well-being. Such efforts have transformed successful aging from a theoretical concept into an achievable goal , 8.The scientific roots of this Research Topic are many, but our (Ory and Smith) personal interest began with the evaluation of the Administration on Aging (AoA)’s national disease prevention initiatives introduced in the 2000s, which will be described in length later in this volume –11. WithAs we explored these questions, we realized the need to look beyond single silos or perspectives to understand how researchers, program developers, and policy makers could work together more closely. Such collaborations are essential to develop, promote, and support evidence-based programing that reflects stakeholders’ perspectives and increases the likelihood of being embedded into existing structures. Ideally, evidence-based programs reflect a translation of testable research theories into key intervention elements that resonate with program adopters and intended participants. However, it is critical that interventions are seen as desirable and feasible for both organizations and intended audiences if they are to be adopted. Thus, a dynamic interaction between research and practice is desirable to ensure the appropriateness of program content and delivery, especially as they are disseminated and evaluated in different populations and settings. Similarly, it is important to examine the role the policy context plays in sustainable program success. For example, healthcare policies are theoretically designed to meet national health care goals. Researchers and practitioners can help document the benefits and consequences of current policies facilitating or impeding the growth and sustainability of evidence-based programming. Research about program effectiveness can inform new policy directions, and practitioners can provide real-world views about the practicality of different service and programing options.In formulating this Research Topic, our collective objective was to identify the most effective programs and to understand individual, social, community, and environmental factors that influence program reach, adoption, implementation, dissemination, and sustainability. This perspective aligns with many emergent themes and frameworks in evidence-based public health and medicine such as the RE-AIM planning and evaluation framework , 15, theTraditional stereotypes of aging viewed older adults as inappropriate targets for community-based health promotion programs because they were believed to be uninterested in such programs and/or unable to benefit from such preventive efforts . HoweverIn the past, there were few researchers involved in developing evidence-based programs for older adults, few community programs adopting these programs, few practitioners delivering these programs, and even fewer policy makers focused on strategies for guaranteeing sustainable funding streams. An initial step in promoting evidence-based programming was informing the aging services provider network about the definition of evidence-based health promotion and disease prevention and its value for practitioners and policy makers.Toward this end, the National Council on Aging (NCOA)  served aThe articles in this Research Topics provide an excellent overview of the evolution from past to present activities, especially related to the dissemination and testing of evidence-based chronic disease self-management programs, physical activity programs, fall prevention programs, and to a lesser extent, behavioral health programs. While great strides have been made over the past three decades, there is still considerable room for improvement related to program delivery, dissemination and sustainability.Authors of this volume were asked to reflect about future implications for research, practice, or policy. Solid groundwork has been laid, suggesting that the evidence-based movement has the foundation for even greater dissemination among an aging population. Our early work focused on the first 100,000 participants in the suite of programs referred to generically as Chronic Disease Self-Management Education (CDSME) programs. Recent statistics indicate the rapid proliferation of programs with over 300,000 persons engaged in evidence-based programs delivered through the aging services network since 2010, including more than 230,000 with CDSME alone .Policy changes, such as the new mandate from ACL limiting Title IIID reimbursement to evidence-based programs, will serve to increase the number of evidence-based programs disseminated to older adults through the aging services network . AdditioWhile the ACL , in partThis section focuses on the evolution of the evidence-based movement from the perspective of the program developers themselves Included is information regarding the processes involved in developing and taking some of the major evidence-based programs for older adults to scale, including: (1) the Stanford suite of CDSME programs ; (2) a MThe program developers generously share the lessons they learned including the importance of: (1) building programs with the end user in mind; (2) defining roles and responsibilities of partners from diverse sectors to build a culture of prevention; (3) setting up a training and certification infrastructure for widespread dissemination with fidelity; and (4) acknowledging the necessity for policy changes to provide sustainable funding streams. Additionally, the contributors express their belief in the true value of having a national data repository for real-time and continued tracking of the reach and representativeness of older participants in evidence-based programs . As withThe national stakeholders have helped spawn networks whose primary missions intersect with the goal of accelerating the implementation, evaluation, and dissemination of evidence-based programs for older adults. The CDC’s Healthy Aging Research Network has been instrumental in advancing science toward action and policy , 13. AddIt is heartening to see a variety of networks working together to promote evidence-based programming that can make a difference in the lives of older adults and their caregivers. Complemented by national stakeholders, these networks are providing the needed research and programmatic infrastructure to accelerate the evidence-based movement. They are also identifying existing policies that can facilitate or impede the broader dissemination and sustainability of evidence-based programs for older adults and addressing them accordingly.In this section, we address what is being learned from national, state, and local studies about the program dissemination and implementation processes and health-related outcomes. These are best characterized as translational or pragmatic research studies conducted in real-world settings , 56. TheThe American Recovery and Reinvestment Act of 2009 (ARRA) provided funds to disseminate CDSME programs in 45 states, Puerto Rico, and the District of Columbia between 2010 and 2012 . This inEven in this brief funding period, we see an evolution in the national roll-out, with participant recruitment accelerated over time . This waThis dataset also offered researchers with opportunities to examine similarities and differences in recruitment and attendance based on participant characteristics based on geographical location  as well as racial/ethnic minority groupings . From these efforts, we see that participants living in rural areas are less likely to have evidence-based programs. Additionally, though individuals from rural areas represented a relatively small proportion of participants (25%), they experienced higher program completion rates . With thAdding to the emerging literature about the effectiveness of CDSMP , 67, sevSeveral articles are focused on the adaptation of CDSMP to different settings and populations. Greater attention needs to be paid to strategies for successfully adopting CDSMP in the workplace to meet the needs of persons not typically enrolled in CDSMP programs . It is sIn response to practice and policy concerns, researchers are starting to examine the cost-effectiveness of different evidence-based programs. Building on prior research documenting the cost-effectiveness of CDSMP , a relatIn addition to the suite of CDSME programs, we invited articles about other evidence-based programs that address major public health issues facing the growing older adult population. With the magnitude and impact of falls on older Americans, it was especially salient to include evidence-based fall prevention programs –83.The CIn addition to programs listed in the CDC falls prevention compendium, there are evidence-based fall risk reduction programs. Analyses on A Matter of Balance, an evidence-based program originally designed to enhance confidence in preventing and managing falls  and reduce fear of falling, reveal the mediating effect of increased physical activity on falls efficacy . A relatAdaptations to a variety of physical activity programs for older adults are being further evaluated. Processes involved in the conversion of a practice-based lifestyle program to a formalized, testable evidence-based program are described . Such trThis Research Topic identifies many cross-cutting issues essential for understanding and enhancing evidence-based program delivery, including perceptions of key stakeholders and lessons learned from the field. With a growing emphasis on translational research to address public health problems, there is now a proliferation of dissemination and implementation frameworks to guide research, practice, and policy related to program planning and implementation. As an example, the ACL has organized many of its initiatives around the RE-AIM Framework . An unknAs evidence-based programs roll-out nationally, there are questions about the ability and value of states and counties to implement multiple evidence-based programs. An early study showed that the majority of older adults lived in regions with access to only one evidence-based program . Since dA major theme throughout this collection of articles is the importance of engaging end users and diverse partners in the design and implementation of evidence-based programs. Thus, before implementing the STEADI Tool Kit, a clinically based fall prevention program, it was important to assess health provider’s perceptions about falls among their older patients and their current fall prevention practices . This inThe importance of building strong linkages across aging, public health, and medical care sectors is becoming well-recognized and is now built into many national and state initiatives . Less apWhereas the literature about the effectiveness of evidence-based programs for CDSME and fall prevention has blossomed in recent years with many meta-analyses –106, litFinally, whereas in the early years there was a lack of information about evidence-based programs, in some areas, there is now a profusion of information, making it hard for practitioners and policy makers to know where to retrieve reliable information for selecting and implementing evidence-based programs. While the national stakeholders have excellent materials on their websites, there is a new Evidence2programs Toolkit and website designed to help community-based organizations navigate through the abundance of information about evidence-based programing .Evidence-based programing for older adults has come of age. Past successes in identifying evidence-based programing have led to new emphases into translating research into practice and policy. There are now dedicated efforts being made to understand and incorporate best practices in building and sustaining programs over time. This includes identifying and employing strategies that will improve delivery system infrastructure for enhancing participant recruitment to, and retention in, evidence-based programs. Additionally, a national system is developing to track the spread of programs across geographical areas and monitor key factors such as delivery sites, participant characteristics, program attendance, and even limited outcome measurements.This Research Topic identifies forces mobilizing the evidence-based movement: perspectives from program developers regarding their successes and remaining challenges; the strength of large and small networks in implementing and disseminating an evidence-based approach across aging, public health, and medical care sectors; factors influencing the dissemination, implementation, and outcomes associated with CDSME programs; the emerging literature specifying what is known about community-based falls, physical activity, and behavioral health interventions; and cross-cutting issues in the field.This collection of articles can be seen as a reflection of the evidence-based programing of the past, present, and future. Dramatic progress has been made over the past three decades. Yet, more attention is needed to monitor and understand the dynamic interplay between specific intervention components  and various health, health care, and cost-related outcomes across different settings and populations. Having a better grasp on such information can guide and drive efforts to better target and tailor interventions for specific populations and settings. We recommend that future actions should be driven by a greater appreciation of interacting research, practice, and policy influences on the development, implementation, dissemination, and sustainability of evidence-based programs. It is our greatest hope that this Research Topic provides guidance to practitioners, stimulates new and unanswered research questions, and informs policy decisions that can help support and strengthen evidence-based programing for older adults.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.This paper is included in the Research Topic, “Evidence-Based Programming for Older Adults.” This Research Topic received partial funding from multiple government and private organizations/agencies; however, the views, findings, and conclusions in these articles are those of the authors and do not necessarily represent the official position of these organizations/agencies. All papers published in the Research Topic received peer review from members of the Frontiers in Public Health  panel of Review Editors. Because this Research Topic represents work closely associated with a nationwide evidence-based movement in the US, many of the authors and/or Review Editors may have worked together previously in some fashion. Review Editors were purposively selected based on their expertise with evaluation and/or evidence-based programming for older adults. Review Editors were independent of named authors on any given article published in this volume.


EHP asserts that scientific research is intrinsically influenced by value judgments and that researchers should declare their interests and values as the best way to promote objectivity, transparency, public trust, and good policy.Scientists increasingly find themselves at the center of contentious public policy debates over issues such as chemical regulation and climate change. The general public and politicians often expect scientific advisors to be purely objective.,,The authors wrote the commentary after observing the uproar last year following a preliminary European Commission (EC) review of its policy on endocrine-disrupting compounds.“Looking beyond the scientific details, the debate suggested that scientists should eliminate all interests and values from their research,” says lead study author Kevin Elliott, a professor of philosophy at Michigan State University. But a growing body of research backs up centuries of philosophical writings on human nature: “Values do affect people’s research,” Elliott says, “and there are benefits to being transparent about these values.”Elliott and coauthor David Resnik, a bioethicist at the National Institute of Environmental Health Sciences, decided to use the conflict over the EC review to illustrate how personal ideals are embedded in the very nature of research itself. They explain that researchers’ values inform their presumptions. For instance, one point of contention over the EC review was whether evidence of toxicity in animals should be presumed to predict human effects in the absence of evidence to the contrary.Investigators further draw on their values when they choose the standard of evidence upon which to base health-relevant policy—for instance, whether animal data alone are sufficient to make decisions, or whether animal and human data are required. In both cases, the choice of which evidence to use hinges on values about what kind of risks are acceptable, not science alone, Elliott says.Critics of the EC review also debated whether endocrine-disrupting compounds have a threshold concentration below which no health effects are seen. “Underlying this dispute are value judgments about how much evidence is required to accept or reject a hypothesis,” Elliott says. Toxicologists traditionally work with the threshold hypothesis and require a lot of evidence before they reject it, he says. “Meanwhile, endocrinologists are not as entrenched in toxicological paradigms and therefore may not demand as much evidence before abandoning the threshold hypothesis.”“Many scientific disagreements boil down to issues that are more at the normative end of the spectrum than the factual end, and this controversy over endocrine disruptors is a perfect example,” says Scott Findlay, an environmental scientist at the University of Ottawa. He agrees with Elliott and Resnik that policy-relevant scientific debates would be more productive and transparent if scientists disclosed their presumptions upfront, disclosed conflicts of interest, and clarified the pros and cons of multiple interpretations of the science.“The advantage of discussing values is that people can start thinking about the fact that they make choices based on these internal aspects, and they can become more aware of those and minimize biases in research,” says James Kehrer, a toxicologist at the University of Alberta. But he sees problems with revealing more than financial and employment-related conflicts of interest. “There is no reasonable way to get at one’s core values without violating privacy,” he says.Findlay points out that discussions about presumptions and standards of evidence could make scientists realize that what they thought was a scientific controversy in fact has little or nothing to do with science. “And if it has little or nothing to do with science,” he says, “then scientists are in no more of a privileged position to render an opinion than anyone else.”


An increase in the prevalence of obesity and the metabolic syndrome as a result of changes in eating habits, both quantitative (over-eating) and qualitative (e.g. high fructose consumption), and a lack of exercise contribute to fatty liver disease. Non-alcoholic fatty liver disease (NAFLD) is a clinical histological spectrum ranging from steatosis to non-alcoholic steatohepatitis (NASH) and affects more than 30% of the general population .The European Association for the Study of the Liver (EASL), the European Association for the Study of Diabetes (EASD) and the European Association for the Study of Obesity (EASO) have recently published the first comprehensive joint evidence-based Clinical Practice Guidelines for the management of NAFLD .While reading these important guidelines two critical questions emerge: (1) is this an algorithm that is likely to be followed in routine care, and (2) what are the health economic implications and therefore what is the likelihood of these guidelines being implemented by healthcare systems?While previous guidelines have suggested similar algorithms , a rIf the resources needed are weighed up against the clinical relevance of the findings in terms of treatment decision making, it is questionable whether liver biopsies should be performed on a large scale to differentiate steatosis from NASH. As long as the major treatment option for both conditions is lifestyle intervention including diet and exercise, with limited evidence for pharmacological treatment, discussions about the indications for biopsy-based differential diagnosis, risk of the intervention and the possible financial implications will and have to continue. Despite the fact that no drug has been approved for the indication of NASH, and guidelines recommend that pharmacological treatment should be reserved for those with biopsy-proven NASH, particularly those with significant fibrosis, in reality, treatment initiation is often solely based on clinical judgement without biopsy . This obHowever, even if evidence-based and approved pharmacological treatment was available, it is questionable to what extent local health budgets may be able to offer it to the individual patient. This raises the additional importance of population strategies. Obesity has been developing to a central disease for many diseases, a gateway for ill health . Given tIn the meantime the present Clinical Practice Guidelines provide the best available evidence and should be followed to provide the best possible treatment for our patients. However, assessment of how the guidelines are translated into clinical practice in Europe and what the particular limitations are, is critical in order to address these issues in future updates.


Virtual reality (VR) has made its way into mainstream psychological research in the last two decades. This technology, with its unique ability to simulate complex, real situations and contexts, offers researchers unprecedented opportunities to investigate human behavior in well controlled designs in the laboratory. One important application of VR is the investigation of pathological processes in mental disorders, especially anxiety disorders. Research on the processes underlying threat perception, fear, and exposure therapy has shed light on more general aspects of the relation between perception and emotion. Being by its nature virtual, i.e., simulation of reality, VR strongly relies on the adequate selection of specific perceptual cues to activate emotions. Emotional experiences in turn are related to presence, another important concept in VR, which describes the user’s sense of being in a VR environment. This paper summarizes current research into perception of fear cues, emotion, and presence, aiming at the identification of the most relevant aspects of emotional experience in VR and their mutual relations. A special focus lies on a series of recent experiments designed to test the relative contribution of perception and conceptual information on fear in VR. This strand of research capitalizes on the dissociation between perception (bottom–up input) and conceptual information (top-down input) that is possible in VR. Further, we review the factors that have so far been recognized to influence presence, with emotions  being the most relevant in the context of clinical psychology. Recent research has highlighted the mutual influence of presence and fear in VR, but has also traced the limits of our current understanding of this relationship. In this paper, the crucial role of perception on eliciting emotional reactions is highlighted, and the role of arousal as a basic dimension of emotional experience is discussed. An interoceptive attribution model of presence is suggested as a first step toward an integrative framework for emotion research in VR. Gaps in the current literature and future directions are outlined. In virtual reality (VR), researchers can simulate intricate real-life situations and contexts to investigate complex human behaviors in highly controlled designs in a laboratory setting. These characteristics of VR have proven especially attractive for the investigation of pathological processes in mental disorders, and this technology has steadily gained momentum since the 1990s . The maiFor research into emotional experiences and emotional behavior, such as fear, anxiety, and exposure effects, it is vital that VR can actually induce emotional reactions. By its very nature, VR as a medium is “unreal” and relies on perceptual stimulation  – in particular, visual cues, sounds, and sometimes touch and smell – to trigger emotional reactions. Historically, the first VR scenarios applied in the field of mental disorders used powerful visual stimuli to provoke emotional responses, in particular, height . Soon, minteroceptive predictive coding model is that presence rests on continuous prediction of emotional (interoceptive) states. For example, when expecting the encounter with an anxiety-related stimulus, the prediction would be fear, together with the changes the organism usually undergoes during fear. When encountering the feared stimulus, the organism compares the actual interoceptive state (fear and its symptoms) with the predicted state. According to Another VR phenomenon linked to emotional experience is presence. Presence is a dimensional construct and describes the extent to which a user feels present in a VR environment . TheorieThe aim of this paper is twofold. First, we provide a review of current research into the relationship between perception and information on emotional experience in VR environments. Since exposure therapy has so far been the most common application of VR technology in clinical psychology, our focus lies on VR concerned with fear and anxiety in both healthy and clinical populations. We present a series of our own experiments that were designed to examine the significance of perception vs. conceptual information and presence for the experience of anxiety, and fear in VR environments. Second, an integration of the literature regarding immersion, presence, and emotional experience in VR is still outstanding. Different VR systems, diverging operationalizations of presence, and study samples ranging from healthy controls to patients with anxiety disorders make it difficult to draw firm conclusions. Based on a review of presence research, we suggest a new interoceptive attribution model of presence as a step toward an integrative framework for emotion research in VR.emotional processing theory by The most influential theoretical conceptualization of dysfunctional fear to date is offered by the reflective-impulsive model of social behavior propose that impulsive, emotional reactions are fast, and governed by the laws of association (spreading activation), while reflective behavior is subject to more flexible, cognitive control. However, the impulsive and the reflective systems are supposed to interact, allowing conceptual information (input to the reflective system) to activate rapid emotional reactions  and the conceptual (fear-related information) paths. Perceptual fear-related cues are assumed to rapidly evoke physiological and behavioral fear reactions, whereas fear-related information is expected to produce subjective fear reactions, but only a poorer physiological activation . Strack eactions . In pracVirtual reality is a particularly suitable tool as it offers an opportunity to differentiate the two paths for eliciting emotion. In VR, cue propositions can be activated by presenting feared objects perceptually , and, unrelated to the perceptual presentation, activating the meaning propositions by informing a person of the existence of a feared object, or situation outside the VR scenario they are immersed in. The laboratory setting of VR further allows the online assessment of different fear reactions  in a highly controlled setting.In a series of studies we investigated the relative importance of perceptual fear-related cues and conceptual fear-related information on the activation of fear in different anxiety disorders. We assumed that fear reactions in specific phobia  are primarily caused by simple perceptual fear-related cues like a spider, whereas the impact of information on fear  should be less pronounced. We directly and separately manipulated the two paths by using VR to present the visual cues on the one hand and the independent information about the existence of a real fear-evoking stimulus on the other hand.Figure 1.In a first study with patients suffering from spider phobia , we founIn a second study, we addressed the question whether these findings generalize to other types of phobias. While in spider phobia, fear is characteristically triggered by a stereotypical object , in other phobias – those of the situational subtype, e.g., claustrophobia – triggers are more context-related, involving more complex perceptual stimuli . TherefoIn summary, in these studies we demonstrated for the first time in an integrated multimodal experiment that perceptual cues and conceptual information can provoke fear reactions in specific phobia, with additive effects if combined. Interestingly, perceptual cues alone seem to induce more self-reported fear than information alone, regardless of the type of specific phobia . This is in line with findings that fear enhances perceptual, but not mental processing e.g., , implyinAs social fears are generally thought to be more cognitive in nature than specific phobias , we expeFigure 2. Finally, we expected high socially anxious participants to show stronger fear reactions than low socially anxious participants. We randomly allocated 48 healthy participants to either the information condition, the cue condition, or the combined condition. . Physiological parameters  decreased significantly over time. There was a trend SCL to differ between groups, with the highest SCL in the visual cue condition (p = 0.066), but there were no other effects of social anxiety or condition. With a mean Social Phobia Inventory , but markedly below the mean  of patients with social phobia reported by We hypothesized that a real observer outside VR (information condition) would evoke significantly stronger subjective and physiological fear reactions than a visual observer in VR . Further, we expected that a combination of real and VR audience (combined condition) would result in the strongest subjective and physiological fear reactions. The experimental conditions are presented in ry SPIN;  score ofin vivo cues vs. imagination). However, the interpretation of our results on social anxiety remains preliminary, as we did not assess patients or acute fear as in the studies of specific phobia. It seems worthwhile to continue this research with different kinds of specific phobias and more complex anxiety disorders like agoraphobia, panic disorder, and social anxiety disorder.In summary, the VR designs reported here confirmed the possibility of eliciting fear reactions via different routes . Patients with specific phobia seem to be particularly sensitive to perceptual cues. Interestingly, this finding was the same for spider phobia  and claustrophobia (situation type). For social anxiety, no differences in activation of the fear structure between the two paths were found. These observations are in line with The association of presence and emotional experience in VR exposure therapy is an issue of debate. Presence is commonly regarded as a necessary mediator that allows real emotions to be activated by a virtual environment . While tPresence has been conceptualized, and consequently operationalized and manipulated, in very different ways. These ranges from a manipulation of presence by providing more or less sophisticated VR technology to the diverse methods of assessing presence, either by subjective ratings taken online during the VR experience, or afterward via questionnaires. Presence questionnaires vary greatly with regard to the constructs they measure; however, what they have in common is that they ask participants for a subjective judgment regarding their experience of presence. With this in mind, we will use the definition of VR simulations can be more or less graphically enhanced, multimodally integrated, and interactive. More sophisticated technology is often thought to result in more presence. Already As for possible effects of immersion on emotions, the picture becomes more complicated. While some authors report an increase in emotional responses in more immersive compared to less immersive VR systems , others Another possibility to test the influence of immersion is the use of different perceptual modalities or multimodal perceptual cues. Thus, we compared tactile cues (touching a spider model) with visual cues  in patients with spider phobia . As expeper se increase emotional experience, but that the emotionally enhancing effect of immersion might be limited to arousing emotions (see the discussion above), supports this conclusion. For example, Taken together, there is considerable evidence that the level of immersion a VR system provides exerts an effect on the presence experienced by the user . This efn = 31) undergoing VR exposure therapy.The association of presence and emotion has been mainly investigated by means of correlations between these two measures. Correlations between presence and emotional experience in VR have been consistently reported, especially in the literature on VR exposure therapy , althougn = 210) of test-anxious students and non-anxious students (groups assigned according to questionnaire scores) to a VR environment that simulated a university exam, and a neutral VR. The authors reported no correlation between fear and presence in the neutral VR, and a considerably stronger correlation between presence and fear in the test anxious group , or because of the nature of the sample , the greater the likelihood of finding a significant correlation between presence and emotion. A possible explanation for this phenomenon could again be arousal. Already, The findings reviewed here highlight important advances in the study of fear and anxiety in VR environments. The data on perceptual fear cues and conceptual information show that both are viable triggers of fear reactions . The posper se enhance therapy effects , presence and emotion is not clear. First, we do not know how the effect of perception vs. information on emotion is produced. On the one hand, fear-related elements in VR are input cues to the fear network – as proposed in emotional processing theory  – and miinteroceptive attribution model of presence . Since presence is a subjective experience, common measures of presence explicitly call the participants to make a judgment of the degree of presence they feel in VR. Based on the results reviewed in this paper, we propose that participants make this judgment based mainly on two sources of information: (1) immersion and (2) the degree of arousal they feel. As for immersion, participants might base their presence judgment on the perceptual distance they experience from the real world setting, i.e., the less stimulation they receive from the real world, and the more stimulation from the VR scenario, the higher the level of presence they will indicate. Of course, this hypothesis needs further empirical confirmation. With regard to emotion, we believe that participants will give higher presence ratings if they feel emotionally affected. As arousal is a particularly strong indicator of emotional involvement, arousing emotions should lead to higher presence ratings, and correlate more closely, with presence ratings, than calm or serene emotional states – a picture that is in fact found in the literature. Interestingly, whether the experience of arousal per se, or the attribution of this arousal to the VR scenario is necessary for the experience of presence has not yet been investigated. Additionally, immersion itself is likely to increase arousal (in vivo exposure as compared to VR exposure, appears vital (Concerning presence, the preliminary conclusion we would draw from the findings reviewed here is that the case for a crucial involvement of arousal in the experience of presence is compelling. However, the mechanism of this effect cannot be discerned yet.  arousal . In esse arousal , and on rs vital . We can rs vital . A direcJulia Diemer: data analysis, wrote the manuscript. Georg W. Alpers: study conception, contribution to data analysis, and the manuscript. Henrik M. Peperkorn: study conception, data acquisition, and analysis, contribution to the manuscript. Youssef Shiban: data acquisition and analysis, contribution to the manuscript. Andreas Mühlberger: study conception, data analysis, contribution to the manuscript. All authors have approved of the final version of the manuscript and its submission.The Guest Associate Editor Fritz Strack declares that, despite being affiliated to the same institution as author Henrik M. Peperkorn, the review process was handled objectively and no conflict of interest exists. Andreas Mühlberger is stakeholder of a commercial company that develops virtual environment research systems. Julia Diemer, Georg W. Alpers, Henrik M. Peperkorn and Youssef Shiban have no conflicts of interest.


The promotion of a sustainable evidence-based health care practice requires multiple efforts. Since master’s level courses in health sciences in Sweden target practicing professionals, integrating knowledge and skills related to evidence-based practice into higher education is one way of reaching this goal. Here we describe the implementation of the course Evidence Based Practice, 7.5 ECTS at the Master of Medical Sciences program at Lund University, Sweden. The course is elective and enrolls registered nurses, physiotherapists and occupational therapists, i.e. the main fields. It is internet based, with five seminars on campus. The learning activities are interprofessional, with each student taking a bearing on their main field. The mode of teaching applied is Targeting Specific Skills of Evidence Based Practice .Examples of learning outcomes are:Independently identify, explain and discuss methods for developing evidence-based practice; independently identify problem areas and formulate appropriate clinical questions (CQ); identify, critically review and discuss evidence-based knowledge in healthcare.Besides lectures and seminars the students have four mandatory assignments:1. Define 3-4 relevant CQ for the course.2. Review the literature related to the CQ above, and classify two papers according to the PICO map .3. A clinical guideline is critically reviewed and summarized according to a checklist .4. A web-site focusing on health information for lay people is critically reviewed and summarized.Teacher feedback is given throughout.The examination is an individual paper. Following the PICO map CQs are identified and formulated. The literature is systematically reviewed and the five papers with the highest evidence are critically appraised and summarized, including a popular science abstract. At the final seminar the students present and defend the paper orally, including an opposition on another student’s paper. In their evaluations the students express above all that they search and review scientific papers more systematically and critically. They state that they now know how to frame CQs and that they apply the acquired knowledge and skills in professional practice. The implementation of this type of course into higher education is a feasible way to enhance the use of evidence based knowledge and skills in health care practice.


Purpose. To evaluate the methodological quality of age-related macular degeneration (AMD) clinical practice guidelines (CPGs). Methods. AMD CPGs published by the American Academy of Ophthalmology (AAO) and Royal College of Ophthalmologists (RCO) were appraised by independent reviewers using the Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument, which comprises six domains , and an Overall Assessment score summarizing methodological quality across all domains. Results. Average domain scores ranged from 35% to 83% for the AAO CPG and from 17% to 83% for the RCO CPG. Intraclass correlation coefficients for the reliability of mean scores for the AAO and RCO CPGs were 0.74 and 0.88, respectively. The strongest domains were Scope and Purpose and Clarity of Presentation. The weakest were Stakeholder Involvement (AAO) and Editorial Independence (RCO). Conclusions. Future AMD CPGs can be improved by involving all relevant stakeholders in guideline development, ensuring transparency of guideline development and review methodology, improving guideline applicability with respect to economic considerations, and addressing potential conflict of interests within the development group. Blindness caused by age-related macular degeneration (AMD) affects an estimated 32.4 million people worldwide, with an additional 191 million people suffering from moderate and severe AMD-related vision impairment . In addiThe Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument is a valid and reliable tool for assessing the methodological quality of CPGs –9. The oThe AGREE II instrument consists of 23 assessment items, with evaluators score on a scale of one (strongly disagree) to seven (strongly agree). The 23 items are organized into six quality domains: (1) Scope and Purpose; (2) Stakeholder Involvement; (3) Rigor of Development; (4) Clarity of Presentation; (5) Applicability; and (6) Editorial Independence . Two finThe AAO CPG outlines recommendations for the diagnosis, treatment (nonneovascular and neovascular AMD), and follow-up care for adult AMD patients . The RCODomain scores are summarized in ICCs for the AAO and RCO CPG scores were 0.74 and 0.88, respectively. Strengths and weaknesses of the guidelines are summarized in To our knowledge, there have been no previous studies critically assessing multiple AMD CPGs using the AGREE II instrument. Our analysis with the AGREE II instrument outlined a number of methodological shortcomings in the guidelines.With respect to Stakeholder Involvement (Domain 2), both CPGs failed to describe participation of patient representatives, if any, in the guideline development process. While the RCO guideline cited the involvement of one patient representative in the guideline development group, it was not possible to determine if patient groups were consulted in the development of the AAO guideline. Neither CPG detailed outcomes of public review nor how reviewer comments were used to guide the process of guideline development. An additional criterion within Domain 2 is the requirement for full disclosure of panel members' individual roles and areas of expertise. Although panel members' occupational degrees and areas of expertise were provided, CPG did not delineate panel members' specific roles within the guideline development group. Similar weaknesses in stakeholder involvement were described in the studies assessing CPGs for the management of cataract and primary open-angle glaucoma , 11. EnsThe AMD CPGs had a few shortcomings in Rigor of Development (Domain 3), the largest AGREE II domain. First, both CPGs failed to disclose details concerning external review methods, including feedback gathered from external review, patient input if any, and how the outcomes informed guideline development. Second, the CPGs lacked detailed search terms, explicit criteria for selecting evidence, names of databases searched (AAO), and dates of search (RCO). Additionally, while the AAO CPG provided an explicit rating system indicating the quality of each primary source, the RCO CPG lacked such information. Finally, the RCO CPG lacked primary source citations following each recommendation, rendering the link between evidence and recommendations unclear. Consistent citation of evidence and disclosure of evidence search methods should be incorporated into AMD CPGs to ensure rigor and transparency of evidence-based recommendation development.Regarding Clarity of Presentation (Domain 4), recommendations were often difficult to identify within the body of each guideline due to the fact that they were either embedded within paragraphs (AAO) or listed without any direct citations of evidence or grade level indicating recommendation strength (RCO). Improving Clarity of Presentation is important as this facilitates guideline interpretation and implementation.In Applicability (Domain 5), a common shortcoming was the lack of research on cost information and description of how economic considerations guided recommendation development. While the RCO CPG mentioned cost considerations as important factors affecting treatment adherence, few provisions were given regarding specific ways to address economic barriers to the implementation of particular recommendations in either guideline. Inclusion of cost and resource information within CPGs can facilitate transparency of treatment selection and is critically important in light of medical technology advancement amid rising healthcare costs . In addiFinally, the AMD CPGs had several gaps in Editorial Independence (Domain 6). Neither CPG delineated methods by which commercial interests were sought nor addressed how potential conflicts of interest (COIs) may have influenced guideline development. The AAO provides a statement on its website regarding its policy on industry relationships within clinical guideline committees ; howeverThere were several potential limitations to this study. First, any rating instrument may lack objectivity. However, the AGREE II instrument has been proven to be reliable and valid, particularly with four independent reviewers. In this study, the independent reviewers were highly reliable (ICCs of 0.74 and 0.88), which is consistent with the existing research that supports the validity of AGREE II as an assessment tool –22. SecoIn summary, the AGREE II evaluation of AMD CPGs highlighted methodological weaknesses in the domains of Stakeholder Involvement, Rigor of Development, Applicability, and Editorial Independence. We recommend a number of improvements for future guidelines, using whenever possible one of the two CPGs reviewed herein as a model . Regular


An increasing number of realist evaluations are being conducted from a wide range of disciplinary perspectives and with diverse, fit-for-purpose methods. This commentary discusses the recent BMC Medicine publication of RAMESES II reporting guidelines for realist evaluations. Knowledge users such as program implementers and decision-makers will benefit from the increased transparency of reporting and interpretation in light of the totality of evidence encouraged by this guidance. It is hoped that these reporting guidelines will eventually lead to improved knowledge synthesis and contribute to the cumulative science regarding realist evaluation.https://bmcmedicine.biomedcentral.com/articles/10.1186/s12916-016-0643-1Please see related article:  Realist evaluations are theory-driven evaluations that seek to understand how complex interventions work, for whom they work, and how programs and their effects are influenced by the context and a setting that is rooted in the philosophy discipline , 2. RealBMC Medicine, Wong and colleagues seek to improve the transparency of reporting realist evaluations by developing consensus and evidence-based reporting guidelines for realist evaluations [BMJ Open [In a recent article in luations . They usluations  and outlluations  to develBMJ Open  and 35 eThe tool consists of 20 items, which have been broken down into the following six sections: Title, Summary of Abstract, Introduction, Methods, Results, and Discussion. Each of the 20 items included in their reporting realist evaluations tool includes a detailed rationale for each item and exemplars of good practice. They have also built in flexibility in terms of the order of reporting and they strongly encourage authors to document a justification for any variance from the reporting items, including omissions of items. The first item is related to identifying the study as a realist evaluation in the title, which will aid in identifying these types of studies in the future. Items 3 , 4 (program theory), 7 , 8 (environment and context), and 10  are particularly important because, based on evaluations of the reporting of realist reviews [The Lancet guidelines [One major advance of this reporting guideline is the encouragement to situate the realist evaluation in the totality of the evidence item 18), which will help program implementers to interpret the findings in light of other relevant evidence while considering the contribution of differences in settings and populations. This is in keeping with other global initiatives to consider the entirety of the evidence when reporting results of primary studies, such as , which widelines . We agreThe authors are already promoting the uptake of these reporting standards through the RAMESES listserv, and training workshops and materials, which will assist in their uptake by program evaluators. Also, the authors plan to evaluate the usefulness and impact of these reporting guidelines in the future. An additional activity that could enhance the impact of these reporting guidelines is registration with the EQUATOR Network . Also, tThe RAMESES II reporting guidelines for realist evaluations is an important initiative to ensure these evaluations are reported in sufficient detail, in the context of existing evidence, and with a rating of strength of evidence for main findings that will greatly assist users of the evaluations. Because reviews are only as good as the included studies, in our opinion, this could also eventually improve realist syntheses that include realist evaluations . We look


Policymakers, stakeholders and researchers have not been able to find research evidence about health systems using an easily understood taxonomy of topics, know when they have conducted a comprehensive search of the many types of research evidence relevant to them, or rapidly identify decision-relevant information in their search results.To address these gaps, we developed an approach to building a ‘one-stop shop’ for research evidence about health systems. We developed a taxonomy of health system topics and iteratively refined it by drawing on existing categorization schemes and by using it to categorize progressively larger bundles of research evidence. We identified systematic reviews, systematic review protocols, and review-derived products through searches of Medline, hand searches of several databases indexing systematic reviews, hand searches of journals, and continuous scanning of listservs and websites. We developed an approach to providing ‘added value’ to existing content  and to expanding the types of evidence eligible for inclusion . Lastly, we developed an approach to continuously updating the online one-stop shop in seven supported languages.The taxonomy is organized by governance, financial, and delivery arrangements and by implementation strategies. The ‘one-stop shop’, called Health Systems Evidence, contains a comprehensive inventory of evidence briefs, overviews of systematic reviews, systematic reviews, systematic review protocols, registered systematic review titles, economic evaluations and costing studies, health reform descriptions and health system descriptions, and many types of added-value coding. It is continuously updated and new content is regularly translated into Arabic, Chinese, English, French, Portuguese, Russian, and Spanish.Policymakers and stakeholders can now easily access and use a wide variety of types of research evidence about health systems to inform decision-making and advocacy. Researchers and research funding agencies can use Health Systems Evidence to identify gaps in the current stock of research evidence and domains that could benefit from primary research, systematic reviews, and review overviews.The online version of this article (doi:10.1186/1478-4505-13-10) contains supplementary material, which is available to authorized users. Such resources have been highlighted as a central pillar of broader efforts to support evidence-informed decision-making about health systems . One-stoDeveloping a one-stop shop for health system policymakers, stakeholders, and researchers requires addressing three challenges. First, these groups need to be able to find research evidence about health systems using an easily understood taxonomy of topics. Such a taxonomy would ideally be organized in a way that reflects the ways in which these groups think about health systems  and using terminology they are familiar with. While a number of taxonomies exist , they teSecond, health system policymakers, stakeholders, and researchers need to know when they have conducted a comprehensive search of the many types of research evidence relevant to them. Providing such reassurance with a single one-stop shop means ensuring that it incorporates the many types of research evidence needed to answer their questions and contains comprehensive inventories of each of these types of research evidence. Many existing one-stop shops address questions about the effectiveness and cost-effectiveness of policy options , but not questions about the policy problem  or about many features of policy options  and implementation considerations  , 15. MorThird, these groups need to be able to rapidly identify decision-relevant information in the results of their search of a given one-stop shop. For systematic reviews, for example, a busy policymaker would ideally like to be able to quickly scan the search results to know whether the retrieved reviews are of high quality, whether the searches for research evidence were conducted recently, and whether the studies included in the reviews were conducted in their own health system or in health systems that share characteristics that are likely to influence the applicability of the findings to their own system . The samOur objective was to develop and refine the methods for an easily searched, comprehensive, free, one-stop shop for research evidence that could provide decision-relevant information about the many types of questions asked by policymakers, stakeholders, and researchers regarding health systems. To address this objective, we proceeded in four stages: i) developing a taxonomy of governance, financial, and delivery arrangements within health systems and of implementation strategies within health systems; ii) building content by identifying, selecting, and categorizing content and by adding value to that content; iii) expanding the types of content; and iv) continuously updating the resulting one-stop shop Table .Table 1We developed a taxonomy of health system topics and iteratively refined it by drawing on existing categorization schemes and by using it to categorize progressively larger bundles of systematic reviews and systematic review protocols. First, we drew on system-wide categorization schemes, such as the WHO’s ‘building blocks of health systems’ , and on Second, two reviewers independently used the taxonomy to categorize systematic reviews and systematic review protocols that were identified by searching Medline (OVID) and by hand searching the Cochrane Database of Systematic Reviews. The Medline search strategy was based on the one used by Cochrane’s Effective Practice and Organization of Care review group, which is the review group focused specifically on producing reviews about health system arrangements and implementation strategies . We iterThe taxonomy continues to be a ‘living document’ that we update periodically and then use to retrospectively re-categorize all records affected by a change in categorization Table . The taxWe identified additional systematic reviews and systematic review protocols, as well as review-derived products (evidence briefs for policy and overviews of systematic reviews) , 18, thrWhile building content for the one-stop shop and drawing on user testing and team member experiences in training and interacting with policymakers and stakeholders, we continued to iteratively refine the taxonomy Table . We deriWe also developed an approach to providing added value to existing content. For each systematic review, two reviewers independently assessed its methodological quality using AMSTAR , coded iIn the third stage, we developed an approach to expanding the types of evidence eligible for inclusion Table . The newFinally, we developed an approach to continuously updating the online ‘one-stop shop’ in all six WHO official languages  as well as Portuguese Table  and, thrhttp://www.healthsystemsevidence.org) – a one-stop shop for research evidence to support policymakers, stakeholders, and researchers interested in how to strengthen or reform health systems or in how to get cost-effective programs, services, and drugs to those who need them. The one-stop shop contains many functionalities designed to address the three challenges described earlier in this paper: health system policymakers, stakeholders, and researchers can now find research evidence about health systems using an easily understood taxonomy of topics, know when they have conducted a comprehensive search of the many types of research evidence relevant to them, and rapidly identify decision-relevant information in the results of their search  developing and iteratively refining a taxonomy based on existing categorization schemes and the practical challenges that arise when using it to code a heterogeneous body of research evidence; ii) building content through systematic searches of a broad array of sources, adding value to that content in ways that have been identified as important to policymakers and stakeholders, and having two reviewers independently participate in each step of the process; iii) expanding the types of content included in the one-stop shop as we achieve comprehensiveness for existing content and recognize the next most pressing gap faced in the type of research evidence needed; iv) designing the one-stop shop in a way that uses links to drive traffic to the websites of those groups involved in producing and/or disseminating these documents  or in preparing decision-relevant summaries  so that these groups get credit for their work in the form of website hits; and v) continuously updating the one-stop shop in all seven supported languages.The limitations of our study are: i) having some sub-categories in the taxonomy containing very large numbers of documents, which could frustrate busy policymakers, stakeholders, and researchers; ii) potentially missing review-derived products, particularly evidence briefs , which aTaxonomy of governance, financial and delivery arrangements within health systems and of implementation strategies within health systems.(DOCX 18 KB)Additional file 1:


The field of research on knowledge mobilization and evidence-informed policy-making has seen enduring debates related to various fundamental assumptions such as the definition of ‘evidence’, the relative validity of various research methods, the actual role of evidence to inform policy-making, etc. In many cases, these discussions serve a useful purpose, but they also stem from serious disagreement on methodological and epistemological issues.This essay reviews the rationale for evidence-informed policy-making by examining some of the common claims made about the aims and practices of this perspective on public policy. Supplementing the existing justifications for evidence-based policy making, we argue in favor of a greater inclusion of research evidence in the policy process but in a structured fashion, based on methodological considerations. In this respect, we present an overview of the intricate relation between policy questions and appropriate research designs.By closely examining the relation between research questions and research designs, we claim that the usual points of disagreement are mitigated. For instance, when focusing on the variety of research designs that can answer a range of policy questions, the common critical claim about ‘RCT-based policy-making’ seems to lose some, if not all of its grip. Under the impulse of evidence-based medicine, evidence-based policy-making now appears to have a life on its own. Even more so, evidence-based policy-making, as a slogan, is now conveniently tossed around by academics and policy actors, sometimes purporting ill-defined meanings. And it is not to say that there haven’t been enough writings about it. To this day, a considerable amount of papers, books and reports have been published – from either a theoretical, empirical and/or normative stance: countless articles devoted to defining what evidence-based policy-making is and is not, what levels of success (or lack thereof) have been reached in implementing evidence-based principles and practices in policy-making, what barriers need to be overcome to fruitfully implement these principles , etc. Tall-things-evidence pessimist – something we erroneously took for granted to be a bygone figure. Then again, a lot has been written both in defense and in reaction to evidence-based policy-making and its normative and practical implications. Indeed, some substantial misunderstandings persist, likely fueled by methodological disputes and epistemologically conflicting views about the role of science in society, in general, and in the policy arena more specifically.In parallel, one still frequently comes across the usual reluctant, skeptical and Our objective here is not to write a definitive defense of evidence-based policy-making but to try and dismiss some of the misunderstandings about it. Our motivation stems from the fact that these conflicting views are in our view the outcome of two intertwined factors: 1) an unsatisfactory portrayal by advocates of evidence-based policy-making; and 2) a misleading depiction by skeptics.Our argument revolves around these lines: advocates of evidence-based policy-making have yet to present a nuanced, epistemologically realist and methodologically substantive perspective; and skeptics have yet to embrace the idea for fear of improbable  implications. We won’t make here the customary distinctions between evidence-based and evidence-informed. But, as shall be clear from this article, what we can expect from evidence – even at best – is a supporting role in the policy-making process. More specifically, we present a detailed view on how relevant research can be sorted out beforehand – and beyond surveying hierarchies of evidence – by examining the adequacy between types of questions in need of evidence and the research designs most likely to provide relevant and valid answers.From its gradual emergence as a potentially viable and fruitful idea, evidence-based policy-making has prompted some reservations. From technocratic suspicions , to evocAs in any area of practical rationality, it is rather unlikely that evidence can be the only decisional factor. Under practical constraints and limited rationality, realistic expectations can only go so far as to make valid and relevant evidence one of the factors weighing in a decision process and, at best, a weighty justification for a specific course of action. On the other hand, and this is possibly where evidence-based policy-making has a strong case, policy processes can be more or less informed by evidence. Alternatively, they can be completely uninformed by evidence, thus giving weight to other sources of justification . Evidence-based policy-making, rather than trying to turn everything into an occasion to delve into scientific literatures, has a clear answer to these alternatives scenarios: one should try and find the most reliable, most objective, most relevant evidence available and make the most out of it within practical constraints.Researchers need a better understanding of the policy process, funding bodies must change their conception of how research influences policy, and policy makers should become more involved in the conceptualisation and conduct of research. Until then, researchers should be cautious about uncritically accepting the notion of evidence based policyInasmuch as evidence-based policy-making requires the interplay of knowledge generation (by scientists and/or academics) and public decision-making processes, it is likely to be a complex enterprise. One obvious reason is that both processes are complex enough on their own and commonly take place independently of one another. As Black  rightfulThis appears entirely reasonable in that it does not discredit the principles themselves, but rather sheds light on their difficult implementation and provides realist expectations about the success rate of evidence-based policy-making practices .So, what does it really imply? Multiple things, mainly along two lines: 1) behavioral change from policy actors and the adaptation of the policy-making process, and 2) behavioral change from researchers in their way of communicating research evidence. On the one hand, proper infrastructures and resources need to be in place to support evidence-based principles , 9 and tanother context is not straightforward, not even with well executed RCT’s , hiding methodological limitations, and presenting sloppy reviews of the literature as evidence based all fail to honor ethical obligations described in professional codes of ethics.Going back to the dilemma expressing two possibilities in the policy process, we here reiterate that the general aim of evidence-based policy-making is to go beyond vested interests, conventional wisdom, preconceived notions, opinions, perceptions, etc., and to ensure greater influence of reliable evidence on policy advice. Then again, the idea is to instill a greater dose of rationality into the policy-making process by taking into account relevant research findings – not to uncritically import everything considered as evidence into the policy process. In short, evidence-based policy has nothing to do with scientism: its aim is rather to make the policy-making process less romantic  and more influenced by reason combined with empirical evidence. The following quote from Gambrill  brillianHere also, a parallel with science is possible where critical rationalism fuels research practices. Taken from philosopher K.R. Popper , 17 thisRCT, randomized controlled trial