Health literacy (HL) is low for 40–50 % of the population in developed nations, and is strongly linked to many undesirable health outcomes. Older adults are particularly at risk. The intervention research on health literacy in ageing populations project systematically created a large inventory of HL interventions targeting adults age 50+ , to support practical production of policy and practice guidelines for promoting health literacy in European populations.We comprehensively surveyed international scientific literature, grey literature and other sources (published 2003+) for implemented HL interventions that involved older adults. Studies were screened for eligibility criteria and further selected for aspects important in European public health policy, including priority diseases, risk factors and vulnerable target groups. Interventions were prioritised using a multiple criteria tool to select final interventions that also featured strong evidence of efficacy and a broad range of strategies.From nearly 7000 written summaries, 1097 met inclusion criteria, of which 233 were chosen for scoring and ranking. Of these, seven had the highest multi-criteria scores. Eight more articles were selected based on rounded criteria including a high multi-criteria score as well as elements of innovation. Final selections were 18 articles describing 15 programmes, which feature strong evidence of efficacy among important diseases or risk factors and vulnerable groups, or that had success with elements of innovation were identified. Most programmes tried to increase skills in communication, self-management and understanding healthcare or lifestyle choices.These programmes have multiple positive attributes which could be used as guidance for developing innovative intervention programmes to trial on European older adults. They provide evidence of efficacy in addressing high priority diseases and risk factors.The online version of this article (doi:10.1186/s13104-016-2056-9) contains supplementary material, which is available to authorized users. Health literacy (HL) may be defined as “the degree to which people are able to access, understand, appraise, and communicate information to engage with the demands of different health contexts to promote and maintain health across the life-course” . Other dHealth literacy is considered relatively low for a high percentage 40–50 %) of the population in developed nations –10. Ther0 % of thThe Intervention Research on Health Literacy in Ageing populations (Irohla) project was funded for 3 years by the European Union (EU) as part of the seventh framework programme. Many (thousands of) peer-reviewed articles have described health literacy interventions that include older adults. Irohla’s ultimate aim was to identify, test and recommend from this large group of studies a maximum of twenty interventions for improving health literacy in older adults (age 50+), as part of larger EU-wide initiatives to promote healthy ageing. Conducting a systematic search and creating a comprehensive inventory was an important methodological task in Irohla, from which a shortlist of promising interventions could later be recommended for adaptation, piloting and implementation within European countries. Interventions were searched for in health care settings or delivered by health care professionals, as reported in scientific or grey literature or in other sources, and which included older adults (age 50+). We developed and applied a multi-criteria selection tool to identify fifteen interventions with the best balance of priority targeting, good quality design, efficacy and innovation , and recommended these programmes for subsequent development and practical implementation in EU countries.The phrase and concept of “health literacy” is most established among English- and Spanish-speaking professionals. Irohla  conceptuThe IIMHL itself is underpinned by a detailed taxonomy of health literacy that includes seven objectives classified as modifiable determinants applicable at the individual or system level such as: enablement through information, education and skills development; behavioural, social and contextual support; professional development; and, environmental change or enhancement  AND (European OR Europe OR European-country name) AND .Any European language, but abstract or summary available in English;publication year = 2003+;target group must not exclude people age 50+;target group must reside in a developed and predominantly western-culture country ;Both scientific and supplemental searches found many review and policy documents that cited evidence from multiple interventions. Most of the review or policy documents, especially if they had specific keywords such as “older adults” , and increases in disease knowledge.Entries were screened independently by two reviewers  on title and abstract for inclusion criteria, with disagreements resolved upon referral to a third screener (TK) or by discussion. Most review or policy documents which had specific relevant keywords were also hand searched for additional intervention studies that met the inclusion criteria.The research objective was to choose a relatively small number <20) of the most promising (and relevant to European priorities) HL intervention programmes to form the basis for practical adaptation and trialling in European populations. Key relevant European public health priorities are many, including: to develop innovative interventions to promote adoption of healthy lifestyles  of the m, to redrHence, from our initial inventory of all HL interventions, we identified (from screening title and abstracts) those articles that addressed the most important (in terms of causes of premature death or disability in Europe) diseases or risk factors , and thaNo ethics approval was required for this systematic survey. Statements regarding ethics approval were clearly stated in most if not all of our selected articles.In total, 6989 intervention descriptions were found from combined sources . Most articles (5561 unique reports) came from the scientific literature search. Snowballing revealed a further 421 items in review and policy documents, 371 items came from the English-language Internet search. An additional 53 items were found from the specific searches for articles about disadvantaged European ethnic minorities. Over 500 articles were found and screened from all other sources. Figure It was desirable that the final chosen programmes offer diverse strategies and address a broad range of diseases and risk factors to ensure study findings have relevance across individual EU countries and communities. After multi-criteria scoring, 15 intervention programmes described in 18 articles were selected as having the best balance of desirable traits, because they demonstrated efficacy among vulnerable target sub-groups, innovative approach to improving HL, greater duration or significance of impacts. These interventions either had the highest scores  or at least one of the following traits: a high breadth of intervention strategies (n = 5), especially uncommon strategies (n = 9), possessed the highest multi-criteria scores of those articles concerned with communication with professionals n = 2) and/or focused on prevention of disease (n = 7). Table  and/or fMost of the intervention participants were in urban communities of the United States. Other interventions took place in Canada , the UniTwo papers , 32 descFrom almost 7000 studies in many western-culture countries, 15 programmes were identified with promising attributes that could be used as guidance for developing innovative intervention programmes to trial on European older adults. The selected items have multiple positive elements. They provide evidence of efficacy in addressing high priority diseases and risk factors as well as the public health policy goal of disease prevention. They describe success in patient groups that are vulnerable or in health professionals trying to address a clinical risk factor. The selected interventions present a wide breadth of health promotion strategies and address diverse health problems. The interventions collected through this comprehensive and systematic survey are supported by multi-cultural theoretical and practical health promotion frameworks –49.Virtually all interventions which scored highly in the multi-criteria scoring exercise aimed to educate, improve patient skills and motivate. Cultural adaptations and actions to change the physical environment (such as paying for travel expenses) were also common. Although target groups in the final interventions are both older adults and professionals, health professionals are much less often the targets of health literacy studies in well-evaluated and documented research, although health literacy often features in professional training . Only siFor now, building on the search and grading work described here, the Irohla project intends to recommend interventions that are most likely to be successful for older adults in Europe. Recommendations will combine the replication of some interventions previously shown as successful, but with further evidence for suitable social and cultural adaptations .We do not know of a similar inventory to the one which we created, or of a similar effort to assess suitability from so many intervention programmes to identify a small number that are best for trial in the European context. Our approach was necessarily driven by the practical needs and objectives of the funders to promote healthy ageing. The diversity of selected interventions, in terms of implementation plans, delivery modes and frequency, as well as duration of measured impacts, is valuable. To be readily adaptable for culturally diverse European populations with increasingly complex health needs and risk factors due to ageing demographics , 57, heaInevitably, the selection process was adapted to the data available and to the demands of the development process needed to inform future practical EU guidance. Search terms were necessarily limited and meant, for instance, that relatively few studies were found about some widely used communication methods such as Teach Back , 59. We Response from the expert survey (to obtain information on tested but unpublished interventions) was low. Non English-language Internet results were generally less useful than those available in the English language. Importantly, the phrase and concept health literacy has not been widely adopted by European public health agencies which led to some subjectivity when screening and applying eligibility criteria. Ethnic minorities specific to the United States (Hispanics and African Americans) were frequent participants in the selected interventions; one of the subsequent challenges for Irohla will be how to appropriately adapt the principles of these successful programmes for use among European ethnic and minority groups.Our multi-criteria scoring was probably weighted in favour of studies with demonstrable improvements in outcomes rather than type of evidence, duration of interventions or innovation. This may be a strength or a limitation of our work (depending on what priorities are foremost in mind of the reader). We were also not able to rigorously evaluate innovative but untested intervention ideas.From thousands of intervention studies, we identified 15 promising programmes that featured strong evidence of efficacy among important diseases or risk factors and vulnerable groups, or that had some evidence of success and important innovation in older populations. To narrow down such a large group of items to such a small selection was challenging and the choices may never be perfect, but they should suggest a broad range of evidence-based approaches for future intervention development and testing on the diverse European continent. HL interventions that work elsewhere may not be able to deliver similar results among European populations. The interventions that we have prioritized are those with (some) evidence of efficacy, targeted at vulnerable groups with health conditions important to the European older adult population.Unfortunately, health literacy interventions tend to lack rigorous study design , 62 and


Early interventions for psychological consequences of traumatic events should be based on the best available evidence while they are delivered in a great variety of contexts Litz, . AdditioThe panelists discussed diverse aspects related to evidence and practice of early interventions. Biological aspects include developing research on prevention by means of pharmacologically reducing hyperarousal and increasing the likelihood of pro-social behavior. The evidence, benefits, and drawbacks of very early psychological structured interventions were highlighted, including the use of new technology as stand-alone interventions or supplementary modules Olff, . The panThe national societies for psychotraumatology may serve as a link to local and regional initiatives for improving psychosocial interventions to international societies. For example, they can provide nationally adapted information about developments in science and practice, and, importantly, facilitate communication and interaction between local researchers and clinicians across countries.This conference was funded by a grant from The Swedish Foundation for Humanities and Social Sciences (F14-1747:1).


Evidence-informed policymaking has been promoted as a means of ensuring better outcomes. However, what counts as evidence in policymaking lies within a spectrum of expert knowledge and scientifically generated information. Since not all forms of evidence share an equal validity or weighting for policymakers, it is important to understand the key factors that influence their preferences for different types of evidence in policy and strategy development.A retrospective study was carried out at the national level in Nigeria using a case-study approach to examine the Nigerian Integrated Maternal Newborn and Child Health (IMNCH) strategy. Two frameworks were used for conceptualization and data analysis, namely (1) to analyse the role of evidence in policymaking and (2) the policy triangle. They were used to explore the key contextual and participatory influences on choice of evidence in developing the IMNCH strategy. Data was collected through review of relevant national documents and in-depth interviews of purposively selected key policy and strategic decision makers. Thematic analysis was applied to generate information from collected data.The breadth of evidence used was wide, ranging from expert opinions to systematic reviews. The choice of different types of evidence was found to overlap across actor categories. Key influences over actors’ choice of evidence were: (1) perceived robustness of evidence – comprehensive, representative, recent, scientifically sound; (2) roles in evidence process, i.e. their degree and level of participation in evidence generation and dissemination, with regards to their role in the policy process; and (3) contextual factors such as global agenda and influence, timeline for strategy development, availability of resources for evidence generation, and lessons learnt from previous unsuccessful policies/plans.Actors’ preferences for different types of evidence for policy are influenced not only by the characteristics of evidence itself, but on actors’ roles in the evidence process, their power to influence the policy, and the context in which evidence is used.The online version of this article (doi:10.1186/s12961-016-0098-z) contains supplementary material, which is available to authorized users. It is indeed recognized that policymaking is a complex process  and healEvidence has the potential to influence policy processes at different stages and different types of evidence might be needed for different stages of policy development . At the The emphasis placed on evidence-informed policymaking is as a result of the understanding that rational analysis and use of available evidence better informs policy decisions. Evidence is needed to understand the policy environment, appraise the likely effects of policy changes to enable choice, and demonstrate linkages between strategic direction, intended outcomes and policy objectives . Using eWhat counts as evidence in policymaking is contentious and has been described to include expert knowledge, published research and output from economic and statistical modelling, outcomes from stakeholder consultations, and previous policy evaluations, among other . DrawingThe aim of this study was to understand factors that influenced the use of evidence in the development of the Nigerian Integrated Maternal Newborn and Child Health (IMNCH) strategy. We examined actors’ roles in developing the IMNCH strategy, their perceptions of evidence and its use in developing the strategy and the key contextual influences on their preferences for and use of evidence in developing the strategy. This paper contributes to existing knowledge on the implications of context and actors’ roles in policy development on their degree of support for and use of evidence in policy development.Nigeria contributes about 10% of the global total of under-5 and maternal mortality, the second largest in the world . About 2The Global Partnership for Maternal, Newborn and Child Health (PMNCH) was formed with the specific aim to support the scale-up of high-impact low-cost interventions in six countries, including Nigeria. Following a Partnership Grant from PMNCH in 2006, the Federal Ministry of Health was tasked with the responsibility of coordinating actions and partners in accelerating the reduction in maternal, newborn and child mortality , and in The overall objective of the IMNCH strategy is to reduce maternal, newborn and child morbidity and mortality, in line with MDGs 4 and 5, through improved access to quality services, adequate funding and management of MNCH services, strengthened community participation and monitoring and evaluation systems, and sustained partnerships for implementation. The strategy is based on the principles of continuum of care and a seamless linkage between family, community and health facilities, coherent integration of priority interventions into the health system, rights-based planning, equitable access, multisector collaboration, and effective partnerships for leveraging on resources and avoiding wastage and duplication.The conceptual framework for the study was adapted from the EVAL-health conceptual framework for assessing the role of evidence in policy development  and the The interplay between evidence and policy processes influences the role of evidence in policymaking; and policy actors, through the roles they play in both processes, and their interests or agenda, determine this interplay. All of these occur within a wider context of national, international and global influences. The degree to which evidence informs policy is initiated and facilitated by policy actors (communities and networks). These varied actors have different roles in evidence and policy processes, and their specific priorities and capacities could influence their perceptions of what makes ‘robust’ evidence. This ultimately affects their preferences for and use of evidence for policy decisions. Actors will also have relationships  between themselves, which can make them more or less powerful within the policy arena and affect their ability to influence policy through evidence use. These networks could also influence their perceptions of robust evidence.The ideology, values, belief system and organisational culture of decision making also characterise policy actors and influence their perception of robust evidence. Actors who influence policies through financing of health services, exercise enough power through their support in determining issues that are legitimate and feasible enough to be taken up on the policy agenda , 19.This study was conducted at the Federal Capital Territory of Nigeria, where all national policies are made. Nigeria is a country in West Africa with a population of approximately 170 million people. The 2013 Nigeria Demographic and Health Survey reported a maternal mortality ratio of 576/100,000 live births, an infant mortality rate of 69/1000 live births and an under-5 mortality rate of 128/1000 live births. The health system is organized into three levels, namely tertiary, secondary and primary, which provide varying degrees of maternal and child health services. The primary level provides normal delivery services, basic emergency obstetric care, immunization services and treatment of common childhood illnesses. Added to these, the secondary level offers comprehensive emergency obstetric care services, management of complications of common childhood illnesses, and in-patient services. Specialized care is provided at tertiary levels of care.This was a retrospective study that covers what happened from September 2006, when development of the IMNCH strategy started, until March 2007, when it was presented to stakeholders for validation. Drawing on the IMNCH strategy development in Nigeria as a case study, we explored the implications of actors’ roles and policy context on their preference for and use of evidence in policy development. The IMNCH strategy was purposively selected for the following reasons: (1) there exists a clear written strategy statement; (2) it is a relatively new strategy document that was developed in 2007 ; (3) it concerns a national priority area of health; and (4) there are available national and international data on maternal and child health.Data was collected using government documents review and in-depth interviews of key respondents to examine (1) the roles of different actors in the development of the IMNCH strategy, (2) actors’ preferences for and use of evidence in the strategy development, and (3) the key positional and contextual influences on their preferences for and use of certain types of evidence in developing the IMNCH strategy.Relevant government documents that were reviewed are the IMNCH strategic plan; Workshop summary on reducing maternal and infant mortality in Nigeria; NSHDP 2009–2015; Communication strategy for Community IMNCH; Report of the Hon. Minister for Health’s presentation on MNCH in Nigeria. Abstraction of relevant data from documents was performed using a template.The selection of key respondents for the in-depth interviews was informed first from initial review of the strategy document, followed by consultation with key stakeholders and sequential referral from respondents during data collection. They were purposively selected to represent key policymaker/influencer groups such as: government officials (5 key informants (KIs)), development partners (2 KIs), academia/professional group (1 KI), health workers (1 KI), and civil society (1 KI). A total of 10 KIs were interviewed, nine through face-to-face interviews and one by telephone interview. Respondents’ preference for and use of evidence was explored by asking them what they understood evidence to mean; what they considered to be robust evidence for policymaking and their reasons for saying so; and what types of evidence they felt were most useful for developing the IMNCH strategy. Their roles in strategy development and evidence process were explored.Digital recordings of interviews were transcribed in English. Transcripts were managed and coded in NVivo 8 using a combined thematic and framework approach. Coded data were organized into tables and charts according to major themes based on the study objectives, and other themes that emerged from the interviews.Ethical approval was obtained from the University of Nigeria Teaching Hospital Ethical Review Board before the commencement of the primary data collection. The conventional ethical considerations for conducting research  were complied with.All stake holders were really involved in the policy process ranging from government where we involved the states, to development partners and civil society organization […] and professional associations like SOGON” – (Public policy elite).Different groups of actors were involved in developing the IMNCH strategy and they played varied roles in both the strategy development and evidence process – generation, dissemination and use. Table In the Federal Ministry of Health, the lead department was Family Health, and then we have the development partners especially the UNH4, and professional associations like SOGON” – ; “Paediatric association of Nigeria was involved; National Association of Nurses and Midwives were involved” – (Civil Society Organisation (CSO)). Some category of actors, such as healthcare providers and academia, were not specifically mentioned. However, they are members of professional associations and often constitute the technical working groups in policy development, and in that capacity contribute in developing policies and strategies.The public policy elite category includes high level government officers in the Ministry of Health like the Permanent Secretary, unit and department heads; political office holders like the Minister for Health and members of board of directors in the Ministry of Health; and National Council on Health and Federal Executive Committee. The development partner category includes the three UN agencies involved in maternal and child health in Nigeria – WHO, UNICEF, UNFPA, which advise and inform policymakers on options, their alternatives and the resultant effects. The non-governmental and civil society groups includes not-for-profit and for-profit organisations and corporations who have a special interest in maternal and child health issues; they often represent public interest. The professional group category is a collection of professional groups involved in maternal and child health (MCH) such as the Society of Gynaecologists and Obstetricians of Nigeria (SOGON), the Conference of Paediatricians Association of Nigeria, and the National Association of Nurses and Midwives: “The idea came from the Federal Ministry of Health and was spearheaded by them” (Policy elite); “most of what we did was funded by partners” ; “The partnership for MNCH had mobilized a lot of resources also and we could not have done it without that” (Policy elite). Another group of policy elites, the Federal Executive Council and the National Council on Health, who may not have largely participated in the strategy process, appeared to play a determinant role on whether the strategy would see the light of day. Without their approval and endorsement, there would not have been an IMNCH strategy. Their influence on policies/strategies developed in the country is very high given the role that they play.An analysis of the influence of actors who were involved in developing the IMNCH strategy in both strategy development and evidence processes as perceived by other actors is summarized in Table whether the international community came and influenced what was happening in Nigeria, nobody wants to say that, because the extent which they could influence was based on the funding they could provide. I want to tell you that the federal government, on its own, did not launch that program, the fund was from international community” .The level of influence of the international community in the policy process was perceived by representatives of CSOs and professional groups to be very high: “people were asked to say their views about how things were going and those things were never taken into account” .Respondents’ perceptions of level of involvement and degree of participation of actors in the policy process differed. While some government officials and policymakers felt that all groups of actors participated fully in the policy development, representatives of professional/health worker groups felt their involvement was limited to the strategy roll-out stage, and with superficial participation at that. There was a feeling that the inputs made by the CSOs and professional groups were not considered or taken into account: “[….] Lancet survival series, (2) survey reports and (3) situation analysis, appeared to be the recurring evidences in most interviews. Following a stakeholder sensitization on the need for the IMNCH strategy, a technical working group was formed which held a series of meetings to scope for and identify what evidence was in existence in relation to maternal and child health. With financial assistance from development partners, the evidences gathered were disseminated to policymakers.Various types of locally and internationally generated evidence were said to be used in developing the IMNCH strategy. The evidence broadly includes published articles, lessons from international experience, epidemiological reports, stakeholder consultations, existing survey reports, situation analysis and existing policy documents. Although different groups of respondents acknowledged specific types of evidence, three types of evidence, namely (1) published research articles, particularly the Evidence is a documentation of something that has proven effectiveness. [….] if you are writing on a national document then you use data that is representative of the whole country” (Policymaker)“Because they [Lancet] were publishing, they had a comprehensive document. [….] it is important that when you work that you have the current information on whatever issue you are working on.” “Evidence comes from properly designed randomized control trials with appropriate conceptualization and allocation”“Different groups of actors had varied opinions of what constitutes robust and appropriate evidence for policy. While the policy elites and government officials felt that documentation, comprehensiveness, representativeness and proven effectiveness were enough to confer robustness on available evidence, academics and professionals would only characterize such information as robust evidence if it was obtained through rigorous scientific research/work. Development partners on the other hand would describe evidence as robust if in addition to being of proven effect, it provides clear guidance to decision making. Supporting quotes are:These actors’ perceptions of what constitute robust evidence can be seen to reflect on the types of evidence they found useful for developing the IMNCH strategy ; and their roles in funding the generation and dissemination of certain types of evidence could count as the basis of their perception of those types of evidence as useful for developing the IMNCH strategy: “A robust Situation Analysis was supported by partners. We support generation of DHS” (Development partner). There was the perception that development partners are mostly driven by their own agenda or self-set goals, which they would push for regardless of its alignment with the country’s felt need: “Often times the partners will have their own self set goals that they want to achieve” (Policy elite). Duplication of efforts with resultant neglect of certain areas of health need was stated as the consequence of having different development partners working in parallel without a well thought out plan for partner coordination by the government: “So often times you see duplication of efforts, […] while some other areas are neglected. So if there could be a synergy where the partners will be involved completely with the government, [….] it will improve our system” (Policy elite).The development partners played a prominent role in the generation and dissemination of types of evidence that were used in developing the IMNCH strategy, “It is the core working group that scopes for evidence and then synthesizes them […] and identifies the ones to be selected” (Policy elite). This group of experts is comprised of academics and representatives of professional groups who provide technical advice to policy elites on policy direction and content. Their bias for scientific rigor contributes significantly to what types of evidence get used in decision making, as typified by the IMNCH strategy, where rigorous research articles (systematic reviews) were used.The technical working group  appeared to have driven the process of identifying and synthesizing relevant information: “The government officials and policy elites are mandated to use available evidences in developing the strategy. Their role (or the limits of it) in evidence generation and dissemination could explain their perception of all the types of evidence as useful. Their degree of perception of usefulness, however, can be seen to vary depending on the type of evidence, and this could be attributed to the ease of access of evidence types.National and international contextual factors influenced actors’ preferences for and use of evidence in developing the IMNCH strategy. Table The Nigerian health system operates a federal structure with the three levels of operation – national, state and local government – having autonomy to make decisions and implement plans. However, health policymaking is a prerogative of the national level and most state policies in health are adapted from those made at the national level. The political transition to democracy, accompanied by an improvement in the budgetary allocation to health, provided a window of opportunity to scale-up high-impact interventions in maternal and child health . The strWe need to keep abreast of new guidelines, new ways of providing support to maternal and child health; we very much align with WHO, the benchmark” (CSO).The high maternal and child mortality rates, despite numerous interventions in the country, pointed to the need for a change in strategy and approach. The country’s slow progress towards attaining the MDG targets for maternal and child health and a need for its activities to align with the global movement, were perceived as major contextual influences on the strategy development: “The choice of interventions to be included in the strategy was influenced by the challenges of limited human, financial and infrastructural resources in the country, and the availability of evidence of low-cost high-impact interventions that were promoted by development partners. Most of the evidence-based interventions identified at the global level were, according to the strategy document, already being implemented in Nigeria. The major problem, however, was that their coverage was low. The marginal budgeting for bottlenecks approach was promoted by the development partners and was used to systematically identify health system constraints to MCH and operational strategies for overcoming them. This marginal budgeting for bottlenecks tool development was funded by the development partners.we need to factor the fact that stakeholders have other engagements, so enough time …. will also help partners to articulate their own inputs” (CSO).National level policymakers, on the other hand, were tasked with the responsibility of producing a comprehensive plan of action for the recently developed Child Health Policy, but also a strategy which in addition to that will address the high maternal mortality indices. They were therefore open to any type of evidence that could support the policy and its implementation. However, they were constrained by the limited timeline to produce a strategy and this appears to have resulted in limited actor engagement and information/evidence gathering. As stated by a stakeholder: “The breadth of evidence used in developing the IMNCH strategy was wide and could be linked to the involvement of various groups of actors at different stages of the policy development. Although most evidence-to-policy frameworks have focused on bridging the gap between researchers and policymakers , 21, it Different actors played different roles in developing the IMNCH strategy as well as in generating, disseminating or using evidence for strategy development. Although there is an overlap in the categories of actors involved in the evidence process and strategy development, some people played more prominent roles than others. In Nigeria, like in most other countries, public policymaking is the mandate of elected or appointed government officials who are influenced by other actors with diverse interests . EvidencThe capacity of these different actors to influence policy is directly linked to the type and amount of influence they possess , 28 and Lancet series, were of the opinion that this was very useful for developing the IMNCH strategy, whereas government officials leaned more towards reports of expert consultations, which they organized. Lehman and Gilson [The degree of actors’ support for different types of evidence for policymaking overlaps across actor categories, and this was largely influenced by their roles in evidence generation and/or dissemination and their participation in policy development. Although characteristics of evidence such as representation, completeness, availability and timeliness influenced respondents’ perception of best evidence for policy development, their roles in generating and disseminating evidence determined what types of evidence they preferred as useful for making the IMNCH strategy. For instance, technical experts and development partners who played key roles in the generation and dissemination of evidence of high-impact low-cost interventions on MCH in the d Gilson , in theid Gilson . In addid Gilson .Other contextual factors, such as maternal and child mortality rates, resource availability, coverage of MCH services, existing related health policy, global targets and time to develop policy affected actors’ decisions on evidence use for developing the IMNCH strategy. The high rates of maternal and child mortality in Nigeria informed actors’ preference for the Demographic Health Survey that reported these rates. The inadequacy of human resources for health and limited government expenditure on health in the country led to the decision to include evidence of low-cost high-impact interventions in MCH that have been applied in similar contexts. The strategy document was developed just before the MDGs mid-term review when it became clear that Nigeria was making very slow progress and was unlikely to meet the targets of MDGs 4 and 5 and time was of essence. This underpinned the decision to use already existing evidence, rather than implement and evaluate pilot interventions, as well as the decision to use evidence of low-cost high-impact interventions. Contextual factors present policymakers with options of how much (breadth and depth) and how well evidence can be used to make policies . It alsoThe scope of this study only allows us to make inferences about the contributions of the range of actors to evidence use in policymaking. It would be useful to explore the extent to which actors’ contributions of evidence informed policy development by also examining policy implementation; this would also allow us to objectively assess whether the evidence used was helpful for achieving the goal of the strategy.Contributors of evidence who also have the power to influence the decision-making process are better positioned to determine the types of evidence that eventually get used in policy and strategy development. The value in having different groups of actors contribute to evidence use in policymaking is that different types of useful evidence are identified, and policymakers have as much information as is possible for decision making. Government officials, who have the mandate to make policies within a range of contexts, are also expected to involve and coordinate as many stakeholders as possible in the process. They must, therefore, pay attention to the fact that actors are driven by contextual factors in their local realities, interests and experiences, even when it concerns evidence-informed policymaking, and all of these peculiarities need to be managed.


Evidence-informed decisions can strengthen health systems, improve health, and reduce health inequities. Despite the Beijing, Montreux, and Bamako calls for action, literature shows that research evidence is underemployed in policymaking, especially in the East Mediterranean region (EMR). Selecting the draft nursing practice law as a case study, this policy analysis exercise aims at generating in-depth insights on the public policymaking process, identifying the factors that influence policymaking and assessing to what extent evidence is used in this process.This study utilized a qualitative research design using a case study approach and was conducted in two phases: data collection and analysis, and validation. In the first phase, data was collected through key informant interviews that covered 17 stakeholders. In the second phase, a panel discussion was organized to validate the findings, identify any gaps, and gain insights and feedback of the panelists. Thematic analysis was conducted and guided by the Walt & Gilson’s “Policy Triangle Framework” as themes were categorized into content, actors, process, and context.Findings shed light on the complex nature of health policymaking and the unstructured approach of decision making. This study uncovered the barriers that hindered the progress of the draft nursing law and the main barriers against the use of evidence in policymaking. Findings also uncovered the risk involved in the use of international recommendations without the involvement of stakeholders and without accounting for contextual factors and implementation barriers. Findings were interpreted within the context of the Lebanese political environment and the power play between stakeholders, taking into account equity considerations.This policy analysis exercise presents findings that are helpful for policymakers and all other stakeholders and can feed into revising the draft nursing law to reach an effective alternative that is feasible in Lebanon. Our findings are relevant in local and regional context as policymakers and other stakeholders can benefit from this experience when drafting laws and at the global context, as international organizations can consider this case study when developing global guidance and recommendations. Evidence-informed decisions can strengthen health systems, improve health, and reduce health inequities . The BeiIn line with the World Health Organization (WHO) recommendations, a recent priority setting exercise in the EMR revealed that one of the key health systems priority concerns is related to shortages in human resources for health (HRH) particularly in the nursing workforce , 8. NursThe WHO, the International Council of Nurses (ICN), and the EMR Advisory Panel on Nursing have outlined recommendations for global nursing educational standards and competencies to standardize and enhance the quality of nursing care . HoweverTo gain insight into the policymaking process, including the use of evidence, we have selected the draft nursing practice law in Lebanon as a case study for a policy analysis exercise. The draft nursing law was selected due to its national and regional importance and relevance, its recognition as a policy challenge, and the presence of an opportunity for change –16. ThisThe Lebanese political system is a parliamentary democratic system consisting of three powers: legislative power (represented by the Parliament elected from the people), executive power, and judicial power. Public policies can take the form of laws stemming from the legislative or decrees that implement the laws developed by executive authorities . The polThe Lebanese healthcare system is characterized as pluralistic and fragmented due to the heavy involvement of the private sector in the delivery and financing of care . The Minth lowest nursing density in the EMR , our efforts were successful due to circumstances: his advisor was married to a nurse… I went to him directly and talked to him… and finally the advisor convinced the minister.” – A Researcher“In addition, it was mentioned how personal interests influenced policy making process such as favoritism. …in Lebanon everything is subject to political and sectarian issues and no one thinks of the health and safety of the citizens and of the harm this can bring to them.” – A Policymaker“Some members of parliament (MP)s agreed on it in the joint committees but those same MPs refused it when it arrived to the general board. They refused because of the favoritism and personal interests.” – A Policymaker“Sectarianism, political interests and pressures were also considered to impact policymaking and compromise the transparency of the decision making process. Though sectarianism did not appear to play a key role in the unfolding of events regarding this particular draft law, participants did mention that some stakeholders , were mostly predominated by specific sects (like Shiite sect), which added another layer of political pressure.You can always blame politics but I’m not sure it’s the right reason. People don’t know how slow the process is.” – A Stakeholder“Participants explained how economic interests translated into political positions as a result of underlying vested interests policymakers had regarding this law. For example, some participants felt that one of the reasons for the opposition to this law is that some hospitals are owned by politicians and influential people, and many vocational schools are owned by religious associations with their own agendas. Neither group would be willing to pass a law that would require them to relinquish their influence or financial gain. However, few participants showed no concern for political problems in the draft nursing law. They indicated that even though there may be issues among political parties, it is more of a financial problem and less of a political one. Another aspect of the formulation of the draft nursing practice law that could have played a role in delaying it was the balance of power and authority among physicians and nurses. Literature indicates that in most Arab countries the medical profession is dominated by males, whereas women dominate the nursing and midwifery professions, which are perceived as “female oriented” and require little or no education, thus falsely creating the poor image of nurses in Lebanon and the region . This geWith respect to this draft nursing law, the position of the participants in this study ranged from support to opposition Figure . As suchThe argument for the draft law revolved mainly around three factors: positive impact on nursing profession, enhanced quality of care at the hospital level, and feasibility of switching to the BS degree only.Regarding impact on nursing profession, supporters expressed that upgrading the nursing degree to university level would give more value to the nursing profession, which is a main contributor to patients’ health and wellbeing. This would then enhance the image of the nursing profession in Lebanon and lower the rate of migration of nurses thus decreasing the nursing shortage. Supporters criticized the current nursing education system in Lebanon by citing discrepancies in competencies between vocational and university education and ineffectiveness of government examinations in technical schools. All this in their opinion made it necessary to standardize competencies and evaluations. Supporters also claimed that upgrading nursing degrees would lead to improvement in patient health by having better educated nurses and higher quality of services.We have a shortage of nurses and this new law will increase it. No one will go to a vocational school to become an ‘assistant’ nurse.” – A Stakeholder“On the contrary, opponents expected the nursing shortage to be aggravated due to increased migration rates. Lebanese BS level nurses are sought after by recruitment agencies, and nearly 65% of migrant nurses have a BS degree . OpponenThe study that is needed is the one that informs us on how to increase the number of nurses (how to overcome the shortage) regardless of converting TS to BS. When we have enough BS we can abolish TS and convert them but we are not at this stage yet.” – A Policymaker“Other participants felt that this draft law was not addressing the real problem, which is the nursing shortage, and that there is a need for more policy-relevant research on the nursing shortage. Other options according to the opponents included, enhancing the curricula of technical programs, training vocational students, and establishing accreditation of universities and technical institutions to achieve standardization of nursing education.When it came to the implementation of this draft law, some supporters felt that it was feasible through creating bridging programs between the vocational (TS) degree and the BS degree as a transition phase, and the reliance on the Lebanese University (free public university) as a venue where educational programs are offered in addition to offering financial aid to students enrolled in private universities.…technical institutes shouldn’t be closed but instead the quality of their education should be enhanced since the presence of these technical institutes is crucial especially in rural areas where there are no universities and where there is a great need for nurses.” – A Researcher“However, opponents mentioned several implementation barriers like the unaffordability of the education in private institutions, the poor access of students in rural areas to the Lebanese University, and the weak capacity of this university to host enough nursing students to respond to market needs. As such, they predicted that the draft law would decrease enrollment rates and exacerbate the nursing shortage. Meanwhile, technical institutes are available in many areas in Lebanon and provide affordable and accessible nursing education. Regarding cost considerations, supporters believed that higher education would benefit hospitals by helping them conform to accreditation standards. Better healthcare quality was indicated by some to improve efficiency, enhance patient outcomes, and reduce costs. The syndicate of hospitals countered this argument by claiming that they could not afford to increase the salaries of nurses to the levels required by higher degrees due to the current financial difficulties already observed in trying to pay salaries today. Even though enhancing levels of education would enhance quality of care, thus increasing wages and the cost on patients as well.Supporters criticized the opponents by claiming that most technical institutes were open for political reasons and are not needs-based, and that their opposition to the draft law was rooted in financial reasons, as they did not want their degrees abolished and to suffer the resultant loss in revenue.This section presents the process of the development of the draft nursing practice law and findings are categorized into subthemes including: problem identification, formulation, negotiation, and use of evidence in policy.The draft nursing law was presented to the parliament in 2000 and has been pending for around 13 years. Work on the draft nursing law restarted with the establishment of the ONL in 2002, which was established in response to the need for nurses to have an organized body that enhanced their image and gave them a voice. Hence, the ONL decided to take the lead on re-examining the regulations governing the nursing practice in Lebanon.Findings revealed that the draft nursing practice law in Lebanon was prompted by the ICN international standards and WHO recommendations concerning organizing the nursing profession. The shortage of nurses in Lebanon and the perceived poor quality of nursing care also played a role in recognizing this issue as a problem and initiating the draft nursing law. One factor that helped get this draft law on the policymaker’s agenda was the fact that the minister of health’s wife was a nurse and the ONL was able to persuade him to champion this draft nursing law.One of the participants involved in the development phase of the draft nursing law mentioned that the ONL invited representatives of schools of nursing of the major universities in Lebanon to help in preparing the draft Nursing practice Law. A WHO representative was also present to make sure that the new draft law was in line with and up to the WHO standards of care. Different groups within the ONL were established to work on the competencies, code of ethics and bridging program in the draft nursing law. With respect to the role of research in the development of the content of this draft nursing law, participants cited sources including international reports  and information from the ONL. Despite the fact that many said there was poor use of evidence, few participants insisted that this draft nursing law was evidence based, particularly since it was initiated by professionals who understand the importance of research in policymaking processes.We have data on the number of nurses graduated but we don’t have information and studies on their career path and their performance” – A Policymaker“However, local evidence was generally limited to numbers and basic statistical information, and some participants indicated particular types of data that are missing. Some participants mentioned utilizing personal contacts midst the absence of reliable sources for obtaining the needed information while questioning its validity.Even though rigorous local research was limited, some stated that international reports should not be considered as sufficient evidence as they were not specific to the context of Lebanon. Opponents of this draft nursing law believed that, although international standards for nursing are important, examining this profession within the local context should be given equal consideration.After the final draft was prepared, it was submitted to the MOPH for revision and approval. A meeting was held in the MOPH where key pertinent stakeholders were invited to get detailed information and explanations about the draft law and consequently provide their feedback. The majority of the feedback was positive, but representatives of the Order of Private Hospitals and the Vocational and Technical Education expressed various reservations concerning the law. In this meeting, implementation barriers were voiced but not addressed. With respect to the circulation of documents and information pertaining to the draft, some participants indicated that it was not passed to all those concerned.There was no evidence presented by the Order of Nurses or Ministry of Health. They only replied verbally to what was presented by the directorate of the technical and vocational education” – A Policymaker“The dearth of research evidence to support the draft nursing law was mentioned as one of the reasons the committee of justice stopped working on it. These conflicts continued on to be reflected in the meetings of the parliamentary committees, and eventually paralyzed their work and halted the progress of this law.It depends on a window of opportunity.” – A Researcher“Here, politics impedes the use of science” – A Policymaker“As for the role of research in informing policymaking in Lebanon, some participants said that evidence has a minimal influence due to the overwhelming power of personal and political interest. We cannot keep on taking decisions haphazardly.” – A Stakeholder“Many participants considered that policy-relevant research in Lebanon is needed in addition to dialogue between policymakers and stakeholders to deliberate about problems and potential solutions. Other factors that hinder the use of evidence as mentioned by the participants include the high turnover rate of ministers and policymakers which yields poor commitment to long term strategic planning, and limited research funding by the government, leading to dependence on international financial support. Figure Study findings shed light on the complexity of the policymaking process and its influencing factors. This policy analysis exercise revealed that successful policy development should take into consideration implementation barriers in the formulation stage. Findings show that the formulation of the draft nursing practice law was hindered by a lack of clarity on the problem, its underlying factors. However, addressing a health systems and policy problem requires working through the underlying problem, options for actions, and implementation issues . The uncThe study also revealed an absence of a structured decision making approach that utilizes research evidence. Interestingly, despite the availability of some local studies on nurses in Lebanon that could have informed the development of this draft, they were not used in policy development and were disregarded. This implies weakness in translation of knowledge and absence of effective communication between researchers and policymakers. It was observed that policymakers rarely access evidence generated from academic institutions, research centers, or think tanks to address their knowledge needs. Findings suggest the need to establish institutional linkages between policymakers and researchers. Findings also showed that the barriers against the use of evidence in policymaking in Lebanon include political influence, personal interests, lack of research funding, and poor commitment to long-term strategic planning.Several barriers impeded the progress of the proposed draft nursing law including financial hardships of hospitals, disparity of access of students to nursing programs, and authority gradient in the medical field, in addition to the characteristics of the policy environment . This led to the resistance of various stakeholders involved in this draft nursing law. Stakeholders were given an opportunity to weigh in and voice their concerns but the implementation barriers that they suggested  were never addressed.Regarding the stakeholders, each had their own sources of power and influence. The main supporters of the nursing practice law were the ONL, MOPH, the Ministry of Education, and the Order of Physicians, who formed a network of actors representing the “State” and medical associations. The power of these groups within the network stemmed from tangible and intangible political resources . TangiblAs for opponents, the major players were the Order of Private Hospitals and the technical education institutes. As such, this group represented the non-state, private sector and in particular “economic groups” which were industries affected by the health policy . The tanIn addition, the particularities of the Lebanese political system played a role in delaying this draft law. In Lebanon, a law cannot pass until the majority of the parliament, or members of the parliamentary committee vote for it. Members of Parliament and committee members represent majorities and minorities in the Lebanese political scene. This dispersal of power and mode of decision making gave more attention to smaller interest groups than in legislatures with winner-take-all single member constituencies . AccordiMoreover, equity played a major role in the implementation considerations of this draft law. Although equitable and sustainable development had long been a goal of the international and local community, little has trickled down into actions. Disparity in quality of education between nursing schools, disparity in performance of nursing levels, disparity of working conditions across regions and nursing levels, and disparity in geographical and financial access to nursing education are only a few equity problems plaguing the nursing workforce in Lebanon. As such, although equity is well recognized as a focal point for action, it is not being implemented, which highlights the role of other factors specific to Lebanon such as favoritism, sectarianism, and corruption in directing decision making.This case study also demonstrated the difficulty in the use of global evidence and international recommendations in a local setting without accounting for local applicability. In the development of the draft nursing law, recommendations from international reports were applied as is, with little or no regard to contextual factors and implementations considerations. This resulted in major conflict, delay, and eventual failure to pass the draft nursing practice law. However, health systems guidance should assist decision making , not repPolicy making processes and their influence on health care reform and practice – specifically nursing practice – in Lebanon and the EMR are severely understudied. However, this study confirms what has been revealed by other international and regional research on the barriers to policy making. For example, the review by Innvaer et al.  reportedOur study has five main strengths. First, to our knowledge, it is one of the very first country case studies conducted in the EMR to closely investigate the policymaking process of a policy that has not yet been ratified. Second, the policy triangle framework, in combination with the stakeholder analysis framework that was used for analysis, helped build a comprehensive understanding of the draft nursing law by identifying its content and objectives, the actors involved in this draft nursing law, the process of policy initiation and formulation, and the context within which the draft law was developed. Third, we interviewed all the key people who were involved in the development of the draft nursing law. This is particularly important in light of the limited documentation. Fourth, the data analysis was conducted through triangulation of data by having two independent reviewers (RH and NA). Fifth, following the interviews, a panel discussion was conducted to validate the results and cross-check the information thereby enhancing the credibility of the study.Several limitations are also acknowledged, the first of which is temporality and recall bias. Participants might not have been able to correctly recall the events that occurred during the formulation and communication phase of the draft nursing law which spanned over a decade. Some of the information as to the history and process of the draft nursing law was contradicting among several participants, creating difficulties in determining the most accurate and precise information. However, the panel discussion helped in the validation of the process and the accuracy of the information collected. Second, dealing with such a sensitive issue was challenging and effectively engaging the participants was difficult. However, to overcome this obstacle, the team carefully explained to the participants the importance of the case study/research, the neutrality of the researchers, the absence of pre-conceived judgments, and the confidentiality in the way the data was to be dealt with. Further, the research team heavily engaged the ONL in order to garner their support and to ensure that the results of the study would be picked up and fed into the next policy cycle.A third limitation was the positionality of the researchers. The lead researcher was heavily involved within this field  and had contributed substantially to the literature on this topic as evident by the relatively large number of references attributed to him. This meant the researcher was an insider in the policy environment which could have biased the research . HoweverFourth, another limitation may have been the agenda of the researchers and their focus on the role of evidence in policymaking as a key tenet in the analysis, which could have biased the interpretation of the data. However, the choice of research question and design of the study was not made based on the researchers’ personal agenda, but in response to policymakers, international calls , and well-acknowledged gaps in knowledge translation and health policy analysis –6.Fifth, as for the application of the two policy frameworks in this policy analysis case study, which is a first in the region, they provided guidance as to how to breakdown the policy development process, identify relevant contextual factors and analyze the stakeholders regarding distribution of power and influence, and problem framing. However, these frameworks  come from the developed/western countries and have not been adapted to suit the political environment in the EMR. As such, interpreting our findings within the dimensions of these frameworks required a degree of adaptation and contextualization. Further, this particular policy analysis deals with a policy that was never passed but stagnated at the phase of development and, as such, the implementation phase in process category was not addressed.Sixth, from the time of the initial drafting of this case study there might be additional updates regarding the policymaking process. Thus, the information in this study is limited to the date of the end of data collection, which is June 2013.Study findings suggest that there are certain steps to be taken before moving on to changing the law governing nursing practice. These steps include standardizing the nursing education programs  and examinations to reflect the competencies required and conducting studies on the best context-specific alternative to the current nursing care model. Accordingly, this study triggers multiple research questions that need to be answered before reaching a best-fit policy change at the level of the nursing education and practice: i) What is the current and projected demand and supply for HRH and nursing workforce in Lebanon? ii) How does the quality of care and performance differ between different nursing levels? iii) What are the different nursing care delivery models followed within the Lebanese hospitals? And what are the educational competencies needed for every model? iv) What would be the optimal nursing care model that can raise the quality of nursing care within the context and resources available while decreasing the nursing shortage? v) What would a collaborative nursing education system look like in Lebanon and how can it be regulated and governed?Further, it is worth noting that the draft nursing practice law only addressed one aspect of the nursing profession, namely education and competencies, without touching upon other issues. For example, despite the presence of local evidence on the issue, the draft nursing law did not tackle the working conditions of nurses. This indicates the need for a comprehensive nursing law that governs the profession.Despite the stagnation of the draft law, a recent decision was taken to reduce the years of study of the higher vocational degree (TS) from 3 to 2 years. This caused problems in providing nursing practice licenses and currently the ONL is trying to mitigate this issue using temporary solutions through providing a proposal to the Ministry of Justice. This decision has only increased the discrepancies in the nursing workforce and makes it even more critical to work on advancing a comprehensive draft law.In addition to grounding the draft law with evidence and answering information gaps, the formulation and negotiation strategies of the law should be changed. The law, its purpose, and implications should be drafted and framed during negotiations in “win-win” terms rather than “win-lose” terms . MoreoveFindings revealed the risk involved in relying blindly on international recommendations and using them without accounting for implementation barriers, contextual factors, and local applicability. When selecting a policy option, an assessment of the key features of a health system that can influence decision making, including governance, financial and delivery arrangements should be conducted . One wayThe significance of the use of evidence in policymaking should be communicated to the public through raising awareness, dissemination of research results, and the media. Public awareness would enhance accountability of policymakers and encourage evidence-informed policymaking. Moreover, the gaps in knowledge revealed by this study, and which hindered informed decision making, suggest the need for better communication between researchers and policymakers.Findings shed light on the complex nature of health policymaking, its influencing factors, and the unstructured approach of decision making. This policy analysis case study revealed the barriers to the development and adoption of the draft nursing practice law and to the use of evidence in policymaking. Findings also uncovered the risk involved in the use of international recommendations without the involvement of stakeholders and without accounting for contextual factors, implementation barriers, and local applicability. This exercise presents findings that are useful to legislative bodies and all other stakeholders for strengthening and revising the existing draft nursing law in order to develop an effective alternative that is applicable in Lebanon. This is particularly important since the ONL, under new leadership, is currently making significant efforts in building its capacity in policymaking, in terms of accessing and using evidence, influencing policymaking and legislations, and promoting evidence-based advocacy. In addition, the ONL is engaging and deliberating with other ministries, including the MOPH, Ministry of Education, and Ministry of Labor, about the nursing workforce and other health policy-related matters. Our findings are relevant in the context of Lebanon and the region as policymakers and other stakeholders can learn from this experience when drafting laws. Findings are also relevant at the global level as international organizations can take this case study into account when developing global guidance and recommendations.


An 11-year-old girl visited her general dentist for a routine check up. Upon clinical andradiographic examination, no signs of dental caries were observed, but the dentist noticedthat her maxillary canines may have been impacted and referred the patient to theorthodontist. After clinical and radiographic examination, the orthodontist concluded thatthe teeth were palatally displaced and needed to make a decision for a course of action.The orthodontist suggested extracting the deciduous canines. The parents were hesitant andwanted to wait for the deciduous teeth to exfoliate and the permanent teeth to erupt. Theorthodontist then decided to look into published literature for the best evidence availablefor the treatment of impacted canines. The orthodontist decided to use the five steps ofEvidence-based practice: Ask, Acquire, Appraise, Apply and Assess . » Ask a focused question: Use the PICOT  acronym to create a focused question. The clinical question was: In an 11-year-oldfemale patient with suspected palatally displaced maxillary canines, does the extraction ofthe deciduous canine, compared to no treatment or watchful waiting, result in successfuleruption of canines after 4 years? » Acquire the evidence: The orthodontist performed a "Narrow scope" search in PubmedClinical QueriesIn the first study, a female patient had deciduous canine extracted and had the permanentcanine successfully erupted. In the second study, orthodontists invited their patients, whoeither would have deciduous canines extracted or did not receive any treatment, toparticipate in the study. After 4 years, they assessed how many patients in each group haderupted permanent canines. In the third study, patients were randomly allocated by theinvestigators to have deciduous canines extracted or to have no treatment at all; patientswere followed up for 4 years. At the end of the study, they reported how many patients ineach group had erupted canines. The first study is a case report; the second, a cohort study; and the third, a randomizedcontrolled trial. The case report does not answer the orthodontist's question because itreports a single patient without comparison with a group of patients who did not receiveany treatment. The cohort study, however, does have a comparison group, but the decision onwhether to extract or not the deciduous canines was based on the clinician's preference andmay be biased if the reasons for providing treatment were based on better prognosis, forinstance. The orthodontist chose the randomized controlled trial in which assignment totreatment and comparison groups was done by a random process; which is less likely to bebiased by personal preferences or favorable prognosis. Then the orthodontist retrieved thefull article.» Appraise the evidence: After reading the full text of the research paper,In addition, the sample size was not calculated for the primary outcome of canine eruption,but for a secondary outcome of positional change of the permanent canine. The orthodontistconcluded that the trial had low risk of bias, with generalizability limited to childrenwith similar clinical characteristics as those in the trial (e.g. no moderate to severecrowding in the maxilla). The results of the study were: 69% and 39% of permanent canines erupted in the extractiongroup and in the control group, respectively. Measures of association were not reported inthe paper, so the orthodontist calculated them using the GATE calculator » Apply the evidence to practice: Before finding the research paper, the orthodontist hadcarefully conducted a clinical and radiographic examination of the patient and communicatedwith the patient and her parents to assess their preferences. After critically appraisingthe best evidence available, the orthodontist needs to assess his clinical experience andpersonal preferences and communicate again with the patient and his/her parents so as tomake a clinical decision. Taking into consideration all the elements of the X factor» Assess actual practice against best evidence-based practice: After treating the patient,the orthodontist considered whether treatment was successful and well accepted by thepatient and her parents. The orthodontist also pondered what lessons were learned thatcould be incorporated into practice so as to make a better decision next time. Byre-evaluating his/her own practice, the orthodontist is constantly reshaping theorthodontic practice in light of new research evidence and increased clinical expertise. Depending on the research question, specific research designs are more appropriate toanswer it. They will have corresponding critical appraisal tools, such as those forintervention studies, diagnostic test accuracy, risk factors and prognostic studies, andsystematic reviews. In the hierarchy of evidence-based practice, randomized controlledtrials are the highest level of evidence, only below systematic reviews of randomizedcontrolled trials. In the present study, we reviewed the appraisal of a singlerandomized controlled trial for therapy using the 5 five steps of evidence-basedpractice and the GATE frame


I read with interest the article “Science Policy, and the Transparency of Values”  and wantFirst, the authors noted that different levels of scientific evidence are necessary depending on the venue in which that evidence is going to be used: High standards of evidence are necessary to infer causal relationships, with weaker standards of evidence required to protect the public from perceived risks. Although I agree that different venues or contexts require different standards of evidence, it is important to emphasize that the actual scientific evidence remains constant. For example, anecdotal evidence is the weakest level of scientific evidence on the hierarchy, regardless of the context in which it is being employed. There might be a legitimate rationale to relax the level of evidence required to inform policy within a given context, but that does not change the nature of the evidence itself. If we rely on “weaker” standards of evidence to protect the public, the basis of the decision and the nature of the evidence relied upon should be made transparent. This is not always the case.Second, the authors combined tort law and chemical regulations as examples in which weaker standards of evidence can suffice to inform policy. I disagree with treating these two venues as equivalent. Rather, tort law should be part of the first venue: inferring causal relationships. After all, the goal of a civil tort is to prove that a chemical causes the alleged injury. Legal tort cases are not intended to be theoretical or precautionary exercises.Third, in the “Discussion” of their paper, Finally, it seems incomplete to assess the role of science in public policy without a discussion of the importance of evaluating risk–benefit relationships. Clearly, society is willing to tolerate health risks from certain chemical exposures when those risks are deemed to be outweighed by the benefits. This risk–benefit assessment is made every time a new drug is considered for approval. In this context, policy makers are willing to tolerate great risk if the benefits of a pharmaceutical agent are deemed to outweigh the risks (and if the appropriate warnings are made). Conversely, policy makers are appropriately unwilling to tolerate health risks when the benefits are minimal or inadequately defined. Characterizing the risk–benefit relationship is critical to setting rational and appropriate public policy. Any discussion of the role of science in this endeavor is inadequately served without discussing these important relationships.I am a scientific consultant who works with commercial manufacturers to help them understand the science related to their products. I have a deep and enduring respect for rigorous scientific inquiry using the best and most appropriate scientific principles. These principles and methodologies are clearly defined and should be carefully and systematically applied before using scientific findings to influence policy e.g., . The nat


Patients and clinicians expect patient decision aids to be based on the best available research evidence. Since 2005, this expectation has translated into a quality dimension of the International Patient Decision Aid Standards.We reviewed the 2005 standards and the available literature on the evidence base of decision aids as well as searched for parallel activities in which evidence is brought to bear to inform clinical decisions. In conducting this work, we noted emerging and research issues that require attention and may inform this quality dimension in the future.This dimension requires patient decision aids to be based on research evidence about the relevant options and the nature and likelihood of their effect on outcomes that matter to patients. The synthesis of evidence should be comprehensive and up-to-date, and the evidence itself subject to critical appraisal. Ethical (informed patient choice), quality-of-care (patient-centered care), and scientific (evidence-based medicine) arguments justify this requirement. Empirical evidence suggests that over two thirds of available decision aids are based on high-quality evidence syntheses. Emerging issues identified include the duties of developers regarding the conduct of systematic reviews, the impact of comparative effectiveness research, their link with guidelines based on the same evidence, and how to present the developers’ confidence in the estimates to the end-users. Systematic application of the GRADE system, common in contemporary practice guideline development, could enhance satisfaction of this dimension.While theoretical and practical issues remained to be addressed, high-quality patient decision aids should adhere to this dimension requiring they be based on comprehensive and up-to-date summaries of critically appraised evidence. The original 2005 International Patient Decision Aid Standards (IPDAS) formulation included a key quality dimension then labeled as “basing information on up-to-date scientific evidence”. This dimension reflected the Standards panel’s belief that these tools should advance evidence-based practice. Since 2005, much work has taken place that advances the use of research evidence to develop tools that support clinical decision making, requiring a review and update in the formulation of this quality dimension. For this iteration, we put forward a more detailed description of this dimension. We now require patient decision aids to base their information on comprehensive, critically appraised, and up-to-date syntheses of the scientific evidence.In this manuscript, we present a detailed description of this quality dimension, the theoretical and empirical justification for evaluating patient decision aids according to this dimension, and discuss the emerging issues and research areas for consideration and discussion identified during this endeavor.IPDAS requires patient decision aids to base their information on comprehensive, critically appraised, and up-to-date syntheses of the scientific evidence. This refers to the information about the various relevant options, and about the descriptions and likelihoods of those options’ effects on the outcomes of most importance to patients.The term “scientific evidence” refers to a body of empirical observations about the options and their consequences. These observations should be made with some degree of protection against systematic and random error.By “comprehensive”, we do not mean the scope and range of the information presented in the decision aid. Instead, in defining this quality dimension, “comprehensive” refers to the extent to which decision aid developers have thoroughly considered all the pertinent scientific evidence addressing each aspect that they chose to present in their decision aid.The requirement that the evidence be “critically appraised” means that decision aid developers will use evidence derived from systematic reviews that: a) avoid selection bias ; b) carefully and reproducibly assess the quality of the incorporated reviewed studies ; c) summarize the estimated pertinent effects ; d) indicate the extent to which these estimates are trustworthy; and e) assess the extent to which selective reporting and publication bias may corrupt the body of evidence. Developers must present this synthesized evidence in the decision aid itself in a way such that it a) conveys that the aid is offering the “best available” synthesized information, and b) indicates—using symbols, numbers, or phrases—the degree of confidence attributed to that information, given the quality of the scientific evidence upon which it is based.The dates when the relevant systematic reviews were searched for, compiled, critically appraised, and synthesized by decision aid developers should be reported and should be sufficiently recent given the pace of progress in the particular field to be considered “up-to-date”. This implies that decision aid developers should develop a sense of the speed with which evidence that matters accrues in their area of work, and implement an update policy. Developers should report a version date, and, when pertinent, a “better by” or “expiration date” to communicate to decision aid users about the speed of evidence accrual in the field and the optimal timing of future updates.As we described above, clinicians and patients expect patient decision aids to be evidence-based tools. This alone could justify this dimension (and indeed it did in its 2005 formulation). Here, we push the rationale further a) by connecting it with three fundamental arguments – an ethical argument, a quality-of-care argument, and a scientific argument – that are central to contemporary clinical care, and b) by describing recent advances in the rating of the degree of confidence in the estimates of effect justified by the state of the available scientific evidence.Three fundamental arguments central to contemporary clinical care justify the IPDAS’s evidence dimension.One key principle driving the development of patient decision aids is the ethical argument for informing patients about their health care choices. Respect for patient autonomy is a governing principle of medical ethics , and is high-quality information and education to be among the important characteristics of patient-centered care [The growing emphasis on patient-centered care is another driving force behind the development of patient decision aids. Principles of patient-centered care require that patients actively participate in decision making and be provided with the information and support they need to make informed choices. Work by the Picker Institute and others has identified respect for the patient’s values and preferences, as well as access to clear, red care . PatientOver the last 20 years, evidence-based medicine (EBM) has strongly influenced the practice of medicine  as it haThe first principle recognizes that not all observations and experiments are similarly protected from random and systematic error. The degree of protection from error offers confidence in the estimates of effect. Therefore, in the interests of fostering EBM, the information presented in patients’ decision aids should include not only evidence-based estimates of the effects of the various relevant options, but also an indication of the extent to which this evidence is protected from error .The second principle holds that the evidence alone is never sufficient to fully inform a clinical decision. The evidence will be considered during the transactions between the treating clinician with a particular level of expertise and the patient with unique goals, values, and preferences; furthermore, the evidence will be applied in a particular biological, psychological, and sociocultural context. Therefore, in the interests of fostering EBM, the information presented in patients’ decision aids needs to be directly applicable to the patients and practitioners using the decision aid, and tailored to individual patients’ characteristics.Ethical (informed patient choice), quality (patient-centered care), and scientific (evidence-based medicine) justifications make it imperative that decision aids be based on comprehensive, critically appraised, and up-to-date scientific evidence.Decision aid developers may want to include information in their tool that is based on different forms of evidence. Natural history and prognostic information, for instance, often requires the developer to use large and long observational studies. Adverse effects, particularly those that are rare, may be better characterized in case reports, and their linkage to exposures ascertained through case-control studies. This notion of a hierarchy of evidence has received much attention and has resulted in an emerging consensus in the practice guidelines movement about how to assess the confidence in the estimates of effect from the body of pertinent evidence: the GRADE approach. While we will now focus on this approach, we recognize that other approaches to grading the evidence exist, but they have substantial shortcomings that this method obviates. Furthermore, consistency in rating evidence from guideline to decision aid may facilitate the development of decision aids in conjunction with clinical practice guidelines efforts.GRADE  is the most comprehensive approach developed for the purposes of formulating clinical practice guidelines. It coulhttp://www.gradeworkinggroup.org).We will summarize the GRADE approach to grading evidence of effectiveness here, but developers should review the extensive published guidance . This includes: a) the likelihood of bias ; b) the Randomized trials often provide high confidence in effect estimates. The task is to identify limitations that would reduce our confidence in those estimates. Observational studies often provide low confidence in effect estimates, limitations further reduce that confidence, but some features, such as strong evidence of a dose-response relationship or evidence of a very large effect , increashttp://decisionaid.ohri.ca/azinvent.php). Out of 257 decision aids included in the inventory, 134 provided references to scientific evidence used, when they were last updated (n = 135), and whether they were available on the Internet (n = 134). In a random sample of those decision aids (n = 20), stratified to correct for differences between providers, ten aids used a high quality systematic review/meta-analysis , such that it is sometimes advisable to use estimates from the general population . The charectness . The degrectness  may enhaThe frequency with which systematic reviews should be updated – and, by extension, the products derived from these reviews – is still subject to research. A comprehensive technical review commissioned by the Agency for Healthcare Research and Quality and published in 2007  found thDecision aids often need to present absolute risk estimates that relate to the individual characteristics of the patient of interest. There is considerable uncertainty about such baseline risks . PrognosCommunicating the confidence that decision aid developers have in the estimates of the benefits and harms associated with the selected options would help decision aid users to make sense of the magnitude and trustworthiness of the estimates. The communication of this confidence in the evidence should be done in a way that is simple and understandable, yet remains precise, without adding cognitive burden on decision aid users.There is limited empirical work on how to communicate confidence in effect estimates to stakeholders in general, let alone decision aid users. A systematic review of information in decision aids reported that very few decision aid developers had addressed or incorporated confidence in effect estimates in their tools . The wayThe GRADE approach for guidelines offers a simple way to report on confidence in effect estimates, using categorical labels—for example, “very low”, “low”, “moderate”, and “high” confidence. ,21. An oA key issue with decision aids is their underuse in practice, despite increasing evidence of their effectiveness and surging policy support in some regions. Thus, efforts to explicitly link their design and content to clinical policy and workflow may facilitate their adoption. This supports in part our approach of linking the development and content of decision aids to state-of-the-art approaches to the development and content of evidence-based practice guidelines.The “ideal” situation indicating the potential usefulness of a patient decision aid is one in which there is evidence producing high-confidence estimates linking options to outcomes, but the options are closely matched, and the choice of the “best” course of action will depend mostly on the patient’s values and preferences. Guideline developers following GRADE usually will offer a weak or conditional suggestion in those circumstances. A suggestion based on high-confidence evidence would require the incorporation of patient values and preferences for implementation, that is, would benefit from a decision aid. This linkage between guidelines and decision aids may affect the development and uptake of decision aids. Furthermore, implementation of the guidelines, for example, in quality improvement efforts, could then be linked to the implementation of decision aids for weak or conditional suggestions.A key aspect worthy of surveillance is the fate of decision aids that have been developed based on the best available evidence, but which disagree in their presentation with extant guidelines and with the quality-of-care parameters derived from these guidelines. These discrepancies can appear because decision aids may use more recently updated summaries of evidence, or because the recommendations apply to a different context or resulted from panels with distorted values . In one example, this apparent divergence led to difficulties in the use of the decision aid and to its nonuse .a priori reason to believe that the quality of the process of evidence synthesis and its adaptation into evidence-based decision aids will lead to better decision processes and outcomes. Yet, basing their content on accurate rather than inaccurate estimates may lead to better outcomes. These central concerns, therefore, remain important research questions.Given that the uptake in practice of decision aids remains limited, there is no strong evidence that decision aids supported by high quality and updated summaries of the body of evidence would be more likely to be taken up in practice. It is plausible, however, that adherence to this domain may create a reputation for high credibility that could drive the uptake of decision aids. There is no A key quality dimension for patient decision aids is how developers identified, summarized, and used research evidence to inform the content of these tools. Patients and clinicians have the expectation that these tools will support evidence-based practice. Ethical, quality-of-care, and scientific arguments justify making this an IPDAS standard. Empirical evidence suggests that there is much room for improvement in developers’ adherence to this standard. Applying the methodological developments of the GRADE working group to this task may help developers enhance their adherence to this domain and thus the quality of their patient decision aids. It may also help position the development of decision aids in the same development stream as that of high quality clinical practice guidelines. Yet, much room exists for research and innovation into the processes that create patient decision aids based on comprehensive and up-to-date syntheses of all the pertinent and critically appraised scientific evidence.AGREE II: Appraisal of Guidelines for Research and Evaluation, second version; AMSTAR: Assessment of Multiple Systematic Reviews; EBM: Evidence-based Medicine; GRADE: Grading of Recommendations Assessment, Development and Evaluation; IPDAS: International Patient Decision Aid Standards.http://shareddecisions.mayoclinic.org).VMM is a member of the IPDAS Steering Committee and of the GRADE Working Group. His research includes the development of patient decision aids. He does not receive financial compensation for any of these activities, and his research group, the KER UNIT, does not receive funding from for-profit pharmaceutical or device manufacturers. Furthermore, the decision aids developed by the KER UNIT are freely available . The Foundation develops content for patient education programs. The Foundation has an arrangement with a for-profit company, Health Dialog, to co-produce these programs. The programs are used as part of the decision support and disease management services Health Dialog provides to consumers through health care organizations and employers.DS receives salary support as Chief Production Officer for the Informed Medical Decisions Foundation, a not-for-profit (501 (c)3) private foundation  as well as discussion and critical revision of the whole manuscript.DS contributed to discussions and critical revisions of the whole manuscript.AT contributed to the search and appraisal of evidence reporting in decision aids, as well as discussion and critical revision of the whole manuscript.VMM is a clinician and Professor of Medicine at Mayo Clinic in the United States, and has over a decade of experience developing and evaluating decision aids for patients with chronic conditions, conducting systematic reviews of healthcare interventions and formulating practice guidelines, and conducting research about evidence-based medicine, patient-centered care, and clinical practice guidelines. He leads the Shared Decision Making National Resource Center at Mayo Clinic.AL is Assistant Professor of Health Services Research at Mayo Clinic, Rochester, MN, USA. She works in the development, evaluation, and implementation of shared decision making interventions, including decision aids, in both acute and chronic care settings.AB is Senior Researcher at the Department of Medical Psychology at the Hamburg University Medical Center and has been involved in the development and evaluation of several shared decision-making interventions in various health contexts for the past four years.DS is Chief Production Officer at the Informed Medical Decisions Foundation, Boston, Massachusetts, USA. Over the past decade, she has been involved in the development, evaluation, and implementation of dozens of decision aids.AT is Assistant Professor of Medicine at Aristotle University, Thessaloniki, Greece, with a special interest in evidence-based medicine, evidence synthesis, and use of decision aids for chronic conditions.