Essential Evidence Plus (EEP) is a point-of-care tool that was created to assist clinicians with evidence-based practice. According to the creators, the database was designed with the work-flow of health care providers in mind in order to facilitate clinical decision making . It contThe editorial board is composed primarily of physicians , and theEEP consists of several databases that can be searched individually or simultaneously by entering the search terms into a single search box. No advanced search option exists, but Boolean operators can be used. When searching for specific terms across all databases, options are available to narrow the search by content or resource. The results can also be sorted by relevance, date, or title. Descriptions of the individual database options follow.This database is organized with listings of broad subject areas; selecting one of these reveals more specific subtopics. After selecting a subtopic, users are then presented with an “Overall Bottom Line,” which is a summary of the latest clinical evidence. Each statement is followed by a level-of-evidence grade, which is also a link. Hovering the cursor over the link provides brief information about the evidence rating that was used in creating the grade. Selecting the link leads to a more detailed description of the grading procedure.There is also more specific information provided on the topic, such as background information, prevention, diagnosis, treatment, and so on. Hovering the cursor over the reference numbers provided after some of the statements reveals the citation of the original study, as well as a link to the article record in PubMed and a Cochrane abstract or relevant Patient-Oriented Evidence that Matters (POEM) research summaries (see below), if available. Some topics include an “Interactive Tools” box, which contains various instruments such as calculators and/or algorithms specific to the topic.Summaries of Cochrane Systematic Reviews can be searched by entering terms in the search box or by navigating a hierarchy of topics and subtopics similar to those used for Essential Evidence Topics. A link to the full review is provided, but it can only be accessed if the institution or individual has a subscription to Cochrane.One unique feature of EEP is a daily POEM research summary. These brief research study descriptions focus on evidence-based outcomes that are important to patients, such as better quality of life and longer life expectancy, as opposed to other outcomes that are considered less important to patients, such as improved lab values. In addition, POEMs suggest a change in practice for issues regularly encountered in clinical settings . For exaAs the title implies, these databases contain evidence-based medicine guidelines and evidence-based summaries, respectively. They can be searched using a search box or browsed by topics and subheadings.This area contains calculators and other tools to aid clinicians in making evidence-based treatment decisions based on individual patient factors .These calculators facilitate the clinician’s ability to determine a patient’s risk for developing a specific medical condition . Links tThese calculators can be used by clinicians to interpret lab results and determine which laboratory tests to order . Links tThis tool provides images of various skin conditions to aid in diagnosis.This resource is designed to facilitate reimbursement from Medicare.EEP enables users to access the original research easily, as well as detailed information regarding the methods used to determine the levels of evidence. These features allow users to review and evaluate the information in EEP for themselves in order to determine if they would like to incorporate it into their practices.According to EEP, “Within 72 hours of new, serious drug safety concerns or drug withdrawals, the Essential Evidence Topics will be up-dated and an e-mail alerting service will notify users of the latest safety information” . These aPhysicians can receive continuing medical education (CME) credits through conducting specific types of searches using EEP and by providing documentation of the searches.EEP can be accessed using recent versions of Firefox, Chrome, Explorer, and Safari. Institutional access can be provided through Internet protocol (IP) authentication. While a mobile app is not available, EEP offers a mobile-friendly interface when users access the site with mobile devices. One limitation of the mobile version is that the content cannot be downloaded onto mobile devices, so access requires an Internet connection.The cost of an individual subscription is significantly less than the cost of other similar point-of-care tools. For example, an individual physician subscription to DynaMed Plus costs $395 annually, and a professional subscription to UpToDate costs $499 annually. Institutions must contact the vendor for pricing; however, based on the author’s own experience, generally the institutional cost of EEP is significantly lower than other point-of-care tools. EEP offers free 30-day trials for both types of subscribers.EEP was included in a 2016 study that Johnson, Emani, and Ren conducted evaluating six mobile point-of-care tools on how effectively they were able to meet the clinical information needs of internal medicine clinicians. The tools were examined for breadth of content, usability, and quality. The authors found that all of the tools reviewed were relatively similar across these dimensions, but the mobile version of EEP scored somewhat lower than the other tools on usability .EEP is a robust, cost-effective, and efficient tool for providing clinicians with current information to facilitate evidence-based practice. The credentials of the editors and the Medical Advisory Board suggest that it is a resource that is carefully vetted by medical experts, so the information is most likely highly reliable. While there are a number of other point-of-care clinical tools available on the market, EEP’s special features such as the daily POEM alerts, the clearly labeled levels of evidence, and the Drug Safety Alerts set this database apart. The low cost as well as the similarity to other point-of-care tools in terms of content and quality make it an appealing option to those with limited budgets.


Discussions within the health community routinely emphasise the importance of evidence in informing policy formulation and implementation. Much of the support for the evidence-based policy movement draws from concern that policy decisions are often based on inadequate engagement with high-quality evidence. In many such discussions, evidence is treated as differing only in quality, and assumed to improve decisions if it can only be used more. In contrast, political science scholars have described this as an overly simplistic view of the policy-making process, noting that research ‘use’ can mean a variety of things and relies on nuanced aspects of political systems. An approach more in recognition of how policy-making systems operate in practice can be to consider how institutions and ideas influence which pieces of evidence appear to be relevant for, and are used within, different policy processes.Drawing on in-depth interviews undertaken in 2015–2016 with key health sector stakeholders in Cambodia, we investigate the evidence perceived to be relevant to policy decisions for three contrasting health policy examples, namely tobacco control, HIV/AIDS and performance-based salary incentives. These cases allow us to examine the ways that policy-relevant evidence may differ given the framing of the issue and the broader institutional context in which evidence is considered.The three health issues show few similarities in how pieces of evidence were used in various aspects of policy-making, despite all being discussed within a broad policy environment in which evidence-based policy-making is rhetorically championed. Instead, we find that evidence use can be better understood by mapping how these health policy issues differ in terms of the issue characteristics, and also in terms of the stakeholders structurally established as having a dominant influence for each issue. Both of these have important implications for evidence use. Contrasting concerns of key stakeholders meant that evidence related to differing issues could be understood in terms of how it was relevant to policy. The stakeholders involved, however, could further be seen to possess differing logics about how to go about achieving their various outcomes – logics that could further help explain the differences seen in evidence utilisation.A comparative approach reiterates that evidence is not a uniform concept for which more is obviously better, but rather illustrates how different constructions and pieces of evidence become relevant in relation to the features of specific health policy decisions. An institutional approach that considers the structural position of stakeholders with differing core goals or objectives, as well as their logics related to evidence utilisation, can further help to understand some of the complexities of evidence use in health policy-making. Increasingly, policy-studies scholars have explored aspects of the political system that may shape when, how and the types of evidence used within policy-making )The policy sciences have thus been increasingly applied to questions of evidence use in health policy-making and beyond. These perspectives allow consideration of the multiple interests and multiple bodies of evidence that are important to a policy decision, while further recognising the ways that institutional and ideational factors can lead to differing constructions of what evidence is seen to be appropriate to address any given interest in the first place – with institutional forms and ideas closely linked to the relative influence of different stakeholders in policy processes.In this paper, we embrace this approach, applying ideas from new institutionalism to explore the competing or contrasting constructions of evidence use for a set of three differing health policy concerns in the setting of Cambodia. On the one hand, new institutionalism highlights not just the structures in place that shape decision-making processes and outcomes, but also the importance of rules and norms within organisations that guide actor behaviours or decisions , 42. TheApplying such an approach to the question of evidence use for health thus allows us to focus on multiple issues. First, we can consider the power or influence that different stakeholders have over policy processes based on their structural positions for a given policy issue – reflecting on how different bodies of evidence may be more or less relevant to given stakeholders with influence. However, this approach also allows exploration of the institutional logics which those stakeholders possess c.f. , 44) tha tha44]) an attractive and strategically important target” [the significant revenues generated by tobacco advertising [for the government] will, in the short term, delay anti-smoking initiatives until alternative forms of revenue are guaranteed”  always complain that if we increase taxes, farmers will lose their job. So, you have to explain to the government that, if you increase taxes, the margin will not affect the industry. Also, we have to explain that farmers do not rely on one crop only, so reduced tobacco production will not significantly affect them.” “Tobacco industry is powerful and has money. Some people are lobbied by tobacco corporations. They [tobacco corporations] have a lot of experience. They can approach friends or members of the family and get confidential information about policy making. Then, people that are lobbied create opposition at the inter-ministerial meetings. They often say that tobacco control will impact on the economy and farmers.” “The context of a deeply entrenched and powerful tobacco interest was also manifested in how respondents conceptualised the evidence that was relevant for moving tobacco control policies forward. For example, a number of civil servants interviewed stressed the need to counter other evidence that the tobacco industry uses to frame tobacco control in a way that suits industry interests. For example:You have to find a way to convince people… also because policy is multisectoral. It’s not that one minister decides. If you want to increase tax, this is not an issue of the Ministry of Health. We don’t have the power to do this. We can do a smoke-free policy, but tax is under the Ministry of Finance. So you have to work closely with the Ministry of Finance. In Cambodia, when you talk to the Ministry of Finance, first you have you prove to them they can make more money… The industry can say ‘oh if you increase taxes, you will lose revenue’. And you have to present evidence that increasing taxes is not a loss of revenue.” “We explain to the government that an increase in tax does not change the overall volume of cigarettes that are sold in the market. The case of Thailand shows this. Why? Because smoking prevalence decreases, but population increases. Cambodia is the same. Smoking prevalence has gone from 49.6% to 42.6%, however the absolute number of smoking is always 2 million because population increased… and you have to tell the government these facts. So you have to do a lot of work with the government to prove this. And of course, the tobacco industry makes a lot of money. We cannot stop them.” “Respondents also emphasised the importance of making different evidence-based arguments to different actors. In particular, it was noted that the MEF needed different evidence from that required by the MoH regarding tobacco control to try to convince it to support policy action. As one civil servant explained:Usually in Cambodia we present evidence or examples from ASEAN [Association of South East Asian Nations] countries. How is Vietnam doing? Thailand? Indonesia? Then, we also do international. But ASEAN is very important, also because we are approaching the ASEAN [Economic] Community in 2015, and member countries do not want to be left behind.” .“This civil servant also explained how evidence from neighbouring countries was considered influential, as was the normative element of the FCTC.Overall, whilst many of the respondents spoke of the slow progress of tobacco control policy in Cambodia, the government has made substantial progress in tobacco control by banning the advertisement of tobacco products in 2011 as well as banning smoking in workplaces and public spaces in 2016 using sub-decrees. However, it was not until April 2015 that the Cambodian National Assembly passed the country’s first-ever law on tobacco control, which was ratified later the same month. The new law tackles tobacco from a variety of angles, including through import and sales restrictions, and bans on sales to minors and pregnant women .used high-quality strategic information to inform a [successful] evidence-based response” [I would say for HIV/AIDS it’s that way [evidence use is more technical than in other areas of policy] every time. I feel like that’s brilliant and the model of HIV/AIDS [should] be replicated to other diseases, for example, we still have a very high number of deaths among pregnant women, the baby, the infant. So why don’t they learn from the HIV/AIDS program.” “HIV/AIDS policy-making illustrates a radically different political context in which the utilisation of health policy-relevant evidence can be explored. UNAIDS has stated that Cambodia has “esponse” , and intYes they [policy-makers] welcome [overseas evidence] in the HIV area. I don’t know about the other area. They welcome to learn the best practice from the region. This week one [NGO] staff member. He joined the field in Bangkok.” “When interviewees were asked about the use of evidence within particular policy developments, a range of evidence types were described in addition to the epidemiological modelling and cost-effectiveness studies mentioned above. Other relevant evidence was said to include pilot studies, used, for example, to inform the development of a community-based testing approach implemented in 2013, where the HIV testing is performed by lay counsellors – volunteers from population groups at higher risk of HIV infection. National prevalence estimates and international evidence were also evidence types that were frequently mentioned, particularly with international evidence from other Southeast Asian countries, and particularly Thailand. The importance of international evidence may also reflect the strong role of donors in HIV/AIDS policy-making in Cambodia.Epidemiological modelling of future prevalence scenarios has been considered key to informing policy with regards to the prioritisation and targeting of interventions, preparation of operational plans, budgets and resource mobilisation efforts , 12. FurYes it’s different [the policy-making process for HIV/AIDS compared to that for other health issues]. I think this is because donor support, and I think the other thing is because of resource, donor support and resource. Resource, I would say financial resource and human capacity resource, let’s say for HIV/AIDS they have more educated [staff] and they adhere to plan, they adhere to target and they target evidence and I feel like the government take that approach very well, participatory approach, it’s very well, because it’s an emergency situation but it is also in the situation where funding is allowing so that’s why we feel like they are open.” “When asked about the reasons why HIV/AIDS policy-making stood out in terms of the use of what is more typically considered policy-relevant health evidence, respondents particularly spoke of the strong donor interest and support for HIV policy-making in Cambodia. They felt that this was key to driving the type of evidence being used in policy-making, and also the relatively well-functioning institutional entry points for such evidence, including the relevant technical working groups (TWGs) of the MoH. One NGO respondent explained:Well I think it’s coordination. Because there’s basically the different donor programme, donors put all the evidence together. It sits on programmes that are… like some donors who are actually doing their own evidence, but not systematically led by the national programme and disseminated in a timely manner.” “However, there were downsides to a donor-driven approach also described, including in relation to siloed, non-integrated evidence gathering. When asked what could be improved, one interviewee explained:The policy-makers they are more open now… for HIV the policy-maker they are more open and learn from the [experience of high-risk groups].” “Interestingly, while HIV/AIDS policy-making has at times been seen as controversial or contested in some countries, related to the highly stigmatised nature of HIV transmission in some contexts , 63, we work to promote that involvement in the policy-making as well, not only [the NGO] but also civil society. But we try to involve the key representatives from each key population to enrol in the policy-making process… we use the number, we use the finding, we use the civil society. But also bring the key population to talk during the meeting is more powerful… So this is like an MSM [men who have sex with men] person or a sex worker, an entertainment worker could stand and speak about their challenges and the law enforcement people they listen to this. They’re part of the meeting. Everyone is part of the meeting… They also listen and sometimes they [describe] their challenges and you can see some improvement.” “[The NGO] The respondents from a key NGO also spoke at length about the effort made by the NGO to engage high-risk population groups in the policy-making process, particularly with regards to supporting representatives to speak at community meetings and at the MoH TWG meetings.I don’t think anyone guided the government to design that policy. But it came clearly from many dialogues that the pay was not enough, that the arrangements did not encourage midwives to work in remote health centres, and did not encourage mothers to use health facilities.” “A few years ago there was a policy to put one midwife in each health centre and the midwifery incentive… Hun Sen acted on this, and the policy was implemented immediately and very effectively… the trigger was the Demographic and Health Survey… it has quite a bit of impact, and there was a lot of pressure from the international community… it was a relatively ‘easy fix’, a simple solution.” “In contrast to the evidence types seen as relevant for tobacco control and HIV/AIDS policy-making, when asked about evidence for policy-making in regard to PBF schemes generally, these schemes were described as reliant almost solely on evidence from pilot studies. Interview respondents spoke of various pilot schemes of PBF that had been run over the years in different districts and by different groups, often by NGOs, but also in workplaces of NCHADS. The perceived dominance of pilots as an evidence type is likely due to specific PBF policies being scaled up based on a pilot, but such schemes are likely also informed by evidence from health economics more broadly  that incentives can achieve outcomes . When pua country of pilots” .However, the dominance of pilots as an evidence type fits with observations from van de Poel et al. , as statThat [the GMIS] was an example of policy being changed by the government. It’s the government’s job, without any evidence.” “You know, in the United States the evidence has, as far as can tell, no effect on congress. But what happens is they pass laws and then health and human services when they’re putting out the regulations or something, that’s where the evidence comes in. Here it’s more like everything at the congress level, even there is no, it doesn’t get more rational as it comes down through the MoH… People have very set ideas about things and those aren’t going to change no matter what evidence is put in front of them.” “You can present the evidence and present it passionately and you can present it unanimously when you are heard, but there’s never really a proper policy dialogue. So saying well I could go and speak to the Ministry of Economy and Finance about that, we need extra money or we need to look at the budgets or perhaps we need to revise the [pre-service] training curriculum. You don’t get that.” “We don’t even know who made it. There’s a re-writing of history that claims the MoH thought about it but there was no evidence. I was here right after it started and know many people here when it started. At the time no-one was claiming the MoH invented it, so it came out of the MEF, the Ministry of Economy and Finance. It was actually hugely successful.” “In contrast, however, there was also considerable discussion of the fact that, at times, policy directives come from high levels of government – within the MoH or as a decision made by the Prime Minister himself – and that, in such situations, evidence is perceived to be of limited importance. The GMIS was described as an example of this:I think in Cambodia evidence  government want the community to deliver their baby at the health facility so the government just simply providing incentive to the health staff, community wide, so for delivery of one live birth delivery, they get US$15.00 this is the decision by the government and government budget and then see if they implemented that.” “Another respondent from a donor agency was unable to name evidence in support of the policy, and instead described how he saw the policy process for the GMIS.This perception of success appears to come from data showing increasing facility utilisation for delivery and falling mortality rates nationally after implementation of the programme. Since then, the percentage of deliveries in public health facilities has increased substantially, from 29% in 2006 to 57% in 2011, and the maternal mortality ratio has declined substantially from 473 per 100,000 live births in 2005 to 206 in 2010 . Care, oThe three policy areas presented show few similarities in how pieces of evidence were used in various aspects of policy-making, despite all being discussed or undertaken within a single MoH, and within a broad policy environment in which ‘evidence-based policy-making’ is rhetorically championed. In this section, however, we draw out some of the particular institutional and ideational features of the three health policy concerns that may help to explain these findings.A starting point is to compare the institutionalised positions of influence of the key stakeholders in each case in order to reflect on how the relevance of particular evidence types fit with the interests of such stakeholders. This can then be followed by considering any contrasting institutional logics that similarly might help explain differences in evidence utilisation. Such logics could either be direct thinking about which evidence is relevant and why , or they may be related to the overarching goals or expectations of the actors involved, which subsequently shapes their uses of evidence .In the case of tobacco control, the historical influence of the tobacco industry appears particularly relevant, and the nature of contestation for this issue appeared to be principally framed in terms of financial implications of tobacco control. Our respondents described the financial importance of tobacco for the agricultural sector and the national economy as the paramount concerns for any policy change. Tobacco control advocates are well aware of the need to present different evidence and frame the issue differently to address the concerns of the most influential stakeholders, with a particular distinction made between the health and economic evidence needed when speaking to policy-makers from the MoH and MEF, respectively. The need for taxation and other regulatory policy to be made outside the MoH illustrated how limited health-related evidence of tobacco harms could be in driving tobacco control policy forward on its own.Evidence from neighbouring countries, and the FCTC, were said to be influential. However, even so, and despite considerable progress, policy change in line with these was described as particularly slow, as too was the development of what were considered more appropriate forms of evidence to guide tobacco policy from a public health perspective such as national smoking prevalence surveys. Despite global evidence on tobacco harms and increasing local data on smoking, it was the concerns regarding economic growth and the industry’s entrenched interests and lobbying documented in our study and elsewhere , 67, 68 big on evidence” .In contrast, the HIV/AIDS policy response in Cambodia developed in a rather different political context. With HIV/AIDS, there was no establishment of corporate interests and little obvious financial interest at stake for any major stakeholders. Instead, the issue may have achieved a relatively high level of priority for policy action in Cambodia due to the attention and resources this issue has been afforded by donor agencies and, possibly related to this, the well-functioning TWGs of the MoH for HIV policy-making, as described by our respondents. The institutionalisation of donor influence is, in fact, reflected within the country’s various strategic documents for guiding programme implementation, including the National Strategic Plan for HIV/AIDS 2011–2015 and Cambodia 3.0, a strategy developed by the country’s MoH to eliminate new HIV infections and congenital syphilis by 2020. Both are considered to be in line with the global targets and foci established by UNAIDS and the US PEPFAR programme (from which Cambodia is a recipient of funds) , 69. HowIndeed, while in many countries the issue of HIV testing has been subject to debate or controversy, particularly around issues of disclosure or confidentiality of people living with HIV, and the challenges associated with addressing HIV in often stigmatised groups such as men who have sex with men, transgender people and sex workers, these concerns were considered by our respondents to have been relatively unimportant in Cambodia (even if admittedly seen as sensitive). That HIV/AIDS is an issue with social connotations in Cambodia perhaps explains the use of narrative evidence – stories of the lived experiences of marginalised groups – to influence policy-making. However, the relatively low level of moral contestation for this issue in Cambodia was noted and used to explain the recent introduction of community-based rapid HIV testing  by lay volunteer counsellors for high-risk population groups. Within this recent national policy, however, it was again evidence of effectiveness provided from a pilot study that could be seen to lead to policy change , 71.Nevertheless, the importance of pilots was much more apparent, and described as the primary source of policy-relevant evidence for PBF. This was in contrast to tobacco policy appearing to require discussion of evidence of financial impact (linked to the influence of one set of interest groups), and HIV policy drawing particularly on epidemiological models and surveys . Again, we can look to the most influential stakeholders involved and their institutional logics to help explain the emphasis on pilots as a form of evidence in this case. Indeed, this helps to move away from the often criticised over-reliance on the idea that a single hierarchy of evidence can guide policy decisions, to instead consider the policy ‘appropriateness’ of particular forms of evidence –74.Unlike the previous cases, the GMIS policy appears to have had few stakeholders outside the government itself. It was reportedly made from the highest levels of the government, with some interviewees speculating that it was driven by the Prime Minister’s office in particular in response to a feeling that some action must be taken to help achieve the maternal health MDG by 2015. The power and influence in this case appeared to be particularly hierarchical, with decision-making made through a planning and management orientation. Pilot studies, which examine feasibility of an intervention, are a first step in exploring novel interventions, novel applications of an intervention, or the feasibility of an intervention in a particular context when the effectiveness of that intervention may be context dependent . For thiIn Table Despite a common use of the language of evidence-based policy-making in the health sector, there are in fact many types of evidence that can speak to a variety of political concerns and mandates that play out in the policy process. Sometimes evidence use differs because the evidence needs vary in order to solve an agreed upon problem. At other times however, there may not be any agreement on the nature of the problem, and as such it is the features of power, interests and framing that serve as important drivers shaping which evidence informs policy considerations. An institutional approach can help to understand some of these dynamics. It can identify which stakeholders have established positions of power within differing health policy issues, linking their interests to the nature of evidence used. It can also help reflect on the institutional logics of these stakeholders which may further influence when or how evidence is utilised.For tobacco, large and expensive national prevalence surveys were considered necessary evidence for intervention, even given considerable evidence from smaller studies of a high prevalence of smoking in the country, and the irrefutable global evidence linking tobacco to numerous diseases and mortality. However, such surveys were needed because of a demand for evidence that could speak to the dominant concerns of financial impact and the logic that evidence was needed to illustrate economic impact, rather than any public health logic of evidence to show medical harm to the population. The importance of the MEF is thus apparent in this case – illustrating both its dominant policy concern in terms of the economy, but also its logic of what forms of evidence are required to speak to that concern. Furthermore, it is of course critical to understand the historical influence and role of the tobacco industry in the country, which no doubt has played an important role in shifting the terms of tobacco policy to one of revenue.In contrast, for the case of HIV/AIDS, the dominance of global donors in supporting this health issue, and the apparent limited contestation at a local level, appears to have led to the explicit embrace of epidemiological evidence that is widely held to be appropriate for HIV planning within the global health community. However, in the final case of PBF, it was the government that drove both the initiation and implementation of the policy response. This state-controlled process appeared to reflect a belief that national action must be taken to address an existing priority (in the form of a MDG). This, in turn, naturally led to a logic which saw relatively small studies focussed on implementation to be the most relevant to policy. Although it is worth noting that, in the case of the GMIS, some believed that evidence was not perceived as important at all due to the policy being driven by higher level political authorities. Indeed, some stakeholders simply referred to the GMIS policy when asked for a good example of the use of evidence, since it was national action based around the ‘evidence’ that maternal mortality rates were too high. This is a much simpler logic of evidence-informed policy-making, whereby evidence of a problem is seen to justify widespread action, in contrast to more traditional health sector descriptions of evidence use being concerned with the effectiveness of interventions or possible alternative priorities or approaches.Policy studies scholars would not be surprised that powerful stakeholders (or ‘vested interests’) end up shaping the understanding of evidence, or which pieces of evidence are championed as relevant for policy-making. This is perhaps most clearly illustrated in how evidence has been presented or selected by the tobacco industry in relation to policy-making debates c.f. , 78, 79), 7978, 7In the three contrasting case studies of evidence use in health policy-making examined in this study, evidence types – and their framing – were found to differ greatly, despite them taking place in the same country setting. The findings reiterate past authors’ understandings that ‘evidence’ is not a uniform concept for which more is obviously better, or where a single model of ‘evidence-based policy-making’ can prevail, but rather that different constructions and pieces of evidence become relevant given the politics involved in policy decisions, the nature of institutions involved, and the framing and conceptualisations of the issues themselves. Our comparative analysis helps to begin to trace out themes in linkages between the nature of contestation of health issues, the interests of established dominant stakeholders, and the logics by which those stakeholders operate – all of which work to shape which evidence is utilised or seen as policy relevant to inform health decisions. Whilst considerable further empirical research is needed in this area, this more nuanced understanding of evidence use may be of relevance to health policy-makers and others considering how to improve the role of evidence in health policy-making.


EDPs specifically achieve this by promoting early deliberation among a wide variety of stakeholders to identify, reflect and learn about the meaning and importance of values, informed by evidence on these values.3 Being a generic value-assessment framework, it may be applied to prioritization exercises with either broad or narrow scopes and should be contextualized to its local setting.Evidence-informed deliberative processes (EDPs) to support priority setting for universal health coverage (UHC), as described in our editorial,4 We also concur with Lauer et al that priority setting should take into account higher level, systemic activities that can strengthen progression towards UHC such as ‘improving health-system governance,’ ‘ensuring equitable access to quality services,’ ‘separating prescribing from dispensing,’ or ‘setting up a pooled funding mechanism to purchase services.’5 Moreover, EDPs “should adapt to a diverse set of factors shaping the relationship between evidence and policy.”4 Ideally, EDPs are initiated at an early stage, as policy-relevant evidence can still be commissioned or searched for during this stage, and there is still time to reflect on input or ideas put forward by stakeholders. As Gopinathan and Ottersen point out, this requires involvement of the right stakeholders, including non-health stakeholders where relevant.4 At the same time, Lauer et al raise the question “who should be invited to the deliberative dialogue?”5 We argue that those affected by decisions should at least be provided the opportunity to participate and provide relevant reasons or evidence3 – and we acknowledge that it is hard to determine who the relevant stakeholders are, and that standardized approaches are required to identify and engage relevant stakeholders. The description by Gopinathan and Ottersen of the complex relationship between evidence and policy further demonstrates the challenge of knowledge translation and how evidence can eventually be presented so as to facilitate its uptake and inclusion in policy formulation or revision.6 More broadly speaking, these comments highlight the challenge of institutionalization: how to work towards a situation that priority setting for UHC gets integrated in the routine decision making processes at both national and sub-national levels.We agree with Gopinathan and Ottersen that the “focus of such processes [to support priority setting for UHC] should go beyond clinical services to accommodate also public health interventions.”5 Universal goals do need to be aligned with local context and priorities. We also recognize the point made by Lauer et al that EDPs, generic as they are, are already in place to a certain degree at the country level and are supported by the WHO Secretariat,5 in the form of consultative groups, and part of health-sector reviews and strategic planning and budgeting. Likewise, as Chalkidou et al point out, the International Decision Support Initiative  has been established to strengthen in-country institutional and technical capacity together with open participative processes for evidence-informed policy-making.7 We hope that such initiatives will benefit from the further development of EDPs.With regard to whether stakeholder deliberation as part of EDPs departs from a ‘blank slate,’ we agree with Lauer et al that countries should not ignore global level policy as formulated and endorsed by the World Health Organization (WHO) member states.7 On the contrary, evidence on the cost-effectiveness of a clinical or public health intervention provides relevant information on the opportunity costs of alternative investments that are foregone. Hence we do not oppose, but instead welcome efforts to integrate evidence on multiple criteria as is the case in the extended cost-effectiveness analysis approach.9 Nevertheless, we agree with Chalkidou et al and would like to highlight again that “global approaches to CEA can hardly be too context-sensitive” for the very reason that “studies done by global players that ignore local contexts but nonetheless presume to advise may undermine local priorities and distort local spending decisions.”7 We furthermore applaud WHO for providing broader support and guidance to countries.5 Yet, we observe in practice that cost-effectiveness is often considered the dominant criterion when used in priority setting. This effectively makes the generation of other evidence and deliberation on additional criteria secondary to cost-effectiveness, which may undermine the legitimacy of decisions at the country level.That said, it is not our intention to devalue cost-effectiveness evidence in itself – or to criticize agencies specialized in producing this particular type of evidence.We thank all authors who commented on our editorial, and we look forward to working together in the coming years to help harmonize EDPs with like-minded initiatives and align them with sustainable country-led decision-making processes.Not applicable.Authors declare that they have no competing interests.MPJ wrote the first draft; all other authors contributed to revisions. All authors approved the final draft.1Radboud Institute for Health Sciences, Radboud University Medical Center,Nijmegen, The Netherlands. 2KIT , Amsterdam, The Netherlands. 3Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands. 4Institute of Public Health, Heidelberg University, Heidelberg, Germany. 5Africa Health Research Institute, Mtubatuba, South Africa.


The overall purpose of the ‘Statistical Points and Pitfalls’ series is to help readers and researchers alike increase awareness of how to use statistics and why/how we fall into inappropriate choices or interpretations. We hope to help readers understand common misconceptions and give clear guidance on how to avoid common pitfalls by offering simple tips to improve your reporting of quantitative research findings. Each entry discusses a commonly encountered inappropriate practice and alternatives from a pragmatic perspective with minimal mathematics involved. We encourage readers to share comments on or suggestions for this section on Twitter, using the hashtag: #mededstats.null findings, as evidence in favour of a null hypothesis . Null findings are frequently interpreted as ‘informing’ theory or as ‘confirming’ theoretical expectations. In this entry, we explain two arguments against the habit of interpreting a null finding as evidence in favour of a null hypothesis. Based on these arguments, we explain that statistical power and required sample size calculations along with replication research and meta-analysis can help us counter the habit of interpreting non-significant findings as evidence in favour of the null hypothesis, and that Bayesian hypothesis testing can help researchers to evaluate the strength of evidence in favour of the null hypothesis or against it.There is a widespread habit in educational research of interpreting statistically non-significant findings, also called n = 20) and an example-problem (n = 20) condition. Students in the problem-problem condition solve two problems – problem A and problem B – that follow the same structure and are of similar difficulty. In the example-problem condition, students first study a worked example of problem A and then solve problem B. Subsequently, students in both conditions complete the same post-test, which comprises ten problems of the same structure as problem A and B and are of similar difficulty. Each post-test problem is scored ‘0’ whenever a student provides an incorrect solution and ‘1’ when that student provides a correct solution. Hence, the total score on the post-test can range from 0 to 10.One area of study in educational research compares learning from examples with learning by solving problems , 2. SuppSD) of 1.16, while the example-problem condition yields an average score of 6.05 with an SD of 0.89. The 95% confidence interval of the difference between average scores (6.05–5.75 = 0.30) [p-value instead of a confidence interval do a t-test on the difference between the average scores of the two conditions with the null hypothesis of ‘no difference’ against the alternative hypothesis of ‘there is a difference’ [p = 0.36.The researchers compute post-test scores accordingly for each student and find that the two conditions do not differ much in post-test score: the problem-problem condition yields an average score of 5.775 with a standard deviation ( = 0.30) . Researcp-value of 0.36 – or the 95% confidence interval that includes ‘0’ for that matter – to conclude that there is ‘no difference’ between the two conditions and hence it does not matter whether in a practical situation we let students solve problems by themselves right away or we first provide them with an example. Two main arguments against this ‘confirming the null’ approach are discussed in the following.In many cases, the researchers use the p-values as evidence in favour of the null hypothesis comes from scholars who note that studies with sample sizes that are common practice in psychology and education (e. g. our example study) often lack statistical power and may therefore frequently fail to reject the null hypothesis even if it is not true (i. e. Type II error) [A first argument against interpreting non-significant I error) –7. StatiI error) , 9 ; witG*Power [n = 20, taking as a starting assumption for their calculation half a standard deviation difference in the population of interest and testing at the conventional α = 0.05 significance level, they would have learned that their study has a statistical power of only 0.34. In other words, even if there is such a difference in the population they sampled their students from, only about one of every three studies of this size (two groups of n = 20) would reject the null hypothesis of ‘no difference’ . This is the same as saying that we would fail to reject the null hypothesis in about two of every three studies of this size.Had the researchers of our example study, prior to conducting the study, calculated the statistical power (e. g. using G*Power ) for a sα = 0.05 significance level , they would have learned that they need two groups of n = 64 each [Had the researchers calculated in advance what sample size they would need for a statistical power of 0.80, assuming half a standard deviation difference in the population of interest and testing at the conventional  64 each .p-value as evidence in favour of the null hypothesis, a study with two groups of n = 20 is unlikely to detect a substantial difference between groups in the first place.Thus, while the researchers in our example study interpret a non-significant p-values as evidence in favour of the null hypothesis comes fromscholars who point at the fact that a statistical significance test uses the p-value as a probability under the null hypothesis but disregardssuch a probability under the alternative hypothesis [likelihood ratio or Bayes factor then expresses under which of the two hypotheses –null or alternative – the observed finding is more likely to have occurred [A second argument against interpreting non-significant pothesis . Scholaroccurred :\documenp-value. Table A Bayes factor of 1 would indicate that the observed finding is equally likely under both hypotheses . A Bayes factor for the alternative hypothesis (numerator) vs. the null hypothesis (denominator) of 2 corresponds with a Bayes factor for the null hypothesis (numerator) vs. the alternative hypothesis (denominator) of 0.5 (i. e. 1/2 = 0.5) and indicates that the finding is twice as likely to have occurred under the alternative hypothesis. Analogously, a Bayes factor for the alternative hypothesis vs. the null hypothesis of 0.5 is the same as a Bayes factor for the null hypothesis vs. the alternative hypothesis of 2 (i. e. 1/0.5 = 2) and indicates that the finding is twice as likely to have occurred under the null hypothesis. Such an interpretation is impossible to achieve with a t-test as an alternative to the t-test that we have been using in medical education research for decades.For more details on the use and interpretation of Bayes factors, we refer to Rouder et al. , who proboth p-values and Bayes factors [There is a free SPSS-like software program that enables researchers to calculate  factors . Using tp-values in terms of evidence in favour of the null hypothesis. Moreover, the researchers from our example study could have a look at other studies that have also made problem-problem comparisons [Following the argument of limited statistical power, calculations of statistical power and sample size to achieve a high statistical power can help us reduce the likelihood of planning a study that is too small to have a decent statistical power and can help us counter the habit of interpreting non-significant parisons , 2 and wp-value as evidence in favour of the null hypothesis regardless of statistical power or sample size [p-values, Bayes factors of single studies can easily be combined in meta-analysis to provide a more accurate picture across studies with regard to the evidence in favour of or against the null hypothesis [However, in the light of the argument that we ought to compare the likelihood of a finding under the null vs. under the alternative hypothesis, we should not interpret a non-significant ple size . If we aple size , 15 and ple size  can helppothesis , 12.p-values and confidence intervals may provide some evidence against a null hypothesis, but cannot provide evidence in favour of a null hypothesis. With statistical power and required sample size calculations as well as replication research and meta-analysis, we have powerful tools for countering the habit of interpreting non-significant p-values as evidence in favour of a null hypothesis. Besides, Bayesian hypothesis tests provide researchers with a tool to address the question of evidence in favour of the null hypothesis when that question is of interest.Absence of evidence is not the same as evidence of absence;


F1L and E3L. Here, we tested MVA vaccine preparations on the basis of the deletion mutant viruses MVA-ΔF1L and MVA-ΔE3L for efficacy against ectromelia virus (ECTV) challenge infections in mice. In non-permissive human tissue culture the MVA deletion mutant viruses produced reduced levels of the VACV envelope antigen B5. Upon mousepox challenge at three weeks after vaccination, MVA-ΔF1L and MVA-ΔE3L exhibited reduced protective capacity in comparison to wildtype MVA. Surprisingly, however, all vaccines proved equally protective against a lethal ECTV infection at two days after vaccination. Accordingly, the deletion mutant MVA vaccines induced high levels of virus-specific CD8+ T cells previously shown to be essential for rapidly protective MVA vaccination. These results suggest that inactivation of the anti-apoptotic genes F1L or E3L modulates the protective capacity of MVA vaccination most likely through the induction of distinct orthopoxvirus specific immunity in the absence of these viral regulatory proteins.The highly attenuated Modified Vaccinia virus Ankara (MVA) lacks most of the known vaccinia virus (VACV) virulence and immune evasion genes. Today MVA can serve as a safety-tested next-generation smallpox vaccine. Yet, we still need to learn about regulatory gene functions preserved in the MVA genome, such as the apoptosis inhibitor genes  E3L and F1L are functionally retained in the MVA genome [F1L gene encoding sequences are highly conserved among orthopoxviruses. In previous studies, deletion of the F1L gene in the VACV or MVA genome has been confirmed to enhance the induction of apoptosis after in vitro infection using different cell lines [E3L gene encodes for a 25 kDa protein that features an amino-terminal Z-DNA-binding domain and a carboxyl-terminal RNA-binding domain. The carboxyterminal domain has been shown to inhibit double-stranded RNA (dsRNA)-activated protein kinase (PKR) by binding and sequestering dsRNA produced during VACV life cycle. Another function of the E3 polypeptide is to block phosphorylation and thus activation of interferon (IFN) regulatory factor 3 (IRF3) and IRF7, required for viral induction of IFN-α/β. In previous studies, an MVA E3L deletion mutant (MVA-ΔE3L) was found replication-deficient in chicken embryo fibroblasts (CEF) resulting in insufficient viral DNA and protein synthesis, enhanced induction of apoptosis, and increased production of chicken IFN-α/β [F1L or E3L gene functions could be a promising approach to further improve the efficacy of MVA vaccination. However, the anti-apoptotic viral genes F1L and E3L are functional in MVA genome and the deletion of these genes also impairs the synthesis of late viral antigen in human and murine cells [E3L or F1L genes exhibited reduced protective capacity in comparison to the MVA control vaccine at three weeks after single shot vaccination. Surprisingly, however, we found all vaccines equally efficient for rapidly protective immunization against a lethal ECTV challenge infection given two days after vaccination. In accordance with this finding, the deletion mutant MVA vaccines induced high levels virus-specific CD8+ T cells previously shown to be essential for rapidly protective MVA vaccination. These results suggest that inactivation of the anti-apoptotic genes F1L or E3L modulates the protective capacity of MVA vaccination against ECTV infection most likely through the induction of distinct orthopoxvirus specific immunity in the absence of these viral regulatory proteins.Variola virus (VARV), the causative agent of human smallpox, has been successfully eradicated by prophylactic vaccination using live vaccinia virus (VACV). However, even today more than thirty years after this famous achievement in medicine, there are still concerns that VARV may be used as bioterroristic weapon and zoonotic monkeypox or cowpox remain threatening infections in humans. Thus, developing improved vaccination principles ready to use in an immediate public health response are essential. The Modified Vaccinia virus Ankara (MVA) is a replication-deficient and safety tested VACV that is already licensed as a next-generation smallpox vaccine in Europe and Canada. Moreover, MVA has been actively investigated as a non-replicating multipurpose viral vector vaccine against other infections and cancer diseases ,2. Thus,A genome ,20,21,22A genome ,23. OnceA genome . The F1Lll lines ,23. The MA-104 ATCC CCL-26), HeLa (ATCC CCL-2), NIH/3T3 (ATCC CRL-1658), BHK-21 (ATCC CCL-10) and chicken embryo fibroblast (CEF) cells were used and routinely maintained as previously described. Plaque purified Ectromelia virus (ECTV) strain Moscow  was propagated on MA-104 cells. As experimental vaccine served Modified Vaccinia virus Ankara (MVA)  ; all rec, HeLa (AConfluent monolayers of HeLa cells or NIH/3T3 cells were infected at a multiplicity of infection (MOI) of 5 with Modified Vaccinia virus Ankara (MVA) , MVA-ΔF1L, MVA-ΔE3L and revertant viruses. Cell lysates were prepared at different time points after infection . Lysates from uninfected cells or wild-type MVA-infected cells served as controls. Polypeptides were separated by sodium dodecyl sulfate polyacrylamide gel electrophoresis (SDS-PAGE) and electroblotted onto a polyvinylidene difluoride (PVDF) membrane. After blocking, membranes were incubated with primary antibodies polymerase (PARP) Cell Signaling Technology  diluted 1:1000; mouse anti-β-Actin Sigma-Aldrich  diluted 1:1000) at 4 °C overnight. After washing, the blots were incubated with secondary antibodies for one hour at room temperature.047R (F17R) mRNAs and reverse primers containing a T7-RNA polymerase promoter were synthesized as previously described [For analysis of mRNA, cells were infected with MVA, MVA-ΔE3L or the revertant virus MVA-ΔE3Lrev at an MOI of 5. PBS-infected cells served as mock-control. Total RNA was isolated with TRIzol reagent , following the manufacturer’s instructions. Total RNA was separated by electrophoresis in 1% agarose formaldehyde gels. Following, RNA was transferred onto positively charged nylon membranes . Riboprobes for detection of MVA-encoded escribed . In vitrFemale C57BL/6N mice (6–10 weeks old) were purchased from Charles River Laboratories . For experimental work, mice were housed in an isolated (ISO) cage unit  and had free access to food and water. All animal experiments were handled in compliance with the German regulations for animal experimentation .5 or 106 pfu of MVA, MVA-ΔF1L, MVA-ΔE3L or phosphate-buffered saline (PBS) into the left hind leg. For intranasal infection, mice were anesthesized by intraperitoneal (i.p.) injection with 1 mg ketamine and 0.04 mg xylazine per 10 g body weight. Intranasal infection was performed by instillation of 20 µL of virus suspension containing 200 pfu (~3 × LD50) or 2000 pfu (~30 × LD50) ECTV. Signs of illness, weight loss and survival were monitored daily for at least three weeks. In all experiments inoculations of corresponding amounts of PBS were used as controls (mock vaccine).Intramuscular (i.m.) vaccination was performed by injection of 50 µL of virus suspension containing 10TM MicroWellTM 96 well solid plates, Sigma-Aldrich, Munich, Germany) were coated with 2 µg/mL sucrose gradient-purified MVA F6 in coating buffer  for 3 h at 37 °C and overnight at 4 °C. After blocking with PBS supplemented with 10% heat-inactivated fetal calf serum and 0.05% Tween-20, the plates were incubated with serial dilutions of sera from immunized mice. Antibody titers were determined using an alkaline-phosphatase conjugated anti-mouse antibody and p-nitrophenyl phosphate  as substrate.Firstly, 96-well immunoplates (Nunc-Immuno2 for 48 h. In the following media were removed and the cells were fixed with ice-cold methanol-aceton for 5 min. Fixed cell monolayers were immunostained using primary rabbit anti-VACV antibody (diluted 1:2000) and secondary antibody  followed by incubation with True Blue Peroxidase Reagent . Blue plaques were counted and MVA neutralization titers were determined to be the last dilution of serum that reduced the number of plaques by 60% compared with control wells.For analysis of MVA neutralizing antibodies we used the reference clonal isolate MVA F6 purified by ultracentrifugation through sucrose cushions. Serum samples from immunized animals or mock-vaccinated animals were tested in duplicate or triplicate in plaque reduction assays on CEF cell monolayers grown in 24-well tissue culture plates. Eight two-fold serial dilutions of serum were mixed with 200 pfu of MVA F6 at 37 °C for 2 h. Following incubation, 100 μL of the reaction mix was loaded onto the CEF cell monolayers. Cells were over-laid with medium and incubated at 37 °C with 5% COl-glutamine and 100 IU/mL penicillin-streptomycin. Interferon gammas (IFN-γ)-secreting CD8+ T cells were analyzed by using the Enzyme Linked Immuno Spot Assay (ELISPOT) PLUS kit for mouse IFN-γ  following the manufacturer’s instructions. ELISPOT plates were preincubated overnight with the antibody solution and then incubated with the cell suspension that had been stimulated with the virus-specific peptide B8R20–27 . After centrifugation, the cell pellet was dissolved in Roswell Park Memorial Institute (RPMI) medium supplemented with 10% fetal calf serum, 2 mM P values were adjusted with the Bonferroni method. CD8+ T cell responses were compared by T test. The statistical evaluation was performed with GraphPad Prism for Windows .Statistical comparison of different groups of mice was analyzed by one-factorial analysis of variance (ANOVA) for the area under the percentage-of-initial-weight curves (AUC). The differences between vaccination groups were analyzed with a one-factorial analysis of variance model. For multiple comparisons, 047R (F17R)  of VACV proteins. Among the viral early proteins are antigens containing important determinants of T cell immunity that are presented via MHC-I molecules to CD8+ T cells ,28. LateR (F17R) B. PrevioR (F17R) C. Though5 or 106 pfu MVA-ΔF1L and MVA-ΔE3L twenty-one days before the high dose respiratory infection with ECTV (2000 pfu corresponding to about 30 × LD50) . In MVA-ΔE3L vaccinated mice, the signs of disease were more severe and culminated with a loss of up to 15% of the body weight at day eleven post infection. Inoculations with doses of 105 pfu of the MVA vaccines resulted in a more pronounced but similar outcome with regard to the induction of protective immunity against the high-dose ECTV infection. All mice that had been vaccinated with 105 pfu MVA-ΔE3L developed systemic mousepox disease with progressive body weight loss starting nine dpi, and died or had to be euthanized until thirteen dpi. In contrast, vaccinations with 105 pfu of either the MVA-ΔF1L or MVA vaccine protected against lethal mousepox. MVA-ΔF1L vaccinated mice, however, developed severe signs of illness which correlated with the body weight loss. In contrast, in MVA immunized mice we observed no obvious weight loss or signs of mousepox illness. Taken together, our data indicate that in comparison to standard MVA, vaccination immunizations with the MVA mutant viruses deleted in the regulatory genes F1L or E3L have a reduced capacity to induce protective immunity against severe mousepox.To elucidate the role of the F1 and E3 proteins in protective immunization, we tested MVA-ΔF1L and MVA-ΔE3L in the C57BL/6 mouse-ECTV challenge model ,5,30. Br × LD50) . Control6 pfu of MVA, MVA-ΔF1L or MVA-ΔE3L and analysed VACV-specific CD8+ T cells by IFN-γ-ELISPOT. Single intramuscular immunizations with all test vaccines induced substantial levels of B8R20–27-epitope-specific CD8+ T cells with median absolute numbers of 3.15 × 103 IFN-γ spot forming cells (SFC) per 106 splenocytes. Of note, MVA, MVA-ΔF1L and MVA-ΔE3L vaccines induced close to identical magnitudes of CD8+ T cell responses  is sufficient to robustly protect against a lethal dose of ECTV inoculated only two days after immunization [5 pfu of either MVA, MVA-ΔF1L, or MVA-ΔE3L vaccine. Injections with PBS served as mock vaccines. Two days after immunization, mice were intranasally inoculated with ECTV (200 pfu corresponding to approximately 3 × LD50) [A sought-after feature of next-generation smallpox vaccines is the capacity to elicit rapidly protective immunity against severe orthopoxvirus infections. In previous work, we demonstrated that even a low dose of MVA vaccine  . As expeE3L and F1L gene sequences are fully conserved in the MVA genome and the deletion mutant viruses MVA-ΔF1L and MVA-ΔE3L demonstrate impaired viral life cycles and elicit increased levels of cell death in various cell types [in vitro antigen production by our MVA-ΔF1L and MVA-ΔE3L candidate vaccines we clearly demonstrate a decreased synthesis of the VACV envelope antigen B5 in human HeLa cells  to test for rapidly protective immunization, and (ii) at 21 days after vaccination a 2000 pfu ECTV challenge (~30 × LD50) resulting in pronounced respiratory disease and early death of infected animals to evaluate the protective capacity of a more conventional orthopoxvirus-specific immunization. The vaccines were given by intramuscular inoculation (a route routinely used for immunization in humans) and at doses (105 or 106 pfu) that were known to be fully protective against mousepox virus challenge infection [6 pfu MVA, MVA-ΔF1L or MVA-ΔE3L vaccine did not reveal differences in the levels of virus particle binding or virus neutralizing antibodies (data not shown). Our failure seems to reflect the common inability of standard immune monitoring assays to reveal correlates for vaccine-mediated protection in vivo. Clearly, more detailed studies are needed to identify the distinct correlate(s) of protection induced by conventional MVA immunization in comparison to MVA-ΔE3L in our ECTV mouse challenge model. Orthopoxviruses induce complex antibody responses reactive against many diverse antigens [20–27 specific CD8+ T cells. This observation is in accordance with the results from other studies suggesting VACV early proteins as major source for virus-specific CD8+ T cell determinants in mice [The replication-deficient and safety tested virus MVA has been licensed as a third-generation smallpox vaccine in Europe and in Canada to replace the conventional live vaccinia virus vaccines . The purll types ,20. In pll types ,22. Herell types ,20,32,33ll types ,35. Corrll types . InevitaLa cells C. As B5 antigens . In our allenge ~ × LD50 rnfection ,39. Our nfection ,41,42. Tantigens . Therefoantigens . Since t in mice ,45. We pE3L or F1L efficiently prime antigens-specific CD8+ T cells, allow for rapidly protective immunization against a lethal ECTV infection but do not improve the overall protective efficacy of MVA in conventional vaccination against a harsh respiratory ECTV challenge.Our study in an experimental murine model for human smallpox suggests that MVA vaccines with inactivated regulatory genes


This study aimed to evaluate the demographic, clinical features, laboratory data, pathology and other survey in pediatric patients with cryofibrinogenemia.A 12-year retrospective chart review identified eight pediatric patients at Mackay Memorial Hospital, Taipei, Taiwan.The female-to-male ratio was 3:1. The mean age at symptom onset and of diagnosis was 10.3 ± 4.6 years and 12.3 ± 4 years, respectively. One child (12.5%) had primary cryofibrinogenemia. The common symptoms were purpura, arthralgia, and muscle weakness (100%). On laboratory examination, cryofibrinogen was positive in all patients. All patients had increased anti-thrombin III while 87.5% and 62.5% had abnormal protein S and protein C, respectively. All eight also complained of neurologic symptoms. One had vertebral artery narrowing, two showed increased T2-weighted signal intensity on the thalamus or white matter, and one had acute hemorrhagic encephalomyelitis on brain magnetic resonance imaging.This study reports on the presentations of cryofibrinogenemia, which is rare in children. Most cases are associated with autoimmune disease and have severe and complex presentations. Central nervous system involvement is common. Cryofibrinogenemia is a rare hematologic disorder wherein plasma forms a cryoprecipitate that consists of fibrinogen, fibrin, fibronectin, factor VII, and smaller amounts of various plasma proteins. Cryofibrinogen precipitates at a low temperature (4 °C) and re-dissolves on warming to (37 °C) . Its exaThe most common symptoms of cryofibrinogenemia are due to cutaneous ischemia and include purpura, livedo reticularis, blisters, tissue necrosis, and ulceration. Systemic symptoms have been reported to be glomerulonephritis and thrombophlebitis . Cases iAfter a review of the diagnostic criteria and clinical features, a medical chart review of eight patients younger than 18 years with a diagnosis of cryofibrinogenemia at the pediatric department of Mackay Memorial Hospital, Taipei, Taiwan, between May 2010 and July 2015. The diagnostic criteria originally proposed by essential and supportive evidence. Essential evidence included: 1) appropriate clinical presentation of sudden onset of skin changes and constitutional symptoms, with or without thrombosis, bleeding, or exposure to cold; 2) presence of cryofibrinogen in plasma; and 3) absence of cryoglobulins. Essential cryofibrinogenemia meant no secondary causes of cryofibrinogens and no evidence of other vaso-occlusive diseases.Supportive evidence was non-specific and not compulsory. However, when present along with essential evidence, supportive evidence improved diagnostic accuracy. Supportive evidence included: 1) angiogram with abrupt occlusion of small to medium sized arteries; 2) typical skin biopsy findings of cryofibrinogen plugging vessels, leukocytoclastic vasculitis, or dermal necrosis; and 3) elevated serum levels of α1-antitrypsin and α2-macroglobulin .In conclusion, this study reports on the presentations of cryofibrinogenemia, which is rare in children. Most pediatric cases are associated with autoimmune diseases and have more severe and complex presentations. Although rarely reported previously, CNS involvement is common in our study. When children complain of persistent unexplained pain on multiple sites with purpura or ecchymosis, cryofibrinogenemia should be considered and further investigated.


National guidelines for smoking cessation in primary care can be effective in improving clinical practice. This study assessed which parties are involved in the development of such guidelines worldwide, which national guidelines address primary care, what recommendations are made for primary care settings, and how these recommendations correlate with each other and with current evidence. We identified national guidelines using an online resource. Only the most recent version of a guideline was included. If an English version was not available, we requested a translation or summary of the recommendations from the authors. Two researchers independently extracted data on funding sources, development methodologies, involved parties, and recommendations made within the guidelines. These recommendations were categorised using the pile-sort method. Each recommendation was cross-checked with the latest evidence and was awarded an evidence-rating. We identified 43 guidelines from 39 countries and after exclusion, we analysed 26 guidelines . Twelve categories of recommendations for primary care were identified. There was almost universal agreement regarding the need to identify smokers, advice them to quit and offer behavioural and pharmacological quit smoking support. Discrepancies were greatest for specific recommendations regarding behavioural and pharmacological support, which are likely to be due to different interpretations of evidence and/or differences in contextual health environments. Based on these findings, we developed a universal checklist of guideline recommendations as a practice tool for primary care professionals and future guideline developers. An international team call for a universal guideline for primary-care practitioners who help patients to stop smoking. Although many nations have such guidelines, no studies have examined whether these guidelines are consistent with the current evidence. Marjolein Verbiest at the National Institute for Health Innovation, The University of Auckland, New Zealand, and co-workers of the International Primary Care Respiratory Group and the National Centre for Smoking Cessation and Training reviewed, evaluated and compared 26 national guidelines. Almost all guidelines place importance on identifying smokers, advising them to quit and providing behavioural and medication-based support. However, there were discrepancies in the support offered, which could be due to different interpretations of evidence, costs of medication and cultural differences. The authors offer a checklist for primary care that can inform future universal guidelines suitable for primary care. Smoking prevalence is, however, substantially higher among males (36%) than females (8%) , general practice is the optimum environment for the identification and referral of smokers to take place. For example, in the UK almost 300 million smoking cessation consultations a year and around 90% of all National Health Service contacts take place in a general practice setting.12 Evidence for the effectiveness of interventions in this setting is well established13; rates of smoking abstinence are increased when health professionals identify smokers, prompt quit attempts,13 and provide assistance to quit smoking, including pharmacotherapy.15In many countries, smokers are most often identified, advised and offered quit support in a primary care setting.16 although the effects depend upon factors such as guideline quality, context and professional experience.19 Guidelines also offer an opportunity for raising the profile of smoking cessation and facilitate the implementation of the WHO Framework Convention on Tobacco Control.20Guidelines in which this evidence is communicated to health professionals can be effective in improving clinical practice,This study aims to assess the nature and extent of the current national guidelines available for the treatment of tobacco dependence in primary care. As such, the objective of the study is threefold and includes an assessment of: (1) the parties involved in the development of these guidelines, (2) the recommendations made within these guidelines for primary care and (3) how these recommendations correlate with each other (consistency) and with the state-of-the-art evidence of what is effective .We identified a total of 43 guidelines from 39 countries. After initial review, we excluded three guidelines. Reasons for exclusion were that they either merely presented minimum specifications for a national smoking cessation service (Northern Ireland and Spain), or they focused on treatment of asthma and chronic obstructive pulmonary disease (COPD) other than on smoking cessation (Greece). Of the remaining 40 guidelines, only 16 guidelines were initially available in English. Authors of another 10 guidelines were willing to complete our coding framework on the key recommendations made within the guidelines, resulting in a total of 26 guidelines that were fully assessed for this review  were involved in the development of the majority of the included guidelines (19/26); in six guidelines the lead-author of the guideline was a GP. In three guidelines, the development was led by an association of GPs and in ten guidelines at least one GP was involved in the development among many other authors. Only four guidelines were developed without any involvements of GPs and for the remaining three guidelines it was not possible to identify the profession of the authors and, therefore, GP involvement remains unclear .n = 22), two focused specifically on smoking cessation during pregnancy (France and Canada), one on smoking cessation among COPD patients (Germany) and one on smoking cessation among perioperative patients (France). Table The majority of the included guidelines focused on smoking cessation in the general population (n = 22), we were able to categorise recommendations into 12 intervention types recommended for primary care:Each patient’s smoking status should be identified and recorded (20/22);All smokers should be given brief advice to stop (20/22);Smokers’ motivation to quit should be assessed (17/22);All smokers wishing to stop should be offered assistance (22/22);All smokers wishing to stop should be offered/encouraged to use pharmacotherapy (22/22);All smokers wishing to stop should be offered behavioural support (19/22);Self-help materials should be offered as part of tobacco dependence treatment (12/22);Smoking abstinence should be evaluated following treatment (5/22);The ‘5A’s Framework’ should be used to guide brief intervention (16/22);The ‘ABC Framework’ should be used to guide brief intervention (3/22);Health-care professionals should be trained in delivering tobacco dependence treatment (13/22);Hypnotherapy and acupuncture are not effective smoking cessation treatments (6/22).With regard to guidelines that focus on the treatment of tobacco dependence in the general population . Moreover, several countries developed specific guidelines for the treatment of tobacco dependence among such subpopulations. The most frequently mentioned recommendations across guidelines are described in this section  can be used to assist smoking cessation attempts made by pregnant women. In several countries, intermittent-dosage forms of NRT  are recommended as a preferred pharmacotherapy over patches , New Zealand, Portugal and India). In contrast, guidelines from Norway, Scotland, the United States, Japan and Kyrgyzstan all recommended that NRT should not be given to pregnant women.Guidelines from five countries made recommendations for black and minority ethnic (BME) groups: Canada, New Zealand, Norway, Portugal and Sweden. All recommend that wherever possible, culturally appropriate smoking cessation support should be offered. These guidelines also recommend that health-care workers should receive additional training in delivering smoking cessation support to BME groups.In total, 16 guidelines made specific recommendations for children and adolescents. Overall, these include: (1) information about tobacco use among children and adolescents should be obtained on a regular basis, (2) children and adolescents should be counselled to encourage abstinence and (3) parents who smoke should be offered smoking cessation support in order to limit children's exposure to secondhand tobacco smoke. Of these guidelines, 11 guidelines recommend that NRT could be offered to adolescents who show evidence of nicotine dependence. In contrast, several other guidelines recommend that pharmacotherapy should not be offered to children and adolescents .Eleven guidelines made specific recommendations for smokers with a mental illness or other substance addiction. Most commonly, guidelines recommend to offer counselling that incorporates all known effective components and to carefully monitor smoking cessation in which medication dosages should be adjusted if necessary. The types of pharmacotherapy recommended for the treatment of tobacco dependence among these patients differed across countries Table .In total, 15 guidelines made specific recommendations for people with a chronic illness. Most guidelines recommend that NRT is safe to use in people with stable cardiovascular disease. However, NRT should be used with caution in patients with unstable cardiovascular disease . Additionally, several guidelines also recommend bupropion as a safe treatment for nicotine dependence in people with cardiovascular disease .Table Recommendations concerning pregnant women, BME groups and people with mental illness were assigned an evidence rating ‘B’ (supported by reasonable evidence). People with other addictions were assigned the rating ‘C’ (supported by expert opinion only). For the purposes of our review, mental illness and other substance addictions were placed in a single group to reflect the content of the national guidelines; these groups are treated separately in the Service and Delivery Guidance. Children and adolescents were not assigned an evidence rating as a priority group. Rather, the Service and Delivery Guidance assigned a rating of ‘I’ (insufficient evidence to make a recommendation) to stop smoking interventions and ‘B’ to prevention and tobacco control. The use of NRT for pregnant women was assigned the evidence rating ‘C’. The use of pharmacotherapy for children and adolescents, people with cardiovascular disease and people with mental illness were not assigned an evidence rating as these were not included in the Service and Delivery Guidance.This study aimed to assess: (1) the parties which were involved in the development of national guidelines for smoking cessation, (2) the recommendations that are made for primary care within these guidelines and (3) how these recommendations correlate with each other (consistency) and with the state-of-the arts evidence of what is effective .In our study, 26 guidelines from 22 countries were included in the analyses. Four of these guidelines focused on the treatment of tobacco dependence among specific subpopulations: pregnant smokers (Canada and France), perioperative (France) and COPD patients (Germany). Most guidelines were produced/commissioned by governmental organisations, followed by medical societies, multiple organisations, research networks and research centers. Although most guidelines were developed in collaboration with GPs, only a minority of the guidelines was developed by a GP as the lead author.Overall, recommendations that focused on the treatment of tobacco dependence in primary care among the general population corresponded well with each other across guidelines. The majority of these guidelines recommended that smokers should be identified, be offered a brief advice to quit, be assessed for motivation to quit, and be offered assistance to quit with behavioural support and pharmacotherapy. Also, the majority recommended the use of the ‘5A’s’ framework to guide brief intervention.We found more inconsistency across details of specific recommendations. For example, the specific content  and delivery format  of behavioural support differed greatly throughout guidelines. These inconsistencies can be partly explained by the differences in the services available at the country level .Other inconsistencies among recommendations were related to the provision of pharmacotherapy for smoking cessation among the general population. Excluding the Canadian and Scottish guidelines, who did not make recommendations relating to the use of pharmacotherapy, NRT was recommended by all guidelines. Although a combination of NRT products was recommended by a majority of the guidelines (19/22), varenicline was recommended by only 15 of 22 guidelines. Recommendations to the use of nortriptyline (9/22), a combination of NRT and bupropion (9/22), and clonidine (4/22) were less common. The discrepancies between these recommendations may be due to differences in medication licensing across countries, or to a difference in access or interpretation of the current available scientific evidence. Also, we argue that costs of medication could be a possible barrier in certain countries for uptake of certain recommendations into the guideline.Finally, recommendations for specific subpopulations were also less consistent throughout guidelines. For example, 15 guidelines recommended that NRT could be used when needed for pregnant women, while five guidelines recommended that NRT should not be used during pregnancy. Similarly, 10 guidelines recommended that NRT could be used for children and adolescents, while four recommended that NRT should not be used in this group. Possibly these inconsistencies are related to the level of evidence available for these subpopulations, which mostly undefined and otherwise limited to level B or lower Table . CulturaThe current study is one of the first to systematically identify and analyse the nature and extent of a large number of practice guidelines for the treatment of tobacco dependence in primary care. Moreover, it is the first to compare guideline recommendations across countries and with state-of-the-art evidence for both the general population and specific subpopulations.However, some limitations need to be taken into account when interpreting the results. Firstly, we only included guidelines that were written or translated into English. Non-English guidelines that could not be translated were only included if the guideline authors were willing to provide the necessary information, and data was thus not derived directly from these guidelines.Secondly, several guideline recommendations made in the Service and Delivery Guidance were not mentioned in any of the included national guidelines. These include types of pharmacotherapy treatment (nicotine-assisted reduction to stop—evidence rating B), unlicensed stop smoking treatment , and the identification of routine and manual workers as an additional priority group (evidence rating B). The lack of uptake of recommendations regarding electronic cigarettes can be explained by their recent rapid rise on the market, which mainly started only after the development of most of the included guidelines (most of them dating from 2011 and earlier). It illustrates a need of frequent updates of the national guidelines. The lack of inclusion of other recommendations in the identified guidelines may indicate a translational gap of the latest evidence. We, therefore, recommend future studies that aim to identify this translational gap, raise awareness among guideline developers and trigger them to update guidelines and include the latest evidence.Thirdly, we were unable to rate the quality of several recommendations within the guidelines because they did not match with the recommendations within the Service and Delivery Guidance . This was likely due to vague descriptions of the recommendations within the guidelines.Finally, although we managed to analyse the consistency and validity of the guideline recommendations, this study does not provide insight into how these guideline recommendations are currently being implemented in each country.Results of our study indicate that, although some consensus on smoking cessation recommendations already exist, there is room for improvement with regard to the inconsistencies we found. Therefore, we suggest that the development of an international smoking cessation guideline for primary care, drawing upon the latest evidence and written by international clinical, policy and academic experts could provide a template to optimise future national guidelines. Based on our study findings, we developed a checklist of recommended smoking cessation intervention components as a ‘tool for practice’ for primary care professionals . AlthougOur study shows that there is almost universal agreement between guidelines regarding the need in primary care to identify smokers, to offer some form of advice to quit and to offer behavioural and pharmacological support to quit. Discrepancies between guideline recommendations were greatest for these latter interventions, which are likely due to different interpretations of the evidence and/or differences in the contextual health environments of countries. International primary care guidance for smoking cessation that is dynamic, (co-) written by primary care experts and drawing from the latest evidence would be a very useful resource for clinicians and policy makers to develop and optimise national guidelines.www.treatobacco.net). Additional search methods included the consultation of international colleagues within the field of tobacco dependence treatment. The initial search was performed in December 2014 and was updated in September 2015. Only the most recent version of each available national guideline was included. If multiple guidelines for specific subpopulations  were available for one country, all those guidelines were included.To identify national guidelines for smoking cessation, we used an independent online resource which hosts a compilation of national treatment guidelines from across the world . In case the communication with an author resulted in the availability of a more recent version of the guideline or to a version with a more specific focus on primary care, this version of the guideline replaced the initially included guidelines of this country.We excluded guidelines where smoking cessation was only part of a guideline on a different topic .http://data.worldbank.org/about/country-and-lending-groups).From each country’s guideline we extracted information regarding the lead authors, the year of publication, and the lead organisation(s) involved in commissioning and/or producing the guideline . We also extracted information on funding sources for guideline development and on the development methodology. We classified the country as a high-, middle- or low-income country based on the information from the World Bank income group  the text was explicitly specified as a recommendation and (2) the recommendation was made specifically for a primary care setting or primary healthcare practitioner, or was relevant to either one of these. Recommendations were extracted in their smallest, irreducible form. For example, a recommendation might state that “pregnant women should only be offered NRT if they are unable to quit otherwise, and intermittent forms of NRT are preferable to patches in this population”. We recorded this as two separate recommendations: (1) “pregnant women should only be offered NRT if they are unable to quit otherwise” and (2) “intermittent forms of NRT are preferable to patches for pregnant women”. Recording the recommendations in this way facilitated the comparison of recommendations across guideline documents.To ensure consistency in the extraction of recommendations, two researchers (K.S. and G.A.) independently extracted the recommendations from two pre-selected guidelines. These two researchers discussed any discrepancies in their extraction and resolved them through discussion or through arbitration with the third researcher (A.Mc.). After having agreed upon a consistent approach to extraction, K.S. and G.A. extracted both 50% of the remainder guideline recommendations.To establish how consistent the recommendations were internationally, we compared each of the individual national recommendations and contrasted them against all others. Those recommendations that were identical, or very similar, were assigned to categories . We used a ‘pile-sort’ method in order to establish this categorisation. Each individual recommendation was printed out on a separate piece of card along with a unique identifier. The cards were sorted independently by two researchers (K.S. and G.A.) into piles of identical/very similar individual recommendations. Each researcher assigned a category label to individual piles, and subsequently met to discuss the categorisations. Disagreements in categorisation were discussed with the third author (A.Mc.) until resolved, with category labels being finalised and agreed upon at this stage.21 This publication makes recommendations for how stop smoking services in the UK should be commissioned, delivered and monitored. Most importantly, it includes an updated evidence review of a number of recommendations for the treatment of tobacco dependence, including behavioural support, pharmacotherapy and the treatment among specific subpopulations.To establish how well the recommendations in each of the guidelines correspond to state-of-the-art evidence, recommendations were cross-checked against the most recent English Service and Delivery Guidance for local Stop Smoking Services.22 The SIGN system includes five ratings: “A” (the recommendation is supported by good/strong evidence), “B” , “C” (the recommendation is supported by published expert opinion only), “I” (there is insufficient evidence to make a recommendation) and “✓” (good practice point in the opinion of the guidance development group).Each recommendation within the included national guidelines was assigned an evidence rating using the SIGN rating system.We developed a checklist of components as a practice tool for primary care professionals based on: (1) the analysis of recommendations in the identified national guidelines, (2) the consistency and validity of these recommendations and (3) experts’ opinions from the International Primary Care Respiratory Group. Supplementary AppendixSupplementary Material


For genomic medicine research to be fully translated into clinical care, it is critical for researchers to engage stakeholders who ultimately regulate the use of genomic technologies and therapeutics within healthcare practice. Herein, we describe an example of how this might work. Notwithstanding the historical hype of genomic medicine , there iProducing the evidence is but the first step toward implementation of genomic medicine. Efforts to engage with target communities and patient stakeholders are now a frequent component of clinical genomics research protocols . Successhttps://cser-consortium.org/), Implementing Genomics in Practice , and Clinical Genome Resource . A primary aim of these public-funded projects is to elucidate whether and how genomic technologies can be utilized to improve patient outcomes and optimize healthcare for the population and individual. In other words, much of research is focused on the analytic and clinical validity as well as clinical utility of genomic technologies. Researchers’ definitions of these concepts may vary considerably from that of regulatory groups, however. Rather than producing scientific evidence with the assumption that it will meet the evidence requirements for policy development, it is important to engage individuals making both coverage and regulatory decisions to understand the evidence needs and assessment processes of these stakeholders. Such engagement may facilitate understanding between researchers and regulators and help ensure study designs and outcomes achieve their intended goals.Currently, many of the research programs producing the growing body of evidence for healthcare applications of genomic medicine are associated with or directly linked through large-scale consortia funded by the National Human Genome Research Institute (NHGRI), under the purview of the National Institutes of Health (NIH). Examples include Clinical Sequencing Exploratory Research  in genomic medicine and their evidence/data needs for policy development and decision-making. Key questions may include the following: (1) What is each regulatory group’s role(s) in genomic medicine and how can research efforts contribute to their specific aims? (2) What types of evidence are needed to support each group’s regulatory decisions? (3) Do the research aims align with the evidentiary needs and/or can the study protocol be amended in response to the required data/evidence? (4) Are there mutual benefits to establishing a relationship? (5) Is it feasible to build and sustain a relationship of transparency, communication, and dissemination?The US Food and Drug Administration (FDA) proactively expressed interest in ClinGen’s approach to curation of genes and variants. Based on this interest, ClinGen sought to establish a process of engagement with the FDA as the first stakeholder. The interactions to date between ClinGen researchers and the FDA described here are presented as an example of how such engagement could work. To facilitate the development of this stakeholder relationship, a small workgroup of key program contacts/liaisons from the relevant organizations  met monthly via conference calls. At the outset, roles and interests were defined as summarized in Fig. The open dialogue generated by establishing this relationship has provided a means through which to clarify and/or contextualize confusing or misunderstood jargon and concepts in terms of the research and the regulatory processes. Acknowledging the need for shared learning is essential to building a mutually respectful relationship. The formal development of a dedicated liaison and workgroup tasked with engaging regulatory groups has allowed broader contextualization of overarching goals and minimized the possibility for ambiguous messages or distorted perceptions.The relationship has allowed ClinGen researchers to understand the potential uses of ClinGen resources for FDA regulatory purposes and to align curation processes to produce a resource consistent with FDA draft guidelines on regulatory-grade databases . The accThrough this experience, it has become evident that engaging with regulatory stakeholders, such as the FDA, can not only be helpful in the development of the ClinGen resource, but is essential to guiding appropriate utilization of the generated evidence to support improved health through genomic medicine Fig. . This exThe need for continued and increased regulatory engagement activities among genomic medicine research is great, and the potential impact of these types of interactions to further clinical genomic medicine may be well worth the effort.