Supplemental Digital Content is available in the text The safety and effectiveness of using the direct thrombin inhibitor bivalirudin during transcatheter coronary interventional procedures remains uncertain.This study aimed to systematically assess anticoagulation with bivalirudin alone or bivalirudin plus glycoprotein (GP) IIb/IIIa inhibitors  in patients undergoing percutaneous coronary intervention (PCI) procedures by a meta-analysis of randomized controlled trials (RCTs).Systematical searches of the MEDLINE, EMBASE, and Cochrane databases were conducted. RCTs comparing bivalirudin-based anticoagulant therapy with a comparable heparin therapy in patients undergoing PCI were eligible. Risk ratios (RRs) with 95% confidence intervals (CIs) served as summary statistics.P > 0.05). Notably, bivalirudin-based therapy showed a highly significant 34% decrease in the incidence of major bleeding  and a 28% reduction in the need for blood transfusion . Meta-regression analyses demonstrated that additional administration of GP IIb/IIIa receptor inhibitors (P = 0.01), especially eptifibatide (P = 0.001) and tirofiban (P = 0.002), was likely to increase the major bleeding risk associated with bivalirudin.A total of 38,096 patients from 17 RCTs were randomized to the bivalirudin group  or heparin group  in the meta-analysis. No significant differences in death, myocardial infarction or reinfarction, ischemia-driven revascularization, or in-stent thrombosis were observed between the 2 groups (all Bivalirudin, in comparison to heparin, is associated with a markedly lower risk of major bleeding, and the additional use of GP IIb/IIIa inhibitors may weaken this benefit. A previous meta-analysis compared bivalirudin mono- or bivalirudin-based  anticoagulant therapy versus heparin-based anticoagulation  in patients undergoing percutaneous coronary intervention (PCI).6 However, the influence of the adjunctive use of GP IIb/IIIa inhibitors and other important clinical factors on ischemic and bleeding endpoints was not defined in the study. Recently, 2 meta-analyses investigated the clinical utility of bivalirudin versus UFH during PCI without planned use of GP IIb/IIIa inhibitors7 and only with the use of GP IIb/IIIa inhibitors,8 respectively. Neither study comprehensively showed the efficacy and safety profile of bivalirudin in patients undergoing coronary interventional procedures. Additionally, more recently reported results of several new trials and longer-term observations from previous trials can potentially contribute to the development of antithrombotic therapy during the procedures.12 We therefore performed a meta-analysis of randomized controlled trials (RCTs) to systematically evaluate the efficacy and safety of bivalirudin mono- or bivalirudin-based anticoagulant therapy in patients undergoing PCI. Meanwhile, the effects of additional use of GP IIb/IIIa inhibitors and other clinical factors on ischemic and bleeding outcomes were also investigated in the meta-analysis.In patients undergoing transcatheter procedures for the treatment of coronary diseases, the optimal antithrombotic regimens for maximizing clinical efficacy and minimizing the risk of bleeding complications have been widely investigated over the past decade. The relatively new direct thrombin inhibitor bivalirudin, which offers a low bleeding risk, might be promising as an alternative to unfractionated heparin (UFH), which is routinely used during coronary interventional procedures. Before the widespread use of clopidogrel or prasugrel pretreatment, bivalirudin was associated with lower incidences of periprocedural major bleeding as well as ischemic outcomes compared to UFH.A computerized literature search was conducted of studies published from January 1990 through January 2015 in the MEDLINE, EMBASE, and Cochrane Central Register of Controlled Trials databases using the following search terms: bivalirudin, hirulog, heparin, low-molecular-weight heparin, unfractionated heparin, UFH, coronary artery/heart disease, myocardial infarction, acute coronary syndrome, unstable angina, angioplasty, percutaneous coronary intervention, PCI, invasive strategy, randomized, and human. In addition, an extensive manual searching was also performed using cross-references from the eligible articles and relevant reviews. The search was restricted to English-language literature.RCTs were eligible for inclusion if they compared the efficacy or safety of bivalirudin mono- or bivalirudin-based anticoagulant therapy with comparable heparin therapy during PCI and reported clinical outcomes of interest. Bivalirudin/heparin-based regimens were defined as anticoagulation with bivalirudin/heparin (UFH or low-molecular-weight heparin) plus planned or provisional GP IIb/IIIa inhibitors . Subgroup analyses within the eligible trials were excluded. Moreover, articles published before the year 2000 and those in the form of study designs, editorials, and reviews also were excluded.13 a numerical score between 0 and 5, was used to qualitatively assess the quality of the included studies.Two investigators (JL and SY) reviewed all the citations in duplicate to identify eligible studies and independently conducted data extraction and quality assessment using a standardized approach. Data regarding ischemic outcomes  and bleeding complications  were extracted from each of the eligible studies. The reviewers resolved differences through consensus, and any disagreements were resolved by the principal investigator of the present study (JJ). All eligible trials were assessed by the following quality criteria recommended by the Cochrane Collaboration: sequence generation of the allocation; concealment of allocation; blinding of participants, personnel, and outcome assessors; use of intention to treat analysis; and description of withdrawals and dropouts. In addition, the Jadad scale,14 Heterogeneity across studies was quantified using the I2 statistic. I2 values greater than 25%, 50%, and 75% were considered evidence of low, moderate, and severe statistical heterogeneity, respectively. Sensitivity analyses, in which the pooled estimates were recalculated omitting 1 study at a time, were conducted to assess the impact of individual studies on the summary estimate of effect. Subgroup analyses were performed to assess the impacts of anticoagulant regimens, clinical settings, invasive strategies, and follow-up duration on overall estimates. Meta-regression analyses were also performed to determine the influences of clinical and demographic factors on the overall results. We assessed publication bias using a Begg funnel plot.15 Pooling analyses were performed with the RevMan 5.2 software , and meta-regression analyses were conducted with STATA 10.0 software . The results were considered statistically significant at P < 0.05 (2-sided). The study was performed in compliance with the Quality of Reporting of Meta-analyses (QUOROM) guidelines.16We used risk ratios (RRs) with 95% confidence intervals (CIs) to express the combined results of individual studies. The pooled effects were calculated according to the Mantel–Haenszel random effects model. For studies with no event of interest in a treatment group, 1.0 was added to all cells for continuity correction.36 involving 17 RCTs were eligible for the final analysis ],28 and the prevalence of previous myocardial infarction ranged from about 11% [the harmonizing outcomes with revascularization and Stents in acute myocardial infarction (HORIZONS-AMI)]23 to 45% (NAPLES).28 Transcatheter procedures were performed in the individual trials mainly through transfemoral access except for HEAT-PPCI trial10 (Table http://links.lww.com/MD/A342).A total of 38,096 patients included in the present study was randomized to the bivalirudin-treatment group  or UFH-treatment group n = 19,218; 50.4%). The study and demographic characteristics are shown in Tables 18; 50.4%P = 0.85; I2 = 42%). Moreover, the neutral finding was also consistently found in subgroup analyses regardless of anticoagulant regimens, clinical settings, or follow-up duration .The pooled analysis showed that bivalirudin was associated with a similar rate of the composite endpoint as compared with UFH . However, when the intracoronary stenting and antithrombotic regimen–rapid early action for coronary treatment (ISAR-REACT) 4 study12 or the evaluate the relative protection against post-PCI microvascular dysfunction and post-PCI ischemia among anti-platelet and anti-thrombotic agents-thrombolysis in myocardial infarction-30 (PROTECT-TIMI-30) study29 were removed from the subgroup of bivalirudin alone or bivalirudin plus provisional GP IIb/IIIa inhibitors versus UFH plus planned GP IIb/IIIa inhibitors, we found that the intrasubgroup difference became statistically significant . Nevertheless, this process did not markedly influence the overall estimate. Moreover, in subgroup analyses and meta-regression analyses, the predefined clinical factors did not have statistically significant influences on the pooled result (Table http://links.lww.com/MD/A342).Overall, 559 of 18,878 patients died from all causes in the bivalirudin-treatment group compared with 584 of 19,218 patients in the UFH-treatment group, with no significant difference between the groups  compared with UFH. Moreover, the benefit of bivalirudin in lowering the risk of major bleeding and subsequent need for blood transfusion was statistically significant in the subgroup of bivalirudin alone or bivalirudin plus provisional GP IIb/IIIa inhibitors versus UFH plus planned GP IIb/IIIa inhibitors (P < 0.01). Furthermore, the beneficial effect of bivalirudin was consistently shown in the subgroup analyses stratified by follow-up duration .Bivalirudin showed a highly significant 34% decrease in the incidence of major bleeding .There was no evidence for publication bias among the included studies. Funnel plots were generated for the composite endpoint, all-cause death, and major bleeding, and essential symmetries were found. Begg tests based on these data did not show any statistical significances , bivalirudin treatment might result in a bleeding risk almost identical to that of UFH therapy, and this result was also identified by our subgroup analyses based on anticoagulant regimens.The combination of a potent anticoagulant  with antiplatelet therapy  is routinely used during transcatheter coronary interventional procedures. Recently, the use of bivalirudin as a specific and reversible direct thrombin inhibitor is gradually increasing in order to overcome the limitations encountered with heparin during coronary interventional procedures.39 Major bleeding may be a powerful predictor of death or poor prognosis in patients undergoing PCI.40 The HORIZONS-AMI study,23 a prospective randomized trial involving patients with ST-segment elevation myocardial infarction undergoing primary PCI, demonstrated that bivalirudin plus provisional GP IIb/IIIa inhibitors improved the event-free survival at 30 days, mainly due to a significant reduction in major bleeding as compared with that experienced with UFH plus planned GP IIb/IIIa inhibitors. However, the present study did not identify a relationship between bleeding events and ischemic outcomes of all-cause death, myocardial infarction or reinfarction, ischemia-driven revascularization, or in-stent thrombosis. Nevertheless, relative to heparin, bivalirudin did not significantly increase the incidence of composite and individual ischemic outcomes. Moreover, the neutral effect on ischemic outcomes remained highly consistent in our subgroup analyses and meta-regression analyses. Additionally, the present study did not show a pronounced additional influence of GP IIb/IIIa inhibitors on clinical prognosis.Presently, achieving a balance between ischemic outcomes and bleeding events is essential in the field of antithrombotic therapy. Emerging evidence indicates the independent relationship between major bleeding with or without blood transfusion and subsequent death.42 Therefore, it remains uncertain whether the use of prasugrel or ticagrelor could change the findings regarding the effect of bivalirudin versus UFH in patients undergoing PCI. In addition, as in other nonpatient-level meta-analyses, the present study utilized summarized published events for each trial as opposed to individual patient data. Nevertheless, the findings in the meta-analysis were generated based on a large-scale population from RCTs, and appropriate meta-analytic techniques with random-effect models were used to pool the effect variables. Moreover, our overall analyses were not influenced by publication bias, and sensitivity analysis further confirmed the credibility of the overall estimates.Several limitations of the meta-analysis should be considered. The majority of the included trials did not provide data regarding the precise dose of bivalirudin used. As a result, we did not consider the impact of the bivalirudin dose on its efficacy and safety endpoints, and this meta-analysis still could not confirm whether bivalirudin therapy had a dose-specific effect on ischemic and bleeding outcomes. Moreover, all of the included trials involved the use of clopidogrel, rather than prasugrel or ticagrelor, which are more effective antiplatelet agents for reducing the cardiovascular death/stroke/infarction rate, according to the recommendation for oral dual antiplatelet therapy.In summary, bivalirudin was found to be superior to UFH for reducing the risk of major bleeding and need for blood transfusion, with no increase in the incidence of ischemic outcomes, in patients undergoing PCI. Notably, the adjunctive use of GP IIb/IIIa inhibitors during PCI may weaken the favorable effect of bivalirudin on lowering bleeding risk.This work was supported by a grant from the National Natural Science Foundation of China (NSFC) (No. 81470300).


There is currently no mechanism in place outside of government to provide rapid syntheses of the best available research evidence about problems, options and/or implementation considerations related to a specific health system challenge that Canadian health system decision-makers need to address in a timely manner. A ‘rapid-response’ program could address this gap by providing access to optimally packaged, relevant and high-quality research evidence over short periods of time (i.e. days or weeks).www.healthsystemsevidence.org) and drew on an existing analytical framework for how knowledge-brokering organizations can organize themselves to operationalize the program features. The issue brief was then used to inform a half-day stakeholder dialogue about whether and how to develop a rapid-response program for health system decision-makers in Canada. We thematically synthesized the deliberations.We prepared an issue brief that describes the best available research evidence related to the problem, three broad features of a program that addresses the problem and implementation considerations. We identified systematic reviews by searching for organization-targeted implementation strategies in Health Systems Evidence (We found very few relevant systematic reviews but used frameworks and examples from existing programs to 1) outline key considerations for organizing a rapid-response program,, 2) determine what can be done in timelines ranging from 3 to 10 and 30 business days, and 3) define success and measure it. The 11 dialogue participants from across Canada largely agreed with the content presented in the brief, but noted two key challenges to consider: securing stable, long-term funding and finding a way to effectively and equitably manage the expected demand. Recommendations and suggestions for next steps from dialogue participants included taking an ‘organic’ approach to developing a pan-Canadian network and including jurisdictional scans as a type of product to deliver through the program (rather than only syntheses of research evidence).Dialogue participants clearly signalled that there is an appetite for a rapid-response program for health system decision-makers in Canada. To ‘organically’ build such a program, we are currently engaging in efforts to build partnerships and secure funding to support the creation of a pan-Canadian network for conducting rapid syntheses for health system decision-makers in Canada.The online version of this article (doi:10.1186/s13643-015-0009-3) contains supplementary material, which is available to authorized users. A ‘rapid-response’ program could fill this gap in situations where health system decision-makers need support with accessing and synthesizing optimally packaged, relevant and high-quality research evidence over days or weeks .A gap exists in efforts to support the use of research evidence between ‘self-serve’ approaches  or to researchers with whom they have an established relationship. Alternatively, they may turn to one of very few formalized rapid-response programs in Canada, if their question is about drugs and other health technologies , if theyDepending on the timelines provided, products provided through a rapid-response program might include a listing of relevant research evidence (if the timeline is very short), a brief synthesis of the results (if the timeline permits) or a more detailed summary (if given a longer period of time). Some rapid-response programs also supplement these products by conducting briefings with decision-makers based on the research evidence identified . One addWe see rapid syntheses as being distinct from a rapid systematic review  in several ways. First, those requesting a rapid synthesis typically set the timeline within which it needs to be prepared . In contrast, a rapid review is typically a comprehensive systematic review conducted in a condensed timeline (e.g. 6 months), rather than a more standard timeline like 1 or 2 years. Second, the nature of the questions addressed can take many forms, and relate to a problem, options or implementation considerations, as opposed to a rapid review of the effects of a single option. Lastly, rapid syntheses typically include existing systematic reviews and occasionally single studies, whereas rapid reviews focus on single studies.We recently created a rapid-response program at the McMaster Health Forum that provides a summary  of research evidence based on a systematic search for information about problems, options and/or implementation considerations related to a specific health system challenge . At presConvening stakeholder dialogues is one promising approach to addressing health system issues . Stakeholder dialogues are designed to support evidence-informed decisions by pairing the best available research evidence with a robust deliberative process that gives stakeholders the opportunity to bring their tacit knowledge and their own views and experiences to bear on a pressing health system problem, three options to address it and implementation considerations . GeneralWe describe below our methods for preparing the issue brief and convening the dialogue. For those who are interested in more detail, a separate paper outlines the overall approach for developing evidence/issue briefs and convening stakeholder dialogues .We prepared the issue brief through four steps: 1) convening a steering committee; 2) developing and refining a terms of reference for an issue brief; 3) identifying, selecting, appraising and extracting key findings from the research evidence about the problem, program features and implementation considerations; and 4) synthesizing the findings in the form of an issue brief. Our steering committee was comprised of representatives from partner organizations and stakeholder groups and provided guidance throughout the process as well as literature that would be relevant to preparing the brief. The terms of reference for the issue brief provided a preliminary outline to clarify the problem, framed three options for addressing it  and identified implementation considerations. We developed and iteratively refined the terms of reference in collaboration with our steering committee.www.healthsystemsevidence.org), which is a continuously updated database, that contains (as of January 2015) 4,200 systematic reviews and more than 2,200 economic evaluations of delivery, financial and governance arrangements within health systems. For more information about the sources searched to identify documents included in Health Systems Evidence, see http://www.healthsystemsevidence.org/why-use-it.aspx. The reviews and economic evaluations were identified by searching the category for organization-targeted implementation strategies in Health Systems Evidence. The searches were reviewed by one of us (MGW) with another (JNL) checking reviews for which there was uncertainty about their inclusion. For each systematic review, we extracted the focus of the review, key findings, the last year the literature was searched, the methodological quality  by searching Health Systems Evidence  the problem, 2) three broad features of a program that could address the problem and 3) possible barriers to implementation of the options at the levels of individuals, providers, organizations and systems. The brief was then merit reviewed by a small number of policymakers, stakeholders and researchers to ensure its system relevance and scientific rigour.We worked collaboratively with the steering committee to identify health system stakeholders . For each dialogue we convene, we invite those who have the ability to 1) bring unique views and experiences to bear on the challenge and learn from the research evidence and from others’ views and experiences and 2) champion within their respective constituencies the actions that will address the challenge creatively. Participants were principally identified from suggestions provided by members of the steering committee, which we supplemented by reviewing government directories and websites of relevant organizations.per se, but rather provide a space where diverging opinions could be shared and discussed and to identify possible synergistic efforts among stakeholders. In addition, the dialogue followed the Chatham House Rule , nor that of any other participant, may be revealed’) [The dialogue was facilitated by one of us (JNL) and included deliberations about the same topics addressed in each of the three sections of the brief . These were followed by a fourth deliberation about the next steps that could be taken. Participants were sent the brief 2 weeks before the dialogue and were requested to read it before arriving. The goal was not to aim for consensus vealed’) . Lastly,www.mcmasterhealthforum.org).We present below a summary of the key findings from the issue brief and the key themes of the deliberations. For those who are interested in more information about each, the issue brief  and diallimited number of formalized supports in place to provide decision-makers with rapid syntheses of the best available research evidence about problems, options and/or implementation considerations related to health system challenges (i.e. providing the right product at the right time);inconsistent interaction between researchers and decision-makers to ensure that the priorities of decision-makers are addressed (i.e. having the right people developing products on the right issues); anduncertainty about what success looks like given the long chain of potential causal relationships between an intervention/program (e.g. a rapid-response program) and relevant outcomes (e.g. whether decision-makers’ needs are met and/or their use of research evidence).Despite a range of approaches that are available in Canada to support health system decision-makers’ efforts to find and use research evidence efficiently, significant barriers and challenges exist. We identified the following as key components of the problem:To promote discussion during the stakeholder dialogue about the pros and cons of a potentially viable approach to developing a rapid-response program for health system decision-makers in Canada, we identified three broad program features. The three program features were developed and refined through consultation with the steering committee and include activities related to 1) organizing a rapid-response program, 2) establishing what can be done in what timelines and 3) defining success and measuring it.We did not identify any systematic reviews related to organizing a program to support the use of research evidence (program feature 1). What we do know is that it is essential to match form to function when organizing such a program . To do sgives policymakers, stakeholders and researchers an explicit role in its governance and ensures they exercise their role with transparency and objectivity;has and enforces rules that ensure independence in how health system information is produced, packaged and shared and that address conflicts of interest;grants the director the authority needed to ensure the accountability of the entire organization to its knowledge-brokering mandate;ensures an appropriate size, mix and capacity of staff with knowledge-brokering responsibilities;ensures an appropriate size of budget and an appropriate mix of funding sources for knowledge-brokering activities;has an explicit approach to prioritizing knowledge-brokering activities and accepting commissions or requests from policymakers and stakeholders;is located within another organization or network that supports its knowledge-brokering activities;collaborates with other knowledge-brokering organizations in its knowledge-brokering activities; andestablishes functional linkages with policymaking and stakeholder organizations.The same policy summary also provides a set of nine criteria for assessing organizational models for knowledge brokering, which were derived from a multi-method study consisting of a systematic review, scoping review of knowledge-brokering mechanisms and models, website review of existing knowledge-brokering organizations, sites visits and case studies. The summary was designed for those involved in establishing or leading organizations that support the use of research evidence in developing health policy . The folIn Table We also documented features of several existing rapid-response programs that target, at least in part, health system decision-makers. We identified the programs in collaboration with our steering committee as well as from reviews of mechanisms designed to promote the use of research evidence by policymakers ,15 and fFor program feature 2 (deciding what can be done in what timelines), we identified three different timelines in which a request can be made to the rapid-response program , which we summarize in Table For program feature 3 (defining success and measuring it), we identified four short- and medium-term areas where the success of a rapid-response program can be measured using a brief survey administered following receipt of a rapid synthesis and short interviews approximately 6 months later. The four areas of success include 1) program organization , 2) final product (e.g. was the synthesis presented in way that was easy to understand?), 3) influence on behavioural intention to use research evidence, and 4) whether and how the synthesis was used. In Table The only systematic reviews we identified for this program feature related to the use of the theory of planned behaviour. One older low-quality systematic review and an older overview of systematic reviews from the psychology field found that the theory explains approximately 39% of the variance in intention and about 27% of the variance in behaviour ,21. Linksystem leaders increasingly working collaboratively to advance the timely translation of research evidence to improve the financing, sustainability and governance of the healthcare system  (program feature 1);the many lessons that have been learned from existing rapid-response programs at the local, national and international levels to decide what can be done in what timeframes (program feature 2); andapproaches to evaluation used by other programs that can be built upon to contribute to a broader evidence base about whether and how rapid-response programs work (program feature 3).We identified several possible barriers to implementing the program features, which we list in Table In addition to the facilitator (JNL) and two members of the McMaster Health Forum team (MGW and FPG), the dialogue brought together a diverse group of 11 stakeholders  from across Canada. Of these, five joined the dialogue in-person and the remaining six joined through video teleconference. The number of dialogue participants was purposefully lower than our typical range of 18 to 22 given our mix of in-person and video participation, and the dialogue was purposefully shorter than our usual full-day dialogue given our smaller number of participants and the challenge of sustaining attention among video participants.Dialogue participants generally agreed that health system decision-makers faced the three general challenges outlined earlier for finding and using research evidence. During the deliberation about these challenges, participants emphasized five specific challenges. First, the difficulty in accessing the best available research evidence in a timely fashion was highlighted by many. It was specifically noted that the definition of ‘timely’ often differs significantly between researchers and decision-makers, which poses challenges for accessing research evidence through most researchers. Second, contextualizing the research evidence was noted as being challenging but critical for helping to identify what the research evidence means for a particular jurisdiction at a given time. Third, accessing expertise was raised as another challenge, particularly when there is only a limited body of research evidence available from which to draw from. Fourth, participants emphasized the lack of capacity to find and use research evidence , both in terms of human resources and organizational structures for ‘policy shops’ within ministries of health. Lastly, ensuring the confidentiality of politically sensitive requests  was seen as a challenge that would need to be addressed (particularly if the products of a rapid-response program are routinely made publicly available).Dialogue participants agreed with the core components of the three program features and offered several suggestions and/or challenges related to them. First, while agreeing with many of the proposed organizational features in program feature 1, most supported the suggestion from one participant that the initial focus should be on the ‘organic’ development of a pan-Canadian network, with the McMaster Health Forum as a national coordinating hub. With respect to what can be done in what timelines (program feature 2), some questioned whether the three-business-day product would be requested often , whether jurisdictional scans about ‘who’s doing what’ could be offered as a fourth type of product, the feasibility of consistently preparing these products within the proposed timelines and if the products can be translated to make them available in both of the country’s official languages. Lastly, participants noted challenges in the fourth area of program feature 3 (measuring whether and how the product was used). Specifically, several dialogue participants argued that it is difficult to assess the extent to which a product informed or influenced a policy decision. In particular, participants emphasized that the timeline for conducting the evaluation is unlikely to align with the timelines for policy development and decision-making processes , thereby limiting the ability to make definitive statements about whether and how a product was used. In addition to the proposed measurement approaches, participants suggested that the possibility of capturing additional data from requestors (before responding to the request) and from others who could benefit from but did not make the request (e.g. download statistics) would be worthwhile to explore.The main challenges identified for moving forward were securing stable, long-term funding and finding a way to effectively and equitably manage the expected demand. Dialogue participants indicated support for engaging in a process to ‘organically’ build a pan-Canadian network, which they saw as needing to support efforts to 1) develop a common vision for the network, 2) explore how to link with organizations that could contribute to establishing and maintaining the rapid-response network, 3) map potential key individuals and organizations that could be involved within their respective jurisdictions that could contribute to the rapid-response network and identify ways to bring them together, 4) find ‘kindred spirits’ in other provinces and territories  that could join a pan-Canadian rapid-response network and 5) encourage people to direct their health-system-related questions to the newly established rapid-response network.We found very few systematic reviews related to the three broad features  of a rapid-response program for health system decision-makers. However, we used an existing analytical framework for how knowledge-brokering organizations can organize themselves  to derivn = 11) than the 18 to 22 participants that we have previously engaged in other stakeholder dialogues and, hence, may have missed divergent views held by some health system decision-makers across the country.The principal strength of our process is that we paired an issue brief outlining research evidence, analytical frameworks and characteristics of existing rapid-response programs focused on health systems with a robust deliberative process that gave voice to the tacit knowledge and real-world views and experiences of those involved in and/or affected by the issue. The main limitation is that we engaged a relatively smaller number of dialogue participants , networks and ‘kindred spirits’ that could join and contribute to sustaining a pan-Canadian rapid-response network. We believe that building such a network will make a significant contribution to addressing a gap in knowledge translation efforts for health system decision-makers in Canada.


However, older adults often lack sufficient eHealth literacy to maximize their benefit from these resources.This review evaluates the research design, methods, and findings of eHealth literacy interventions for older adults.A systematic review of peer-reviewed research articles from 28 databases in 9 fields was carried out in January 2013. Four rounds of screening of articles in these databases resulted in a final sample of 23 articles.Findings indicated a significant gap in the literature for eHealth literacy interventions evaluating health outcomes as the outcome of interest, a lack of theory-based interventions, and few studies applied high-quality research design.Our findings emphasize the need for researchers to develop and assess theory-based interventions applying high-quality research design in eHealth literacy interventions targeting the older population. This deBuilding on the concept of health literacy, eHealth literacy emphasizes information and communication technologies’ (ICTs) growing role in health information. Examples of ICTs relevant to individuals’ health management include patient portals, telehealth systems, and online support systems. eHealth literacy requires a mix of health, information, scientific, media, computer, and Internet literacy . Given IIn this new (but growing) field, few studies have yet developed and tested eHealth literacy specific theories. Rather, eHealth interventions often use learning theory to guide interventions eg, Xie -20). The. The20])Older adults’ distinct characteristics may explain why learning theories have not generalized to eHealth literacy interventions. Cognitive aging examines age-related changes in cognition, such as reduced information processing speed or a diminished ability to coordinate and integrate information . CognitiDiversity within the older population may also affect intervention outcomes. This diversity includes chronological age, along with race and ethnicity. Chronological age can range from 50 to over 100, while racial and ethnic minorities comprise 21% of the US population over age 65 . This diA systematic review of eHealth literacy interventions for older adults can provide a foundation for improving intervention outcomes. A recent systematic review investigated health literacy interventions for older adults  but exclWe address a significant gap in the literature by providing a more comprehensive review that includes eHealth literacy interventions with both computer literate and illiterate older adults. This approach is necessary because (1) many older adults lack computer literacy , (2) excWe performed four rounds of systematic selection in January 2013 to identify relevant articles: (1) database selection, (2) keyword search, (3) screening the titles and abstracts, and (4) screening the full text.We conducted search queries with electronic databases accessible at the University of Texas at Austin. Database selection involved two steps. First, we identified academic fields pertinent to the literature review, resulting in a list of nine fields with a total of 159 databases  screened the titles and abstracts for the 253 articles to ensure each study included Older adults must make up a significant proportion of study participants. For the purpose of this study, older adults are defined as individuals age 50 years and above. This definition expands the scope of the review and is consistent with growing appreciation of the role that health behavior interventions play in healthy aging for those under age 65 [1. r age 65 . Studiesr age 65 ) were elintervention using empirical data. To expand the scope of the review, we included qualitative studies and studies with a non-experimental research design, provided that these studies evaluated an intervention. Studies not reporting original and empirical data, such as literature reviews [2. The study involved evaluation of an  reviews  , were eeHealth literacy or improving a health outcome by improving eHealth literacy. To be more inclusive, we included interventions focused on health literacy related to specific health conditions, such as mental health literacy  or oral, Walker ), or a s, Walker ).One author (IW) reviewed the full text of the remaining 30 articles to confirm consistency with the three criteria applied to review article titles and abstracts during the third round. We eliminated another 7 articles in Round 4. Articles were eliminated for not reporting empirical data -39, not The 23 articles were published between 2003 and 2013 and report results from 23 independent studies -36,44-59We identified several intervention strategies among the sampled studies, including collaborative learning and tailored intervention content. Collaborative learning “involves the construction of meaning through interaction with others and can be characterized by a joint commitment to a shared goal” . Three sSimilarly, four studies (17%) tailored intervention content ,44,45,61Interventions applying collaborative learning strategies significantly improved participants’ computer and Web knowledge and skill ,20, eHeaThe sampled studies provided evidence for the effectiveness of eHealth literacy interventions for older adults. The studies that conducted RCTs all found significant improvement for outcome measures from pre- to post-intervention ,46,47,61About half of the studies  applied no theoretical framework. The 11 studies that applied a theoretical framework drew on two fields: (1) health behavior (seven studies) and (2) education (4 studies). Of these 11 studies, 7 30%) used health behavior theories: three studies (13%) used the Transtheoretical Model ,50,55, t% used heThe studies used a variety of research designs, consistent with the broad selection criteria used in our searches. Just under half 11/23, 48%) used designs with pretest and posttest of a single condition ,36,48-54% used deFive studies (22%) used either the Test of Functional Health Literacy in Adults (TOFHLA) or the Short Test of Functional Literacy in Adults (S-TOFHLA) -47,50,51Sample size varied between 11 participants in a pilot study and 909 participants for an RCT  reported data on participants’ income ,49,50,61% reporteTen studies (43%) targeted a specific health outcome or behavior as the outcome of interest, with health literacy serving as an independent variable ,55,56,61Most (6/8) of the eHealth literacy interventions used instructional materials developed by the National Institute on Aging (NIA) of the National Institutes of Health (NIH) ,49,57,58Fourteen studies 61%) occurred in informal learning settings  -54,57,58% occurreeHealth literacy interventions can provide older adults with the skills and knowledge necessary to benefit from eHealth resources -20. HoweThe sampled eHealth interventions were inconsistent with the clinical perspective in that none of the interventions included a health outcome as an outcome measure . IncludiSimilarly, developing and applying theory could enhance the quality of research on eHealth literacy interventions. Theory development advances emerging fields by shifting the research focus away from simply discovering new facts to explaining facts, predicting outcomes , and genNone of the sampled studies used an eHealth literacy-specific theoretical framework, such as the Lily model or Chan and Kaufman’s  proposedWhile the Lily model describes the skills and knowledge necessary for eHealth literacy, it includes no cognitive, social, or environmental variables and lacks empirical validation. The Chan and Kaufman  frameworAlong with a lack of theory, poor research design makes evaluating intervention outcomes problematic. Consistent with recent health literacy reviews ,13, mostA notable distinction between health literacy and eHealth literacy appeared in the intervention materials. The health literacy interventions used various materials, including condition-specific materials, such as an ulcer pamphlet . In contThe studies examined in this systematic review included participants that varied considerably in age, but most of these studies did not report participants’ income, education, race, or ethnicity. Mean participant age ranged from 61-84 years, raising the question of whether interventions effective for the “younger” segment of the older population can generalize to the “oldest” old. The failure to report participants’ income or education is problematic given the low median income of older adults in the United States ,71 and tTailoring offers a solution for addressing the influence of individual characteristics on health outcomes. While tailoring has been proven effective in health interventions ,72, onlyThis systematic literature review contains several limitations. The keyword search did not use a controlled vocabulary  and was restricted to the title, keywords, and abstract for each article. This inclusion criterion may have excluded studies that deal with aspects of eHealth literacy but do not contain these exact keywords we used. Only studies with full text written in English were included in the sample, which excluded articles in non-English journals. Also, this review did not include studies that may have contributed to the development of the eHealth literacy construct (that pre-dated it and thus did not use the exact terms to be included in our searches). Nonetheless, this review identified important gaps in the literature that require future research. These gaps include (1) What is the relationship between eHealth literacy and health outcomes for older adults?, (2) Which theoretical frameworks are effective for developing and assessing eHealth literacy interventions for older adults, and which are more effective than others in what context?, (3) What is the relationship between eHealth literacy interventions conducted in clinical settings and those conducted in informal learning settings, and what factors should be considered when implementing eHealth literacy interventions in clinical settings?, (4) What instructional materials best facilitate older adults’ improvements in their eHealth literacy, and how can materials stay current given ICTs’ rapid development?, and (5) Which individual characteristics, such as health literacy level, computer experience, or SES, should be considered in interventions to tailor the health content and delivery strategies?This paper reports findings from a systematic review of 23 articles on health literacy interventions and eHealth literacy interventions for older adults drawn from 28 relevant databases in nine fields. The eHealth literacy interventions in the sampled articles used eHealth literacy as an outcome of interest, applied learning theories, and occurred in informal learning settings such as senior centers and public libraries. In contrast, health literacy interventions (that involved ICTs as a key aspect of their interventions) often targeted specific health outcomes, applied health behavior theories, and occurred in both informal learning and clinical settings. These results indicate a significant gap in the literature on eHealth literacy interventions that use health outcomes for outcome measures. Additionally, most of the studies used no theoretical framework, and only seven studies were RCTs. These results highlight a great need to develop and assess theory-based interventions applying high-quality research design.


Although increasing attention is being paid to knowledge translation (KT), research findings are not being utilized to the desired extent. The present study explores the role of evidence, barriers, and factors facilitating the uptake of evidence in the change in malaria treatment policy in Uganda, building on previous work in Uganda that led to the development of a middle range theory (MRT) outlining the main facilitatory factors for KT. Application of the MRT to a health policy case will contribute to refining it.Using a case study approach and mixed methods, perceptions of respondents on whether evidence was available, had been considered and barriers and facilitatory factors to the uptake of evidence were explored. In addition, the respondents’ rating of the degree of consistency between the policy decision and available evidence was assessed. Data collection methods included key informant interviews and document review. Qualitative data were analysed using content thematic analysis, whereas quantitative data were analysed using Excel spreadsheets. The two data sets were eventually triangulated.Evidence was used to change the malaria treatment policy, though the consistency between evidence and policy decisions varied along the policy development cycle. The availability of high-quality and contextualized evidence, including effective dissemination, Ministry of Health institutional capacity to lead the KT process, intervention of the WHO and a regional professional network, the existence of partnerships for KT with mutual trust and availability of funding, tools, and inputs to implement evidence, were the most important facilitatory factors that enhanced the uptake of evidence. Among the barriers that had to be overcome were resistance from implementers, the health system capacity to implement evidence, and financial sustainability.The results agree with facilitatory factors identified in the earlier developed MRT, though additional factors emerged. These results refine the earlier MRT stating that high-quality and contextualized evidence will be taken up in policies, leading to evidence-informed policies when the MoH leads the KT process, partnerships are in place for KT, the WHO and regional professional bodies play a role, and funding, tools, and required inputs for implementing evidence are available.The online version of this article (doi:10.1186/1475-2875-13-345) contains supplementary material, which is available to authorized users. Although commitment to knowledge translation (KT) has been an issue of interest to funders of research, researchers, and policymakers, there is a concern that research findings are not being utilized to the extent that they should –4. SeverIn the present study, evidence is broadly defined to include research study results (both published and unpublished), findings from monitoring and evaluation (M&E) studies and population-based surveys, Ministry of Health (MoH) reports, community complaints, and clinician observations , 15. TheThis study, which looks at the uptake of evidence in policy development, specifically in reference to changes in the malaria treatment policy and its implementation, is part of a larger study exploring ways to improve KT in Uganda. Previous work in Uganda led to the development of a middle range theory (MRT) outlining the main facilitating factors for translating evidence into policymaking . MRTs arHigh-quality and contextualized evidence will be taken up in policies so as to lead to evidence-informed policies in instances where the MoH leads the KT process and there are partnerships for KT in place.“The MRT detailing facilitating factors to the uptake of evidence as identified by policy actors in Uganda states the following: Evidence must be of high quality, contextualized, providing economically feasible recommendations, and produced in a timely manner by credible researchers. Use of local researchers is helpful but there is need for separation of roles between researchers and policymakers.KT requires strengthened MoH institutional capacity to lead the KT process. Institutionalized platforms for engagement between researchers and policymakers including civil society need to be in place, and mechanisms to coordinate evidence generation and synthesis need to be mainstreamed within the MoH. The capacity of policy makers in knowledge management needs to be strengthened and the policy making process need not be very bureaucratic.These contribute to higher ownership, adoption, and better application of evidence[”.evidence”.Partnerships for KT need to be in place and all relevant stakeholders must be involved throughout the process to improve trust and build interest. Communities need to be involvement in evidence generation and KT as well. The MRT was developed on the basis of a literature review and then validated with policy actors in Uganda. The facilitating factors were collected from respondents without a specific reference to a given research project and policy outcome; the extent to which they are valid in other settings needs to be tested in specific policy case studies. This study explores the place of evidence in the design and implementation of the change in malaria treatment policy in Uganda using a case study approach. Specifically, the study seeks to assess the extent to which the previously developed MRT explains the uptake of evidence in policy development from a policymaking perspective and explore the barriers and facilitatory factors to the uptake of evidence in the malaria treatment policy change. Eventually, the application of this MRT to concrete, selected health policy cases will contribute to refining and enriching the previously developed MRT.et al.[The background to this case study was published by Nanyunja et al.. The incet al.. EANMAT et al., 22. Thuet al., 23, 24.et al.. The malet al.. The proThe case study approach was used based on the need to understand complex contextual issues . The casIn a quest to improve the comprehensiveness and validity of the findings, the present study employed both qualitative and quantitative methods (QUAL + quant) which are increasingly being applied to the investigation of complex issues in health systems research , 31. A tThe selection of respondents was guided by the study design. Using the timeline of key events, institutions were identified that were involved in the policy process. Key informants (KIs) were selected using purposive sampling, with the main criterion being their involvement in the research, design, or implementation of the malaria treatment policy change . From eaTo obtain perceptions from across the spectrum of the healthcare delivery system, two districts with high malaria endemicity  were purThe selected respondents included donor representatives, public policy makers, civil society organizations (CSOs), researchers, the media, and representatives of the pharmaceutical sector. Managers of health services at the district level, health care providers from the public and private not-for-profit health facilities, senior officials from the national medical stores (NMS) in charge of medicine procurement and distribution, and managers from the national drug authority (NDA) in charge of medicine regulation were also interviewed. Details of the selected respondents are shown in Table The timeline of key events guided the identification of relevant documents to be reviewed. A broad range of documents relevant to the case were included in order to ascertain the processes and stakeholders involved and assess the use of evidence.The qualitative part of the study assessed whether evidence was available to guide policy decision-making, the nature of the evidence that was available, the extent to which evidence was disseminated and discussed in relevant forums, whether and why evidence had or had not been considered in policy development and implementation, and whether policy decisions were in line with the available evidence. The barriers and facilitatory factors to the uptake of evidence in the malaria treatment policy change and implementation process were explored.Data collection methods included a review of documents and interviews with KIs. Interviews were conducted with KIs using an in-depth interview guide consisting of open-ended questions. The interview guide was developed by the first author and was reviewed and refined by the research team prior to pretesting it with volunteer colleagues in the WHO Uganda office (n = 2), technical officers in the MoH (n = 2), and one researcher from the Makerere University School of Public Health in Uganda. KIs were contacted and invited by email or telephone to participate in the study. All identified respondents accepted to participate and were interviewed. All interviews were conducted by the first author face-to-face in English . Using a semi-structured questionnaire, KIs were asked to rate the consistency between policy decisions and available evidence. The consistency between evidence and policy decisions were rated using scales developed by Hanney et al.[The document review entailed quantifying the frequency with which evidence was cited, including the type of evidence  and Uganda National Council for Science and Technology (number SS 2920).This is mainly scientific because we are dealing with technical issues, hence no room for guessing.As civil society, although we reach out mainly to communities, we were at task to explain the need for a policy change using scientific explanations”.The change in malaria treatment policy was reported to be very technical, and in which the role of evidence was very important. A civil society respondent stated more specifically that, “The evidence that was reported to have been available and considered in the policy process was categorized into nine areas Table , namely We used evidence to change the malaria treatment policy. We mainly used evidence on the efficacy of the drugs that we were using (CQ/SP). There was good evidence from different sites in the country showing that the efficacy of the drugs was really declining”. This evidence was deemed to be of high quality because multiple study sites were using a WHO-approved protocol. Technical support was provided by EANMAT and the WHO when setting up the sites and for the development of research protocols, data analysis, and interpretation. Results from the different sites were consistent, and the results were consistent with those of other countries with similar malaria endemicity: The methodology to undertake studies on the efficacy of medicines was thorough, so when we presented the evidence, there were no loopholes. We made sure that results were not from only one study area. Otherwise, with the challenges we had with this malaria treatment policy shift, if there were loopholes in the data we would have been shot down, especially at the point when people started thinking that maybe some drug companies were the ones pushing it on Uganda and that the country was not going to afford the new policy”. (Researcher respondent)“Evidence on the efficacy of CQ/SP and ACT was reviewed, discussed, and guided decision-making, as highlighted by a donor respondent: “WHO is seen as the authority on clinical matters; when the WHO takes a stance and says that this drug is better for patients than the old drug, the country will often take that recommendation very seriously”. (Donor respondent)“Uganda is a member of the WHO - so we tend to take advice and guidance from the WHO. Whenever WHO updates guidelines or sends out new information, we take that advice. One of the reasons why a country, like for example Uganda, should change the policy is because they are being guided by an international body like the WHO, which is our technical arm in policy around malaria and other diseases”. (MoH respondent)“The WHO provided evidence from global and regional levels, as well as standards with regards to drug resistance cut-off levels at which a country should embark on changing their malaria treatment policy. This was important in guiding decision-making, as emphasized in the following quotes: There was a public outcry, and the public outcry comes in many forms, either through newspaper articles or, particularly if you are the health worker, people will tell you ‘doctor you gave me this drug for a fever last week and it is not getting better!’ That is how they communicate to you their concerns, and that is strong enough. So to me, I think that is strong enough evidence and we used it”.Cries from the community further created impetus to take action, as one donor respondent stated, “When we came in with Coartem, three months later we started getting complaints from the community that the new drug was not as effective as CQ; when you take it you still feel weak and the temperature does not go down. The problem was not that the new drug was not working, but it did not have the property of reducing temperature. So we had to go on radio to make the announcements nationally saying ‘when you are taking Coartem®, take an antipyretic like Aspirin or Panadol together with Coartem,’ and it worked”. (MoH respondent)“At the implementation stage, community complaints, which referred to perceptions of the effectiveness of Coartem®/AL, were used to address operational challenges: we submit our monthly reports to the District Health office, and these reports are used in assessing delivery of services, including malaria, but I have never seen any change. No feedback to improve the quality of data”.Routine M & E and evidence from supervision reports were also used at the implementation stage. However, reservations were expressed by service providers regarding the quality of routine data stating that “Respondents identified factors that facilitated the uptake of evidence, and these were categorized into five themes: 1) characteristics of the available evidence, 2) MoH institutional capacity to lead the KT process, 3) partnerships for KT, 4) availability of tools and inputs to implement evidence, and 5) intervention of the WHO. These themes are summarized in Table The characteristics of available evidence encompassed several dimensions, including the availability of high-quality local evidence, the availability of competent in-country researchers, consistent results from multiple studies performed by different researchers, evidence generated by credible international researchers/regional network, and consensus on research results.the study design also influenced the uptake of evidence. For example, they used the WHO tools; people will easily accept such evidence other than coming up with individually designed tools. The WHO tools are already tested methods”.High-quality local evidence on the efficacy of CQ/SP was available from sentinel sites set up in Uganda by the MoH with support from EANMAT and the WHO. A private pharmaceutical representative noted that “Uganda is fortunate that it has a lot of leading thinkers in malaria who know a lot about malaria. They were part of the process and were leading in all the studies. The availability of local data made it easier for running this case through the different levels of policy formulation, which would be different I suppose if you were in a country where there are no systems for research”.“Some research studies were performed by competent in-country researchers who were deemed credible, and this helped the acceptance of results, as highlighted in the following quote from a donor respondent: The regional network EARMAT is highly respected as a body of senior malaria experts, their support to sentinel sites in the regions gave a lot of credibility to the data, and they have been very supportive in terms of helping us to understand what is in the background”.International researchers who had generated evidence from outside the country were also credible, as the MoH respondent stated, “Evidence sometimes is not easily accepted, but there was consistency in results from different sites. If you have several studies from different settings undertaken by different researchers showing the same results, it is easily accepted”.Consistency in the research results of different study sites in the country and studies from other East African countries was a key factor, as stated by a donor respondent, “I remember we had a report coming out every Monday in the newspapers publicising the malaria burden”. However, some felt that the extent to which evidence influenced decision-making could not be ascertained. A journalist remarked, “We certainly published a number of stories on drug resistance in the newspapers although I am not sure to what extent those publications influenced decisions”.Evidence was disseminated through the media, as a MoH respondent stated, “Similarly, the review of documents pointed to the availability of contextualized evidence on the efficacy of anti-malarials from studies performed in the country. Other sources of high-quality evidence included Demographic Health Survey (DHS) data supported by Macro International, the census and national household survey employing internationally agreed upon methodologies, and malaria economic studies, which were carried out in several countries following the WHO protocol.a, the Interagency Coordination Committee for Malariab, and the national stakeholder forum. A research committee was put in place and included Malaria programme staff, researchers, and representatives of technical partners . Research priorities were identified in a meeting that brought together all relevant stakeholders.There was separation of roles, the researchers conducted the studies and policy makers played a leadership role, receiving and discussing results. Research findings were discussed in several partnership forums. Evidence was discussed in the Malaria Case Management Technical Working Group (MCMWG)The MoH institutional capacity to lead the KT process encompassed several dimensions, including leadership role, the willingness of MoH to use evidence, MoH involvement in research studies, and a culture of the MoH using evidence to change treatment policies.I recall that around the time that we transitioned from CQ/SP to ACT, the NMCP was in the hands of two people; both of them believed in evidence”.The strong leadership of the MoH and a culture of using evidence in policy development, facilitated partly by previous experiences, were echoed by several respondents. A researcher remarked, “Our research programme has been working closely with the MoH and that helps; although were not always on the same page we had a good relationship. The sentinel sites were established by the malaria control programme a long time ago and we are using them. We have continued that relationship and, when we do studies, we often discuss with them what we are going to do and get a go ahead from the ministry. We have also had funding coming through the Ministry”.“There was close collaboration between researchers and MoH policy makers, as reported by a researcher: We have people in the ministry that are not well versed with the evidence they are using to decide whether this policy makes sense or not, and therefore you have people that think they know the evidence but really they do not”.“However, one donor respondent expressed reservations about the quality of MoH participation: The reviewed documents further confirmed that an institutionalized and systematic data collection system on drug efficacy was in place through the MoH-established sentinel sites (established in 1997). In addition, the Health Management Information System (HMIS), which collects data on service utilization and malaria cases, was also in place and managed by the MoH. These sources provided data that were referenced in the majority of the reviewed documents.Furthermore, the MoH commissioned several studies, including a review of all available data, and closely followed up with researchers through the implementation of the research studies. Broad institutionalized platforms were in place for engaging all stakeholders. The MoH took leadership of the knowledge synthesis and application process through participation in, and chairing, the working group charged with synthesising all available evidence and making recommendations to the steering committee, which consisted of the decision-makers.Partnerships for KT encompassed: the availability of platforms and structures within the MoH to discuss evidence, the interest of stakeholders to see that evidence is adopted into policy, and civil society involvement.At that time we had a very good team in the MoH and the opportunity to discuss things. There were systems and we had regular meetings and annual performance reviews. We had quarterly performance evaluations, which were very important and everyone had to be there right from the minister, so the climate for evidence-based decision-making, whether it was accidental or not, was there in the vision of the leadership”. (MoH respondent)“The availability of structures within the MoH to enable systematic dialogue was highlighted as a factor that improved the uptake of evidence: The way the policy process worked is that the malaria programme in the ministry called together all its technical stakeholders - all its partners - everyone, government, academia, NGOs, etc. Everyone sat in one room and debated what they thought the best policy option should be. I thought this was an excellent process”. (Donor respondent)“As a malaria programme, we have what we call a Malaria Case Management Technical Working Group, which periodically meets to review how our medicines are working in the country; so that is another arm which facilitated the uptake of evidence because we are able to detect a need for change”. (MoH respondent)“Platforms to enable inclusive participation were also in place for evidence to be discussed, which facilitated consensus building, as highlighted in the following quotes: The document review showed that partnerships were put in place at several stages of knowledge generation, synthesis, and application, bringing together relevant stakeholders. Figure c was put in place to synthesize available data and make evidence-based recommendations to the steering committee and MoH. The MCMWG concluded that, “After careful consideration of the evidence for CQ/SP treatment failure, the meeting agreed that there was need to change the anti-malarial drug policy (AMDP). In line with June 2000 recommendations to adopt a long-term policy and WHO recommendations, ACT was considered the most viable option to change to.” A policy decision that conformed to the available evidence.The MCMWG Ahead of the national consultative meeting, evidence on drug resistance against the first-line treatment for malaria, as well as its interpretation and implications, was also presented to influential stakeholders in a form that was easy to understand. A national consultative stakeholder forum was then held, bringing together researchers, policy makers, clinicians, donors, CSOs, and district level health managers, and evidence synthesized by the MCMWG guided the dialogue. The national stakeholder forum identified information gaps, enabling all partners to participate in setting the research agenda. The meeting identified several studies covering several aspects to provide comprehensive evidence on cost-effectiveness, acceptability, and the feasibility of implementation. These studies were then commissioned by the MoH.A series of meetings were held ensuring sustained dialogue throughout the policy change process. The steering committee provided regular briefings to members of MoH top management to apprise them of the situation, including presentations of synthesized evidence.After a decision was made to change the first-line treatment for uncomplicated malaria to AL, the MoH commissioned task forces that brought together relevant stakeholders to work out the implementation process and mainstream implementation of the new policy in the routine system Figure . In a meAll stakeholders were represented in the different working groups and worked together to review the evidence, make policy decisions, and produce operational tools. All of the task forces were chaired by senior MoH officials.The provision of guidelines, medicines, and training for health workers on the new policy are factors that favoured the implementation of evidence.some health workers were totally green about the new anti-malarials; so we developed training materials and we trained health workers throughout the country.” A service provider stated that “the MoH provided us with guidelines which were very useful when it came to actual implementation of the policy. The civil society helped us to print more copies of these guidelines and provide them to health facilities”.A MoH official stated that “Around that time, there were was an opening for the Global Fund, and the possibility of applying for the Global Fund round 2 to support the policy change was raised. That was the only way we could realize the change to the new drug, and when we applied for funding and got it, that drove the decision”. (MoH respondent)“There was the Global Fund at that point, otherwise it would have taken us a long time to change the treatment policy if we were to foot it alone. The Global Fund came on board with rounds where we had to make proposals to get funding for the new treatment, and when they gave us funds, we took up the policy”. (MoH respondent)“Affordability issues were also raised, despite the availability of evidence. Some respondents reported that the decision to move to ACT was influenced by the availability of funding from the Global Fund (GF), as highlighted in the following quotes: The document review showed that the MoH put in place a mechanism to enable the implementation of evidence; the four task forces shown in Figure there is a general tendency of human beings resisting change just by nature, because they have learned the old ways of giving medicine and nobody is willing to adjust. So there was a problem with some of our health workers’ attitude”.Respondents mentioned resistance from implementers as one of the barriers that had to be overcome. One MoH official remarked that “You know as part of market dynamics- you will find that once a product has been in the country for some time, there is a system which promotes its sale; so when a change is proposed, there is resistance because someone has been promoting a product which the company has been selling for some time. That is one arm that can bar you from adopting good evidence”. Health system considerations and sustainability of the new policy were among the additional challenges. Concerns also existed about whether the supply chain system would ensure the availability of ACT and whether the country could sustain the policy from its own resources.Another reported barrier was the influence of drug companies, as stated by a service provider, “All 18 of the reviewed documents referenced evidence; the documents cited evidence 57 times. Each type of evidence was counted only once in each reviewed document. The evidence cited most was mainly locally generated evidence, as opposed to international, efficacy studies, M & E, and WHO guidance  put in place to synthesize available data  and make evidence-based recommendations to the steering committee and MoH. Malaria Case Management Group comprised the MoH , WHO, NDA, NMS, JMS, DHO, hospitals, CSO, public health lab, researchers, UNICEF, and MoF.In-depth interview guide for policy makers and researchers. Open-ended question to elicit responses from respondents. (DOCX 15 KB)Additional file 1:Guideline for document review. Guide used in reviewing documents. (DOCX 12 KB)Additional file 2:Documents reviewed. Details of milestones and list of documents reviewed. (DOCX 24 KB)Additional file 3:Factors that facilitated the uptake of evidence in the malaria treatment policy change. Details of facilitatory factors under the different themes. (DOCX 20 KB)Additional file 4:


People intending to travel may seek information on malaria prevention from a range of sources. To ensure the best protection, this information needs to be reliable, up-to-date, consistent, and useful to their decision making. This study appraises current international and national guidelines written in English for malaria prevention in travellers, and whether any recommendations conflict.We systematically identified national or international English-language guidelines on malaria prevention in travellers to July 2013 using standard and multiple searching methods. We critically appraised guidelines using the AGREE II tool, and report inconsistent recommendations within guidelines.We identified five sets of English-language guidelines on preventing malaria for travellers. Assessment against AGREE II indicate that all of the guidelines fall short of internationally accepted standards in guideline development: none include a transparent description of methods; only one describes sources of funding or potential conflicts of interest; and only one includes formal presentation of the evidence alongside transparent assessment of the quality of that evidence. There were a number of important discrepancies between guidelines, and some omit information about effectiveness, safety and adverse effects of chemoprophylaxis options.The methods used for developing guidelines for malaria prevention in travellers lags behind current internationally recognized standards. Healthcare professionals as well as travellers themselves could be better informed if guidelines were more systematic and transparent summaries of the current knowledge on drug interventions in relation to effects, safety, administration and contra-indications.The online version of this article (doi:10.1186/1471-2458-14-1129) contains supplementary material, which is available to authorized users. In the UK, approximately 2,000 episodes of malaria are recorded each year in people returning from travel to endemic countries, and travellers still die from malaria . TravellPeople intending to travel may access information on malaria prevention through on-line resources, by visiting their own doctor, through specialist travel clinics, or through various travel operators . To ensuTravellers to malaria endemic settings require information that enables them, and convinces them, to choose the interventions that are most effective and safe for them. This is likely to involve some quantification of the relative benefits and harms of each intervention. To facilitate this, good quality guidelines should present this information in a clear and user-friendly format .In this study, we aim to a) appraise the available national and international English-language guidelines on malaria prevention against international best practice in guideline development, b) evaluate the degree of consistency or conflict between guidelines, and c) consider how well the information facilitates informed decision making.Two authors  searched independently for international or national guidelines for malaria prevention in travellers using: Google, the websites of major public health bodies from English-speaking non-endemic countries , the TRIP database, the National Guideline Clearing House, and pubmed  AND guid* AND travel) for articles published until the end of July 2013. Guidelines limited to malaria prevention in sub-groups (such as children), or in languages other than English, were excluded.Two authors  independently appraised each identified guideline using the AGREE II guideline appraisal tool . The AGRTwo authors  independently reviewed each guideline to identify both explicit and implicit recommendations relating to prevention of malaria. Explicit recommendations were classified as those where something was clearly being advised . Implicit recommendations were classified as statements where judgements are being made . Two reviewers mapped recommendations to a standardised table independently and compared and contrasted them. Discrepancies were addressed through discussion between the two reviewers and where discrepancies could not be resolved, a third reviewer (DS) was consulted. Two reviewers  compared the recommendations made by each of the guidelines to identify any conflicts between sets of guidelines.Five guidelines met our inclusion criteria, developed by: the World Health Organization (WHO) , the PubWe excluded an article providing advice for travellers from New Zealand, as it was not official national guidance , and altThe summary AGREE II scores for each of the five guidelines are shown in Table All five guidelines covered both drug and non-drug interventions to prevent malaria in travellers, and all aimed to assist clinicians as they advise travellers. The UK and HK guidelines provided the most complete description, clearly outlining the objectives, health issue and target population to which the guidelines apply.All guidelines gave a named list of the individuals who primarily authored the document, although only the UK guidelines stated the roles and responsibilities of these individuals. Three guidelines  also referred to working groups or external contributors but these were not named. None of the guidelines reported involvement of the target population (travellers).None of the guidelines included a Methods section. None described methods to systematically search for and appraise evidence, or to move from evidence to formulating recommendations. None of the documents described external peer review prior to publication, and only the WHO guideline made reference to a process for future updates.The Canadian guideline presented the evidence base summarized in GRADE tables. All other guidelines summarized evidence in text form, with variable levels of referencing provided to validate recommendations. USA and WHO guidelines provide no references, HK provide some references and UK guidelines provide a reference for many of the recommendations made.All guidelines discussed the health benefits and side effects of interventions to some extent, although the level of detail, particularly around safety was highly variable. The Canadian and WHO guidelines gave some risk estimates for the potential benefits and harms.All of the guidelines presented some clear recommendations that were unambiguous and specific, and clearly identifiable. However, all guidelines also included discussions of options without clear recommendations, or implied recommendations incorporated into the text and without clear signposting. The Canadian guidelines were presented most clearly, listing all explicit recommendations in a table.None of the guidelines discussed the resource implications of the recommendations, and none described potential barriers or facilitators to implementation. None presented audit criteria. Only the USA guideline presented an outline of the advantages and disadvantages of each drug in a user-friendly table.Only the USA guideline described the sources of funding, and made any reference to potential conflicts of interest among guideline developers. The remaining guidelines said nothing, and therefore scored zero for this domain.All guidelines state that atovaquone-proguanil, doxycycline and mefloquine are effective against chloroquine resistant falciparum malaria. Two guidelines quantify this: the Canadian guidelines state that they are 95% effective and the UK guidelines state that they are 90% effective, each referencing different documents.The Canadian, Hong Kong and USA guidelines state that atovaquone-proguanil has an excellent safety profile, with only the Canadian guidelines quantifying that 8% to 15% of individuals experience nausea, vomiting, abdominal pain or diarrhoea. The guidelines state that the most common side effects are headache and gastrointestinal upsets, but again do not quantify this.Although the Canadian guideline outlines that atovaquone-proguanil has an excellent safety profile, it outlines that it should be used in caution in patients with HIV and liver disease. This is not discussed in other guidelines;The US guideline states that atovaquone-proguanil cannot be used in children under 5 kg, whereas all other guidelines state 11 kg as the cut-off;All guidelines state that doxycycline may be photosensitising and can lead to a rash, however no guidelines quantify this risk;All guidelines state that doxycycline may lead to gastrointestinal upset and oesophageal ulceration, however no guidelines quantify this risk;The UK guideline states that doxycycline is contraindicated in children under 12 years, while all other guidelines say 8 years;All guidelines state that mefloquine can lead to major neuropsychiatric side effects such as seizure, psychosis and suicidal ideation. The Canadian and WHO guidelines quantify this. Canadian guidelines suggest that 1 in 6,000 to 13,000 have seizure or psychosis; and WHO guidelines suggest severe neuropsychiatric disturbance in 1 in 10,000.The Canadian, Hong Kong and USA guidelines state that mefloquine can lead to minor side effects such as nausea, strange vivid dreams, dizziness, mood changes, insomnia, headache and diarrhoea. The UK and WHO guidelines do not mention this. Only the Canadian guideline quantifies this, suggesting that 95% of people have no or mild, temporary side effects, with 1 in 250 to 500 having mild neuropsychological reactions e.g. anxiety or mood change;The US guideline states that mefloquine can be given to all ages, while other guidelines state it cannot be used in children below 5 kg;Stand-by treatments is recommended as being valuable in all guidelines, although the USA guidelines provide very little information regarding this.A summary of the drug recommendations made by each of the guidelines has been presented in Additional file A full outline of the non-drug recommendations made by each of the guidelines has been presented in Additional file The WHO guidelines states insecticide-treated clothing lasts longer than skin applied insecticide, whereas UK guidelines states the opposite. Other guidelines provide no recommendations regarding this;Concentrations of DEET recommended in children vary from 10%  to 50% (UK), and in adults vary from 35%  to 50%  in adults. WHO guidelines state that DEET should be used according to manufacturers’ recommendations;The Canadian and WHO guidelines state that DEET can be used at any age, while the UK and USA guidelines state it should be restricted to children over the age of 2 months. Hong Kong guidelines do not provide recommendations on age limits of DEET;The WHO guidelines state that no other protection is required if a hotel is air-conditioned. The UK and USA guidelines imply that malaria transmission is reduced in air-conditioned rooms, and the Canadian and Hong Kong guideline do not provide recommendations on air-conditioning;Light covering clothing is recommended by Canada and Hong Kong, while the UK guidelines state this is not effective. USA and WHO guidelines do not provide recommendations on the use of light covered clothing.The main conflicts and omissions are that:Appraisal of the five guidelines using the AGREE II tool indicates that all of the English guidelines surveyed  fall short of internationally accepted standards in guideline development. None include a transparent description of methods used, only one describes sources of funding and potential conflicts of interest, and only one includes formal presentation of the evidence alongside transparent assessments of the quality of that evidence. There were a number of important discrepancies between guidelines; and some omit information about effectiveness, safety and adverse effects of chemoprophylaxis options.Moving from research evidence to healthcare recommendations is not straightforward. High quality evidence that an intervention is effective does not automatically lead to a strong recommendation to implement it, and low quality evidence does not preclude a positive recommendation . The proA second clear deficiency common across guidelines was the lack of user-friendly resources for implementation. Although most guidelines stated that they should be used by healthcare workers and travellers to identify the most suitable interventions for individuals, most did not provide adequate information to help these decisions between different strategies, and quantified estimates of the proportion of people who would be likely to benefit or be harmed by an intervention were rarely given. This was particularly true of drug interventions where side effects are often one of the most important factor influencing choice , 13. TheIn addition there were a number of inconsistencies or discrepancies between guidelines. Most were omissions of information, but some reflected different or conflicting recommendations. Identifying these may suggest this is where the evidence is weak or limited, and expert judgment alone has been the main driver of the recommendation. Being clear and transparent about the presence or absence of high quality evidence behind a recommendation helps identify priority research questions, and stimulate a genuinely useful research agenda.One limitation of this study is that we only included English-language guidelines. Italian guidelines, published as a summary in English after the literature search was conducted for this paper, outlines that national and internationals guidelines have moved in a varying degree away from routinely recommending chemoprophylaxis to a greater use of a wide range of preventative methods . There mWe evaluated five sets of guidelines written in English on the prevention of malaria for travellers. Guidelines had a number of omissions and conflicts, and were not clearly based on evidence. Healthcare professionals and travellers would benefit from consistent, easy to use guidelines, based clearly on evidence, and outlining current knowledge on effectiveness of interventions, safety, administration and contra-indications.Consensus between guidelines for drug interventions.(DOC 144 KB)Additional file 1:Consensus between guidelines for non-drug interventions.(DOC 115 KB)Additional file 2: