The translation of research into clinical practice is a key component of evidence-informed decision making. We implemented a multi-component dissemination and implementation strategy for healthcare professionals (HCPs) called Evidence Rounds. We report the findings of focus groups and interviews with HCPs to explore their perceptions of Evidence Rounds and help inform the implementation of future similar initiatives. This is the second paper in a two-part series.We employed total population, purposive sampling by targeting all of the health care professionals who attended or presented at group sessions exploring the evidence on clinical questions or topics chosen and presented by the HCPs. We conducted and audio-recorded in-person focus groups and one-to-one interviews, which were then transcribed verbatim. Two authors independently coded transcripts. NVivo software was used to collate the primary data and codes. We analysed data guided by the five steps involved in framework analysis; 1) familiarization 2) identifying a thematic framework 3) indexing 4) charting 5) mapping and interpretation.Thirteen HCPs participated, of which 6 were medical doctors an d 7 were nursing or midwifery staff. We identified the following key domains; organisational readiness for change, barriers and facilitators to attendance, barriers and facilitators to presenting, communication and dissemination of information, and sustainability. During focus groups and interviews HCPs reported that Evidence Rounds had a positive impact on their continuing education and clinical practice. They also provided insights into how future initiatives could be optimised to support and enable them to narrow the gap between research evidence and practice.Individual, departmental and organisational level contextual factors can play a major role in implementation within complex health services. HCPs highlighted how in combination with clinical guideline development, implementation of evidence could be increased. Further research after a longer period of implementation could investigate how initiatives might be optimised to promote the uptake of evidence, improve implementation and expedite behaviour change.The online version of this article (10.1186/s12909-019-1488-z) contains supplementary material, which is available to authorized users. Evidence-informed decision making is fundamental to the provision of healthcare and central to this is the translation of research evidence into clinical practice. The use of the term “evidence-informed” highlights the need to acknowledge and address contextual influences and consider how the best available evidence can be used in specific circumstances .There is a need to improve translation of research evidence into practice . The eveKT strategies can employ single or multiple components such as professional educational meetings eg. journal clubs, educational materials, educational outreach visits, knowledge brokers, audit and feedback etc. A limitation of the traditional educational model of journal club is that its primary focus is on the critical appraisal of a single source . A CochrWhile outcome evaluations tell us whether an implementation programme does or does not work, they can ignore confounding contextual factors  and failWhile a number of definitions for the term dissemination have been suggested, in this paper we define it as “an active approach of spreading evidence-based interventions to the target audience via determined channels using planned strategies” . McCormaWe utilised a multi-faceted KT strategy to actively disseminate evidence to healthcare professionals and promote evidence informed practice including implementation of evidence where appropriate . Our iniThe objectives of this study were to use focus groups and interviews a) to identify HCP-reported barriers and facilitators to attending and presenting at educational group sessions b) to explore HCPs’ views of Evidence Rounds, particularly as a dissemination strategy, and c) to generate insights to improve the sustainability of future initiatives because evidence about the sustainability of KT interventions is still lacking , 23.We utilised a qualitative study design which can provide valuable insights into contextual factors and intervention features which influence the success of KT interventions . We usedWe gave potential participants the choice to take part in either a focus group or an interview according to their individual preferences. We displayed posters in areas frequently accessed by our target population. To enhance recruitment, we entered each participant into a draw to win a voucher for a local restaurant. We developed an interview guide around the aims of the study Identifying a thematic frameworkAC who had been present at all recordings re-listened and where appropriate, made corrections and where possible, filled in sections of speech that were inaudible to the transcriber. Two authors (AC and MD) listened to the audio files while reading the corrected transcripts. AC reviewed the observational notes collected by the assistant moderator during the focus groups. Two authors (AC and MD) independently coded the transcripts and noted key points, repeated themes and issues considered important by participants.3.)IndexingWe began to create a thematic framework drawing from a list of 54 a priori key issues deemed relevant to our study aims, 21 additional emergent issues based from participant responses, and began to connect and look for patterns in participant responses to form analytical themes. The thematic framework took several iterations.4.)ChartingWe uploaded the transcripts to NVivo Version 11 and systematically applied the thematic framework by assigning nodes and sub-nodes to text within each transcript. As is common in framework analysis papers, some text was coded into multiple nodes , others 5.)Mapping and interpretationWe reviewed the data and made a decision to chart by core themes rather than cases. We created five tables each with a unique domain and used themes, sub-themes and illustrative quotes that demonstrated the range of participant responses. All authors reviewed the tables and made revisions to improve the presentation of data.We referred to the main aims of the study and reviewed the tables. We considered the nature and range of participant perspectives. Using this method, it was possible to extract key dimensions of the barriers and facilitators to attending and presenting at Evidence Rounds, their perspectives of our dissemination strategy, and suggestions to make future initiatives more sustainable.Thirteen HCPs participated in three focus groups (of between two and four participants), and five in one-to-one interviews. Six medical doctors and seven nursing or midwifery staff participated, of which four were male and nine were female. Our data analysis revealed five core domains regarding HCPs perspectives of Evidence Rounds: (1) barriers and facilitators to attending; (2) barriers and facilitators to presenting; (3) organisational readiness for change; (4) communication and dissemination of information; and (5) sustainability.This domain included three themes namely; departmental context and resources, social context and individual level factors. Our study demonstrated that attendance levels at Evidence Rounds were determined by the availability and workloads of staff, the organisational climate, the attendance of others (colleagues and senior-level staff), level of interest in the topic and extrinsic factors such as certificates of attendance and continuing education units from a professional body. HCPs who had control over the timing for their daily activities experienced less scheduling-related restrictions compared to those who were providing front-line care on hospital wards. Lunchtime was identified as the most likely time to suit the majority of people. The provision of food and beverages was a facilitator to attendance especially for HCPs who would not get another opportunity to eat during their work shift. Keeping sessions within the advertised timeframe was appreciated by busy HCPs. A number of staff came into work on their days off to attend Evidence Rounds. Some line managers agreed to allow time in lieu for these staff. However, this was not offered to all employees and in general, being off duty was a barrier to attendance. We also identified a previously unknown scheduling conflict with a lunchtime meeting for obstetric staff. This may contribute to the fact that there were fewer attendees from this department. Busy workloads and inadequate staffing levels were barriers to HCPs attending sessions. Understandably, clinical care took priority and staff reported that some colleagues had trouble attending even mandatory training sessions (attendance at Evidence Rounds was voluntary).All staff viewed the interprofessional and multiple disciplinary nature of Evidence Rounds as a facilitator to their attendance. Teamwork and the breaking down of professional silos were among the positive effects they saw from this approach. Consultant attendance and management support for Evidence Rounds was mentioned repeatedly as having a positive effect on non-consultant hospital doctor (NCHD), nursing and midwifery staff attendance. Senior staff acknowledged that their attendance set an example for junior staff. Some HCPs were motivated by a self-perceived benefit to attending e.g., obtaining professional credits for attendance, certificates of attendance or participation, claiming back time spent or enjoying a free lunch Table This domain included two themes of individual level factors, and departmental context and resources. The perceived benefit of taking part and having an interest in the topic or format facilitated presenting at Evidence Rounds. Presenting was considered as a more active way to engage with the literature. Some participants had a long-standing interest in their topic and viewed Evidence Rounds as a platform to promote discussion with colleagues. One participant took part because they wanted to experience giving a presentation in an alternative format to a journal club. Another participant shared that recruiting presenters was a continuing problem.Staffing issues also influenced decisions to present at Evidence Rounds. Even though evidence was presented by 3 HCPs per session, a lot of preparatory work was required from each individual. Another important finding was that some staff were motivated by a strong interest in the topic, a need to set an example for less experienced HCPs or the desire to experience presenting in this unique initiative. Our study found multi-dimensional factors that can be both barriers and enablers to different individuals, at different times and under different circumstances. For example, the level of self-confidence in presenting in front of others could either encourage or discourage potential presenters from taking part.Health care professionals who saw themselves or others as being deficient in knowledge, skills, or education or those without a research background, identified this as a barrier to presenting. For some participants, their taking part was done to motivate others and learn the process so that they could provide assistance to future presenters. Others presented because they were well-versed on the topic and felt confident to present. One participant mentioned their fear of being asked difficult questions by attendees but chose to present regardless.The structure of Evidence Rounds whereby three HCPs presented at each session was encouraging for some staff. Some topics can cause information overload if there is a lot of published evidence so sharing the literature and the workload with colleagues helped to minimise any negative impact on work-life balance. Nevertheless, some HCPs viewed their busy schedules and the extra work associated with presenting as barriers.The transience of junior medical staff was identified as a barrier because they were rotated to different hospital departments or hospitals every 6 months. They were perceived as being less willing to take part because they would be moved soon afterwards. Support from line managers i.e. protected time to prepare for their presentation, was identified as a determinant that would encourage staff to present Table This domain included two themes of acceptability and appropriateness, and pushing and changing slowly. All participants viewed Evidence Rounds as having a positive impact on their practice and education. It highlighted the need to improve their communication with colleagues in relation to approaches to care delivery. Evidence Rounds helped to ensure practice was based on research evidence as well as their own clinical experience and that of their colleagues. The initiative was acknowledged as having a wider scope, decreased risk of bias and more applicability to decision-making for clinical care than traditional journal clubs. Participants welcomed the opportunity for interprofessional collaboration across multiple professions and disciplines and saw this as a means to network and discuss key issues with colleagues they might work with infrequently. There was recognition that getting together for Evidence Rounds sessions helped to foster links between the midwifery, obstetric and neonatal departments.Most participants acknowledged that key research findings highlighted as actions from Evidence Rounds were slow to be implemented although some recommendations had been implemented in practice. Bridging the gap between research and practice is often contingent on additional steps. Evidence Rounds was seen as a platform to begin a conversation and start to plan the formal process of updating or creating new guidelines so that there could be a widespread change in practice. One participant noted that having the relevant guideline developer in attendance would increase the likelihood of getting the evidence into practice Table This domain included two themes; modes of delivery and communication and dissemination strategy considerations. We asked participants questions to gain insight into their preferred modes of delivery when receiving communications and disseminated information. One important finding is that participant feedback did not identify a single mode of delivery of information that would have engaged all staff. Therefore, our decision to employ a multi-component strategy was appropriate for our target population despite a lack of evidence that this is the most effective approach , 28. HCPFinally, we asked HCPs about their perceptions of the sustainability of Evidence Rounds and how they would make future initiatives more sustainable. Sustainability is difficult to measure  so our qThis domain included two themes; staff engagement and collaboration and individual and departmental influences on sustainability. Perhaps the most striking finding is the influence of resources and particularly the HCPs themselves, on sustainability. Their availability, schedules and workloads, level of interest and motivation, the engagement of senior-level staff and their willingness to lead and become champions for initiatives were hugely important factors. These results corroborate suggestions that behaviour change theory may be useful to positively impact implementation processes. HCPs identified a number of factors key to the sustainability of Evidence Rounds and similar initiatives after the support of the KT professional would be terminated. Staff representatives from both the neonatal and obstetric units would need to take ownership and assume responsibility for administrative tasks such as planning and scheduling the meetings. Some participants viewed champions as essential for sustainability. Two participants believed that there needed to be a dedicated person whose job it was to oversee education and another thought their role should include developing clinical practice guidelines.All participants remarked positively on either or both of the interprofessional and multiple disciplinary aspects of the initiative. One individual believed that senior level staff from one discipline were more invested in keeping it going than those from the other discipline and worried about the impact of this. There was a sense that it was not always easy to come up with topics of simultaneous interest to midwifery, neonatal and obstetric departments. Evidence Rounds was just one of many educational opportunities open to staff during their working week. Taking into consideration the already busy workloads of the healthcare professionals, it was not easy to find staff to volunteer to take on the extra responsibility required to keep it going. Buy-in from senior level staff and having a consultant run the sessions were considered factors that might encourage staff to attend. Rotating presenters and dividing tasks between a team of three was a means of keeping the workload associated with presenting at a manageable level.Assigning a HCP to pre-schedule the sessions for the coming year was suggested by multiple participants. Participants mentioned the need to involve someone with experience in performing systematic literature searches and to provide additional support to upcoming presenters Table This study sought to identify the barriers and facilitators to attending and presenting at Evidence Rounds. Our findings agreed with evidence from other studies that the provision of refreshments may be associated with increased HCP attendance at educational events , 31.We wanted to improve our understanding of HCPs’ perspectives of Evidence Rounds as a dissemination strategy. We asked multiple questions to gain insight into their preferred modes of delivery when receiving communication and disseminated information.Overall, our study findings were consistent with a mixed methods study also conducted in Ireland, to reach consensus on priorities for clinical learning environments for postgraduate medical education . Among tEvidence Rounds was an example of pragmatic, community-engaged dissemination and implementation research , 34 in wIt is not clear whether our participants were a representative sample of the population. More than half had presented or were involved in the co-design or implementation of the initiative. Therefore, this group may be more invested in Evidence Rounds than other potential participants. We did not capture the perspectives of healthcare professionals who did not attend Evidence Rounds. The inclusion criteria for our study specified that participants must have attended at least one group session.Another limitation of the study is that the main researcher who implemented the initiative also moderated the focus groups and interviews and was involved in analysis and interpretation. Participants may have felt reluctant to share negative perceptions. To address this concern, at the start of each interview or focus group we emphasised that both positive and negative feedback was being sought to continue the initiative and make it more effective or to make recommendations for future initiatives.In one systematic literature review, the authors reported that a timeframe of two or more years is required to examine the sustainability of evidence based interventions . Tricco Framework analysis uses an applied rather than a theoretical approach to research . TherefoFurther research might explore how to leverage social media platforms to effectively communicate and disseminate evidence to a targeted population. Evidence Rounds was an initiative for HCPs in Ireland, which is classified as a high income country . QuestioThe findings of this study uncovered a number of important points to inform individuals planning, developing or implementing initiatives aimed at HCPs. We encourage others to consider interprofessional and multiple professional/disciplinary platforms for these types of initiatives as this approach was valued highly by staff. Those planning similar initiatives may consider multi-component strategies. Our HCPs found more benefit relating to the provision of patient care in group sessions focusing on the best available evidence than on previous events like journal club which critically appraised single articles. Effective communication and dissemination aimed at HCPs requires careful consideration of a number of factors including the mode of delivery, scheduling, frequency, and organisational, departmental and individual-level preferences. Feedback during implementation from the target population may guide decisions to maintain, remove or modify aspects of the strategy. Others implementing similar initiatives may consider factoring in the provision of support and training for presenters who need help with critical appraisal, data presentation, statistical inference etc. The development of a plan for presenters and attendees would be ideal to build organisational capacity. Our health service staff did not feel that they had the skills to perform adequate searches on clinical topics or questions. Like other authors, we recommend the involvement of information specialists, librarians or individuals with experience of designing and conducting search strategies . We alsoWe set out to identify barriers and facilitators to attending and presenting at group sessions from the perspectives of HCPs, to gain an understanding of their views of Evidence Rounds as a dissemination strategy and to generate insights to improve the sustainability of future initiatives. The results of this study and our analysis have extended our understanding and may be useful for guiding the development and implementation of future KT strategies for HCPs. Our focus groups and interviews highlighted the variability in preferences of mode of delivery in our target audience suggesting the multi-component approaches can be useful. They helped us gain insight into the influence of organisational and individual level factors  on the willingness and ability of HCPs to attend, present at and sustain these types of initiatives.Although HCPs invariably work in complex systems with unique contexts, this paper may help others to understand factors that can impact the implementation of initiatives to disseminate key research findings and promote evidence informed practice.Additional file 1:FGI Guide_BMC Med Ed. (PDF 261 kb)Additional file 2:Informed consent_BMC Med Ed. (PDF 140 kb)


Health care delivery and outcomes can be improved by using innovations  supported by scientific evidence. However, scientific evidence may not be the foremost factor in adoption decisions and is rarely sufficient. The objective of this study was to examine the role of scientific evidence in decisions to adopt complex innovations in cancer care.Using an explanatory, multiple case study design, we examined the adoption of complex innovations in five purposively sampled cases in Nova Scotia, Canada. Data were collected via documents and key informant interviews. Data analysis involved an in-depth analysis of each case, followed by a cross-case analysis to develop theoretically informed, generalizable knowledge on the role of scientific evidence in innovation adoption that may be applied to similar settings and contexts.The analyses identified key concepts alongside important caveats and considerations. Key concepts were (1) scientific evidence underpinned the adoption process, (2) evidence from multiple sources informed decision-making, (3) decision-makers considered three key issues when making decisions, and (4) champions were essential to eventual adoption. Caveats and considerations related to the presence of urgent problems and short-term financial pressures and minimizing risk.The findings revealed the different types of issues decision-makers consider while making these decisions and why different sources of evidence are needed in these processes. Future research should examine how different types of evidence are legitimized and why some types are prioritized over others.The online version of this article (10.1186/s13012-019-0859-5) contains supplementary material, which is available to authorized users. Contributions to the literatureResearch has shown that the scientific evidence does not always have a large influence on decisions to adopt innovations in health care. For many decision-makers, experiential knowledge can be more relevant and applicable.Although we found scientific evidence typically underpinned the adoption process, the types of evidence most valued by strategic-level decision-makers were insights into real-world implementation challenges and impact obtained from other jurisdictions.These findings contribute to recognized gaps in the literature, including ascertaining how, when, and why different types of evidence are used during decisions to adopt innovations in health care.Health care delivery and outcomes can be improved by using innovations  supporthow and why different factors come into play during the policymaking process . Interviews were carried out face-to-face or via telephone, depending upon practical considerations. All were audiotaped to ensure the data were captured and retrievable in true form, and transcribed verbatim by an experienced research coordinator. The interviewers met regularly to review the questions, ensure they were being answered in sufficient depth, and revise the questions or probes, as needed, before subsequent interviews.Detailed case descriptions were constructed for each case to describe its history, organization, and context. Data analysis involved an in-depth analysis of each case, followed by a cross-case analysis. First, the cases were treated as separate studies and analyzed independently using a thematic analysis approach , which iFor each case, codes were collapsed into categories and larger themes through an iterative process that included critically analyzing each concept and category to identify similar and distinct concepts and categories; linking concepts and categories across all the data collected; reviewing the study objectives, conceptual framework , and other literature sources; regular meetings with three team members ; and discussions with the full research team to facilitate questioning of the data and its analyses. The findings of each case were fully delineated before moving to the next case.Next, a cross-case analysis was conducted to develop theoretically informed, generalizable knowledge on the role of scientific evidence in innovation adoption that may be applied to similar settings and contexts . MultiplAcross the five cases, data were collected from 32 key informants and > 100 documents. All interviews took place between December 2014 and April 2015. Key informants played differing roles in the cases, including advocating for the innovation, making formal requests for adoption/funding, and/or making the eventual adoption decision. They included clinical and administrative champions, departmental managers, department chiefs/heads, executive-level administrators , policymakers from the provincial ministry of health, and community advocates. Due to the nature of their decision-making role, two key informants played roles in more than one case and were questioned about their role in each case.The five selected cases are described in Table The cross-case analysis revealed similarities across most cases. However, one case (case 5) fundamentally differed from the other cases in terms of the role of scientific evidence and the impetus for adoption. The findings are discussed below and broadly categorized into the “We relied on the expertise of the Diagnostic Imaging professionals … I cannot speak to the evidence, I cannot point to the studies, except to say that it was confirmed with us, by both the hospital executive and DI, that there was ample evidence that this was the standard for technology. [P18]The documentary and interview data revealed the adoption process played out on a continuum, from someone’s initial conviction the innovation should be implemented , to advocating for the innovation , to the decision to adopt the innovation at the departmental, organizational, and/or healthcare system levels. With the exception of one case (case 5), scientific evidence underpinned these processes. However, scientific evidence played a much greater role in early adoption processes when frontline personnel decided to advocate for the innovation. During subsequent decision-making , scientific evidence played a limited explicit role in decision-making processes, with decision-makers typically trusting the individuals who brought the innovation forward. As stated by one decision-maker involved in case 1:A decision-maker in case 4 put it this way: “in a perfect world [where] we had lots of staff and lots of time … we’d probably have a more rigorous process and more data considered but … you know, we’d need a whole department for this” [P17]. In other words, at the formal approval levels, scientific evidence received only limited attention and consideration since most individuals at these levels felt the innovation would not be under consideration unless there was scientific evidence to support its adoption. In fact, application and proposal documents included limited information about the scientific basis for adoption. Rather, at these levels, both documentary and key informant data revealed issues related to capacity and costs dominated decision-making processes.In addition to scientific evidence, the data showed evidence from multiple sources informed decision-making, including (i) clinical experience, (ii) local data, (iii) patient experience, and (iv) information from other jurisdictions. Table … to see whether we have this ability or capacity in Nova Scotia and to see whether the technique is really robust … it’s just showing local data to convince [the] committee you know, yes, [we] can do it. It can work here.” [P24]The use of local simulation data was also particularly useful in quantifying the advantage of this technology over conventional radiotherapy for the local patient population and thereby strengthening the support of local clinicians. Similarly, local data were key to influencing decision-makers to approve microsatellite instability (MSI) testing in Nova Scotia (case 3). Specifically, data demonstrating high accuracy and reproducibility of the test locally was paramount to the adoption decision. As one participant stated:Use of patient experience was explicit in case 1 only. Patient representatives, speaking at town hall meetings and with government officials and media, described the benefits of positron electron tomography (PET) and shared their experiences with having to access this technology outside the province. Their stories were well-articulated, emotive, and illustrative of the potential benefits of PET, helping to convince decision-makers and fundraising parties of the need for this technology in Nova Scotia. On speaking about the influence of patients, one participant said, “Even though [PET] had all these implications for research, you know, and … patient care and all these sorts of things … the part that really resonated with people was the story [patients] told” [P18].potential impact for patients or the health system, understanding what had happened in other jurisdictions provided evidence into the implications and impacts outside of controlled research environments. As such, any scientific evidence that entered the decision-making process had to be negotiated alongside other sources of evidence, particularly as it provided limited guidance for action. One participant summarized the experience of scientific evidence and information from other jurisdictions by saying, “I think people prefer, you know, the highest level of research evidence possible. Uhm, but you know, there is great consideration given to what happens in other jurisdictions” [P17].Information from other jurisdictions was a particularly valued source of evidence for organizational-level decision-makers, providing them important insight into implementation challenges and real-world impact. This was especially pertinent to cases 2, 4, and 5. Whereas scientific evidence could provide insight into the Related, the role of scientific evidence appeared less pertinent to adoption decisions when the innovation had been extensively adopted elsewhere, with the data suggesting real-world evidence and experience gained from others’ adoption was more highly valued.Multiple sources of evidence sources were used given that decision-makers considered three key issues when making decisions: (i) expected budgetary and operational implications, (ii) expected impact on patients, and (iii) equitable access to care. These three issues prevailed regardless of what level the adoption decision occurred  though the different issues did not receive equivalent attention across cases or levels of decision-making.The last one was a big lab meeting… and we were kind of a little bit grilled. And, you know, what would this be and you are going to pay the cost and it was mostly about budget … So it was mostly people who were worried about dollars … it felt very much like, ‘tell us how this is not going to cost us anything and we might let you do it.’ [P27]Across all cases, cost considerations and financial pressures were pervasive. This was highlighted by a participant in case 3 when describing the approval process:Sometimes those big initiatives get juggled around from year to year because there’s just not enough money in the pot. And here’s the thing, there’s no unworthy things that come to our table. … There’s nothing that you can say that’s frivolous and we do not really need it. It’s unlimited need. So we are, we do ration, you know? [P14]While cases 1 and 4 involved substantive capital expenditure costs, decision-making often involved consideration of other types of costs, including resource/training costs, ongoing operational costs, and opportunity costs. Regarding the latter, decision-makers continually remarked on the ubiquitous nature of competing priorities and recognized investing in an innovation will mean resources are not provided to other important programs, services, or innovations. This was summarized by one participant as:Another key budgetary and operational consideration related to maintaining scope. Specifically, decision-makers recognized that once an innovation is implemented, it is difficult to maintain boundaries around its use, particularly in a complex clinical care environment where evidence continues to emerge and evolve. This was true for cases 1–3, which were clinical (diagnostic and therapeutic) innovations. As stated by one participant in case 2, “We couldn’t open the floodgates. We couldn’t overload the system with a whole range of indications that could be treated, even though many, it seemed at that time, would benefit from this technology” [P1]. Thus, policies were put in place to ensure use of the innovations would be limited to specific indications and referring clinicians to contain costs and resources once implemented.When a patient gets xerostomia, it’s terrible. They have to carry around a water bottle for the rest of their lives because the damage is permanent, and pretty much every nasopharyngeal patient had that side effect prior to this technology … So there was some very compelling data … and the answer was that this side effect was virtually gone. [P1]The second key issue was expected patient impact, that is, expectations about the nature and magnitude of patient impact were important criteria when considering a particular innovation given competing priorities. Expectations of patient impact were informed by a variety of sources, including the scientific evidence base and information from other jurisdictions. For example, one participant from case 2 noted the tremendous impact IMRT has on patient quality of life and how this influenced adoption:Another participant more generally noted, “there has to be a benefit somehow to patient care. The cost is something, probably the next thing we look at” [P10].The fact that we didn’t have [PET] in Atlantic Canada was also a great motivation to bring it here because it said that the standards of care … were better frankly, everywhere else in Canada except here. [P18]The third issue in the decision-making process was whether or not a particular technology or service was standard of care elsewhere—specifically, equitable access to care was viewed as an important value, and if other Canadians had access to a particular technology/service, then patients in Nova Scotia should as well. Those advocating for the innovation and its adoption often presented this equity argument, highlighted by the following participants from cases 1 and 2, respectively:Our argument was always like ‘we can do it here, [so] why can we not do it here?’ Why should patients have to travel all that way to Toronto? They do not want to travel. Who wants to travel out of province for a treatment? And so we said ‘well look, if we don’t do this now, we’re always going to be sending people out of province, and what good is that?’ [P4]At the same time, decision-makers at the organizational and system levels described reconciling their goal of equitable access with real-world cost and resource considerations as well as expected patient impact .Across all five cases, local champions, who were dedicated to the innovation and promoted it tirelessly within their departments and organizations, were critical to eventual adoption. In cases 1–3, it was the mounting scientific evidence that influenced these individuals to become champions for the innovations. Champions were instrumental in motivating others to support the innovation and overcoming barriers to its adoption and eventual implementation. These individuals undertook a number of activities, including gathering and disseminating multiple types of evidence to communicate benefits, formally and informally advocating for the innovation across levels of the organization/system, collecting local data to demonstrate need and/or value, and lobbying with external players  to garner moral and material support. For cases wherein barriers were substantial , the champion played a larger role in selling and moving the idea forward. As one participant stated, “he is the driving force behind it … The reason the PET machine is here is because of [him]. He was the champion for it and he wouldn’t take no for an answer. He made it happen” [P18]. In fact, the data suggested even in cases were the evidentiary basis and existing infrastructure were robust, adoption would not have happened without a champion continually pushing the innovation forward.do something about medication access within the cancer center. At the same time, frontline staff  felt unable to effectively navigate the increasingly complex environment of medication access and to adequately meet the high psychosocial needs of their patients because of the large amount of time they spent on medication access issues.So it was hugely time consuming … and it was also difficult to keep in your head all of the ins and outs and the complexities of all of these medications … we did not always have the capacity to be able to jump through all the hoops quickly enough because there were so many needs … So more and more and more time was being spent on medication resource issues and less and less time was being spent on practicing scope. [P28]Case 5 differed in many respects from the other four cases. In particular, the innovation  was implemented due to a perceived need to address an urgent problem at the local level. Specifically, the organizational  and socio-political  context at the time resulted in an urgency to In fact, documentary data demonstrated that, in 2003, social workers spent more than 30% of their time coordinating medication access and funding.In terms of this particular medication resource specialist, as far as my understanding was, there was no actual evidence to support that it was going to help us … it was just such an obvious need you didn’t need any evidence. [P29]While the need was informed by various sources of evidence , there was limited scientific evidence available to inform the development of the resulting innovation. This was expressed by one participant as:Thus, the data suggested the differences inherent in this case were due to the nature of the problem, and not necessarily the nature of the innovation. Development of the MRS position was viewed as a creative solution to a substantial local need, and one that could generate evidence versus being informed by evidence.Case 5 was also unique in that the solution was addressing a cost and resource problem, with administrators anticipating substantive cost benefit in the short term. Specifically, as cancer center staff felt unable to effectively navigate the complex environment of medication access and coverage, they increasingly relied on the hospital’s No Insurance (NOINS) program, which covered urgent medication costs for patients when all other options were exhausted. One participant described the perception that NOINS “was being overused, not because people thought it was easier or an easy way out, it was purely because people didn’t know what else to do” [P31]. As these costs escalated, a solution to medication assistance became critical. Many participants perceived this cost crisis as central to administrators’ support for the innovation and its expediency into practice. Unlike the other cases, the MRS position was directly solving a problem for administrators themselves in addition to a clinician and patient problem. Indeed, administrators were highly involved in and supportive of the innovation and allowed it to evolve over time as needed, as noted by one participant who stated: “we had good support from our senior leader” [P32]. Documentary evidence demonstrated a 47% reduction in NOINS costs over the first 3 years of the position.For cases 1–4, data pertaining to decision-making at the organizational and/or health system levels uncovered the wide-ranging view that adopting innovations after they have been adopted and implemented elsewhere had discernible advantages. Specifically, adopting later in time (relative to peer institutions elsewhere) minimized risk and allowed managers/administrators to acquire valuable evidence from elsewhere to understand the resource implications and real-world patient/health system impacts of implementation. This evidence could inform adoption and implementation efforts and lower any ambiguity associated with the innovation. This was described by one participant who said, “Yes, it’s new, it’s great, it’s wonderful. But we’re not ready for that. And let’s let some other institution get the bugs out and then we’ll go forward” [P11].Participants across these four cases described an implementation cascade where innovations were developed and implemented elsewhere, and eventually trickled down to their organization. While this cascade had ostensible benefits for decision-makers, frontline clinicians were continually frustrated by what they saw as a lack of innovation and improvement in their respective settings.This study sought to better understand the role of scientific evidence in adoption decisions. We found scientific evidence typically underpinned the adoption process (with the exception of one case), yet had a declining role in decision-making as the decision progressed to high levels of the organization and system. We also found that decision-makers often relied on multiple types of evidence, which were necessary due to the multiple issues they considered as they made their decisions. The most relevant type of evidence for organizational-level decision-makers was information from other jurisdictions that had previously implemented the innovation since this evidence provided them with key insights into implementation challenges and real-world impact. These findings contribute to recognized gaps in the literature base, including ascertaining how and why different types of evidence are used during decisions to adopt innovations in health care , 16.Research has shown that the strength or quality of scientific evidence does not always have a large influence on the decision to adopt innovations in health care , 28. ForIn four cases, decision-makers used local data to inform their decisions on whether or not to adopt the innovation. While the nature of the local data varied , the findings suggested the desire for local data was twofold: it allowed decision-makers to better understand the nature and scope of the problem in their setting and/or enabled them to better estimate the local impact on their patient population. Indeed, when deciding to adopt complex innovations, many of which are resource-intensive, decision-makers must negotiate the knowledge base  with local needs . The proclivity for local data aligns with a recent study on policymakers’ uses and preferences for evidence in public health whereby policymakers reported local data as their most highly valued type of evidence .Organizational decision-makers placed especially high value on information they acquired from other jurisdictions. This information was particularly germane to evaluating the potential risks of adoption in that it provided real-world experiences about the budgetary, operational, and patient impacts of the innovation, as well as implementation challenges and successes. Thus, while such evidence may be used to understand the real-world benefits of an innovation, it may also be used to mitigate potential risks to the organization. Decision-makers across cases discussed acquiring such information from colleagues in other jurisdictions, gray literature sources, industry/vendors, and/or networking at meetings and conferences. Recent studies have shown that policymakers in public health report their most frequent source of evidence as “other people”   and servThe enabling influence of on-the-ground champions cannot be understated. Across all cases, the presence and role of champions was paramount to decisions to adopt the innovations. The importance of champions in adoption and implementation processes has been demonstrated in many studies –40 and iCase 5 differed from the other cases in many respects. Certainly, the nature of this innovation  was quite different than the other innovations . At the same time, this case may be characterized as a problem-based initiative wherein a solution was sought to a perceived urgent problem, whereas the other cases may be characterized more as solution-based initiatives with clinicians advocating for specific solutions (innovations) they perceived would benefit patient care, safety, or outcomes in their respective areas. Moreover, the data suggested the differences inherent in this case were due to the nature of the problem and not necessarily the nature of the innovation, that is, senior administrators were keenly aware of the high, and ostensibly avoidable, cost and resource burden of the existing way of accessing medications—and reducing these costs became a clear priority of these organizational leaders. Indeed, innovations are more likely to be adopted when it aligns with the organizational needs and addresses clear, practical problems . CriticaThere are several practical implications of these findings in terms of informing adoption efforts. One relates to the value of information on real-world implementation and impacts. Given that decision-makers must always examine the potential benefits and costs  of an innovation , real-woThis study has a number of limitations. First, this study was conducted in one jurisdiction only, which may limit the applicability of findings to other settings. However, our sampling strategy ensured the cases varied on a number of key features  and thus sought to optimize transferability across types of innovations and contexts. In addition, healthcare systems generally have a number of defining features regardless of their location, including wide-ranging and diverse stakeholders and complex governance arrangements . Both shSecond, in each case, some individuals were difficult to reach because they had changed positions or retired since the innovation was adopted and implemented. As a result of not reaching certain individuals, we may have missed some important data. Similarly, given that the time period between adoption and interviews was lengthy for several cases , and health administration, ensuring the data were viewed and questioned through multi-disciplinary perspectives. As such, this study contributes important data to help address identified gaps in the literature on how evidence is used in decisions to adopt innovations in health care organizations.This study demonstrated how research evidence is considered among other sources of evidence as decisions are made to adopt innovations in health care. The findings revealed the different types of issues decision-makers consider while making these decisions and why different sources of evidence are needed in these processes. Future research should continue to examine how evidence is used in adoption decisions, including how different types of evidence are legitimized and why some types are prioritized over others. Given the high value placed on information from other jurisdictions, a better understanding of how this evidence is constructed and applied during decision-making processes is also warranted.Additional file 1:Draft Interview Guide: Clinician or administrator involved in the adoption process. (DOC 29 kb)


Significant resources are required to write de novo clinical practice guidelines (CPGs). There are many freely-available CPGs internationally, for many health conditions. Developing countries rarely have the resources for de novo CPGs, and there could be efficiencies in using CPGs developed elsewhere. This paper outlines a novel process developed and tested in a resource-constrained country (South Africa) to synthesise findings from multiple international CPGs on allied health (AH) stroke rehabilitation.Methodologists, policy-makers, content experts and consumers collaborated to describe the pathway of an ‘average’ stroke patient through the South African public healthcare system and pose questions about best-practice stroke rehabilitation along this pathway. A comprehensive search identified international guidance documents published since January 2010. These were scanned for relevance to the South African AH stroke rehabilitation questions and critically appraised for methodological quality. Recommendations were extracted from guidance documents for each question. Strength of the body of evidence (SoBE) gradings underpinning recommendations were standardised, and composite recommendations were developed using qualitative synthesis. An algorithm was developed to guide assignment of overall SoBE gradings to composite recommendations.Sixteen CPGs were identified, and all were included, as they answered different project questions differently. Methodological quality varied and was unrelated to currency. Seven clusters, outlining 20 composite recommendations were proposed .The methodological development process, tested by writing a South African AH stroke rehabilitation guideline from existing evidence sources, took 9 months. The process was efficient, collaborative, effective, rewarding and positive. Using the proposed methods, similar synthesis of existing evidence could be conducted in shorter time periods, in other resource-constrained countries, avoiding the need for expensive and time-consuming de novo CPG development.The online version of this article (10.1186/s13012-018-0803-0) contains supplementary material, which is available to authorized users. Over the past decade, the South African burden of disease has swung towards chronic conditions, trauma and disability, as more lives are saved from communicable diseases , 2. The Effectively implementing evidence-based practice, particularly using comprehensive high quality clinical practice guidelines (CPGs), has been the subject of considerable research in high-income countries over the past three decades . Much ofResearch into evidence implementation and uptake in low and lower-middle-income countries has largely been in knowledge translation into policy, which has concurrently identified gaps between research and end-user/stakeholder needs for guidance . The chaThis paper outlines a novel process of using existing best evidence sources to develop new, composite, implementation-ready, evidence-based recommendations. The process was tested on allied health (AH) rehabilitation for a high-burden health condition in a resource-constrained environment (stroke). AH disciplines such as physiotherapy, occupational therapy, speech and language therapy, clinical nutrition and social work provide rehabilitation services to stroke survivors, with the aim of optimising function and quality of life. A 2015 summary of systematic reviews of the impact of AH care on stroke rehabilitation outcomes showed significant economic benefits . Key barThe World Health Organization (WHO) notes that AH rehabilitation services are generally poorly accessed and/or suboptimal in many low and low-middle economy countries . The WorStroke is a leading cause of disability worldwide . Over thIn SA, stroke is a high health burden , 2. It iSouth African AH stroke rehabilitation services are currently not supported by local CPGs that summarise the AH literature . MoreoveThe aims of the research were to develop a template for producing defensible, locally-implementable recommendations from existing evidence sources, and test this by writing an implementation-ready SA-contextualised AH stroke rehabilitation guideline relevant to any South African public healthcare setting.The RIGHT Statement was applied  See Add.Stellenbosch University Africa Human Research Ethics Committee (#0642).Seed funding was provided from Stellenbosch University and the WHO Alliance for Health Policy and Systems Research in February 2017. The funders had no influence on research design, conduct or reporting.This comprised 15 expert clinicians, academics, policy-makers and consumer representatives and five researchers. Other than the consumer representative (a stroke survivor), each team member had an AH discipline background. Each team member brought different understandings of stroke rehabilitation, such as how, when and why stroke rehabilitation is offered, what care is provided, who provides it, how it is received and in what rehabilitation settings. Project funding was sufficient for travel and meeting costs and to support one researcher 2 days/week. All other team members participated voluntarily or as part of usual work commitments.The timeframe was short (9 months) (Feb–Nov 2017), reflecting the urgent need for effective, efficient, equitable and safe AH rehabilitation for patients with acute and chronic stroke across South African healthcare settings. There was no time or funding to develop a de novo CPG for South African AH stroke rehabilitation, which outlined ‘what’ to do. The focus needed to be on implementing best evidence into local practice, by addressing service delivery questions of ‘who’, ‘how’, ‘when’, ‘where’, ‘why’ and ‘how much’. The project team agreed that existing CPGs would form the evidence base.The research adopted the WHO characteristics for good quality service delivery . This seAny document published from 2010, which provided freely- and publicly-available guidance to inform best practice AH rehabilitation for any adult stroke sufferer was eligible for consideration. The 7-year search window encompassed the usual five-yearly update period for CPGs . For incTo ensure the common understanding of purpose and an efficient focus for discussions, agreement was reached at the first meeting on the underpinning project premises  . ProjectA comprehensive search of internet repositories was undertaken to identify guidance documents for AH stroke rehabilitation published since January 2010, by any organisation, in any country. The search strategy is provided in Additional file Irrespective of their nomenclature  , methodohigh quality (HQ) being 80% + of the total possible AGREE II score,moderate quality (MQ) being 60–79% of the total possible AGREE II score andpoor quality (PQ) being < 60% of the total AGREE II score.Independent scorer dyads, experienced in using AGREE II, scored randomly-allocated guidance documents, and the scoring rubric was calculated independently using a MSExcel macro , 23. Whi‘Usual’ patient care pathway(s) were established during the first project team meeting . These oThe project team then discussed AH rehabilitation activities related to the patient care pathways and raised issues for which there was variable practice and/or uncertainty about what to do. This process highlighted the complexities of delivering best-practice SA stroke rehabilitation, particularly the need for guidance on the workforce, training, organisation of services and communication. These issues are reported in the second layer of Fig. The team constructed 47 questions for which evidence-based answers were required. Questions were classified relevant to second tier of the patient pathway in Fig. The project questions were condensed to 38 by team discussion Table . These wThe project team anticipated that no one guidance document might answer all project questions. This was because of the specific nature of the South African stroke rehabilitation project but also because of the different purposes for which current international AH stroke rehabilitation guidelines might be written. Thus, if a guidance document identified in the search did not answer a project question, it was assumed that this question was not relevant to its scope or purpose.The project team identified, discussed and agreed on key questions which, if answered by current best-evidence, could improve the quality of South African AH rehabilitation. A five-step process was taken.The scope, purpose and questions addressed in the guidance documents identified in the search were mapped to the South African project questions. To be included, a guidance document should answer at least one South African project question, provided in the form of a recommendation (this is defined in a later section). Poor methodological quality was not a reason for exclusion, as quality was taken account in the calculation of the overall strength of the body of evidence (OSoBE) (see later section).A summary of the relevant evidence usually underpins recommendations in good quality guidance documents evidence, reported as SoBE , 26. Berwording in guidance documents that was clearly labelled ‘recommendation’  and accompanied by a SoBE grading, orwording that appeared in the text, not necessarily labelled ‘recommendation’ but which had the intent of being a recommendation, in terms of intention words such as ‘should’, ‘could’, ‘might consider’, accompanied with an SoBE grading.There was no standard way of defining a ‘recommendation’. Expanding Alper’s  and SchiNot considered as recommendations were words which appeared in the body of guidance documents which were not labelled ‘recommendation’, did not have the intent of a recommendation or had no SoBE grading.Separate purpose-built data extraction sheets were developed for each project question. These recorded the details of the guidance documents which answered it , recommendations relevant to the question, and the SoBE gradings for recommendations (in whichever way they were reported). Moreover, any guidance document which provided ‘how to do it’ information was identified as potentially providing useful tier 3 documents. These could include, but were not limited to, protocols, patient management or service decision-making tools, organisational flowcharts, stroke team construction, workforce issues, assessment criteria, specific assessment tools, outcome measures, minimal clinically significant changes from interventions, discharge planning checklists and patient information material.A common, readily interpreted set of ‘faces’, was developed for this project to standardise the different ways in which SoBE was presented in included guidance documents. A smiley face was used for positive evidence, a neutral face for insufficient or conflicting evidence and a frowning face for negative evidence. Details are provided in the ‘Results’ section.those which were not answered by any guidance document (absence of evidence),those answered by only one–two guidance documents (scant evidence) andthose with inconsistent SoBE (inconsistent evidence).A summary table was developed from the individual data extraction files, reporting the guidance documents which provided answers to each project question (volume of evidence), year of production of guidance documents (currency of evidence), their methodological quality and the consistency of recommendations . Project questions were then classified assubjective experience within the “taken-for-granted, commonsense” world of the daily life of individuals’ and takes the view ‘that people living in the world of daily life are able to ascribe meaning to a situation and then make judgments. It is the subjective meaning of experience that was the topic for interpretation in this study’ ).Cochrane –31, 33.For each project question, researcher dyads used an inductive analysis approach and discussed and compared wording, content, intent and meaning of recommendations extracted verbatim from guidance documents. Draft composite recommendations for each question were proposed to the other members of the project team, which discussed and ratified them for relevance to South African contexts.the determination methods of Gonzalez-Suarez et al.  . The AGREE II domain scores are reported in Additional file Methodological quality varied  is provided in Fig.  is proviMost of the composite recommendations were underpinned by moderate to good OSoBE, which provided believable tier 1 evidence to support contextualised implementation of recommendations into SA healthcare settings. Additional file This is the first research of which we are aware that has developed and tested processes to transparently summarise recommendations from multiple guidance documents, to answer questions relevant to a local context problem. Our project builds on the research reported by Alper  and Shifstandardise differently reported SoBE gradings,synthesis intent and wording from multiple recommendations that reported the evidence for one question in different ways anddetermine an OSoBE grading for a composite recommendation which summarised recommendations from multiple guidance documents, for one question.The new methods address the lack of guidance in the literature about how toThe test vehicle for this work was AH stroke rehabilitation delivered in a country (South Africa) that cannot afford the time or finances to develop its own de novo CPG. Moreover, because of the burden that stroke imposes on the SA economy, on communities, families and individuals, AH stroke rehabilitation in SA needs best evidence recommendations to ensure that scant resources are used wisely and equitably , 13, 15.These new methods add to discussions on methods of CPG repurposing and transferability from one context to another. The methods should assist other South African CPG teams and teams in other resource-constrained organisations or countries, to efficiently contextualise currently available guidance documents for other health conditions. Our work provides a simple, new step-by-step approach that focuses efforts on evidence implementation, rather than on development of de novo CPGs , when thProject team members with little background in methodology found the step-by-step process simple to understand, and they required little further explanation than was provided in the figures reported in this paper. The simplicity of the process meant that future work should be able to be undertaken with minimal training by non-academics  or even students. Those managers and clinicians on the project team who were less focused on methodology, and more interested in recommendation relevance and implementation in local settings, validated the usefulness of the wording and intent of the composite recommendations, as well as the way the OSoBE had been developed. They indicated that the recommendations aligned with their understanding of current best practice undertaken elsewhere in the world. The policy-makers expressed confidence that the composite SA–relevant recommendations positioned them to progress evidence-informed discussions on future funding and policy priorities. Consumers considered that the process encouraged engagement of consumers on future guidance teams. The recommendations would also improve SA patient understanding of rehabilitation, as well as shared decision-making.Integrating existing recommendations was far more efficient than starting ‘from scratch’ with a de novo CPG oriented to South African contexts , 26. ThiThe need for this project was highlighted as there was no one international CPG which answered all the project questions. Moreover, the project questions contained few clinical ones (‘what to do’) and mostly focused on implementation of evidence locally . This validated the theoretical model proposed by Project SAGE , of usinThe importance of including the multiple components in the OSBoE decision-making framework was highlighted by the variability in the number and currency of included guidance documents providing answers to project questions. The differing SoBE gradings for extracted recommendations relevant to a project question mostly related to the underpinning research design(s) which formed the evidence base. This reflects a fundamental issue with determining appropriate SoBE gradings related to AH stroke rehabilitation. The notion of SoBE has its genesis in intervention studies, largely delivered using a medical model , 54, 55.The applicability of our novel method requires further testing by others, in other healthcare settings, for stroke and other health conditions. The proposed methods to combine recommendations and SoBE gradings need validation in different types of research evidence. However, critical to update of this method is the issue of transferability of recommendations from existing CPGs to new environments. This must be guided by tier 2 . Evidence that has been developed on one population may not be immediately transferrable to other populations, even if patients come from similar socioeconomic environments , 28, 29.The limitations were that a new process was developed and tested on one set of questions . This condition provided a vehicle. The project team was learning as it went, and errors and mis-interpretations were sure to have been made, particularly in the formulation of composite recommendations from the included guidance documents. The process of combining evidence sources into one composite set of recommendations needs to be tested by other teams for other conditions.Our practical innovative methods add to the scarce body of evidence in repurposing and transferring CPGs from one etting to another. Given the urgent need to implement evidence in resource-constrained settings to reduce alarming disease burdens, the costs and time to develop de novo CPGs, and the untenable burden the lack of appropriate evidence places on many low- and lower-middle economy countries, our methods provide a feasible, efficient process to repurpose and transfer CPGs, rather than produce new ones. The SA AH stroke rehabilitation recommendations developed using this new method will ensure that appropriate treatments are offered at appropriate times, in appropriate ways, relevant to local contexts, to improve stroke survivors’ access to ongoing care and quality post-stroke lives.Additional file 1:Underpinning premises for this study. (DOCX 55 kb)


The World Anti-Doping Agency is responsible for maintaining a Prohibited List that describes the use of substances and methods that are prohibited for athletes. The list currently contains 23 substance classes, and an important reason for the existence of this list is to prevent unfair competition due to pharmacologically enhanced performance. The aim of this review was to give an overview of the available evidence for performance enhancement of these substance classes. We searched the scientific literature through PubMed for studies and reviews evaluating the effects of substance classes on performance. Findings from double-blind, randomized controlled trials were considered as evidence for (the absence of) effects if they were performed in trained subjects measuring relevant performance outcomes. Only 5 of 23 substance classes show evidence of having the ability to enhance actual sports performance, i.e. anabolic agents, β2-agonists, stimulants, glucocorticoids and β-blockers. One additional class, growth hormone, has similar evidence but only in untrained subjects. The observed effects all relate to strength or sprint performance (and accuracy for β-blockers); there are no studies showing positive effects on reliable markers of endurance performance. For 11 classes, no well-designed studies are available, and, for the remaining six classes, there is evidence of an absence of a positive effect. In conclusion, for the majority of substance classes, no convincing evidence for performance enhancement is available, while, for the remaining classes, the evidence is based on a total of only 266 subjects from 11 studies. Medical or other scientific evidence, pharmacological effect or experience that the substance or method, alone or in combination with other substances or methods, has the potential to enhance, or enhances, sport performance.Medical or other scientific evidence, pharmacological effect or experience that the use of the substance or method represents an actual or potential health risk to the athlete.WADA’s determination that the use of the substance or method violates the spirit of sport, described in the introduction to the World Anti-Doping Code.The mission of the World Anti-Doping Agency (WADA) is to lead a collaborative worldwide movement for doping-free sport, and its activities focus on the responsibilities given by the World Anti-Doping Code . One of The third criterion is clearly most subjective and is more a fundamental and philosophical question than a scientific one . Howeverthe potential to enhance performance is sufficient to place a substance on the Prohibited List, in this review clinical pharmacological evidence for actual performance enhancement was considered essential to determine that a substance or category of substances has a positive effect on performance. In other words, similar to any other therapeutic review, to make an evidence-based conclusion that there are performance-enhancing effects, the level of evidence should preferably be high (level 1), meaning that evidence should come from double-blind, randomized controlled trials  , increased running endurance . Howeversurement . In bothWith regard to insulin, Kuipers and van Dugteren indicated that based on several observations, this drug is not expected to have a physiologically significant effect on muscle growth, even in combination with glucose and/or amino acids . HoweverFinally, inhibitors of fatty acid oxidation belong to this category. Meldonium is classified as a partial inhibitor of fatty acid oxidation, but, in a recent editorial, Greenblatt and Greenblatt concluded that no studies have evaluated the performance-enhancing properties of meldonium in trained subjects . AnotherVO2max compared with baseline measurements, but not compared with controls, in untrained subjects. Armstrong et al. [VO2 uptake during this test [VO2max, but time to exhaustion was reduced by 29% in a continuous exercise to exhaustion [VO2max and Pmax [The category of diuretics and masking agents is not necessarily included in the Prohibited List for their potential to enhance performance. Masking agents are supposed to interfere with analytical testing of markers or other substances on the Prohibited List. Diuretics increase urine production and by this effect are thought to dilute, and therefore interfere with the detection of, banned substances in urine. This increased water excretion caused by diuretics might also improve performance as it can quickly reduce weight, which might give a competitive advantage. In sports with weight classes, for example, this effect could place athletes in a lighter category, and, in speed or endurance sports, lighter athletes might have an advantage. Cadwallander et al.  reviewedg et al.  found thhis test . Anotherhaustion . Finallyand Pmax . An addiand Pmax  showed tand Pmax , but didThere are several non-pharmacological interventions that are prohibited at all times, termed prohibited methods. These are manipulation of blood and blood components (e.g. blood transfusion), chemical and physical manipulation (e.g. tampering with a sample or intravenous infusions of fluid) and gene doping. As this review focuses on pharmacological interventions, evidence for the effects on performance of these categories is not discussed here.VO2max was not affected in either study and endurance performance  was not investigated in these studies. Of note, the former study used no baseline correction  and, for the latter study, it is unclear whether it was (double-)blinded, which may both make the results less robust. Another study with a higher dose of methylphenidate showed no effect on time-trial performance in normal temperature, but there was an improvement of 15% average power output compared with placebo in the heat (30°) in trained subjects [Stimulants are thought to potentially improve performance via the effects on neurotransmitter levels in the brain, predominantly dopamine and norepinephrine. Research into the effects of stimulants on performance has mainly focused on a few drug classes. Amphetamines such as amphetamine sulfate  showed psubjects . Levometsubjects .Ephedrine, pseudoephedrine and phenylpropanolamine have a similar mechanism of action to amphetamines. Two studies investigating the effects of ephedrine showed positive effects. One study found an effect on peak Wingate sprint power (+ 0.6%), but not on time to exhaustion , in untrFor cocaine, another well-known stimulant that is on the Prohibited List, there are no well-designed studies evaluating its effects on performance.Overall, studies on the effects of these stimulants show varying results, making it unclear whether they improve performance, as was concluded in a review published by Clarkson and Thompson in 1997 . In certThe narcotics category consists of strong analgesics, all belonging to the opioids class. Surprisingly, although not all opioids are currently banned , substances such as morphine and its analogs, and fentanyl and its derivatives, are. Although analgesic effects might enhance performance, common side effects of opioids, including nausea, sedation and respiratory depression, would equally argue against any beneficial effects. One study showed that an intrathecal injection of fentanyl did not impact average power output during a 5-km cycling time trial in trained cyclists ; howeverCannabinoids are known to affect perceptual function, and in a review of (non-sport) performance, Huestis  concludeVO2max and ventilator threshold of 5 days of dexamethasone in untrained subjects [Pmax of 4 weeks of budesonide treatment in trained subjects [Glucocorticoids act on metabolism and the immune system, and, through that mechanism, potentially affect performance. For this reason, systemic doses are prohibited in competition. A recent review showed there are varying results of glucocorticoid treatment in performance tests . The twosubjects , and on subjects . The effsubjects . In contsubjects . Sprint subjects . Howeversubjects , a findisubjects . Howeversubjects . An almosubjects . Althougsubjects . HoweverThis category covers substances prohibited in particular sports  and contains only the group of β-blockers. This group of substances inhibits β-adrenergic receptors, thereby reducing heart rate, anxiety and tremulousness, which could potentially enhance performance in sports where precision and accuracy are vital. A double-blind, randomized, placebo-controlled, crossover study has been conducted evaluating the effect of metoprolol on shooting performance in amateur marksmen . The stuVO2max and Pmax, but the available evidence shows no effect on actual endurance performance. What also becomes clear from this overview is that for most substance classes (11 of 23), there are no well-designed studies evaluating the effects on performance (in trained subjects), meaning there is an absence of level 1 evidence. Physicians involved in administering such substances in particular are performing practices similar to off-license prescribing, and prescribing without evidence is considered bad medical practice. In contrast, for another six substance classes, well-designed studies are available that show some evidence of absence of (relevant) effects on performance. Overall, this review therefore shows that for the majority of substance classes (17 of 23), there is no convincing evidence that they enhance the performance of athletes. Moreover, in regard to the other five classes that are prohibited in all sports that do have evidence-based effects, it is unproven whether such effects are relevant or useful in many types of sport (e.g. endurance sports) as they only improve specific performance tasks (mainly strength and sprint power). Although these aspects do play some role in many sport disciplines , it is not very clear whether these effects would also impact actual performance in those disciplines. In any case, there are no studies investigating this, as we have shown. These findings together seem discordant with the general perception that substances on the Prohibited List, by definition, improve performance (to a great extent). This is especially evident when it is considered that a total of only 266 subjects (from 11 studies) form the clinical pharmacological level 1 evidence base for performance enhancement, the main reason for anti-doping efforts. Although the WADA Code only requires evidence for the potential to enhance performance, and there are two other criteria that can be applied to make a substance prohibited, we conclude there is a lack of high-level evidence for improvement of actual performance based on this review. Undertaking more high-quality clinical research to provide the level 1 evidence base for the current Prohibited List could fill some of these gaps. Some of this research could be impossible due to practical or ethical objections, but the current level of randomized evidence is low and there appear to be many areas where such research is possible. Furthermore, if there is clear evidence that there are no performance-enhancing effects of a certain class, athletes should be informed of this. This could potentially lead to fewer athletes being tempted to use these substances. Finally, such steps would lead to a more transparent and high-level evidence-based fight against doping, and possibly reduce the efforts and resources needed to test for abuse.Of all 23 specific substance classes defined in the 2018 Prohibited List, only five classes show evidence of having the ability to enhance actual sports performance (see Table


ERAS® Society guidelines are holistic, multidisciplinary tools designed to improve outcomes after surgery. The enhanced recovery after surgery (ERAS) approach was initially developed for colorectal surgery and has been implemented successfully across a large number of settings, resulting in improved patient outcomes. As the ERAS approach is increasingly being adopted worldwide and new guidelines are being generated for new populations, there is a need to define an ERAS® Society guideline and the methodology that should be followed in its development.The ERAS® Society recommended approach for developing new guidelines is based on the creation of multidisciplinary guideline development groups responsible for defining topics, planning the literature search, and assessing the quality of the evidence.Clear definitions for the elements of an ERAS guideline involve multimodal and multidisciplinary approaches impacting on multiple patient outcomes. Recommended methodology for guideline development follows a rigorous approach with systematic identification and evaluation of evidence, and consensus‐based development of recommendations. Guidelines should then be evaluated and reviewed regularly to ensure that the best and most up‐to‐date evidence is used consistently to support surgical patients.There is a need for a standardized, evidence‐informed approach to both the development of new ERAS® Society guidelines, and the adaptation and revision of existing guidelines. This paper presents the ERAS® Society recommended approach for developing new guidelines. Elements that define an enhanced recovery after surgery (ERAS) guideline include a multimodal, multidisciplinary approach impacting on multiple patient outcomes. Recommended methodology for guideline development follows a rigorous approach with systematic identification and evaluation of evidence, and consensus‐based development of recommendations.Clear guidance for guideline development Enhanced Recovery After Surgery (ERAS®) Society consensus statements and guidelines are powerful tools that have been implemented across hospitals and healthcare systems worldwide to improve the quality of surgical care. The ERAS approach was initially conceived by a group of surgeons in northern Europe, based on the principle that actions undertaken to modulate postoperative stress can improve recovery after surgery. The ERAS approach has shown that early mobilization, early reintroduction of nutrition, and rapid discharge are feasible and beneficial for a large number of postoperative patientsThe first ERAS consensus paper was produced for colonic surgery in 2005The initial ERAS consensus paperThe strong principles underlying the ERAS concept, including its multidisciplinary and multimodal approach to perioperative care, and results of ERAS implementation may all have contributed to the success and wide adoption of the initial guideline in improving patient outcomes. Subsequent iterations and adaptations of the initial ERAS guideline have been produced using various methodologies, but have frequently used the initial consensus paperSince the initial introduction of ERAS nearly 20 years ago, publications on ERAS have increased, with more than 1300 published in the past 5 years. New guidelines are created with variable quality and rigour from a wide variety of authors.As the ERAS approach is being adopted increasingly worldwide and new guidelines are being generated for new populations, there has been, and remains, a need to define more precisely what constitutes an ERAS® Society guideline and what methodology should be followed for development. The ERAS® Society recognizes that there is a real need for a standardized, evidence‐informed approach to both the adaptation and the revision of existing guidelines, in addition to the development of new guidelines. ERAS guidelines, with their focus on broad multimodal care across a spectrum of time points and involving multiple specialties, require an ERAS‐specific approach.In December 2018, an ERAS® Society guideline steering group was formed to standardize and improve the guideline development process, in order to transform it into a regulatory framework that could be followed by all surgical specialties that plan to update or establish new ERAS® Society guidelines. This document outlines the ERAS® Society guideline for ERAS® Society guideline development. This includes the general principles of what differentiates an ERAS® Society guideline from other guidelines and the recommended methods for developing new ERAS® Society guidelines. This framework is presented as a detailed step‐by‐step manual, with comment and justification of the proposed development process.Table ERAS® Society guidelines can be differentiated from other clinical guidelines in a number of ways, including their holistic, multidisciplinary design, rigorous and broad literature review, and strong grounding in ERAS expertise Table .To ensure that future guidelines are up to date with a rapidly moving field of research, a clear time frame for completion should be established. The recommended time span used for the development of new guidelines should be as short as is feasible, with most guidelines completed within 10–12 months.When developing new guidelines, the ERAS® Society will work with lead authors to establish a guideline development group (GDG) that has international representation and will drive the process of guideline development. One or two lead authors will assume first and/or last author position on the published guideline and be responsible for ensuring the process is completed appropriately in a timely fashion. The number of members in an ERAS GDG should normally not exceed ten individuals in total, and should include two or three surgeons from the specialty under study and two anaesthetists and potentially other experts (see below). Input from patients and a diverse number of stakeholders is an important aspect of guideline development. Members of the GDG should be familiar with research methods and critical evaluation, whereas participation of other members can be included within the consensus generation process (see step 5) or in consultation for particular aspects of the guideline. At least one member of each GDG must have in‐depth knowledge of epidemiology and/or statistics to be able to judge the scientific quality of new and existing research adequately. All members of the GDG will be jointly responsible for the recommendations and co‐author the new guideline.GDG members and expert consultants should include individuals with specialist knowledge in their field who can evaluate available evidence and understand the aspects of implementation when considering the value of specific recommendations. Specialist knowledge may accrue from recognized qualifications, the study of evidenced research, or clinical experience.Patients and caregivers provide important perspectives on the appropriateness and acceptability of guidelines for patients and familiesExperts involved in guideline development could include: relevant healthcare specialists , representatives of professional bodies, including national organizations, researchers  and health economists, as well as patients.de novo. When appropriate, existing guidelines may provide a framework for the development of a new guideline. If this process is followed, each element adapted from the previous guideline must be assessed individually for appropriateness for the new patient population for which the guideline is being created. This assessment should take the form of a series of focused literature reviews, with assessment of the quality of the evidence and development of recommendations. Recommendations should be specific to the specialty with elements added, removed or edited based on the literature review. The GDG should have a process whereby new relevant elements can be identified for inclusion. This may take the form of a formal Delphi processERAS® Society guidelines may be adapted from existing guidelines or created de novo for unique populations. Methods to develop proposed guideline elements should take the form of a Delphi process, with development of a large number of proposed topics or elements that are voted on for inclusion by the GDG and informed by existing relevant guidelines. The process of focused literature reviews should be followed for each proposed recommendation.ERAS guidelines can be developed The first task for the GDG is to scope the guideline, and plan and carry out the relevant literature search. Relevant existing guidelines and priority areas should be identifiedTable If the GDG does not have expertise in systematic reviews or knowledge syntheses, this should be done in consultation with a research librarian or other knowledge synthesis expert. The PICO  frameworkUnlike a systematic review, screening titles, abstracts and full texts for the multiple searches can be performed by a single individual, but the final body of literature should be reviewed by the GDG to ensure that relevant papers familiar to the other group members are captured. The screening process for each question and each search should be recorded within a PRISMAAll studies captured in the literature search should undergo a standardized process of evaluation, regardless of study design. Each single intervention in the ERAS protocol should be scrutinized, and quality assessed according to the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approachper se15. Therefore, the quality of evidence provided by the studies included in the final analysis should be classified as high, moderate, low and very low by assessing the following aspects: importance of outcomes, risk of bias, heterogeneity, indirectness, imprecision and publication bias .The Delphi process is one recommended method to achieve expert consensusFig. When all of the above steps have been completed Fig. , the finFig. The format of the guideline should adhere to the guidelines of the scientific journal, but should include a clear description of the methods outlined above, including the identification of proposed elements, the literature search, the rating of quality, the generation of final recommendations, and the rating of the strength of those recommendations. Results should include an ERAS® Society diagram Fig. 18 contaiEvery 2–3 years, the lead author or designated alternative will present a formal report on the guideline to the ERAS® Society, with a brief re‐evaluation of the literature. If there is substantial new information, a guideline update will be performed. This review should be a facilitated process, generally lasting no more than 3–4 months. Relevant recommendations should have an updated search performed to include papers published since the last guideline. Inclusion of the recommendation, its wording, the quality of evidence and strength of recommendations should be considered in light of new evidence. The GDG should also consider whether new recommendations should be added to the guideline. As with creation of a new guideline, the revised guideline should be reviewed by external reviewers identified by the ERAS® Society. In the case of a simple revision without substantial changes (no changes to the recommendations themselves), a single reviewer would suffice. In the case of revised recommendations, or if recommendations are eliminated or added, two reviewers and potentially the Delphi process would be advised.Table Since 2005, 23 ERAS® Society publications of 20 guidelines have been published for surgery and perioperative care in various specialties and subspecialtiesAll authors are members of the ERAS® Society.Appendix S1. Supporting informationClick here for additional data file.


MMWR if approved by the director. This report briefly summarizes an update to the ACIP process for developing evidence-based recommendations that ACIP adopted at its February 2018 meeting.The Advisory Committee on Immunization Practices (ACIP)https://www.cdc.gov/vaccines/acip/recs/grade/table-refs.html). However, during the process of recommendation formulation, in addition to the certainty in the evidence as presented in such evidence profiles, panels consider myriad factors such as the values held by the target population regarding the outcomes, health economic data, and implementation issues. Significant additions in this area have been incorporated into GRADE methodology in the past 8 years, particularly the use of Evidence to Decision or Evidence to Recommendation (EtR) frameworks to support the process of moving from evidence to decision and to provide transparency regarding the impact of additional factors on deliberations when considering a recommendation  approach for developing evidence-based recommendations (https://www.cdc.gov/vaccines/acip/recs/grade/downloads/ACIP-evidence-rec-frame-508.pdf), which was formally adopted by a unanimous vote at the February 2018 ACIP meeting. Other guideline development panels, including National Immunization Technical Advisory Groups and the World Health Organization’s Strategic Advisory Group of Experts on Immunization, have adopted the new GRADE approach and developed EtR frameworks for use in formulating recommendations .New or substantially revised ACIP recommendations for vaccination will use the EtR framework to communicate the deliberations and judgments made by ACIP during formulation of its recommendations. Recommendations will be communicated in the framework in one of three categories: 1) ACIP recommends vaccination for all persons in an age group or a group at increased risk for vaccine-preventable disease; 2) ACIP does not recommend the use of a vaccine; or 3) the ACIP recommendation relies upon guidance of the clinician in the context of individual clinician-patient interactions to determine whether or not vaccination is appropriate for a specific patient. In some instances , ACIP might not make a recommendation, and this option is also reflected in the EtR framework separately  implemented the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach for developing evidence-based recommendations.Since the original adoption of the GRADE evidence-based recommendation process by ACIP, the use of Evidence to Decision or Evidence to Recommendation (EtR) frameworks have been incorporated into GRADE methodology. ACIP adopted the use of an EtR framework at its February 2018 meeting.The EtR framework elucidates the additional factors considered in vaccine recommendation deliberations and facilitates transparency, consistency, and communication of recommendations to health care providers, partner organizations, and the public.