When evidence‐based medicine (EBM) became established, its dominant rhetoric was empiricist, in spite of rationalist elements in its practice. Exploring some of the key statements about EBM down the years, the paper examines the tensions between empiricism and rationalism and argues for a rationalist turn in EBM to help to develop the next generation of scholarship in the field. The paper will explore the tension between that rhetoric and the rationalist elements of its practice. The history and the interaction between empiricist claims and underlying rationalist assumptions will be explored by examining the contributions of writers like Cochrane and Sackett and institutions like the Cochrane Collaboration and the National Institute of Clinical Excellence (NICE). The terms rationalism and empiricism are used here in Hume's sense of rationalism being about the relationship between ideas and empiricism being concerned with matters of fact.There are a number of landmarks in the development of EBM. The ones chosen to illustrate the argument here are Cochrane's critique of the medical profession, the appearance of the McMaster school and the influential writings of Sackett and his colleagues, the founding of the Cochrane Collaboration, and then the establishment of NICE. Archie Cochrane focussed on ways to determine effectiveness and cost‐effectiveness; the McMaster group were intent on applying the lessons of public health epidemiology to clinical medicine; the main thrust of the Cochrane Collaboration was the improvement in methods of review and scientific reporting, and NICE was about putting EBM into practice. All had an explicit focus on empiricist methods and a very strong rhetoric about objective science and matters of fact. Yet in the work of the McMaster group, the Cochrane Collaboration, and NICE, there was always a tension. Some problems simply could not be solved by empiricist solutions, and rationalism never really went away.2Effectiveness and Efficiency: Random Reflections on Health ServicesThe contribution of Cochrane's book Cochrane concluded that the optimal way to determine whether a drug worked and whether it was better or worse than usual treatment or some other drug was the evidence from randomized controlled trials—now of course the standard method, but not then. He also called for a system to monitor and systematically record trial results. Further, although he did not describe it as such, Cochrane provided a sociological analysis of the way the medical profession operated. Powerful medical schools and senior professors in them—in England in the highly prestigious London Teaching Hospitals for example—dominated pedagogy, curricula, teaching, and paradigmatic understandings of disease aetiology, diagnostics, and therapeutics. The paradigms inevitably lagged some way behind leading research discoveries and innovations. They still do.So in the questions that Cochrane asked, we see a strong emphasis on evidence of observable phenomena based on scientific rigour and method. Evidence should drive practice, not opinions and social networks. His ideas in other words were strongly empiricist (in Hume's sense). Of course, Cochrane was a very sophisticated thinker on a range of medical and scientific matters,3“Evidence‐based medicine de‐emphasizes intuition, unsystematic clinical experience, and pathophysiologic rationale as sufficient grounds for clinical decision making and stresses the examination of evidence from clinical research”.The story shifts to North America and to a group of doctors and scholars linked to David Sackett at McMaster University in Hamilton, Ontario. They coined the term EBM. Sackett's original idea was to apply the principles of public health epidemiology to clinical medicine.“Understanding certain rules of evidence is necessary to correctly interpret literature on causation, prognosis, diagnostic tests, and treatment strategy. It follows that clinicians should regularly consult the original literature  in solving clinical problems and providing optimal patient care. It also follows that clinicians must be ready to accept and live with uncertainty and to acknowledge that management decisions are often made in the face of relative ignorance of their true impact.” Note that from the outset, they acknowledged the role of clinical experience and judgement, but argued that on their own, these were not sufficient for effective medical practice and were a source of potential bias. They emphasized instead the importance of systematic, reproducible unbiased observations. The group argued that the understanding of basic mechanisms of disease is a necessary but not sufficient guide for clinical practice. Their contention was that:rules of evidence. By this was meant procedures and protocols to make sense of evidence. Although the group acknowledged clinical experience and judgement, they did not seek to apply the idea of rules to these judgements and neither did they acknowledge that the application of rules is an interpretive process, itself requiring judgement.They set out to put this into practice in their teaching at McMaster.In 1995, Sackett and Rosenberg outlined 5 principles of EBM. These were that clinical decisions should be based on the best patient, population, and laboratory evidence; that evidence rather than habits, protocols, or traditions should drive practice; that the best evidence integrated epidemiology, biostatistics, and pathophysiology, and techniques like meta‐analyses of randomized trials and economic analyses  should be used to appraise that evidence; that evidence must inform practice; and that practice should be continuously evaluated.“the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research”.In 1996 in a landmark and much quoted editorial, Sackett and colleagues provided what has become the standard definition of EBM as“clinically relevant research, often from the basic sciences of medicine, but especially from patient centred clinical research into the accuracy and precision of diagnostic tests , the power of prognostic markers, and the efficacy and safety of therapeutic, rehabilitative, and preventive regimens … [EBM] involves tracking down the best external evidence with which to answer our clinical questions … Because the randomised trial, and especially the systematic review of several randomised trials, is so much more likely to inform us and so much less likely to mislead us, it has become the ‘gold standard’ for judging whether a treatment does more good than harm.”Sackett et al defined external evidence as:Again, it is important to reflect on the words “best” and “gold standard”. What is best and a gold standard are judgements, albeit judgements that can be assisted by certain rules and protocols, but they are still judgements.sine qua non of EBM. The tone is conciliatory up to a point, acknowledging the role of experience and judgement, but the question of how experience and judgement integrate with the rules remains oblique.This editorial was published against a background of growing opposition to EBM and not a little controversy. One objection was that EBM was nothing new and that doctors were already practicing EBM. Sackett and colleagues' riposte was that in medical practice, things had not changed since the publication of Cochrane's book 22 years before. The authors were keen to emphasize that best evidence did not replace the diagnostic or therapeutic skills of the physician. They were at pains to point out that the best evidence was a platform for better clinical judgement and that it would not in any sense replace it. They also noted that the rate of publication of new papers made it simply impossible for even the most conscientious physician to keep up with the literature, so to consult reviews of accumulated evidence was a So, the ideas from Sackett and colleagues did articulate a view in which observable and replicable evidence, properly reported and appraised, should be the cornerstone of practice. The methods that they advocated like the randomized controlled trial (RCT) and the focus on the elimination of bias in the relationship between independent and dependent variables were inherently reductionist. The notions of causation and association embedded in trial design are linear. The emphasis on the importance of evidence over opinion elevates science above other kinds of values in the same way that Cochrane had argued in his original book. At the same time, the thorny problem of clinical judgement and expertise and experience hovers round the margins of the discussion and never really goes away.4In 1993, the Cochrane Collaboration was established, taking its name from Archie Cochrane and his suggestion that there should be a system for making available systematic reviews of all relevant RCTs of health care. It was led initially by Iain Chalmers and took on the task, through a network of volunteers, of cataloguing and reviewing the extant trials and establishing them in an accessible data base.The existence of the Cochrane Collaboration and all the work it has done has been a major pillar in the development of EBM. It is a touchstone of content, techniques, and method, and it has helped to build the reputation of EBM. It has become a resource much used by researchers, guideline developers, and clinicians worldwide. It has generated an industry of methodologists exploring many of the finer details of process and procedure. The original idea that there should be a repository of trials and that through the process of accumulation, synthesis, and appraisal that repository can provide better answers to questions of efficacy and effectiveness remains at the heart of what those associated with Cochrane do.5In 1999 in England, NICE was established to conduct appraisals of new drugs to determine their value for money for the NHS. It was an attempt to end the so‐called postcode lottery of health care in England and Wales, where treatments that were available depended upon the NHS Health Authority area in which the patient happened to live. Soon afterwards, NICE began developing clinical guidelines. In April 2005, NICE amalgamated with the Health Development Agency to become the new National Institute for Health and Clinical Excellence (still abbreviated as NICE) now with responsibility for developing public health guidelines. Following the Health and Social Care Act 2012, NICE was renamed the National Institute for Health and Care Excellence on April 1, 2013 (still with the acronym NICE) reflecting its then latest responsibilities for publishing social care guidelines.The National Institute of Clinical Excellence is important in the story of EBM because more than any other public body in the UK, and a public body that found itself very much in the in the public eye, it executed and practiced the principles of EBM. It drew heavily on the thinking that had followed in the wake of the 1990s NHS R&D strategy.However, it was in the execution of its tasks that it became clear to a number of the leading figures at NICE, especially Sir Michael Rawlins the Chair, that there were underlying assumptions, which presented NICE's appraisal and guideline development committees with some very knotty problems that were not amenable to solutions derived from the empiricist precepts of EBM. The first efforts NICE made to deal with these was with respect to values. The National Institute of Clinical Excellence established a Citizens' Council to deliberate on value issues and published its own social value judgement paper (which went through a couple of editions) to try and sort the issue out. In turn, it was hoped that this would help its appraisal and guideline committees as they went about their work.The limitations and difficulties of the processes used by NICE at the time were explored by a number of members of the senior team at NICE as they wrestled with the difficulties involved.The knowledge and evidence to be found in the social sciences and humanities are particularly germane to this problem. Psychology, sociology, anthropology, political science, history, and philosophy all in various ways could make significant contributions. These disciplines describe and investigate human behaviour, individually, in groups and communities, in organizations, and in particular times and places. They constitute discrete evidence bases, which for the most part are never pressed into service by EBM. Psychology and Economics have made the greatest inroads but both within relatively narrow confines of health‐related behaviour change (psychology) and cost effectiveness and the application of cost utility theory (economics). Important as these contributions are, there is a very much wider knowledge base that could be used. Presently if this material appears at all, it is as a basis for describing context and social background, not as evidence that might be used to address questions of effectiveness or how to bring about change in complex systems. Taken as a whole, the world of EBM excludes much that, as is argued below, if a rationalist turn was taken, could help.6An Enquiry Concerning Human Understanding, Hume demarcated 2 types of knowledge, which he called rationalism and empiricism.Empiricism is used here following Hume. In Factual reasoning on the other hand for Hume is inductive and proceeds from the particular to the general, from observation and experience to more complex ideas. It involves drawing reasonable but not logically certain conclusions from empirical evidence, experience, or testimony. Factual reasoning is the basis of empiricism and is a posteriori, ie, after the fact of experience or observation. To learn from what we have observed, we must extrapolate beyond experience. To do this, we must draw out factual or inductive inferences from what we have observed to that, which we have not. Such inferences are contingent. This is because the future may not resemble the past and what we observe now or in the future may not be the same as we have observed in the past. For Hume, the relations between things, and especially causal relations, can only be known on the basis of experience. Factual reasoning, concerning the relation between things, is based on an assumption, used by all of us all as we go about our business in the world, which is the things we have observed in the past are a reasonable, but not infallible, guide to what we will observe in the future.The Enquiry ends with his famous words.“If we take in our hand any volume of divinity or school of metaphysics, for instance; let us ask, Does it contain any abstract reasoning concerning quantity or number? No. Does it contain any experimental reasoning concerning matters of fact and existence? No. Cast it then into flames. For it can be nothing but sophistry and illusion.”Matters of fact are not just about ideas and how they relate to each other but they are also about how things work in the real world. Hume concluded that rationalism and empiricism are fundamentally irreconcilable, and he opted for empiricism. The empiricism in Hume's sense is clear in the writings of Cochrane and Sackett, in the precepts of the Cochrane Collaboration and indeed in NICE's manuals. The empiricist position was clearly stated by Sackett and Rosenberg for example who argued that doctors should always seek to base their decisions on the best available evidence.The rejection of rationalist thinking (in Hume's sense) exhibited by Sackett is more than a preference for particular methods; it is an epistemological position.Further, not only are patients biologically heterogeneous they are also heterogeneous socially and psychologically. There are very important biological and social differences between individual patients that are concealed by the epistemological precepts of the RCT.“do not include many types of treatments or patients seen in clinical practice; and the results show comparative efficacy of treatment for an ‘average’ randomized patient, not for pertinent subgroups formed by such cogent clinical features as severity of symptoms, illness, comorbidity, and other clinical nuances. The intention‐to‐treat analyses do not reflect important post‐randomization events leading to altered treatment; and the results seldom provide suitable background data when therapy is given prophylactically rather than remedially, or when therapeutic advantages are equivocal. Randomized trial information is also seldom available for issues in aetiology, diagnosis, and prognosis, and for clinical decisions that depend on pathophysiologic changes, psychosocial factors and support, personal preferences of patients, and strategies for giving comfort and reassurance.”Feinstein and Horwitz put it succinctly, arguing that EBM dataIn other words, there is knowledge both from other empirical approaches as well as rationalist ways of thinking that are quintessential to the clinical task, which in a narrow interpretation of EBM will be side‐lined.Tonelli's criticisms of EBM echo this. He has argued that there are 5 types of consideration in making decisions about individual patients. These are as follows. Empirical evidence derived from clinical research. Experiential evidence derived from personal clinical experience or the clinical experience of others . The pathophysiologic rationale based on underlying theories of physiology, disease, and healing. Patient values and preferences derived from personal interaction with individual patients. And system features including resource availability, societal and professional values, and legal and cultural concerns. Casuistic reasoning, he suggests, allows the physician to weight these different elements.Evidence‐based medicine has a tendency towards reductionism that derives from the centrality of the RCT. This strategy has important benefits. Biomedical reductionism has been very successful in helping to identify causal factors and in finding curative or palliative drugs and other interventions for many diseases. But because of biological variation between individuals, neither potentially causal factors nor pharmaceutical agents are universal in their causes and effects. Heterogeneity is endemic to medicine. Epidemiological studies and RCTs describe average effects in populations and subpopulations, and interpretation always has to be mindful of outliers consequent on heterogeneity. What the RCT does is reduce phenomena down to a unit of analysis in such a way that there can be clarity in the relationship (which is often an association rather than a mechanism)However, the analytic strategy is an abstraction from the real world, not the real world itself. The moment that the fact that it is an analytic strategy is lost sight of, problems will ensue. Much of the methodological literature about RCTs appears to reify the abstraction as an objective account of reality itself. In complex biosystems, in complex social systems, and in the complex relations between them, the notion that two variables act one on the other and on nothing else is clearly wrongheaded. Systems approaches draw attention to multiplex phenomenon with multiple interacting parts, which are in continuous and iterative and recursive relations. The models constructed of these are abstractions, not reality itself. So, the linearity implied in trials as well as their reductionism is not wrong in itself—understanding is advanced by using such strategies—but to shift the reduced linearity to the status of the real world is either deliberate or very naïve scientific legerdemain producing a tendency towards prescriptiveness.evidence is a misnomer because it is a hierarchy of methods with the methods that are best at reducing bias between dependent and independent variables at the top.The hierarchy of evidence places meta‐analyses of well‐conducted RCTs at the top of the hierarchy and expert opinion, consensus, and case reports at the bottom. The hierarchy of Evidence‐based medicine has a fundamental problem in dealing with the absence of evidence in two senses. First, when evidence exists, it is excluded on epistemic grounds—ie, it is excluded because it is not derived from RCTs or meta‐analyses of RCTs by invoking the hierarchy of evidence. What is happening here is that EBM is using judgements  either explicitly or implicitly from the very outset to determine what constitutes evidence in the first place. So contra the rhetoric, which makes claims about the importance of judgements in individual patient treatment decisions, judgements come into the process long before the patient encounter. Second, judgements come into play when the evidence does not exist. Evidence‐based medicine has a strong rhetoric about being evidence based, but often is not in its own terms when the evidence is not there. It faces the problem of what to do then. Sometimes, the argument is made that this is not actually a problem so long as the evidence has been searched for properly so what is unknown is really unknown“To obtain ‘trustworthy’ information, randomized trials have concentrated on getting ‘hard’ data about death, disease, and demography. The patient's baseline condition, before therapy, is regularly characterized with the ‘reliable’ information of age, gender, race, imaging, endoscopy, biopsy, cytology, and laboratory tests. The therapeutic outcome is cited, whenever pertinent, as death, or in global ratings for certain symptoms  that need not be specified more precisely because the ‘double‐blind’ observations will presumably avoid bias”.Feinstein and Horwitz note thatTherefore, the belief that randomization solves problems of heterogeneity is misplaced.There is one further important point to note about the idea of chance and statistical significance in assessing trial results. This is also a theoretical principle. Specifically, the idea is about reducing the likelihood that what has been observed is the consequence of chance rather than the hypothesized relationship which is being investigated. However, actually nothing happens by chance; all things have preceding causes. When we say it happened by chance, it really means we did not know in advance that there was some alternative cause for what had happened to that which we are investigating or have hypothesized. To argue that this helps to overcome the fact that we do not know in advance all possible reasons for things happening is a logical principle. It is not an empirical one. It is a helpful principle when it comes to interpreting statistical data, but a rationalist principle it remains. To say something may have happened by chance is a rationalist theoretical construct not an empirical one grounded in evidence.And the final irony is this. The orthodoxy of EBM belongs firmly in the rationalist camp. It is an overarching set of theoretical ideas and logical principles about methods. The structure of the design of the RCT and the hierarchy of evidence are prime examples as are certain epidemiological and statistical techniques; the hierarchy of evidence and the associated idea of internal validity are rationalist constructs not empirical ones. The principles of trial design and internal validity upon which it is based are also rationalist ideas. This means that they are derived from a priori theoretical and logical principles rather than a posteriori observations. These things may give the impression of being about hard empirical science, when in fact they are rationalist a priori constructs. A strict application of the principles of EBM will drive out rationalist knowledge, an anxiety echoed early on by Tonelli.7However, we must be careful not to throw the baby out with the bathwater. The fact is that despite the overarching tendency to emphasize a particular version of empiricism, EBM and its aspirations are worthy and worthwhile, and the move to placing evidence at the heart of decision making is certainly one that deserves support. The difficulty arises with the knotty problem of judgement and other forms of rationalist knowledge, which while they have been acknowledged from the very beginning of EBM still appear to sit uncomfortably with the empiricist orthodoxy. Others have considered this problem. Jenicek argued for a new type of science to help to find a way through the difficulties.The solution lies in rectifying the enormous imbalance between the amount of effort that has gone into refining certain methods like the trial, and the associated statistics and epidemiology on the one hand and the forms of reasoning so often consigned to the bottom of the evidence hierarchy on the other. There are well‐defined scientific protocols for methods and interpretation of results. The methods for understanding processes of inference and judgement beyond the method protocols are less well understood or articulated, and they should be developed.As a result, enormous effort has also gone into refining certain but not all aspects of the competencies involved, and a huge amount of development in the infrastructure to support evidence cumulation and synthesis. Judgement and interpretation as part of the practice have been set to one side in favour of a highly empiricist account of the activity. Kathryn Montgomery has plausibly argued that to understand medicine, we must get behind its rhetoric to the actual practice of medicine.In the same way that several generations ago Archie Cochrane and David Sackett mapped the problems and set in motion an agenda that has served us well, we must now try to do the same for the bits that have not been discussed anywhere nearly so precisely. We also need to make plain that this will not in some sense diminish EBM—it is not a retreat from science back to anecdote. Rather, if EBM is to progress, it will have to get to grips with a set of problems, which if forever consigned to the status of non or soft knowledge will seriously hinder further scientific progress.


Little work investigates the effect of behavioral health system efforts to increase use of evidence-based practices or how organizational characteristics moderate the effect of these efforts. The objective of this study was to investigate clinician practice change in a system encouraging implementation of evidence-based practices over 5 years and how organizational characteristics moderate this effect. We hypothesized that evidence-based techniques would increase over time, whereas use of non-evidence-based techniques would remain static.n = 340; overall response rate = 60%). All organizations and clinicians were exposed to system-level support provided by the Evidence-based Practice Innovation Center from 2013 to 2017. Additionally, approximately half of the clinicians participated in city-funded evidence-based practice training initiatives. The main outcome included clinician self-reported use of cognitive-behavioral and psychodynamic techniques measured by the Therapy Procedures Checklist-Family Revised.Using a repeated cross-sectional design, data were collected three times from 2013 to 2017 in Philadelphia’s public behavioral health system. Clinicians from 20 behavioral health outpatient clinics serving youth were surveyed three times over 5 years (SD = 11.40); there were no significant differences in clinician characteristics across waves . Controlling for organizational and clinician covariates, average use of CBT techniques increased by 6% from wave 1 (M = 3.18) to wave 3 , compared to no change in psychodynamic techniques (p = .570). Each evidence-based practice training initiative in which clinicians participated predicted a 3% increase in CBT use (p = .019) but no change in psychodynamic technique use (p = .709). In organizations with more proficient cultures at baseline, clinicians exhibited greater increases in CBT use compared to organizations with less proficient cultures .Clinicians were 80% female and averaged 37.52 years of age (System implementation of evidence-based practices is associated with modest changes in clinician practice; these effects are moderated by organizational characteristics. Findings identify preliminary targets to improve implementation. The last two decades have shown increasing emphasis on the implementation of evidence-based practices (EBPs) in publicly funded behavioral health systems nationally , 2. PoliThe majority of what is known about system-wide EBP implementation is largely descriptive in nature . One set of studies takes a broad perspective and reports on national trends across system EBP implementations. For example, a study reporting on a set of national surveys conducted with state mental health directors found increases in states offering EBPs for youth from 2001 to 2012 . Anothern = 255,968) received at least one session of EBP for PTSD. Another study using administrative claims data found that there was an increased rate of EBP claims over time within the context of a fiscally mandated implementation effort in Los Angeles County 84] ratheDespite substantial efforts to implement EBPs over the past 20 years, downstream effects have rarely been systematically measured . This st


To explore the extent to which doctors and dentists in training within the UK and Republic of Ireland (RoI) engage in and with evidence-based practice (EBP), and to identify motivators and barriers to them doing so.An observational, prepiloted web-based survey developed by a trainee-led focus group.The survey instrument was disseminated to doctors and dentists in training within the UK and RoI during June 2017 via social media and through deaneries, Royal Colleges and specialty-specific mailing lists.Data from 243 trainees were analysed; 188 doctors from 31 specialties and 55 dentists from 9 specialties. Responses were received from trainees at all stages of postgraduate training though the overall response rate was low.The motivators and barriers to, and the extent of, trainee engagement with EBP.Cronbach’s α was 0.83. Most trainees (87.6% (n=148) of doctors and 75.1% (n=39) of dentists) consulted the evidence base at least monthly, while 23.1%  of both specialties did so daily. The two most commonly cited barriers to engagement with EBP for both doctors and dentists, respectively, were insufficient time (57.6% (n=95) and 45.1% (n=23)) and a tendency to follow departmental practice (40.6% (n=67) and 45.1% (n=23)). Key motivators for EBP included curiosity, following the example set by senior colleagues and a desire to avoid harm. Most trainees reported high levels of confidence interpreting evidence yet for 26.8% (n=45) of doctors and 36.5% (n=19) of dentists, medical hierarchy would impede them querying a colleague’s management plan based on their own reading of the evidence.Time, accepted departmental practice and the behaviour of senior clinicians all highly impact on trainee engagement with EBP. Given the low response rate, the extent to which these data represent the overall population is unclear. National survey detailing medical and dental trainees’ level of engagement with evidence-based practice.Provides policymakers, trainers and healthcare organisations with evidence for areas in which trainees’ engagement with evidence-based practice can be encouraged and facilitated.Reach extends across the UK and Republic of Ireland but results may not represent trainees’ experiences in other countries.Both dentists and doctors in training sampled but modest sample size may limit generalisability.Evidence-based practice (EBP) describes the integration of a patient’s values and a clinician’s expertise with the contemporary scientific evidence base.Within the UK and the Republic of Ireland (RoI), EBP is firmly embedded within both medical and dental curricula. Reflecting this, the General Medical Council recommends that new doctors must be able to ‘access and analyse reliable sources of current clinical evidence and guidance and have established methods for making sure their practice is consistent with these’.Despite these recommendations and the known tangible benefits of EBP to patient outcomes and care, it is recognised that clinical practice is not universally ‘evidence based’.The extent to which these barriers specifically apply to postgraduate medical and dental trainees, and the degree to which they therefore engage with EBP, is for the most part unknown. Much of the evidence cited to date is instead drawn from grouped populations of undergraduate and postgraduate trainees, or from cohorts consisting of both trainees and more senior clinicians who are beyond completion of training.The Cochrane UK & Ireland Trainees Advisory Group was established as the trainee-led arm of Cochrane UK & Ireland in 2016. It is formed of 15 medical and dental trainees from across the UK and RoI who represent a diverse range of specialties and who are together tasked with enhancing trainee engagement with EBP. Given the lack of evidence concerning trainees’ engagement with EBP in the contemporary UK and RoI healthcare systems, the Trainees Advisory Group undertook a national survey focused on trainees’ use of the evidence base in routine clinical practice and the wider motivators for and barriers to use of the evidence base among these doctors and dentists in training.We conducted a cross-sectional study targeting doctors and dentists in postgraduate training from across the UK and RoI using an anonymous electronic survey between June and October 2017.Members of the Cochrane UK & Ireland Trainees Advisory Group formed a focus group in order to develop an online survey specifically for this study. The rationale for designing a bespoke instrument for this survey was based on our literature review which revealed no validated instrument to cover all areas of focus for our survey. Drawing on their own experiences, the Trainees Advisory Group defined the following as areas of focus for the survey: (A) trainee engagement with EBP, (B) motivators and barriers to engagement with EBP; and (C) trainees’ preferences towards approaches to enhance engagement in EBP. Engagement in this context was defined as maintaining sufficient knowledge of the contemporary evidence base relevant to one’s clinical role to influence views of or specific practice within that role.The survey instrument was devised and refined through an iterative process that included two pilot surveys with 19 postgraduate trainees who represented 13 specialties in the fields of medicine and dentistry, and seven geographical locations within the UK and RoI. In its final form , we disseminated the survey via email to relevant professional organisations with oversight of doctors and dentists in training across the UK and RoI. We also used the email marketing service MailChimp to disseminate the survey to medical deaneries across the UK and to 49 trainees subscribed to the Cochrane UK & Ireland Trainees Advisory Group newsletter see . MailChi10.1136/bmjopen-2019-031809.supp1Supplementary dataDuring this same week, a link to the survey was posted online using the Cochrane UK Twitter account (@CochraneUK), which at the time had in excess of 40 000 followers. In line with an a priori planned social media strategy, this link was retweeted nine times by this same account and by members of the Cochrane UK & Ireland Trainees Advisory Group within the same week. Tweets were flagged for the attention of the Twitter accounts of relevant Royal Colleges and other bodies overseeing trainees.In week 2, we once again contacted those professional bodies who had been contacted during week 1 but who had not responded to the first survey approach. Contemporaneously, further tweets outlining the survey were distributed by the Cochrane UK & Ireland Twitter account. These continued into the third week. The survey subsequently remained open until October 2017 in order to permit further responses to be collected.The unit of analysis of this study is the individual respondent (trainee). Data are provided for doctors and dentists separately where possible. There were insufficient participant numbers to analyse by specialty. Data are shown for separate stages of training where numbers allow. Descriptive analyses were undertaken using a statistical software R . We computed the frequency of answers and where relevant expressed this as a percentage of total respondents. For Likert-type scaled response answers, we assumed answers to be an interval-level measurement to allow visualisation using bar plots.et al.We conducted a thematic analysis of the free-text responses using an inductive approach, following the phases of thematic analysis outlined by Clarke There was no direct patient involvement in the design or conduct of this study.The survey had high internal consistency with a Cronbach’s α of 0.83.A total of 243 responses were received; 188 (83.3%) from doctors in 31 specialties, and 55 (16.7%) from dentists in 9 specialties see . Most ) were supportive of its use. Popular forms of engagement were via email updates (53.7% (n=117)), an online journal club (47.3% (n=101)) and through a podcast (40.4% (n=88)).There was some variation between doctors and dentists in the factor respondents highlighted as their motivation for consulting the evidence base . For denThese findings were supported by thematic analysis of free-text comments, which revealed that motivators for EBP included trainees’ desire to provide a good standard of care at either the individual patient level, or for public health trainees, at the population level. The responses captured the inspiration to do so, which included examples of harm caused to patients by non-EBP, involvement in research projects and following the example set by senior colleagues:I think it is important to review literature as often ingrained practice has no real proof of benefit. I also think that by understanding the evidence you learn better and you understand its applicability and are better able to decide if it applicable for your patient cohort.The ethical implications of continuing to practice in an evidence-light environment following the withdrawal of the Liverpool Care Pathway…My consultants are also very much into [EBP] & quote papers all the time. I feel that I’m expected to know the evidence behind the treatments I offer patients & I think this is a good thing as it makes me much more confident at recommending a particular treatment to a patient.Barriers to EBP reported by trainees are summarised in Thematic analysis of free-text responses supported these data, and highlighted that trainees considered time and access as important barriers to EBP:By far the greatest barrier to taking an evidence-based approach to my work is the lack of access to the evidence. I am a public health registrar with a passion for the wider determinants of health. At the moment I have very limited Athens access through the training scheme, but following this I'm not sure if I will even have that…I am limited by the insufficient electronic journal access provided as part of NHS Athens, which is inferior to the access provided by a good university account. I do manage to work around this, but it takes extra time and limits the sources I can use.Some trainees felt that the access issue was twofold: electronic access to articles being the first hurdle, and having the information technology facility within the NHS premises to allow access to the information being the next one:…the evidence base is spread across multiple resources, some of which are, with the best will in the world difficult to use in day to day practice, from crappy, locked down NHS computers. If they are free to use at all.Poor wifi signal often frustrates me when I want to look at a resource such as NICE or UptoDate on the wards.Some trainees felt that the use of evidence was not prioritised in postgraduate training despite its importance, and that those who require further training on this may struggle to find the time due to other priorities:We are not taught how to use the evidence base. Yeah sure, the local library might offer some introductory courses  but it is unacceptable that ‘moving & handling’ is a mandatory course but searching and using the evidence base is treated as optional.I was lucky enough to learn about evidence-based research at medical school and taught how to conduct literature reviews but it is something I just don’t have the time for now.Dentists in training more commonly cited a lack of relevant or high-quality evidence as a barrier than doctors (37.3% (n=19) vs 12.7% (n=21)). Thematic analysis also revealed the challenges trainees face in specialties in which evidence is scarce and where studies have unclearly reported methodology or questionable generalisability to their patients:…there is a really lack of randomised controlled trials in my specialty and rejection of them by some of the large research groups. The question of how much evidence is enough evidence in palliative medicine remains to be answered.Reliance on secondary evidence such as local, specialty-specific or national guidelines was evident in the free-text responses. Trainees perceived secondary evidence to be of high quality and therefore their default first choice of avenue for evidence:I use individual papers very rarely. Think Royal College of Obstetrics and Gynaecology does a good job of synthesising current information and producing guidance on common topics.Commonly used guidelines have been memorised—for example, asthma, transient loss of consciousness.When trainees were asked if they would feel comfortable querying a colleague’s management plan based on the trainee’s reading of the evidence, 57.7% (n=97) of doctors and 51.9% (n=27) of dentists were unconditionally so. Other trainees reported that they would feel confident only if the colleague was more junior or less experienced than them (19.6% (n=33) and 30.8% (n=16)), or that this depended on the situation (15.5% (n=26) and 11.5% (n=6)). A smaller proportion of respondents stated that they would feel uncomfortable about querying a colleague’s management in any case (7.1% (n=12) and 5.8% (n=3)).On analysing the free-text responses, a passive attitude of following generally accepted departmental practice was a frequent feature. It was evident that the working relationship with, or the personality of a senior colleague was more important to trainees than the seniority itself when considering whether to question a senior colleague’s decision. Embedded within the responses was a situational analysis process that trainees go through in their mind before deciding to approach their senior colleague. Some trainees have developed a strategy where their decision to question a senior colleague would be conditional—for example, only if the impact to the patient is large, only if they predict a positive response from the colleague and only if they have direct responsibility for care :Some colleague’s personalities may colour my comfort level to query/discuss management!Consultants who are confident and progressive would be willing to discuss. Some seniors would find this threatening as they are insecure in their own practice.…you have to read the situation…It really depends on so many factors  In general, I would only do it if I feel that there will be a positive response. If I feel that it is only going to generate conflict, then, what's the point?… I pick my fights as such and only raise it if I think it is important/going to make a difference etc.There was also a response which pointed out that a clinician’s unfounded preconceived notions may act as a barrier to a scientific enquiry:The problem is that the concept of ‘evidence’ has lost its meaning. People use for all sorts of things…I mean that people, when faced by a public health issue, they don't ask themselves ‘What does the evidence say about how to resolve this issue?’ What usually happens is that they have a pre-defined position and they ask themselves ‘What is the evidence that supports my position?’ That influences how searches are done, how bias is assessed and so on.The extent to which doctors and dentists in contemporary training schemes engage with the evidence base, and the barriers and motivators to them doing so, is underexplored. We outline here a nationwide survey of UK and RoI doctors and dentists from a large number of specialties and at all stages of training. Most actively engage with literature at least monthly, while a quarter does so daily. Dentists in training most frequently cited curiosity as their main motivator for EBP and reported a lack of relevant evidence as a key barrier to further engagement with the evidence base. On the other hand, doctors frequently cited their engagement with EBP in the context of accepted practice for the departments in which they work. For both doctors and dentists, a lack of time and poor access to relevant literature were key barriers to further engagement in EBP. Trainees nevertheless reported high levels of confidence in searching for evidence and interpreting basic statistics. However, the majority also reported that they would not query a more senior colleague based on their own reading of the evidence, thereby potentially limiting the extent to which their confidence translates in to EBP.The notion that trainees appreciate EBP but do not consider their practice to be consistently evidence based is supported elsewhere in the literature.Our findings have a number of additional implications, which the Cochrane UK & Ireland Trainees Advisory Group have reviewed with a view to formulating strategic, concrete plan to promote EBP. The higher reported confidence in EBP of trainees who have undertaken formal postgraduate academic training demonstrates the value of academic training schemes to healthcare settings. Notably, that the hierarchy and departmental practice so considerably shape the degree to which trainees engage in EBP demonstrates the parallel importance of ensuring an appropriate training environment. In view of this, the Trainees Advisory Group plans to collect examples of good practice as a result of postgraduate academic training and positive influence from colleagues or department practice, and disseminate them through its online platforms. The data depicted in our survey additionally demonstrate a clear need to support busy trainees in digesting evidence, whether through directly managing their workload or by ensuring that sources of evidence are readily available and their conclusions efficiently consumable. Other implications of this work are likely to be harder to address. The paucity of relevant evidence highlighted by dental trainees is one such area but may be challenged by promoting greater awareness of EBP within the profession.The number of trainees reached by the survey is not known. However, there are over 60 000 doctors and nearly 3000 dentists in training in the UK alone.This survey nevertheless had a wide geographical reach across both the UK and RoI, incorporated a significant number of specialties and featured responses from both doctors and dentists in training. Internal consistency was also high, as measured by a Cronbach’s α of 0.83. The data provide a number of clear themes, some of which are specific to the postgraduate trainee population. For example, the important influence of hierarchy, both as a motivator and a barrier, is a trainee-specific theme that emerged prominently as a result of our focus on the postgraduate trainee population. These themes provide a starting point for further research and intervention from both healthcare professionals and policymakers. The validity of these themes is supported by their concordance with historical evidence,Time and a lack of access to key resources limit the ability of doctors and dentists in training within the UK and RoI to engage with EBP. While the majority of these trainees report high levels of confidence in searching and interpreting statistics in the literature, a significant proportion highlight that hierarchy and departmental practice significantly shape their EBP. The extent to which these data vary between specialties and by stage of training is unknown and requires further research. Healthcare professionals and policymakers should ensure that trainees are provided with the time and ability to efficiently access sources of evidence, and should engender cultures in which EBP is encouraged.


We have read with great interest the paper by Patıroğlu et al.  on the mThe patient herein is a physician and the senior author of this paper (İ.C.H.). He is also the mentor of the first author (R.Ç.). The experimental therapeutic drug is ABS, which was developed as a medicine with numerous clinical studies (https://www.ncbi.nlm.nih.gov/m/pubmed/?term=ankaferd), mostly authored by İ.C.H. himself. ABS is the first topical hemostatic agent acting on red blood cells and fibrinogen gamma interactions to be tested in clinical trials . ABS is At the time of the burn accident, ABS for burn wound management had only been demonstrated in rats ,5. The bThe emergency state of the severe burn lesions and the availability of ABS at the time of the accident enabled us to apply it topically to the burn lesions of İ.C.H. The burn lesions were clearly regressed and wound healing occurred with no complications upon the usage of ABS in our physician patient .Helicobacter pylori bacteria himself and developed stomach ulcers within a few days. He later successfully treated himself with antibiotics and went on to win the Nobel Prize [In the history of medical science, there are many inventors that applied their own therapeutic tools for the management of their own diseases, such as Dr. Barry J. Marshall. He drank el Prize . The conclusions that were drawn from our unique clinical story are as follows:- Medical inventors and researchers are enthusiastic for the use of their experimental drugs in clinical situations.- The weakest aspect of evidence-based medicine is the ‘lack of evidence’ in the related particular clinical problem. This represents a great challenge, particularly for real-life medical emergencies.- Sometimes medical doctors have to make clinical decisions despite the lack of solid scientific evidence in the presence of urgent medical needs.- Rat and animal studies may be the only source of evidence for human use in some medical emergencies.- Nevertheless, the best clinical practice should rely on the best current evidence obtained through randomized controlled clinical trials.


Patient and Public Involvement (PPI) strategic documents are viewed as an essential feature of organisational commitment to openness and transparency. They provide a mechanism to communicate opportunities for wider community influence in healthcare. The absence of documentation can be negatively interpreted, for example during regulatory inspection, as a lack of intent by organisations to collaborate with a broad constituency. Published literature paints a confusing picture of rationale and evidence that could provide the foundation for strategic action. This makes it difficult for those responsible for turning goals into meaningful involvement. We investigated how content is presented and organised in strategic documents. This pragmatic study is intended to stimulate reflective practice, promote debate and generate further inquiry with a wide audience.We created and iterated a framework adapted from 4Pi National Involvement Standards to analyse organisational PPI strategic documents against five domains which are principles, purpose, presence, process and impact. Fifteen strategic documents were grouped into four categories  and included for analysis. A matrix was produced. By reading the matrix vertically (down) and horizontally (across), comparisons can be made between 4Pi domains and across organisations.There was no discernible pattern between domains or between organisations. There was variation in the level to which criteria were met. No single strategy fully met the criteria for all five domains of 4Pi National Involvement Standards. The criteria for purpose was fully met in eight strategic documents. Only two documents fully met impact criteria. Four organisations showed better completeness with fully or partially met criteria across five domains. A single organisation partially met the criteria for all domains. The remaining 10 were unable to meet the criteria in at least one domain.Our findings align with published literature that suggests the underpinning rationale for PPI is confusing. A strategic aim is difficult to articulate. Context and complexity are at play making the sharing of generalisable knowledge elusive. We offer further critique about the value of these documents and consider: ‘is there an alternative approach to construct PPI strategy to generate theory, capture learning and evaluate effectiveness at the same time?’ We suggest testing the adoption of programme theory in PPI. The emergent nature and context sensitive features of programme theory enable curiosity, creativity and critical appraisal. It has the potential to release practitioners from the tokenistic cycle of monitoring and reporting and replace this with a richer understanding of ‘what’ works and ‘how’ tied to a ‘why’ – in order to achieve a shared aim that everyone can get behind. The availability of organisational Patient and Public Involvement (PPI) strategic documents suggests that there is a commitment to work with a wider community to improve healthcare and strengthen health research. Drawing on our experience working with practitioners we noticed documents are not always available or written in a way a non-expert can understand. Proposed actions are not necessarily linked to a defined goal. This makes it difficult for those responsible for turning strategic goals into real action. People who could help, especially public contributors, may find it difficult to understand when, where and how their knowledge and experience could be best used. This led us to investigate further. We tested a framework adapted from 4Pi National Involvement Standards to methodically assess 15 documents from different healthcare organisations including those active in community care, research and education. We examined how document content is presented and organised. No single document met the full criteria for addressing the standards which are organised into five domains called principles, purpose, presence, process and impact. There was variation. We critique the value of these documents in practice and ask: ‘is there an alternative approach to construct PPI strategy to generate theory, capture learning and evaluate effectiveness at the same time?’ We open debate about alternative ways to show and evidence organisational practice and learning. We suggest an approach called ‘programme theory’ which has the potential to minimise tokenistic practice and connect learning about the ‘why’, ‘what’, and ‘how’ of PPI.Patient and Public Involvement (PPI) strategic documents are viewed as an essential feature of organisational commitment to openness and transparency. They can provide a mechanism to communicate opportunities for wider community influence in healthcare. The presence of a PPI strategic document on a website or the production of this evidence on request is interpreted as a signal that efforts are being made to connect with people who use services or with those who may be able to contribute to the generation of new knowledge about health and healthcare, for example through research and quality improvement (QI) –3. The aDespite the proliferation of evidence, policy, guidance and toolkits about PPI in healthcare and research, published literature offers a complex and confusing picture about the underlying rationale for involvement practice –14. MartAgainst this complex background, we decided to further investigate documentary evidence to explore the content and construction of PPI strategic documents across a range of organisations. This was in part prompted by a challenge experienced in practice when invited to conduct an informal peer review of a draft strategy in a partner organisation. We noticed the proposed actions appeared disconnected from the overall aim. Documentary evidence is not a proxy for actual practice and it is not our intention to offer judgements about practice or the individuals responsible for the construction of documents. Our plan was two-fold. First to adapt and test an existing framework to inform analysis and second to methodically critique documents against this benchmark which attends to important features of meaningful involvement.It possesses universal relevance;It is underpinned by experiential and research evidence;It is firmly grounded in service user experience and was constructed through partnership working;It explicitly states the core purpose of involvement to be improvement;The standards are written in plain English and offer information and explanations that are of benefit to people new to involvement;Mental health service user experience is at the forefront of championing active and progressive involvement in healthcare.We identified the 4Pi National Involvement Standards (4Pi)  as a valWe were familiar with and had expertise in using 4Pi to support healthcare professionals and patients in QI teams frame their PPI plans, actions and impact assessments , 21. 4Pivertically between 4Pi domains and horizontally across each organisation. The 4Pi domains of principles, purpose, presence, process and impact were adapted as a coding framework. We developed an iterative and pragmatic search and sampling strategy based on available time and resource. This was conducted between July and September 2016. We aimed to source PPI strategic documents for 63 partner organisations associated with NIHR Collaboration for Leadership in Applied Health Research and Care Northwest London (CLAHRC NWL)  as their broad remit and geographical reach extended beyond Northwest London. Voluntary, third sector and charitable organisations were excluded for similar reasons. Disruption caused by local authority restructuring and financial constraints at the time of the search made it difficult to find active websites with current strategic documents. Fifteen documents were selected for final analysis. Identifying features were removed to preserve organisational anonymity Table . Documenn = 1) against the criteria, discussing differences of opinion to inform any further adaptations. Following this test we introduced sub-questions to strengthen inter-rater reliability. MK and RM subsequently analysed three documents together: one acute trust; one clinical commissioning group; and one community healthcare trust (n = 3). No further iterations were made to the framework. Analysis was independently conducted for the remaining documents (n = 11). Further discussion was initiated with AB if there was perceived ambiguity and difference of opinion to reach consensus. NVivo 10 software was used to store, order and code data and to select relevant supporting quotes. A rating was assigned to each strategy and corresponding domain coding cell ” [B8]“We have a duty and obligation to inform, engage and consult with the public to ensure accountability and build trust.”[B10]Other aims implied a technocratic purpose:“Bringing together the expertise of local people and professionals to champion and drive high quality health and wellbeing for all in our community and overcome health inequalities.” [B7]PPI is also articulated as a mechanism through which change could be achieved to deliver more equitable care and health outcomes:“This strategy describes the shared vision of [organisation] and its User Panel” [B9]“The User Panel has been central in defining the overall mission for our CCG” [B9]“Co-production of our involvement strategy with the strategic lay forum – and … other patients, carers and local people … we have run two ‘co-design’ events to produce our longer-term involvement strategy and implementation plan.” [A3]One strategic document fully met the criteria. Ten partially met the criteria and four did not meet it. There was some indication about who is involved but it was often vague and lacking in detail. There was little or no evidence to show any understanding of the demographics of local populations or service users that might inform the strategic document. If users were referred to, they were more likely to be presented as a homogenous group:“ … the PPE objective of ensuring inclusion of patients and public from across our constituent communities and groups and address health inequalities.” [B5]The absence of specific target groups presents difficulties when selecting appropriate involvement mechanisms. There was limited assurance that involvement would be shaped through the knowledge and experience of their local population groups:Some documents did name or acknowledge those who had contributed to the content but this was rare.“[organisation name] is committed to using PPE at all stages of the commissioning cycle and in particular: ... Specifying outcomes and procuring services: Engaging patients in service design and improvement...” [B10]“As well as quarterly meetings with Theme leads and regular reports for the … Committee, we will hold workshops for PPI leads / champions within the [organisation], and use [Forum] to bring together stakeholders from across NW London.” [D12]One strategic document fully met the criteria, 11 partially met the criteria and three did not meet the criteria. There was variable evidence to explain how the overall purpose of the strategy was to be achieved. Whilst objectives and actions were included, it was difficult to understand how they linked to the overall aim and what the predicted impact of action might be. Reporting, monitoring and organisational accountability were variably explained. It was rare for documents to provide sufficient detail that would enable a reader with limited knowledge to understand what might happen in practice.Only two strategic documents fully met this criteria. It was hard to find examples of desired impact together with mechanisms that enabled it to be assessed, evaluated or measured.Some strategic documents provided statements explaining ‘good’ implementation of the strategy [C11] and how it will make a difference [A1].“Basic level: Starting off – Increased staff awareness of PPE requirementsGold level: Exemplar – Service redesign activity conducted with evidenced input from service users” [C11]One strategy provided a useful framework to assess impact for those with knowledge and experience of involvement, and those who may need extra support. The strategy [C11] suggested how involvement would be evidenced at various organisational levels , alongside a traffic-light system for example:“Minutes and documents must evidence that service users were involved, how they felt about their involvement and what impact it had” [A2]Whilst these may indicate the impact of improved processes, they do little to shed light on the desired outcome. Examples of evidence are stated as:“Measuring the level of impact people feel that they are achieving through our engagement activities” [B9]It was more common for documents to provide general statements about measuring impact but not enough clarity for anybody to meaningfully do so:There is no discernible pattern by organisation type. A1, B7, B9 and D15 show the best completeness with domains that are fully and partially completed and none that are unmet. A3 shows partial fulfilment across all five domains. All remaining organisations have domains which do not meet the criteria in at least one domain. Only C9 fully meets the presence criteria. D15 alone fully meets the process criteria. C8 and C11 fully meet the impact criteria.Our results demonstrate a varied picture of involvement intent across different organisations. No single document addressed all five domains of 4Pi. Document readers are required to work hard to search for and disentangle the ‘why’ (strategic aim) and ‘how’ (plan of action) of involvement. Dense language has to be unpacked to gain an understanding of aspiration and direction. The documents differed in accessibility of language and content. Some were closer to Plain English and clearly explained NHS or organisational structures. Managerial and technical language with jargon and acronyms dominated the sample. Very few documents stated any meaningful detail about who was to be involved, for example by offering data about the local community or demographic information about the patient population across services. This was especially concerning when considering issues of inclusivity, equity and equality.In our introduction we identified that there is limited actionable evidence to support those tasked with developing, implementing and evaluating meaningful patient and public involvement. The identified policy tensions are reflected in the documents we investigated , 10, 12.In drawing out the key results, the purpose domain of 4Pi is the one which was fully met in eight out 15 documents. This leaves seven documents without a clearly defined purpose. If we look at how impact is addressed, only two out of 15 documents fully met our criteria. Six documents were unable to meet the criteria for impact in any way. This raises questions about if and how purpose and impact can be logically linked. These domains are essential features of contemporary involvement practice. Each may be considered separately and can be valued differently. For example, the demonstration of impact continues to invite debate –32. TherThe relationship between the theory and practice of involvement has parallels with other fields, for example QI . Whilst A gap between the theory and practice of involvement remains. This explains a commonly expressed frustration in the field when attempting to answer ‘why’ we should involve and difficulty in describing ‘how’ to do it with greater certainty that we will achieve meaningful results to improve care or strengthen research. We therefore open the debate by posing the question: ‘is there an alternative approach to construct PPI strategy to generate context specific theory, capture learning and evaluate effectiveness at the same time?’We offer perspective from our applied health research programme where PPI is considered an integral component in supporting the translation of research evidence into practice using a QI approach . We suggThere is growing interest in the relationship between PPI and QI. Bergerum and colleagues  report aThe action effect diagram Fig.  demonstrThere are limitations to our research. The study is localised, however the use of documentation in PPI is widespread and will be relevant internationally and for readers from different backgrounds. Further work could be conducted to test transferability to other settings. The final sample documents were not constructed with knowledge of or alignment with 4Pi. Our intention was to bring discipline and order to the subjective interpretation of qualitative information. We cannot rule out unconscious bias in our appraisal of the sample.PPI is an established feature in healthcare policy and research in the UK and internationally. Patients, practitioners and researchers continue to grapple with the day to day reality of demonstrating meaningful practice that will further consolidate the value of involvement. In the absence of compelling actionable, context specific evidence, the gains are fragile. We open the debate about the limitations of PPI strategic documents and question the desirability of generating generalisable evidence and associated attempts to achieve consensus in how to capture impact. Testing the adoption of programme theory in the field of PPI, for example through the application of the action effect method, could introduce a structured approach to continually test and learn from practice. The emergent nature of programme theory enables shared curiosity, creativity and critical appraisal. It has the potential to release practitioners from the tokenistic cycle of monitoring and reporting strategic progress, replacing it with a richer understanding of ‘what’ works and ‘how’ tied to a ‘why’ – in order to achieve a shared aim that everyone can get behind.


Current methods for assessing strength of evidence prioritize the contributions of randomized controlled trials (RCTs). The objective of this study was to characterize strength of evidence (SOE) tools in recent use, identify their application to lifestyle interventions for improved longevity, vitality, or successful aging, and to assess implications of the findings.The search strategy was created in PubMed and modified as needed for four additional databases: Embase, AnthropologyPlus, PsycINFO, and Ageline, supplemented by manual searching. Systematic reviews and meta-analyses of intervention trials or observational studies relevant to lifestyle intervention were included if they used a specified SOE tool. Data was collected for each SOE tool. Conditions necessary for assigning the highest SOE grading and treatment of prospective cohort studies within each SOE rating framework were summarized. The expert panel convened to discuss the implications of findings for assessing evidence in the domain of lifestyle medicine.Hierarchies of Evidence Applied to Lifestyle Medicine (HEALM), to illustrate the feasibility of a tool based on the specific contributions of diverse research methods to understanding lifetime effects of health behaviors. Assessment of evidence relevant to lifestyle medicine requires a potential adaptation of SOE approaches when outcomes and/or exposures obviate exclusive or preferential reliance on RCTs. This systematic review was registered with the International Prospective Register of Systematic Reviews, PROSPERO [CRD42018082148].A total of 15 unique tools were identified. Ten were tools developed and used by governmental agencies or other equivalent professional bodies and were applicable in a variety of settings. Of these 10, four require consistent results from RCTs of high quality to award the highest rating of evidence. Most SOE tools include prospective cohort studies only to note their secondary contribution to overall SOE as compared to RCTs. We developed a new construct, The online version of this article (10.1186/s12874-019-0811-z) contains supplementary material, which is available to authorized users. There is at present lively debate in the peer-reviewed literature regarding the nature of evidence supporting specific recommendations pertaining to nutrition , 2 and oAssessment of scientific evidence for a given question has evolved in academic publications from the presentation of an individual author’s conclusions into a formalized process –7 that iEvaluating SOE for research questions related to health behaviors of individuals is of high importance for public health professionals and clinicians focusing on behavioral modification as part of clinical practice. Interest in lifestyle medicine is rapidly expanding globally . LifestyThe majority of current systems for evaluating scientific evidence are well-suited to conventional medical treatment such as pharmacotherapy and discrete procedures. The movement towards evidence-based medicine (EBM) in recent years has emphasized the commonly accepted hierarchy of evidence and generally places results from RCTs above other study designs , 17. WhiCost constraints and challenges with adherence makes it difficult to randomize individuals to lifestyle interventions and maintain the prescribed behaviors for sufficient time periods (decades) to investigate the effects of such exposures on mortality or long-term morbidity , 27.Blinding of the treatment group is only possible when the treatment is ostensibly similar to the placebo. While this is straightforward in drug trials, it is difficult at best, and often impossible when modifying health behaviors.The generalizability of results in intervention trials to the broader population may be limited.Specifically, RCTs have methodological limitations that impede application to the investigation of longevity, overall vitality , compresSome debate exists around differences in results seen between observational studies and RCTs. Depending on the research questions, evidence from observational cohort studies may be substantially more informative in drawing conclusions about overall SOE . There mWe conducted a methodologic systematic review of SOE tools to inform the answer to this question: When RCTs cannot, for whatever reason, serve as the primary evidence source, are there alternative assemblies of evidence that can be used to achieve comparable confidence in a given exposure-outcome relationship?The research team was convened by the American College of Lifestyle Medicine (ACLM) in joint auspices with the True Health Initiative (THI) to (1) conduct a methodological systematic review of SOE grading tools in recent or current use to characterize which assemblies of evidence produce an evidence rating of highest strength, and (2) analyze the findings and their implications for potentially developing a new grading tool to evaluate SOE in the specific context of lifestyle medicine, where often good RCTs are not available or possible.The Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) statement was followed in reporting this systematic review . The proThe search strategy was built in PubMed in consultation with a librarian and modified as needed for four additional databases: Embase, AnthropologyPlus, PsycINFO, and Ageline. The databases were searched for studies containing keywords related to either lifestyle or longevity. To identify only SOE tools in recent or current use, searches included studies published during the previous five years from the start of the project, from 01/01/2013–11/07/2017. There were seven exposures of interest related to lifestyle: diet, exercise, stress, social relationships/support, addiction(s), sleep, and genetic-based factors with potential for epigenetic modification. Additional search terms were included to restrict the scope of our literature search to papers related to avoidance of chronic disease: longevity, vitality, and healthy or successful aging. Keywords used in the search strategy are presented in Table To identify relevant SOE tools in current or recent use, we included systematic reviews and meta-analyses of intervention trials or observational studies that both examined lifestyle medicine exposures and outcomes and evaluated SOE using a specified SOE tool. The inclusion and exclusion criteria applied in abstract and full-text screening are presented in Table After merging results from all five databases and removing duplicates, all citations were title-screened by a single investigator [MK] to exclude in vitro, cell and stem cell studies, animal studies, and studies whose designs were clearly not a systematic review or meta-analysis, such as studies that used other study designs in the title  All studies with ambiguous titles were included at this stage of screening. All abstracts identified via the literature searches were then independently double-screened (independently screened by two different investigators)  using the inclusion and exclusion criteria Table  via the Additionally, the results from the systematic search process were complemented with manual searching on websites of major agencies recommended by the expert panel that conduct or commission systematic reviews. Agency websites were searched for officially adopted SOE tools . A list of unique tools was compiled from the combination of the systematic and manual searches [MK].Data extraction forms were created and received approval from the entire research team prior to use. The information extracted included the following: date first published; purpose of the evaluation; intended audience; number of levels of SOE; the definition of the highest level of SOE; and the placement of cohort studies in the framework of SOE. All data extractors initially extracted 10% of the articles to pilot uniformity of extractions. For all remaining articles, each article was extracted by one investigator and reviewed and confirmed by a second . Any disagreements were discussed among the research team and resolved via group consensus.As this systematic review’s focus is on SOE grading systems related to lifestyle medicine outcomes and not studies’ specific lifestyle-related findings, ROB assessments were not conducted. However, if ROB assessments played a role in the included SOE grading systems, details were extracted.Data were summarized in narrative form with regard to the conditions necessary for assigning the highest SOE grading . Next, the treatment of prospective cohort studies within each SOE rating framework was qualitatively summarized .The PRISMA flow diagram for study selection and exclusion is presented in Fig. All 15 tools rated SOE using three to five levels, with the exception of the US Food and Drug Administration (FDA) tool in reference to qualified health claims  Guidance to synthesize findings from diverse study designs, except to prioritize RCTs over observational studies.Upon completion of the systematic review, the expert panel convened to discuss the findings. The results confirmed that the following methodological elements within existing SOE tools in recent use are lacking:To address these issues, the Hierarchies of Evidence Applied to Lifestyle Medicine (HEALM) investigators enumerated the particular contributions of diverse research methods into a complete understanding of exposure/treatment effects, as shown in Table Evidence Threshold Pathway Mapping  . Such coHierarchies of Evidence Applied to Lifestyle Medicine (HEALM) shown in Table Thus, we introduce a new construct- HEALM incorporates the variety of sources of evidence available and synthesizes their contributions into one rating. It is important to note that the method described in Tables Lifestyle behaviors are among the leading determinants of health outcomes, with non-communicable disease causing nearly three-quarters of death globally . DietaryAssessment of SOE requires grading the methodological quality and ROB of individual included studies, assessing the consistency and internal validity of studies addressing a specific research question, and forming conclusion statements. Such SOE conclusions can thus inform the discussion on the weight of evidence, informed by multiple studies providing for external validity or generalizability to various populations, settings, and circumstances.Evidence Threshold Pathway Mapping contends that the same level of confidence, and the same strength of evidence, can be achieved by a variety of assemblies of evidence. The approach respects the unique value of RCTs in establishing attribution and does not assume RCTs are interchangeable with other study designs. Rather, Evidence Threshold Pathway Mapping acknowledges that RCTs may be precluded for various reasons with regard to a given outcome and that other complementary evidence should be considered. Even then, such trials may contribute to understanding by assessing attribution with use of interim measures, and/or surrogate markers. This method of identifying the SOE approach used for evaluation based on the nature of the question being asked is informed by the approach taken in the OCEBM tool , which tHEALM, derived from application of the Evidence Threshold Pathway Mapping approach, is one unique, potential approach organized to frame discussion of existing evidence available to answer specific research questions relevant to lifestyle medicine when existing tools such as GRADE are not viable options . The scoring, similar to other SOE tools, relies on expert consensus, but is also informed by quantitative scoring considerations used in umbrella reviews  to evaluThe need for innovation in SOE assessment is in part because the RCT holds a position of relative primacy in the adjudication of medical evidence. Arguments favoring reliance on RCTs rightly invoke the merits in this methodology, namely defense against diverse kinds of bias, and protection against confounders both known and unknown  thus priAlso of concern are the cases in which observational and intervention trial results appear to be in conflict with one another. In some cases, RCTs may be testing different hypotheses than observational studies, and conclusions from one investigation may not be generalizable to all populations. For example, a review analysis on the use of hormone replacement therapy (HRT) among women in the Women’s Health Initiative affirms the consistency of findings across observational and intervention data if the age at time of starting HRT is considered .consistently superior may be exaggerated, with the benefits and risks of hormone replacement therapy providing an example of the partial contributions to understanding made possible by both RCTs and observational cohort studies [A recent Cochrane systematic review concluded such differences are likely not due to differences in study design alone; rather, RCTs and observational studies tend to produce similar effect sizes for a range of health outcomes and disagreements are likely due to other study characteristics  such as  studies –67.In contrast, there are clear cases in which observational studies offer a superior method of evaluating questions concerning the cumulative, lifetime effects of lifestyle practices. A key example of such trials whose recruitment is designed to maximize the number of endpoints is the Alpha-Tocopherol, Beta-carotene Lung Cancer Prevention (ATBC) Study which targeted male smokers . In captThe HEALM tool scores evidence, lending particular weight to RCTs for the clarification of causal effects and attribution. The tool, however, allows for rating evidence as strong even if RCT data are not more than suggestive, provided evidence from all other complementary research approaches are decisive and aligned. More importantly, short-term evidence from RCTs, or focus on isolated biomarkers, absent any suitably long-term data addressing hard outcomes would not score as “strong” in the realm of lifestyle medicine because of the great potential divide between short and long-term effects. As an example, many serious infectious diseases lower weight and blood lipids; such “favorable” trends in biomarkers are obviously not indicative of beneficial health effects in the long term. This adaptation of established approaches readily accommodates the imperative of judging the impact of lifestyle practices on health outcomes over the full human life span.The strength of this study was to take an approach of a methodological systematic review to capture existing and recently used SOE tools, thus ensuring that a new method proposed would offer a novel contribution to address current methodological gaps. Limitations of this study included the focus in the search strategy on healthy aging as an outcome, rather than risk for specific chronic diseases. The search strategy was constructed in alignment with the target outcomes of lifestyle medicine practice , and inclusion of all major chronic disease outcomes would not have been practical due to the large number of search results. Additionally, the search strategy was limited to systematic reviews of studies conducted among those ≥65 years, not because lifestyle medicine is only relevant for older populations, but because this focused the search strategy to identify studies in the domain of longevity. SOE tools used in these contexts would be potential best matches for evaluating evidence concerning other lifestyle medicine-type questions. However, manual searching for SOE tools based on expert panel recommendations augmented the systematic review results to the degree that all major tools known by the expert panel are included in our results.Finally, the HEALM construct is dependent on conclusions about the “plurality” of evidence from distinct research methods. Other than results produced from systematic review of meta-analysis, there is no universal standard for sufficient or sufficiently consistent evidence to establish the veracity of a given causal pathway or weight of the evidence for a given research topic. Even meta-analyses and systematic reviews fail to reach this standard, because in “crowded” research domains more than one such study is common and they may conflict with one another. The Community Preventive Services Task Force (CPSTF)  providesThe problem of establishing an operational definition for the “weight of evidence,” or a decisive plurality of studies, is in no way specific to lifestyle medicine. This is a generic challenge pertaining to all assessments of overall evidence, and thus deemed beyond the scope of this particular effort. This group simply notes the importance of this issue, and its pertinence to both Evidence Threshold Pathway Mapping and HEALM. This paper invites attention to the matter and highlights the opportunity to fortify operational definitions in this area.This project was commissioned with a preferential focus on lifestyle medicine, but the implications apply broadly to public health. Lifestyle practices and exposures- dietary patterns, physical activity patterns, sleep patterns, tobacco and alcohol exposures, psychological stressors, social connections- while uniquely emphasized in lifestyle medicine (4), pertain to all fields of medicine and public health and to all health professionals.Future research should test application of Evidence Threshold Pathway Mapping and HEALM by conducting systematic reviews on specific research questions in the domain of lifestyle medicine. The HEALM construct should evolve, informed by research in which it is applied.SOE tools in current use are generally poorly suited to long-term effects of lifestyle choices such as diet, exercise, sleep, and stress. Evidence Threshold Pathway Mapping, a method for identifying multiple assemblies of evidence to achieve a given grade, extends the robust assessment of evidence to a wider array of questions important to medicine and public health. HEALM is proposed as one example of a tool specifically adapted to questions in lifestyle medicine and nutrition. Application, testing, and validation of the performance of HEALM and consideration of its relevance to this domain of medicine are encouraged.Additional file 1:Expert panel (DOCX 13 kb)Additional file 2:Search strategy (DOCX 17 kb)Additional file 3:Keywords used in text mining (DOCX 13 kb)


Adenoviruses (AdVs) are major contributors to clinical illnesses. Novel human and animal AdVs continue to be identified and characterized. Comparative analyses using bioinformatic methods and Omics-based technologies allow insights into how these human pathogens have emerged and their potential for host cross-species transmission. Systematic review of literature published across ProQuest, Pubmed, and Web of Science databases for evidence of adenoviral zoonotic potential identified 589 citations. After removing duplicates, 327 citations were screened for relevance; of which, 74 articles received full-text reviews. Among these, 24 were included here, of which 16 demonstrated evidence of zoonotic transmission of AdVs. These documented instances of AdV crossing host species barriers between humans and non-human primate, bat, feline, swine, canine, ovine, and caprine. Eight studies sought to but did not find evidence of zoonosis. The findings demonstrate substantial evidence suggesting AdVs have previously and will continue crossing host species barriers. These have human health consequences both in terms of novel pathogen emergence and epidemic outbreaks, and of appropriate and safe use of non-human adenoviruses for therapeutics. As routine human clinical diagnostics may miss a novel cross-species adenovirus infection in humans, next generation sequencing or panspecies molecular diagnostics may be necessary to detect such incursions. Subsequpathogen  as it copathogen . HAdV-55pathogen ,18, illupathogen .Mastadenovirus, there were 51 serotypes that are now integrated into the 103 human AdV genotypes described in high resolution using genomics [Mastadenovirus infect a wide range of mammalian hosts including bats, bovines, canines, deer, dolphins, equines, murines, non-human primates (NHPs), ovines, swine, sea lions, skunks, squirrels, and tree shrews. Further, there are AdVs in four other genera within the Adenoviridae family that infect avian, reptilian, and fish species- essentially covering all vertebrate species studied to date. As with humans, AdV infections in animals can cause diseases that range from asymptomatic to fatal.Within the genus genomics . These aIn addition to increased attention to HAdVs due to infection-associated morbidity, the biochemical and clinical characteristics of the virus, including its stability and ability to induce innate and adaptive immune responses in mammals, have made AdVs particularly attractive vectors for vaccines and gene therapy. The use of non-human simian AdVs (SAdVs)  to circuGiven that humans and animals are known hosts to AdVs, it is reasonable to consider the possibility of zoonotic or cross-species transmission of AdVs. Hence, we sought to review literature surrounding the zoonotic potential of adenoviruses. For the purposes of this systematic review, “zoonoses” was considered using a One Health definition, as a “two-way street, with humans infecting animals as well as the other way round” . In instOn November 8, 2018, we conducted a search of literature for evidence that AdV can be transmitted between humans and other animals through ProQuest, Pubmed, and Web of Science. This search followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. To be captured in the search, the terms “adenovirus”  along with a term implying transmission of virus between humans and other animals in either direction  must have been included in the text. The complete search string used was, “((adenovirus* OR mastadenovirus OR atadenovirus OR aviadenovirus OR ichtadenovirus OR siadenovirus) AND (cross-species OR interspecies OR zoono* OR anthropono* OR anthropozoono* OR zooanthropono*)).” Citations were exported from the three databases into Endnote, where duplicates were removed. All abstracts and titles were screened, and relevant articles available in English or French were selected for a full review if they mentioned adenovirus and language indicating an investigation of evidence of cross-species virus transmission. At each step, articles were screened separately by J.K.F. and L.K.B. and then reviewed and discussed until the authors were in agreement with respect to interpretation.The initial search generated 589 citations with 221 through ProQuest, 228 through Pubmed, and 140 through Web of Science . After rOf the 24 studies, 18 (75%) provided phylogenetic evidence, four (17%) provided serological evidence, one provided both phylogenetic and serological data, and one relied on molecular evidence supporting or refuting the zoonotic potential of the viruses. Of the studies presenting phylogenetic evidence of zoonoses, three demonstrated interspecies recombination of HAdVs and simian AdVs (SAdVs) ,25,35 anThe study by Chen, et al. provided both phylogenetic evidence using whole-genome sequencing and serological evidence of human infections with a titi monkey adenovirus (TMAdV) producing arguably the strongest evidence of a anthropozoonotic transmission of adenoviruses . In thisAnother report supporting anthropozoonotic transmission of adenoviruses to humans from primates noted a 1997 outbreak of acute respiratory illness in baboons at a primate research facility in Texas. In this outbreak, six novel baboons adenoviruses (BaAdVs) comprising a proposed novel “simian adenovirus C (SAdV-C)” species were isolated from asymptomatic and sick baboons, including two fatalities. Although no humans showed symptoms, antibodies to these new BaAdVs were found in staff personnel at the facility, as well as in other baboons .DNA sequencing data show genomic “footprints” of cross-species transmission as two nearly identical SAdVs, isolated independently from a chimpanzee at the New Iberia Research Center  and a bonobo in a zoological setting  by Roy, et al. , containWe identified six studies (25%) demonstrating evidence of anthropozoonoses, five (21%) of which present transmission of AdVs from NHPs to humans. In one of these studies, Xiang et al. presented serological evidence of anthropozoonotic transmission of AdV to NHPs from five countries across three continents . The stuSignificantly, one of the first human viral pathogens isolated and characterized in 1953 as cytopathogenic respiratory illness agent RI-67 , later rAnother study by Phan et al. reported RT–PCR detection of feline AdV in a one-year old child hospitalized for acute gastroenteritis . ThroughThe review found eight studies (33%) demonstrating evidence of zooanthroponoses, including six studies (25%) that provided evidence of transmission of AdVs from humans to NHPs. Across those six studies, the most common HAdV species detected in NHPs were HAdV-B, -C, -E, and -F, though the global study of fecal, blood and tissue samples from wild and captive NHPs conducted by Wevers et al. found all HAdVs species A-G .Eidolon helvum or fruit bat AdV-1 hexon forms a phylogenetic clade separate from previously identified bat AdVs, and is distinct in that it clusters with the HAdV clade. The high bootstrap value of 93 in their phylogenetic analysis indicates strong support that will likely be preserved with additional sampling. It was noted by Baker et al. that the hexon shared 77% and 90% amino acid similarities with the HAdV counterpart across 58 and 63 amino acids, respectively, of the 900 amino acid hexon protein [Baker et al. isolated a novel AdV from bat urine in Ghana and sequenced the hexon gene . This Ei protein .Additionally, Pauly et al. detected HAdV in fecal samples collected from pigs, dogs, sheep, and goats in Côte d’Ivoire . Even moThe review captured eight studies which did not find evidence of the zoonotic potential of AdVs. Of those studies, four specifically sought to find evidence of zoonanthroponoses and three specifically sought to find evidence of anthropozoonoses. Five studies provided phylogenetic evidence characterizing fecal samples from bats, rodents or NHPs, determining that the AdVs detected were genetically distinct from HAdVs. One negative study, Wang et al., was premised on the absence of molecular presence of AdV . The NkoUnlike our previously published systematic review of the zoonotic potential of enteroviruses (EVs) , in whicOf the negative studies, five were conducted in the United States, and one each in China, Gabon, and Tanzania. While this regional distribution may suggest the zoonotic potential of adenoviruses is not geographically isolated or clustered, it is important to note that the two U.S.-based studies demonstrating zooanthroponosis were based in primate facilities ,34. AddiThe majority of the studies identified as providing evidence for zoonotic transmission of AdVs reported species crossing between humans and non-human primates. In addition to these 13 reports, a recent study that was published after the search date for this review by Dehghan et al. unearthed evidence of a unique “ping-pong” pattern of AdV transmission between humans, chimpanzees, and bonobos when conducting genomics analyses of archived AdV sequences . Of noteAdditional relevant studies were published following the initial review in November 2018, including the previously cited Dehghan et al. study , OngrádiThis systematic review had several limitations. While a number of studies with minimal or no evidence of zoonotic adenovirus transmission were identified, these negative studies may have been more likely to be missed in this search. Additionally, our search strategy excluded studies with evidence of cross-species infections among nonhuman animals. Indeed, of the 50 articles excluded following the full-text review, 13 were excluded from the final analysis as they strictly investigated cross-species infections between two non-human animals.Given the evidence in this review, it is apparent that AdVs have crossed, and will continue to cross host species. This has been especially true for the species barrier between humans and NHPs (and likely between different NHP species), but it may also occur between humans and other animal species. Humans with intense exposure to NHPs, such as though occupational exposure at primate facilities, may be at higher risk of becoming infected with a zoonotic adenovirus. Routine clinical diagnostics are not likely to detect a novel animal adenovirus infection in humans. Hence, clinicians must have a strong degree of suspicion and apply next-generation sequencing and/or panspecies diagnostics  in detecting such a novel strain. Wellehan et al  have rep