Physicians often select clinical management strategies not strongly supported by evidence or guidelines. Our objective was to examine the likelihood of selecting, and rationale for pursuing, clinical management strategies with more or less guideline support among physicians using clinical vignettes of eight common medical admissions.We conducted a cross-sectional survey using clinical vignettes of attending physicians and housestaff at one internal medicine program in New York City. Each clinical vignette included a brief clinical scenario and a varying number of clinical management strategies: diagnostic tests, consultations, and treatments, some of which had strong evidence or guideline support (Level 1 strategies) while others had limited evidence or guideline support (Level 3 strategies). Likelihood of selecting a given management strategy was assessed using Likert scales and multiple response options were used to indicate rationale(s) for selections.Our sample included 79 physicians; 68 (86%) were younger than 40 years of age, 34 (43%) were female. There were 31 attending physicians (39%) and 48 housestaff (61%) and 39 (49%) had or planned to have primarily primary care internal medicine clinical responsibilities. Overall, physicians were more likely to select Level 1 strategies “always” or “most of the time” when compared with Level 3 strategies , with wide variation across the eight medical admissions. There were no differences between attending and housestaff physician likelihood of selecting Level 3 strategies . Supportive evidence and local practice patterns were the two most common rationales behind selections; supportive evidence was cited as the most common rationale for selecting Level 1 when compared with Level 3 strategies , whereas ruling out other severe conditions was cited most often for Level 3 strategies.For eight common medical admissions, physicians selected more than 80% of management strategies with strong evidence or guideline support, but also selected more than 40% of strategies for which there was limited evidence or guideline support. The promotion of evidence-based care, including the avoidance of care that is not strongly supported by evidence or guidelines, may require better evidence dissemination and educational outreach to physicians. The American health care system suffers from uneven quality  and wideWhile there is a rich literature on general guideline attitudes and adherence, the differences in likelihood and reasoning for selecting between clinical management strategies that have different levels of guideline support has been sparingly investigated. Within most guidelines there is a gradation of recommendations from those with high quality evidence to those with little or no evidence of efficacy. Moreover, these selections may be different for management strategies that are focused on treatments, as opposed to imaging, laboratory testing, or consultations.Understanding whether physicians use different justifications or explanations for selecting evidence-based, highly supported medical care versus poorly supported, or even contraindicated, care could have substantial implications for reducing the provision of wasteful and potentially dangerous medical care. To investigate differences in likelihood of selecting, and rationale for pursuing, clinical management strategies with more or less guideline support among physicians, we conducted a cross-sectional study using clinical vignettes of common medical admissions among attending and housestaff physicians at one academic medical center.We developed a cross-sectional, self-administered survey for physicians and housestaff working within the Department of Internal Medicine at the Mount Sinai Hospital, a large tertiary care academic medical center in New York City, New York, USA. The Department has 400 full-time faculty, 140 categorical, preliminary, and research track housestaff, and more than 10,000 annual hospital discharges. At the time the study was conducted, all investigators (with the exception of JA) were either on the faculty or trainees at the institution.Attending physicians in the Department of Internal Medicine who worked at least 4-weeks per year on the general medicine or geriatrics teaching services and housestaff in the Department of Internal Medicine at the post-graduate year two level and above were eligible. Participation was voluntary and participants received a $5 appreciation gift card. This study was approved by the Mount Sinai School of Medicine Institutional Review Board and all study subjects underwent informed consent.We designed a survey to examine physician clinical decision making for eight common medical admissions. All chosen medical admissions were conditions for which published evidence-based guidelines were available that rated the strength of existing evidence supporting the clinical recommendations. The eight chosen conditions were congestive heart failure ,18, chroEach vignette was constructed through an iterative process by three of the authors , which included solicited feedback from a second general internist and an appropriate subspecialist as to whether the clinical scenario and decisions in the vignettes were ambiguous. We then piloted the survey on three housestaff and two attending physicians and feedback was incorporated into the final vignettes. Our eight vignettes each had two parts. Part one described the initial patient presentation to the hospital and a series of possible clinical management strategies from all four core areas of patient care, including treatments, imaging, laboratory tests, and consultations. Part two presented results of the diagnostic tests included in part one and additional clinical management strategies. All responses for part one were collected before participants received the second part of the vignettes, so part one responses could not be changed after receiving additional information during part two.The likelihood of selecting a clinical management strategy was asked using a 5-point ordinal scale . For each selection, respondents were asked to indicate one or more rationales, including: local practice patterns, supervisor expectations, supporting data or guidelines (evidence), malpractice concerns, academic purposes , or to rule out more serious conditions. These rationales were derived from several group sessions with attending and housestaff physicians where clinical management strategies were discussed. Each vignette listed clinical strategies for which there was a range of evidence. We included management strategies that were either recommended or not recommended by the guidelines, based on clinical evidence or expert opinion. Recommended strategies included both class I and II recommendations  and are henceforth referred to as Level 1 strategies; non-recommended strategies include class III recommendations  and are henceforth referred to as Level 3 strategies. In particular, class I and II guidelines recommend management strategies for which the benefits are greater than or equal to the risk, whereas class III guidelines suggest that the risk outweighs the benefits. In all, the 8 clinical vignettes included 64 Level 1 strategies and 78 Level 3 strategies.We also collected information about demographic and professional characteristics of respondents such as: age, gender, intended or current subspecialty, and intended or current clinical activities. Respondents were also asked to rate factors that influence their practice patterns using a 5-point ordinal scale and sources of information that inform their clinical practice on a 7-point ordinal scale. Housestaff and attending physicians completed the surveys during regularly scheduled noon-conferences and administrative time respectively.Responses to the likelihood of making a Level 1 or Level 3 clinical strategies were collapsed into two groups; “Always” and “Most of the time” versus “Some of the time”, “Rarely” and “Never.” The proportion of respondents selecting Level 1 and Level 3 strategies ‘always’ and ‘most of the time’ was calculated for each clinical scenario. Physicians’ decision selection within each of the four core areas of care; laboratory testing, imaging, treatment and consult decisions, and the rationales behind selected clinical decisions were determined. We compared the proportions of respondents that selected “All” or “Most” of the time for Level 1 and Level 3 decisions by demographic and clinical practice characteristics using Chi Squared-test and Mann–Whitney U-test as appropriate. As the clinical scenarios varied in disease and severity of presentation, to inform future work, we also compared how often Level 1 and Level 3 management strategies were selected within each vignette and in comparison across clinical scenarios, using community acquired pneumonia as the reference scenario. Finally, the rationales for were compared for selecting Level 1 and Level 3 clinical management strategies. We used a type 1 error level of 0.01 to account for multiple comparisons. All analyses were completed using SPSS® version 18.0 .Of the 129 physicians eligible for the study, 79 (61%) completed the survey; made up of 48 (61%) housestaff and 31 (39%) attending physicians. Sixty eight (86%) physicians were younger than 40 years of age and 34 (43%) were female  of the respondents indicated their preferred practice setting as metropolitan, with none of the respondents having any intentions of practicing in a rural setting. When asked to describe factors that influence their clinical practice, the most common reasons cited were accumulated research evidence , hospital culture  and individual practice experience . Only a third cited malpractice concerns . When asked about the main sources of information used to inform clinical practice, the most common sources cited were peer-reviewed journals  and practice guidelines .Overall, physicians were more likely to select Level 1 management strategies “always” or “most of the time” when compared with Level 3 management strategies   to develop and maintain hospital performance measures that are used for public reporting; he is also a member of a scientific advisory board for FAIR Health, Inc. No other authors have potential competing interests to report.KLS and JSR were responsible for the conception and design of this work and drafted the manuscript. KLS, SA, and MM were responsible for acquisition of data. JA conducted the statistical analysis. JSR provided supervision. KLS, SA, JA, MM, and JSR participated in the analysis and interpretation of the data and critically revised the manuscript for important intellectual content. All authors read and approved the final manuscript.The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/14/176/prepubTriple C manuscript 2013-10-16 appendix.doc, 502 K http://www.biomedcentral.com/imedia/1827684101160557/supp1.doc.Click here for file


The ability to do online searches for health information has led to concerns that patients find the results confusing and that they often lead to expectations for treatments that have little supportive evidence. At the same time, the science of summarizing research evidence has advanced to the point where it is increasingly possible to quantify treatment tradeoffs and to describe the balance between harms and benefits for individual patients.Trustworthy clinical practice guidelines provide evidence-based recommendations to health care practitioners based on assessments of study-level averages. In an effort to customize the use of evidence and ensure that choices are consistent with their personal preferences, tools for patients have been developed. Gradually, there is recognition that the audience for high quality evidence is much wider than merely health care professionals – and that there is a case to be made for creating tools that translate existing evidence into tools to help patients and clinicians work together to decide next steps.We observe two processes occurring: first, is the recognition that decision making in healthcare requires collaboration and deliberation, and second, to achieve this, we need tools designed to customize care at the level of individuals. In November 2014, the Flemish government paid the Belgian Center of Evidence-Based Medicine to create a video telling patients to avoid Google before going to see their own doctor [It was 40 years ago when Archie Cochrane wrote: “It is surely a great criticism of our profession that we have not organized a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomized control trials .” This oHowever, guidelines have limitations, especially when we consider the necessity to customize treatments and decisions. It is not only difficult to find guidelines, it is also difficult to know which ones are of high quality. Guidelines can only make recommendations for typical patients, and cannot ensure that individual values and preferences are accounted for in the ultimate management decisions.Despite the enormous investment in guideline production, only a limited number have been successfully tailored so that they are optimally useful to clinicians and very few, if any, have been tailored for optimal usefulness to patients. Clinicians have the task of tailoring evidence to individuals – to people at different ages and with different sets of problems, concerns, preferences, and priorities. Many existing clinical practice guidelines do not support this task despite recent efforts to include patients as stakeholders in guideline development , 9, and free access to the sum of all medical knowledge” [There have been efforts to make clinical guidelines more accessible to a wider audience, typically by making them shorter and easier to read. Others want to create more recognizable knowledge sources for patients Table . For exaowledge” . Howeverowledge” . NeverthGuidelines that summarize scientific evidence and provide recommendations that apply at a general level are necessary but far from sufficient. Nevertheless, however much we work to make guidelines readable and accessible to a wider audience, they will be limited in their application to the situations – likely a majority – that are sensitive to patients’ differing circumstances, preferences, and priorities. Additional tools are necessary to ensure that guidelines are optimally applicable – tools that are just as rigorous when it comes to evidence but that also help individualize decisions to the patient at hand. Such tools would compare treatments and make explicit the tradeoffs between the benefits and harms. Such tools exist, and have been termed patient decision support.Less well known than guidelines, patient decision support tools or aids have been developed over the last few decades. Their goal is to support patients to compare alternative treatment options. Some tools are designed for use by patients before clinical encounters and other much shorter versions are designed for use by patients and clinicians together , 13. A sMany of these tools also include information about the experiences of patients, and describe what it is like to have the illness and to experience the treatment, including side effects. Patients and their families find this kind of information highly valuable, and as important as understanding the probabilities of likely outcomes. An excellent example of this kind of information is the Healthtalk Online website –18. PatiWhilst we can confidently make the argument for the adoption of evidence-based tools to help produce customized care for individuals, there are many steps required before this vision becomes a reality. Evidence synthesis will become more efficient, automated, and will increase the speed at which tools can be populated with risk data that can be used at the level of individuals. There is considerable research published on how to produce these types of tools and about how to communicate risk that it is understood well by both clinicians and patients. There are also sources that provide patients and their families with information about illness and treatment experience. Bringing these elements together will be a challenge, and there will be a need to have versions that can be viewed or used independently as well as versions that are brief enough for use in clinical encounters.How to make these widely accessible is going to be another significant hurdle. Clinical practice guidelines have been made available to clinicians by a range of different arrangements. Patient-facing decision support tools are not available in the same way, they are available commercially by a small number of producers, or are part of research evaluations. Patients and their families cannot easily find them using Internet search engines.Here, then, are some steps that would help us achieve a future where evidence-based customized care tools might become more available.The work required to locate the highest quality evidence before producing the best estimates of treatment effects, burden , benefitPatients are, of course, interested in more than evidence-based estimates of risk. They want to know exactly what they will face as they develop an illness, and what do other patients say about their experiences of illness. What are their main concerns, questions, and priorities , 21? ResAs mentioned, different types of tools will be necessary to facilitate shared decision making. What ‘matters most to patients’ should guide the presentation of evidence, specifically in terms of organizing the content and conveying the probabilities of treatment effects, harms, and burden in ways that communicate risk most effectively. Short tools such as Issue Cards , Option The three areas described above – evidence synthesis, patient experience, and patient-facing tool production – have stemmed from independent areas of work. There are, however, signs of increasing overlap between the groups. A joint conference focusing on shared decision making and evidence-based healthcare will be held in Australia in 2015 . Many frProgress will depend on advances in all these areas if we are to ensure that trustworthy evidence can be used collaboratively in clinical encounters , with clCreate trustworthy evidence – in addition to applying new standards and systems for evidence synthesis, further work should focus on optimal presentation and dissemination strategies, and a commitment to create public-facing versions with optimal patient engagement strategiesCreate generic customized care tools – if patients can understand them, so will clinicians, and there is more chance that better conversations will be generated. Busy clinicians value short tools, but will be willing to read longer versions if they need to know more about the underlying evidenceMake the tools easy to find – use search engine optimization methods to ensure the tools are highly ranked in resultsCreate trusted sources – people are becoming more skeptical, we need to create trusted sources of evidence and be transparent about limitations in the quality of the evidenceBring tools to the clinical encounter – innovative presentation formats, based on trustworthy guidelines, designed for use in clinical encounters could support collaboration and deliberation


Myositis ossificans usually occurs in the vicinity of the elbow, knee joints, or hip joints, following obvious trauma or surgery. This is the first report on myositis ossificans of the serratus anterior.In this report we present a case of myositis ossificans within the serratus anterior which developed as a complication due to long-term nape massage. The patient was a 29-year-old Han woman. Because heterotopic ossificans constricted her brachial plexus the surface of her right upper arm was slightly numb; the symptom disappeared after surgery.This case highlights that myositis ossificans can occur in the serratus anterior following long-term nape massage. Myositis ossificans is a heterotopic bone formation within a muscle. It is mainly found in the muscle of extremities, but myositis ossificans of the serratus anterior has never previously been reported. The current report presents the case of a 29-year-old woman with a very rare form of myositis ossificans of the serratus anterior developed due to long-term nape massage.A 29-year-old Han woman presented with a chief complaint of a tumor under her right clavicle for 5 months, and right shoulder soreness and numbness of the exterior of her right upper arm for 10 days. At physical examination there was a red and warm mass over her right shoulder with mild pain and tenderness. The mass was tough, hardly moveable, and base-fixed. The strength of her right upper limb muscles was 4+. A chest X-ray showed a mass with a high density shadow at the superior posterior right clavicle Fig. . An elecMyositis ossificans is characterized by abnormal heterotopic ossification formation, typically involving muscles, tendons, ligaments, fascia, and aponeurosis. It can be divided into three subtypes: traumatic myositis ossificans, nervous myositis ossificans, and progressiva myositis ossificans. Progressiva myositis ossificans is a rare, hereditary, progressive connective tissue disorder, characterized by congenital malformation of the great toes; progressive heterotopic ossification occurs mainly in the neck, chest and back . This paTreatment is usually conservative with analgesics and physical therapy, and excision is considered when excessive pain, joint limitation, or nerve compression is present. Surgery is generally chosen when myositis ossificans is ripe, identified by a higher bone density in X-ray findings and normal red blood cell sedimentation rate and alkaline phosphatase . For ourThis is the first case report on myositis ossificans within the serratus anterior, and it suggests that myositis ossificans can occur in the serratus anterior following long-term nape massage. So people who like massages should be advised to refuse rough massage and to question long-term massage that gives them pain.Written informed consent was obtained from the patient for publication of this case report and accompanying images. A copy of the written consent is available for review by the Editor-in-Chief of this journal.


Application for the sequence { The sinc function is defined to beThis function plays a key role in many areas of mathematics and its applications –6.x = π/2.The following result that provides a lower bound for the sinc is well known as Jordan inequality :(2)sincThis inequality has been further refined by many authors in the past few years –35.In , it was We noticed that the lower bound in  is the fTo the best of the authors' knowledge, few results have been obtained on fractional lower bound for the sinc function. It is the first aim of the present paper to establish the following fractional lower bound for the sinc function.x ∈ , one hasFor any m, the following Carleman type inequality holds:an⩾0, n = 1,2, 3,…, with 0 < ∑n=1∞an < ∞, whereIn , Yang prbn}n=1∞ has very interesting properties. Yang [n,From a mathematical point of view, the sequence {es. Yang  and Gylles. Yang  have proIn , the autbn}n=1∞.As an application of n⩾2, one hasFor any positive integer The proof is not based on . We firsx ∈ ⩾0. Elementary calculations reveal thatt < 1/3, we haveSet equality  is equivquality =t2(−x) and 16π4/(3π2+x2)2, we find thatx2/π2 = t, 0 ⩽ t ⩽ 1. Consider the function f(t) defined byf(0) = 2/9 and f(1) = 0. n⩾4,n⩾4, we havec with  such that g(c) = 0. Using Maple, we find that on the open interval  the equation g(t) = 0 has a unique real root t1 ≈ 0.89⋯.By using the power series expansions of sin(ind that1+sinc(x)Hence, from  we get, a, a31)f(f((31)f = 2/π, H(1) = 1, and H is concave up, it follows thatTaking the natural log giveslnH(x)=(1.Using  from (42ows thatH(x)⩽2(1−


Bronchiolitis is the commonest respiratory infection in children less than 12 months and cause of hospitalisation in infants under 6 months of age in Australasia. Unfortunately there is substantial variation in management, despite high levels of supporting evidence. This paper reports on the process, strengths and challenges of the hybrid approach used to develop the first Australasian management guideline relevant to the local population.An adaption of the nine steps recommended by the National Health and Medical Research Council (NHMRC) and the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology were utilised. Following establishment of the Guideline Development Committee (GDC), we identified the population, intervention, comparator, outcomes and time of interest (PICOt) questions, undertook a systematic literature search and graded the evidence and recommendations using the NHMRC and GRADE processes. Using Nominal Group Techniques (NGT), consensus was sought in formulating the clinical practice recommendations and practice points. Key health professional bodies were consulted to ensure relevance in the Australasian emergency and ward settings.From 33 PICOT questions, clinical recommendations for practice that were deemed relevant to the Australasian population were identified. Specific considerations for the management of Australian and New Zealand indigenous infants in relation to the use of azithromycin and risk factors for more serious illness are included.Using NGT, consensus demonstrated by a median Likert score > 8 for all recommendations was achieved. The guideline presents clinical guidance, followed by the key recommendations and evidence review behind each recommendation.Developing evidence-based clinical guidelines is a complex process with considerable challenges. Challenges included having committee members located over two countries and five time zones, large volume of literature and variation of member’s knowledge of grading of evidence and recommendations. The GRADE and NHMRC processes provided a systematic and transparent approach ensuring a final structure including bedside interface, and a descriptive summary of the evidence base and tables for each key statement. Involvement of stakeholders who will ultimately be end-users as members of the GDC provided valuable knowledge. Lessons learnt during this guideline development process provide valuable insight for those planning development of evidence-based guidelines.The online version of this article (10.1186/s12874-018-0478-x) contains supplementary material, which is available to authorized users. Bronchiolitis, the commonest lower respiratory tract infection in children less than 12 months of age, is the most frequent cause of hospitalisation in infants under six months of age . Groups Bronchiolitis is caused by a viral infection, most commonly respiratory syncytial virus and is characterised by acute inflammation, oedema and necrosis of epithelial cells lining small airways, increased mucus production, and bronchospasm. Signs and symptoms are typically rhinitis, tachypnoea, wheezing, cough, crackles/inspiratory crepitations, use of accessory muscles, and/or nasal flaring , all conEvidence based clinical practice guidelines translate findings from health research into recommendations for clinical practice . GuideliAlthough high quality international clinical practice guidelines for children with bronchiolitis exist for practitioners in the United States  and the At present many Australasian tertiary children’s hospitals and State Government departments have developed and utilise their own local guidelines for the management of bronchiolitis in their setting –19. HoweAn Australasian guideline for the management of bronchiolitis has the potential to standardise care for infants presenting with one of the most common conditions for children admitted to hospital. A guideline can enable professionals from different disciplines to come to an agreement about treatment and devise a quality framework against which practice can be measured, not only for tertiary hospital care but smaller regional, rural and remote health facilities. The development and implementation of an evidence based guideline has the potential to align practice standards across Australasia, allowing benchmarking, and individual institutions to focus on suitable implementation strategies for knowledge translation in the local context. With such standardisation there are expected health benefits as hospitalisation remains the primary determinant of health care expenditure in bronchiolitis . A high-Clinical guidelines can be developed by organisations, individuals, task forces or institutions that have a common aim to identify the current best practice based on the available research evidence in the wider context of patient and clinician experience. Guidelines are rarely based on research evidence alone, but generally also incorporate the consensus view of experts. Rigor and transparency are required in the process selected to develop consensus  and seveThe NGT can involve approximately eight to twelve people who identify the questions to be included in a guideline, express their views in private, and then discuss areas of disagreement. After the discussion the group members again provide their views in private to the steering committee or organisers who analyse the responses to achieve group consensus. This process allows all group members the opportunity to express their views, reduces the risk of misunderstandings whilst identifying the reasons for differences of opinion, and is a structured, transparent, and replicable way of synthesising individual judgments .A Delphi method involves two or more rounds of postal or electronic questionnaires to obtain consensus or expert opinion. This method allows involvement of a larger and more geographically dispersed group of participants and avoids the risk of some individuals exercising undue influence . The anoHybrid approaches occur where a combination of questionnaire and face-to-face meetings are utilised to reach consensus. This method can allow for greater inclusion of stakeholder viewpoints  but can A key determinant of an evidence based guideline is the methodological approach used to assess the quality of the research reviewed. Assessment of quality directly guides the strength of the final recommendations for practice. There are several internationally accepted approaches for assessing and grading quality of evidence. Examples of these processes are the Grading of Recommendations Assessment, Development and Evaluation (GRADE), National Health and Medical Research Council (NHMRC) Evaluation of Evidence process, Strength of Recommendation Taxonomy (SORT) and the Scottish Intercollegiate Guideline Network (SIGN) . GradingThe GRADE method entails an assessment of the quality of a body of evidence for each individual outcome to a given clinical question, including consideration of within-study risk of bias , the directness of the evidence, heterogeneity, the precision of effect estimates and the risk of publication bias . The GRAThe NHMRC Evaluation of Evidence process is Australian and addresses the evidence to support clinical questions such as those relating to interventions, diagnosis, prognosis, aetiology and screening which are specifically related to guideline development . The NHMThe SIGN process encompasses a system that maintains the link between the strength of the available evidence and the grade of the recommendation. The levels of evidence are based on study design and methodological quality of individual studies. Utilising the SIGN process, after synthesis of the evidence base, the Guideline Development Committee must make a subjective judgement on the recommendations. This judgement is based on their clinical experience in addition to their knowledge of the evidence base .Shukla et al.  reviewedThis paper reports on the methodology utilised to develop a specific guideline for Australasian clinical practice; assessment of evidence using the NHMRC and GRADE processes along with a hybrid consensus approach to develop clinical recommendations in a final clinical guideline for the management of bronchiolitis in the Australasian setting. The strengths and challenges encountered are presented.A structured guideline development process was utilised see Fig. . The aimThe principal target group to utilise the guideline will be staff of, and health systems supporting, Australasian EDs and general paediatric wards. The guideline will be structured as a useable clinical interface with bed-side functionality, including flow diagrams and tables of key points. A descriptive summary of the evidence base and evidence based tables from the GRADE and NHMRC evidence review processes will sit behind the useable clinical interface in the final document.A Guideline Advisory Group (GAG) was formed which consisted of the project chief investigators; three paediatric emergency physicians, one paediatrician and the project coordinator. The role of this group was to provide expert advice and contribute to the guideline development process including construction of population, intervention, comparator, outcomes and time of interest (PICOt) questions, defining the guideline scope and target audience, and oversight of the project.Further, a multidisciplinary Guideline Development Committee (GDC) which included the members of the GAG, was convened in accordance with the NHMRC recommendations for guideline development . This coThe GAG initially identified 26 key PICOt questions and relevant topics to be included based on the American Academy of Paediatrics (AAP) 2014 bronchiolitis statement . A face-Population: infants under 24 months of age with bronchiolitis.Interventions: diagnostic tests and investigations, oxygen therapies, medications, rehydration, scoring systems and others that met individual research questions Table Comparators: no treatment as a control, standard care or treatment as usual or an alternative intervention; comparisons between different modes of delivery, frequency, dose or duration of interventions were included.Outcomes: length of stay, admission, readmission, death.Time: 1 January 2000 to 17 December 2015. Given the GAGs’ extensive experience and previous knowledge of the field, a fifteen year search range was thought to be appropriate, realising that sentinel work that had occurred prior to this would have been included in relevant Cochrane reviews and other meta-analysis.Language: English language.A systematic literature search for all 33 questions was undertaken. The search strategy inclusion criteria included:The complete search strategy is included as Additional file Exclusion Criteria included publications of case reports, commentaries, editorials and letters, and those of infants with bronchiolitis obliterans.The following electronic databases were searched: Ovid Medline, Ovid Embase, PubMed, Cinahl (EBSCO), Cochrane Library, Cochrane Database of Systematic Reviews (CDSR) and Cochrane Central Register of Controlled Trials (CENTRAL) for systematic reviews and randomised controlled trials (RCTs). Where systematic reviews and RCTs were not found, quasi-experimental and non-experimental observational studies (e.g. case control studies and cohort studies) were retrieved. If a systematic review published in the Cochrane Library was relevant to a question, only systematic reviews and RCTs from the year of, and years subsequent to, the documented search date in the Cochrane systematic review were included.The study selection process was conducted in a three step process:Step 1 – One of five members of the GAG reviewed the title and abstracts of the articles identified in the literature search. Articles that met the inclusion criteria and were relevant to any of the 33 questions were included.Step 2 – Where screening by title and abstract was insufficient to make a decision as to relevance, a copy of the complete article was sourced and reviewed in order to make a decision regarding possible relevance. If further uncertainty existed regarding possible relevance the article was reviewed by a second person. Where there was disagreement between reviewers or uncertainty final inclusion or exclusion was decided upon by the five person GAG.See Additional file Step 3 – Included articles were then divided into groups according to each question for which they may be relevant. Where an article was relevant to more than one question, it was placed into each appropriate group.Members of the GDC were divided into four working groups according to expertise to allow more effective interaction via teleconference. The 33 questions were divided into areas of clinical similarities and allocated to a single group. Each article that was potentially relevant to an individual question was independently reviewed by two members for quality and the level of evidence utilising both the GRADE  and NHMROn completion of the evidence grading, recommendations were reviewed within working groups via teleconference to be included in a draft guideline. In areas with a poor evidence base, the working groups discussed the evidence base and current clinical practice and made clinical practice recommendations and practice points for a draft guideline. Alongside evidence tables, summaries of evidence were prepared for each question with review by the working groups and the GDC. Where possible, the evidence was based on systematic reviews and randomised controlled trials. Where there was only low level, unsupported evidence, clinical care statements outlining current accepted practice points were included.Draft evidence tables and recommendations were shared electronically with all members of the GDC. A face-to-face meeting was then conducted where each recommendation was presented and all GDC members given the opportunity to present their view without interruption. Discussion ensued and any alterations were made before GDC members were requested to vote. To ascertain consensus from the multidisciplinary GDC, NGT principles  were useA draft guideline was produced including flow diagrams of patient care for infants with bronchiolitis, and highlighting key assessment and treatment points throughout the patient journey from ED presentation to ward admission, inpatient stay and discharge. The structure for the draft guideline was agreed by all members of the GDC.The final step was to undertake stakeholder/knowledge user consultation. Formal feedback was sought from Australian and New Zealand professional medical and nursing colleges , Royal Australasian College of Physicians (RACP), New Zealand Emergency Medicine Network, Australian College of Children and Young Peoples Nurses (ACCYPN), New Zealand Nursing Council and Children’s Healthcare Australasia (CHA), in addition to the clinical leads of general paediatrics and ED’s at Australian and New Zealand tertiary paediatric hospitals and consumers . All feedback was reviewed by the GAG and changes and corrections were made if deemed appropriate. This review was the final consultation between researchers and clinicians (knowledge users) to ensure relevance of the final guideline to the Australasian emergency and paediatric ward setting.n = 20) resulted in achievement of a median Likert score > 8 for all recommendations. The guideline is presented in such a way that the important clinical guidance is at the front, followed by the key recommendations and the evidence review behind each of these recommendations.Utilising the GRADE and NRMRC processes for assessing evidence and a hybrid NGT method to achieve consensus recommendations, an evidence based Australasian Guideline for the management of Bronchiolitis in infants was developed. From the 33 PICOT questions, clinical recommendations for practice that were deemed relevant to the Australasian population were identified and are included in the guideline. Specific considerations for the management of Australian and New Zealand indigenous infants in relation to the use of azithromycin and risk factors for more serious illness are included. Utilising the NGT method to ascertain consensus amongst the GDC  with a descriptive summary of the evidence base and evidence tables (NHMRC and GRADE) for each key statement. This clinical practice guideline provides an opportunity to close the gaps between current clinical practice and the best available evidence. An implementation strategy incorporating a knowledge translation project is planned to be conducted by the PREDICT network as the next step towards consistent, evidence informed practice.Additional file 1:Australasian Bronchiolitis Guideline Literature search strategy. Literature search strategy used to identify relevant articles for the guideline (DOCX 20 kb)Additional file 2:Australasian Bronchiolitis Guideline Prisma Diagram. Prisma flow diagram of search strategy (DOCX 29 kb)


This systematic review assessed the effectiveness of capacity building interventions relevant to public health practice. The aim is to inform and improve capacity building interventions.Four strategies were used: 1) electronic database searching; 2) reference lists of included papers; 3) key informant consultation; and 4) grey literature searching. Inclusion  and exclusion criteria  are outlined with included papers focusing on capacity building, learning plans, or professional development plans within public health and related settings, such as non-governmental organizations, government, or community-based organizations relating to public health or healthcare. Outcomes of interest included changes in knowledge, skill or confidence (self-efficacy), changes in practice (application or intent), and perceived support or supportive environments, with outcomes reported at the individual, organizational or systems level(s). Quality assessment of all included papers was completed.Fourteen papers were included in this review. These papers reported on six intervention types: 1) internet-based instruction, 2) training and workshops, 3) technical assistance, 4) education using self-directed learning, 5) communities of practice, and 6) multi-strategy interventions. The available literature showed improvements in one or more capacity-building outcomes of interest, mainly in terms of individual-level outcomes. The available literature was moderate in quality and showed a range of methodological issues.There is evidence to inform capacity building programming and how interventions can be selected to optimize impact. Organizations should carefully consider methods for analysis of capacity building interventions offered; specifically, through which mechanisms, to whom, and for which purpose. Capacity-building interventions can enhance knowledge, skill, self-efficacy (including confidence), changes in practice or policies, behaviour change, application, and system-level capacity. However in applying available evidence, organizations should consider the outcomes of highest priority, selecting intervention(s) effective for the outcome(s) of interest. Examples are given for selecting intervention(s) to match priorities and context, knowing effectiveness evidence is only one consideration in decision making. Future evaluations should: extend beyond the individual level, assess outcomes at organizational and systems levels, include objective measures of effect, assess baseline conditions, and evaluate features most critical to the success of interventions.The online version of this article (10.1186/s12889-018-5591-6) contains supplementary material, which is available to authorized users. Fast-paced globalization of health challenges the capacity of the public health field to not only adopt evidence, but also to adapt it locally to keep pace with public health events , 2. GlobDespite demands placed on public health to meet new and future challenges, skill deficits in the public health workforce are evident and include insufficient preparation via education and training for the jobs performed, and an overreliance on experience and on-the-job trial and error . Self asGiven the deficits reported, there is a need for capacity building. Capacity building is defined by the World Health Organization (WHO) as “the development of knowledge, skills, commitment, structures, systems and leadership to enable effective health promotion...[with] actions to improve health at three levels: the advancement of knowledge and skills among practitioners; the expansion of support and infrastructure for health promotion in organizations, and; the development of cohesiveness and partnerships for health in communities” . EngaginEfforts to foster public health capacity have included the development of core competencies, credentialing, accreditation, and a number of policy recommendations, which include the systematic assessment and development of the public health workforce. , 16–20 DThere appears to be limited research on the effectiveness of capacity building interventions related to individual and organizational capacity building in public health. One systematic review was found that explored community-based interventions; however, this review was limited to community-based interventions and studies published in academic journals, with included papers not quality appraised . To our The purpose of this systematic review is to assess the effectiveness of capacity building interventions relevant to public health practice. The aim is to inform and improve capacity building interventions.A framework for completing the review was constructed in advance and included the following steps: 1) search strategy, 2) determining inclusion and exclusion criteria, 3) screening papers, 4) methodological quality assessment of primary studies or reviews, followed by 5) data extraction and 6) synthesis of results . A protoFour strategies were used: 1) electronic database searching; 2) reference lists of included papers; 3) key informant consultation; and 4) grey literature searching. A systematic electronic search was conducted by Public Health Ontario (PHO) Library Services on September 29, 2015 and updated on September 29, 2016 in four databases: 1) Ovid MEDLINE, 2) Embase, 3) CINAHL Plus with Full Text, and 4) PsycINFO. The search aimed to locate articles on the impact of capacity building in public health and general healthcare and included “Capacity building” [MeSH] as well as keywords “capacity building”, “prevention capacity”, “health promotion or public health”, “build or increase capacity” and “learning plan”. See Additional file Articles and reports were included if they were published in English over the last 11 years, were about capacity building interventions such as learning plans, or professional development plans within public health or related settings , offered summary-level evidence (guidelines), synthesis-level research or primary studies that included evaluations of capacity building interventions, and included outcome data related to effectiveness which could be organized and analyzed according to intervention type . Outcomes of interest included changes in knowledge, skill or confidence (self-efficacy), changes in practice (application or intent), and perceived support or supportive environments, with outcomes reported at the individual, organizational or systems level(s). Outcomes measures were determined by the authors prior to the review work, and reflected relevant measures for the health promotion and public health system. Commentaries were eligible for inclusion if they reported data on outcomes.Exclusion criteria included non-English language papers published earlier than 2005, articles focusing on curriculum development in academic settings (university research centres and departments), acute care facilities, hospitals, long-term care centres, agricultural agencies, afterschool agencies, community development and community capacity building interventions, articles that did not provide outcome data, and interventions in low resource and/or low-to-middle income countries related to obtaining credits for continuing professional education or medical education, or capacity building .Titles and abstracts of all identified articles in the original 2015 search were screened by two review authors (KD and KB), who independently screened 20% of the search results for relevance and had an agreement score greater than 80%. The remaining 80% of results were split in half and independently screened (KD and KB). Two team members (KD and GM) independently screened the full set of the updated search conducted in 2016. At full-text relevance screening, two authors (KD and GM) independently screened 20% of all full-texts, and the remaining 80% was screened by one author (GM). Any discrepancies were resolved by discussion until consensus was reached, and occasionally a third reviewer from the team, or the full research team, was consulted. The reference lists of all relevant articles were searched to identify additional articles. Any potentially-relevant additional papers were retrieved in full document versions and screened for inclusion.Titles and abstracts of all grey literature search results were screened by one author (KD). All sources identified through key informant consultation were screened for relevance by one author (GM). Full-text assessment was independently carried out by two authors (KD and GM). Consensus was reached on all discrepancies via discussion between the two authors.Methodological quality was independently assessed by two authors (KD and GM) using criteria appropriate to the design of the included papers. Specifically, the Health Evidence Quality Assessment Tool for systematic reviews and meta-analyses was used for systematic review and/or meta-analysis articles . ArticleA data extraction table was drafted to meet the purpose of the project, and refined by discussion among the research team. The form was then piloted and discussed by two authors . The resulting discussion generated a guide for data extraction with instructions for completing each item, to achieve consistency. Data extraction was then completed by one author (GM) and reviewed by a second author (KD) to ensure that appropriate data were extracted from all included papers. Authors were not contacted to validate data published in the included papers. Information extracted from each paper included: author and year of publication, purpose or objective, type of intervention, population and setting, providers, study design, context, theories and frameworks cited, theories and frameworks used to develop interventions, measurement tool, outcomes, findings/results, implications for practice, and study limitations. Findings included a range of outcome measures determined to be relevant to the health promotion and public health capacity building field and are outlined with their methods of measurement in Table We conducted a systematic review that focused on the effectiveness of all included papers relevant to the impact of interventions for capacity building. For the purpose of this paper, a systematic review is defined as an evidence synthesis that adheres to guidelines on the conduct of a review . Our revA total of 38 papers were assessed as relevant and then appraised for methodological quality, with 24 scoring low in quality and being excluded from the synthesis due to lack of quality. The remaining 14 papers were included. For appraising quality, an appropriate tool was selected for each of the research designs present. Tools were selected for systematic review quality appraisal , quantitative studies, qualitative studies, and mixed methods designs. Please see the PRISMA flowchart , staff at community-based and national organizations, community health workers, public health staff/community health staff, some education sector staff , and national health service staff. Providers for these interventions included expert consultants, evaluators (manager or leader), clients, multidisciplinary teams, health professionals , resource centre staff, public health staff, researchers or research centres, academic institution representatives and community health centre staff, and in two papers, providers were unspecified.The types of research designs used to evaluate capacity building interventions include: systematic review , mixed methods, qualitative, and quantitative methods, represented by simple before-after evaluations. The range of outcomes and measurement methods for the included studies are shown in Table In terms of quality of the available literature (Appendix C), review-level papers  were well-done with no serious flaws identified after quality appraisal. Quantitative evaluations were moderately well-done, with none rated high in quality. Quantitative papers’ flaws included a general lack of controls, failure to measure at multiple time points (simple pre-post), failure to use reliable or valid tools to measure, use of self-report rather than objective measurement of outcomes, and lack of rationale for drop outs from their projects.All included papers that were qualitative in nature rated moderate in quality, and none rated high in quality. Qualitative papers’ flaws typically were that they did not explain rationale for selection of a particular method, lacked an audit trail to support understanding what was done to obtain results, lacked reflection/explanation about the possible impact of the researcher or person doing the intervention/data collection on the data itself and on the participants/intervention, and failed to consistently address ethical concerns. Mixed methods papers featured a combination of the flaws identified in both the quantitative and qualitative literature separately.As part of data extraction, we recorded whether authors cited a particular theory or framework related to their research. Notably, only four of 14 papers cited a particular framework, with no overlap in theories/frameworks cited across the four papers –34. HoweThe following sections report outcomes organized by each of the six intervention types noted in the currently available literature: 1) internet-based instruction, 2) training and workshops, 3) technical assistance, 4) education using self-directed learning (SDL), 5) communities of practice (CoP), and 6) multi-strategy interventions. Each intervention type reports a description of the intervention results for that intervention type by each outcome.A well-done meta-analysis by Cook et al., incorporated 201 individual studies that assessed 214 separate interventions . The revp < 0.001) [p = 0.002), and for longer courses (p = 0.03) [Compared to no intervention, internet-based instruction showed a large effect on knowledge  . In term = 0.03) . Level op < 0.05) [Compared to other forms of instruction , internet-based instruction offered a statistically significant benefit but the effect was quite small  . Thus ovp < 0.001). Practice exercises seemed to enhance the effect of internet-based instruction on skills, while interactivity, repetition and discussion did not influence outcomes.Similar to the impact of internet-based instruction on knowledge, compared to no intervention, there was a large effect on skills as an outcome  [Also aligned with results related to knowledge, internet-based instruction showed no benefit to skill development when compared to other forms of intervention  .Although curricula were not included and analyzed as part of this review, we were interested in education programs that used self-directed learning (SDL). A single, well-done meta-analysis reviewed SDL for capacity building . The intMurad’s meta-analysis showed a moderate, significant increase in knowledge  using self-directed learning, compared to traditional teaching methods and no significant skill increases for SDL strategies over traditional teaching methods . The revFor the purpose of analysis, training and workshops were grouped together since their definitions in the literature didn’t allow for distinctions between interventions labelled ‘training’ and what was considered ‘workshops’.Seven papers reported training and workshop outcomes that included: knowledge, understanding, skill, confidence (self-efficacy), changes in practice and policy, behaviour change, and application of knowledge , 37–40. p value. However, knowledge ‘scores’ were considered high if 8 or greater on an eleven-point Likert scale [p = 0.986) [Four papers assessed knowledge after training and workshops , 37, 39.rt scale . Conflic= 0.986) , 39. Bot= 0.986) , 34, 37.A single paper reported increased understanding after a second of two training sessions, with the impact sustained after 5 months post-intervention; however, there were significant differences between groups at baseline . These dp = 0.003), critically reading research literature (p = 0.047), developing a research study (p = 0.008), developing a data collection tool (p = 0.033); [p = 0.0001) [p = 0.0001) [Three papers reported consistently positive, quantitative skill gains for training and workshops , 34, 39.increased confidence and comfort with their content area  ,increased confidence and motivation to include  promotion activities within their roles ,confidence in their ability to use skills, intentions related to applying learning and changing their approach , andgreater confidence to use skills and feelings of empowerment and motivation .All three papers that evaluated this outcome reported positive gains in confidence/self-efficacy, with all three reporting qualitative findings and two also reporting quantitative findings, with findings from each of the two methods validating each other. Descriptive  findings showed increased confidence reported by participants as:Ruiz reported a 23% increase in confidence , with Kep values, confidence intervals) to assess whether differences observed were significant; however, engagement across all activities showed increases. For the purpose of highlighting potentially-meaningful gains, we note that four areas showed large gains of 30% or greater, in terms of the proportion of participants active in capacity-building practices at the individual, organizational, and inter-organizational levels after the training compared to prior [A single study by Keogh reported on practice and policy change with both quantitative and qualitative outcomes with quantitative change assessed as percentage change . Both itto prior . These wto prior . Qualitato prior . Though A single study reported behaviour change as an outcome evaluated qualitatively via questionnaire  . ParticiApplication of knowledge from capacity building training and workshops was evaluated by two studies that reported consistent positive results. Qualitative findings showed community health workers immediately applied knowledge from training by educating family, community, and other community health workers . AdditioTwo papers qualitatively evaluated technical assistance (TA), with both reporting individual-level knowledge and skill, leadership, and confidence , 42. BotIn both settings, TA clients reported knowledge gains relevant to their content area, including the development of new skills such as enhanced communication and presentation skills, overcoming barriers at council meetings, and media and communication , 42.Both Kegler and Lambracki reported increased confidence in one’s own work, with Lambracki also reporting commitment to take on new challenges and overcome barriers , 42. ForIn the area of practice and policy change, TA resulted in service improvement, foundational work for policy action, increased use of evidence, and enhanced collaboration (via development of partnerships and network building) . These fAt an organizational level, TA clients reported neutral or mixed feelings about the impact of TA services on organizational support . HoweverA single paper reported outcomes for communities of practice (CoP) . Althougp < 0.00 for all four knowledge statements evaluated) [p < 0.00 and p < 0.002 across three statements), and action statements related to knowledge about their own practices (p < 0.00 for both statements) from pre to post test [Capacity building being perceived as meaningful and enjoyable Changes in thinking because of new knowledge in mental health Experience evoking strong emotions related to practitioners’ own identities Reported practice changes  Bazyk reported significant knowledge gains measured quantitatively via pre and post-test results, finding that CoP participation increased knowledge statements  . Althougost test . KnowledThis mixed-methods study reported changes at post-test and did not report whether knowledge gains and reported practice changes were sustained .Two papers evaluated multi-strategy interventions , 43. Forp < 0.01), and individual trauma knowledge and practice also improved significantly  [Knowledge improved via multi-component, capacity building interventions , 43. Pre < 0.01) .In addition to knowledge, skills were also enhanced by multi-component interventions . All parp < 0.01), trauma supervision and supports , access to trauma-informed services , and local agency collaboration (trauma)  [Lang evaluated a system-level, multi-component intervention delivered over 2 years and aimed at generating system-level change in evidence-informed trauma care . Pre-pos < 0.01) . There w < 0.01) . AlthougThis systematic review assessed public health capacity building interventions for their relative impact on knowledge, skill, self-efficacy (including confidence), changes in practice or policies, behaviour change, application, organizational support, and perception of system-level capacity, to inform capacity building practices and services at multiple levels. Given the time frame over which capacity building interventions have been in place (and capacity building defined), the evaluations available in the current literature were surprisingly sparse. More consistent evaluation of the interventions would offer a firm foundation upon which capacity-building organizations can base decisions about how they deliver services.For the evaluations reported in the current literature, methodological quality was low to moderate, without high-quality in any of the four research designs located and appraised, with the exception of the meta-analysis by Cook et al. . Given hThe range of evaluated interventions represents the scope of what is commonly referred to as capacity building in a public health context. However, the papers remain focused on capacity building at the individual level, despite acknowledgement that individual behaviour is shaped by interaction with the environment at multiple levels , and calEach of the six interventions included in the current literature found improvements in one or more capacity-building outcomes of interest. A large, well-done meta-analysis found that internet-based capacity building enhanced both knowledge and skill, but only in comparison to no other intervention . ImportaSelf-directed learning was also evaluated via meta-analysis, and enhanced knowledge but made no significant difference in skill. Self-directed learning may prove an effective strategy where knowledge alone is the outcome of interest, but where skill gains are expected, programs should consider alternate strategies. That SDL could impact knowledge in a significant way is promising since it represents a first step or initial outcome that could be built upon in a phased approach or in a series of interventions to advance learners along a continuum, potentially reaching behaviour change and practice and policy change at later stages. It also offers the advantage of requiring fewer resources than more intensive interventions like technical assistance.Training and workshops were evaluated by seven included studies that showed enhancements across almost all outcomes including knowledge, skill, confidence, improvements in practice and policy, behaviour change, and application of knowledge , 37–40. Technical assistance was defined consistently by two studies that evaluated its impact , 42. A cThe potential of TA to impact actual practice and policy change is meaningful in that it represents an outcome beyond the individual level. Practice and policy change was also affected by training and workshops, so the features and potential complementary nature of the two strategies could be considered where practice and policy change are goals.Multi-component interventions were consistent in enhancing both knowledge and skill; however, there was limited evidence, and the evidence did not report on which key elements contributed most to their observed impacts. It is unknown which components of multi-strategy interventions most influenced their impact. Multi-strategy interventions should offer some assessment of which components were critical, and which combination of interventions is optimal. Those ideas were not explored in the available literature, but would contribute to awareness of which combination of strategies would work best together to impact which outcomes, and at which level(s).Findings from this review support the effectiveness of capacity-building interventions to impact knowledge, skill, self-efficacy (including confidence), changes in practice or policies, behaviour change, application, and perceptions of system-level capacity. However, this support exists mainly at the individual level. Despite lack of evidence for capacity building impact beyond the individual level; impacts on behaviour change, practice and policy change, and application are promising.There may be other relevant documents beyond published articles and grey literature searches, which are not available in the public domain and/or there may be articles published in another language on this topic. This systematic review’s limitations relate mainly to the state of the current, available literature. The literature was searched for English language papers only. Limitations of the literature at a study level include the quality of the available literature overall and the fact that evaluation methods used were not strong, with some consistent methodological issues: the extent to which measurement tools were reliable and valid, and data consistently relying on self report. The literature is not clear about the role of public health staff in capacity building, with a lack of clarity on providers. There was consistently greater focus on describing the target audience in setting the approach, with less information about the role of capacity-building providers.A limitation at the review level was that the sectors from which evidence was drawn were limited to those related to public health and health promotion specifically. Having reviewed the evidence from the health sectors only, our search strategy could have missed literature in related fields such as implementation science, secondary education, and knowledge utilization as examples. However, this review’s focus was on the public health context, and this review established the state of the literature nearest to that context, based on the most relevant and current papers.Strengths of this review include the author’s collective experience working in a capacity building organization, the use of a comprehensive search strategies  and assessment of the quality of included articles.Although data gaps exist and most evaluation rests at the individual level, the public health field must still take action. We should apply the available effectiveness evidence to programming and planning moving forward, and continue to advance the evaluation related to capacity building.Based on current evidence about what works, capacity building programming should consider which outcomes are of highest priority. Priority setting done by a working group or other body, or determined via prioritization of audience needs could be useful in seeing which interventions could offer the best fit. High-priority outcomes could be aligned with intervention strategies that offer the best chance of achieving those outcomes. An assessment of learning objectives that aligned outcomes with the most effective intervention(s) and consideration of context may also be useful. For example, if delivering interventions in remote regions or where there aren’t currently options available, internet-based interventions may work well and be the most feasible. In contrast, where a host of interventions are available, other factors such as outcome goals matched with the evidence on what works may be weighted more heavily. The evidence also provides direction to optimize delivery of interventions; for example, when applying SDL, involving learners in their own learning resource selection results in greater improvements, as well as targeting those who are more advanced in learning. Thus, SDL may be particularly appropriate, and feasible, for capacity building beyond introductory level material.Challenges related to budgets and funding commitments, available resources, and other organizational considerations will undoubtedly influence how capacity building is done and evaluated. Interventions themselves can require a high volume of resources, particularly where the intervention involves web development, moderation for discussion groups, or technical assistance as an example of a more personalized (and in-person) approach to capacity building. Where outcomes of interest can be achieved equally well via a less intense option, such as training and workshops, the latter option may be more viable.Opportunities for long-term planning lie in combining the practical considerations that accompany the effectiveness evidence in selecting the most relevant, appropriate interventions, and then determining whether the desired impact is achieved and sustainable, to contribute to what is known about how capacity building interventions work. Given certain outcomes are reported mainly at the individual level, once an intervention is selected, those evaluating capacity building interventions should work to further advance the evidence. The current literature can help by offering strategies for conceptualizing measures beyond the individual level. Organization and systems-level measurement strategies such as those used to operationalize Lang’s system capacity and readiness offer an example . WorkingOngoing training and multi-level strategies are needed in developing and delivering programs, and to adapt and measure effectiveness of their interventions . This meWhile a range of capacity building interventions have been set in place and have become standard, there is less evaluation of which intervention work best in a public health or health promotion context. It is unclear based on currently-available literature how capacity building interventions are selected and applied, and where they are used in the absence of evaluation data to show they work. Organizations should carefully consider methods for analysis of capacity building interventions being offered, through which mechanisms they work best, for whom, and for what purpose.Regardless of the need for enhanced evaluation and consideration of effectiveness evidence in implementation, there is clear advantage to particular interventions. Capacity building does enhance outcomes related to the work of public health. In future it will be important to assess how these can be influenced at different levels beyond that of the individual and considering how strategies work for entire organizations and systems that operate in different contexts.Additional file 1:Appendix A and B. Search strategy examples. Full Medline search strategy and a general web search for grey literature. (DOCX 52 kb)Additional file 2:Appendix C. Results of quality assessment of the included 14 papers with final ratings. (DOCX 26 kb)Additional file 3:Table S1. (Characteristics of included papers) to be linked approximately here (DOCX 37 kb)Additional file 4:Table S2. (Intervention types and outcomes) to be linked approximately here (DOCX 51 kb)


Currently, little is known about the types of evidence used by policy makers. This study aimed to investigate how policy makers in the health domain use and evaluate evidence and how this differs from academic epidemiologists. By having a better understanding of how policy makers select, evaluate, and use evidence, academics can tailor the way in which that evidence is produced, potentially leading to more effective knowledge translation.via an anonymous online survey (n = 28), with sampling from three health-related government and non-government organizations. Semi-structured interviews with policy makers (n = 20) and epidemiologists (n = 6) were conducted to gather qualitative data.An exploratory mixed-methods study design was used. Quantitative measures were collected Policy makers indicated systematic reviews were the preferred research resource (19%), followed closely by qualitative research (16%). Neither policy makers nor epidemiologists used grading instruments to evaluate evidence. In the web survey, policy makers reported that consistency and strength of evidence (93%), the quality of data (93%), bias in the evidence (79%), and recency of evidence (79%) were the most important factors taken into consideration when evaluating the available evidence. The same results were found in the qualitative interviews. Epidemiologists focused on the methodology used in the study. The most cited barriers to using robust evidence, according to policy makers, were political considerations (60%), time limitations (55%), funding (50%), and research not being applicable to current policies (50%).The policy maker’s investigation did not report a systematic approach to evaluating evidence. Although there was some overlap between what policy makers and epidemiologists identified as high-quality evidence, there was also some important differences. This suggests that the best scientific evidence may not routinely be used in the development of policy. In essence, the policy-making process relied on other jurisdictions’ policies and the opinions of internal staff members as primary evidence sources to inform policy decisions. Findings of this study suggest that efforts should be directed toward making scientific information more systematically available to policy makers. There has been increasing discussion that in order to improve public health outcomes quality scientific research should be used throughout the development of health policies . The proIn the “real world” of policy making, scientific research is just one of many types of information used . Policy Previous research has focused on the apparently low uptake of academic research by policy makers, with particular attention given to understanding how and under what circumstances policy makers access and use academic evidence . HoweverCurrently, little is known about what types of information and evidence is normally used as part of the policy development process or the extent to which political agendas and budgetary constraints influence the design and choice of policy options , 9, 14. By investigating the type of research that policy makers use to inform policy decisions, how they identify evidence and what other factors may influence policy decisions, we can identify what information is viewed as more relevant and timely . This maThere is also a need to investigate how policy makers select and evaluate the quality of evidence. One way of selecting and evaluating evidence is by using an “evidence hierarchy.” This hierarchy may consider certain types of experimental research, for example randomized controlled trials (RCTs) and systematic reviews of RCTs, as highest in methodological quality . ResearcAlthough the use of evidence hierarchy and grading systems may provide an easier, or at least more streamlined, way of identifying high-quality evidence, in many situations RCTs may not be the most appropriate research methodology to answer specific policy questions, particularly in the sphere of public health. For example, findings from RCTs do not usually take into account the political, social, or economic context –20. RCTsEpidemiological data and research is typically valued highly as “objective” or “hard” data compared to qualitative data or case studies . FindingGiven the importance of promptly incorporating new and robust scientific evidence into policy and the barriers to KT identified above, there is an urgent need to better understand how policy makers evaluate and use evidence. Therefore, the current study had two intersecting aims. The first aim was to gain an understanding of the role of research evidence in policy making. The second was to investigate how policy makers in the health domain select this evidence and whether they systematically assess evidence quality and how this differs from academic epidemiologists.n = 13 and 6) and one quantitative survey (n = 28). Both interviews and survey are included in Supplementary Material. Written informed consent was obtained from all participants prior to involvement in the study. The Australian National University Human Research Ethics Committee (HREC) and the Australian Capital Territory (ACT) Health HREC Survey Resource Group approved the study.An exploratory mixed-methods study design was used in order to provide a deeper understanding. The design involved the collection and data analysis of two sets of qualitative interviews  how policy makers locate and use evidence from observational and other research; (2) factors influencing their choice of evidence sources; (3) how policy makers deal with conflicting evidence from specific topics; (4) how policy makers evaluate the quality of research; and (5) how policy makers view researchers. The interviews also sought information on perceived barriers to KT. The interview questions were developed in consultation with research experts and senior staff from the health department, Alzheimer’s Australia, and NHMRC. Interviews were recorded and transcribed. Interviews were done one-on-one and took approximately 1 h.The transcribed interviews were uploaded into NVivo  and themThe second set of interviews focused on a purposive sample of chronic disease experts, known to the authors, who were approached to provide their views regarding the characteristics of high-quality observational research, their opinion about the currently available evidence rating systems, and the implications of grading observational research.Participants were asked to answer seven open-ended questions in paper form seeking their views on: (1) their opinion of what constitutes high-quality observational research and how it compares with other types of research; (2) their understanding and use of current rating systems for grading evidence; and (3) the consequences of inappropriately rating observational research. Participants were provided with a one-page guide to grading instruments to clarify what the authors defined as a grading instrument .Thematic analysis using a step by step process was conducted to analyze the interviews. The interview transcripts were repeatedly screened in order to identify a list of recurring themes that appeared critical to evaluating evidence. These themes included evidence sources, choice of evidence sources, evaluating evidence, grading evidence, and observational research. Average and percentage calculations were also applied.via email to complete the survey. The selection of participation was not reliant on age, gender, or policy experience; however, the survey was only distributed to staff who were not involved in the qualitative interviews. Participants were not offered any incentives for completing the survey. The survey was accessible to participants for 6 months. In the time the survey was accessible, 58 participants began the survey, but only 28 participants provided responses to all questions.An anonymous survey was compiled using the online survey tool Qualtrics . Senior The focus of the survey was barriers to knowledge-uptake, knowledge needs at the time of the survey, and the accessibility of information. The survey comprised 18 questions. Of these six were multiple-choice, five were rating scales, six were open-ended, and one was a combination of multiple-choice and open ended . The questions in the survey were developed after examination of surveys that had been used in related studies, consideration of results of the qualitative interviews, and after consultation with collaborators from the University, the Health Department and Alzheimer’s Australia.Participants with missing data were excluded at the item level. Data were analyzed using Microsoft Excel. Descriptive statistics and bar graphs were used to illustrate response patterns to survey items.All but one participant (95%) reported that policies and program decisions were often based on work, including current programs or policies, which had been done in other jurisdictions or by other organizations that were presumed to have better resources for seeking evidence . In this context, respondents reported that greater emphasis was placed on the experience of running the program or implementing the policy than on the evidence base behind it, which was typically not systematically checked. As an example, one participant noted a program implemented in another state that was “taken up” and resulted in a lot of problems. Subsequent contact with those who had set up the original program revealed that they too had had a lot of problems but had not reported them.No respondents identified a systematic approach to gathering evidence for policy. Fourteen participants (70%) mentioned that part of their research strategy included talking to experts, including academics and consultants. Eleven participants (55%) gained most of their information from consumer input and subscribed to publications by the Australian Healthcare and Hospitals Association. Academic journals and Institutional research/library services were only used by five participants (25%).the politicians are wanting to say ‘we’ve made a decision, this is what we’re going to do’… and if there are votes in it [the politicians] will do it regardless of the evidence” . Eleven participants (55%) also cited that consumer or community views were another policy driver.Twelve participants (60%) mentioned politics or political agenda as a significant contributor to the policy formation process. The political agenda may drive what research is used and is not, regardless of the quality of the research. For example, one participant said “… I think it’s a bit of an issue that we’ve seen in terms of being able to identify well what is good evidence, what’s real evidence, what evidence should you use for a policy … what evidence should you be using to back that up. Don’t just go to a website and copy something – that happens, you know, which is not very good but it happens” . Policy makers identified the following as the most common factors which affect evidence choice: the type of evidence (60%), the reputation of the evidence source (55%), quality of the evidence (45%), and local applicability (40%).Eight participants (40%) discussed that there was not a great understanding of what constitutes good or strong evidence. For example, one participant said “it’s hard to find any RCTs for the issues we’re after and whether they’re appropriate anyway in some contexts … in terms of policy what’s really good is a Cochrane review or something that’s looked at a bunch of things across everywhere and synthesized it and so then you can look at what the general opinion or picture looks like” .Only three participants (15%) knew of grading systems and they did not use grading systems to evaluate evidence. Two of these participants discussed the mismatch between grading systems and policy, with RCTs not necessarily being applicable in the policy decision-making, but rather social research being more likely to inform a policy decision. One of the participants highlighted this mismatch and the use of systematic reviews, stating: “research takes time, as well as money and effort …. Policies have a different timeframe. So if a government is going to move in a particular area, or feels inclined or compelled that it needs to come up with something, it might not be able to wait for research”  Two participants also stated that government department employees were risk averse and so would “perpetuate current practice” rather than suggesting and evaluating “original ideas” based on new research.The most cited barriers to using robust evidence were political agenda (60%), time limits (55%), funding (50%), and research not being applicable for current policies (50%). For example, one participant stated “would make policy development a lot easier because you would have shown quite clearly due to the collaborative nature of the research that you’ve considered a large number of things and it would seem to provide a very solid finding because of that” (NGO Manager). Two participants (10%) stated that being able to access collated information would be helpful as it would reduce the amount of time spent looking for applicable research.When policy makers were asked what could improve the use of evidence in developing policy, six participants (30%) stated that there should be more “links” or collaborations between government staff and researchers. According to one policy maker these linkages “Seven epidemiologists were asked to participate; however, only six agreed and completed the interview. All interviewees had a post-doctoral degree, and all but one was a researcher from an Australian university. There was an even number of male and female respondents.All respondents cited that they had heard of grading system but tended not to use them to evaluate research evidence, rather they had their own way of evaluating evidence. One participant stated that they evaluated studies from first principles , and another admitted to giving more credibility to studies published in prestigious journals as they tended to undergo more rigorous peer-review and methodological editing.All respondents cited that although RCTs are considered at the top of the hierarchy of evidence and observational research lower, RCTs are not necessarily the most efficient or applicable evidence. Respondents found several problems with using RCTs, including unsuitable research questions , limited generalizability, and bias. All respondents argued that it is more important to look at the design and conduct of the study – for example cohort size, duration, evaluation of relevant covariates/confounders – than it is to look at what rating the evidence is.Responses to what constituted as high-quality observational research all focused on the rigor of the methodology. All respondents agreed that high-quality observational research should address bias and ensure that the data are valid. Four respondents also argued that the sample had to be representative of the target population and large.The majority of respondents were aged between 35 and 44 years (32%) followed closely by 45–54 (29%) and 55–64 years (25%). Respondents were mostly female (71%) and had completed a postgraduate qualification (82%). Of the 28 participants who responded to all questions, 13 (46%) described their level within the organization as “middle management or project/policy officer with some management responsibilities.”When asked to indicate preferred research methods, respondents (19%) indicated that systematic reviews were the preferred research method. Qualitative research and RCTs followed with response rates of 16 and 13% respectively. Only 7% of respondents indicated a preference for observational research.The most easily understood sources of evidence were trusted organizations (96%), other internal staff (92%), consumer views (85%), policies from other jurisdictions (81%), and expert opinions (73%). The most difficult evidence sources to understand were researchers and existing academic research (42%) and internal statistical data (35%).The most important factors taken into consideration when evaluating evidence are shown in Figure The relative ranking of specific research methods and data synthesis techniques, as indicated by policy makers, is shown in Figure The aims of this study were to gain an understanding of the role of research evidence in policy making, investigate how policy makers in the health domain select this evidence, and whether they systematically assess evidence quality. While use of evidence differs somewhat across policy makers, it appears that the reliance on direct scientific evidence in the policy development process is low. The policy maker’s investigation did not seem to have a methodical approach to evaluating evidence. Although there was some overlap between what policy makers and epidemiologists identified as high-quality evidence, there was also some important differences which suggests that the best scientific evidence is not frequently used in the development of policy. Differences between epidemiologists and policy makers included the way evidence was evaluated and the importance placed on study’s methodology.Systematic reviews were the preferred research method in the policy-making process. Observational research came last and ranked as the lowest quality. Previous research has found that policy makers perceived systematic reviews as better suited to identifying gaps in existing research rather than providing answers to policy questions . This reThe information provided by policy makers about the use of academic resources in the policy process is inconsistent. In the web survey, all responding participants indicated that academic research was the most often utilized evidence source in the policy process. However, in the interviews, only one-quarter of participants stated that they referred to academic journals when gathering evidence for policy. Furthermore, participants from the web survey stated that academics and existing academic research were the most difficult evidence source to understand. This difference between responses may indicate that what policy makers think they are using, or what they should ideally be using, is not what they actually use and that they may not fully understand the academic research that they are using. Studies that had similar results found that respondents did not use academic literature because they did not have access to libraries or online journals, they were not trained in how to use academic search engines and because they found academic literature complex and frequently contradictory .Results from both the web survey and interviews found that the majority of policy makers used work which had been done by other jurisdictions or organizations as a base for policy and program decisions. The use of other jurisdictions programs/policies may be a feasible option as it fits with the policy environment and provides a sense of security that the intended outcomes will be achieved within the decided timelines. However, as participants pointed out, this transferability either may not be applicable to the adopting jurisdiction or key information and supporting evidence may not be provided by the other jurisdiction. Given that respondents stated they usually did not check the quality of the evidence to these programs, or the applicability of this evidence to the situation, then the policy/program objective may not be met.Respondents from both the interviews and web survey also mentioned that other internal staff members were one of the most frequently utilized source of evidence, and that part of their research strategy included talking to others, such as experts or consultants. This has been found in previous studies and may be a way of gaining accurate information quickly , 14, 33.Policy maker’s reliance on peers and other jurisdictions, rather than evidence, could indicate several possible characteristics of policy makers. First, this might suggest that respondents are assuming that someone else has checked and evaluated the evidence. Second, policy makers may lack the skills to evaluate the evidence themselves, or lack confidence in their own skills. A third possibility, cited by two participants, is that individuals within the government health department are risk averse. Individuals may feel that the culture within the public service discourages innovative programs and policies as such evaluations may fail and result in damage to the government and individual’s reputations.Previous research has found that political opinion or targets influenced the adoption of particular policies or programs . Within Just under half of participants in the interviews discussed that there was not a great understanding among policy makers of what makes good quality evidence. This has been found in previous studies . AlthougBoth parties agreed that RCTs, followed by systematic reviews, provided the highest quality evidence and that observational research was ranked the lowest. However, both policy makers and epidemiologists cited problems with using RCTs in their respective fields. For policy makers, RCTs were not applicable in the policy decision-making process, whereas epidemiologists had methodological issues with RCT designs . These findings are similar to previous research , 23.The most cited barrier to using robust evidence was political agenda and time limits. Previous research has also found that the short time periods, or need for action, within the policy making sphere meant that decisions were often made whether “strong” evidence was there or not , 35. ResHalf of our policy-maker participants stated that barriers to using research were lack of funding and research not being applicable to current policy. This has been found in previous research , 36. It The establishment of more links, or collaborations, between policy makers and researchers was cited by one-third of policy makers as a way to improve the use of evidence in the policy-making process. This strategy has been frequently discussed in previous research , 35. PreThis study has some limitations. First, the sample size was small and only a few organizations within a single Australian provincial-level jurisdiction were surveyed and as such may not be more widely generalized. Second, due to survey design, we could not analyze how policy maker’s level of research training affected their use of scientific research. For example, it is possible that those with a specific health-related Masters or Postgraduate degree are more likely to use peer reviewed literature. Despite these limitations, this study gathered data on a process about which little is known or understood. Furthermore, it used different methodologies in order to gain a more comprehensive understanding of the issue and different organizations from both government and non-government were involved.To facilitate the use and assessment of academic research in the policy-making process, we have four recommendations. The first is to build policy makers capacity to appraise evidence, through strategies, such as training and participation in internships. This recommendation is based on our finding that policy makers did not have a great understanding of what makes good quality evidence nor did they use a standardized way of evaluating evidence. As only a small number of policy makers in this study referred to academic sources, the second recommendation is to ensure that policy makers can access robust sources of scientific evidence, for example online peer-reviewed journals. Third, because the policy and scientific processes occur on different time scales, which policy makers in this study cited as a barrier to using robust evidence, the sharing of evidence between researchers and public servants should be facilitated through new channels and ways of conducting business. This is particularly important for health issues for which the scientific data may vary substantially over time. Finally, we recommend developing mechanisms through which scientists with specific expertise are invited into a particular department for a “scientific chat” to openly discuss planned policies. This would be particularly useful in cases where commissioning new research would take too long but where substantial “soft” evidence is already available in the scientific field.Based on our findings that policy makers cited researchers and existing academic research as one of the most difficult evidence sources to understand and that a barrier to using robust evidence was research not being applicable to current policies, we have three recommendations for researchers. The first is to build awareness among researchers producing policy-relevant material that this information cannot be communicated exclusively through typical scientific dissemination processes . Furthermore, academic research with policy-relevant material should include a clearly identified policy-relevant section that can easily be identified by policy makers, and the language and statistics included should be tailored in a way that makes them usable by policy makers. Second, training on the production and effective ways to communicate policy-relevant material in scientific research should be provided to researchers. Finally, forums where scientists and policy makers can interact and demonstrate their viability and effectiveness should be established.This study has found that neither policy makers nor epidemiologists are using grading systems to evaluate evidence, rather each have their own ways of assessing the evidence. Both policy makers and epidemiologists recognized that RCTs were usually at the top of these hierarchies, but that RCTs were not always the most efficient or applicable evidence upon which to base population health policies and that there were some problems with RCT designs. Policy makers in this study demonstrated a good understanding that they need to have an evidence base, that it is an important part of the process, and that it justifies the policy. However, the time and resources to form that evidence base, as well as an understanding of what constitutes good evidence and how to evaluate it was lacking. This study is limited by its small sample size; however, by having both in-depth interviews and the web survey we are provided with more and often conflicting information than previous research has found using just survey data. Finally, this study focused on the use of observational evidence and interviewed only one type of public health researcher, academic epidemiologists. By using this approach, the authors have not examined the use of intervention research which provides direct evidence on how to produce change and which may be more relevant to policy makers . FindingThe study concept and design was done by KA, NC, and PK. All authors contributed to the analysis and interpretation of data and drafting of the manuscript. LJ conducted all statistical analysis. All authors have read the final paper and have agreed to be listed as authors.PK is employed by ACT Health Directorate as a policy maker, a co-investigator in the study as well as being an author. The other authors declare no conflict of interest.