The complexity of modern practice requires health professionals to be active information-seekers.Our aim was to review the quality and progress of point-of-care information summaries—Web-based medical compendia that are specifically designed to deliver pre-digested, rapidly accessible, comprehensive, and periodically updated information to health care providers. We aimed to evaluate product claims of being evidence-based.We updated our previous evaluations by searching Medline, Google, librarian association websites, and conference proceedings from August 2012 to December 2014. We included Web-based, regularly updated point-of-care information summaries with claims of being evidence-based. We extracted data on the general characteristics and content presentation of products, and we quantitatively assessed their breadth of disease coverage, editorial quality, and evidence-based methodology. We assessed potential relationships between these dimensions and compared them with our 2008 assessment.r=.41, P=.0496). However, all dimensions improved from 2008: editorial quality (P=.01), evidence-based methodology (P=.015), and volume of diseases and medical conditions (P<.001).We screened 58 products; 26 met our inclusion criteria. Nearly a quarter  were newly identified in 2014. We accessed and analyzed 23 products for content presentation and quantitative dimensions. Most summaries were developed by major publishers in the United States and the United Kingdom; no products derived from low- and middle-income countries. The main target audience remained physicians, although nurses and physiotherapists were increasingly represented. Best Practice, Dynamed, and UptoDate scored the highest across all dimensions. The majority of products did not excel across all dimensions: we found only a moderate positive correlation between editorial quality and evidence-based methodology (Medical and scientific publishers are investing substantial resources towards the development and maintenance of point-of-care summaries. The number of these products has increased since 2008 along with their quality. Best Practice, Dynamed, and UptoDate scored the highest across all dimensions, while others that were marketed as evidence-based were less reliable. Individuals and institutions should regularly assess the value of point-of-care summaries as their quality changes rapidly over time. Pressed for time and obliged to navigate ever-expanding medical literature, doctors are increasingly relying on online information tools to accelerate the search process without compromising the reliability and quality of information retrieved. Point-of-care information summaries offer predigested syntheses of medical research intended to be used when the patient and physician interact  . Web-basThe medical information technology market parallels the efforts by national health systems to streamline clinical workflow and align clinicians’ behavior with best practice strategies. Point-of-care summaries play a central role: they increasingly form the knowledge basis of complex information systems, such as computerized physician order entry and computer decision support systems -5. In thAs point-of-care information summaries gain ground in the culture of medical practice as stand-alone products or integrated with other systems, their validity must be assessed against marketing claims that they are evidence-based. This review examines the quality of Web-based point-of-care information summaries and their development and progress since 2008.As this is an update of analyses done in 2008  and 2012To identify the point-of-care information summaries, we re-examined the eligibility of all products that were included or excluded in the 2008 and 2012 analyses. To find new summaries, we searched Medline from August 2012 to December 2014 with the following terms: ((“Evidence-Based Medicine”[Mesh]) AND ) AND ((“Online Systems”[Mesh]) OR (“Point-of-Care Systems”[Mesh])). We scanned the references of the papers retrieved and used the Google search engine to identify additional products that may not have been reported in the medical literature. We explored various publisher and librarian association websites  -16, and One reviewer examined the search results, screened the titles and abstracts of papers identified through Medline, and evaluated the eligibility of products integrating additional information found on product websites. If there was doubt about the inclusion of a product, all authors discussed the eligibility until a consensus was reached. We recorded the reasons for exclusion.One reviewer extracted information on the general features of each point-of-care information summary. Products that could not be accessed  were excluded. One reviewer collected data on the general characteristics of products and their content presentation for qualitative (descriptive) evaluation, along with information about the editorial quality, evidence-based methodology, and content volume  for empirical quantitative analysis. A second reviewer checked the extractions.International Classification of Diseases (ICD)-10 code searches.For each summary included, we collected the following general details: country of development, year of release, vendor or publisher, marketing claims, format , access and subscription options, annual costs, and targeted audience. Since the 2012 analysis, we have introduced an additional component: ability to be integrated into an EHR system. This entails the capacity to access information from the point-of-care summary directly through the EHR interface. For example, when a physician clicks on a condition written in the patient record, the physician is directed to a new screen detailing disease information and treatment options. A point-of-care summary search tool may be additionally available on the EHR interface to make free-text and Content presentation was analyzed in summaries that we accessed. We examined the different outputs , use of formal ontology, flexibility, and reporting of references . We also assessed products’ adoption of an intent to recommend, use of a formal strength of recommendation system, as well as the availability of continuing medical education programs or credits, other education materials , and patient handouts.Two reviewers extracted information about three key dimensions: quality of the editorial process, quality of the evidence-based approach to content development , and volume or breadth of the medical conditions covered. We described products quantitatively using three separate scores that covered components relevant to each dimension. Disagreements were resolved by discussion between the reviewers. A third author was consulted for any unsolved discordances. All Web pages providing useful data were saved and stored in an electronic archive. When information about a particular component  was unclear or could not be found, we contacted publishers by email requesting additional information and clarification of contents. All emails were stored in an electronic archive.We adopted the following indicators of transparency to evaluate the methodological quality of the editorial process: authorship (reporting of authors for each summary), reviewing , updating (whether or not summaries had been revised or updated in the previous 2 years), conflicts of interest (disclosure of contributing authors’ conflict of interest), and commercial support for content development. For this last component, we assigned three points if commercial support was not accepted, one point if commercial support was accepted and reported, and no points if the product developer did not present sufficient information for us to make a judgement. For the remaining items, we assigned three points if the component was judged as “adequate,” one point if “unclear,” and none if “not adequate” or “not reported.” We arbitrarily decided to award three points instead of two for the adequate fulfillment of a criteria in order to give more weight to transparent and accountable reporting, and increase variability within the sample.In the 2008 and 2012 reviews, we assessed the authorship, authors’ conflicts of interest, and updating of products based on the editorial policy statements. If the information provided was insufficient to make an accurate evaluation, we referred to a nonrandom selection of sections (often referred as topics) to assess the dimensions. In the effort to minimize bias between reporting and implementation in the 2014 analysis, we evaluated these dimensions through a random sample of topics. We randomly selected ten blocks or categories of diseases from ICD-10 . If any The following components were used to evaluate the strength of the evidence-based methodology for content development: implementation of a literature search or surveillance strategy to identify current information, cumulative versus discretionary approach (prioritization of systematic reviews over other evidence sources), critical appraisal, formal grading of evidence, and citation of expert opinions (separation of expert opinions from other evidence sources in summaries). Three points were assigned if the component was judged “adequate,” one if considered “unclear,” and none if “not adequate” or “not reported.”As it was not feasible to count the total number of diseases and medical conditions covered in each product, we estimated the comprehensiveness of disease coverage by verifying the presence or absence of a random sample of diseases from the ICD-10 . We randP values. Changes in the strength of the products from 2008-2014 were assessed using the matched pairs Wilcoxon signed-rank test. For hypothesis testing, a probability of <.05 was considered statistically significant. All statistical tests were two-sided. Stata software was used for statistical analyses.Volume and quality indicator scores are presented with medians and interquartile ranges (IQR). Point-of-care information summaries were ranked on the basis of (1) editorial quality, (2) the use of an evidence-based approach, and (3) the volume of diseases covered based on a random sample of ICD-10 chapters. Correlations between these three dimensions were assessed by Spearman rank correlation coefficients and their respective The search strategy identified 58 products for potential inclusion. After screening, 26 fulfilled our inclusion criteria. Sixteen of these were previously included in the 2008 and 2012 reviews . Four products changed into a new product since 2012 . Six products were newly identified in this review . Prodigy, which is connected with Clinical Knowledge Summaries (CKS), was considered a new product since CKS was discontinued for some time and only in 2012 was restarted. In order to access the 26 products, we registered for free-trial access online whenever available or contacted the publishers directly requesting temporary access to perform the evaluation. We did not receive a response from the publishers of three products , which were prevented from further evaluation. A total of 23 products were included in the content presentation and quantitative analysis.General features are summarized in Editorial quality and evidence-based methodology are summarized in r=.41, P=.0496). No correlations were found between editorial quality and volume , or between evidence-based methodology and volume .The ranking of point-of-care information summaries based on their strength of volume, editorial quality, and evidence-based methodology is shown in P=.01), evidence-based methodology (P=.015), and volume (P<.001). Compared to the 2008 evaluation, there were significant improvements in all three dimensions: editorial quality , information must be re-filtered to meet personal practice needs. Users will need to personalize the product, setting filters to isolate specific information  that is relevant to individual clinical practice. This will prioritize information that can engender changes in health professional behavior .Second, high-quality point-of-care summaries should be integrated into computer decision support systems for EHRs. These computer systems may represent the future of clinical decision making in which evidence-based knowledge from point-of-care summaries is linked with patient information from EHRs to generate case-specific guidance messages through rule- or algorithm-based software ,38. CompThird, the potential integration of point-of-care summaries into continuing medical education programs should be recognized . Doubts The maturation of point-of-care summaries can be seen as a virtuous circle . It star


Dirofilaria immitis and Dirofilaria repens are mosquito-transmitted zoonotic nematodes, causing heartworm disease and skin lesions, respectively, in carnivores. In Europe, the domestic dog is apparently the main definitive host, but patent infections occur also in other species of carnivores. The rapid spread of the golden jackals (Canis aureus) throughout Europe opens a question of involvement of this species in the sylvatic cycle of pathogens in the colonised territories, including Dirofilaria spp.D. immitis, D. repens and Acanthocheilonema reconditum by multiplex PCR DNA amplification.Between January 2014 and May 2015, 54 golden jackals from 18 localities in Romania were examined by full necropsy for the presence of adult filarioid nematodes and blood samples from all animals were screened for the presence of microfilariae of D. immitis were found in 18.52 % of the animals, originating from the southern part of Romania. No D. repens or A. reconditum were found at necropsy. The molecular prevalence in blood samples from the same animals was 9.26 % for D. immitis and 1.85 % for D. repens. All samples were negative by PCR for A. reconditum.Nematodes morphologically identified as Dirofilaria spp. infections in golden jackals from Romania together with the increasing density of the jackal populations highlight their potential role in the transmission of these zoonotic parasites and in the maintenance of natural disease foci.The relatively high prevalence of  Dirofilaria infecting European carnivores are D. immitis, a widespread nematode that causes severe cardiovascular disease (heartworm disease) [D. repens, which localises sucutaneously and is associated with a variety of dermatological clinical signs [Canis familiaris), but patent infections have been reported also in wild canids, such as wolves (Canis lupus), red foxes (Vulpes vulpes) and golden jackals (Canis aureus) [D. immitis infection and their reservoir role has been demonstrated in ferrets (Mustela putorius furo) [Dirofilaria spp. are zoonotic; however, D. repens is the one involved in most human cases recorded in Europe [The main species of disease)  and D. ral signs . Carnivoal signs . The typ aureus) –6. Musteus furo) . Both Din Europe , 9.D. immitis, D. repens and Acanthocheilonema reconditum have been reported in dogs and humans from different areas of the country [Dirofilariases are widely distributed in Europe. The southern countries are historically endemic or even hyperendemic territories, and during the last decades, the infection has spread also into the north-eastern and central Europe , 10. In  country –13. HoweIn the past decades, the populations of golden jackals in Europe underwent considerable expansion. The species originated in the Arabian Peninsula and has initially spread in three directions: westwards to Europe, to Africa and to eastern Asia . More reGolden jackals are known as important reservoir hosts for several parasitic and infectious diseases, many of them having medical or veterinary importance , 19. HowD. repens and A. reconditum. The heart, pulmonary arteries and lungs were longitudinally dissected and macroscopically examined for the evidence of D. immitis. All filarial nematodes were collected and identified based on morphological keys and morphometric data available in the literature [Between January 2014 and May 2015, 54 legally hunted golden jackals originating from 18 localities of Romania  of spirurid nematodes was amplified following [D. immitis, D. repens and A. reconditum) was performed [Additionally, clotted blood samples were taken from the heart of each examined animal and further processed by molecular methods . Genomicollowing , while ferformed . PCR proP-values were lower than 0.05.Statistical analysis was performed using EpiInfo™ 7 software . The frequency and prevalence of infection and the 95 % confidence intervals (95 % CI) were established and the differences in prevalence were assessed using chi-square testing. The differences were considered significant if χ2 = 2.90, df = 1, P = 0.04), whereas the difference between sexes was not significant . The intensity of infection varied between one and seven nematodes/animal (mean intensity three), with seven jackals harboring at least one sexually mature pair .In total, 27 male and 27 female jackals were necropsied; 24 individuals were juveniles, while the other 30 were adults. Nematodes were retrieved from the right ventricle and pulmonary arteries of ten jackals  originating from six localities  and showed a 100 % similarity with strains of D. immitis collected from dogs from China and Australia (accession numbers EU159111 and AJ537512).Overall, 30 nematode specimens were recovered  and identified as D. immitis , one sample was PCR-positive for D. repens  and all samples were PCR-negative for A. reconditum. All five blood samples PCR positive for D. immitis. corresponded to animals which were positive also at necropsy , most likely due to a longer exposure to infected mosquitoes. Dirofilaria repens was molecularly identified in one animal (1.86 %) that was also positive for D. immitis at necropsy. The D. repens sequence was deposited in the GenBank database (accession number KU321603) and showed 95 % similarity to other European strains of D. repens .Five blood samples were positive by PCR for cical”, an animal coming from the south of the Danube [The golden jackal is considered to be one of the most widespread wild canids. In Europe, an ongoing expansion of the Balkan populations has been recorded, their presence having been observed, with records as far west as Switzerland, as far north as Estonia and as far east as the Ukraine and the Russian Federation , 16. In e Danube . During e Danube . Howevere Danube , 28. More Danube , 31. Cure Danube , 32.Dirofilaria spp. is conditioned by a series of factors, mainly the existence of suitable hosts and the availability of mosquito vectors [Culex pipiens complex, which are the most efficient natural vectors of D. immitis [Aedes albopictus, another important natural vector for Dirofilaria spp. [Dirofilaria spp. and for a high density of Culex spp. [During the last decades, golden jackals have colonised and are currently present in all areas that are endemic for canine dirofilariases in Romania , 33–38.  vectors . Species immitis , and Aedria spp. , are comria spp. , 43. Morria spp.  that arelex spp. .D. immitis infection established by necropsy was of 18.52 %, showing a relatively high value, compared to other post-mortem studies performed in neighbouring European countries. In Bulgaria, the recorded prevalence ranged between 4.4 % (2/45) and 37.54 % (122/325) [D. immitis in golden jackals was of 7.32 % (32/437) between 2009 and 2013 [D. immitis by means of PCR were either harbouring nematodes of the same sex (n = 3) or were at juvenile age (n = 2), corresponding to the prepatency period, which ranges between six and nine months [D. repens together with the immune responsiveness of the host may have inhibited the production of D. immitis microfilariae [In the present study, the prevalence of 122/325) –48. In H122/325) . In Serband 2013 . Interesand 2013 . All indfilariae .Dirofilaria repens infections have been previously reported in golden jackals from Africa and Asia [D. repens was significantly lower  compared to D. immitis.and Asia , 54, butD. immitis infections in golden jackals and wolves are an epi-phenomenon of dog infection, and that they do not represent a real wild reservoir of the infection [Dirofilaria spp. infections are only sporadically reported, as individual cases [D. immitis or D. repens in some epidemiological settings [Dirofilaria spp., particularly D. immitis.Some authors consider that nfection . Indeed,al cases , 55–57. settings , 58–60. Dirofilaria spp. compared to the wild canids in sympatric areas. However, in the present study the overall prevalence of D. immitis infection in golden jacakals exceeded that of dogs originating from the same counties (e.g. 25 vs 15.94 % in Tulcea County) [D. immitis infections in Romanian golden jackals suggests the existence and perpetuation of a sylvatic life-cycle and highlights the importance of jackals as free-ranging resevoirs.Undoubtedly, the domestic dogs are the most abundant carnivores throughout Europe and as such, represent a dominant reservoir of  County) . The higD. repens infection in golden jackals for the first time in Europe. We highlight the relatively high prevalence of patent Dirofilaria spp. infection in golden jackals in Romania, underlining their possible role as reservoir host involved in the dissemination of these filarioids, particularly D. immitis, in their newly colonised territories.To the best of our knowledge, the present study reports


Evidence synthesis techniques in healthcare education have been enhanced through the activities of experts in the field and the Best Evidence Medical Education (BEME) collaborative. Despite this, significant heterogeneity in techniques and reporting of healthcare education systematic review still exist and limit the usefulness of such reports. The aim of this project was to produce the STORIES  statement to offer a guide for reporting evidence synthesis in health education for use by authors and journal editors.A review of existing published evidence synthesis consensus statements was undertaken. A modified Delphi process was used. In stage one, expert participants were asked to state whether common existing items identified were relevant, to suggest relevant texts and specify any items they feel should be included. The results were analysed and a second stage commenced where all synthesised items were presented and participants asked to state whether they should be included or amend as needed. After further analysis, the full statement was sent for final review and comment.Nineteen experts participated in the panel from 35 invitations. Thirteen text sources were proposed, six existing items amended and twelve new items synthesised. After stage two, 25 amended consensus items were proposed for inclusion. The final statement contains several items unique to this context, including description of relevant conceptual frameworks or theoretical constructs, description of qualitative methodologies with rationale for their choice and presenting the implications for educators in practice of the results obtained.An international expert panel has agreed upon a consensus statement of 25 items for the reporting of evidence synthesis within healthcare education. This unique set of items is focused on context, rather than a specific methodology. This statement can be used for those writing for publication and reviewing such manuscripts to ensure reporting supports and best informs the wider healthcare education community.The online version of this article (doi:10.1186/s12916-014-0143-0) contains supplementary material, which is available to authorized users. Evidence-based health care involves the systematic collection, synthesis and application of all available scientific evidence, not just the opinion of experts . The intFor more than a decade, there have been calls for medical education to become more evidence-based -7. In paThe Best Evidence Medical Education (BEME) collaborative was established in 1999 , rejectiSince the inception of BEME, the conversations within the medical education field have radically changed, with a new zeitgeist disseminating through such research. Key to this revolution has been substituting the often self evident question as to ‘whether’ education interventions are effective  and focuThe challenge for those completing secondary evidence synthesis within healthcare education has been to evolve techniques that can reflect this revolution in primary education research. Organisations such as BEME have embraced such changes, leading to rapidly updated author guidance, as well as current structural reorganisations to widen methodological and general education expertise within the organisation . ScholarAn example can be found in the often cited landmark systematic review on Internet based medical education published in 2008 which concluded that e-learning is better than no intervention and similar (on average) to traditional instruction . SpecifiPublication standards are common, useful and often required by those conducting health research. Examples include CONSORT for randomized controlled trials , PRISMA Reporting In healthcare education of Evidence Synthesis) statement as a baseline set of reporting criteria.This study set out to develop the STORIES  AND (‘checklist’ OR ‘reporting’ OR ‘statement’) was undertaken in the Medline database from 1993 to the present day. Papers reporting a standardised set of criteria for any form of evidence synthesis in healthcare were included. Four such publications were deemed relevant ,27-29. ACommon items to reported systematic review statements/checklists• Describe as a systematic review piece, with specific type mentioned• Provide a structured summary• Describe the rationale for the review in the context of what is already known• Provide an explicit statement of questions being addressed• State why this method of review was selected• State and provide a rationale for how the searching was done• Provide details on all the sources of information and dates searched• Electronic database details should include full search terms for at least one database• If individuals familiar with the relevant literature and/or topic area were contacted, indicate how they were identified, selected, contacted and what they contributed• Explain how judgements were made about inclusion/exclusion• Describe the process of data extraction and any process of contacting authors for confirmation of/or more data• Describe and justify the method of analysis and how quality was assessed• Give a flow diagram summarising study selection• Provide the characteristics of all included documents• Present the main findings in light of the reviews objectives• Discuss strengths and limitations of the review and its findings, commenting on the strength of the evidence• Give guidance for future research• Provide details of fundingThese items were presented as a bare minimum set of reporting items for any evidence synthesis.Medical Education, Medical Teacher, Postgraduate Medical Journal, Academic Medicine, Advances in Health Sciences Education: Theory and Practice, BMC Medical Education, British Medical Journal, Journal of the American Medical Association, Lancet and New England Journal of Medicine.An online Delphi method was used. Three authors of prior published BEME systematic reviews volunteered to pilot the survey, with amendments and suggestions incorporated. Members of the expert panel were then recruited through several methods. Firstly, two additional members of the BEME editorial board were recruited. Secondly, invitations were sent to all corresponding authors for published BEME reviews. Finally, all corresponding authors for evidence synthesis publications within key journals were invited to take part. Articles from August 2012 to July 2013 were considered through hand searching in the following journals: The Delphi panel was conducted online using the online survey tool, ‘surveymonkey’. It began with a ‘brainstorm’ round (‘round 1’). Participants were sent the preliminary consensus item list and asked to comment on the suitability of these items for inclusion. Additionally, expert participants were invited to submit personal views, exchange theoretical and empirical papers on the topic and suggest items for inclusion. These were collated by the authors in preparation for the next round (‘round 2’). The final list of consensus items was presented, as well as new amended items, with participants asked ‘yes/no’ for inclusion. Each of the suggested resources, as well as empirical items for inclusion, were presented as a questionnaire with likert scales to indicate relative importance validity and sent to participants for ranking (‘round 2’). Once again, the responses were collated by the authors and as consensus was reached on the majority of items, a final first draft of the statement was sent to participants for review.The statement was used to assess a contemporaneous published medical education systematic review from a leading journal by local colleagues and collaborators representing various professions, including nursing, medicine, health informatics and allied health professionals. Additionally, panel members were invited to review the same study and consistency of responses analysed, as well as free text feedback on the statement collated.A total of 35 potential expert panel members were identified to invite for participation – 10 authors of BEME reviews and a further 25 from published reviews within the hand searched journals. Within this group were two members of the BEME editorial board and so no further BEME members were invited to participate. Nineteen experts agreed to take part and completed the Delphi survey (54%), with four continents represented . The Delphi panels ran between September 2013 and January 2014. All participants completed all the questions in round 1. This was also the case in round 2, with 100% response in all sections of the survey. Feedback leading to changes of the final document was received from four experts, with no further concerns raised from the other experts. The statement was sent out to the panel for piloting with a contemporaneous published education systematic review. This led to a 75% overall agreement in judgements made by returning panel members. Additionally, there was universal feedback that the STORIES statement supported the judging of the report.The STORIES statement is presented in Additional file The structure of the STORIES statement has drawn on previous methodological publications and, in particular, on the ‘Explanations and Elaborations’ document of the PRISMA statement  and rece1. Use a title that includes a description of the aims of the piece  and the method of evidence synthesis ‘Internet-based medical education: a realist review of what works, for whom and in what circumstances.’ .In is clear that a title for evidence synthesis has multiple purposes in this context. It was apparent from the first part of this study that within healthcare education this is still often not clearly indicated by authors and only implicit. The primary requirement is to set out the aims of the study and as such tell readers whether the focus of the review is one of description, justification or clarification . Additio2. Provide a structured summaryThis item was consistent across all other publication standards for evidence synthesis. In the context of education evidence synthesis, this should seek to highlight the key items from the statement. In particular, the implications of the work for future educational research (item 22) and the implication for educators (item 23).3. Describe the rationale for the review in the context of what is already known4. Provide a statement of the questions being addressed by the study‘There has not been a systematic review of the literature on teaching professionalism. The heterogeneity of learning theories and teaching approaches employed makes such a review a difficult undertaking. This difficulty is compounded by the varying ways that professionalism has been defined and the lack of consensus on what criteria make up medical professionalism.’’ [‘Our research question was: What teaching processes, systems, and approaches have been found to work to ensure an ethos of professionalism in medical graduates? We sought to discover: What works in teaching professionalism? (Method), How does it work? (Methodology), Why does it work? (Theory) and What does it teach?’ .Given the massive and often heterogeneous evidence base that exists within healthcare education research, it is key for the study to address both why the review is being completed (by discussing what is the current state of the field) and by addressing the context for this particular review. This may include justification as to why specific learner groups, educational environments or content topics are being selected for this piece of work and how this relates to existing works, possibly within other fields.The expert panel also expressed a strong desire for a specific set of questions to be stated as the aims for the work and for there to be a match between these questions, the title of the review (item 1) and the methods of synthesis (item 5). Whilst this may seem implicit, it is clear that often these three elements are not aligned ,34. This5. State why this method of evidence synthesis was selected within the context of the questions being asked‘A realist review using Pawson’s techniques of realist enquiry to include a wider range of outcomes and criteria for inclusion including grey literature and other non-empirical reports and documents. This would consider the complexity and other phenomena associated with CEME. A realist synthesis review attempts to answer the question “how and when does it work and for whom”, in doing so it draws on a wider range of sources than a systematic review and it is based on synthesizing commentaries rather than ratings. Again we would seek to answer the following core questions: What key factors define CEME? What are the strengths and limitations of CEME?’ .In keeping with items 3 and 4, as well as stating a clear objective and why this has been selected, authors must particularly relate these to the methods of evidence synthesis selected. This is not because such decisions can be judged as right or wrong, as it is entirely reasonable to approach the same review questions with different methodologies. Given this diverse range of methodologies and the various epistemological stances that can be taken by authors, understanding the decisions that have been made allows readers to more fully interpret the results and conclusions of the study .6. State and provide a rationale for how the searching was done7. Provide details on all the sources of information and dates searched8. Electronic databases - provide full search terms for at least one database, with details of deviations in subsequent searches‘Our objectives for evidence synthesis were not aligned with a particular epistemological stance and thus we did not take a strict positivist or constructionist approach. Rather, we followed a pluralistic model, not using a single arbiter for quality assessment and including a mixture of evidence types.The following online databases were searched to June 2011 using a standardised search strategy (Appendix online).’ .As with many of the areas discussed, our panel did not feel that there were any specific limitations or expectations for searching that should be imposed, merely a description should be offered with the rationale for such decisions to support the readers in understanding the breadth of data that the search is likely to cover. As with all evidence synthesis, sufficient detail should be given to support replication.9. Describe the process of data extraction and any process of contacting authors for confirmation of/or more data10. Explain the method for judging inclusion/exclusion‘If a study reported an intervention in limited detail or commented on improved handover without presenting evidence in support of the improvement, we attempted to contact the author for further details’ .Common with evidence synthesis in other contexts, if methodological data is missing so as to limit judgments on quality, then attempts should be made to contact the authors. However, it is also commonplace in educational systematic review to find that published educational interventions lack the detail to understand fully the education in question. This may consist of a lack of detail as to the structure, resources or learning outcomes and can limit usefulness for readers  when synSimilarly to other methodological areas, the panel did not feel prescriptive guidance on inclusion and exclusion of studies based on any set criteria was needed, but simply for authors to clearly highlight any decisions or assumptions made to readers.11. If quality appraisal tools are used, please describe and justify their choiceThese items were the focus of much discussion amongst the panel and highlight some of the issues in reporting that led to the conception of these works. There are many different appraisal tools that have been applied in this context ,39-41. A12. Describe qualitative methods for synthesising primary evidence (where appropriate) and the goal of these methods, such as thematic analysis; meta-ethnography, and realist synthesisQualitative synthesis combines rigorous processes and authorial judgement to present the collective meaning of research outputs . As with13. Describe quantitative methods for synthesising primary evidence (where appropriate), such as meta-analysis and how issues of heterogeneity will be considered‘We planned for both quantitative and qualitative evidence synthesis. Because there was high between-study inconsistency in prior analyses, we planned to use random effects meta-analysis to quantitatively pool results, organized by comparison . Additionally, we planned subgroup analyses based on trainee level, topic, and the key instructional design features noted above. Finally, we conducted subgroup analyses based on key study design elements (randomization and blinding of assessment) and sensitivity analyses, excluding the results of studies with imprecise effect size calculations.’ .The role of quantitative methods of synthesis, such as meta-analysis, within the context of education systematic review is difficult to define. In many cases, the naturally heterogeneous nature of educational interventions and learner groups limits the scope for such analysis. However, when reporting such work, it is important for authors to report in what circumstances they would have used such methods and if indeed they were used, what measures were taken to address heterogeneity. The example above eloquently describes methods that consider the three main forms of heterogeneity that should be accounted for: educational, methodological and statistical heterogeneity.14. Give a flow diagram summarising study selection15. If individuals familiar with the relevant literature and/or topic area were contacted, provide a summary of the contact and information obtained16. Provide summarised details of included works, considering elements such as methodology, key results and conclusions17. Describe methods of quality assessment of education reported, including all parameters considered 18. Describe quality assessment of the research methods of included studies19. Present the results of qualitative and/or quantitative evidence synthesisall reported evidence synthesis guidance and was agreed by the panel unanimously for inclusion. Item 15 reports the outcomes of item 9 and, as such, was agreed for inclusion by the panel.The focus of panel discussions was to what extent included works should be presented. It was felt that three broad areas should be reported. These are a summary of the educational research activity itself (item 16), a summary of the education employed in studies (item 17) and a summary of the quality of research methodology of primary studies (item 18). Consistent with other items, the panel avoided prescriptive items and, therefore, attempted to provide broad guidance (Items 16 and 18).Item 17 gives examples of potential areas to report when discussing actual educational interventions from primary studies. Whilst it could be argued that if a review is focused on effectiveness (justification), such information is not required, panel members argued that having a piece of evidence synthesis that is educational in nature and not presenting any information to inform the reader as to what such education looks like is a missed opportunity . Additio20. Present the main findings in light of the review objectivesWhilst discussing the findings of the review it is key to relate this to the review objectives. This will often be a superfluous item for authors, but the panel noted that it is possible in the context of certain review methodologies, such as realist reviews , that th21. Discuss strengths and limitations of the review and its findings, commenting on the strength of the evidence‘Overall, inconsistencies were evident in methodological reporting and quality, in the 14 studies included in this review. None of the studies reviewed provided an appropriate framework for defining, measuring or understanding emotional intelligence (EI) within their work. This resulted in the inclusion in this review of a wide range of EI proxy measures, thus illustrates the problems caused by the broad definition of EI and related constructs within medical education.’ .Commenting on the limitations of the review was common to all other published guidance. The panel highlighted that such discussions should particularly relate how the quality of primary extracted data has impacted, and possibly limited, the strength of conclusions made.22. Discuss how the findings of the evidence synthesis impact future primary research‘We have several recommendations for improving the research base for journal clubs. The study design should be based on pedagogical theory, for example using the principles of adult learning.Each element of the intervention needs to be operationalised with sufficient definition of the variables to facilitate comparison of the active ingredients within the delivery framework. Evaluation questions need to be matched to the goals of the intervention, to the effectiveness measures, and to the level of effect.’ .This was a common item from existing guidance, but the panel reflected that they thought it was in many ways more important in the context and generally poorly reported by authors. The review team have an extremely in depth knowledge of the state of the field and, as such, are very well placed to highlight explicitly directions for future work. This is one of the least objective and paradoxically most important elements of the reported work.23. Describe possible implications of the findings for educatorsThe panel once again reflected that whilst a rather self evident item, this is often missing from existing reviews. Insights should not be limited to the clinical teacher, but where appropriate, give suggestions for curriculum developers and educational policy makers. This allows reviews to be relevant at both the micro and macro educational level.24. Provide details of funding25. Describe the skills and expertise of the review team and acknowledge any outside helpThese final items were the most contentious amongst the panel. Some felt them to be unnecessary, but in the final stages of the process, the case for inclusion was decided to be strong. Item 24 was argued to be crucial in all scholarly endeavours. Whilst education is often felt to be less at risk from conflicts of interest and the influence of commercial interests, the nature of granted educational work could still present a potential source of bias and, as such, should be transparently reported to all. Finally, item 25 is a requirement when completing works for organisations such as BEME and Cochrane. The panel did not feel this was primarily to judge whether a team should have completed a review, but to allow readers to frame methodological choices within the context of the team’s expertise.The STORIES statement has been produced through a multistage process that involved synthesis of existing consensus guidance, a Delphi process and finally the piloting of the draft statement. The aim of this process has been to support reporting of healthcare evidence synthesis that is more informative to researchers, educators and policy makers.The STORIES statement has clearly been built on previous consensus statements and, as such, it is interesting to consider it within this context  and item 11  both reflect the profoundly multidisciplinary nature of health education and the range of methodologies employed by researchers in the field. In this manner, the STORIES statement is completely unique, focusing on the context of the synthesis, rather than methodology.The STORIES statement cannot replace primary methodological guidance on how to carry out health education evidence synthesis, but through the expert Delphi panel, a number of key relevant resources have been highlighted for authors. Given the vast array of evidence available in health education research, the kaleidoscopic nature of the evidence and the significant range of valid methodological choices facing authors, the STORIES statement is presented as a framework to support clear reporting. STORIES is not meant to be a prescriptive checklist, but a tool to support authors in considering the choices they have made and to act as a prompt to support transparency in reporting these choices. Many items simply highlight the existence of choice and so it is left to the individual acumen of authors, reviewers and readers to make and judge specific decisions made.In formulating the STORIES statement, an expert panel was selected that represented a range of professionals working in a vast array of educational settings. The common skill of all panel members was practical experience completing evidence synthesis. As such, it is hoped that the finished statement is pragmatic, grounded in the realities facing authors and reflecting needs with the field.The STORIES statement is not based on any new empirical works. The outcome of the process is nothing but opinion and the results of the process are only as valid as the opinions of the experts who formed the panel. Whilst the panel did represent those publishing within the peer reviewed literature, there are notable gaps in the range of expertise represented. For example, there is a predominance of medicine educators and a lack any PHD medical educators. This may not invalidate the statement, but limit our ability to comment in any meaningful way on its application within other contexts.The Delphi process employed a bare consensus set of reporting items within stage one. It can be considered a limitation that this new set of reporting criteria is grounded within previous consensus statements that had ultimately been judged as incomplete within the medical education setting. However, given that the majority of such items were accepted by our panel, it is our assertion that these items are indeed a pragmatic and appropriate backbone for evidence synthesis reporting. Indeed, the value of the STORIES statement is in clarifying that this is the case within the medical education context and the addition of several novel high value items that are situation specific. The STORIES statement is presented not as a final product, but in response to a need. Future work is needed from researchers and clinical teachers to comment on, refine or possibly reject STORIES.This study has sought to gain a consensus for reporting of health education evidence synthesis. A review of existing guidance, followed by the convening of a Delphi process involving an international expert panel, has agreed upon a consensus statement of 25 items for reporting. This unique set of items is focused on context, rather than a specific methodology and as such can be applied to all evidence synthesis works within health education. This statement can be used for those writing for publication and reviewing such manuscripts to ensure that reporting supports and best informs the wider healthcare education community.


This paper presents a typology of available evidence to inform physical activity policy. It aims to refine the distinction between three types of evidence relating to physical activity and to compare these types for the purpose of clarifying potential research gaps.A scoping review explored the extent, range and nature of three types of physical activity-related evidence available in reviews: (I) health outcomes/risk factors, (II) interventions and (III) policy-making. A six-step qualitative, iterative process with expert consultation guided data coding and analysis in EPPI Reviewer 4.856 Type I reviews, 350 Type II reviews and 40 Type III reviews were identified. Type I reviews heavily focused on obesity issues (18 %). Reviews of a systematic nature were more prominent in the Type II (>50 %). Type III reviews tended to conflate research about policy intervention effectiveness and research about policymaking processes. The majority of reviews came from the United States, United Kingdom, Australia and Canada.Although evidence gaps exist regarding evidence Types I and II, the most prominent gap regards Type III, i.e. research pertaining to physical activity policymaking. The findings presented herein will be used to inform physical activity policy development and future research.The online version of this article (doi:10.1007/s00038-016-0807-y) contains supplementary material, which is available to authorized users. The need for a scoping review on physical activity evidence types is two-fold: First, the World Health Organization (WHO) Regional Office for Europe and its Member States are implementing the first European Strategy on Physical Activity to support the reversal of trends in rates of certain non-communicable diseases and obesity  number of reviews found, (b) kinds of reviews, (c) first author country affiliation, (d) dates of publications, and (e) most frequent and/or trending topics. Fourth, in the discussion and concluding remarks we relate results to previous work, discuss limitations of the study and indicate future research needs.A key aim of this paper is to refine the distinction between three types of evidence relating to physical activity. A second aim is to compare evidence types for the purpose of identifying potential research gaps. Type I evidence focuses on the link between physical activity and health status, i.e. the type, amount and intensity of physical activity and its effects on different health outcomes. For example, this type of evidence might demonstrate how physical activity reduces the risk of diabetes. Type II evidence links interventions with physical activity, i.e. the type of intervention, setting and/or environment that influences physical activity. For example, this type of evidence might demonstrate how settings-based interventions support increased physical activity. Effective environmental and policy interventions to promote physical activity also belong to Type II evidence. Type III evidence focuses on the effects of policy-making on physical activity interventions, i.e. the policy agendas, structures, funding and processes that affect development, implementation or adaptation of physical activity interventions. For example, this type of evidence may demonstrate how cross-sectoral approaches to policy-making help position physical activity promotion on the agendas of different policy sectors and policy levels.Regarding Type II and Type III evidence, policy plays a role in both. Policy is involved in Type II evidence as a part of interventions to increase physical activity. Policy-involved interventions, however, result from the dynamics of policy-making processes (Type III evidence). For example, tax incentives that encourage active transportation may result in more active lifestyles in the general population. These incentives, as part of interventions, can be deemed Type II evidence. The policy-making process of reaching consensus on the incentives and examining the political environment can be deemed Type III evidence.The distinction between Type I and Type II evidence is widely addressed in the literature. Making the distinction between Type II and III evidence is more innovative. Doing so may help overcome conceptual hurdles in previously published literature, which often conflates “policy and environmental approaches” or deals with policy approaches as a type of  intervention  as a framework for policy development  and IBSS  were searched independently by two researchers and in consultation with an institutional librarian. Searching incorporated Boolean operators, advanced search features and grammatical variations of search terms. While the type and number of search terms could easily be expanded in any direction, especially in relation to synonyms for physical activity (e.g. exercise), doing so would make the comparison of results across evidence types inconsistent. This review, however, is intended to be the first of a series of reviews that will build upon results in a step-wise process. The next round of reviews may tailor new combinations of keywords based upon what was learned from this review.At the outset, it was recognized that there might exist a vast amount of research in one area and a limited amount of research in another. Therefore, it was determined that prior to exploring single studies, the literature would be searched for various types of reviews. A variety of kinds of reviews was purposefully sought , which fostered openness to identifying evidence produced from different disciplinary sources . All inclusion/exclusion criteria for the initial search, which sought to compare across the three different types of evidence, was the same. Any document type in English, French, German or Spanish was considered. Any study focusing on humans that addressed physical activity and was discussed by the authors in relation to effectiveness of evidence was considered, regardless of outcome, type of intervention, target group or exposure.Documents were excluded if they did not address humans, made no connection to physical activity or were not considered to be an evidence review. For example, while some documents may have been retrieved that contained the word “review”, they may have, in fact, been a single study. Those remaining were included as part of conceptual mapping and comparative analysis.n = 958, Type II n = 1106, Type III n = 146) were removed, the research team conducted a preliminary screening of the remaining documents  in each category. After the preliminary screening it was found that the titles and abstracts contained a significant overlap in relation to the different evidence types  , eliminate duplicates (n = 756) in a second step and then conduct a screening and sorting process on the remaining documents (n = 1496).Relevant studies were selected and analyzed between December 2014 and May 2015. A team of three researchers checked for duplicate references using EndNote plus a manual duplicate search. Once duplicates (Type I n = 250) . All documents not identified as false positives were deemed relevant and selected for qualitative comparison (n = 1246).References were then imported into EPPI Reviewer 4 where a team of researchers independently assessed a sample of the documents. A final screening of all references eliminated false positives  kind of review, (2) country of first author’s institution, (3) date of publication, and (4) emergent themes.There are a variety of kinds of reviews to consider when ascertaining the extent, range and nature of research that originates from different disciplines  Fig. . This waThis discussion addresses how the results of this scoping review achieve the aims of clarifying the distinction between evidence types and helping understand where potential research gaps can be filled. It also provides insights into evidence-based policy-making in the areas of health and health promotion in several ways.First, the outcomes confirmed the hypothesis that evidence for the three types of physical activity research differs in terms of quantity and characteristics of reviews available in scientific journals. For example, the number of reviews on health effects of physical activity (Type I) was more than twenty times higher than those on physical activity policy-making (Type III). The kind of reviews ranked highest in the hierarchy  extend the scoping review to other than English languages/data bases to determine the availability of evidence from non-English language sources, (2) consider placing more emphasis on policy-related physical activity research in low and middle income countries   Below is the link to the electronic supplementary material.


In trying to improve health, particularly for the millions of older Americans with chronic conditions, many researchers focus solely on improving the professional health care system. Better medications, care protocols, and other clinical interventions are important, but much of chronic care takes place away from the clinic or hospital and in one’s home or community . EvidencArchstone Foundation is a private independent grant making foundation with a mission of preparing society to meet the needs of an aging population. During the last two decades, the Foundation has supported the development, evaluation, and dissemination of a range of evidence-based programs in areas including fall prevention, physical activity, chronic disease management, caregiver support, and mental health. It also funded the development of Better Choices, Better Health, the online version of the Stanford Chronic Disease Self-Management Course, to broaden its reach to underserved populations. Archstone Foundation has supported the cultural adaptation of EBP to meet the needs of increasingly diverse older adults. It has funded local and national coalitions to support the dissemination of EBP, including the Falls Free Initiative led by the National Council on Aging . Most reWhether it is foundation funding, Older Americans Act funds, or resources of the organizations offering the programs, funds are limited. So are participants’ and program providers’ time and energy. The development of EBP for older adults allows limited resources to be directed to programs with the greatest probability for positive impact. As a funder looking at hundreds of requests each year, confidence in the science is critically important. We recognize that when we support any given program, other programs may struggle. EBP give funders greater assurance that what we fund will deliver meaningful results.Archstone Foundation and many other funders have supported EBP out of a desire to improve health outcomes and quality of life for older adults. We value the programs’ self-management strategies that empower older adults, while effectively improving their health. EBP have been supported out of a desire to provide high quality, effective, and sustainable programs with a broad reach, and proven outcomes when done with fidelity to the original model .Healthy aging initiatives for older adults require broad and effective community collaborations . ResearcReflecting upon the last few decades, there has been a remarkable improvement in the number, variety, and quality of evidence-based programs. A few examples of the programs include: the Chronic Disease Self-Management Program, addressing several chronic conditions; A Matter of Balance, Stepping On, and Tai Chi for Better Balance, addressing fall prevention, PEARLS , and IMPACT (Improving Mood-Promoting Access to Collaborative Treatment) addressing depression; Fit and Strong! and EnhanceFitness, addressing exercise. There is now an alternative to the free-for-all of “do it yourself” programs, developed without standards or proof of outcomes, as was the case only a few years ago. We now have a system of programs with varying levels of evidence targeting a number of health and quality of life concerns. The Centers for Disease Control and Prevention, the SAMHSA National Registry of Evidence-Based Programs and Practices, the Administration of Community Living, and the Agency for Healthcare Research and Quality are a few of the agencies that have established processes for evaluating, and certifying or registering programs as evidence-based. We are also seeing a growing number of programs addressing the needs of our diverse older adults . This enIt is important to recognize that there has been resistance to the widespread use of evidence-based programs. This resistance offers lessons in how to improve the field . Some haLooking to the future, our challenge is to expand the breadth of offerings, improve the quality, and ease the adoption of evidence-based programs. To realize the potential of EBP, we will need to prepare our workforce to understand, implement, manage, and promote the programs . The useThe author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.This paper is included in the Research Topic, “Evidence-Based Programming for Older Adults.” This Research Topic received partial funding from multiple government and private organizations/agencies; however, the views, findings, and conclusions in these articles are those of the authors and do not necessarily represent the official position of these organizations/agencies. All papers published in the Research Topic received peer review from members of the Frontiers in Public Health  panel of Review Editors. Because this Research Topic represents work closely associated with a nationwide evidence-based movement in the US, many of the authors and/or Review Editors may have worked together previously in some fashion. Review Editors were purposively selected based on their expertise with evaluation and/or evidence-based programming for older adults. Review Editors were independent of named authors on any given article published in this volume.


Actinic keratosis (AK) is a common precancerous skin lesion and many AK management guidelines exist, but there has been limited investigation into the quality of these documents. The objective of this study was to assess the strengths and weaknesses of guidelines that address AK management. A systematic search for guidelines with recommendations for AK was performed. The Appraisal of Guidelines for Research and Evaluation (AGREE II) was used to appraise the quality of guidelines. Multiple raters independently reviewed each of the guidelines and applied the AGREE II tool and scores were calculated. Overall, 2,307 citations were identified and 7 fulfilled the study criteria. The Cancer Council of Australia/Australian Cancer Network guideline had the highest mean scores and was the only guideline to include a systematic review, include an evidence rating for recommendations, and report conflicts of interest and funding sources. High-quality, effective guidelines are evidence-based with recommendations that are concise and organized, so practical application is facilitated. Features such as concise tables, pictorial diagrams, and explicit links to evidence are helpful. However, the rigor and validity of some guidelines were weak. So, it is important for providers to be aware of the features that contribute to a high-quality, practical document. Actinic keratosis (AK), or solar keratosis, is a common dysplastic lesion of the keratinocyte , 2. OverClinical practice guidelines (CPG) can guide clinicians' diagnostic and therapeutic decisions by succinctly reviewing the literature and proposing evidence-based management recommendations. CPG are defined by the Institute of Medicine as “statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options. Rather than dictating a one-size-fits-all approach to patient care, CPG offer an evaluation of the breadth and quality of the relevant scientific literature and an assessment of the likely benefits and harms of a particular treatment” . Thus, aThe PRISMA checklist was used as a guide for this investigation. A medical librarian performed a systematic search of the medical literature for CPG with recommendations for AK. An extensive search of Medline/PubMed  was conducted for relevant clinical practice guidelines using a combination of the key terms “actinic” and “keratosis” and “guideline” as well as “keratosis, actinic” as a major medical subject heading. Results were limited to practice guideline and guideline publication types. There were no date or language restrictions. In addition the National Guideline Clearinghouse, National Institute for Health and Clinical Excellence, European Academy of Dermatology and Venereology, Guidelines International Network, TRIP Database, Australian Government Clinical Practice Guidelines Portal, Scottish Intercollegiate Guidelines Network, Guidelines and Audit Implementation Network, Professional Organizations and Royal Colleges, Health Service Executive Guidelines and the American Academy Dermatology web sites were searched using key terms “actinic” and “keratosis.” a priori and (1) included an explicit statement identifying the document as a management guideline, (2) were written by multiple authors including at least one dermatologist, and (3) made recommendations concerning multiple options to manage AK. Exclusion criteria included (1) consensus statements or review articles that did not include identifiable management recommendations, (2) reviews of a published guideline, (3) focus on only preventative, epidemiology, or research methods, (4) focus on a limited or specific patient population, and (5) discussion of one form of management.The inclusion criteria were establishedTwo reviewers (TS and JK) independently examined the retrieved titles and abstracts to assess the articles for inclusion and exclusion criteria. The full text of these articles was retrieved and the same two reviewers (TS and JK) reviewed the selected articles again for eligibility. Disagreement was resolved through discussion by the reviewers (TS and JK). Seven of the guidelines were selected for analysis.The AGREE II tool was used to describe the quality of the selected CPG. It was developed to evaluate the validity and feasibility of CPG as well as assess sources of bias . The AGRThe AGREE II instrument was used to define the assessment variables and rating system and to store data . This inThe minimum standardized score for each domain was, therefore, 0% and the maximum was 100%. A standardized domain score above 60% has been suggested as the threshold to indicate sufficient minimum quality to consider practical use of this portion of the guideline . This stp value of <.05 was considered significant. All analyses were performed using SAS 9.3 .The scores were compiled and reviewed; upon visual inspection one investigator had scores that appeared to deviate from the other three. The Pearson correlation coefficient is very sensitive to outlying values so Spearman's Rho was used to determine interrater reliability . The degp = .38]). This rater was a first-year medical student and it was presumed that this reflected clinical experience rather than CPG quality so his scores were removed and subsequent calculations did not include these scores. The correlation coefficients among the three remaining appraisers, which included two dermatologists and a family practitioner, were moderate or better (0.54–0.86).The search strategy identified 2,307 records and after review ultimately seven records were included in the study . The chaThe CCA/ACN guideline  was the Three guidelines were recommended by at least three raters . The editorial independence domain includes a rating of COI and funding sources and had the lowest mean score of all the domains. COI reporting by the CPG contributors varied greatly. The Stockfleth et al. 2008  article One of the primary purposes of CPG is to make practical, evidence-based recommendations about management of a condition. A summary of the recommendations from the three CPG with the highest scores is presented in The results of this study should be considered in the context of the limitations, including CPG that were not included and the effects of score variation between raters and within raters. Since the literature search was completed in 2014, this review does not include CPG published more recently which may be valuable , 28. ThiEffective CPG are evidence-based with recommendations that are concise and organized, so practical application is facilitated –31. The


The Guidelines International Network (G-I-N) aims to promote high quality clinical guideline development and implementation. Guideline-based performance measures are a key implementation tool and are widely used internationally for quality improvement, quality assurance, and pay for performance in health care. There is, however, no international consensus on best methods for guideline-based performance measures. In order to address this issue, the G-I-N Performance Measures Working Group aimed to develop a set of consensus-based reporting standards for guideline-based performance measure development and re-evaluation.Methodology publications on guideline-based performance measures were identified from a systematic literature review and analyzed. Core criteria for the development and evaluation process of guideline-based performance measures were determined and refined into draft standards with an associated rationale and description of the evidence base. In a two-round Delphi-process, the group members appraised and approved the draft standards. After the first round, the group met to discuss comments and revised the drafts accordingly.Twenty-one methodology publications were reviewed. The group reached strong consensus on nine reporting standards concerning: (1) selection of clinical guidelines, (2) extraction of clinical guideline recommendations, (3) description of the measure development process, (4) measure appraisal, (5) measure specification, (6) description of the intended use of the measure, (7) measure testing/validating, (8) measure review/re-evaluation, and (9) composition of the measure development panel.These proposed international reporting standards address core components of guideline-based performance measure development and re-evaluation. They are intended to contribute to international reporting harmonization and improvement of methods for performance measures. Further research is required regarding validity, acceptability, and practicality.The online version of this article (doi:10.1186/s13012-015-0369-z) contains supplementary material, which is available to authorized users. Clinical practice guidelines aim to improve the quality of patient care . The InsExisting guidance on PM development or appraisal highlights that clinical guidelines are an important source for quality measures –11. Fromhttp://www.g-i-n.net/) aims to lead, strengthen, and support collaboration in guideline development, adaptation, and implementation [The Guidelines International Network (G-I-N)  included in the systematic review  was exclOf the 30 experts invited to participate in round 1, 90 % responded and 70 % (21/30) completed the online Delphi questionnaire. The 27 respondents of round 1 were invited to take part in round 2. Of those, 44 % (12/27) completed the rating of the criteria and approved the rationales . No additional criteria were proposed. The main outcome of the discussion in between the Delphi rounds was the need for consistent wording expressing the goal of developing reporting standards as opposed to developing a quality appraisal tool. Revision of the criteria was made accordingly. In Delphi round 2, the overall agreement for every revised criterion and rationale was at least 90 % (see Table a. State the currency of the guideline(s) used for performance measure development and state if it/they meet the criteria set out by the Guidelines International Network (G-I-N). Describe the guideline quality using a validated guideline appraisal tool, such as AGREE II.b. Indicate additional (evidence) sources, if used and the rationale for their use.To provide a high quality, trustworthy base for PM, the source clinical guideline should meet the criteria published by G-I-N   which ideally are under the control of health care professionals. Structures and outcomes of care are more determined by influencing systemic factors and regional or national conditions .Internationally, there is debate about the type of recommendations which are adequate sources for PM. Recommendations with a strong grade of recommendation  are meant to be useful and valid for the majority of patients whereas weak recommendations describe options for individual situations. Recommendations with a weak grade of recommendation  are therefore not considered suitable for PM intended to serve for external quality assurance, in particular accountability purposes . The strength of a recommendation does not only express the level of the underlying evidence but also other considerations like the level of confidence that implementing this recommendation will do more good than harm or will avoid harm in case of negative recommendations , 44. StrLevels of evidence and/or grade of recommendations of the underlying guideline recommendation are provided in about two third of respective publications .Describe the consensus methods used to select the performance measures from the supporting clinical guideline recommendations in detail.The process of selection implies specification of candidate PM (see criterion 5), critical appraisal according to the specific criteria or core attributes (see criterion 4), and final decision-making by an expert panel (see criterion 9) in a consensus process. To minimize bias, it is recommended to use formal consensus methods , 34–37.Appraisal of guideline-based PM should be done in several steps. Some attributes can be appraised directly, before implementing the measure (see criterion 4); others can only be appraised when there are already data of the respective measure available .Reporting the consensus methods used for selection and how they were conducted helps to assess the quality of the selection process. To our knowledge, there are presently no studies comparing different selection methods. The majority of publications identified stated the use of formal consensus methods to develop PM. Nearly half of the studies available cite the Research and Development Corporation (RAND) Appropriateness Method (RAM) developed by the RAND and the University of California, Los Angeles (UCLA) or a modified RAM using the consensus method with varying appraisal criteria and cutoff levels for agreement. Also mentioned is a RAND-modified Delphi method or a Delphi process .Relevance Scientific Soundness (as a minimum: the evidence supporting the measure)Feasibility (as a minimum: clarity of definition and measurability)State the consideration of the following attributes during the performance measure development process:Frequently reported appraisal criteria in the methodology publications include usefulness of PM for improving patient outcomes, relevance, and feasibility of monitoring , 35, 37.Given different possible perspectives, developers should explain which aspects they summarize under “relevance” or “importance” of a measure” . Further appraisal and selection after having done a practice test  is reasonable. If a recommendation is potentially measurable, the lack of measurement feasibility in current health information systems should not be used as the sole criterion for determining a measure to be invalid as improvements to existing data can result in a measure becoming feasible.Specify the performance measure numerator and denominator unambiguously and in detail.To be measurable, a guideline recommendation has to be transformed into a rate or proportional based measure consisting of numerator and denominator. Only in a few cases, PM consist of so-called sentinel events which should be presented as counts, rather than proportions—for example, maternal deaths . InclusiRecommendation: For patients with R0 resected stage III colon cancer, adjuvant chemotherapy is indicated. Numerator: Number of patients who have undergone adjuvant chemotherapy.Denominator: All patients with colon cancer Union internationale contre le cancer (UICC) stage III who have had an R0-resection of the primary tumor.Further specification: substances, minimal number of chemotherapy cycles required, timeframe.Exceptions: Patient refusal, contraindications to be specified.Provide a clear description of the intended use of the performance measure  and at what level in the health system it is used .Quality improvement (QI)—for internal quality improvement purposes such as voluntary initiatives of health professionals Certification: for quality assurance at a regional or national level, such as being used as a compulsory instrument for quality assurance in hospitals and/or a defined ambulatory care setting or for various aspects of the in- and/or out-patient sector in general . ImportaPay for performance, where payment is attached to defined levels of achievement of the measure with the intention of improving the quality of care , 49.Accountability—containing different aspects, for example:The intended use of guideline-based PM should be stated as PM are to be used at different levels within health systems:In addition, quality measures may be used in research to develop or produce new knowledge about the health care system.The different uses of PM influence the indicator development methods that have to be applied. It seems useful to distinguish PM into those to be used for QI and those used for accountability purposes , 51. A kProvide a full description of the practice test (piloting) prior to using the guideline-based performance measure. If no practice test is done, provide the reason for this. Provide information about any other validation process in use.All measures but especially indicators that are used for regional or national reporting purposes or pay for performance are at risk of having unintended consequences , 54. A pState the currency of the performance measures in use. State the criteria for deciding to change or stop using performance measures.There can be several reasons to stop a measurement, for example, if measures show unintended consequences or lack of reliability . Other rDescribe clearly the composition of the panel deciding on the performance measures with information on participation of multidisciplinary experts, stakeholders in the field, experts in quality measurement, and patient representatives.Similar principles apply to the composition of guideline-based PM groups as to clinical practice guidelines, including the consideration of conflicts of interest, but there is very little evidence for this rationale. According to the “good practice,” the group should be multidisciplinary and include content experts in the field—usually health care professionals involved in the relevant clinical guideline development including stakeholders who are being measured and, if applicable and possible stakeholders who will use this data to inform decisions, patient representatives and experts in quality measurement (representing the organizations which measure) . This isWe have developed a reporting standard for guideline-based PM with nine criteria, using formal written consensus methods (two Delphi rounds). This is the first work to our knowledge presenting a consensus on transparent reporting of methodological requirements of guideline-based PM by an international group. Applying these criteria aims to make guideline-based PM more comprehensible and valuable. The fact that all criteria were confirmed after the first round indicates their international meaningfulness for the participating guideline and PM experts. There was a strong view on these criteria to be used as standard reporting criteria. The group judged the available evidence not yet sufficient for the development of an appraisal tool.The methodology publications identified were heterogeneous concerning criteria for guideline selection, selection of guideline recommendations, panel selection, consensus methods, appraisal criteria, and proposals for practice tests. No paper included and specified all nine criteria presented here. In light of missing comparative studies, the structured consensus process was most important to ensure the best possible expert-based PM reporting criteria. It is notable that one of the steps in PM development identified in the systematic review —“topic sWe identify the following limitations of this work: First, the present criteria do not address any qualitative aspects of patient care to be reported. It is recognized that different patient groups do have different needs and the best quality of care may be different for the same condition, requiring differentiated patient-oriented measures  . Second,Furthermore, the consensus process was done with a small group of experts with a low participation rate in the second round. However, there was a high acceptance already in the first round and the final draft was approved by the whole group.Finally, not all criteria were judged to be of similar importance see Fig. .It is planned that the reporting standards will be tested in different countries and settings in order to evaluate their validity, acceptability, and practicality.These reporting standards provide international consensus on the best practice criteria for reporting guideline-based PM development and re-evaluation. Better reporting of methods used by PM developers should improve both the quality and consistency of guidelines-based PM. The PMWG encourages research on the validity, acceptability, and practicality of the respective criteria.Additional file 1:Study protocol (2013). (PDF 427 kb)Additional file 2:AMSTAR appraisal of the systematic review Kötter et al., 2012. (PDF 13 kb)Additional file 3:G-I-N survey round I, copy of survey monkeys results and comments. (PDF 366 kb)Additional file 4:G-I-N survey round 2; copy of results. (PDF 266 kb)


Narrative exposure therapy (NET) is a recently developed, short-term treatment for patients with a posttraumatic stress disorder (PTSD) as a result of multiple trauma. NET can be applied very successfully in patients with complex trauma complaints , stones (traumatic events), sometimes candles (grief), or recently also sticks for aggressive acts . For culturally diverse populations, NET is recommended as the most evidence-based trauma treatment, besides culturally adapted CBT. NET has been investigated in different populations in Africa, Europe, and Asia. In Asia, research has been carried out in Sri Lanka as well as in China. In China, NET was conducted and investigated with survivors of the Sichuan earthquake (Zang, Hunt, & Cox, To date, there is good evidence NET is effective in the treatment of PTSD patients, with support from 18 RCTs (


Information and communication technology (ICT) tools are known to facilitate communication and processing of information and sharing of knowledge by electronic means. In Nigeria, the lack of adequate capacity on the use of ICT by health sector policymakers constitutes a major impediment to the uptake of research evidence into the policymaking process. The objective of this study was to improve the knowledge and capacity of policymakers to access and utilize policy relevant evidence. A modified “before and after” intervention study design was used in which outcomes were measured on the target participants both before the intervention is implemented and after. A 4-point likert scale according to the degree of adequacy; 1 = grossly inadequate, 4 = very adequate was employed. This study was conducted in Ebonyi State, south-eastern Nigeria and the participants were career health policy makers. A two-day intensive ICT training workshop was organized for policymakers who had 52 participants in attendance. Topics covered included: (i). intersectoral partnership/collaboration; (ii). Engaging ICT in evidence-informed policy making; use of ICT for evidence synthesis; (iv) capacity development on the use of computer, internet and other ICT. The pre-workshop mean of knowledge and capacity for use of ICT ranged from 2.19-3.05, while the post-workshop mean ranged from 2.67-3.67 on 4-point scale. The percentage increase in mean of knowledge and capacity at the end of the workshop ranged from 8.3%-39.1%. Findings of this study suggest that policymakers’ ICT competence relevant to evidence-informed policymaking can be enhanced through training workshop. All over the world, there is increasing recognition of importance of making health policies with best available evidence. The process which is known as evidence-informed health policymaking is characterized by the systematic and transparent access to, and appraisal of, evidence as an input into policymaking . Accordibefore the programme (intervention) is implemented and after [The aim of this study is to improve the knowledge and capacity of Nigerian policymakers to access and utilize policy relevant evidence via an ICT training workshop. In this workshop, a modified “before and after” intervention study design was used in which outcomes were measured on the eligible population (target participants) both nd after . The difThis research was a sub-national study and participants consisted of individuals whose geographical area of operation is south-eastern Nigeria, with emphasis on Ebonyi State. The target participants were the career health policy makers, this category of policymakers included: health professionals in charge of the health systems; regional, state and local government directors of the health ministry; directors of primary health care at the local government level; health professionals working with specific programmes in the health ministry; staff and consultants involved in public health issues within the health ministry; programme/project managers under the health ministry; chief executive officers of civil society groups, including non-governmental organizations; leaders of national health-based associations  This category of policymakers were selected as key participants in this study because findings from our previous studies , 18, 19 Approval for this study was obtained from the Directorate of Research, Innovation & Commercialization (DRIC), Ebonyi State University, Abakaliki Nigeria. The approval was based on the agreement that participation in the research was voluntary following informed consent; that participants’ anonymity would be maintained; and that every finding would be treated with utmost confidentiality and for the purpose of this research. These were adhered to in this study.A total of five facilitators were engaged in the workshop based on their expertise in the training contents. The facilitators included four senior academics from Ebonyi State University Abakaliki Nigeria and a senior director from the Ministry of Health. The academics included a Professor of infectious diseases and epidemiology with vast experience in intersectoral partnership/ collaboration in health policymaking; the remaining three senior academics have vast experience in evidence-informed policy making and knowledge translation in the Nigeria context. The only policymaker facilitator has vast experience working with the government in the formulation of policy in the south eastern Nigeria.A total of 70 policymakers were mapped out for this study. The mapped out participants were divided into two batches of 35 persons each. We organized a two-day intensive health-policy based ICT training workshop at the Ebonyi State University, Information and Communication Technology (ICT) (E-Library) Centre Abakaliki Nigeria, for the policymakers. All the policymakers were invited to the workshop by invitation letters which were sent 2 weeks before the event and was followed-up with a text message reminder to their mobile phones a day before the programme.The first batch of policymakers had their workshop in April 2014, while the second batch had theirs in May 2014. The duration of the workshop each day was five hours from 10am-3pm (with a break between 12:30pm-1pm). A pre-workshop assessment questionnaire , was administered prior to actual training to assess the level of knowledge and capacity of the participants on the specific topics to be covered within the theme of the workshop. After the administration of the pre-workshop questionnaire the training commenced and was facilitated by four resource persons . The workshop covered the following topics: (i). Need for intersectoral partnership/ collaboration in health policymaking in Ebonyi State; (ii). Benefits of engaging ICT in evidence-informed policy making for control of infectious diseases of poverty and running the health sector; (iii). Introduction to use of ICT for evidence synthesis;(iv) Development of the capacity of the policymakers on the use of computer, internet and other ICT the following: (a). The search protocol for health information and policy relevant research evidence; (b). Identification of and search strategies for evidence of a wide range of electronic resources, in addition to the traditional scientific and clinical databases ; (c). Information/evidence audit to accompany search strategies.All teaching sessions were done using power-point presentation and handouts on each topic were produced and distributed to all participants. It was made mandatory for all lectures to be delivered in simplified, practical and easily comprehensible patterns, with little or no emphasis on complex mathematical or scientific computations/models for the benefit of non-specialists who constituted the majority of the participants. Practical sessions were held during the workshop in which each participant was able to use an internet connected computer to practice the acquisition of research evidence from relevant electronic databases. At the end of the workshop, a post-workshop assessment questionnaire was administered to the participants to evaluate the impact of the workshop.The data collected via the questionnaire  was analyzed using the methods developed at McMaster University Canada by Johnson and Lavis . The anaOut of the 70 policymakers and stakeholders invited for the workshop, a total of 52 individuals attended. The official designation attributes as indicated in intersectoral partnership/ collaboration in health policymaking”, the PRE-ICT mean ranged from 2.99-3.05, while the POST-ICT mean ranged from 3.37-3.67, with the percentage increase ranging from 14.3%-22.6%. In terms of “engaging ICT in evidence-informed policy making for control of infectious diseases of poverty and in running the health”, the PRE-ICT mean ranged from 2.26-2.55, while the POST-ICT mean ranged from 3.14-3.41, with the percentage increase ranging from 25.1%-39.1%. In terms of “knowledge & application of information/communication technology for Internet use”, the PRE-ICT mean ranged from 2.21-3.03, while the POST-ICT mean ranged from 2.69-3.51, with the percentage increase ranging from 8.3%-24.1%. Concerning the “use of ICT for evidence synthesis”, the PRE-ICT mean ranged from 2.19-2.49, while the POST-ICT mean ranged from 2.67-3.08, with the percentage increase ranging from 16.6%-32.8%   . We focuIn the present study, it is worthy of note that the knowledge of the use of ICT for evidence synthesis improved considerably among the policymakers after the training workshop. The percentage increase ranged from 16.6% to as high as 32.8%. The implication of this is that the policymakers have acquired some abilities to use of ICT for accessing policy relevant information; know the sources of evidence and electronic databases relevant for evidence in policy making; and to search and identify existing research evidence in policy making context; as well as assess the authenticity, validity, reliability, high quality of research evidence. There are numerous reports including randomized controlled trials that have demonstrated considerable improvement in the capacity of health professionals to use ICT effectively to improve healthcare capacity following training/instructional programme/workshops –31. It iIt is pertinent to state that although notable improvement in knowledge and skill was witnessed in this study among the participants, the study was not without a number of limitations. First, the ICT training workshop lasted only two days. This could be considered rather too short a time duration to assess the real impact of the training. Furthermore the post-workshop assessment that was conducted may not have been very adequate to evaluate the impact of the training skills acquired. A follow-up of participants to see how far they are able to use the skill acquired in the policymaking process would have provided a more excellent evaluation as we observed in our previous study . The fin