While there is an abundance of evidence-based practice (EBP) recommendations guiding management of various chronic diseases, evidence suggesting best practice for using social media to improve health outcomes is inadequate. The variety of social media platforms, multiple potential uses, inconsistent definitions, and paucity of rigorous studies, make it difficult to measure health outcomes reliably in chronic disease management. Most published investigations report on an earlier generation of online tools, which are not as user-centered, participatory, engaging, or collaborative, and thus may work differently for health self-management.The challenge to establish a sound evidence base for social media use in chronic disease starts with the need to define criteria and methods to generate and evaluate evidence. The authors’ key objective is to develop a framework for research and practice that addresses this challenge.This paper forms part of a larger research project that presents a conceptual framework of how evidence of health outcomes can be generated from social media use, allowing social media to be utilized in chronic disease management more effectively. Using mixed methods incorporating a qualitative literature review, a survey and a pilot intervention, the research closely examines the therapeutic affordances of social media, people with chronic pain (PWCP) as a subset of chronic disease management, valid outcome measurement of patient-reported  outcomes (PRO), the individual needs of people living with chronic disease, and finally translation of the combined results to improve evidence-based decision making about social media use in this context.Extensive review highlights various affordances of social media that may prove valuable to understanding social media’s effect on individual health outcomes. However, without standardized PRO instruments, we are unable to definitively investigate these effects. The proposed framework that we offer outlines how therapeutic affordances of social media coupled with valid and reliable PRO measurement may be used to generate evidence of improvements in health outcomes, as well as guide evidence-based decision making in the future about social media use as part of chronic disease self-management.The results will (1) inform a framework for conducting research into health outcomes from social media use in chronic disease, as well as support translating the findings into evidence of improved health outcomes, and (2) inform a set of recommendations for evidence-based decision making about social media use as part of chronic disease self-management. These outcomes will fill a gap in the knowledge and resources available to individuals managing a chronic disease, their clinicians and other researchers in chronic disease and the field of medicine 2.0. Presently, no rigorous frameworks exist informing how to generate evidence of improved health outcomes from social media use in chronic disease management based on robust research design. We wish to address this problem. The work presented in this paper is part of a larger research project, which has two aims. First, on a theoretical level it aims to develop a framework for generating evidence of improved health outcomes from social media use as part of chronic disease self-management. Second, on a practical level it aims to produce a series of recommendations for clinicians suggesting evidence-based decision making about social media use in the same setting . We also recognize the shortcomings of much of the research in this domain. Namely, there is a failure to discuss and unpack fundamentals within the research context. In the case of this research, addressing the above study aims requires careful consideration of the following key concepts: social media, evidence-based practice (EBP), affordances, and patient-reported outcomes (PRO). They will be outlined and described in more detail.Attempts to ratify definitions of social media remain problematic, partly because agreements remain elusive. However, the underlying principles of communication, participation, collaboration and user-centeredness are commonalities . Social People dealing with chronic disease are increasingly communicating their health concerns online, with poorer health status, stigmatization, isolation, and disconnection outlined as major reasons . Social Despite the apparent infiltration of the social Web into chronic disease management, there is a challenge for clinicians who wish to use social media in patient management within an evidence-based practice (EBP) framework. The challenge is establishing robust recommendations for their use based on best available evidence, while also taking into consideration both clinician expertise and patient preferences ,8. We thEBP refers to providing the most effective care to patients based on the best available evidence ,8. WhileA four-step process  suggestsThis conventional approach to EBP provides the scope for the current research and is applicable to how social media in clinical practice may be considered. As highlighted above, evidence-based decision making about social media use in chronic disease to improve health outcomes also relies not only on an efficient set of processes but on the best available information and guidance being available to clinicians, researchers and patients to make informed decisions . HoweverOffline approaches to chronic disease management have been commonly “unidirectional”. This means they have emphasized clinical research findings above all else, suggesting research should inform clinical practice . However“Affordance” as a theory may be relatively foreign to health care and more specifically to this research in understanding how social media are used in chronic disease management to affect health outcomes. Notably, this is perhaps because its origins are from perceptual psychology. However, it has been appropriated to human-computer interaction ,12. The Contemplating the importance of the therapeutic affordances of social media has been a key motivation behind our work. We theorize that different social media interactions can precipitate different effects for different people self-managing chronic disease. We anticipate that this approach may help to guide researchers when conducting research projects in this domain and also guide clinicians when deciding whether social media may form a meaningful part of patient management.Patient-reported outcome (PRO) measurement has long been an accepted means to evaluate the success of medical interventions and present evidence of changes to health outcomes. This approach is intended to foster the patient’s perspective of an intervention via outcome measure questionnaires . They prThe chronic disease landscape in particular pushes us to establish valid PRO measurement research methodologies. The breadth of chronic conditions  creates a relative lack of consistency in regards to the measures chosen to assess health outcomes. In chronic disease, PRO tools are generally designed to assess functional limitations, symptoms, health status and health-related quality of life (HRQL) . Common To consider how to generate evidence of health outcomes from social media use we propose a dual method that harnesses both qualitative and quantitative research findings and allows them to be combined.identification and examination of the therapeutic affordances of social media that can help to explain how use of these platforms may underlie favorable health outcomes.The first part of the method focuses on We feel that it is important to examine more closely by what mechanisms social media actually impact health outcomes. As implied in , researcWe have conducted an extensive review that has been published, of empirical and theoretical literature in order to define potentially therapeutic affordances of social media in chronic disease management . The finChronic pain has been selected as a suitable subset of chronic disease self-management for our study purposes. The reason is because while chronic pain is a recognized chronic disease in its own right, it is also a common manifestation or comorbidity of many other chronic diseases. This provides us an opportunity to generalize across various chronic diseases in the clinical setting. This is further highlighted in the same literature review we have conducted, presenting examples of various social Web-based interventions impacting health outcomes in chronic pain related studies .validated and appropriate outcome measurement to reliably assess health outcomes from social media use .The other part of the method focuses on Pertinent to our current research is that social media’s validity as chronic disease management tools is uncertain and still largely untested. Formal measurement of health outcomes is required to actively assess whether social media interventions are effective for improving health outcomes in chronic disease . In our Our literature review identified evidence of self-reported health outcomes and other effects seen from social media use in different chronic disease scenarios. This evidence is presented in full in the review, which has been published elsewhere . To brieWe have decided to explore and utilize a particular instrument of PRO measurement, the Patient-Reported Outcomes Measurement Information System (PROMIS). We are doing this because PROMIS is an item bank system of commonly studied PROs that has been tested and calibrated, demonstrating good reliability and validity across a range of chronic diseases, and shows moderate to strong correlations with other common outcome measures . PROMIS While literature exists outlining health outcomes from social media use, few attempts have been made to investigate how social media operate to meet the specific and individual needs of different chronic disease patients. As more social media uses emerge and further reports are published, researchers will require even more comprehensive methodologies and meta-analytic research designs to synthesize collective knowledge in the quest towards incorporating social media use into EBP ,10. The Our proposed framework represents a research approach for generating evidence of health outcomes from social media use in chronic disease management . Its dessurveyed regarding their perceptions of the therapeutic affordances of social media and how social media use may lead to health outcomes. Then, they undergo an online pilot intervention testing how social media can be targeted (considering these affordances) to better tailor management to individual needs. Finally, PRO from both survey and online intervention are measured using specific item banks from PROMIS to provide empirical evidence of health outcomes. Using standardized PROMIS item banks allows for health outcome questions in the survey and online intervention to be tailored depending on the chronic disease being studied, as well as the primary outcome measure of interest  [The proposed framework follows several steps. First, taking into account the uses and interactions social media affords people with chronic disease, the framework begins with a thorough review of the literature of social media use in chronic disease management . Second,OL, etc) .As outlined, progress to date has expressed the complexities involved and how important it is to be systematic when approaching the study of health outcomes in chronic disease using social media. This is why we feel research would benefit from frameworks detailing the evidence synthesis process. We began our research by examining the chronic disease landscape to gain a deeper knowledge of management both offline and online. This enabled a more focused approach to then explore social media . We empThe global online survey is now closed and recruited 231 participants. In the coming months we plan to present findings of the survey and describe them in a future paper, which is currently beyond the scope of the present paper. We anticipate the results will provide a refined understanding of both how social media affect health PROs and also how people with chronic pain perceive this to be so. The steady flow of contact and interest in the survey, and coherence of preliminary results indicates that participants have been able to follow and make sense of the work. This provides early evidence and validation supporting the theoretical basis of our framework. Unfortunately, this is not expected to be rigorous enough to inform decision-making about social media use in the clinical setting. Future plans of our research are to conduct a pilot intervention in order to further develop and validate our framework and bring us one step close to evidence-based decision making about social media use in chronic disease self management.Standardized outcome measurement lies at the heart of bringing our research together and without it the reliability, validity, and generalizability of this project will be of limited value. For this reason we have selected and discussed PROMIS as the outcome instrument we are using to investigate the PROs in both our survey and also the pilot intervention. PROMIS has many strengths that suit this research. Its item banks (or outcome domains) can be translated into “short-forms” of targeted questions to suit any study, its item banks have been tested amongst large heterogeneous patient cohorts and they have been tested against other commonly used outcomes measures . HoweverConversely, we acknowledge that while initial PROMIS item banks have been shown to display reliability, validity and accuracy when compared to other common outcome measures, longitudinal data is still to be finalized. PROMIS researchers are confident however, that this will also be shown .As research and clinical practice progresses, the challenge for clinicians who plan to use social media in patient management or recommend them as part of individual self-management will persist. This will continue unless research into social media in this domain establishes evidence-based frameworks . While wOur research to date on framework development for conducting research into health outcomes in chronic disease centers on a deeper investigation of the therapeutic affordances of social media in this context. Second, we emphasize the importance of valid and standardized PRO measurement. Together, affordances and PRO form the basis of a novel methodological approach for how to generate evidence of health outcomes from social media use, as well as clinical recommendations for evidence-based decision-making about social media use in chronic disease management. With further work and collaboration, this research method and framework may aid research design for social media interventions and allow for greater improvements in health outcomes to be recognized.


Primary mediastinal extraskeletal Ewing’s sarcoma (EES) is quite rare. To the best of our knowledge, only five cases have been reported. Given the paucity of data, there is consequently no optimal treatment strategy available. The current study presents the case of a 51-year-old female with unresectable EES of the posterior mediastinum. Chemoradiotherapy achieved near complete remission without severe side-effects. The literature associated with EES is also reviewed. The present case highlights the possibility of the diagnosis of EES for a mediastinal mass. Chemoradiotherapy may be a good option for unresectable cases. In the future, large-scale collaborative clinical trials should be initiated to provide an improved understanding of the characteristics of EES and the best treatment strategy. Extraskeletal Ewing’s sarcoma (EES)/primitive neuroectodermal tumor (PNET) is a rare entity, accounting for 15% of all Ewing’s sarcomas . PatholoPrimary mediastinal EES is extremely rare. The present study describes the case of a middle-aged female with mediastinal EES who received sequential chemotherapy and radiotherapy and achieved a marked response. Written informed consent was obtained from the patient. Additionally, the associated studies on EES are also reviewed. Further studies are required to establish the standard treatment strategy for EES. Written informed consent was obtained from the patient.On December 25, 2012, a 51-year-old female presented with intermittent chest pain that had been apparent for one year. The physical examination was unremarkable. The patient’s performance status was 1 according to an Eastern Cooperative Oncology Group (ECOG) evaluation .Laboratory investigations revealed a normal complete blood cell count, coagulation routine and serum biochemical profile. The thoracic computed tomography (CT) scan revealed a huge mass in the posterior mediastinum, with invasion of the esophagus, descending aorta and right pulmonary artery . On Dece2). Following one cycle of chemotherapy, the tumor rapidly decreased in size  . CD99-po margins .et al (For a long period of time, EES had been regarded as exhibiting no significant differences to osseous Ewing’s sarcoma. A study by Applebaum et al  was the et al (et al recommended an intense strategy of administering vincristine, Adriamycin, cyclophosphamide and actinomycin D alternately with IE (et al held the view that the use of platinum-based chemotherapy should be considered (EES is highly aggressive. Local relapse and distant metastases are frequent. Multidisciplinary measures are important for improvements in survival. A retrospective study of 24 cases of EES showed a 61% five-year overall survival rate following multimodality therapies . In a stet al , two caset al . Extendeet al . However with IE . Tao et In terms of local control, no random trials have yet been initiated to demonstrate whether surgery combined with radiotherapy is superior to surgery alone. However, complete resection plus radiotherapy is known to have an advantage over radiotherapy. For unresectable cases or tumors that cannot be removed completely, radiotherapy with a radical dose of 50–60 Gy should be performed .In conclusion, the present study describes a relatively rare case of primary mediastinal EES/PNET that could not be resected. Sequential chemotherapy and radiotherapy achieved a favorable response with mild side effects. We propose that dacarbazine and pirarubicin chemotherapy should be considered for unresectable EES/PNET cases for its beneficial effect, and chemoradiotherapy is a good treatment option for such cases. Successful published protocols using collaborative clinical trials will aid in the development of a standard treatment strategy.


Many aspects of clinical management of Lyme neuroborreliosis are subject to intense debates. Guidelines show considerable variability in their recommendations, leading to divergent treatment regimes. The most pronounced differences in recommendations exist between guidelines from scientific societies and from patient advocacy groups. Assessment of the methodological quality of these contradictory guideline recommendations can be helpful for healthcare professionals.Systematic searches were conducted in MEDLINE and databases of four international and national guideline organizations for guidelines on Lyme neuroborreliosis published from 1999–2014. Characteristics  and key recommendations were extracted from each guideline. Two independent reviewers assessed the methodological quality of each guideline according to the Appraisal of Guidelines for Research and Evaluation II (AGREE II) tool. AGREE II scores from guidelines developed by scientific societies and from patient advocacy groups were compared across domains.n = 6 were developed by scientific societies and n = 2 by patient advocacy groups. Agreement on AGREE II scores was good . Three guidelines, all from scientific societies, had an overall quality score of ≥ 50 %. Two of them were recommended for use according to the AGREE II criteria. Across all guidelines, the AGREE II domain with the highest scores was “Clarity of Presentation” ; all other domains had scores < 50 % with the domain “Applicability” having the lowest scores . Guidelines developed by scientific societies had statistically significantly higher scores regarding clarity of presentation than guidelines from patient advocacy groups (p = 0.0151). No statistically significant differences were found in other domains.We identified eight eligible guidelines of which Current guidelines on Lyme neuroborreliosis vary in methodological quality and content. Health care providers and patients need to be aware of this variability in quality when choosing recommendations for their treatment decisions regarding Lyme neuroborreliosis. No statement can be given on quality of content and validity of recommendations, as these issues are not subject to assessment with the AGREE II tool and are prone to individual interpretation of the available evidence by the corresponding guideline panels. To enhance guideline quality, guideline panels should put more emphasis on linking recommendations to the available evidence, transparency in reporting how evidence was searched for and evaluated, and the implementation of recommendations into clinical practice.The online version of this article (doi:10.1186/s12883-015-0501-3) contains supplementary material, which is available to authorized users. Borrelia burgdorferi sensu lato. Lyme disease can affect multiple organ systems, common manifestations are dermatologic manifestations (e.g. erythema migrans), Lyme arthritis or Lyme neuroborreliosis [Lyme disease is a tick-borne infectious disease caused by the spirochete bacterium reliosis .Diagnosis of Lyme disease and Lyme neuroborreliosis is usually based on consensus-derived case definitions . Tiered Many aspects of disease management are subject to controversy, sometimes referred to as the ‘Lyme wars’ . DespiteThese different opinions resulted in different, partially contradicting guideline recommendations for Lyme neuroborreliosis. One example is duration of treatment, for which three guidelines Furthermore, guidelines from patient advocacy groups recommend treatment with multiple antibiotic agents and additional adjuvant agents (e.g. hydroxychloroquine) simultaneously , 8, wherThese divergent, partly contradicting recommendations lead to uncertainty and doubt in patients and healthcare providers. Evidence from high-quality studies is scarce for treatment of Lyme neuroborreliosis and of limited methodological quality. Hence, contradicting guideline recommendations may exist for conditions with limited evidence, and individual opinions and experiences that are not evidence-based gain increasing influence on guideline recommendations. This, in turn, emphasizes the need for transparent and comprehensible methods for developing guidelines as a sound basis for clinical decisions. To investigate methodological rigour of development of these guidelines with contradicting recommendations, we performed a systematic review focusing on the quality of guidelines for Lyme neuroborreliosis.http://www.guideline.gov/), the International Guideline Library of the Guidelines International Network , The National Institute for Health and Care Excellence  and the Arbeitsgemeinschaft der Wissenschaftlichen Medizinischen Fachgesellschaften  from 1999–2014  and the databases of the National Guideline Clearinghouse  extracted the data, whereas a second reviewer (IT) checked the first reviewer’s data for completeness and accuracy. Differences in opinion were resolved through discussion. Data was collected on year of publication, country, type of producing organization (scientific society or patient advocacy group), as well as key recommendations for diagnosis and treatment of Lyme neuroborreliosis.Methodological quality was assessed using the Appraisal of Guidelines Research and Evaluation II (AGREE II) instrument , which cFrom the rating, domain scores were calculated as described by AGREE II by the following formula: (obtained score - minimum possible score) / (maximum possible score - minimum possible score) The maximum possible score was: maximum possible score x number of items in domain x number of appraisers. The minimum possible score was: minimum possible score x number of items in domain x number of appraisers. There is no defined threshold for the domain scores of the AGREE II tool to make a distinction between high quality and low quality guidelines, albeit some authors consider domain scores <50 % pragmatically as low quality , 17.According to the AGREE II method, each guideline is rated as either ‘recommended’, ‘recommended with modifications’ or ‘not recommended’ taking into account the appraisal items considered in the assessment process.If manuals of methodology were accessible for the development process of single guidelines, their content was considered when performing the AGREE II assessment.As differences in recommendations regarding Lyme neuroborreliosis from different panels were of interest, we compared domain scores between them to investigate whether differences may be paralleled by differences in quality. Additionally, we investigated whether guideline quality was associated with year of publication as an explorative analysis, as guideline quality might increase over time and availability of rating instruments e.g. AGREE II. Statistical comparisons between scores were performed with two-sided t-test. Correlation calculations were performed with Spearman’s rank correlation coefficient. Statistical analyses were conducted with R and Prism 4.0b for Macintosh , 19.As this study does not include any patient data, no ethical approval or consent was needed.We identified 177 records; of which 168 were excluded  and in domain 3 .Inter-rater agreement was high according to Cohen’s weighted kappa  and Lin’s concordance correlation coefficient . Quality of included guidelines was <50 % in most AGREE II domains; the only domain with a score >50 % was clarity of presentation . Other domain scores did not correlate with ‚Overall guideline assessment’ (not shown). Year of guideline publication did not correlate with any domain score (Table Scores for domain 3 (‘Rigour of development’) correlate with ‚Overall guideline assessment’ , differences in other domains were not statistically significant. Overall guideline assessment was not significantly different for guidelines from scientific societies and from patient advocacy groups (p = 0.4534, Table Guidelines developed by scientific societies scored different results than guidelines developed by patient advocacy groups. Results are shown in Table Methodological quality of existing guidelines for treatment of Lyme neuroborreliosis is limited and shows considerable variability across individual guidelines identified. Quality assessments of many domains were unsatisfactory according to the AGREE II tool. Quality scores partly differed between guidelines developed by scientific societies and guidelines developed by patient advocacy groups, with statistically significant differences in clarity of presentation. Interestingly enough, year of publication did not correlate with any of the quality score, albeit this could be to the low sample size. Due to language limitations, we may have missed guidelines which could not be assessed in this review.Discussion on items that were rated >1 point apart by the two raters had to be performed in all domains, mostly because information on single items was scattered through individual guidelines and was difficult to gather. Such disagreements could easily be cleared in the consensus discussion.Quality assessment of guidelines with the AGREE II tool covers issues of methodological rigour of guideline development, applicability and transparency. No statement can be given on quality of content and validity of recommendations drawn from the available body of evidence, as these issues are not subject to assessment with the AGREE II tool and are prone to individual interpretation of the available evidence by the corresponding guideline panels. However, as these issues cannot be assessed directly, the process of selecting evidence and linking of recommendations to supporting evidence in an individual guideline should be transparent and comprehensible, which was insufficient in most included guidelines. The importance of methodological rigour of development for guideline development is illustrated by our finding that the domain score for ‘rigour of development’ correlated statistically significantly with the overall guideline assessment.Whereas clarity of presentation was acceptable in some guidelines, more emphasis on rigour of development and especially systematic search methods, criteria for selecting evidence and linking of recommendations to supporting evidence as well as addressing applicability issues could lead to improved quality and better usability of guidelines for management of Lyme disease. The credibility of a guideline is diminished when it is not clear whether systematic search methods were used to gather available evidence or how recommendations are linked to supporting evidence.Linkage of recommendations to the available evidence could be improved by implementing summary of findings tables according to the GRADE approach . ReadersApplicability of guidelines could be improved by providing advice or tools on how the recommendations can be put into practice, e.g. short versions, clinical pathways or clear accentuation of key recommendations. Potential resource implications could be applied, e.g. the need for specialized laboratories on serologic testing.Monitoring and auditing criteria were not mentioned in any guideline, although it may be difficult to determine such criteria for treatment of Lyme neuroborreliosis. Included guidelines did not intend to provide monitoring criteria but rather stated that the intention was to provide support and guidance for clinicians and patients in treatment decision.In the light of contradicting recommendations in a field with limited evidence, it seems necessary to provide guidelines which are transparently developed and provide evidence based recommendations for clinicians. Individual guidelines with highest overall scores on guideline quality, which were also the two guidelines recommended according to AGREE II ratings were the EFNS and the AAN guidelines.These two guidelines that were ‘recommended’ according to AGREE II had especially high scores in the domains ‘rigour of development’ and ‘clarity of presentation’ compared to the guidelines that were not recommended. All other guidelines, including the current guideline from the DGN, were not rated as ‘recommended’.Both guidelines stem from scientific societies. None of the guidelines from patient advocacy groups were rated as ‘recommended’. Guidelines from patient advocacy groups endorsed extended antibiotic treatments longer than 28 days, whereas guidelines from scientific societies recommended antibiotic treatments for 14−28 days. Differences in these recommendations might partly be explained by differences in methodological quality of guidelines.Clinicians and patients faced with treatment decisions on Lyme neuroborreliosis can use the provided quality assessment of the available guidelines to choose individual guidelines showing high methodological quality according to the AGREE II tool.


Prenatal records are potentially powerful tools for the translation of best-practice evidence into routine prenatal care. Although all jurisdictions in Canada use standardized prenatal records to guide care and provide data for health surveillance, their content related to risk factors such as maternal smoking and alcohol use varies widely. Literature is lacking on how prenatal records are developed or updated to integrate research evidence. This multiphase project aimed to identify key contextual factors influencing decision-making and evidence use among Canadian prenatal record committees (PRCs), and formulate recommendations for the prenatal record review process in Canada.Phase 1 comprised key informant interviews with PRC leaders across 10 Canadian jurisdictions. Phase 2, was a qualitative comparative case study of PRC factors influencing evidence-use and decision-making in five selected jurisdictions. Interview data were analysed using qualitative content analysis. Phase 3 involved a dissemination workshop with key stakeholders to review and refine recommendations derived from Phases 1 and 2.Prenatal record review processes differed considerably across Canadian jurisdictions. PRC decision-making was complex, revealing the competing functions of the prenatal record as a clinical guide, documentation tool and data source. Internal contextual factors influencing evidence use included PRC resources to conduct evidence reviews; group composition and dynamics; perceived function of the prenatal record; and expert opinions. External contextual factors included concerns about user buy-in; health system capacities; and pressures from public health stakeholders. Our recommendations highlight the need for: broader stakeholder involvement and explicit use of decision-support strategies to support the revision process; a national template of evidence-informed changes that can be used across jurisdictions; consideration of both clinical and surveillance functions of the prenatal record; and dissemination plans to communicate prenatal record modifications.Decision-making related to prenatal record content involves a negotiated effort to balance research evidence with the needs and preferences of prenatal care providers, health system capacities as well as population health priorities. The development of a national template for prenatal records would reduce unnecessary duplication of PRC work and enhance the consistency of prenatal care delivery and perinatal surveillance data across Canada. Prenatal medical records have long been considered important tools for the clinical management of obstetrical care, in addition to being vehicles for quality assurance, compensation and medico-legal documentation . As a clIn Canada, health care is constitutionally under the authority of provinces and territories. Maternity care delivery models thus vary across Canada in accordance with local resources, regulatory, educational, and population characteristics . HoweverDespite the widespread adoption of standardized prenatal records in maternity care, literature on prenatal records is virtually non-existent. The only paper found comparing general prenatal record content dated 1991) revealed that most American prenatal records contained items of traditional medical-obstetrical significance , as opposed to more “contemporary” issues such as patient education and social risk factors such as smoking and drinking 991 revea. SimilarBroader literature on the development of clinical management tools acknowledges that guideline recommendations are influenced not only by scientific evidence but also by clinical experience, expert opinion, patient preferences and feasibility -20. GrouWe found only one study that examined decision-making processes used during expert group development of clinical guidelines for population-wide application. This study, by Dobrow and colleagues, yielded a conceptual framework for “context-based evidence-based decision-making” to identify contextual influences on evidence use in the “shift from an individual-clinical to a population-policy level” ,29. TheiAs prenatal records play a central role in guiding and monitoring the delivery of prenatal care in Canada, it is critical that the decision-making processes underlying their content are made more transparent. This paper describes a multiphase, knowledge translation project that aimed to 1) describe and compare how prenatal records are reviewed and updated across different Canadian jurisdictions; 2) identify contextual factors influencing decision-making and evidence use during the process of prenatal record review; and 3) formulate recommendations for optimizing the prenatal record review process in Canada.The project involved two data collection phases and a dissemination phase  from all 10 Canadian jurisdictions that had developed their own prenatal record. PRC leaders whose names were in the public domain were recruited directly via telephone and/or e-mail; the others were solicited via letters or e-mails forwarded through the organizational bodies responsible for prenatal record review. Interview questions were informed by Dobrow and colleagues’ conceptualization of the internal decision-making context and explored PRC views of the purpose of prenatal record review; PRC membership, mandates, and resources available to conduct the committee’s work; methods used to access and review research evidence; and strategies used for obtaining committee consensus on proposed modifications to the prenatal record.The main research question for Phase 1 was: Interviews were conducted in English via telephone by two trained research assistants except for two conducted in French by SS. Interviews were audio-taped and transcribed verbatim; French transcripts were professionally translated into English prior to analysis. Descriptive content analysis of the transcripts was conducted to summarize the prenatal record review process within each jurisdiction and to identify commonalities and differences across jurisdictions. Two investigators  reviewed the Phase 1 transcripts and extracted all pertinent information related to the main interview questions. The findings were then summarized and tabulated in a matrix, and double-checked for accuracy by a research assistant who reread all the Phase 1 transcripts. We also retrieved paper copies of each jurisdiction’s prenatal record as well as any accompanying user guides or assessment forms, to enable comparisons of specific content across jurisdictions and to facilitate understanding of interviewee’s descriptions or explanations related to the use of the form.What internal and external contextual factors influence evidence use during the prenatal record revision process? We used a qualitative multiple-case design with embedded units of analysis  or even all of Canada…I think that there would be room for supporting the application of evidence for each of the provinces by some national group…so that the research does not have to be done on every question locally in 10 provincial jurisdictions” (Case 5).Given the burden of evidence reviews, PRC members from several cases suggested having a national initiative to facilitate updating of their prenatal records rather than having to “reinvent the wheel” during each review:“…committee members were experts within their own field…you know, specialists, general practitioners so what they did is responded to the recommendations based on what they thought was practical” (Case 1). All five PRCs used informal means  to arrive at consensus about revisions to the prenatal records . In some PRCs, this allowed decision-making to be influenced by professional hierarchies among group members: “…the majority of changes were made by a small elite group of people, you know, the obstetricians and a few specialists and physicians with very little input from other people…” (Case 5).A preponderance of clinical experts (primarily physicians) and relative absence of methodologists on the PRCs contributed to a focus on the usefulness, acceptability and feasibility of research evidence, rather than debates over evidentiary quality: ds Table , which m“very helpful layer” to practical deliberations of prenatal record content: “…sometimes they have a perspective of, ‘Well this seems very reasonable, I know, it’s because I’ve had a baby” (Case 2). Two PRCs invited experts in aboriginal care on their PRC to “ensure questions related to issues such as substance abuse were asked in a sensitive manner…political-wise it was very important to be seen that we were approaching the (native women’s groups) and getting information from them” (Case 3).PRC composition also influenced the nature of the evidence brought to the decision-making table, in addition to power dynamics. Participants from two PRCs noted that their committee was under-represented by nurses and midwives, given their expanding role in the delivery of prenatal care. Only one PRC involved patient representatives, who were seen as adding a “So one of the things that we’re looking at in our prenatal program, is how we’re going to be able to collect some of the variables on the prenatal record for further analysis and research and program planning. But in that light, we’re very, very cognizant that first and foremost we need it to be a clinical tool and a communication tool and it’s not there for data mining… Ensuring the committee has a clinical focus as its number one priority just has to be a principle” (Case 1).PRC views of the primary function of prenatal records determined the focus of the review process and what types of evidence were considered in decisions about prenatal record items. Members from all five PRCs reported that the main purpose of the prenatal record was to guide and communicate clinical care:“We wanted [the prenatal record] to collect as much, what we thought would be useful information and do it in such a way that when we had such a transitory healthcare provision…that it would be relatively easy and straightforward to get this information and to pass it on” (Case 3).The importance of the prenatal record as a communication tool was recognized in particular by PRCs in jurisdictions with multiple  or more transitory, rural prenatal care providers:“Over the last few years we’ve been trying to get up a perinatal database, look at outcomes and all various different factors. And I think that played a big part in some of the questions that we sort of wanted to ask… I mean we even looked at second hand smoking as, is this relevant data to be collecting?” (Case 3).The prenatal record’s priority function as a clinical  tool was also reflected in the relative absence of population health and data management experts on the PRCs. Nonetheless, in several cases PRC decision-making about the specific content of the prenatal record forms took into account the prenatal record’s current or future role as a source of data for perinatal health surveillance:“a more planned and integrated approach…with the same wording in the prenatal record and the maternal child data base” (Case 4). Clinical and population-health functions of the prenatal record sometimes required different types of information, highlighting the complexity of decision-making around the specific items:“Our [Public Health] individuals wanted to use what primarily was reported in the literature for indicating morbidity such as number of drinks per week. And yet our clinical individuals would say, for us on the intervention perspective, the number of drinks per week is not the issue; the issue is, is there any alcohol consumption and what’s the maximal alcohol consumption on any one occasion” (Case 1).Participants described the challenge of ensuring that prenatal record questions were appropriate for both individual patient care and population health monitoring, noting the need for “I mean smoking is always a challenging variable to look at…so we’re constantly trying to improve the data based on what we get when we search for things, and also based on what the coders actually tell us they’re able to collect” (Case 4).Poor compliance with prenatal care provider documentation of such “sensitive” questions as smoking or alcohol use was noted as a particular problem for perinatal data base managers:PRCs that were developing electronic versions of their jurisdiction’s prenatal record faced the additional challenge of matching paper and electronic copies of the forms. Nevertheless, participants acknowledged that electronic prenatal records facilitated information-sharing among clinicians, expanded opportunities for linking with perinatal databases and created possibilities for tailoring forms to the particular monitoring needs of individual patients.“There was some significant discussion on the committee about this [incorporating suggestions about smoking cessation], and the reason is the rather dismal performance of most of the attempts to try and secure smoking cessation. I mean unfortunately, it’s all very well to identify smokers, but there’s not a whole lot that has been demonstrated to work that results in smokers decreasing their smoking even during pregnancy” (Case 5).PRCs resorted to expert opinion when research evidence needed to inform decisions about modifying items  was viewed as negative, equivocal or lacking; or was in conflict with personal opinions or experiences of committee members. For example, one small PRC decided not to add more assessment questions or intervention guides for maternal smoking based largely on members’ beliefs that such strategies were ineffective, despite the availability of strong research evidence to the contrary:“When we try and get evidence to make these kinds of decisions, we are doing it each as individuals and there’s no question that there is a subjective angle to that…we pick and choose things that we like…” (Case 5). Decision-making around items related to alcohol use appeared particularly complex for the PRCs due to mixed research findings about the impact of social drinking:“There was some discussion around the alcohol question, ‘cause there certainly is, you know, different opinions in the literature around the effects of alcohol, and how much alcohol can cause damage. But there certainly was agreement that certainly you would want to be promoting ‘no alcohol’ as best” (Case 1).Another participant from the same PRC noted that relying on a few volunteers to research the literature was a potential source of bias: External contextual factors  that influenced PRC decisions about prenatal record content included concerns about user buy-in; health system capacities; and pressures from public health stakeholders. Table “Right now (the form) sits with 72 questions that you’re supposed to ask in the first  interview. Well, that’s not humanly possible…”(Case 1). Several participants noted that a more comprehensive, detailed prenatal record based on the latest evidence might not receive buy-in from prenatal care providers:“…a committee themselves can be sold on the evidence and the requirement for information on the prenatal record. The challenge would be how you best sell that to the practitioners when you’re implementing the record because of the reaction to the increased time and resources required to interview the mother and document the information and then knowing what to do with that information” (Case 1).PRCs considered the acceptability of prenatal record items to the clinicians who would be completing the forms, given resource constraints within the practice setting: “The page was already pretty full, so if we wanted to add something, we thought well we don’t want a multi-page form because people won’t use it… what can we eliminate so that we can add some stuff?”(Case 5). Considerations of item relevance involved weighing evidence for efficacy against considerations of feasibility given the PRC’s jurisdictional context:“If I think back to the smoking one…it was space on the paper if I remember correctly, and you know, “And is there a value in doing it?” … “Is it a ‘nice to know’, or a ‘need to know’?” And is it applicable in this practice setting? Is it a big issue?” (Case 2).To address potential resistance to change, all five PRCs solicited feedback from clinician stakeholders on proposed revisions to the forms, through individual consultations or small-scale piloting of the revised prenatal record forms. Participants from four PRCs also stated that they explicitly aimed to keep their prenatal record short and “user-friendly”, necessitating committee debate about the relative importance of prenatal records items: “When we talked to people who were using the current prenatal record, and wondering why these particular points were not sort of not being checked off or asked about…they either felt uncomfortable about the question or they didn’t know what to do if the [answer to the] question was positive…” (Case 5).Several participants raised the particular issues of clinician discomfort addressing complex social issues such as prenatal substance abuse, and high rates of non-compliance with documentation of maternal smoking and alcohol use items:“We found that sometimes the physicians were uncomfortable about asking those questions…so we introduced the [X] questionnaire, which is completed by the mother herself (Case 1). Members from two PRCs with large aboriginal populations and high rates of maternal substance use also addressed the acceptability of the prenatal record items to pregnant women, acknowledging the potential negative impact of substance use questions on patient-provider trust, patient privacy, and child protection involvement:“So if there was concern by the mother that this information [substance abuse question]…may lead to her social service referral, or child welfare referral and her loss of the child…it certainly could impact on how she answers the questions.” (Case 3).One PRC created a self-report questionnaire as a companion document for their prenatal record, to facilitate collection of personal information: “Social economically…we had to be very aware on the committee of what resources were available in the big center, as well as the smaller centers” (Case 3). Clinician skills and the availability of local smoking cessation or addiction services were important health system considerations when debating whether to include more screening questions or intervention prompts related to smoking/alcohol use. For example, Case 4 added items to prompt a more detailed assessment of smoking patterns given their confidence that health system resources were available for referrals: “…supports are provided throughout the province where people can get support and information and smoking cessation aids for free so that we could tag into that information when we’re supporting people to reduce or quit smoking…” Yet they decided not to add a screening tool for alcohol use partly in consideration of the lack of alcohol treatment programs for pregnant women in their jurisdiction: “Probably the biggest challenge with all of that for care providers is…figuring out what do we do now when someone says, “Yes, I’m using alcohol,” and the approach to that and the resources…”PRC decisions about the applicability of evidence also considered the availability of health system resources to implement prenatal record recommendations at the population level: “There was quite a vocal group from the African Canadians saying we weren’t paying enough attention to things like Sickle Cell” (Case 4). PRCs from jurisdictions with high aboriginal populations were under particular pressure to improve prenatal record screening for maternal substance use:“I mean we do have very high rates of smoking, very high rates of alcohol abuse and substance abuse here in [X], for a small population. And there’s always been outside pressure to…ask the right questions and provide interventions that would help that. A lot of government and non-government sort of organizations do give their input into what we should ask, or why we should be asking these questions” (Case 3).Local public health priorities and external pressures from public health stakeholders or community groups were additional influences on whether to modify prenatal record items and what questions to prioritize in the competition for prenatal record space: “And the smoking issue, I can’t remember what research we had but…like the old prenatal record just said, you know, how many cigarettes you have in a day. This time it was more like we have to take a look all the time at harm reduction so how much were they smoking before they got pregnant” (Case 4).In several cases, the public health trend towards harm reduction approaches to substance use influenced the specific content and wording of smoking and alcohol use items:“There were things like the T-ACE screen brought forward for looking at people’s addiction, that type of thing. I remember taking a look at some of those research things, but decided that in [Province X] these are the things right now with this prenatal record, we need to focus on obesity”.For the case that opted not to enhance their prenatal record’s alcohol screening questions despite evidence-based recommendations, the PRC ultimately considered whether the limited prenatal record space would better serve more pressing public health priorities:Nine members of the research team and 26 invited stakeholders participated in the dissemination workshop. Invitees included five representatives of national professional associations ; six stakeholders from different branches of Canada’s federal health agency or national maternal child health organizations; three representatives from national or provincial perinatal health surveillance programs; and several nationally-known perinatal or health promotion researchers. Based on group discussions, participants’ written feedback and ratings of the draft recommendations, we modified and synthesized the original nine recommendations for strengthening the prenatal record review process into five main messages. In brief, our recommendations address: involving multi-disciplinary stakeholders in the revision process; adopting the explicit use of decision-support tools, and processes to support the revision process; developing a national template of evidence-informed changes that can be used by provinces; considering both clinical and surveillance functions of the prenatal record; and developing a dissemination plan to communicate changes to the prenatal record. The recommendations and related key points derived from the interview and dissemination workshop are summarized in Table This is the first study we are aware of to examine decision-making processes underlying the content of standardized prenatal records. Our use of Dobrow and colleague’s framework for context-based evidence-based decision-making  helped eIn our findings and proposed recommendations, we highlight several important issues related to the development and updating of prenatal records. First, input from all relevant stakeholders and the use of systematic strategies for reviewing the extant literature and guiding group decision-making would help reduce potential biases in the prenatal record review process. We found little documentation of PRC membership or processes available in the public domain, raising questions about the transparency and/or accountabilities of PRCs across Canada. The PRCs we examined were clinician-dominated with sometimes little (or no) representation or supports from methodology experts to help evaluate research evidence, or other persons who could advise on other functions  of the prenatal record. Previous studies of evidence use by clinical guideline groups have found that clinician-led decision-making favours professional values, expert opinion, and practical experience over research evidence ,24,34,35Findings underscore the importance of adequate resourcing of the work of PRCs, to allow for a systematic evaluation of the updated literature and adequate time for consultation and decision-making processes. Larger-scale guideline initiatives that involve stakeholders from practitioners to policy makers require sufficient time, commitment and resources to ensure effective communication with all concerned parties and to support their implementation . We founAn additional key issue raised by our study was the relative lack of attention paid to the potential population health implications of prenatal record items, such as how they could be used to lever improvements in wider health care system characteristics  that might enhance the health of future pregnant women and infants. Although providing better care for population sub-groups such as Aboriginal women was identified as a consideration by some PRCs, none of the participants expressed PRC intentions to lobby for improvements to the health care system. Rather, limitations of the care system were identified as reasons for not making changes on the prenatal record forms. Perhaps this orientation of PRCs is related to a perception of their mandate, the large amount of work involved in reviewing many items, and the mix of items on the forms .PRC composition and decision-making around prenatal smoking/alcohol use items revealed a primary focus on the prenatal record as a tool for individual clinicians. The use of prenatal records as a legal documentation tool with the potential implications of missing data for clinician accountability may have accounted in part for PRC reluctance to include recommendations that were not feasible within the context of local health care resources .In contrast to concerns about prenatal record use by clinicians, the role of the prenatal record as an important data source for perinatal health surveillance and the consequences of absent or poorly completed items for population health researchers  appeared to be of less direct concern to PRCs. There also seemed to be a tension in deliberations about using the record for clinical versus population level goals. The issue of under-reporting of stigmatizing behaviors such as alcohol use during pregnancy is well-documented , and womAlthough not explicitly identified as ethical considerations by participants, several domains of ethical issues were raised during interviews: a) professional ethics, i.e., how the prenatal record forms might aid clinicians to meet their professional obligations; b) patient ethics, i.e., how the inclusion of items on the forms might assist clinicians to optimize benefits and minimize harms to patients; and, c) population health ethics, i.e., how characteristics of the population, varying patterns of service access across population sub-groups and underlying factors that influence these access patterns should be taken into consideration in decisions about modifications to prenatal record items. For example, the underlying ethical dilemma of minimizing harms may have been a consideration in the integration of more comprehensive assessment questions about maternal smoking and alcohol use. A key barrier for discussing and documenting alcohol use in the perinatal period is not only clinicians’ preoccupation with the dilemma of what to do with a positive answer (prompting the need to identify multilevel support and treatment for substance use and to engage in evidence-based and compassionate brief interventions), but also potentially adding to women’s burdens by asking them about something that is highly stigmatized and for which child protection may become involved . The neeThere was a surprising absence of discussion about governance and financing models in the Canadian health care system, and how these affected PRC choices about items for inclusion on the prenatal record forms. A number of participants made reference to issues of time constraints, busy clinicians and provider mix, which their PRCs had considered in reviewing practical limitations to the addition of assessment questions. However, underlying health system drivers such as fee for service models were not addressed. Clinician reluctance to “open Pandora’s box” by asking about complex social issues during the limited time allocated to prenatal visits may have been an unspoken concern of the PRCs. Fee for service funding models may have broader implications for PRC discussion about pragmatics, the length of the prenatal record forms, trading off concerns about user buy-in with public health priorities and sources of clinician resistance to change. The potential influence of these more fundamental health system issues on the prenatal record review process needs to be made more transparent.Finally, the impact of prenatal records as knowledge translation tools ultimately rests on individual clinician adherence to the record’s guidelines, which will vary according to such factors as their perceived relevance or degree of sanctions for non-compliance with documentation on the forms . The incThere were several methodological limitations to this study. The number of participants was small and while purposively sampled, may or may not reflect the perceptions of other committee members. We also examined PRC decision-making using a retrospective, cross-sectional case study design. In some jurisdictions the prenatal record review process had taken place months or years prior to our interviews, therefore some participants may have had difficulty recalling specific details of their committee’s decision-making process. There has also been a time lapse between the interviews and publication of our findings, and contextual factors related to PRC decision-making may have changed in some jurisdictions since our data collection . However, a re-review of all Canadian prenatal records conducted in January 2014 revealed that only one of the 10 Canadian jurisdictions involved in our project had modified their prenatal record content since 2009, and Canada still lacks a national process for prenatal record review. Future research should consider prospective designs using non-participatory observation of PRC decision-making ,44, as wUniversal prenatal records are potentially powerful tools for the translation of best-practice evidence into routine prenatal care, and thus merit rigorous processes for ensuring the quality of their content. Decision-making related to prenatal record content needs to consider the multiple functions of the tool, and involves a negotiated effort to balance evidence-based recommendations with the needs and preferences of prenatal care providers, health system capacities as well as population health priorities. Broad stakeholder participation, systematic evaluation of the research evidence and formal strategies for arriving at group consensus are needed to reduce potential biases in PRC decision-making. The development of a national template for prenatal records could reduce duplication of work by PRCs in different Canadian jurisdiction and enhance the quality and consistency of prenatal care delivery and perinatal surveillance data across Canada.


With rising health care costs and the diversity of scientific and clinical information available to health care providers it is essential to have methodologies that synthesize and distill the quality of information and make it practical to clinicians, patients and policy makers. Too often research synthesis results in the statement that “more and better research is needed” or the conclusions are slanted toward the biases of one type of stakeholder. Such conclusions are discouraging to clinicians and patients who need better guidance on the decisions they make every day.consensus panels, and appropriateness panels. These types of panels are utilized by organizations such as the National Institutes of Health, the Institute of Medicine, RAND, and other organizations to provide clinical guidance. However, there is a need for a more cost effective and efficient approach in conducting these panels. In this paper we describe both types of expert panels and ways to adapt those models to form part of Samueli Institute’s Scientific Evaluation and Research of Claims in Health Care (SEaRCH™) program.Expert panels are one method for offering valuable insight into the scientific evidence and what experts believe about its application to a given clinical situation. However, with improper management their conclusions can end up being biased or even wrong. There are several types of expert panels, but two that have been extensively involved in bringing evidence to bear onclinical practice are Expert Panels provide evidence-based information to guide research, practice and health care decision making. The panel process used in SEaRCH seeks to customize, synthesize and streamline these methods. By making the process transparent the panel process informs decisions about clinical appropriateness and research agenda decisions. One source of information is scientific evidence derived from research studies. However, since a single study is seldom definitive, a systematic review of the numerous studies published in the literature is preferable. Systematic reviews of the literature provide a method of assessing the quality of each individual study , 2, as w©) . However©) , 7. The ©) . The sitOne approach that has emerged to overcome this problem has been the use of expert panels (EP). While in systematic reviews the opinion of experts is considered at the bottom of the evidence hierarchy, in expert panels, which combine evidence and clinical acumen, the opinions are made transparent and subjected to critical appraisal. This overcomes the major objection to the opinion of experts vs. systematic reviews and has the additional value of allowing the introduction of information closer to clinical practice and a better interface with research evidence.Samueli Institute uses the following expert panels: a Clinical Expert Panel (CEP), and a Research Expert Panel (REP), with others designed for policy (PoEP) and patient expert panel (PaEP) decisions, in development. These types of panels form a part of the Scientific Evaluation and Review of Claims in Health Care (SEaRCH™) process.  They draSamueli Institute’s EP processes have drawn from three methods developed over time to deal with the application of evidence to practice. These are the National Institutes of Health Consensus Development Conference (NIH CDC), the Institute of Medicine (IOM) report process and the RAND Expert Panel process. Because we draw on these three methods they will be described briefly in what follows.The National Institute of Health (NIH) consensus development conference method for resolving evidence judgment issues was begun in 1977 by NIH as a method by which the scientific community could bring relevant research to bear on the quality of health care –10. The NIH panels focused largely on questions of efficacy and safety , and areThere are, however, several challenges which limit the usefulness of the NIH Consensus Development Panels. If the focus is only on those issues that can be resolved by scientific evidence, it is necessarily confined to those issues with scientific support. Unfortunately, within the field of health care, many of the most problematic issues cannot be resolved by evidence alone. In such a situation the NIH CDC is forced to report simply on the state of the science. In addition, the judgment processes used for the final conclusions are also done behind closed doors and so are not completely transparent.The NIH panel places a premium on research findings, and gives less weight to clinical experience or acumen . This undermines the credibility of the NIH panels with the very persons it seeks to influence—practitioners and patients. Additionally, the panels favor largely scientific  in both the clinical and related aspects of the topic , 9. RelaAnother limiting factor of NIH panels is that they are consensus-based, and while minority reports are possible, participants are strongly encouraged to agree on assessments and recommendations. Such consensus panels may be more prone to bias from dominant panel members . The recIn situations where the evidence is insufficient to derive consensus statements or as part of the consensus panel for clinical practice, a state of the science special panel can be created. This panel is also further limited by the relative lack of transparency and specificity related to gap assessment and research recommendations. Little systematic information is given on how group decisions are made regarding literature gap assessment, including the relevance of a particular gap for clinical practice or policy, or the types of research designs best suited to address specific gaps. Finally, as noted by the Institute of Medicine (IOM), many expert panels do not make a clear distinction between the quality of evidence vs. strength of recommendations. While the NIH State-of-the-Science Panels are ostensibly run when there is a perception that the evidence base is “weaker”, the methods used in assessing the quality of evidence may vary and strength of recommendations often varies. Recommendations from these “State-of- the-Science” panels are not presented based on the relevance of a particular gap in terms of research, practice, or policy.The Institute of Medicine (IOM) also produces reports that often summarize evidence and make recommendations about practice and research. While their methods can vary, the basic approach is similar to the NIH CDC in that they pick scientific experts, receive outside input and sequester the participants and seek consensus whenever possible. Unlike the NIH CDC, the panels usually meet several times and have more time for discussion and analysis. In addition, the reports are longer and more thoroughly developed. However, despite a recent IOM report on criteria for systematic reviews and guidelines, each panel does not yet always conduct or rely on systematic reviews and their processes are sometimes even less transparent than those of NIH , 13. TheThe RAND method has been extensively described elsewhere in the literature. One of its essential features vis-a vis the NIH or IOM approaches is that the RAND approach is oriented less to scientific experts and more to clinical experts , 9. In aCoulter , 14 has First, in a RAND panel an extensive review of the literature is conducted and a systematic review written  . In the accepted best clinical practice for the year in which the panel is conducted. Considering an average group of patients with each listed indication, presenting to an average practitioner in the US, the ratings should reflect the panelist’s own best clinical or expert judgment. In this way the judgments are a combination of evidence from the literature and clinical acumen or experience with realistic variations as they often happen in the clinic. This forms a bridge from evidence to practice in a more realistic way than simple evidence summaries do, but with a more systematic and balanced process than individual clinical opinion.After these lists are compiled, the process uses a modified Delphi method where the indications created are sent to the nine panelists along with the literature synthesis. The panelists then independently rate the appropriateness of the procedure based on the evidence from the literature review and their clinical experience. The ratings for appropriateness are from 1 to 9, with 1 representing extremely inappropriate and 9 extremely appropriate. Appropriate care is defined as when “the expected health benefit to the patient  exceed expected health risks  by a sufficiently wide margin that the procedure is worth doing” , 15. TheThese ratings are quantitatively summarized by the research staff. Then, in a second round of ratings the panelists are brought together in a face-to-face meeting. Each panelist is shown his/her rating for each indication, and the distribution of the ratings of the panel as a whole is presented for each indication. Individual panelists must reconsider their judgment and although they are not forced to defend it in most cases, where the individual does differ from the group, he/she will usually do so or at least explain the logic or evidence for that position. Following the discussion, the panelists re-rate the appropriateness of the procedure again. From the second rating, it is possible to determine the degrees of agreement in the panel, and to calculate the average median ratings, and the average dispersion measures for the procedures. Consensus is not required. However, in most instances the dispersion decreases during the second rating as the panelists come closer to a consensus. Once the work of the expert panel is completed, the team then compiles a set of indications for performing a procedure based on the evidence and their clinical experience, which can then be used to compare to actual practice. This allows researchers to calculate a rate of appropriate/inappropriate care present in practice.In the RAND panels, consensus is not required but its degree is measured. RAND has utilized two approaches to measure consensus. In the first there is consensus if all raters’ responses fall within one of the three point ranges of the scale . This would mean all the raters agreed that the procedure should not be performed ; they agreed that it was questionable or uncertain ; or, they agreed that it should be performed . The second method is to define agreement if all the ratings fell within any 3 point range. Furthermore, agreement can be determined using both methods but by rejecting one extreme high or low rating. Similarly, disagreement can be calculated using two methods; if at least one rater chose a 1 and one chose a 9; or if at least one rater fell in the lowest three point region and at least one in the highest. As with disagreement, the extreme ratings can be discarded. A procedure can be judged inappropriate if its median rating is in the range 1–3, and without disagreement; uncertain if the median rating is in the range 4–6, and appropriate if it is 7–9, without disagreement. Finally, the outcome can also be that the panelists disagreed on the proper rating (they were indeterminate) –16.A unique feature of the RAND appropriateness panel is the amount of research that has been done on its reliability and validity, making the RAND panel process, unlike other expert panel processes, truly evidenced-based. For validity of the RAND appropriateness panel, studies have examined the relationship between ratings and the literature , face anThe SEaRCH Expert Panel Process draws heavily on the work done at NIH, IOM and RAND. The process integrates the most reliable and useful aspects of these panels. The desire to create a streamlined approach was driven by the need to reduce cost and create more efficient and transparent expert panels.One example of streamlining is to have expert panelists utilize the online technology database to enter data related to appropriateness ratings. Another is to hold meetings via teleconference, which allows for reduced costs as well as the inclusion of key expert panelists that otherwise may not be able to attend an in-person meeting. This also allows for the overall panel process to happen more expediently and to be more inclusive of the right experts for balance. In addition, it allows for a greater variety of panel types to be developed to serve different purposes, such as panels focused on policy and patient preferences.Currently, the SEaRCH Expert Panel Process (SEPP) uses expert panels for two primary purposes –clinical appropriateness and research agendas. A Clinical Expert Panel (CEP) is used to determine when care is appropriate or inappropriate combining both evidence and clinical acumen, similar to the RAND process. A Research Expert Panel (REP) is designed to examine the state-of-science and to identify and prioritize gaps in the scientific evidence vis-à-vis practice. Both the CEP and REP are used particularly in areas where the evidence is either lacking in quality or insufficient to determine appropriate care using usual consensus or guideline processes. While the two expert panels differ, the processes they use are similar and will be described jointly below.A request for a SEaRCH Expert Panel Process (SEPP) typically comes from an outside person or group who needs a balanced and objective way to make research or clinical recommendations on a particular topic. The requestor often seeks to obtain an evidence-based, expert judgment to help determine the appropriateness for clinical use of, or develop the research needs for, a product, practice, program or policy. In the case of a clinical expert panel (CEP), the requestor seeks recommendations for an intervention within a given setting or recommendations around a particular treatment approach in various settings. In the case of a research expert panel (REP), the requestor seeks specific research recommendations based on current gaps in the evidence and may include an assessment of available resources and readiness to conduct such research. The following describes the steps in the SEaRCH Expert Panel Process (SEPP). Figure The first step of the SEPP is for an Expert Panel Manager to meet with the client to create and refine the question to be answered. This meeting also examines the focus of the other integrated SEaRCH components such as the REAL (Rapid Evidence Assessment of the Literature) which is a streamlined, reliable, systematic review process; , 23 and As stated earlier, one of the benefits of the SEaRCH model is the communication that occurs between the description, evidence and judgment components  needed to answer important health information questions. For example, using the REAL process for informing the Expert Panels provides a systematized assessment of research quantity and quality. For the clinical expert panel, specific SMEs develop the clinical scenarios that will be rated by the panelists. Once the EP team and panel members have been established and the clinical scenarios created, the actual panel process begins.Each panelist integrates information from the REAL and/or CAP evidence with their own clinical judgment for each scenario. They rate the appropriateness of use of the intervention for each scenario and enter their ratings into an online database. The use of the database allows for less error, more opportunity for statistical review and a much faster turnaround of results for phase II.The second phase of the CEP consists of all nine panelists meeting face to face, either in person or through virtual means to review their clinical appropriateness rating scores and discuss them among the group. After the panel discussion, panelists are asked to re-rate the scenarios using the online database. The computerized program can be used to tabulate the new re-rated scenarios.As the flow chart in Fig. Similar to the Clinical EP, the Research EP is a two-phased process. However, in this case, initial ratings entered in the online technology database are not based on appropriateness of clinical care, but rather on elucidating and clarifying research gaps, as well as identifying research designs to address the most relevant gaps.Similar to the Clinical EP, the Research EP member integrates his/her knowledge and expertise with the available evidence from the REAL and/or CAP, and then enters this information into a computerized database. The initial ratings provided by the research expert panelists help to systematize and make transparent specific recommendations on research directions in particular gap areas. In the case where research recommendations are being made specifically for an institution that wishes to carry out a research project, tailored recommendations also are based on the available resources for that institution (when explicated through the CAP). The process for convening the Research Expert Panel in Phase II is similar to the Clinical Expert Panel.The goal of the research expert panel meeting, however, is for the panelists to discuss areas where they may disagree on gaps and next research priorities. As with the clinical expert panel, after the panel discussion, panelists are then asked to re-rate the research form. Panelists input their ratings in the database which tabulates the scores.The last phase of the process is to summarize and deliver the panel recommendations in a report with quantified ratings. Recommendations and specific content of each summary report will vary depending on the type of panel  and the question(s) that the client is seeking to answer. For these panels, the summary includes an in depth description of the EP methodology and recommendations based upon the findings.It is important to note that this panel model can be modified to address questions on multiple issues related to health care such as policy implementation and patient-centeredness. For example, a Policy Expert Panel (PoEP) is a derivative of the Research Expert Panel and focuses on making evidenced based policy judgments  to direct implementation of a practice claim. In fact, this panel is often used to explore direct implementation issues even when a policy issue is not a factor.One of the biggest challenges in all panel processes is that they have difficulty obtaining patient input early and continuously in the decision making. Usually, patient representatives are placed on advisory boards where input comes late in the decision making process. In addition, patients may feel intimidated or lost when on panels with scientists and clinical experts, especially if they are not trained or comfortable executing their role and using the panel methodology. The Samueli SEPP has patients and patient interest groups involved in every phase of the process,. It may even be deemed appropriate to convene a Patient-only Expert Panel (PoEP) to achieve a complete patient perspective. Patients can be incorporated into panels in various ratios such as equal (1:1:1 with scientists and clinicians) or weighted toward patients (2:1:1). In addition, use of the anonymous Delphi and virtual meeting processes can further empower patient input more deeply in judging either clinical or research relevance, making both methods more patient-centered. The REAL review process also trains all potential panel members (no matter their expertise) in how to use the results of a systematic review. Special training of the patient panel members and of panel moderators also enhances communication and input from the patient’s perspective , 25.There is an essential need for evidence-based information to guide research, practice and health care decision making. Expert panels contribute to evidence-based decision making in both research and practice by closing the gap between the usual evidence summaries and the needs of clinicians, policy makers, patients and researchers for using such evidence in daily decisions. There is a clear need for more transparent, systematized, and efficient processes for expert panels, especially in terms of providing recommendations. This process seeks to fill those gaps. As part of the SEaRCH program, Samueli Institute streamlined an expert panel process that was driven by the need for cost effective, efficient and transparent approaches to addressing health care needs. It is designed to expand diverse stakeholder input into the research judgment process. This allows for easier delivery of expert opinion and patient input for making decisions about research priorities and clinical appropriateness. This methodology is integrated with other SEaRCH components such as the Rapid Evidence Assessment of the Literature (REAL) and the Claims Assessment Profile (CAP). This methodology is still new and will need to be validated in the future. The use of information from these three evidence based components, will allow for more customized, expedient and evidence-based recommendations on therapeutic claims of interest.The expert panel process is the final step in the integrated SEaRCH process described in this series of articles. It improves the link between research evidence, research agendas, real world practice decisions and patient needs and preferences. Together these integrated strategies allow development of true evidence-based health care.


Background: Mapping the evidence is a relatively new methodological approach and may be helpful for the development of research questions and decisions about their relevance and priority. However, the amount of data available today leads to challenges for scientists sometimes being confronted with literature searches retrieving over 30,000 results for screening. Objectives: We conducted an evidence mapping of the topic “diabetes and driving” to investigate its suitability for an evidence-based national clinical guideline. In addition, we compared a highly sensitive search with a highly specific one.Methods: Based on a systematic review, our database searches were limited to publications from 2002 to present in English and German language. Results: Due to the strongly focused topic and the limits, our sensitive search identified a manageable number of references including sufficient evidence to answer our research question. Using the specific search strategy, we achieved a reduction of citations by 25%, concurrently identifying 88% of relevant references.Conclusions: Evidence mapping with the intention of gaining an overview of a research field does not require high level accuracy in contrary to systematic reviews. Keeping this distinction in mind, a mass of extraneous information will be avoided by using specific instead of highly sensitive search strategies. The term evidence mapping describes a system targeting an overview of the extent, nature and characteristics of a research field , often cAccording to the Global Evidence Mapping Initiative GEM , the mapDeveloping and running literature searches is the methodological core of scientific overviews. The design of a search strategy can be a challenge , wherebyOur aim was to determine the value of creating a full evidence synthesis concerning the research topic “diabetes and driving” and to investigate its suitability for a national clinical guideline. We therefore conducted an evidence mapping and compared a highly sensitive systematic literature search with a highly specific one. The evidence mapping was intended to reveal the amount of existing literature in general, as well as the amount of high quality evidence, e.g. systematic reviews and controlled trials. Content analyses of the identified literature had not been commissioned.We developed a highly sensitive search strategy and systematic electronic searches were conducted in April 2014 using the databases Medline, Embase and The Cochrane Library. Without a clear intervention, we used a modified PICO scheme being reflected in our search strategy see . PrelimiUnusually for evidence mapping, we only had one research question that was even clearly built up, meaning that both search strategies were not very extensive: the highly sensitive one consisted of about 70 terms, including MeSH and EMTREEs, respectively, and using truncations, synonyms and related terms. The complete highly sensitive search string as well as the highly specific one can be found in Using the highly sensitive search strategy limited by language and time period, we identified 884 references  could not be detected by the specific search which would have provided an adequate basis for the decision to initiate a full systematic evaluation.The information overload nowadays evokes a need for strategies to cope with this. Richard Smith describes that a consistent proceeding according to the methods of evidence-based medicine can hardly be pursued because almost nobody has the time. He therefore proposes helpful machines as the ultimate solution , althougFor evidence mapping regarding prevention and treatment interventions for depression in young people, Callahan et al.  used a sIt is certainly possible to screen up to 150 titles or abstracts in one hour, although it is hardly possible to do so effectively eight hours a day for five days a week due to dwindling concentration. Therefore, screening tens of thousands of references would take up to at least one month and cost a fulltime-job, if the screening process should be operated in a high quality manner by two independent reviewers. Ordering and procuring full-text publications would take additional time before thousands of full-text publications could be screened. Moreover, going back to the beginning of the process, building up and testing a complex search strategy also takes considerably more time than developing a specific search string. Therefore, the sense and purpose of broad literature searches should be well-conceived.As in geography, where a small scale map is chosen to represent a target in detail and a high scale to provide an overview of a larger area, highly sensitive search strategies for evidence maps will produce high accuracy and precise estimates, albeit only at the end of a screening procedure starting with substantially larger numbers of retrievals compared to highly specific searches yielding less accuracy but also fewer irrelevant retrievals at the beginning.To clarify the target, and in contrary to systematic reviews where it is absolutely necessary to identify the whole literature being relevant for avoidance of bias, the particular research questions and its consequences for the workload following must be thoroughly considered in evidence mapping. If it is intended to provide an overview, “a high scale” should be chosen because breadth rather than depth is the goal, like a small scale map of a continent compromising the handling of the surface and making an overview hardly possible. Parkhill et al.  tested aThis is the first empirical study concerning a research question from a public health area comparing a highly sensitive literature search and a highly specific one for evidence mapping. Our findings confirm the results of Parkhill et al.  regardinBased on our results identifying approximately the same number of relevant citations using different interfaces, we would prefer searches via Ovid or PubMed reducing the number of references to be screened by half compared to Elsevier and thus being clearly more efficient. However, this may not apply for certain topics, such as searching the literature for specific terms, e.g. aboriginal, which is currently undergoing evolving political and cultural terminology .According to Katz et al. , the breOur results are limited by a single and clearly focused research question with a very low total number of citations retrieved, being uncommon for evidence mapping. In addition, the evidence assessment took place at an abstract level lacking plenty of information. Nevertheless, our results in a public health question can serve as an appeal for thorough considerations about the sense, purpose, and consequences of the methods used for evidence mapping. Further research ascertaining our findings and recommendations concerning the combination of database and interface is necessary in a broad research question and should be at least expanded to the database Embase searched via different interfaces.Using highly specific search strategies instead of sensitive ones is fully adequate for evidence maps with the aim of covering mainly the breadth rather than depth of a research spectrum. In this case, a highly specific strategy should be given preference to save human and financial resources as well as avoiding the risk of a conclusion being out of date.The authors declare that they have no competing interests.Not required.None.


The advent of major investments in biomedical research, leading to new and better tests and treatments, has spurred the development of critical appraisal of the medical literature and evidence-based medicine , and appEvidence-based medicine, whose philosophical origins extend back to mid-19th century Paris and earlier, is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients . The praEvidence-based medicine initially focused mainly on determining the best research evidence relevant to a clinical problem or decision and applying that evidence to resolve the issue. Subsequent versions of evidence-based decision making have emphasized that research evidence alone is not an adequate guide to action. Rather, clinicians must apply their expertise to assess the patient's problem and must also incorporate the research evidence and the patient's preferences or values before making a management recommendation .Similar to other fields, specialists in the field of cardiovascular medicine are faced with the dilemma that many clinical questions in their daily practice to do not have universally agreed answers, but patients increasingly demand the ‘best practice’ from their doctors. In addition time pressures mean that clinicians are unable to keep up with the full spectrum of published research and current resources that collate evidence for clinicians. In our scrupulous scrutiny to provide up-to-date and practical data, our fully online and peer-reviewed journal, Research in Cardiovascular Medicine (RCVM), was founded in 2011. RCVM, the official journal of Rajaie Cardiovascular Medical and Research Center, is dedicated to the clinical and basic researches in the field of cardiovascular medicine. This authoritative medical journal publishes peer-reviewed articles on all aspects of cardiovascular disease, including original articles, systematic reviews, meta-analyses, narrative reviews, short articles, letters, case reports, and interesting images. This journal accepts the articles only from an online submission system. This online editorial system is prepared for manuscript submission, peer reviewing and tracking replications. Our system is customized with training and step-by-step tips in order to provide a very convenient service for authors, reviewers and editors. By available tracking service, authors can track the review process of their submitted manuscript. RCVM is an open-access journal, in order to provide appropriate updated scientific information in the field of cardiovascular medicine for the relevant readers, all around the world and is published in co-operation with Tehran University of Medical Sciences. I hope RCVM meet the high expectations of our readers by providing evidence based data in the field of cardiovascular medicine.