It has been hypothesized that adaptation of health practice guidelines to the local setting is expected to improve their uptake and implementation while cutting on required resources. We recently adapted the published American College of Rheumatology (ACR) Rheumatoid Arthritis (RA) treatment guideline to the Eastern Mediterranean Region (EMR). The objective of this paper is to describe the process used for the adaptation of the 2015 ACR guideline on the treatment of RA for the EMR.We used the GRADE-Adolopment methodology for the guideline adaptation process. We describe in detail how adolopment enhanced the efficiency of the following steps of the guideline adaptation process: (1) groups and roles, (2) selecting guideline topics, (3) identifying and training guideline panelists, (4) prioritizing questions and outcomes, (5) identifying, updating or conducting systematic reviews, (6) preparing GRADE evidence tables and EtD frameworks, (7) formulating and grading strength of recommendations, (8) using the GRADEpro-GDT software.n = 5 [100%]), impact on health equities (n = 4 [80%]), the balance of benefits and harms (n = 1 [20%]) and acceptability (n = 1 [20%]).The adolopment process took 6 months from January to June 2016 with a project coordinator dedicating 40% of her time, and the two co-chairs dedicating 5% and 10% of their times respectively. In addition, a research assistant worked 60% of her time over the last 3 months of the project. We held our face-to-face panel meeting in Qatar. Our literature update included five newly published trials. The certainty of the evidence of three of the eight recommendations changed: one from moderate to very low and two from low to very low. The factors that justified a very low certainty of the evidence in the three recommendations were: serious risk of bias and very serious imprecision. The strength of five of the recommendations changed from strong to conditional. The factors that justified the conditional strength of these 5 recommendations were: cost (This project confirmed the feasibility of GRADE-Adolopment. It also highlighted the value of collaboration with the organization that had originally developed the treatment guideline. We discuss the implications for both guideline adaptation and future research to advance the field. Guidelines are considered an integral aspect of the development of a standardized high quality health care using evidence-based practices . GuideliDevelopment of guidelines de novo faces multiple challenges including financial and human resource requirements and time constraints . On the Adaptation of guidelines would address the above challenges and limitations, through modifying the recommendations to account for contextual factors. It has been hypothesized that adaptation of guidelines to the local setting is expected to improve their uptake and implementation . One of A recently published  survey identified eight methods for guidelines adaptation, one of which is GRADE- Adolopment . AdolopmAmerican University of Beirut (AUB) GRADE Center.We adoloped the recently published American College of Rheumatology (ACR) Rheumatoid Arthritis (RA) treatment guideline to the Eastern Mediterranean Region . This prThe objective of this paper is to describe the process used for the adolopment of the 2015 ACR guideline on the treatment of RA for the Eastern Mediterranean Region. We report on the specific recommendations resulting from this process in a separate paper.We used the adolopment methodology to adapt the 2015 ACR RA treatment guideline  to the EGroups and rolesSelecting guideline topicsIdentifying and training guideline panelistsPrioritizing questions and outcomesIdentifying, updating or conducting systematic reviewsPreparing GRADE evidence tables and EtD frameworksFormulating and grading strength of recommendationsUsing the GRADEPro-GDT softwareWe structured the process for the ‘adoloped guideline’ using the Guidelines 2.0 comprehensive checklist for guideline development, which consists of 18 steps . Table 1This step included identifying and recruiting individuals for the following three groups: a guideline executive committee, a guideline coordination team, and a guideline panel (see Table In term of conflicts of interest (COI) rules, we asked the panel members to fill out an online COI declaration form, adopted from the World Health Organization. We will include these forms as an appendix in the adopted guideline manuscript. All panelists received a COI policy document explaining the approach to managing conflicts during the guideline process. The two panel co-chairs exercised the rule of “no strong advocacy” to minimize the effect of the COIs. Accordingly, panelists were invited to share their opinions and positions on each recommendation, while avoiding repetition or being verbally forceful. The co-chairs had the right to exclude panel members with COI from discussions or decision-making on recommendations they were conflicted on.i) management of patients with early RA (15 questions), (ii) management of patients with established RA (44 questions), (iii) management of patients with established RA with high risk (24 questions), and (iv) live vaccines in early or established RA patients (5 questions). Given the limited time and resources, the guideline executive committee chose to address a set of prioritized questions from the first topic, i.e., management of patients with early RA. While this choice was pragmatic it also aimed to produce a coherent set of questions, i.e., within the same type of patient population.The source guideline (the 2015 ACR RA treatment guideline) addressed four different topics: (AUB GRADE and the McMaster GRADE (Mac GRADE) centers. The regional rheumatologists were members of the Middle East Rheumatoid Arthritis Consortium (MERAC). MERAC is based at the Weill Cornell Medical College in Qatar and is comprised of rheumatologists from Jordan, Kingdom of Saudi Arabia, Lebanon, United Arab Emirates, and Qatar. MERAC’s research focuses on studying rheumatoid arthritis in the Middle East.The executive committee recruited guideline panelists in a way to represent a diverse set of expertise. Panelists included regional and international rheumatologists, a nurse practicing in Qatar, and methodologists from MacGRADE center. Also, the first day of the guideline meeting consisted of a workshop on the use of the GRADE Adolopment methodology. The workshop used material related to the one of the planned guideline questions.One month ahead of the panel meeting, we shared with the panelists training videos on the GRADE approach for guideline development found on the webpage of the The core team at the AUB GRADE Center developed a prioritization survey that included all questions from the source guideline addressing the prioritized topic, i.e., early RA. We asked the panelists to rate the importance of these questions on a Likert scale of 1–5 (least - most important), from the perspective of patients in the EMR. We selected 8 questions taking into consideration panelists’ ratings and the coherence amongst those questions. We communicated the list of selected questions to the panelists for approval. It was not possible afterwards to change or add questions to the project.Then, we sent the panelists a survey to prioritize patient important outcomes relating to the selected questions. The survey specifically asked participants to “consider outcomes that might be important to someone making a decision to use or not to use the treatment”. The panelists used a Likert scale of 1–9 to rate the importance of these outcomes for making a decision as follows: (1–3) not important; (4–6) important but not critical; and (7–9) critical.Figure We used standards systematic review methodology including duplicate and independent approach to title and abstract screening, full text screening, and data abstraction. We conducted calibration exercises, used standardized and pilot tested forms, and relied on a third reviewer to resolve disagreements.Relevance (directness): we assessed the relevance of identified systematic reviews by matching their PICO to the PICO of the guideline questions. The minimum requirement was for the Population, Intervention and Control elements to match to a reasonable degree, i.e., not to have serious indirectness for more than one of the three elements.Quality (risk of bias): we assessed the risk of bias of relevant systematic reviews using AMSTAR . If we iBeing Up to date: we assessed whether the systematic review judged to be relevant and of highest quality was up to date. In case we had identified more than one systematic review, the judgment of relative up-to-dateness would have considered whether the systematic reviews included all relevant studies. When we identified new primary studies, we integrated the findings in the chosen systematic review.When evaluating the potential use of identified systematic review, we considered the following three characteristics as important:When we identified no usable systematic review (based on the three above criteria), we updated the systematic review conducted by the source guideline-working group using the results of the search for primary studies.In addition, we searched the literature for studies and data relevant to patients’ values and preferences and economic data. The eligibility criteria related to both the population of interest (rheumatoid arthritis) and the interventions of interest. We searched Medline using both medical subject headings (MeSH) and text words related to ‘rheumatoid arthritis’ and the interventions of interest. We used a filter-limiting search to EMR region when searching for economic data but not when searching for values and preferences. Also, we limited to a period covering the last 10 years when searching for values and preferences. In addition, we solicited panelists for additional studies on baseline risks and economic data.www.gradepro.org) to develop for each recommendation question the standard tables proposed by the GRADE working group to facilitate the process [Evidence Tables: they provide for each of the outcomes of interest a summary of the synthesized evidence  and the rating of the certainty of the evidence. The GRADE working group has developed two versions of these evidence tables: Evidence Profiles (EP) (a more detailed version) and Summary of Findings tables (SoF) (a less detailed version) ;Evidence to Decision frameworks (EtD): they are intended to facilitate the panel’s decision-making process for going from evidence to recommendation by summarizing in a structured and transparent way the evidence for the following factors: benefits and harms, values and preferences, cost, cost effectiveness, equity, acceptability and feasibility , 13. As In preparation for the guideline panel meeting, we used GRADEpro-GDT guideline development tool .The guideline panelists revised the Evidence Profiles and provided input prior to the panel (face-to-face) meeting. During the panel meeting, we used the GRADE Evidence-to-Decision frameworks to assist the panel in formulating and grading the final recommendations. During this process, the panel decided to downgrade the evidence for three recommendations from low to very low due to a judgment for the imprecision factor that was different from that of the source guideline.Adoption of the recommendation, i.e., the use of the original recommendation as is;Adaptation of the recommendation, i.e., the modification of the original recommendation;De novo development of the recommendation, i.e., the creation of a new recommendation.We intentionally did not expose the panelists to the recommendations from the source guideline prior to finalization of the recommendations. For each of these recommendations, we expected the use of the adolopment process to lead in one of three potential outcomes:Collect the COI disclosures statements of panelistsUpdate the evidence profiles and develop EtD frameworks for each of the guideline’s questions;Facilitate the panel discussions by projecting the tables in real time during the meeting;Export tables in word format to include in the final guideline report.We used the GRADEpro-GDT software to conduct the following tasks:The adolopment process yielded eight recommendations, which will be detailed in another paper. Below we provide a description relating to the: (1) timeframe of the process, (2) results of the literature review, and (3) change in the certainty of evidence and the strength of recommendation.Table Table With regards to our search for studies on patients’ values and preferences, we identified 16 relevant studies but none were specific to the Eastern Mediterranean Region context. The information we retrieved looked at outcome valuation and medication preference in general . We did not identify any studies on resource use relevant to the Eastern Mediterranean Region.n = 5), impact on health equities (n = 4), the balance of benefits and harms (n = 1) and acceptability (n = 1).After we formulated the eight final recommendations, we compared the certainty and strength of each of the adoloped recommendations to corresponding recommendations from the source guideline. The certainty of the evidence of three of the eight recommendations changed: one from moderate to very low and two from low to very low. The factors that justified a very low certainty of the evidence in these three recommendations were: serious risk of bias and very serious imprecision. The strength of five out of the eight recommendations changed from strong to conditional. The factors that justified the conditional strength of these 5 recommendations were the following: cost  UseOur collaboration with the source guideline organization proved to be crucial for the success of the project. A major contributing factor was the willingness of the senior director of quality for the ACR to allow the unrestricted use of their recently published guideline as the basis for the adolopment process. As a result, we used ACR RA treatment guideline recommendation questions as a starting point to prioritize our questions; the participation of the chair and the methodologist of the ACR treatment guideline gave the panel unique insight into the source guideline’s panel decision-making process; it helped with clarifying any uncertainties regarding the evidence synthesis and formulation of the final recommendations. In addition, the fact that the two guideline efforts used the same methodology , and the same the tools  made our process more efficient.This project was not without any challenges or barriers. One major challenge was the scarcity of evidence for contextual factors such as values and preferences, and economic data. The panel relied on its members’ expert opinion to judge those factors as part of the EtD. Another challenge was the need to develop a plan (as describe above) to build the capacity of the panelists and of some members of the technical team in guideline development and adolopment. We were also challenged with the relatively short timeframe of 6 months. We managed by employing efficient processes, having the members of the core team dedicate a larger amount of their time to the project while limiting the number of questions to address.Accepting definitions or classifications used by the source guidelines for the condition, the interventions, and the outcomesUsing either the exact same question or a question with narrower scope, compared to the source guidelines. Otherwise, the adapting group must run a new search strategy to ensure capturing all studies relevant to their new questionAccepting the outcomes and the associated timeframes as prioritized by the source guidelinesAccepting any limitations in the process of the source guidelines related to the systematic review process .To keep adolopment of guidelines efficient, guideline developers might need to accept a number of restrictions based on our experience:The indirectness of the evidence as it relates to the local population ;The indirectness of the evidence as it relates to the intervention ;The baseline risk of the outcomes of interest The values that the local population attach to the outcomes of interest  and their preferences for the interventions of interest ;Contextual factors of the EtD, including priority of the problem, cost, cost effectiveness, impact on equity, acceptability and feasibility.Having considered the above, it is also ideal that the guideline adolopment panel makes its own judgments about context sensitive aspects of the decision-making process:In terms of implications for future research, the adaptation process allowed us to highlight the scarcity of regional data related to rheumatoid arthritis when looking at contextual factors such as cost, cost effectiveness and values and preferences. This should encourage researchers to address these gaps through conducting primary studies. This will allow the adaptation process to be better contextualized and the data on which we rely on to be more systematic and evidence based rather than solely relying on the panels knowledge and opinions.Schünemann et al. described a vision of guideline development through an “international collaboration with common aims and free of proprietary influences” . SeveralIt is important for major guideline developers interested in others building on their guidelines, to develop them in a manner that facilitates the later adaptation process. This includes making explicit the methods used in the evidence synthesis, evidence grading and recommendation development processes to allow reproducibility. It also includes making the products of those processes available to other groups . Finally, making explicit the judgments and underlying justifications on the EtD factors would make it easier to other groups to make their own judgments.


Patients are the same all over the world. Evidence is the same in the entire world. However, care policies are not the same. This is impossible to understand and unacceptable, since there is no different evidence for public and private care; it is not a matter of being capitalist or socialist, of presence or distribution of money; it is the fight against consumerist, untruthful and voracious capitalism, encouraging licit “robbery”; it is an issue of equity and ethics; of assured constitutional rights; in which doing something means always doing it correctly and for everyone; where there is no freedom and no prohibition at the same time; where everything starts at standardization; where there is no room for futility; where excesses are banned; where not only is there orientation as to what to do, but also as to what not to do; where the patient is not the means, but the end; where variation by VALUE is combated; and where the concept of VALUE considers costs, as well as benefits and damage.1Nevertheless, how is it possible to not use evidence in health policies? Since evidence defines what is necessary and what is superfluous; provides options of equivalent benefits and forbidden practices; informs about the estimated amount of benefit and damage, and regarding the adoption of new practices with no benefit; defines the concept of value and the use of money with discrimination; leads the market towards its function and the media towards appropriate criticism; gives orientation as to priorities and strategies; reduces variations in practice and conflicts; and balances interests.2What has been done? How has the “quasi-evidence” been considered in the definition of access to care? Well, the health authorities are limited in deciding “what and how much is paid (or not paid)”, by means of their “technology” assessment centers, using expert ghost consultants, before, during, and after issuing their opinions based on “God only knows” what - but always aligned with their own interests.What could and should be done initially with the evidence available?It is considered that, through the same health authorities, four different and complementary offices would be created with a core purpose of defining what is right or wrong to be offered to all patients, contributing to establishing the key strategies and priorities of the national policy: (1) managerial office in evidence-based medicine; (2) evidence-generation offices; (3) evidence-implementation offices; and (4) conflict-arbitration offices.Further, there are some requirements for these offices, so that they may survive, influence, disseminate, and guarantee quality care at national level:–There is a need for infrastructure and human resources , which can assure the appropriate use of scientific information in care, by means of performance of those involved with their results (benefits and safety) and with structuring the service, conformed to the image of the evidence.–It is necessary to trust evidence, determined by its strength and consistency, which by producing a lower level of uncertainty, allows one to believe and be secure in the practice recommendations and syntheses.–External credibility  of the service is a factor that should also be present, since patients should be satisfied regarding quality of care and results obtained.–Healthcare professionals should be homogeneously informed, trained, and skilled to make decisions in each specific situation; moreover, by means of experience accumulated they can appropriately compare practice and evidence. With adequate preparation, there is a greater possibility of reproducing the outcomes described in the evidence, in which doing means always doing the right thing.–Participation of patients in decision making about their care is compulsory; and in the presence of strong evidence, should have their expectations considered and doubts explained, so that the decision is safely made and without regret. In the absence of evidence or in the case of weak evidence, expectations and medical experience should be shared.3–The various stakeholders must minimally master knowledge about evidence, or this knowledge, once ignored, will lead to skewed language and understanding of the same evidence. Thus, to know how to interpret the strength of the evidence, with its several components, such as size and variation of results (effect), not only leads to an adequate medical comprehension, but also to suitable communication with patients regarding the degree of uncertainty in the decision to be made.–Lastly, all need to discuss the concept of value, which despite varying according to the location, tends to be distributed in a “normal” manner. In this debate, a few issues should be on the common agenda of the use of evidence, such as: What is the value of health? How to be mortal? Personalized medicine? Quality of life? What is the value of life? Social values? What is benefit? What damage should be tolerated? This can only be reached through education, consciousness and perception of all being aware that we share the same healthcare system and services, and for any attitudes or actions taken, there will be direct or indirect consequences, near or far for all, hindering the consolidation of equity in health in our country.,5(–)Although many perceive the agenda to implement evidence in health policies as a simplistic use of a “cookbook”, the effects of no adequate use of evidence speak for themselves. The requirements to implement evidence are fundamentally associated with generation, interpretation, communication, and adequate use. However, three more prerequisites are necessary - which may precede evidence or not - so that doubts, priorities and strategies be focused on local needs:–8–A precise mapping and diagnosis of distribution of the major health problems, which will guide the priorities according to the local severity and prevalence.–The record and control of the population's health information, which will contribute to diagnosis besides generating or validating evidence implemented or to be implemented.–Education actions including training, structuring, combating, defusing, audit, measurements, analyses, reassessments, and follow-up.Is scientific evidence in public health policy possible? Is this question relevant? Have we already done this? What would be the symptoms of this policy? How to identify when this is being done? What are the practical outcomes?(–)We can declare that in a country whose health policy is based on evidence, there is disinvestment in the unnecessary and investment where there is need. Overdiagnosis, overtreatment, and undertreatment are combated. There are parameters to help in judicialized conflicts. There is control of the activity and of the industrial health market. There is identification of the opportunity for prevention, research, and innovation, since with standardization underway, one can stop to think. There is no imitation; there is personality, enjoying the advantage of the Internet and of global evidence available. There is control of actions, having as target patient well-being and safety, and not the economy of resources at the expense of damage. There is central control of the responsibility of caring for one's patients. There is an environment where the patients really feel cared for. There is participation of all in monitoring the practical use of evidence when delivering care to the population.–8We do not need to move from this country. We can - here and now - make way for scientific evidence to determine the actions; for professionals to accompany and produce the results; and for the population to give feedback.


The effective use of Objective Structured Clinical Examinations (OSCEs) requires careful attention to the construction of clinical scenarios and to the way clinical performance will be measured.Assessments must be aligned to the curriculum and when constructing an OSCE station, care must be taken to set a scenario which tests the desired learning outcomes.  This allows the student to demonstrate evidence of the clinical skills they have acquired and will create measures of clinical performance likely to be valid and reliable. However, there are many sources of variability which can influence the generalizability and reliability of the scores. For example, the assessors’ judgment, students’ ability and the sampling of tasks/cases.An assessor’s judgment may lose validity if the tasks within a station are so complex and extensive, resulting in examiner ‘demoralization and fatigue’.The judgments that assessors make may also be prone to measurement error leading to issues of inter-rater reliability i.e., would the student have achieved the same performance rating in the “abdominal” station if a different examiner had rated them?Performance ratings are susceptible to the various types of internal and external errors which can contribute to less reliable OSCE performance ratings. An example of an internal error would be variations in the level of student’s interest and motivation, whereas differences in circuits, student gender, the ethnicity of standardised patients and students,,To standard set an OSCE it is common place to use a borderline group or regression method. This requires assessors to both score the task set in a station, using either a checklist or domain-based marking scheme and also to give their global rating of the overall students’ performance.  It is particularly important that the assessors are aware of the standard of the set of students being examined, especially that of a “borderline candidate”. More than a century ago, Thorndike and Hagan concluded that “the ideal rater is the person who has had a great deal of opportunity to observe the person being rated in situations in which he would be likely to show the qualities on which ratings are desired”.To our knowledge scant attention has been paid to visualise the checklist scores and global ratings together to obtain a clear picture of the quality of performance ratings. This paper therefore considers assessor performance and addresses how visualising checklist scores and global ratings with boxplots provide substantial detail of the distribution of scores from different assessors examining the same station and its relationship with global ratings.Assessor bias error occurs when the ratings given are not grounded in the performance of the student on the domain being assessed. For each OSCE station, the performance of the task is rated by assessors who assign a score to each of the items or domains being assessed, creating a total mark achieved by the student.  For standard setting or psychometric study purposes, assessors are also asked to make an overall judgment about the quality of student performance using a global rating scale independent of the individual marks awarded. Many sources of variability can influence such ratings and judgements, resulting in a lack of generalizability and poor reliability of OSCE scores.Experimental studies by Schneier showed that the cognitively complex assessors were more confident in their ratings, were less lenient and subject to less halo effect in their ratings, and also showed less restrictions on range errors. Therefore, it is important for assessment leads to match the cognitive characteristics of assessors to checklists and global rating scales to reduce the bias of leniency and stringency in ratings.  A further external factor which results in a positively or negatively biased assessment is the mood of assessors. According to the Affect Infusion Model, affective states can cause a ‘colouring of the judgemental outcome’.The characteristics of assessors, internal and external error factors, may result in bias of underrating or overrating of student performance.  This in turn threatens the validity and reliability of OSCE scores. One should question the reliability of student scores and the overall fairness of an assessment when there are significant differences between assessor ratings. This may need to be addressed through moderation of the assessment, where marks may be adjusted to compensate for unacceptable levels of variation and error from assessors and other sources; stations be reviewed and improved for future use; and assessment leads consider and implement ways to improve assessor performance.There are a number of psychometric methods to estimate such differences between assessors and we will describe one simple yet effective approach. A visual example of this is shown in There are different plausible approaches  to estimate the assessor leniency and stringency effects, to calculate test-scorereliability, and to make suggestions for enhancing the rating process,The larger the standard score, the more extreme the score comparatively to others. Therefore, a standard score of 0.5, being close to the mean, indicates that the assessor was not particularly lenient or harsh. If a standard score is -2 on the whole distribution, this indicates the assessor scored -2 standard deviations below the mean and that the assessor is likely to be stringent or ‘hawkish’ compared to the average assessor. If a standard score is greater than 2 this indicates the assessor is lenient or “dovish”.  Using a standard score greater than 2 or less than -2 is an arbitrary approach. However, when the scores are normally distributed, it is exceptional to get a standard score greater than +3 or less than -3.  When we convert the assessors’ scores to standard scores, we can plot box plots for the standard scores.  As shown in Improving the quality of assessor observational ratings in OSCEs is very important and this can be achieved through visualisation of checklist scores and global rating scale. As discussed earlier, multiple facets can engender assessor errors and biases, e.g., halo effect, severity or kindness error, central tendency, liking, first impression, companionship, and hence make students’ scores less decisive and unreliable.  However, caution should be taken when interpreting the potential effect of error, as a score may also be a reflection of true performance, and not be affected by potential error.Box plots are a powerful and useful tool for displaying specific summary statistics in order to obtain an immediate impression of assessor bias effects. What is relevant here is the means by which the box plots can be fed back to assessors so that their ability to discriminate between high and low performers is enhanced. Therefore, based on these boxplots, assessor training on rating accuracy can be discussed and improved so that they can make more informed judgments concerning performance rating of students and hence provide a fairer and more accurate score. Providing accurate scores may also increase the motivation of students, and hence they may consider the feedback provided to enhance their performance.The authors declare that they have no conflict of interest.


The judgment and decision making process during guideline development is central for producing high-quality clinical practice guidelines, but the topic is relatively underexplored in the guideline research literature. We have studied the development process of national guidelines with a disease-prevention scope produced by the National board of Health and Welfare (NBHW) in Sweden. The NBHW formal guideline development model states that guideline recommendations should be based on five decision-criteria: research evidence; curative/preventive effect size, severity of the condition; cost-effectiveness; and ethical considerations. A group of health profession representatives (i.e. a prioritization group) was assigned the task of ranking condition-intervention pairs for guideline recommendations, taking into consideration the multiple decision criteria. The aim of this study was to investigate the decision making process during the two-year development of national guidelines for methods of preventing disease.A qualitative inductive longitudinal case study approach was used to investigate the decision making process. Questionnaires, non-participant observations of nine two-day group meetings, and documents provided data for the analysis. Conventional and summative qualitative content analysis was used to analyse data.The guideline development model was modified ad-hoc as the group encountered three main types of dilemmas: high quality evidence vs. low adoptability of recommendation; insufficient evidence vs. high urgency to act; and incoherence in assessment and prioritization within and between four different lifestyle areas. The formal guideline development model guided the decision-criteria used, but three new or revised criteria were added by the group: ‘clinical knowledge and experience’, ‘potential guideline consequences’ and ‘needs of vulnerable groups’. The frequency of the use of various criteria in discussions varied over time. Gender, professional status, and interpersonal skills were perceived to affect individuals’ relative influence on group discussions.The study shows that guideline development groups make compromises between rigour and pragmatism. The formal guideline development model incorporated multiple aspects, but offered few details on how the different criteria should be handled. The guideline development model devoted little attention to the role of the decision-model and group-related factors. Guideline development models could benefit from clarifying the role of the group-related factors and non-research evidence, such as clinical experience and ethical considerations, in decision-processes during guideline development. During the last decades, clinical practice guidelines  have become increasingly used tools for health care systems that aim to provide efficient and safe health care. Although guidelines have a common goal of informing health decision making based on patient preferences and the judgment of health professionals, the impact of guidelines has varied –4. ChallSignificant efforts have been put into increasing the trustworthiness, quality, and implementability of clinical guidelines –10. The Development of high-quality guidelines involves technical, cognitive, and interpersonal processes . ConsideGuideline development generally includes identifying and refining the subject area, assembling panels, identifying and assessing evidence, translating evidence into clinical practice guidelines, and finally reviewing and updating the guidelines , 24. GuiDecisions about guideline recommendations have usually been made by a group of health profession representatives so as to assemble a wide range of knowledge, experiences, and opinions. Considerations made when recruiting individuals to such groups have involved variation in geographic representation, disciplines, stakeholder interests, gender, and different schools of thought , 31, 32.Guideline development processes have typically been informed by manuals or models that share common features . RecurriThe functioning of GDGs has been shown to influence how information is shared and processed and ultimately affects the guideline quality , 38. In For many patient populations and conditions evidence is limited . To aid The traditional Delphi method is characterized by iterations of individual decisions and group feedback to reach consensus, although participants do not meet with each other face-to-face and therefore can present and react to ideas unbiased by other participants , 43. TheMembers of guideline groups tend to be biased in favour of procedures or theories linked to their field of knowledge or experience –49. HighGroup size and structure have been found to be relevant for the validity and reliability of guideline recommendations . BetweenIn Sweden, the National Board for Health and Welfare (NBHW) is responsible for the development of national guidelines. Guideline development is led by an executive board based at NBHW. The formal purpose of guidelines is to support and guide resource allocation within health and medical care and the social services system. Primary target audience is decision-makers within health care, while professionals, patients, and the public are secondary target groups.NBHW guideline policies and instructions for guideline development state that rigorous assessment and use of scientific evidence should be the basis for guideline recommendations. Evidence is ranked by using the GRADE system  . GuideliGuideline development at NBHW consists of four phases and shares the characteristics of several other guideline frameworks . The devUp until todayNBHW has developed 15 guidelines using the process outlined above.Our focus is on the judgement and decision making process of the Prioritization group in the third phase of the NBHW guideline development model, i.e. prioritizing and deciding on guideline recommendations. Accordingly, the aim of this study was to investigate the bases for decisions and the decision making process of the Prioritization group during development of clinical guidelines with a disease preventive scope in Sweden.I.Which decision making criteria were used, and how did research evidence influence the Prioritization group’s judgment and decision making process?II.Did the composition of decision criteria change over time, and if so, how?III. Did the Prioritization group encounter conflicts or dilemmas during judgement and decision making? If so, on what subjects and how were these conflicts or dilemmas managed?Three more specific research questions were posed:A qualitative, longitudinal, case-study approach was used to investigate decision making. In phase three Fig. , the Prind meeting, Sept 2009), Q2 in the middle , and Q3 at the end of the development process .To answer the first research question an inductive approach was undertaken and data were collected from questionnaires, non-participant observations of meetings, and documents. For the second research question non-participant observations were the main data source. The third research question used data from observations and questionnaires. Two researchers (LRS and MN) conducted non-participant observations of 18 days of group meetings (Ʃ104 hours) using a semi-structured protocol indicating time and pre-set meeting agenda. Observational data contained aspects of verbal content  and the process (e.g. types of activities and procedures) and type of actor that made input (member or executive board members). Three questionnaires  were developed guided by the research questions and used to collect data on members’ experiences of the process. The questionnaires were distributed in connection to meetings, Q1 in the beginning  contained a section with close-ended questions concerning socio-demographics . The reason for repeating these questions was to make analysis of the roles of these factors in each phase of the guideline development possible.In addition the questionnaires contained open-ended questions concerning experiences from the decision making process. Questionnaire number 1 (Q1) focused on members’ anticipation and experience of the initial engagement in the guideline development process (example of question in Q1: How did you prepare for participating in the prioritization group?). Questionnaire number 2 (Q2) focused on Prioritization group members’ experience of the NBHW model for prioritization and what decision criteria they based their decisions on (example of question in Q3: “Describe the principles for taking decisions in the prioritization group?”). Questionnaire number 3 (Q3) addressed Prioritization group members’ perception and experience of the decision making process in retrospect (example of question in Q3:“To what degree have you been able to communicate your opinions and knowledge in the group?”).n = 22, 19, 21). Missing data were due to participants not being present. 18 documents containing data concerning NBHW guideline policies (n = 4), instructions for guideline development (n = 11), the specific task description for the Prioritization group (n = 3) and the final versions of the NBHW National Guidelines for methods of preventing disease were collected. These documents provided background information and a description of the formal procedure and decision criteria to be used by the Prioritization group. In Table The questionnaires were distributed at the end of meetings. Response rates varied between 76% and 88% n = 22, 9, 21. MiData analysis focused on identifying decision criteria used, variation in decision criteria over time, and types of dilemmas encountered (guided by research questions I–III). With these restrictions we used two approaches to qualitative content analysis; conventional and summative . Conventn = 482 pages), questionnaires (n = 62 responses) and documents (n = 89 pages) was read through several times to get a sense of the material as a whole. In a second step, the text was divided into coherent text units and further analysed using software NVivo9. The text units contained information on how  and on what basis (e.g. criteria or arguments) group decisions were made. In a third step text units with coherent statements (i.e. sentences and sections of text with coherent content) were coded with a condensed label intended to capture the essence of the unit’s meaning. Codes with a common meaning were then grouped into categories. In the fourth step, categories with a common meaning were merged into themes. The basis for and labelling of categories and themes were discussed among and decided on by two researchers (LRS and MN). Seven decision criteria were identified during the analysis. To assess the usage and development of various decision criteria during the two year prioritization process, we used summative content analysis [scientific base, scientific investigations and scientific arguments. Key terms and their corresponding synonyms were then sought in the text through computer-assisted searches. Number of occurrences from each key terms and alternative term were summarized. Finally the occurrences of the terms were organized chronologically, indicating if they were expressed in the beginning (meetings 1–3), the middle (meetings 4–6), or the end (meetings 7–9) of the process in Phase 3.The conventional analysis was conducted in several steps. First, the text in observation notes  and interpretations continuously discussed with another researcher (MN). LRS and MN were both aware of the content of the formal guideline development model.The results are presented in three sections congruent with the three research questions: I) Decision criteria used by the Prioritization group II); the Decision making process over time; and III) Emerging dilemmas and related strategies. Results are summarized in Table Group-decisions and judgments were found to be based on seven main criteria. The first four criteria are in accordance with the NBHW decision making model: 1) Research evidence, 2) Severity of the condition, 3) Cost-effectiveness of the intervention, and 4) Ethical considerations. The other three criteria emerged over time as an additional basis for decisions: 5) Needs of vulnerable groups, 6) Clinical knowledge and experience and 7) Potential consequences of guideline recommendations.There were also additional aspects that influenced the decision process: 8) Additional factors – Gender, social status, and interpersonal skills.“At first I tried to use all parts of the model. But now I have learned to use evidence as the main source in making decisions. I think the whole group has learned this.” (Q3)“Ok, this is a given. The evidence is clear and the cost is relatively low. This is something we can recommend.” To use of research evidence as a decision criteria in the prioritization process was a recurrent advice in the NBHW instructions for guideline development. The use of research evidence was also prominent in the observed group discussions and in the questionnaires . Observations indicated that when evidence was considered sufficient and clear the judgments were also highly consistent with individual ratings, resulting in a swift decision process where other criteria had little or no presence in the discussions. The Prioritization group members saw evidence as the superior criterion.The lack of evidence was a recurrent issue; discussed in the group meetings and also described in questionnaires . Even though over 31,000 research articles had been reviewed, several condition-intervention pairs were found to be insufficiently researched. Evidence was labelled as insufficient due to type of study (e.g. non-randomized single studies), too small effects or sample sizes, disparity between study population and guideline target population, disparity between study variables or outcomes and guideline operationalization of key concepts , and studies not being sufficiently detailed about the intervention or method for it to be translated into a guideline recommendation.When evidence was lacking, the NBHW guideline development model recommended that the intervention should be tested and evaluated in a controlled clinical research setting or not be used at all. However, observational data showed that in some cases, when evidence was considered insufficient for conditions with highly prioritized target populations, other formal decision criteria, such as ethical considerations or the severity of the condition, became more prominent and/or new decision criteria (e.g. Needs of vulnerable groups) evolved in order to be able to recommend the intervention. The process to reach consensus then became considerably longer with postponed or repeated voting procedures and increased involvement of the chair and the NBHW guideline executive board. Arguments based on research evidence were associated with logic and rationality both by Prioritization group members and the executive board.Severity of the condition was a key component in the NBHW decision-model and served as a starting point for group discussions. This criterion refers to the potential risk of decreased life years or life-quality as a result of the condition.“Even though evidence may be lacking, this is a very serious health condition. The ethical platform states that patients of greater needs should be given priority, so this is a difficult judgment.” Discussions about the severity of conditions were often linked to argumentation based other criteria, e.g. ethical considerations.“A more time-consuming intervention is also more effective. But it is also more expensive. A short-term intervention is cost effective. It is also more ethical use of the patients’ time.” Observations showed that Cost-effectiveness of the intervention was – as the Severity of the condition-criterion – often used in combination with other criteria in the decision making process Cost-effectiveness was for example used as an argument to make criterion Research evidence or Severity of the condition, stronger or weaker.Responsibilities as health professionalsDuring the observed discussions the members raised ethical issues in relation to responsibilities to represent patients, based on their background as health professionals.“Even though we got this assignment from the NBHW authority, we must put the needs of the patients first.” Paternalism“If a person comes to the primary health care centre with depression or a sore throat, should he be asked questions about his eating habits? Is that the role of health care, to impose advice on people? Isn’t that a bit paternalistic?” Discussions concerning the strive to acknowledge patients’ right to be informed about possible health risks of their lifestyle habits, but not wanting to be paternalistic and telling patients how to live their lifes.Discussions referring the general ethical platform for prioritizations in health care“If we as health professionals are well aware of the risks of unhealthy eating habits, do not our patients also have the right to know?” Members presented arguments referring to patients’ equal rights to receive counselling to support changes in lifestyle.Throughout the development process, we observed an ongoing debate about ethical implications of different guideline propositions. This theme was also elaborated on in questionnaires (Q3). Values and questions about right or wrong  were labelled by the Prioritization group as being ethical issues related to the role and obligations of NBHW, or to the group itself. Ethical considerations also concerned implications of guideline recommendations for health professionals or patients. Ethical reflections were divided into three main categories depending on their focus:Ethical discussions also addressed solidarity, i.e. that patient groups with greater needs and more severe suffering should be prioritized in the guideline recommendations. The need for prioritizing patients with greater needs became more obvious when interventions and conditions were compared. This discussion led the Prioritization group to construct an additional decision criterion – Needs of vulnerable groups.“When we divide the vulnerable groups in this way we will only find studies with low quality of evidence because the groups are so small. How can we recommend the best for these vulnerable groups?” A perceived need to prioritize certain patient groups emerged during the observed group discussions and was described in questionnaires . The severity and consequences of a condition were assessed as higher and more far-reaching for certain groups and/or conditions . Evidence was ranked as low for these groups when sample-sizes were small or when randomization had not been used.The Prioritization group determined, via discussions with the NBHW management board, that certain defined vulnerable groups automatically would get a higher prioritization and stronger recommendation in the guidelines: patients preparing for surgery, pregnant or breastfeeding women, parents, and those with comorbidities. In the final guidelines, needs of vulnerable groups were defined as “increased severity of medical conditions due to certain circumstances” .“This will be difficult to implement! I can imagine the challenge it would be to offer advanced counselling to all patients with unhealthy eating habits.” “But this is not the place for feelings.” “Ok, I need to step back and consider the scientific base for this intervention.” One observed category of arguments was based on Prioritization group members’ previous experience and familiarity with specific interventions and/or health conditions. Some members used their clinical experience to stress the urgency of certain knowledge areas, perspectives, or patient groups. It was common to use clinical examples and express a patient perspective, especially during the first meetings. The Prioritization group members used hypothetical examples to test effects of the recommendations being discussed. However, the NBHW guideline executive board rejected the use of arguments based on clinical experience. These were considered to be subjective and expressions of opinion and, therefore, not valid as formal decision criteria.“Even though you are invited to this group (remark: the prioritization group) based on your clinical expertise, we want you to let your clinical experience to be subordinate to the guideline development model.” The executive board expressed contradictory signals to if and how the Prioritization group members should let their clinical experience influence judgments during guideline formulation.“I think we need to consider how this will be perceived among health professionals. What kind of signal do we want to send?” A consideration that also influenced decisions was how guideline recommendations would be perceived and used by the target populations. This argument was observed in group discussions and also described in a questionnaire (Q3). The Prioritization group members tested different user perspectives, e.g. a physician at a health care centre or a local politician. The group anticipated discussions and reactions on the forthcoming guideline recommendations, especially from health care professionals. Endeavours to be consistent and trustworthy in the eyes of the target audience were reoccurring in the discussions.The above argument met some resistance among executive board members, who expressed that potential negative consequences of the guidelines should not limit the Prioritization group’s conclusions given that the guidelines to some degree had the role as a pioneer intervention to improve preventive health services.“Being male, a good speaker, and having good rhetorical skills provided opportunity to influence. But listening and paying attention to other members’ arguments was also rewarding, if you wanted to influence discussions. Perhaps most crucial was to choose which questions you should invest in and save your arguments for the issues where you had knowledge.” (Q3)“Influence was gained by grounding your arguments on facts and knowledge and staying away from becoming emotional.” (Q3)Additional factors that influenced discussions were also identified, mainly in questionnaire data (Q3) but also noted in observations. The Prioritization group members’ gender and status influenced group discussions. Male members and members of higher status in terms of education or age were perceived as having a greater influence on discussions. Verbal skills, timing, and ability to resist emotional expressions during discussions were also perceived by members as factors that led to greater influence. Abilities to be present, to listen, and to change one’s viewpoint were perceived as good strategies to achieve influence.The summative analysis indicated that the usage of different types of decision criteria in observed Prioritization group discussions varied in frequency over time. Figure “It took me some time to really learn how to understand and use evidence in the discussions. In the beginning I was more emotional, thinking about how this intervention would work for me in my role as a physician.” (Q2)The NBHW executive board continuously interacted in the prioritization process by defining the meeting structure and providing directions to the Prioritization group. Arguments based on research evidence increased over time while arguments based on clinical experience declined by the middle of the process and expressed more seldom towards the end. This development was urged by the executive board members. The Prioritization group members expressed awareness about this variation and described it as constructive and as a learning experience.Severity of the condition and 3) Cost-effectiveness showed relatively stable patterns in occurrence over time. Arguments related to criterion 4) Ethical considerations were most frequent in the beginning and middle sections of the process and declined at the end. The added criterion 5) Needs of vulnerable groups was frequently used after it was established during the third meeting. Discussions on 7) Potential guideline consequences of the proposed recommendations increased in the later part of the process.Terms related to 2) Several dilemmas emerged during the process, closely related to the variation of the discussion of different criteria over time. Three categories of dilemmas were identified, further described below.“This system has its downsides. When we recommend an intervention that we think is the best, we automatically undermine all other methods even though some of them show acceptable levels of evidence and that are working fine in the current preventive health practices. Health care will seek to implement the highest-ranked interventions, but if the gap is too wide and implementation fails, we are left with fewer preventive methods than before the guidelines.” (Q2)The Prioritization group discussed potential implications of recommending interventions based on high quality evidence when these outweighed disease-preventive interventions with moderate (or low) quality of evidence already in use and widely regarded as successful. Some of the proposed interventions with high quality of evidence were extensive (i.e. demanding more time and skills) and they implied a wider gap in relation to current health care practises compared to some interventions supported by low-medium quality of evidence. Some Prioritization group members saw a risk of good-enough practices being abandoned in favour of recommended practices aiming for larger impact. As a solution, it was decided to be transparent about how the evidence was graded and used, and this was explicitly described in a specific section of the final guidelines.“This is a very serious health condition but the evidence is weaker. How do we merge these two aspects of the decisions?” The Prioritization group struggled with cases where the severity of the condition was assessed as very high, but where interventions supported by evidence were lacking.“it is better to offer something than nothing” (Observation meeting no. 5). In these discussions an unwillingness to abandon methods that had insufficient evidence was expressed, and members argued for the need of recommendations even though evidence was insufficient, as there were no alternatives available. On the other hand, recommending methods that were supported only by low quality evidence was seen to potentially jeopardize the trustworthiness of the guideline process.“There is a risk that guidelines from the NBHW become an ideological megaphone that they are not taken seriously if we recommend methods without research evidence.” Some of the Prioritization group members took the standpoint that To avoid discriminating interventions targeting small vulnerable populations, it was decided to routinely increase the prioritization level one step of the recommended interventions for these groups. The purpose of this strategy was to make vulnerable groups visible and highly prioritized in the guideline recommendations. In some cases, vulnerable groups (e.g. pregnant women) were merged with other groups in order to aggregate the effects of the intervention. For new, not fully evaluated interventions where studies indicated promising results, the Prioritization group sometimes suggested further tests of the interventions in clinical research settings. A list of promising, but not yet evaluated interventions was formulated.Needs of vulnerable groups).The condition-intervention pairs were processed one at a time by applying the decision criteria. As the Prioritization group compared assessments of the interventions across lifestyle areas  inconsistencies between some of the assessments were observed by the executive board and/or the Prioritization group members. To address this dilemma, the Prioritization group added an additional step in the decision process. When a decision was reached, the executive-board systematically compared it with decisions made earlier. If inconsistencies between prioritizations were found, the decision was reconsidered. On some occasions previous decisions were fully re-processed and new evidence (i.e. recently published scientific articles) was sought and found. When consistency was reached, a new principle for decision making was formulated by the Prioritization group and thereafter applied  but we think that the process perspective has illuminated some important features of the guideline development process.The NBHW guideline development model required that the condition-intervention pairs were to be prioritized based on research evidence; curative/preventive effect size, severity of the condition; cost-effectiveness; and ethical considerations. The Prioritization group used the formal guideline development model as a starting point for discussions. However, as the Prioritization group encountered dilemmas that the formal model did not cover  the decision model was modified accordingly, by adding new decision-criteria, new rules for decision making and/or by formulating additional writings in the final guidelines. This pragmatic use of research evidence will be discussed with a focus on group decision making, and the role of clinical experience and research evidence in guideline development.Information sharing processes may have been hindered as a consequence of several group-related circumstances  in the studied case. The studied Prioritization group (25 members) was larger than the ideal group size of 6–15 members , making The observed deviation from the NBHW formal guideline development model is also in line with previous findings –35. MostAccepted methodological standards for guideline development have been pushed to continuously higher levels of rigor , 4, withClinical expertise is emphasized in definitions of EBP, stating that EBP is about “integrating individual clinical expertise and the best external evidence” ( p. 312).Research evidence and Clinical knowledge and experience were contrasted in the Prioritization group’s discussions, but research evidence was almost always given a higher priority. The precedence of research evidence was underlined by the NBHW executive board and also supported by the NBHW instructions for guideline development.In the studied case it was found that over time, research evidence out-triumphed arguments based on clinical knowledge and experience. The criteria The purpose of involving clinical experts in the guideline development process was rather vague and their expected contribution remained slightly unclear in the studied case. Diverse approaches to stakeholder involvement among guideline developers have also been seen in previous research. Most guideline developers involve stakeholders , in different stages of the process , mainly This tension between evidence and clinical experience and expertise is not often discussed in the guideline literature. How and to what extent clinical experience can be systematically identified and applied in guideline development remains unclear and is an essential area for further investigation.Some of the key determinants of guideline adherence are settled already during guideline development  , 15, 63.The observed deviation from the pre-set NBHW decision-model is interesting from this implementation perspective as several of the emergent decision-criteria were connected to specific target users, target populations, postulated barriers, or guideline contexts that are linked to implementation determinant framework . In the The Prioritization group struggled to find a balance between recommending major changes to clinical practices based on best available research evidence and suggesting more feasible changes based on a combination of evidence, good practice, stated values of health care professionals, and existing disease-prevention health services. Adaption of a general pre-set decision model to fit the specific topic of the guideline and to the target users is inevitable and might also be beneficial. Adapting models to fit current settings and melding values, evidence, and costs into guideline recommendations might arguably increase the possibilities to successfully implement recommendations in health care organisations.It is well known that clinical experience and research evidence sometimes are in conflict with each other , 59. TheThis study examines only a part of the entire guideline development and implementation process – the process of prioritization when deciding on guideline recommendations. Thus, the reader should bear this in mind when thinking about the recommendations and suggestions made. Examining guideline development by observing it over time, documenting members’ experiences, and collecting documents did allow us to investigate the prolonged process of reasoning, judgment, and decision-making, and also permitted validation of findings from different sources. However, the methods used in this study have some limitations. Trustworthiness of studies is always important to consider. The observations are restricted by the ability of the observers to follow and document the process. Observations could have been improved by using video recordings, but the large number of members and the characteristics of the premises made this very difficult. In this study two researches were involved in observations (4 of 9 meetings) and analyses in order to reduce bias and shortcomings. Furthermore, it was presumed that group decisions were based on arguments explicitly articulated during meetings and/or indicated by members in questionnaires. The increase (or decrease) of an articulated argument over time might not entirely reflect its role in the final decision and should be interpreted with caution. It was though considered to be an acceptable indicator of how decision criteria were used over time.This study of group decision making during guideline development illustrates that guideline development is a complex and dynamic process.The studied case has shown that GDGs make more or less transparent compromises among research evidence, clinical experience and ethical considerations. The emphasis on these criteria also varies over time in Prioritization group’s discussions. Guideline development models are often detailed concerning how research evidence should be identified and ranked. However, the model offered few details on how other criteria  should be handled in group decisions, particularly when the research evidence was sparse or ambiguous. Aiming to solve the dilemma of integrating preferences, the practical field of guideline development could benefit from providing more details on how this could be made possible and transparent.To obtain comprehensive advice on generic dilemmas that GDGs encounter, further research on the strategies used and the processes that take place within guideline development is needed, along with assessment of how guidelines that more or less take clinical expertise and experience into account are translated and used in different context.


In its recent evaluation of the US Environmental Protection Agency’s Integrated Risk Information System assessment process, the National Research Council committee singled out the term “weight of evidence” (WoE) for critique, deeming the process too vague and detractive to the practice of evaluating human health risks of chemicals. Moving the methodology away from qualitative, vague and controversial methods towards generalizable, quantitative and transparent methods for appropriately managing diverse lines of evidence is paramount for both regulatory and public acceptance of the hazard assessments. The choice of terminology notwithstanding, a number of recent Bayesian WoE-based methods, the emergence of multi criteria decision analysis for WoE applications, as well as the general principles behind the foundational concepts of WoE, show promise in how to move forward and regain trust in the data integration step of the assessments. We offer our thoughts on the current state of WoE as a whole and while we acknowledge that many WoE applications have been largely qualitative and subjective in nature, we see this as an opportunity to turn WoE towards a quantitative direction that includes Bayesian and multi criteria decision analysis. The inexact science of converting existing environmental health and toxicology knowledge into risk management decisions and policy relies on a growing volume of increasingly diverse scientific data. Past work in this area was guided by a small number of experimental techniques and models. Today, however, toxicity data is much more diverse. It can be collected by different modes, compounded with experiments or models, and often point in different directions regarding the same assessment endpoint. Individual lines of evidence constitute an information base from which conclusions must be drawn regarding public health and economic development. Weight of evidence (WoE) is an approach that, by means of qualitative or quantitative methods, integrates individual lines of evidence to form a conclusion  and has WoE, as an approach, is currently at a crossroads. While there are significant efforts to formalize WoE methodologies, voices of skepticism about the approach’s utility exist as well. For example, the Organisation for Economic Co-operation and Development (OECD) aims to formalize WoE as a process for developing adverse outcome pathways (AOPs) . On the We argue that regardless of the name, integration of individual lines of evidence is an essential component of environmental assessments that should be standardized to establish consistency and comparability across similar efforts. Even though WoE evaluation can be dated back to Greek mythology with images of Themis, the goddess of law and order, holding two scales presumed to hold the balance of weights for or against a certain hypothesis, it was Professor I. J. Good of Virginia Polytechnic Institute and State University who first proposed the WoE methodology as an inherently Bayesian statistical approach . A BayesResults of this rule can be expressed in various ways, notably with the Bayes factor, i.e., the ratio of the posterior odds to the prior odds. In this case, WoE is defined as the logarithm of the Bayes factor . By the While Good continued to expound on the mathematical relationships linking probability and Bayesian models to WoE into the 1980s, the US EPA and other regulatory agencies started to use the same terminology for very different analytical processes see . Linkov Practice of WoE has been scrutinized recently in several academic efforts as well as by National Academies. Most recent criticism of WoE comes from the NRC review of EPA’s IRIS assessment of formaldehyde and methanol . Broad aTo replace WoE as a means for synthesizing multiple lines of evidence to determine hazard, NRC recommends a set of qualitative and quantitative approaches. The qualitative suggestions include Guided Expert Judgment and Structured Processes such as the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system. The NRC warns against utilizing the Bradford Hill (BH) criteria, which are often applied to analyze strength of association, consistency, specificity, temporality, biologic gradient, plausibility, coherence, experimental evidence and analogy ,b, on thAs for quantitative approaches, NRC recommends meta-analysis, probabilistic analysis, and the Bayesian approach as sufficient replacements for WoE to improve evidence integration in the IRIS process . Meta-anOf the three quantitative evidence integration methods suggested as suitable replacements for WoE by NRC, the Bayesian approach is discussed most extensively. The Bayesian approach is noted as “an opportunity to include as much rigor in constructing a formal model of evidence integration and uncertainty as one wants… with a type of theoretical guarantee” . Bayesia1) webtool offers a promising opportunity to validate quantitative WoE approaches using Bayesian statistics in a transparent and reproducible manner. Even though WoE approaches to hazard assessment have relied heavily on subjective inputs and qualitative analysis, Bayesian methods have been used as well. For example, the Optimized Strategies for Risk Assessment of Industrial Chemicals through Integration of Non-Test and Test Information .Using such a MCDA model, we can then calculate a relative weight for each evidence source i, wMCDA approaches to WoE have already been conducted in a quantitative manner. MCDA application to WoE assessment of nanomaterial hazard was recommended in As high impact regulatory decisions continue to be made in hazard assessment and other environmental health applications, the necessity to provide a clear basis for decision-making is becoming increasingly important. WoE, conducted in a manner consistent with its original intent, should be considered as having valuable scientific merit for evidence integration in toxicology. The present state of WoE, and evidence integration in general, is such that approaches to hazard assessments in toxicology differ greatly between applications, among both qualitative and quantitative assessments.2 is promoting this concept , and more generally offer a transparent way to incorporate judgments even when a formal Bayesian approach cannot be used. Combined with more recent proposals to visualize the relative contributions of various data types through the Toxicological Priority Index (ToxPi) , MCDA or


After being absorbed, drugs distribute in the body in part to reach target tissues, in part to be disposed in tissues where they do not exert clinically-relevant effects. Therapeutically-relevant effects are usually terminated by drug metabolism and/or elimination. The role that has been traditionally ascribed to the spleen in these fundamental pharmacokinetic processes was definitely marginal. However, due to its high blood flow and to the characteristics of its microcirculation, this organ would be expected to be significantly exposed to large, new generation drugs that can hardly penetrate in other tissues with tight endothelial barriers. In the present review, we examine the involvement of the spleen in the disposition of monoclonal antibodies, nanoparticles and exosomes and the possible implications for their therapeutic efficacy and toxicity. The data that we will review lead to the conclusion that a new role is emerging for the spleen in the pharmacokinetics of new generation drugs, hence suggesting that this small, neglected organ will certainly deserve stronger attention by pharmacologists in the future. Dick 1977), and this is the question that probably most of us would ask if we were told that spleen could have something to do with drug action or disposition. The spleen is, indeed, barely mentioned in pharmacology textbooks, and among the physiological roles of this organ, such as immunological surveillance, removal of aged blood cells, hematopoiesis and the regulation of blood volume  have been also developed [99mTc-PEG liposomes after repeated administration in rats and monkeys -labeled exosomes into mice. The differential disposition of exosomes in tissues was measured by Morishita et al. [125l-labeled exosomes. More specifically, they collected the exosomes released by B16-BL6 cells transfected with constructs encoding for a fusion protein of streptavidin and lactadherin. Then, these exosomes were made radioactive by the incubation with 125l-labeled norbiotinamide and intravenously injected in mice. After 4 h 28%, 1.6% and 7% of the radioactivity was detected in the liver, spleen and lung, respectively.Exosomes are nanosized vesicular structures released by a variety of cells in physiological conditions or in disease states ,154. Desa et al.  using 12vβ3/β5 integrins and is expressed in exosomes from dendritic cells [Evidence was obtained that exosomes were mainly captured by macrophages because tissue sequestration was significantly reduced when the mice were pretreated with clodronate-containing liposomes, an experimental procedure that selectively depletes MZ macrophages in the spleen . It has ic cells . Exosomeic cells , and eviic cells . Saunderic cells  reportedic cells . The CD1ic cells  showed tIn conclusion, the spleen is a key anatomical station for miRNA and exosome sequestration, and a thorough understanding of the mechanisms involved in this process will be demanded to develop modified versions of these vesicles with the final aim of escaping their splenic capture and increasing their biological activity.The data that we reviewed in the present paper clearly indicate that because of its anatomical characteristics, such as its high blood flow and loose capillaries, the spleen is highly exposed to circulating drugs. Whereas it is unlikely that the spleen could significantly impact systemic disposition of classical drugs, splenic capture of new generation drugs may have important implications by different mechanisms. More specifically, monoclonal antibodies directed against antigens expressed in the spleen directly affect the function of this organ, whereas drug-conjugated monoclonal antibodies exert a local toxic effect upon release of their conjugated chemotherapeutic drugs. Moreover, the spleen represents an important clearance site for exosomes and nanoparticles and may direct immune responses against their PEG stealth. In conclusion, the spleen, this small, neglected organ, continues to surprise us with unexpected physiological roles: after having been recently linked to the pathophysiology of non-alcoholic fatty liver disease , a new r


Throughout the world, there is increasing awareness and acknowledgement of the value of research evidence in the development of effective health policy and in quality health care practice and administration. Among the major challenges associated with the lack of uptake of research evidence into policy and practice in Nigeria is the capacity constraints of policymakers to use research evidence in policy making.To assess the capacity of maternal and child health policy makers to acquire, access, adapt and apply available research evidence.This cross-sectional quantitative survey was conducted at a national maternal, newborn and child health (MNCH) stakeholders' engagement event. An evidence to policy self-assessment questionnaire was used to assess the capacity of forty MNCH policy makers to acquire, assess, adapt and apply research evidence for policy making.Low mean ratings were observed ranging from 2.68–3.53 on a scale of 5 for knowledge about initiating/conducting research and capacity to assess authenticity, validity, reliability, relevance and applicability of research evidence and for organizational capacity for promoting and using of research for policy making.There is need to institute policy makers' capacity development programmes to improve evidence-informed policymaking. In Nigeria, as in other parts of the world, the policymaking process is very well recognized as a complex one. Nonetheless, there is increasing awareness and acknowledgement in the country of the value of research evidence not only in the development of effective health policy but also in quality health care practice and administrationThere are many potential challenges related to research use in MNCH policy making and these challenges also affect other aspects of evidence-informed health policymaking process. Wilson and colleagues13The major challenges and barriers to the uptake of research evidence into policy and practice in low and middle income countries (LMICs) including Nigeria are associated with the capacity constraints of policy makers to acquire, access, adapt and apply available research evidence in policy makingIt is important to state that this capacity constraint regarding acquisition, assessment, adapting and applying research evidence in policy making is not only limited to the individual level but also at organizational levelHowever, in order to design an effective capacity enhancement strategy for evidence-to-policy process, there is need to assess the existing individual and organizational capacity to acquire, access, adapt and apply available research evidence in policy making. According to Deans and AdemokunThis study was aimed at identifying the capacity gaps at individual and organizational levels regarding MNCH evidence informed policy making in Nigeria from which lessons can be drawn for other LMICs. The study was designed to obtain data from national MNCH stakeholders including tacit knowledge and knowledge relating to evidence transfer. This was to serve as a first step towards effort to build national MNCH policy makers' capacity on issues related to knowledge transfer and use of evidence in MNCH policy making.The study design was a cross-sectional survey undertaken during a one-day national MNCH stakeholders' engagement event convened under the auspices of the West African Health Organization (WAHO) and the Federal Ministry of Health (FMOH) Nigeria, in October 2015 in Abuja, Nigeria. A cross-sectional quantitative survey was used because of the need to capture information based on data gathered for a specific point in time . The study design was also used to generate findings and outcomes that can be analysed in order to develop more in-depth research on policy makers' capacity constraints regarding evidence-informed policy making.Among the objectives of the meeting was to assess the capacity of national policy makers and their organizations to acquire, assess, adapt and apply research evidence for MNCH policy making in Nigeria. Participants were officers of senior cadre drawn from the Federal Ministry of Health (FMOH) Abuja and its associated ministries, departments and agencies; others included directors from selected State Ministry of Health (SMOH); executive officers from development partner organizations (DPOs), civil society organizations (CSOs) and non-governmental organizations (NGOs). As part of the inclusion criterial, only individuals from the FMOH, SMOH, DPOs and CSOs/NGOs who were involved in MNCH programmes and policy making/implementation were invited.http://www.whatworksforchildren.org.uk/docs/tools/SelfAssessmtWEB.pdfhttp://www.cfhi-fcass.ca/sf-docs/default-source/documents/SAT-Self-Assessment-Tool.pdfA questionnaire was administered to participants after the completion of an informed consent form. The questionnaire and the informed consent form were approved by the University Research Ethics Committee of Ebonyi State University Nigeria . The questionnaire was designed to assess participants' knowledge, capacity and organizational process of generation, synthesis and utilization of research evidence in policy making regarding MNCH. The questionnaire used for this assessment were modification of the self-assessment tool produced by the Canadian Health Services Research Foundation (CHSRF) available at the following websites:We chose the CHSRF self-assessment tool as the basis from which we developed our data collection tool because a number of reports have demonstrated that it can help projects evaluate their capacity to use research evidence in the design and delivery of servicesacquisition of research evidence;assessing the validity, quality and applicability of research evidence;adapting the format of the researcher results to provide information useful to decision makers;application of evidence in decision making.The questionnaire contained core questions (hints), which assessed the following:The data collected via the questionnaires was analyzed using the methods developed at McMaster University Canada by Johnson and LavisThe range was recorded as the range of values represented by lowest number chosen from the response scale and the highest . The mean was calculated as the numbers chosen from the response scale divided by the total number of responses to the question. While the median was determined by arranging the values chosen from the response scale in ascending order. In terms of analysis, values ranging from 1.00–3.49 points were considered low, whereas values ranging from 3.50–5.00 points considered high.A total of 40 participants signed the informed consent form and completed the questionnaire . A totalThe outcome of the assessment of acquisition of research evidence relevant to maternal, newborn & child health among the stakeholders' participants is presented in The outcome of assessing the validity, quality & applicability of research evidence relevant to maternal, newborn & child health among the stakeholders' participants is presented in The outcome of adapting the format of the research results to provide information useful to decision makers relevant to maternal, newborn & child health among the participants is presented in The outcome of adapting the format of the research results to provide information useful to decision makers relevant to maternal, newborn & child health among the participants is presented in The outcome of the assessment of individual capacity to acquire, assess and adapt research evidence relevant to MNCH among the participants generally indicated low MNRs. However, there are areas where the MNRs were very low suggesting areas of greatest capacity need. In some previous studies, that made use of the CHSRF self-assessment tool, the areas of lowest ratings indicated areas of greatest capacity constraintsThese findings were not a surprise occurrence as they clearly confirmed earlier reports which indicated that many policy makers particularly the administrators lack research skill to acquire, assess and adapt evidence, which has led to a general lack of understanding about how evidence can be used properlyThe mean ratings of the assessment of level of research incentives available in participants' organizations for acquiring, assessing, adapting and applying research evidence was generally very low. The worst outcome was reordered in terms of the organizational capacity for applying research evidence in which majority of the mean ratings ranged from 2.79–2.97. These findings clearly suggest that there is still a huge gap in terms of the organizational structure in place to support the use of research evidence in policy making in Nigeria. For instance, organizational incentive for research acquisition was as low as 2.68 regarding internet facilities; availability of research grants; in-service training, stipends, bonuses & per-diem for research works; provision of research facilities; sponsorship to conferences/workshops etc.Similarly, organizational incentives for assessing the validity, quality and applicability of research evidence was as low as 2.77 especially concerning existence of quality control units; use of consultants/experts; application of computer software, statistical package; and promotion of ethical standards/practices. Also in terms of the organizational capacity to adapt evidence regarding research evidence implementation committee and availability of administrative process for accepting/implementing research evidence the mean rating was very low at 2.78.These findings clearly indicate that there is a very poor research enabling environment in the participants' organizations. Similar organizational capacity constraints were identified in a number of previous studies as serious barriers to evidence informed policy making process. Wilson and co-workersAmong the key organizational factors incapacitating evidence to policy process identified by some other studies was a lack creation of a learning culture and institution of good management - that promote better decision-makingThis development is worrisome and poses a critical challenge to the evidence-informed policy-making process regarding MNCH in Nigeria, because inadequate organizational capacity and commitment towards evidence-informed policy making can incapacitate even the highly knowledgeable and skillful policy makerAlthough the emphasis that policy makers/decision takers ought to be skilled enough to find, appraise evidence or even conduct research may be based on a futile premise. However, it is our opinion that this capacity constraint may be playing contributory to weak health policy making and implementation in Nigeria. Irrespective of the busy schedule of policy makers, if they have better research skill they may find the evidence-informed policy making process a lot easier. This study is intended to promote the awareness of need for capacity enhancement at national level and the employment of people with research capacities in policy making positions. The study has provided base line scientific information that will be vital to the development of capacity enhancement training programmes on evidence-to-policy process targeted at policy makers at all levels of operation and governance in Nigeria.One of the main limitations of this study was the rather relatively small study sample size of 40. A larger sample size may have provided better insights regarding the study outcome. However, our confidence and reliability of the study comes from the fact that we specifically targeted key national policy makers involved in MNCH programmes, and policy making/implementation. Individuals from the invited organizations who were not involved in MNCH were excluded from the study and that accounted for the limited number. Because there was substantial representation of the MNCH policymakers from the federal and states ministries of health, the findings can to a reasonable extent be generalised as far as FMOH and SMOH are concerned in Nigeria. We used the study to gain understanding at a small scale regarding the MNCH policy makers' capacity constraints on evidence-informed policy making in order to develop more in-depth research. A much more robust research methodology than a simple cross-sectional study and which involves expanded scope of MNCH stakeholders is required in future research on this important subject.