In a world beset by attempts to undermine scientific evidence and evidence-based policy, we emphasize their important role in helping humanity rise to the challenges of our time. The scientific enterprise has great potential to benefit all aspects of society, as well as to increase our understanding of the world and how it is changing in response to our actions. The advances brought about by research are too many to enumerate; one only needs to think of our successes in identifying the causes of disease and developing appropriate treatments, or the myriad technological advances that are part of our daily lives. For research to serve its purpose of benefitting society, however, it needs to engender something that is as important as it is fragile—trust. Trust in science is essential for it to effectively inform policy and more fully benefit society.PLOS Biology, founded on the principle of accelerating progress in science, cannot stand by in silence.But this trust is increasingly being undermined across the world by those elected to make decisions in the name of and for the benefit of society. Both public and political leaders once embraced great advances, such as the polio vaccine, and worrying truths, such as the evidence that our use of chlorofluorocarbons (CFCs) was destroying the ozone layer. As a result, mass vaccination campaigns have all but eradicated polio and the Montreal protocol, a global agreement to phase out ozone-depleting substances, saved the ozone layer. Today, many people reject the possibility of using a coronavirus vaccine even as scientists race to develop one, the US President has pulled out of the World Health Organization (WHO) and the Paris Agreement even as the world burns around us, and the destruction of the Amazon has hit a record high under the current Brazilian Administration. Across the world we are witnessing attacks on scientific independence and the undermining of institutions that have been historical pillars of evidence-based policy, such as leading centers for the control of disease and those that protect the environment and regulate medicine approval. We are also witnessing flagrant misinformation and denial of scientific evidence from many leaders who should be acting in the best interest of their population, even when it concerns the most basic of viral transmission control measures or when confronted with satellite images evidencing the truth. Treatments that are the result of many years of research and development are touted as miracles, undermining the role of scientists in bringing them to the clinic in favour of more demagogic interpretations. Withdrawal from essential international organizations and agreements, and even the gagging or discrediting of individual researchers for political purposes—things that would have been unthinkable a few years ago—have come to pass.The independence of scientific research, a crucial tenet on which scientific advance is based, is important to avoid politization of work that is for the most part funded by public money. For science to maximally benefit society and inform policy, research should be conducted in an unbiased manner, and results should be objectively and freely communicated, independently of the agenda of the body funding the research. Likewise, decisions on science funding and support should not be driven by political or personal beliefs.In these times of social media echo-chambers, disinformation and fake news, leaders would be well advised to use their voices to bolster rather than to undermine trust in science, expertise and evidence-based decision-making. Our societies need voices that point to a way forward in the road ahead—with its uncertainties, certainly, but based on the most solid scientific advice available—rather than contribute to the chaos and polarization. There are excellent examples of evidence-based decision-making in response to the COVID-19 pandemic, and these countries are for the most part controlling it with limited disruption to normal life, whereas chaotic responses and mounting death tolls plague those ignoring the scientific advice. We live in extraordinary times, in which belief in the existence of an infectious agent, or that our planet is warming, seems to depend more on one’s general political ideology than on rational arguments.This editorial could single individuals out by name but the sad truth is that these issues are pervasive throughout the world and at all levels of government. Shining a spotlight in any one place would take it away from others. We are confident that our readers will be able to fill in the gaps relevant to their particular circumstances.We are in the middle of a pandemic and are destroying the one planet we have to live on. Science can help us find ways to protect ourselves from the former and protect the latter from us. This, however, requires that we as a global population embrace evidence-based decision-making and trust our experts to be objectively seeking possible solutions. Those of us that have the good fortune of living in democratic societies will have a chance to make our voices heard. We urge you to take advantage of that opportunity, vote and have these issues uppermost in your mind when casting your ballot. Our votes will decide who the next world leaders will be and where they will take us.


Evidence-Based Clinical Outcome assessment Platform (EB-COP) to systematically and transparently evaluate the suitability of TBI COAs for specific purposes.The high failure rate of clinical trials in traumatic brain injury (TBI) may be attributable, in part, to the use of untested or insensitive measurement instruments. Of more than 1,000 clinical outcome assessment measures (COAs) for TBI, few have been systematically vetted to determine their performance within specific “contexts of use (COU).” As described in guidance issued by the U.S. Food and Drug Administration (FDA), the COU specifies the population of interest and the purpose for which the COA will be employed. COAs are commonly used for screening, diagnostic categorization, outcome prediction, and establishing treatment effectiveness. COA selection typically relies on expert consensus; there is no established methodology to match the appropriateness of a particular COA to a specific COU. We developed and pilot-tested the Following a review of existing literature and published guidelines on psychometric standards for COAs, we developed a 6-step, semi-automated, evidence-based assessment platform to grade COA performance for six specific purposes: diagnosis, symptom detection, prognosis, natural history, subgroup stratification and treatment effectiveness. Mandatory quality indicators (QIs) were identified for each purpose using a modified Delphi consensus-building process. The EB-COP framework was incorporated into a Qualtrics software platform and pilot-tested on the Glasgow Outcome Scale—Extended (GOSE), the most widely-used COA in TBI clinical studies.The EB-COP provides a systematic methodology for conducting more precise, evidence-based assessment of COAs by evaluating performance within specific COUs. The EB-COP platform was shown to be feasible when applied to a TBI COA frequently used to detect treatment effects and can be modified to address other populations and COUs. Additional testing and validation of the EB-COP are warranted. Every year, over 50 million people worldwide sustain traumatic brain injury (TBI). The global burden of TBI is enormous, dramatically impacting economic, societal, professional and personal welfare. Apart from the direct effects, which are often life-changing, there is growing evidence that TBI is a risk factor for development of major co-morbidities and neurodegenerative diseases . As suchThere is increasing concern that the use of insensitive, mis-matched, or psychometrically unproven clinical outcome assessment (COAs) measures in drug and device trials may be contributing to the failure to develop effective therapies for TBI –6. Of thThe Glasgow Outcome Scale-Extended (GOSE)  is the mSeveral research funding agencies, including the National Institute of Neurological Disorders and Stroke (NINDS), Department of Defense (DOD), and National Institute on Disability, Independent Living and Rehabilitation Research (NIDILRR) prioritized the need to identify the most robust TBI COAs in the existing pool of outcome assessment tools. To advance this goal, the NINDS launched the Common Data Element (CDE) Project, which aims to improve the precision, reproducibility and cross-study comparability of data elements, including COAs. CDEs have now been established for 23 different neurologic disorders. The first iteration of outcome CDEs for TBI was published in 2010 , followeWhile the development of TBI CDEs has been an important step toward narrowing the field of potential COAs that may be selected for use in clinical trials and observational studies, a key shortcoming is that the CDEs included in the 2013 update were selected by expert consensus, not through evidentiary review. Many of the major TBI research funding agencies now mandate that grantees incorporate the recommended CDEs into their research design. Thus, investigators should have some assurance that the TBI CDEs will perform adequately, and at least as well as other COAs that have not been designated as CDEs.Evidence-Based Clinical Outcome Assessment Platform (“EB-COP”), a semi-automated tool that guides an investigator through key questions designed to ascertain if the COA fits the intended COU, and presents the supporting evidence base for this assessment. The EB-COP extends the Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) [In 2014, the U.S. Department of Defense established the TBI Endpoint Development (TED) initiative  to identify and validate COAs and biomarkers that could serve as potential FDA Drug Development Tools (DDTs) to more accurately detect injury sequelae, identify patient subpopulations most likely to benefit from therapeutic interventions and detect changes attributable to efficacious therapies. In response to Priority #4 of the TED initiative, which called for the development of processes to assess the strength of clinical outcome measures, the authors were awarded a seed grant to design, build, and pilot-test an evidence-based assessment platform to enable the efficient, transparent, and systematic grading of TBI COAs for use in research and clinical practice. This effort culminated in the development of the We convened a multidisciplinary team of experts to serve as the EB-COP Design Team. The team was composed of two TBI COA content experts who also co-led the Outcomes Core for the TED initiative , an evidence-based medicine methodologist (MJA), two database developers  from the American Academy of Neurology (AAN), a medical librarian (PE) from the Mayo Clinic, a regulatory affairs advisor (AR), and two TED post-doctoral fellows . A glossary of key terms and abbreviations is included in Clinical Practice Guideline Process Manual [Finding What Works in Health Care: Standards for Systematic Reviews,” [“Clinical Practice Guidelines We Can Trust,” [Roadmap to Patient-Focused Outcome Measurement in Clinical Trials [COAs should be assessed within distinct COUsOnly studies meeting accepted standards for high-quality methodology , 24, 25 The review process should be systematic and transparent and lead to a clearly stated recommendation.The conceptual framework adopted by the EB-COP Design Team was informed by the AAN’s s Manual , which feviews,”  and “Cli Trust,”  and guidl Trials .” In addThe criteria used to identify high-quality studies  were informed by existing data quality standards, including the AAN’s Guideline Development Process Manual , COSMIN We selected six of the most common COA applications  for inclusion in the EB-COP: 1) diagnosis, 2) identification of TBI sequelae, 3) stratification of distinct TBI subgroups, 4) outcome prediction, 5) sensitivity to natural history changes, and 6) detection of treatment effects. After consulting the peer-reviewed literature –35 and aTo facilitate the COA review process, the EB-COP was subsequently incorporated into a semi-automated on-line platform using Qualtrics software to facilitate data extraction and synthesis. The accompanying EB-COP Manual of Operating Procedures (MOP) see  providesThe EB-COP review process is comprised of six steps as shown in The readiness of a COA for use within a specific COU is framed around an evidence question. The COU describes the: 1) population , 2) COA purpose , 3) concept of interest (COI)- the domain, activity or function assessed by the COA , and 4) COA of interest . The selected COU parameters align with the TBI CDEs. After the user enters these parameters, the EB-COP software generates an evidence question. For example:In adult patients  with acute mild TBI from blunt trauma (Population), for the purpose of predicting outcome at six months (Purpose), is short-term recall (COI) adequately measured by the Rey Auditory Verbal Learning Test [(COA)?ing Test  (COA)?Assessment of the COA begins by vetting the measure against a set of eight “fundamental” QIs that were selected by the Design Team through the modified Delphi process. These are: 1) the availability of documentation on how the COA was developed, 2) specification of the target population, 3) description of the intended COI, 4) information on content validity and 5) face validity, 6) clarity of the administration and scoring instructions, 7) extent of missing data in the population sampled, and 8) feasibility of application within the COU . The EB-COP’s on-line data entry platform defines each fundamental QI and prompts the user to respond to pre-programmed questions that determine whether the criteria for each QI has been met. The EB-COP MOP provides the user with additional guidance for responding to the questions. If the criteria for all eight fundamental QIs are met, the review process proceeds to the next step. If any one of the fundamental QIs fails, the user is advised that the COA is not appropriate for further review.After establishing that the COA is fundamentally sound, the EB-COP prompts the user to perform a literature search to identify published abstracts of relevant, high-quality studies that have investigated the COA within the intended COU.To assist the user in searching the literature, the Design Team adapted a pre-existing COSMIN filter to facilitate capture of articles that address the TBI population. At least two databases  should be searched to help ensure that the review is exhaustive. Book chapters, conference proceedings, dissertations and case studies are excluded from the review. A list of TBI-related search terms that builds on recommendations from the COSMIN group  is incluThe EB-COP excludes clinical trials and observational studies that used (v. investigated) the COA to assess outcomes. This rule was based on guidance from the COSMIN group, who warned that it is not possible to use the COA as an outcome measure  while concurrently testing its responsiveness . If the Step IV is composed of four parts. In Step IVa, the abstracts retrieved in Step III are screened by the investigators to determine which articles should be acquired for full-text review. This is followed by a full-text review of the surviving studies to confirm the study’s relevance and sample size (Step IVb). The strength of the study design and generalizability of the results to the target population are assessed in Step IVc. Finally, in Step IVd, the mandatory QIs associated with the COA’s selected purpose(s) of use are examined based on the data extracted from the literature in Steps IVa-IVc. Step IV is designed to ensure that the final COA recommendations rely only on methodologically sound studies that can effectively evaluate the designated QIs. To control for bias, standardized data extraction forms are used to guide users through each of the steps described above. To help ensure reproducibility of the findings, two independent reviewers are required to screen the abstracts, extract data from the full-text articles, classify the strength of the study design and reconcile any discrepancies detected. After all of the harmonized data are entered into the on-line platform, the Qualtrics software automatically determines the final COA grade, based on the number of the number of mandatory quality criteria met.Step IVa essentially serves as a high-level filter, eliminating articles that are unlikely to inform the evaluation of the COA due to weaknesses in the study design. The aims of Step IVa are to capture the abstracts of all published studies that may be relevant to the defined COU (as described in Step I), and ensure that extracted studies have a sufficient sample size to assess the performance of the COA for the selected purpose(s) of use. The relevance of the study is determined by reviewing the information provided in the abstract about the: 1) population studied , 2) age group  and 3) measurement properties studied . The user also determines if the study enrolled the minimum number of subjects required to achieve the stated aims. If the abstract indicates that the study does not meet these criteria, it is excluded, but if it lacks information or is ambiguous, the study is retained for full-text review. Upon completion of this step, the EB-COP provides a summary of the user’s responses. Step IVa essentially serves as a high-level filter, eliminating articles that are unlikely to inform the evaluation of the COA due to low relevance to the COU or insufficient sample size.After both reviewers independently complete the abstract review and reconcile any disagreements about inclusion or exclusion, the full-text articles are retrieved by the user to confirm that the abstract describes an original research study and meets the requirements for relevance and sample size. Upon completion of this step, the EB-COP provides a summary of the user’s responses and indicates whether the requirements for proceeding to Step IVc have been met.Step IVc is comprised of a series of questions designed to confirm that: 1) the design of the study under review provides adequate protection against risk of bias, and 2) the results can be generalized to the target population. These criteria follow guidelines proposed by the AAN , COSMIN In step IVd, the user assesses the quality of the study investigating the QIs linked to the selected purpose of use. To ensure that only high-quality COA evidence is included in the data analysis and synthesis, only studies that meet all of the mandatory QI criteria are considered in the grading process and in the final recommendation. After all the queries about the selected QIs have been completed and entered into the EB-COP on-line tool, the data are automatically exported into an Excel-based Evidence Summary table, managed by the EB-COP Administrator based at the AAN. The Evidence Summary is then emailed back to the user by the EB-COP Administrator in preparation for the evidence analysis which is performed in Step V. Studies that survive the EB-COP review process produce evidence that is associated with no more than moderate risk of bias as determined by the AAN evidence classification scheme .Grade I—Recommended without Reservation: For the specified purpose of use, all mandatory QIs were found to be “adequate.”Grade II—Recommended with Reservation(s): For the specified purpose of use, all mandatory QIs were found to be “adequate,” except for no more than 2 mandatory QIs that were found to be “undetermined.” No mandatory QIs were found to be inadequate. Additional research is indicated.Grade III—Not Currently Recommended: For the specified purpose of use, all mandatory QIs were found to be “adequate,” however more than 2 mandatory QIs were found to be “undetermined.” No mandatory QIs were found to be inadequate. Additional research is indicated.Grade IV—Not Recommended: For the specified purpose of use, one or more mandatory QIs were found to be “inadequate.”In Step VI, the user reviews the Evidence Summary table, which includes all the data extracted from the high-quality studies reviewed in Step IV. Acceptability cut-offs for all the QIs  are baseThe EB-COP does not stipulate which statistical tests should be used to calculate the cut-off values, but some general considerations are provided in the EB-COP MOP. The final COA grade and recommendation is based on the number of mandatory QIs that meet the pre-specified cut-off values as shown in We pilot-tested the EB-COP on the GOSE because it is among the most commonly used COAs in TBI research. The primary aim of the pilot-testing was to assess the utility of the EB-COP review process and the feasibility of the data entry platform. Pilot-testing was performed by two post-doctoral fellows who first participated in a primer on evidentiary methods conducted by the Design Team’s evidence-based medicine methodologist (MJA). Although we expected the results to be scientifically informative, our primary aim was to assess the EB-COP’s capacity to interrogate the GOSE within a well-defined COU. We chose detection of treatment effects as the purpose of use, given the GOSE’s widespread use in clinical trials. The results of each step are summarized below.In framing the evidence question concerning the GOSE, we selected the following parameters from the EB-COP’s drop-down menus: population = adults with moderate to severe TBI; injury duration = subacute; cause of injury = blunt trauma; Purpose = detection of treatment effects and concept of interest = global outcome. After these parameters  performed a literature search using the terms, “traumatic brain injury,” “Glasgow Outcome Scale” AND “Glasgow Outcome Scale—Extended.” The search was refined by employing the COSMIN filter  that wasNinety abstracts were retained for full-text review after reconciliation by the two independent reviewers (Step IVa). Approximately 6% (170) of the abstracts required reconciliation. Abstracts that could not be definitively excluded were retained for full-text review. By far, the most common reason for exclusion was that the study clearly used the COA to test an intervention, rather than investigating its psychometric properties. During the full-text review (Step IVb), the reviewers excluded 79 studies for relevance because they used (versus tested) the GOSE as an outcome measure. In Step IVc, nine more studies were excluded, either because the study design had increased risk of bias or low probability of generalization. Finally, one additional study was excluded in Step IVd because the study did not include any of the QIs required for detection of treatment effects. w = 0.92, CI = 0.51–1.00) as shown in The data extracted in Step IV were exported into an Excel spreadsheet by the Database Administrator and an Evidence Summary table was returned to the reviewers, indicating successful execution of Step V. Evidence analysis (Step VI) indicated that only one study met all of the EB-COP criteria required to complete the grading process. This study, published in 2003 by Pettigrew and colleagues , investiIt is important to note that two highly cited papers, the original study of the GOSE structured interview by Wilson and colleagues , which aNot currently recommended” for detecting treatment effects on global function in adult patients with moderate to severe TBI assessed in the subacute setting.In the final analysis, while the EB-COP did not find any “inadequate” GOSE QIs, more than two QIs required for detection of treatment effects were characterized as “undetermined.” No purpose-specific GOSE data were available to assess internal consistency, test-retest reliability , longitudinal inter-rater reliability, validity , responsiveness  or floor and ceiling effects. Consequently, a Grade III rating was assigned- “Implications for clinical trial design. An investigator interested in using the GOSE score at 6 months post-injury as the primary endpoint in a clinical trial designed to test the benefits of a novel drug administered within 72 hours of sustaining severe traumatic brain injury would want to be confident that this measure is appropriate for detecting improvements in function that exceed those expected based on knowledge of the natural history of recovery. To achieve this, the investigator should be reasonably sure that the GOSE:Is comprised of items that are inter-correlated, independent of each other and behave lawfully ;Captures a single underlying construct ;Can be reliably administered (interrater and test-retest reliability);Measures behaviors that are tied to functional ability, behaves like other measures of functional ability and predicts future functional status ;Is robust to cultural differences in the target population Detects “true” change  that is clinically relevant across low and high performers .Failure of a measure to demonstrate any one of these features can jeopardize accurate interpretation of the study findings and lead to either over-estimation or under-estimation of the effects of the intervention. The EB-COP guides the investigator through a systematic process that can be used to identify which COAs are best-suited to accurately detect treatment effects. The failure of a measure to meet the mandatory requirements does not mean that the measure should be scrapped, rather it reveals weaknesses that require further investigation before the measure can be confidently used for the desired purpose.The pilot test results demonstrated that the EB-COP platform is a feasible method of assessing COAs within a specific COU. Guided by a Qualtrics-driven software platform underpinned by branch logic, two independent reviewers systematically and transparently navigated a comprehensive evidence-based COA review of the GOSE. Pilot-testing also provided an opportunity to re-organize the order in which some of the questions are presented and to clarify language ambiguities. These changes have been incorporated into the current version of the EB-COP.The success of a clinical trial hinges, in part, on the performance of the selected COAs. Relying on existing literature on psychometric standards, well-established evidence-based methodology developed by the AAN and published FDA guidance, we developed a six-step evidence-based method  to evaluate COA performance within well-defined COUs, and operationalized this procedure on a semi-automated Qualtrics-driven software platform. To our knowledge, this combination of features is not available in any existing systematic review platform.The EB-COP is distinct from the COSMIN guidelines in a number of important ways. First, it extends COSMIN by incorporating QIs that are specific to, and required for, different COA applications. This is critically important as a COA may perform well for one purpose or population, and poorly for another. As an illustration of this point, EB-COP pilot-testing suggested that the GOSE may not be suitable for detecting treatment effects in patients with moderate to severe TBI because most of the mandatory QIs for this purpose of use have not been investigated. This finding is not surprising since the GOSE was originally developed to assess global functional outcome after brain injury, not to detect treatment effects. Second, a modified Delphi consensus process involving a diverse stakeholder group was used to guide development of the EB-COP. The EB-COP’s six-tiered framework, the criteria for grading study methodology and the purpose-specific QIs were all subjected to an iterative consensus-building process. Third, the EB-COP’s grading system only considers evidence associated with moderate risk of bias or lower in accord with the AAN evidence classification scheme , ensurinThe EB-COP can be viewed as a decision-making tool that minimizes historical, subjective and idiosyncratic biases that influence COA selection in both clinical practice and research trials. We wish to emphasize that the EB-COP is not intended to be the only driver in determining which COAs should be selected for a specific TBI research application. Other factors, both practical and scientific, will also play a role. For example, investigators are likely to encounter situations in which a particular COA is not recommended for a specific purpose of use because of missing or inadequate QIs, but a suitable alternative is not available. Under these circumstances, some judgement will be required to determine whether the level of risk associated with the measure is acceptable, based on the number of missing or inadequate QIs, other COAs included in the assessment battery, the study aims and other considerations.While our original focus was on TBI COAs, the EB-COP can be modified to address other populations and expanded to incorporate additional applications. From a regulatory standpoint, the EB-COP can sharpen the process used to determine the readiness of a COA for use as a clinical endpoint in drug development. This is consistent with the Agency’s efforts to encourage “community collaboration in the development of COAs for unmet measurement needs”  and suppThe EB-COP has some limitations. Although selection of the QIs used in the EB-COP review process was guided by existing psychometric standards and approved by COA content experts from scientific, regulatory and clinical sectors using a formal consensus process, it is important to show that the grading scheme has criterion validity. This is typically accomplished by comparing a new measure to a “gold standard.” While no clear-cut gold standard exists, one approach might be to compare performance ratings generated for the same COA by the EB-COP and the COSMIN Risk of Bias Checklist . A secona-priori-defined COUs. This feature is critically important as it considers the psychometric strengths and limitations of the instrument in relation to the target population and intended purpose. These characteristics may help reduce type II error in research and improve diagnostic and prognostic sensitivity in clinical settings. Although designed for use in TBI, the underpinnings of the EB-COP framework are cross-cutting and may be applicable to other health conditions.We developed a semi-automated evidence-based platform  to facilitate the evaluation of TBI COAs for use in research and clinical practice. This tool is unique in that it includes quality indicators that assess the performance of TBI COAs within specific S1 File(DOCX)Click here for additional data file.S2 FileThe MOP guides the EB-COP user through the review process by asking a series of questions about psychometric indicators that are important in judging the relevance and strength of the COA. Links are provided in the MOP to the on-line Qualtrics Survey Software System. The software platform streamlines the review process by efficiently navigating the EB-COP user through the review process, and ultimately produces a recommendation.(PDF)Click here for additional data file.S3 FileThe EB-COP search filter extends the filter recommended by COSMIN by adding terms that help identify studies focusing on TBI outcome measures.(DOCX)Click here for additional data file.S4 FileIn adult patients with mTBI of subacute (<6mo) duration, for the purpose of (1) detecting PCS, (2) stratifying sub-types, or (3) monitoring the resolution or progression of PCS, are PCS adequately measured by the English version of the patient-reported RPQ? Seventy-nine full-text articles were reviewed by two independent reviewers and four qualified for EB-COP review. The two most common reasons for exclusion were study design  and lack of relevance to the evidence question. Results indicated that there is paucity of relevant, high-quality evidence addressing the performance of the RPQ within the three COUs assessed. As such, the RPQ was not recommended for these applications.The RPQ is a widely-used patient-reported measure of symptom severity following mild TBI. In this study, we assessed the Rivermead to address the following evidence question: (PDF)Click here for additional data file.


Low health literacy is an issue with high prevalence in the UK and internationally. It has a social gradient with higher prevalence in lower social groups and is linked with higher rates of long-term health conditions, lower self-rated health, and greater difficulty self-managing long-term health conditions. Improved medical services and practitioner awareness of a patient’s health literacy can help to address these issues. An intervention was developed to improve General Practitioner and Practice Nurse health literacy skills and practice. A feasibility study was undertaken to examine and improve the elements of the intervention. The intervention had two parts: educating primary care doctors and nurses about identifying and enhancing health literacy  to improve their health literacy practice, and implementation of on-screen ‘pop-up’ notifications that alerted General Practitioners (GPs) and nurses when seeing a patient at risk of low health literacy. Rapid reviews of the literature were undertaken to optimise the intervention. Four General Practices were recruited, and the intervention was then applied to doctors and nurses through training followed by alerts via the practice clinical IT system. After the intervention, focus groups were held with participating practitioners and a patient and carer group to further develop the intervention. The rapid literature reviews identified (i) key elements for effectiveness of doctors and nurse training including multi-component training, role-play, learner reflection, and identification of barriers to changing practice and (ii) key elements for effectiveness of alerts on clinical computer systems including ‘stand-alone’ notification, automatically generated and prominent display of advice, linkage with practitioner education, and use of notifications within a targeted environment. The findings from the post-hoc focus groups indicated that practitioner awareness and skills had improved as a result of the training and that the clinical alerts reminded them to incorporate this into their clinical practice. Suggested improvements to the training included more information on health literacy and how the clinical alerts were generated, and more practical role playing including initiating discussions on health literacy with patients. It was suggested that the wording of the clinical alert be improved to emphasise its purpose in improving practitioner skills. The feasibility study improved the intervention, increasing its potential usefulness and acceptability in clinical practice. Future studies will explore the impact on clinical care through a pilot and a randomised controlled trial. Health literacy can be defined as ‘the motivation, knowledge and competencies to access, understand, appraise and apply health information in order to make judgments and take decisions in everyday life concerning health care, disease prevention and health promotion to maintain or improve quality of life throughout the course of life’ . Health Health literacy is a balance between peoples’ capacities and the complexities of the health system ; thus, iIn England, nearly all general practices use electronic health records (EHRs) for clinical care . IncreasPrevious work, using an English working-age population functional health literacy dataset, has determined a threshold of literacy and numeracy skills required to understand and use health information in common circulation in the UK . BuildinThe current study aimed to build on this previous research by (i) developing an evidence-based intervention that incorporates health literacy awareness training to General Practitioners and Practice Nurses and applies an automatically generated health literacy alert within the General Practice EHR system, (ii) investigating the feasibility of this intervention from the perspectives of participating health care staff and a patient and carer group, and (iii) gathering the views of practitioners and members of a patient and carer group on how the intervention could be improved. The aim of the intervention would be to improve practitioner health literacy awareness, skills, and practice to improve the patient experience.In order to ensure that the intervention components reflected best current knowledge, two rapid reviews were undertaken, one for each component. Rapid reviews aim to retain the rigorous approach undertaken in systematic reviews whilst streamlining the approach to enable identification and synthesis of evidence to be undertaken in a timely manner . The rapThe rapid reviews were limited to peer-reviewed publications available in English. Papers that described a protocol only were excluded. Since both key papers were published in 2009, the search was undertaken from 2008 onwards to the date of the searches (2017). Both searches were designed and tested in MEDLINE  database of journal citations and abstracts for biomedical literature) and then translated and run in the following databases: MEDLINE, Embase (the Excerpta Medica Database), CINAHL  and EPOC (the Effective Practice and Organisation of Care). Example MEDLINE search strategies for the two rapid reviews are shown in At the commencement of the intervention development, it was decided to have two components (1) GP and practice nurse training in health literacy and (2) automatically generated ‘pop-up’ alerts appearing on the GP clinical system consultation screen. The GP and practice nurse training was based on one-day health literacy training sessions that had already been shown to improve practitioner knowledge and confidence . To makeThe IT system coding to generate the clinical alerts was written for SystmOne, a clinical IT system used by more than 2700 of the 7000 General Practices in England . All praGP evaluation: Post-intervention focus groups were conducted in two of the four participating practices to explore key elements of the intervention and study design. Questions explored the acceptability of the study to GPs; ways to encourage participation; the feasibility of the intervention; the GP training and how it might be improved; whether patients or GPs might find elements of the intervention unhelpful and/or intrusive and, if so, how this might be addressed; and the optimal threshold for ‘pop-up’ frequency.Service User Evaluation. The results of the study were presented to members of Voice, a Newcastle-based service user and carer organisation . Views wNHS Ethics approval was obtained from the London City & East Research Ethics Committee reference 17/LO/1277. Low literacy and numeracy carry considerable stigma . We sougThe outcome of the rapid evidence scans of (a) health professional training and (b) factors facilitating the impact of clinical system alerts on clinical practice;Practice recruitment;Qualitative thematic analysis of the post-hoc focus groups of (a) primary care practitioners and (b) patients and carers.This was descriptive and included:The flow chart for this rapid review is shown in Key facilitators for effective health professional training relevant to the intervention being developed were inclusion of mixed teaching methods such as a mixture of didactic teaching and practical/participatory methods ,28,29,30Mono-component training was found to be less effective than multi-component teaching, particularly when didactic-only , and thiThe flow chart for this rapid review is shown in Elements of clinical alerts that promoted effectiveness were those that were directed at changing clinician behavior ; availabThe training element of the intervention  already The clinical alert component of the intervention was being developed de novo and was thus designed to include all the elements promoting effectiveness. The alerts were designed to remind the clinicians to improve their health literacy practice, as learned in the training session. The alerts automatically appeared during the clinical encounter. To maximise specificity of the alerts, the algorithm was triggered if the patient characteristics indicated a 70% or greater likelihood that the patient would fall below the national health literacy thresholds, developed in earlier work by GR et al. . The aleTwelve general practices were eligible as they used the SystmOne clinical system. All were approached. Four practices were recruited, giving a recruitment rate of 33%. Training for three practices was delivered in one group teaching session, and one practice received training via a webinar. Following the training (within two weeks) the clinical alerts were activated in the practices for eight weeks.Focus groups were undertaken at two of the four participating practices. In both, feedback was that the intervention had worked smoothly, the training was well received and effective, and that clinical alerts appeared as expected in a corner of the screen when patient notes were accessed. There was positive feedback on the clinical alerts; participants reported that the alerts reminded them about the training they had received: “it definitely jogged our memory when we saw it”, “it did make me for a few seconds think “Right, I just need to try and speak to them appropriately””. Practitioners did, however, raise some hypothetical concerns, i.e., that patients could be offended if they became aware of the alerts during the consultation. The length of time for which the clinical alerts would be effective was also raised, with staff suggesting that eight weeks was too long, as they felt that effectiveness was diminishing “after a few weeks”. It was highlighted that practitioners get many clinical alerts and “the constant having to click the screens off before you can get started” can leave practitioners frustrated and reduce effectiveness.The practitioners fed back that the training had built their awareness of health literacy; “it really made me think about when I’m in a consultation with patients that actually they might not always understand”. Examples were given of improved practice in communication with patients in the consultation: “it may well have made me think about how … I spoke to the patient … medical jargon”. Perceived gaps in the training were the lack of information about how health literacy differs from learning disability, and the need to include building skills to discuss health literacy with patients. In addition, some practitioners did not understand that the alerts were based on a person’s likelihood, from their socio-demographic characteristics and area of residence, to be below the health literacy threshold, rather than being diagnostic. One participant said “one time the alert came up for a teacher and I thought—that can’t be right…”. Another practitioner, however, recognised that patients with higher everyday life skills might still have low health literacy: “we start spouting off really, you know, medical terminology, different procedures, and things, then actually, unless you work in that area, no matter how intelligent you are, you’re not actually going to know what those terms mean”.One practice received training via webinar. Staff from this practice stated that, whilst the training had been valuable, they would have preferred face-to-face teaching. To help resolve the concern that patients might see, and be offended by, the alerts, practitioners suggested that the wording be tailored to focus on the practitioner rather than the patient. Suggestions for the improved wording included “consider practitioner communication skills”. The practitioners felt that improving the wording would lead to the alerts becoming a “positive thing that’s a reminder (to the practitioner)”, as opposed to something which makes the clinicians “uncomfortable or the patient awkward”.Suggestions for improving training included clarification about the difference between low health literacy and learning disability, and inclusion of role play training on how to have discussions with patients who might notice the pop-ups on their screens. Practitioners also suggested that the training included explanations about how the alerts are generated, i.e., that the patient socio-demographic data and area of residence are used to generate a risk score, which is an indicator rather than being ‘diagnostic’.Practitioners suggested that one approach to reducing the risk of the alerts upsetting patients was for clinical staff to open the patient screens and acknowledge and close any clinical alerts before the patient was called into consultations. Some practitioners felt that this is would not always be possible, citing time constraints. Another suggestion to reduce the risk of patients feeling offended by the alerts was to take a ‘whole practice’ approach to the intervention. It was suggested that information could be prominently displayed throughout the practice, explaining that the practice is seeking to improve communication with patients, and that this would include practice training and reminders to practitioners about communication during some consultations. The results of the study were presented and discussed, as was the feedback from the health care practitioners. The patient and carer group fed back that, since the study was focused on improving the consultation experience for patients with low health literacy, patient recruitment would be best conducted with telephone contact rather than by written invitation. The group supported the practitioner views on improving the training, improving the wording of the clinical alerts, and adopting a practice-wide approach as part of the intervention.The rapid review process ensured that both components of the intervention were in line with current evidence and designed to maximise the effectiveness of the intervention. Key factors in maximising the effectiveness of the educational component were to ensure multi-component training involving small group work and role-play, identification of barriers to change and development of plans to overcome those barriers, and involvement of a local opinion-leader in the training. Key factors in maximising the effectiveness of the clinical alerts were the development of stand-alone, automated, ‘push’ alerts that appeared in the consultation and required acknowledgement from the clinician. The alerts only appeared for patients at high risk of low health literacy and only in consultations where additional time was given and the focus of the consultation was on building patient capacity to self-manage long-term health conditions.Four practices were recruited (33% of eligible practices). Feedback from clinicians was that the intervention had increased their knowledge and skills in relation to health literacy. Feedback included suggestions for improving the intervention, in particular training on how health literacy differs from learning disability, and how to have discussions with patients around health literacy. The feedback from the practitioners also indicated that more information should be given about how the alerts are generated and what they indicate, i.e., that they are not ‘diagnostic’ of low health literacy but rather act as a reminder to the practitioners to use the knowledge and skills acquired through training, and will appear when patients are at greater risk of having low health literacy in view of their socio-demographic characteristics and where they live. As described above, the training had been shortened to 3 h to make it more attractive to busy practitioners, but this feedback indicates that a full day of training may be required to cover all the relevant issues. Suggestions for improving the clinical alerts were to improve the wording to focus on practitioner skills rather than patient skills and to reduce the period of time during which the alerts would appear from the current eight weeks, although no specific time was suggested. A key suggestion was that the intervention should involve the whole practice, with information about health literacy and the purpose of the training and clinical alerts being to improve practice staff communication and consultation skills. The project used an evidence-based step-wise approach to developing the intervention, including building on current knowledge through rapid evidence reviews, and post-hoc feedback from clinicians and a patient and carer group. A key limitation is that, whilst health literacy is a holistic concept that reflects a dynamic balance between the capacities of individuals and the health and wider societal settings in which they are living, the algorithm used in this project was based on the only health literacy data available in England, i.e., functional health literacy. Functional health literacy is correlated with more holistic health literacy capacities, but whilst the correlation is statistically significant, it is only moderate in size . It shouAs this was a development project, no quantitative assessment of the impact of the intervention on clinical communication skills or patient outcomes was undertaken. Assessments of the revised intervention should include more detailed practitioner feedback on the impact of the training on their clinical communication skills, and what ‘added value’ the alert component brings. The current recommendation is for a ‘universal precautions’ approach to health literacy practice, i.e., the provision of high-quality care to all irrespective of health literacy capacities . Whilst To our knowledge, this is the first intervention to use a combination of clinical training and automated clinical alerts on electronic health records to improve clinical health literacy skills and practice. The rapid review process ensured both components of the intervention were evidence based. Post-hoc feedback from clinicians and patients and carers further improved the intervention. The one practice that received web-based training stated that they would have preferred face to face training; this may have implications for scaling up any future interventions. Key limitations identified in this study should be explored in studies of the revised intervention. Health care practitioners have a key role in developing the health literacy responsiveness of health systems to ensure that they address the health literacy needs of the people they serve. The approach to intervention development taken in this project has resulted in an evidence-based intervention that has been further improved by feedback from clinicians who undertook the training and from a patient and carer group. Further work to develop the intervention will be undertaken through more feasibility testing followed by a pilot- and then full-randomised controlled trial. Since the aim of the intervention is to improve the health literacy awareness, skills and practice of health care practitioners, suitable measures of change will be those that capture patient views on practitioner communication, such as those used in the NHS England GP Patient Survey .


Despite a recognized need for midwives to provide post abortion care, there exist barriers preventing them from integrating lifesaving skills such as manual vacuum aspiration (MVA) into practice. This collaborative research with the Professional Association of Congolese Midwives (SCOSAF), sought to understand how certain midwives in the Democratic Republic of Congo (DRC) have overcome barriers to successfully integrate MVA for post abortion care. Specifically, in order to provide locally-driven solutions to the problem of inadequate post abortion care in the DRC, this study aimed to identify examples of positive deviance, or midwives who had successfully integrated MVA in complex working environments following an in-service training facilitated by their midwifery association, SCOSAF.Creswell’s mixed method comparative case study design was used to identify positive deviant midwives who had practiced MVA one or more times post training and to explore their strategies and enabling factors. Other midwives who had not practiced MVA post training permitted for a comparison gro cup and further interpretations. Sources of data included a sequential survey and semi-structured interviews.All 102 midwives invited to be surveyed were recruited and 34% reported practicing MVA post training (positive deviant midwives). No statistical significance was found between the two groups’ demographics and practice facility type. Overall, both groups had positive attitudes regarding midwifery-led MVA and legalization of abortion. Positive deviant midwives demonstrated and described more confidence and competence to practice and teach MVA. They were more likely to identify as teachers and overcome interprofessional barriers by teaching MVA to physicians, medical students and other midwives and position themselves as experts during post abortion emergencies.Results provided important insight to midwives’ integration of post abortion care in Kinshasa. Strategies used by positive deviant midwives in emergencies allowed them to navigate challenging contexts in order to practice MVA, while simultaneously increasing the credibility of their profession and the dissemination of evidenced-based MVA practice. Programs designed to work with and promote positive deviant midwives as knowledge brokers could be tested for their overall impact on the diffusion of midwifery-led MVA to improve access to safe, respectful reproductive care.The online version contains supplementary material available at 10.1186/s12913-020-05997-7. A critical need in many countries is adequate care arising from unsafe abortion practices. Restrictive abortion laws, conflict, displacement, lack of economic autonomy, war-based rape and violence, and legal and cultural restrictions contribute to women’s high rates of unsafe abortions and related complications –4. The mDeaths due to unsafe abortion could be reduced from 22,000 to 9000  per yearCurrently, governmental policies and programs in the DRC have gaps regarding midwifery provision of post abortion care, but there are emerging efforts to address these issues , 20. TheContrary to a barrier-focused approach to health services research, positive deviance involves the identification of best practices within a community, resulting in better outcomes . In otheWhile positive deviancy has been used in similar contexts to explain better maternal health outcomes , this apWho are the positive deviant midwives who integrated MVA into practice post training?What strategies and factors facilitated adopting this skill?How do they compare to midwives who have not integrated MVA?Our approach was to identify and examine “positive deviants” that is, those midwives who overcame barriers to providing post-abortion MVA. Particularly, we aimed to explore those strategies being successfully implemented and the role they had in fostering improvement, and to understand how they compare to midwives who had not yet integrated the skill. We developed the following research questions:Positive deviant scholars recommend mixed-method design to identify and describe positive deviant individuals 10). Comparison to non-positive deviant individuals can aid in properly identifying behaviours and factors unique to positive deviants (44). Our research used Creswell’s mixed method comparative case study design to address the research aims . We iden0. ComparPrior to data collection, we obtained ethics approval from the National Committee of Ethics of the Kinshasa School of Public Health in Kinshasa (#ESP/CE/088/2019), Democratic Republic of Congo and the Research Ethics Board (#6017245) at Laurentian University, Canada. Written informed consent was obtained from each participant.Midwives considered eligible for the study had completed a 7-day emergency obstetric and neonatal care (EmONC) training facilitated by SCOSAF between September 2017 to March 2018, during which 350 were trained from the nine major health zones in the province of Kinshasa. Trainees worked primarily in labour and delivery and varied in their work contexts and experience.Sample size of 5% prevalence of post-training MVA practice was based on previous post-training evaluations of Congolese midwives’ report of frequent barriers to MVA integration , PreviouThe number of positive deviant midwives (cases) was hypothesized to be few. Therefore, the case group was bound by the deviant behaviour or outcome of practicing MVA for post abortion care one time or more since SCOSAF’s training. The comparison group of non-positive deviants was defined as midwives trained who had not practiced MVA for post abortion care since the training.The design of our study required a multi-stage sampling approach for recruitment. We sequentially collected two sources of evidence between June 2019 and January 2020 to conduct the case group comparison. In stage one, to maximize the number of positive deviants, the local research team and KMB used the SCOSAF training database to create a list of all trained midwives working at health facilities offering MVA for post abortion care. The team took health facility characteristics into consideration to ensure diverse representation of working environments.We combined two context specific tools to create the survey for stage one of the research  perceptions of the innovation; 2) characteristics of individuals who do or do not adopt the innovation; and 3) the context in which the innovation is introduced . Within We descriptively summarized the survey responses from stage one for the overall sample of respondents. Participants’ characteristics included socio demographics, professional experience, MVA practice, knowledge and attitudes regarding their role expansion and willingness to provide MVA for post abortion care. Questions regarding attitudes and values of MVA were measured using a Likert scale, which were added to create an overall score , 46. ParUsing SPSS software, we conducted a bivariate analysis between case and comparator groups. We used chi-square tests for categorical variables. Due to non-normal distribution of attitudes and values about MVA, we analyzed for differences in the groups’ scores using the non-parametric Mann Whitney-U test. Probabilities of less than 0.05 rejected the null hypothesis of no differences between the two groups.Semi-structured interviews conducted in Lingala were forward translated to French and back translated by another researcher to ensure conceptual equivalence  . InterviData integration occurred at three points: The first point involved analysis of: 1) survey results between the groups of positive deviants and comparators; 2) qualitative findings from interviews between the two groups; and 3) a side-by-side comparison of results from both sources of evidence to locate convergences or divergences while exploring factors associated with positive deviant behavior. Second, integration occurred when results for each theme were compared between the groups. The third integration involved producing a narrative and joint display.In the DRC, abortion is legally restricted. As a result, clandestine abortions are common and stigmatized , 8. Our 3As midwives belonging to the midwifery association, the team of researchers held a unique and important insider perspective regarding the provision of post abortion care, and a high degree of cultural sensitivity and understanding in relation to the research topic (37). Cultural competence is known to increase the researcher’s ability to properly represent reality and foster trust with participants in abortion research contexts . Furthern = 102) midwives participated in the study. The social, demographical and professional characteristics of the participants and subset are presented in Table 1. A total of one hundred and two n = 10 midwivesA subset of twenty-seven (27) participants were interviewed. We reached saturation with 16 positive deviants and 11 comparators. No potential positive deviants refused to be interviewed, but 11 comparators selected for interviews could not be reached due to changes in contact information. All comparators reached agreed to participate. Table 2.1) Midwifery perceptions of MVA, abortion and midwifery scope of practice, 2.2) Midwives’ confidence and competence of MVA, and, 2.3) Immediate working context. Each category was further divided into sub-themes. The following narrative is an interpretation of data integration and a comparison of positive and non-positive deviant midwives by categoric theme. Divergences and convergences are additionally explored.p = 0.164). The interviews revealed similar changes across groups in perspective and knowledge of midwives’ role with regards to post abortion care from SCOSAF’s MVA training. Specifically, they expressed the opinion that midwives’ involvement in MVA practice increased the credibility of their profession and is an essential skill, as midwives are well positioned and capable of saving the lives of women who had undergone clandestine abortions.In the survey, groups demonstrated equivalent knowledge of the midwifery scope of practice in the DRC . When we followed the training and especially when we had just learned that MVA is part of the field of midwifery practice, (..) since then I work in collaboration with the doctors when it comes to the use of MVA4(case group 1: participant 1).In relation to their ability to integrate MVA, positive deviants described how SCOSAF’s training allowed them to fully understand their autonomy and the key role they hold within the health care system to decrease maternal mortality due to unsafe abortions. This newfound empowerment and self-perception as an MVA provider were catalysts to practice and evolved their own professional identity in favour of practicing MVA:It gave me value and consideration in the eyes of others...I am happy to be the champion and leader in my structure, because it is thanks to me that everyone practices MVA (case group 1: participant #14).Positive deviants also associated their practice of MVA with its overall positive impact on midwifery by increasing midwives’ credibility with colleagues and clients. They saw MVA practice as honourable and important for midwifery and the advancement of maternal and child health. Indeed, this new sense of autonomy and credibility allowed them to advance their own position as midwives within their hospitals, often evolving into leadership roles and propagating the use of MVA:It is really very important that the midwife performs MVA to save the lives of women who die every day as a result of abortions...it should also be noted that MVA is part of the scope of midwifery in the DRC (Case group 2: Participant 25).Comparators had similar knowledge and views of MVA within their scope of practice. In interviews they revealed no negative opinions or views regarding MVA being within the scope of midwifery practice for post abortion care. However, they did not apply this to their own personal practice or lack thereof:p = 0.92). During interviews, both groups acknowledged the rate of unwanted pregnancies as too high in their country, contributing to higher levels of deaths due to clandestine abortions. They expressed belief that midwives must address this problem by advocating for better access to contraception, decreasing illegal activities and improving abortion laws. Both groups spoke of inequities in their health care systems that lead to unsafe abortions, that the system was unfair and had a negative impact on women and girls.I think that many girls or young people resort to clandestine abortions because the Ministry of Health has not set up a safe abortion process, many women do these abortions because their pregnancies are unwanted (case group 1: participant 1).Overall, midwives expressed positive attitudes and professional intent to practice post abortion care. In the survey, midwives scored similarly to questions regarding their willingness to provide post abortion care, to be publicly known as a post abortion provider, and their beliefs of abortion legislation. Mean scores were high at 86.91% (mean S.E. 1.18) for positive deviants and 85.18% (mean S.E. 1.52) for comparators, with no statistical difference between the rank mean scores .Midwives discussed the importance of improving their country’s policies and programs to improve access to birth control and abortion and were clear that midwifery needs to be present at all levels of implementation:p <  0.0005). Positive deviants were more likely than comparators to hold a formal or informal teaching role (p = 0.002). They were also more likely to express confidence in their ability to teach MVA (64.2% compared with 94.3% of comparators expressing only somewhat confidence to teach MVA).In the survey, midwives were asked to rank their confidence to practice and teach MVA post training. Positive deviants were more likely than comparators to report being very confident, while comparators were more likely to be somewhat confident to practice MVA .In the interviews, midwives were asked to describe their levels of confidence including what factors impacted their confidence to practice MVA. Positive deviants used words such as self-worth, personal conviction, motivation and courage to describe the internal or inherent factors that increased their confidence to practice, particularly with regards to the first time they practiced MVA.My level of confidence is stable because I master the technique perfectly and I also train others (...) I have even become a leader in my structure (case group1: participant 6).Midwives described that successfully performing MVA for the first time under extremely stressful conditions reinforced their confidence by positively impacting their self-respect, self-worth and perceived place within their interdisciplinary team. This new confidence as a team member led them to share their knowledge by teaching and training others to use MVA. Being a teaching resource or reference point in their hospital also positively impacted their confidence as practitioners.I think I’ve mastered the technique with what I see from my colleagues. But I just lack determination and a little bit of decisiveness because I wasn’t interested in it at all (case group 2: participant 19).Conversely, comparators described an inherent lack of motivation or interest to practice MVA or to ask someone to mentor them. Midwives suggested that it was purely a matter of self-determination and a personal decision to motivate themselves to practice or to ask for help:The element that impact my confidence is the lack of practice after the training because if you have learned something and you don’t practice, you forget certain steps, but I would need someone to assist me the day I do the MVA (case group 2: participant 22).Some described that their confidence to practice was poor due to the length of time since their training, or lack of exposure in clinical practice. This led to needing more mentorship or supervision, or even more training so they could be confident enough to practice:Yes, I would like to practice MVA 1 day and I know that 1 day I will do it because with the EmONC training and as I already assist those who do it if a case happens and I am alone I will do it. (case group 2: participant 18).Many comparators were working in contexts where MVA was practiced by either physicians or other midwives. They seemed to be content to support their colleagues in MVA procedures instead of performing MVA themselves. However, if faced alone with a clinical situation that demanded MVA, they often described full intent to provide the necessary care, regardless of their confidence levels:p = 0.001). They were also more likely to identify SCOSAF as the primary source of support for successful integration (p = 0.010). In the interviews, both groups discussed the importance of receiving MVA training from SCOSAF and its positive impact on their competence. Discussions differed between the groups based on how much the training and perceived competency impacted their confidence to actually practice on their own. The comparators expressed more reticence towards practicing and discussed needing ancillary supports before they felt completely confident:The factor that impacts my confidence is the EmONC training (...) but I still need to be supported by the experts when I do it for the first time (box group2: participant 20).The most common external factor that influenced midwives’ confidence to practice MVA was their SCOSAF training. In the survey, positive deviants were more likely than comparators to report they felt competent following the training .However, perceived competence achieved from the training to practice MVA did impact positive deviants’ practice, compelling them to apply their knowledge:p = 0.24), however interviews with comparators revealed that their confidence had waned over time:At the moment my confidence level is a bit low (...) which means that for the moment before practicing I first have to redo the theoretical and practical training we received at SCOSAF (case group 2: participant 23).In the survey, length of time since being trained (2017 or 2018) was not associated with whether midwives had initiated MVA practice (p = 0.009) and to have received complementary MVA training (p ≤ 0.002).Another influence appeared to be the amount of training and experience midwives had had prior to the training. The survey revealed that positive deviants were more likely than comparators to have previously assisted with MVA procedures prior to their training (Results for context are divided into two categories addressing: 1) the midwives’ immediate work environments such as supply and equipment and 2) working relationships, particularly in the working environment at the actual moment of the emergency or MVA practice.MVA integration did not differ whether midwives were working for public or private entities nor if their hospital provided basic or comprehensive EmONC care. During the interviews, several participants referred to their hospital-specific training which had left mannequins for training and equipment at their disposal.p = 0.042). MVA equipment was either made available by the hospital, which in many cases was due to the donations of an organization, belonging to a medical colleague, or the midwife themselves. Midwives described MVA availability as inconsistent, caused by stock outages or doctors taking the equipment home with them. This midwife described how she stowed her own MVA to guarantee consistent access in emergencies:Then we brought her to my office and as all the materials are in there, I suctioned out the contents and in a few minutes the woman was saved (case group 1: participant 6).Having immediate access to MVA equipment was statistically associated with practice (p = 0.26), both groups discussed the importance of care provider relationships throughout the interviews. Predominantly, midwives discussed who was present or not during emergencies making clear that the interprofessional encounter was more relevant than overall institutional environment. Interprofessional relationships were most often with physicians followed by medical students or midwives.While the survey found no significance between support from colleagues and MVA practice .Positive deviants in the private sector revealed competing financial interests between themselves and physicians. More complicated procedures, such as dilatation and curettage, cost more for a patient and therefore, were more profitable for physicians. When physicians were present, midwives were less likely to manage the client’s care. MVA profits also caused interprofessional tensions, resulting in a reluctance on the part of some physicians to allow midwives to perform post abortion care.The factors that inhibit me are the lack of support from my colleagues because the materials are there and I often assist them, but I think that I too must have the courage (case group 2: participant 18).For comparators, competing interests between physicians and midwives and their own motivation to practice were described simultaneously. While they expressed a desire for more support from their peers, they often stated that midwives and physicians were already practicing in their setting. They described both a need for their peers to be more inclusive of them, while admitting they could also be more confident and motivated to include themselves:the woman was bleeding so badly that we couldn’t wait many minutes because her life was in danger. I and the other midwife had decided to do the MVA (...), and we aspirated (...) I actively assisted and we were able to save that woman’s life (case group 2: participant 26).Many comparators discussed collaborations with their peers in order to assist them during emergencies and described a sense of accomplishment in these collaborations in order to save a life:The doctor who was there asked that we could prepare the material for the curettage, as I had just been trained and we still had the MVA syringe that had never been used; I proposed to the doctor that we could first try the new aspiration technique; after my explanations on the advantages of the technique, they accepted and I did so in front of the doctor and my colleagues who also learned that day (case group 1: participant 14).The doctor on duty did not have much knowledge of the technique, so I suggested if I could perform MVA. The doctor and my team trusted me and assisted me from start to finish (case group 1: participant 15).Positive deviants also described their interactions with physicians during emergencies as educational encounters. If the physician did not know the technique, the midwife would capitalize on the occasion to inform and teach. Midwives discussed how they were in many cases able to convince their peers of the benefits of MVA during the emergency:I did it 1 day in an emergency and the doctor was busy, and we saved this woman together with the trainee doctor...I master the technique well and I can do it alone without supervision and I teach the other doctors to do it (case group 1: participant 5).Teaching during one emergency could lead to the midwife teaching others as other emergencies arose:Our research found that positive deviants encompassed all ages, types of midwives, experience levels and facilities. While their perceptions of the scope of midwifery and MVA for post abortion care differed little from comparators, one distinction is clear. In an emergency, positive deviants relied on their self-confidence fueled by motivation, sense of professional credibility and competence. In each story, positive deviants described an emergency that became a critical juncture leading them to initiate MVA. With MVA equipment at hand and the right combination of inter- or intra-professional support, those midwives were able to push past institutional and cultural norms regarding their role in post abortion emergencies. Faced with a clinical emergency, this combination of internal and external factors propelled them to courageously apply the MVA technique.Positive deviants further leveraged their confidence as MVA practitioners and as teachers in order to inspire their colleagues to trust in their expertise and to learn the MVA technique from them. Examples included convincing colleagues to abandon dilatation and curettage, a riskier yet commonly practiced skill, and teaching the MVA technique to physicians and residents. This strategy of positioning themselves as mentors in an emergency allowed them to momentarily transcend the traditional patriarchal structures of their institution and health system. This non-threatening approach was acceptable to their physician colleagues in the moment. Following this event, midwives evolved into a mentor and in some cases an expert in their institution.p = 0.015) [Our findings are similar to a study of integrating MVA for post abortion care in Ghana. Post training, investigators reported a similar prevalence (31% = 0.015) . In that= 0.015) , 50.Our qualitative results mirror a systematic review regarding task shifting in midwifery, where sharing a skill across traditional professional boundaries challenged hierarchical structures , 50. AboWhile our research found that physician support and trust were conducive to midwifery practice across both positive deviant and comparator groups, our research adds an important insight to explain how midwives navigated challenging professional boundaries and in doing so, increased the trust and corroboration of their medical peers. Similar to some previous studies, confidence is an important starting point , 53. HowOur research provides further insight into how midwives trained in MVA challenged and overcame interprofessional barriers by strategically positioning themselves in a clinically urgent situation, making a calculated risk to save a life. Positive deviants presented themselves as intermediaries or knowledge brokers, promoting the practice of MVA to their peers, a still uncommon practice in their context, while subtly demonstrating the role of the midwife in a post abortion emergency. Figure Three main strengths of this study include: 1) the mixed method case study design that allowed for an in-depth analysis of the “who” and “how” Congolese midwives integrate MVA into practice; 2) the positive deviant approach that allowed for identifying pragmatic solutions that overcome well documented professional barriers to MVA integration; and 3) the collaborative approach that provided SCOSAF with vital information to leverage midwifery in the context of abortion care in their country.The main limitation of the study was due to the coronavirus pandemic. Travel and ethical restrictions meant a required adaptation to the final step of study. Rather than employing a focus group discussion for validation, results were electronically shared with stakeholders, with dissemination plans occurring at a later date. Another limitation occurred during the data collection period. Frequent deployment of researchers  to teach in rural areas caused delays in recruitment for the semi-structured interviews for comparator midwives. During this time lapse, changes in participants’ contact numbers resulted in some comparators being no longer reachable. Contactable comparators were easily recruited for the qualitative phase.This is the first study to independently examine midwifery integration of MVA in the DRC after training provided by the midwifery association. There is an important link between SCOSAF, the results of our study and the development of the midwifery workforce as it pertains to abortion. Midwifery associations act as a focal point for the profession, ensuring quality midwifery themed training and providing educational support. SCOSAF’s training permitted midwives to understand their professional scope and to develop an openness to post abortion care. Future research is needed to explore the link between the impact of midwifery-led training and supports, professional identity and MVA integration. Further, SCOSAF could collaborate with positive deviants to continue to diffuse midwifery-led MVA and grow their capacity in-house to support midwives in abortion care. As knowledge brokers and leaders, positive deviants could properly support midwives who are interested in providing post abortion care, yet who are less confident or competent to practice. Midwifery associations also act as a voice to represent the profession at local and national levels . MidwiveAdditional file 1.Additional file 2.


Vaccinium macrocarpon) is a distinctive source of polyphenols as flavonoids and phenolic acids that has been described to display beneficial effects against urinary tract infections (UTIs), the second most common type of infections worldwide. UTIs can lead to significant morbidity, especially in healthy females due to high rates of recurrence and antibiotic resistance. Strategies and therapeutic alternatives to antibiotics for prophylaxis and treatment against UTIs are continuously being sought after. Different to cranberry, which have been widely recommended in traditional medicine for UTIs prophylaxis, probiotics have emerged as a new alternative to the use of antibiotics against these infections and are the subject of new research in this area. Besides uropathogenic Escherichia coli (UPEC), the most common bacteria causing uncomplicated UTIs, other etiological agents, such as Klebsiellapneumoniae or Gram-positive bacteria of Enterococcus and Staphylococcus genera, seem to be more widespread than previously appreciated. Considerable current effort is also devoted to the still-unraveled mechanisms that are behind the UTI-protective effects of cranberry, probiotics and their new combined formulations. All these current topics in the understanding of the protective effects of cranberry against UTIs are reviewed in this paper. Further progresses expected in the coming years in these fields are also discussed.Cranberry ( Urinary tract infections (UTIs) are the second most common type of bacterial infections worldwide, behind otitis media. UTIs incidence peaks in individuals are in their early 20s and after age 85 . The lifAccording to the anatomical location of the bacteria, UTIs are categorized as pyelonephritis and kidney infection when they affect the upper part of the urinary tract (ureters and kidney parenchyma), and as cystitis and urethritis when they affect the bladder or urethra (lower infection tract). Clinically, uncomplicated UTIs affect healthy individuals without urinary tract anomalies, while UTIs associated with factors that compromise the urinary tract or host defense are referred as complicated UTIs, including urinary obstruction or retention caused by neurological disease, renal failure or transplantation, pregnancy and the presence of foreign bodies such as kidney stones, indwelling catheters or other drainage devices . Among tEscherichia coli (UPEC) is the primary etiological agent for the majority of these infections, causing around 85% of cystitis, but other Gram-negative bacteria, such as Klebsiella pneumoniae, and some Gram-positive cocci, such as some staphylococcal and enterococcal species, seem to be also implicated in the etiopathogeny of the remaining infections [Uropathogenic fections . Polymicfections .Conventionally, the use of antibiotics to treat this pathology has claimed to be very operative, but they can cause prevalence of resistance among uropathogens and other adverse side effects, such as damage in intestinal microbiota. For these reasons, there is a growing interest in the search of natural therapies for UTI prevention and treatment to face increasing bacterial resistance to antibiotics and high recurrence rates ,11. To pVaccinium macrocarpon) has been extensively recommended for UTIs prophylaxis and relief of adverse symptoms, the UTIs preventive activity of cranberry has been debated in the literature and numerous clinical studies have been carried out, including some recent meta-analyses [Although the consumption of cranberries  can access and grow within the bladder cells to form intracellular bacterial communities, which are detected in exfoliated urothelial cells and associated to suffer recurrent UTIs [A key step in the pathogenesis of infectious diseases is the pathogenic bacteria adherence to host cells. According to Beerepoot et al. , UTIs artruction . Once inent UTIs . Moreoveent UTIs .On the other hand, it is worth mentioning that the uropathogens can form biofilms to adhere more easily and increase its resistance. This has an effect on the rest of microorganisms causing greater adherence and multiplication, being able to form thick biofilms that significantly decrease the effectiveness of antimicrobial treatments. When the biofilm resources are limited, mature bacteria are detached and can colonize a new surface to repeat the cycle .E. coli [The uninfected urinary tract has long been assumed to be sterile in healthy individuals and it was thought that the major uropathogenic organisms only included Gram-negative bacteria, mainly E. coli ,39. CurrE. coli is a rod Gram-negative bacterium that colonizes the gastrointestinal tract of humans and animals after birth, forming part of the natural bowel microbiota. In addition, it can be an opportunistic pathogen that uses the intracellular environment to survive and protect itself against the action of antibiotics. Several epidemiological, serological, and bacteriological studies revealed that UPEC is the pathogen most frequently associated with UTIs [ith UTIs , accountith UTIs ,42. ThesE. coli in the sites where they usually do not live as the uroepitelum, and the invasion of host cells and tissues to trigger the infection later. Although the biogenesis of bacterial adhesins is a tightly controlled process, UPEC can alter binding features in reaction to environmental changes, including temperature, osmolarity, pH, and nutrient availability [E. coli to urogenital epithelial cells was seen compared to healthy controls [UPEC’s ability to bind the host uroepithelial receptors is facilitated by virulence factors, hair-like organelles called fimbriae or adhesins that form part of its bacterial wall. These structures, pili or fimbriae, allow for the attachment of lability . In recucontrols .E. coli adhesins have been described, such as the fimbriae type 1 and type S, both sensitive to the α-d-Mannose receptor, and the fimbriae type P and Afa, resistant to the α-d-Mannose receptor [d-mannose, present in most of the cells of the uroepitelium. In 99% of uropathogenic E. coli strains, genes to encode type 1 fimbriae are present [Different types of receptor . The mos present  and they present . As a re present .E. coli commensal strain into an uropathogen, and, often, the inhibition of a single adhesin may cost enough to a bacterium to lose its virulence [E. coli lineages are more likely to cause UTIs exhibiting an antibiotic resistance phenotype and seem to be more persistent in the rectal tract and pandemic. In this way, the ST131 group of extraintestinal E coli strains exhibiting multidrug resistance to beta-lactams and fluoroquinolones and have been identified globally [Other factors of virulence to highlight are the production of toxins, including α-hemolysin (HlyA), cytotoxic necrotizing factor1 (CNF1), secreted autotransporter toxin (SAT), cytolysin A, plasmid-encoded toxin (PET), vacuolating autotransporter toxin (VAT), Shigella enterotoxin-1 (ShET-1) and arginine succinyltransferase (AST) , and iroirulence . Moreoveirulence . In receirulence  reviewedglobally .E. coli, other Gram-negative bacteria, such as K. pneumoniae, and/or Gram-positive bacteria, including S. saprophyticus, Enterococcus faecalis, and group B Streptococcus, are also implied in UTI pathologic outcomes, especially in polymicrobial infections, but they are systematically overlooked [E. coli and E. faecalis. Conversely, strains from Corynebacterium, Lactobacillus and Ureaplasma genera were rarely isolated from the urinary tract as these routine culture techniques are not accurate enough to identify anaerobic slow-growing organisms. However, using advanced detection technologies, these bacteria have been identified as members of the urinary microbiota in multiple studies [Besides the most causative agent of UTIs, erlooked . Traditi studies .E. coli predominates, followed by other Gram-negative bacilli, Klebsiella spp. and Proteus spp., and Gram-positive cocci, such as S. saprophyticus, Enterococcus spp. and Streptococcus agalactiae. However, in the elderly infections, Enterococcus spp. are the most common agent (11.6%) compared to the rest of the population, which only represent 5.3% of cases [It should be remarked that UTI etiologic agents vary according to age, sex, and underlying pathology. Thus, in ambulatory patients, of cases .The increase in global antibiotic resistance and recurrence rates has prompted the exploration and assessment of new therapeutic strategies by antiadhesive components. As a natural alternative, the intake of cranberries as fresh or dried fruits, as well as products derived from them  has been extensively recommended. Cranberry consumption has been indicated to be effective in decreasing the occurrence and severity of UTIs in women and to prevent the adherence of pathogenic bacteria in the urinary tract ,50,51. MAlthough numerous epidemiological and intervention studies have proved the efficacy of cranberry products in UTI prophylaxis ,17, otheFor the most recent decades, flavonoids as proanthocyanidins (PACs), mainly A-type , were asIn addition, complex carbohydrates and sugars, terpenes, as well as organic acids such as quinic, malic, shikimic, and citric are other preponderant cranberry phytochemicals. Among them, D-mannose has also been reported to inhibit the adherence of UPEC to uroepithelial cells in vitro, although new studies will certainly be needed to confirm it ,60. Vitaex vivo studies have confirmed the antiadhesive activity of urine samples collected from volunteers who consumed cranberry products in comparison to urine samples collected from the placebo group. As example of this, a recent study demonstrates the strong ability of human urine after intake of a cranberry chew compared to a placebo chew to inhibit the ex vivo adherence of both P type and type 1 uropathogenic E. coli in a randomized, double-blind, placebo-controlled, crossover design pilot trial [Candida albicans strain by urine collected after the intake of a cranberry extract and a lecithin formulation with improved oral bioavailability. The mechanisms implied in the preventive effects of cranberry consumption against UTIs are not completely established and several leading hypotheses have been proposed. Thus, cranberry polyphenols, in particular, their microbial-derived metabolites, are claimed to operate in the phase of bacterial adherence to the uroepithelial cells, disabling or inhibiting the adherence of UPEC and, therefore, preventing bacterial colonization and the progression of UTIs ,63. Thisot trial . In the ot trial  reportedCell culture methodologies have also applied to assess the antiadhesive activity of cranberry phenolic compounds/fractions/extracts against uropathogens, specifically in bladder epithelial cell lines ,26,28,64E. coli ATCC® 53503™ into bladder uroepithelial cells  [O-sulfate are able to inhibit the adherence of the uropathogenic bacteria tested at concentrations that may be more physiologically relevant. Additive (and even synergistic) effects among all the cranberry-derived phenolic metabolites present in urine are expected in vivo, which could explain, at least partly, the ex vivo antiadhesive activity of urine samples collected from volunteers who consumed cranberry products [™ cells) ,26,28. A™ cells) ,63, makiproducts .Another interesting finding about the ex vivo capacity to inhibit UPEC adherence after cranberry extract is that of  that fouEmerging evidence shows that the gut microbiota has a key role in homeostasis, regulating health and disease at distal sites throughout the body . Althougn = 10) after 2 weeks of consuming dried cranberries (42 g/day) [Akkermansia in most subjects. Similarity, in a human trial (n = 11) evaluating the effect of a cranberry powder (30 g/day) in the context of an animal-based diet, [Based on this wide influence of gut microbiota in the body, other hypothesis about the mechanisms of action of cranberry flavonoids and phenolic acids against UTIs is that cranberry components  would interact with gut microbiota, modulating its composition and/or functionality in such a way that it prevents microbial dysbiosis. In accordance with this hypothesis, a pilot study of the human faecal microbiome (2 g/day)  indicateed diet,  found thE. coli (ExPEC) and decrease its (transient) intestinal colonization, consequently reducing the risk of UTI incidence [More specifically in relation to UTIs, and as it is becoming evident that the intestine is a reservoir for uropathogenic bacteria, in vitro studies have indicated that cranberry flavonoids and phenolic acids might interact with extra-intestinal ncidence . This poTherefore, the effects of cranberry polyphenols as flavonoids and phenolic acids against UTIs would be influenced by what has been called a “two-way interaction” between polyphenols and gut microbiota  in the sIn other words, the beneficial effects of cranberry against UTIs would be modulated by our intestinal microbiota through its different capacity in metabolizing cranberry flavonoids and phenolic acids into bioactive metabolites. This is the case, for example, in phenyl-γ-valerolactones, one of the most abundant metabolites in the bioactive urine fraction after cranberry intake ,59 and wLactobacillus have been found in vaginal exudates and faecal samples [Lactobacillus translocation from the intestinal mucosa to distal mucosal surfaces has also been reported [The vaginal microbiota, that has predominance of lactobacilli, plays a dynamic and often critical role in UTI pathology by maintaining a low pH and avoiding uropathogen colonization by competitive exclusion. However, hormonal change due to estrogen deficiency, antimicrobial therapy, contraceptives, incomplete healing and recurrence of UTIs lead to a shift in the local microbiota (dysbiosis), increasing the risk of pathogens triggering pathologies  ,76. Ther samples ,78; Lactreported .Lactobacillus strains has been proven in many in vitro studies [Lactobacillus salivarius, Lactobacillus plantarum and Lactobacillus acidophilus on the adherence of various uropathogenics strains from E. coli, Staphylococcus epidermidis and E. faecalis to bladder cells T24. However, the mechanisms behind these inhibitory effects have thus far not been elucidated, with most studies relying on inferred evidence [Lactobacillus spp. seems to inhibit UPEC growth and adversely affects its fimbrial structure and adherence as well as upregulate immunogenic membrane proteins [L. rhamnosus probiotic in the degradation of flavonoids which are present in cranberry pomace fractions, and the effect of the yielded cranberry metabolites on hepatocellular carcinoma HepG2 cells in vitro have been investigated by Rupasinghe et al [The use of probiotics/lactobacilli against UTIs has been assessed . The inh studies ,82,83. A studies  reportedevidence ,84,85. Tproteins ,87. Morehe et al  and theyLactobacillus strains’ consumption to prevent or treat UTIs [Several human intervention studies have evaluated the effectiveness of specific eat UTIs ,35,89. Seat UTIs ,35.Lactobacillus probiotic preparations have recently been lunched onto the market, being available to the general public without prescription. Many of these products do not have scientific support, although some human studies on the combination of cranberry–probiotics have been reported. In this way, Montorsi et al. [Lactobacillus rhamnosus SGL06, cranberry and vitamin C for the prevention of recurrent UTIs. Polewski et al. [E. coli (ExPEC) to intestinal cells. The exploration and incorporation of advanced probiotic formulations and microbiome-targeted treatment strategies within urology is warranted [New products claiming protection against UTIs and combining cranberry extracts and i et al.  have proi et al.  have alsarranted .E. coli, mainly outer membrane proteins are potential immunogens. Thus, bacterial lysates could be potential candidates to create a polyvalent protective vaccine against recurrent E. coli-associated UTIs. Alternatively, the development of a synthetic urinary microbiota for transplantation might lead to an effective treatment for patients with recurrent UTIs, but still all these new approaches are under development.Lastly, new therapies based on vaccination and small-molecule inhibitors targeting uropathogens virulence factors, are being investigated as future treatments against UTIs . In thisin vivo may be impractical and ineffective until more is understood about the various mechanisms by which different cranberry components work together to influence overall health.It is worthy of note that cranberries, like some traditional medicines, are a well-known functional food, whose efficacy has been assayed in clinical trials. However, the compounds directly responsible for many of cranberries’ reported health benefits remain unidentified. They are complex mixtures whose constituents have a synergistic effect by acting at different levels on multiple targets and pathways or reducing toxicity and drug side effects . NowadayVaccinium macrocarpon) has been extensively recommended for UTI prophylaxis and the relief of adverse symptoms. In vivo evidence has been translated to in vitro verification, resulting in the widely recommended use of cranberry juice in traditional medicine for urinary tract pathologies. This review summarizes some relevant considerations about cranberry polyphenols as flavonoids and phenolic acids, and UTIs derived from the last scientific publications. One of our first conclusions is that, besides uropathogenic Escherichia coli (UPEC), other etiological agents such as Klebsiella pneumoniae or the Gram-positive bacteria of Enterococcus and Staphylococcus genera seem to be widely involved in UTIs. In particular, complicated UTIs caused by UPEC and Enterococcus spp. represent a major health care concern nowadays. The application of advanced techniques for UTI diagnosis comprising PCR, microarray and next-generation sequencing, together with the study of the individual urological microbiome, may improve diagnosis and treatment of these infections. The other outcome of this review is that the existing clinical trials support substantial evidence about the use of cranberry products as total or partial therapeutic alternatives to antibiotics in UTIs, although it has also been seen that cranberry effectiveness is individual- and/or case-dependent. This variability among individuals and cases has been attributed to different manufactured cranberry-based products and doses, as well as to the lack of systematic protocols for the selection of subjects and clinical assays. However, as more is known about the metabolism of cranberry polyphenols, new factors such as host microbiota, endogenous metabolism, host genetics and the immune system may also play a role in cranberry effectiveness against uropathogens. A question that remains to be fully answered is which cranberry and/or cranberry-derived components are mainly responsible for its protective effects against UTIs. Although recent studies have shed light on the antiadherence activity derived from cranberry consumption, other mechanisms might be jointly implicated that need further investigation. Knowledge of these mechanisms (and the cranberry components involved) might result in a better management of the potential of cranberry products against infections. For instance, nutraceuticals and other forms containing cranberry bioactive compounds and ensuring optimum bioavailability and effectiveness could be designed. Finally, pieces of evidence regarding treatment against UTIs with probiotics and their combinations with cranberry are too limited to draw any conclusions but they seem to indeed be a promising strategy for future treatments. However, any development must be supported by exhaustive research specific to the strain/s and cranberry products, as well as about the mechanisms involved in these potential beneficial effects of probiotics and their combinations with cranberry.The consumption of cranberry (