Background: The development of reliable, high quality health-related guidelines depends on explicit and transparent processes, methods aimed at minimising risks of bias and the inclusion of all relevant expertise and perspectives. While the methodological aspects of guidelines have been a focus to improve their quality, less is known about the social processes involved, for example, how guideline group members interact and communicate with one another, and how the evidence is considered in informing recommendations. With this in in mind, we aimed to empirically examine the perspectives and experiences of the key participants involved in developing public health guidelines for the Australian National Health and Medical Research Council (NHMRC). Design: This study was conducted using constructivist grounded theory as described by Charmaz, which informed our sampling, data collection, coding and analysis of interviews with key participants involved in developing public health guidelines. Setting: Australian public health guidelines commissioned by the NHMRC. Participants: Twenty experts that were involved in Australian NHMRC public health guideline development, including working committee members with content topic expertise (n=16) and members of evidence review groups responsible for evaluating the evidence (n=4).Results: Public health guideline development in Australia is a divided process. The division is driven by 3 related factors: the divergent disciplinary background and expertise that each group brings to the process; the methodological limitations of the framework, inherited from clinical medicine, that is used to assess the evidence; and barriers to communication between content experts and evidence reviewers around respective roles and methodological limitations.Conclusion: Our findings suggest several improvements for a more functional and unified guideline development process: greater education of the working committee on the methodological process employed to evaluate evidence, improved communication on the role of the evidence review groups and better facilitation of the process so that the evidence review groups feel their contribution is valued. For example, the WHO Handbook for Guideline Development , 2nd Edition requires that the process for developing a recommendation is explicit, transparent, and uses methods aimed to minimise risk of bias; the guideline development group includes all relevant expertise and perspectives, and that recommendations consider benefits and harms as well as other relevant factors.3 Currently in Australia, there are standards for guidelines approved by the National Health and Medical Research Council (NHMRC) and guidance to achieve these standards is being developed.7Due to the large number of organisations and governments that produce guidelines, end-users are often presented with contradictory recommendations and guidelines of varying quality.3 GRADE allows for a transparent rating of the quality of evidence and rates the confidence in effect estimates for benefits and harms as high, moderate, low, or very low.8 GRADE has been optimised for evaluating clinical interventions and randomised controlled trials.GRADE  is widely endorsed as a methodology for clinical guideline development worldwide, including by the WHO and NHMRC.9 For example, NHMRC guidelines assess the health harms of living near a windfarm10 or provide dietary advice for Australians.11 Unlike clinical practice guidelines, the available evidence for the development of public health guidelines is seldom from randomised controlled trials but instead is often derived from observational studies such as cohort studies, case controls, or time series analyses. Further, GRADE has not been developed to account for all important considerations related to public health guideline development, for example it does not provide explicit guidance for when evidence is linked across a causal pathway.12 Depending on the risk of bias tool used, GRADE can also downgrade non-randomised controlled trial evidence, even if this is the only and most appropriate type of evidence available. The use of GRADE for developing public health guidelines or conducting systematic reviews in the field of public health has been previously studied.13 While it has been recognised as a systematic and transparent process of evaluating the evidence, challenges have been identified in its use due to the complex nature of public health exposures.Public health guidelines offer recommendations to prevent ill health or to improve the health of a population, which are tailored to a specific audience .14 In clinical practice guideline development, previous experiences and beliefs that are not consistent with the research evidence are prioritised when developing recommendations.15 Further, the status and, therefore, power of the guideline development groups have been shown to influence the level of input made by individuals in multidisciplinary guideline panels.16Although the most rigorous and trustworthy guidelines are evidence-based, evidence may not be the greatest influence in the formation of guidelines.17 Although the diversity of opinions in such groups brought tensions, diversity was vital in making informed judgements, relevant to making recommendations.18 Tensions have also been experienced between guideline development groups and those conducting the evidence synthesis.19Several studies have examined how participants involved in public health guidelines translate evidence into recommendations. Guideline development group members may conceptualise the guideline development task differently, with some prioritising the evidence in informing their decision-making, while others, their disciplinary expertise.5 less is known about the guideline development groups social processes. We conducted a grounded theory study to understand the perspectives and experiences of those involved in the development of public health guidelines. Consistent with a grounded theory approach, we define social processes as participants’ actions, interactions, and decision-making within the context of a guideline development process with a specified beginning and endpoint.20 We aimed to understand the perspectives and experiences of the 2 key groups of participants involved in developing public health guidelines for the NHMRC: the evidence review groups and the working committees . The interview guide was designed to evoke the participants’ opinions and experiences in being involved in the guideline development process and to remain open to what participants’ deemed salient to the process of guideline development . Thirteen people did not respond to emails and 3 could not commit to a time. We had a lower response rate from individuals in evidence review groups. We interviewed 16 working committee members, and 4 evidence review group members. By interview 16, similar narratives were shared. We recruited 4 additional participants representing diverse perspectives to ensure exploration of the range of experiences and determined saturation at interview 20.The experiences of the guideline development process were markedly different for the 2 key groups of participants involved in the process, the evidence review groups and the working committees. These experiences suggest that public health guideline development in Australia is a divided process. The division is driven by 3 related factors: the divergent disciplinary background and expertise that each group brings to the process; the methodological limitations of the framework, inherited from clinical medicine, that is used to assess the evidence; and barriers to communication between content experts and evidence reviewers around respective roles and methodological limitations.Participants on the working committees had divergent disciplinary backgrounds  compared to those on evidence review groups  and held varying, sometimes conflicting beliefs about what constituted ‘good’ evidence. Many experts on the working committee viewed their primary role as protecting the public’s health through reducing possible harms to hazardous exposures or recommending interventions that would improve health outcomes. These individuals often viewed ‘good’ or ‘important’ evidence from the perspective of their own knowledge and expertise and not from the standards of methodological rigour used by the evidence review groups.For example, one working committee member often felt that studies with statistically significant harms or benefits should be included in the body of evidence to form recommendations, even if they had been excluded by the evidence review groups on methodological grounds.“There was a critical study which, you know, some of the older studies were omitted because they were methodologically not considered acceptable; but, they were strong results, so it was thought they couldn’t be dismissed” .The evidence review groups, however, were contracted to evaluate the evidence using explicit methodological frameworks, such as systematic reviews and GRADE, that had very clear criteria for how the evidence was to be evaluated. The evidence review groups were aware of these disciplinary backgrounds and beliefs that the working committees brought to the guideline development and recognised the importance of having these experts involved in the process. However, they were also aware of the challenges that this presented because the evidence known and used by experts often did not meet the necessary criteria for inclusion.“…So you want experts, sure and of course a lot of these experts, they’re very professional and they produce really good research and of course they’re attached to that research … My experience has been a common thing when you first present the evidence review or the systematic reviews you’ve done, they get upset because it’s not what they know and it’s different – it’s looked through a different lens” .These divergent roles and epistemological beliefs led to conflict and division between the groups. Many of the working committee members believed that in order to best protect the public’s health, the evidence presented by the review groups should at times be challenged.“Yeah, it wasn’t all plain sailing, there were differences of opinion, yep, and there were differences of opinion on what should be included, because some of the people on the committee were experts in their field and had contributed to the literature, so that makes it also that little bit different” .The evidence reviews groups however, saw this process as being hostile, aggressive and at times they felt victimised for doing their job. They felt that the evidence-review role they had been contracted to complete was not respected by some working committee members.“We’re trying to do the best we can, we’re not content experts, we’re methodology experts, you know, we’re not deliberately trying to sabotage the process, we’ve got to work with people, but it always seems to come back to what feels like a very personal attack … it often is quite aggressive” .For at least some participants, the guideline development process was a divided one, with one dominant group offering content expertise and the other group attempting to provide methodological expertise.It was widely considered by all participants that the evidence reviews, and guideline development processes should be rigorous and transparent and that this would enhance the credibility of the guidelines.“I think that’s NHMRCs main goal of this whole process is that it would be as transparent and reproducible as possible that every decision is documented, and process driven as much as possible but there’s a framework for each step, and I think that’s working reasonably well” .Although working committee members were generally supportive of the processes they used, they acknowledged methodological limitations. For example, several working committee members recognised that the methods used in evaluating the evidence for their public health guideline topic were designed for clinical medicine and evaluation of randomised controlled trials.“The problem is that NHMRC holds you to their standards of evidence, which are designed for other forms of evidence. They’re designed mainly in the medical domain and drug domain. So, to apply them to something like (topic) is ridiculous” .They described how relevant and important evidence was consistently being downgraded, leading to a body of evidence used that appeared low quality.“So, because it was this public health type evidence, what it meant was that none of the gradings were very high. And that we thought that – and I think many people have had the same views – that it was really not appropriate. And that the randomised clinical trial approach to evidence obviously a gold standard and so on, but that it was important not to throw out all the other things where – where randomised clinical trials were never going to be possible” .When following GRADE guidance, little or no evidence leads to recommendations that are rated as “weak.” Several working committee participants and chairs were concerned that weak recommendations would not be understood by policy-makers in determining appropriate action to protect the public’s health or would be misrepresented by industries that may wish to discredit their findings.“But we had to, and we argued a lot about how to word this exactly, because if you say there’s no evidence it could mean that there just isn’t enough research to know or there is evidence that it doesn’t cause that. So we had to be very careful with the wording…So I remember we argued, and argued and argued about the wording of that to get it to a situation that we were all happy with” .However, the evidence reviews groups often felt that they, and not the methodological frameworks, were blamed for the way the evidence was evaluated.I mean, that’s essentially what we did but it’s a very uncomfortable position to be in. I feel like they like the idea of it. But in practise when you give the results their sort of like, they’re shocked, and I mean, the methods can only do so much and they’re not flawless, there’s limitations they just don’t often expect what they get at the end ” .“Despite such criticism, the evidence review groups’ members who had intimate knowledge of the methodological process and were aware of how the evidence would be presented, understood why the working committee were frustrated, and were even empathetic to these issues as they knew what the limitations of the framework were.I don’t mean to be critical of the working group or the NHMRC and I think this was very challenging right from the beginning because it’s public health intervention and a good example of using the GRADE process. It’s very difficult because they’re all observational studies…and there were quite a lot of issues ” .“The evidence review groups contracted by the NHMRC worked independently from the working committee for most of the process. Evidence Review group members were unable to share with the working committee insights and opinions on the best approach to identifying or evaluating the evidence. Further, the 2 groups were unable to help one another to understand each other’s point of view. The transfer of ideas, knowledge and expertise was limited by the separation of the 2 groups.“No, no, that’s actually a challenge too, because they decided to outsource them to a body that has expertise in doing systematic reviews, but not topic specific expertise” .Not including content experts in the evidence review process provided the opportunity for working committees to be critical of the evaluations of the evidence review groups. For example, the content experts felt that the evidence review experts lacked the necessary knowledge on a guideline topic to identify all the relevant literature necessary to inform the guideline.While conversely, the complexity of the methodology meant that unless working committee members had a methodological background, they found it difficult to understand and follow the evidence evaluations, when they were presented by the evidence review groups.“So, they would, we’d have these 2-day meetings and the people who run the tender to do the systematic review would kind of explain the methodology and, I’m not a methodologist so a lot of it went past me, about what you should include and what kind of grade recommendations could be supported, by what kind of evidence and so forth” .This highlights how by not working with the evidence review groups regularly from the outset and understanding the methods thoroughly, some working committee members felt limited in their ability to contribute to the guideline process due to their lack of methodological knowledge and training.This separation between the groups was also seen by the evidence review groups as a major limitation in how the guideline process was conducted. The irregularity of the meetings at which the evidence reviews groups presented their findings to the working committees meant it became an ineffective way of communicating with the working committee on how the evidence was being evaluated with the methodology employed.“Here it would include the GRADE process, and everyone goes yeah, okay, we understand that, that’s good, but when it comes to the presentation it’s usually so long after they’ve forgotten – even for us it’s challenging” .As a result of this, when evidence review groups did advise the working committee on what evidence should or should not be included, they were criticised for their suggestions.“I don’t know. What we were told is don’t tell us what to do, which shook us quite a bit because we were like well, we’re just giving advice. Like, we don’t mind if you don’t take it but this is a wee bit challenging. So, then you don’t know what your role is ” .The evidence reviews groups felt that this criticism from the working committees grew from the separation of duties, and failure to have effective communication strategies in place. The evidence review groups members felt that many of the tensions that were experienced between groups of participants could have been limited if working committees were provided with clear information about the different roles of the 2 groups.“But certainly for these 2 NHMRC ones it felt combative and I don’t – and it’s been an unpleasant process for us and we’ve felt that either the Chair should stand up and just – it’s just little things, like just saying, you know, if someone’s attacking the work, just stop in and say, look, these guys have done and spent a lot of time and a lot of work on this so let’s just calm down and let them talk through the methodology of how they’ve done it” .Throughout the process evidence review groups felt a lack of support when delivering their work that was at times confronting and difficult for some working committee members to accept. Evidence review groups recognised an unequal and unfair power dynamic between the 2 groups. This led to the evidence review group members feeling that they were not valued or respected contributors to the guideline process.“But it’s my experience in working with these advisory committees, particularly with the NHMRC that it’s the committee that makes the decision and the evaluation group is very much, you know, in a responsive position and pretty much on the back foot” .While the roles and responsibilities of these 2 groups in the guideline process may not be intended to be equal, it is highlighted here that the evidence review groups felt that they played a passive role. Being in a ‘responsive position’ demonstrates how this division is perceived by the evidence review groups as a process that is dominated by a working committee that do not fully value their contribution to the guideline process.19 Our study however, extends this prior research by not only understanding the experiences of the working committees, but also giving a voice to members of the evidence reviews groups to understand the experience from their perspective, and the social processes involved in public health guideline development.The methods experts and content area experts interviewed for our study agree that rigorous methods should be used to develop public health guidelines that are considered valid and trustworthy. Our findings suggest that more attention needs to be given to the social processes influencing guideline development in order for the experts to achieve this shared goal. The division that is present in public health guideline development in Australia is driven by the divergent disciplines the 2 key groups of participants bring to the process, the methodological limitations of the framework that is used to assess the body of evidence, and the inadequate integration and clarity of the respective roles of the evidence review and working committees. These divisions were emphasised by the lack of interaction between the groups. These themes are echoed in the literature exploring the experiences of different guideline working committees using similar methodological approaches.This is the first comprehensive empirical investigation, to the best of our knowledge, into the process of public health guideline development in Australia. This study included several guideline topics and participants from diverse backgrounds thus allowing us to analyse the experience of different groups of participants involved in the process.This work reflects the opinions and experiences of the participants involved in the development of a sample of guidelines, and therefore it is possible that the experiences represented here may be different to those who did not participate. However, we sought to minimise this bias by including a diverse range of guideline topics. Four fifths of the respondents were working committee members, while only one fifth from the evidence review groups. While we attempted to contact more members of the evidence review groups, their response rate was lower and evidence review groups tend to have about half the number of members as working committees. The reason for this low response rate may be due to the evidence review groups being contracted by the NHMRC to conduct the evidence reviews and they therefore may feel conflicted in contributing as they are paid by the developer. Alternatively, they may have felt uncomfortable with sharing their thoughts and insights on the guidelines process as these experiences revealed in this study were often challenging. Future research should aim to understand these experiences further.24We feel, however, the concepts we represent here were expressed by both groups as we continued our sampling and analysis until we reached thematic saturation.17 A previous qualitative study that explored the social processes of how evidence is understood and used by guideline advisory groups found that different group members prioritised the ‘scientific’ evidence, such as randomised controlled trials in informing their decision-making, while others their professional experiences.17 Clinical and practical experiences have also been shown to take precedence over the evidence in forming recommendations when using the GRADE process in the developing WHO guidelines that included public health topics.18 Our study expands on these studies by showing that the content and evidence review experts have differences in valuing randomised, clinical trials vs. clinical expertise and in how they value observational design studies needed for public health guideline development.The prioritisation of disciplinary expertise that working committee members may have over the methodological expertise typical of the evidence review groups, is consistent with previous studies that have examined how evidence is conceptualised and used in forming recommendations in public health guidelines.13 The difficulties identified included, which studies to include or exclude, the inability to upgrade the quality of evidence from observational studies higher and concerns that policy-makers may potentially misinterpret no or low quality evidence when determining what course of action to take. Further, the limited understanding of the GRADE process used to evaluate the evidence by the working committee discussed in our present study, was also demonstrated with previous investigations into the guideline development process by WHO guideline groups.19The specific methodological challenges involved with evaluating the evidence used in public health guideline development described by the participants in this study have previously been identified in a study that explored the experiences of groups that have applied GRADE for developing guidelines or systematic reviews in the field of public health.19 Although methodologists were also part of the working committee in our current study, the evidence review groups were responsible for conducting and presenting the results of the initial evaluation and grading of the evidence, which makes their experiences similar to this previous investigation. Therefore, the experiences of feeling tension with the working committee members, the need for their role to be clearly articulated and the need to receive greater support from the NHMRC throughout the process, are relevant and consistent with these previous findings.19The tensions felt by the evidence review groups with the working committee members in this present study has also been shown in a previous study that explored the experiences of methodologists working on WHO guidelines with discordant recommendations.12 a number of steps could be put into place to help optimise the public health guideline development process in Australia and globally. Improvements in the social processes of guideline development could reduce the tensions and division that have been identified between the 2 key groups of participants involved in this study. Firstly, more pragmatic advice and training by the NHMRC for the working committee members unfamiliar with the frameworks used to assess the body of evidence is necessary, not only at the commencement of the process but should be ongoing thereafter. Both the working committee and evidence review groups viewed understanding the methods as a significant challenge to the current process. Inadequate understanding of the methods restricts the level of input certain working committee members can have in the process and creates tensions between the groups.19While there are methodological challenges and considerations that go beyond the scope of this paper,19 The evidence review groups also need to feel appropriately supported in their roles when presenting the evidence reviews, particularly by the working groups chairs. Therefore, the power imbalances created between the 2 groups must be minimised through strong facilitation, which would allow for the evidence review groups to feel their contribution is respected and valued and their opinions heard throughout the process. These power imbalances could also potentially be minimised by paying attention to differences in age, experience, gender, or region between the evidence review groups and working committees.25Secondly, the role of the evidence review groups needs to be clearly articulated from the start and reinforced throughout guideline development process.26 By integrating the groups, with subgroups to evaluate the evidence for particular questions, the tensions identified from this lack of contact and communication will be significantly reduced. Furthermore, methodologists who are members of the working committees could facilitate discussions of the reviews presented by the evidence review groups. Their understanding of the challenges associated with evidence review for public health questions could bridge a gap in communication between the working committees and evidence review groups.Finally, there is a need to work collaboratively from the outset and throughout the duration of the guideline process, to make it more collegial, effective and efficient. To enhance the transfer of ideas, knowledge and expertise, the physical separation that is currently present between the 2 groups should be reduced. While there may be benefits to keeping expert opinion influence away from the evidence review process, the infrequent meetings and lack of communication between the groups appears to be a significant factor in tensions that are made apparent when the evidence review groups present their findings to the working committees.Through greater education of the working committee on the methodological process employed to evaluate evidence, improved communication on the role the evidence review groups play, along with better facilitation of the process so that the evidence review groups feel their contribution is respected and valued, an enhanced transfer of ideas, knowledge and expertise in the guideline development process will be possible.We thank the qualitative work group of the Evidence Policy and Influence Collaborative at the University of Sydney, Sydney, NSW, Australia for their involvement in discussions around the emerging concepts from the data. We also thank the staff of the Australian NHMRC for assistance with recruiting participants.This project was approved by The University of Sydney Human Research Ethics Committee (Project number: 2017/220).Authors declare that they have no competing interests.NC and LAB initiated the study. NC designed the study, conducted the interviews and prepared the original drafts of the manuscript. QG and LAB assisted with the design. QG, LAB, and LMP assisted with the data analysis. QG and LMP trained and supported NC in the data collection and analysis methods. All authors made significant contributions to the final manuscript. NC and LAB are guarantors.This work was supported by NHMRC project grant APP1139997. Nicholas Chartres is a recipient of the James Millner PhD Scholarship in Pharmacy from the University of Sydney, Sydney, NSW, Australia. Quinn Grundy was supported by a Postdoctoral Fellowship from the Canadian Institute of Health Research.1Charles Perkins Centre, Faculty of Medicine and Health, School of Pharmacy, The University of Sydney, Sydney, NSW, Australia. 2Faculty of Nursing, University of Toronto, Toronto, ON, Canada.Improvements in the social processes of guideline development could reduce the tensions and division that have been identified between the 2key groups of participants involved in this study – evidence evaluators and content experts.More pragmatic advice and training for the working committee members unfamiliar with the methodological frameworks used to assess thebody of evidence is necessary.The evidence review groups need to feel appropriately supported in their roles when presenting the evidence reviews, particularly by theworking group chairs. There is a need to work collaboratively from the outset and throughout the duration of the guideline process, to make it more collegial, effectiveand efficient. To enhance the transfer of ideas, knowledge and expertise, the physical separation that is currently present between the 2 groups should bereduced by integrating the groups, with subgroups to evaluate the evidence for particular questions.Public health guidelines are designed to protect the public’s health. Therefore, the methods and processes that are used to evaluate the evidence andformulate the recommendations need to be rigorous, transparent and free of bias. Further, the 2 groups that are responsible for developing theseguidelines, the evidence review groups and working committees with content expertise, must also work effectively together. However, the currentpractise of these groups working separately throughout the guideline development process leads to division and conflict. The recommendationswe have made in this study will lead to a more functional and unified guideline development process that best uses all the relevant expertise, andtherefore may contribute to better health outcomes for the public.


Health literacy assessment tools that are designed and validated specifically for East Asian older adults are limited. This session reviews the existing health literacy assessment tools and discusses possible applications in East Asian older adults. As an alternative to the population-level health literacy assessment, the general literacy measures and assessment approach in the Program for International Assessment of Adult Competencies (PIAAC) data are discussed. As a case demonstration, the PIAAC literacy measures in Japan and Korea are summarized and evaluated for the applicability in aging and health research with East Asian older adults.


To establish whether national guidelines for postpartum hemorrhage (PPH) reflect new scientific evidence on misoprostol, and determine the challenges faced in their implementation.A web‐based survey was sent by email to 130 national societies of obstetrics and gynecology (FIGO Member Associations) in 2016. The survey, composed of 18 questions, covered national guidelines on PPH with particular reference to misoprostol, the creation of national guidelines, and challenges to implementation.Completed surveys were received from 69 societies, for a 53% response rate. The key findings were that many countries lacked comprehensive, up‐to‐date, evidence‐based national guidelines providing guidance on misoprostol use; recommended regimens were very different in the national guidelines as well as between international and regional guidelines that are most often used as referencing documents; and there are a variety of challenges to implementation of guidelines.There is a need, especially in countries with high maternal mortality, to establish mechanisms that ensure the existence of up‐to‐date, comprehensive, evidence‐based guidelines on PPH. This can be difficult given conflicting guidance at the international level. Regional and international societies should prioritize clinical updates and ensure their dissemination and implementation. A survey of 69 national societies of obstetrics and gynecology showed that many lacked comprehensive, up‐to‐date, evidence‐based national guidelines on postpartum hemorrhage, including misoprostol use. More than two‐thirds of reported hemorrhage deaths were classified as postpartum hemorrhage (PPH).Owing to the global imperatives to reduce maternal mortality, and given the existing knowledge base, international recommendations specify that the following aspects must be ensured in high‐risk countries: uterotonic availability; medicines are listed on international (WHO) and national essential medicines lists (EML) in correct dosages; and international and national guidelines on PPH management are in place that support the provision of these uterotonics, and that the guidelines are fully utilized.Ensuring the existence and implementation of evidence‐based clinical guidelines is a key objective for many focus areas of global health concern. Regarding maternal health and PPH, evidence shows that it is an effective component. A recent systematic review evaluated the impact of new or updated guidelines for PPH management implemented in clinical practice. In four of seven trials in the review, the numbers of PPH cases declined after the intervention.While an array of international clinical guidelines exist on the prevention and treatment of PPH, little is known about whether these guidelines are utilized, whether they are translated into national guidelines, or used to get medicines listed on national EMLs. The aim of the present study, arising from FIGO's interest in the use of international guidelines, was to understand whether new scientific evidence is incorporated into national clinical guidelines and EMLs—important steps toward improving maternal health.2A web‐based survey was developed and sent by email to 130 FIGO Member Associations (MAs) on February 5, 2016. Consent was requested in the email and at the start of the survey. Three email reminders were sent  before the survey was closed on March 21, 2016. The survey comprised 18 questions, with both single response and multiple response questions. The broad question areas covered were: national guidelines on PPH and their content with regard to misoprostol, the creation of national guidelines, challenges to their implementation, and the inclusion of misoprostol on national EMLs. Data were imported into Excel  where basic analysis was conducted.We decided to ask questions specifically about misoprostol for three reasons. Firstly, misoprostol may be more practical in the absence of oxytocin owing to the reasons noted above, making it an important component of an integrated package of PPH interventions, especially in low‐resource and community settings. Secondly, misoprostol—a relatively new drug supported by new science—can be used as an indicator for how quickly guidelines respond to the latest evidence on PPH management. Both FIGO and WHO produced guidelines for prevention and treatment of PPH in 2012 ; therefore, misoprostol can serve as an interesting point of comparison for both international and national recommendations. Furthermore, in 2011 and 2015, misoprostol was included in the WHO EML for use in PPH prevention and management, respectively. Thirdly, since FIGO has been involved in disseminating information on misoprostol, it was felt that the survey would provide information to improve or redirect its work.3Completed surveys were received from 69 (53%) of the 130 MAs  and Ministry of Health . The most current versions of the guidelines had been published between 2006 and 2016; 20 (36%) had a year scheduled for their review (ranging from 2016 to 2020), 10 (18%) were reported as “currently underway,” and 26 (46%) had no review planned or the timescale for review was not known. Of 55 responders, 53 (95%) reported that their association of obstetrics and gynecology would be involved in the next review of the guidelines, and 45 (80%) reported that minor revisions to the guidelines were possible between reviews. No follow‐up question was asked to identify any cases where minor revisions had been made between reviews.The most common challenges to updating and implementing guidelines were: lack of supportive policy or programs for use of misoprostol; misoprostol not widely and regularly available; misoprostol not included on national EML; healthcare providers not knowing about the guidelines; lack of national guidelines/misoprostol not included in national guidelines; and misoprostol not being registered for use . Of the 69 responders, 15 22%) reported no challenges  was often included in national guidelines, whereas it is not included in FIGO and WHO recommendations owing to its pharmacokinetic profile, which may not be associated with the best efficacy and because there is limited robust evidence on this route, while there is a plethora of evidence supporting the oral and sublingual routes for prevention and treatment. Follow‐up enquires were made after the survey with other obstetricians and gynecologists to understand why this route continued to be recommended. The reasons provided were that in many university hospitals the diagnosis of PPH and, therefore, the use of misoprostol may occur in women who are already being treated at a high level  making the oral routes less favorable. In addition, for the obstetrician wearing surgical gloves these would have to be removed in order to put a tablet in the mouth, which also makes the rectal route favorable. Furthermore, the rectal route would be chosen during cesarean delivery. While these are valid reasons for using the rectal route, these circumstances are very different to what could be happening in a low‐resource setting where nonrectal routes should be used. This again implies that national guidelines may not be suitably tailored for low‐resource settings. In future, either international organizations should work together to synthesize evidence and potentially align their recommendations to ensure greater harmony with the aim of providing greater clarity for the development of national guidelines, or international bodies should work closer with national ones to look at regional needs and decide what is required in their particular setting.In this survey, the most common challenges for implementing guidelines that include use of misoprostol for PPH were that misoprostol is not widely and regularly available and there is a lack of supportive policy and programs. This supports other findings that recognize that guidelines alone are insufficient in the absence of a supportive policy environment, without broad dissemination, scalable programs and training, and ensuring the availability of such commodities throughout all health facilities. A 2008 systematic meta‐review by Francke et al.A limitation of the study was that the information received from MAs was not substantiated. The responses were based on the views and understanding of one senior person within the national society, and it may be possible that the respondent may not have had the most accurate or up‐to‐date information. The response also represented the view of one expert provider in the country and was not based on information from multiple providers. Additionally, what was reported may not reflect actual provider practices, which was beyond the scope of the survey.Another limitation was that the survey was conducted in English, and while Spanish and French translations were provided upon request, this may have impacted on responses from non‐English speaking MAs. The survey also only focused on written national guidelines and, given wide access to the internet, it is unclear how many providers actually know about or rely on national documents versus information they obtain online using a range of sources. In this survey we did not ask if providers actually referred to their national guidelines or explore the use of other online resources such as videos and articles to guide clinical practice.Further, the survey was limited in that it did not ask about which healthcare providers were able to give key medicines such as uterotonics, and was directed at the guidelines for obstetricians and gynecologists rather than other healthcare providers. It would also be interesting to examine guidelines provided for nonspecialist providers, such as midwives, who often attend deliveries, to see whether the same gaps and needs exist.5In FIGO's guideline document on prevention and treatment of postpartum hemorrhage in low‐resource settings,It is hoped that the findings presented here can be used in collaboration with partners to offer assistance to countries that do not have guidelines or are revising national guidelines to ensure that they are comprehensive, evidence‐based, and appropriate for their setting. These findings can also be used with partners to offer assistance to countries that do not have misoprostol listed on their EMLs. They could also be useful for discussion with partners when revising international guidelines to raise issues of conformity, dissemination, and implementation. They will also be used to guide FIGO's work disseminating evidence on misoprostol and other promising technologies for the management of PPH in the future.Both authors contributed to the writing of the commentary. JLM led on the survey design, and coordinated the data collection. JLM conducted the primary data analysis, with SK conducting further analysis.The authors have no conflicts of interest.


Background: There is a growing body of literature on evidence-informed priority setting. However, the literature on the use of evidence when setting healthcare priorities in low-income countries (LICs), tends to treat the healthcare system (HCS) as a single unit, despite the existence of multiple programs within the HCS, some of which are donor supported.Objectives: (i) To examine how Ugandan health policy-makers define and attribute value to the different types of evidence; (ii) Based on 6 health programs  to discuss the policy-makers’ reported access to and use of evidence in priority setting across the 6 health programs in Uganda; and (iii) To identify the challenges related to the access to and use of evidence.Methods: This was a qualitative study based on in-depth key informant interviews with 60 national level  and 27 sub-national (district) level policy-makers. Data were analysed used a modified thematic approach.Results: While all respondents recognized and endeavored to use evidence when setting healthcare priorities across the 6 programs and in the districts; more national level respondents tended to value quantitative evidence, while more district level respondents tended to value qualitative evidence from the community. Challenges to the use of evidence included access, quality, and competing values. Respondents from highly politicized and donor supported programs such as vaccines, HIV and maternal neonatal and child health were more likely to report that they had access to, and consistently used evidence in priority setting.Conclusion: This study highlighted differences in the perceptions, access to, and use of evidence in priority setting in the different programs within a single HCS. The strong infrastructure in place to support for the access to and use of evidence in the politicized and donor supported programs should be leveraged to support the availability and use of evidence in the relatively under-resourced programs. Further research could explore the impact of unequal availability of evidence on priority setting between health programs within the HCS. Systematic healthcare priority setting involves an explicit process of selecting which health interventions to pursue, for the purposes of resource allocation. According to the literature, using evidence  when setting healthcare priorities promotes objectivity, improves consistency, and strengthens the validity of the prioritization process and the decisions. The result is an improved quality of decisions.4 The use of evidence in priority setting is therefore critical, especially in low-income countries (LICs) where resources are scarce.8Evidence2; the World Report on Knowledge for Better Health9 and the latest call which came with the sustainable development goals where the collection and dissemination of high quality evidence is recognized as a key element in facilitating effective planning, follow-up and review of the implementation of the 2030 agenda for sustainable development.10There have been several global level calls to improve evidence-informed policy-making. For example, the statement from the 2004 Ministerial Summit on Health Research in Mexico City; the resolution from the 58th World Health Assembly which urged member states to strengthen knowledge transfer mechanisms; the 2008 Bamako Statement that called for countries to ensure that decision-makers have access to relevant evidence11 Grading of Recommendations Assessment, Development and Evaluation,12 the Disease control priorities project.13 At the regional level initiatives such as the Evidence Informed Policy Network,3 and the International Decision Support Initiative,14 are working to strengthen decision-makers access to and use of evidence in several countries.10 Efforts at the national level have focused on improving the quality of the available evidence eg, in Ethiopia15; as well as facilitating the use of evidence in policy dialogues by conducting rapid reviews and providing readily accessible evidence to policy-makers eg, in Uganda.10 There have been concurrent efforts to facilitate evidence based priority setting at the sub-national levels through projects such as the Community and District Empowerment for Scale-up in Uganda17 and the data-informed platform for health in India, Nigeria and Ethiopia.18 To some extent, these strategies have increased the availability and use of evidence, yet there are still circumstances in which evidence is either lacking, inaccessible, underutilized, or lacks credibility.19 For example, the limited resources that hamper priority setting in LICs may also contribute to a lack of credible evidence.22 Furthermore, there are reported cases where evidence has been disregarded by decision-makers whose priorities lie elsewhere.24The response to these calls has been the development of several initiatives at the global, regional and national that support the generation and use of evidence in decision-making. Examples of global level initiatives include the WHO’s–CHOosing Interventions that are Cost-Effective,25 Since it is often considered to be objective, evidence is thought to provide means through which personal interests and values which may have undue influence on priority setting, can be mitigated.28 However, there is a paucity of literature on how low-income country decision-makers define, access and use evidence when identifying healthcare priorities. To date most of the literature on the use of evidence in LICs has focused on policy-making11 and not specifically on priority setting – which is a critical component of the health policy-making cycle. Some of these studies have, for example identified barriers and facilitators of evidence-informed decision-making,26 while others have focused on approaches and tools to facilitate the use of evidence16 in LICs. The benefits of using evidence in healthcare priority setting may be particularly critical in LICs where priority setting is thought to be fraught with uncertainties, and can be easily influenced by the interests and agendas of various stakeholders.28Evidence is thought to be key to improving the quality of priority setting and resource allocation decisions.34 In many cases, programs with different funding mechanisms have different information gathering and reporting mechanisms, depending on the funding agencies.34 Might there be differences in the way different policy-makers conceptualize, value, access and use evidence in their prioritization processes? This paper responds to this question.The meagre literature on the use of evidence in priority setting in LICs has focused on the health sector as a unit. However, in countries such as Uganda, the health sector comprises of different health programs see  which mai) To examine how Ugandan health policy-makers define and attribute value to the different types of evidence; (ii) Based on 6 health programs  to discuss the policy-makers’ reported access to and use of evidence in priority setting across the 6 health programs in Uganda; and (iii) To identify the challenges related to the access to and use of evidence. This paper also highlights key points of similarity and difference between the health programs.The objectives of this paper are  in Uganda.30 At the national level we targeted respondents who were involved in priority setting with focus on specific programs . The initial respondents (policy-makers heading any of the 6 programs), identified subsequent respondents who they thought were knowledgeable. The interviewees were asked to identify additional respondents until we achieved theoretical saturation—no new themes emerged from subsequent interviews.We sampled respondents who were knowledgeable of priority setting at the national and district levels (policy-makers and development partners-donors). A mixture of purposeful and snowball sampling strategies was used.Three districts were included in the study. The districts were sampled for regional representation and duration since the district was formed . Within each district, we purposed to recruit all members of the district health. Using purposeful and snowball sampling, the initial respondent was the Director of district health services who identified subsequent respondents.We interviewed a total of 60 respondents at the national level (including development assistance partners (DAPs) or donors and policy-makers) and 27 at the sub-national (district) level .The reviewed documents included 3 health sector strategic plans (2000-2015), 14 Annual health sector reports (2001-2015), 2 National health policy statements (1999-2010), Meeting minutes (2009-2015) and media reports (2005-2015). These were aligned with the health strategic planning cycles.The interviews were conducted by a trained research assistant and the principle investigator using a pilot-tested interview guide see . The int36 However, an open stance was maintained throughout the process and any codes that emerged during the process were continually discussed and agreed on codes were added to the code book. While several themes about priority setting emerged from the data .“data and statistics on one hand, and subjective terms such as personal knowledge and experience on the other. For some respondents, evidence encompassed everything from quantifiable epidemiological data to subjective immeasurable concepts such as well-being. In their responses, respondents focused on the aspects of evidence that they value most. Broadly speaking, however, their uses of the term can be categorized as quantitative or qualitative, with national-level respondents prioritizing the former and sub-national–level respondents tending to prefer the latter. Both types of evidence are discussed below.As such, terminology used by the respondents ranged from: objective terms such as a) comparatively easy to measure (b) favorable to politicians and (c) well correlated with opportunities for future funding:Overall, respondents showed a preference for evidence with easily collectable and analyzable metrics. These also reported that they privileged quantitative evidence because it was easily understood. As they explained, quantitative information such as economic evidence, mortality and morbidity rates, and burden of disease are (“…You have to show them (the politicians) figures. When they see the numbers you get on the list (of priorities)...” (D3_2).“…You have to measure the benefits and the cost of any program and if the benefits are greater than the cost then it’s a priority, you should do it...” (DAP_ HIV_2).At the national level, some respondents reported a reliance on broadly generalizable data, which can be applied across countries and regions.“…So most people are not using evidence, what most people are relying on is what the global conventions said we should do…” (MOH_Vaccines_1).However, some respondents noted that contextualizing the quantitative data provided by the international community can be difficult without the necessary technical competence, which further contributes to their reliance on international experts in making use of data.Unlike national-level respondents, who tend to focus on one program, sub-national decision-makers tend to focus on evidence that can help facilitate program implementation across a range of areas within the healthcare system (HCS). Three district level respondents in this study noted that it is important to look beyond the quantitative evidence if priority setting and program implementation are to succeed. These gave an example of HIV whereby they alleged that successful interventions must incorporate cultural preferences and political demands alongside measurable factors such as effectiveness. If costs were the sole metric by which priorities were set, abstinence-based education would be the best approach for HIV prevention. However, a more complex picture, that requires decision-makers to account for a range of political and socio-cultural needs, emerges when information such as beneficiary assessment data and cultural preferences are considered.Acknowledging this, 4 district level respondents mentioned the value of data that incorporates local views, which can be collected through approaches such as conversations between the public and service providers; through the involvement of community resource persons; via suggestion boxes at health facilities, exit interviews in clinics, and community satisfaction surveys.However, some respondents were concerned about the difficulties in assessing the quality of and integrating this qualitative information in actual decision-making;“….I know that there have been big discussions in the organization related to how to try to measure a contribution to for example all activities on communication for development that are related to changing behavior, other things which are also very difficult to measure beyond process indicators …” (DAP_HIV_3).Respondents identified several sources of the evidence they used in priority setting at the national and district levels. These included; Policy documents and Reports; Special research and “experts.”Respondents identified several policy documents and reports they used as sources of evidence for healthcare prioritization. These included national health policy and planning documents, Strategic plans, annual review reports and national statistics. Respondents also identified specific performance reports generated through monitoring and evaluation mechanisms such as the Health management information systems from the districts and lower levels, biannual reviews (one respondent), in addition to monitoring indicators set in strategic and work plans (one respondent). Furthermore, respondent discussed large-scale sector reviews eg, the Health Financing Review,  and the National Health Accounts survey done in 2009/2010 which provided trends and relative proportions of government outlays, donor outlays but also other sources of funding like Out of pocket expenditure.the performance reports are available – the Ministry does performance evaluation every 6 months – we used to do it quarterly but found that it was a bit too frequent” (MOH_HIV_3).“…Respondents discussed research and special studies as an additional source. Specifically, several respondents discussed how the results from the burden of disease study though dated, still played an important role. Other specific studies include beneficiary assessments and specific program reviews and assessments;“The global burden of disease study, the burden of disease study that was done for Uganda in 1993 still influences very much the priorities and the interventions therein that are used for the health sector…” (DAP_HS_2).“Well it is the middle level and high level managers within the organization who regularly review program data and based on that program data they’re going to identify priorities...” (DAP_MNCH_5).Furthermore, interviewees articulated using locally produced evidence in the form of statistical assessments of the resources, databases, and other studies conducted at the different levels within the health sector, pilot studies , consumer satisfaction and governance surveys. In addition to these, respondents reported using evidence on the successes and failures experienced by similar African countries for transposable interventions, programs, and lessons.“…We can also look at the East African community in general how are the other member countries performing? Like in Rwanda, how have they improved their health indicators? We see if we can cut out (implement) those kinds of interventions, maybe we can  improve” (MOH_ HS_12).In contrast some respondents discussed the significance of international evidence;“… International evidence plays a major role in driving local action and…maybe for many reasons (such as) it is published in the lancet, the WHO has endorsed it…” (MOH_NCD_3).While they recognized the importance of international evidence, respondents also highlighted the need for increased support for systematic gathering and use of local evidence. This limitation was emphasized by the district level respondents who talked about how limited resources hampered the generation of local evidence.Consultative meetings with local and international experts were undoubtedly the most commonly stated medium through which evidence, expertise, and recommendations were generated. These meetings occur between diverse and numerous stakeholders within the Uganda health sector, including but not limited to: donors and other development partners, experts and specialists, district-level health officials, service- and care-providers, and community members/service users. The purpose of these meetings are to generate thorough knowledge of the health sector performance, understand where action is required, and to determine the most appropriate interventions;“…technical working groups…people come with presentations, people come with a new areas of research, people come with new evidence…most of the work on priority setting is done at the technical working group…by experts, knowledgeable people that’s where most work is done” (MOH_ MNCH_9).“At community level we create a forum for communities to have an honest discussion with the service providers and make sure that what they are doing responds to their needs” (DAP_HIV_4).DAPs, based on their technical and extensive infrastructure, were somehow expected to provide and to facilitate the gathering of evidence:“…Evidence is a big issue and it should be, from a technical perspective, it’s really their (DAPs) role to … strengthen and build that evidence base in order to provide it during the resource allocation, prioritization processes…” (MOH_HIV_1).The above sources of evidence were corroborated by respondents from the districts, who in addition to the above sources, and possibly due to their proximity to the community, identified the public , and non- government organizations as additional sources of evidence.When discussing the role of evidence when setting priorities, respondents from the national and district levels had varied perspectives. While several reported specific aspects of the prioritization process where evidence was reportedly used, some respondents doubted that evidence played a big role in the process.When discussing how evidence is used in the prioritization processes, the responses were similar at both levels and included;i) When assessing the effectiveness of program implementation - where monitoring and evaluation evidence is collected,.Certain epidemiological indicators can also give insight into how well programs are performing at the local level and can influence priorities that are set by an organization that is involved in implementation. One respondent explicitly described that priorities should be based on the needs of the population, which can be inferred from statistics that reveal gaps in care and service.But at the end of the day, as I said, we need to know how quickly do you diagnose a patient, how much time do you ensure you put in to getting them treated and followed up to completion. So the indicator in that case would be case detection rate. Case identification rate, extreme access rate, cure rate, and the others...” (DAP_HIV_5).“ii) Evidence was also reported to be used; to guide strategic planning, when identifying priority interventions, and in advocating for more resources..iii) Respondents also discussed how evidence was used to improve the quality of their decisions by limiting the role of self- interest. For example, district level respondents discussed how they use evidence to differentiate between the community’s real needs from their expressed needs;.Two respondents emphasized the importance of evidence in successful priority setting pointing out the ways in which a lack of evidence can hamper evidence-informed priority setting. Without evidence to set priorities, and without evidence to evaluate the degree to which the priorities are implemented, effective priority setting can be difficult.i) the availability and access to evidence, and (ii) challenges related to the use of evidence in priority setting, all of which are discussed.Respondents reported a range of challenges that hinder the use of evidence in priority setting. In particular, they highlighted challenges related to .Respondents discussed how poor physical infrastructure, political upheaval, and natural disasters can make it difficult for them to reach the areas where data is to be collected. Deficiencies in infrastructure can be particularly problematic when location-specific information is required yet some locations may be inaccessible due to poor physical infrastructure.As respondent from the emergency program explained,“…I find that sometime it is hard for the committees to visit to the community due to lack of the transport…” (MOH_Emergency_05).Respondents also discussed financial and personnel related challenges. For example, one national-level respondent highlighted how limited resources can hinder the implementation of country-specific studies. This was discussed in reference to other programs that have more resources:“…They (other people) have the resources and people to invest in to do a good job in packaging, delivering, and influencing. If you look at me (from the MOH) what resources do I have to do a good study and disseminate it? Where do I get those resources?” (MOH_MNCH_1).While adequate funding has facilitated the availability of evidence for programs such as HIV, inadequate funding, makes it difficult for policy-makers to generate the evidence needed to support decision-making in relatively under resourced programs.evidence (completeness and credibility) or to the different stakeholders (interests and the pressure to implement).Respondents discussed several factors that affect the use of evidence. These were either related to the incompleteness of the available information. For example, respondents spoke about how they lacked national and sub-national denominators, which numbers would allow priority setters to make inferences about a population:One challenge identified by most of the respondents was the “…What we usually have [are] numerators … and you don’t have information for decision-making because you don’t even have the denominators you just have some numbers…” (MOH_HS_14).It can be difficult to set priorities that are in the interest of the whole population if basic demographic information is not available to provide a clear picture of the population being served, or if such evidence has been manufactured or distorted.While some of the respondents recognized the relevance of qualitative evidence in providing a complete understanding of the health problems, they noted that it can be difficult to incorporate evidence that is not easily quantifiable into an evidence-informed priority setting process. However, omitting qualitative evidence means that sometimes decisions are based on “incomplete” quantitative evidence which may not provide a complete picture of the situation.credibility of some of the evidence is doubtful. For example, one national-level respondent, talking generally about GAVI funding, reported an instance where they think imaginary numbers were used for funding purposes:Although sometimes the decision-makers have access to complete information, the “… So, it’s happened all the time, if you take GAVI application from countries there are many applications where you have the data on vaccine coverage that were just made up just for the sake of getting money…” (DAP_Vaccines_6).Another aspect of the credibility of evidence, is timeliness. Respondents were concerned about having to use out dated information to guide priority setting. These specifically discussed instances where outdated evidence (such as the BOD/CEA study), continues to be used in priority setting.“…The global burden of disease study, the burden of disease study that was done for Uganda in 1993 still influences very much the priorities and the interventions therein that are used for the health sector…” (DAP_ HS_2).Respondents from the HIV and vaccines programs also cited problems with keeping up with incoming data. In particular, they noted the challenges in staying up-to-date with and subsequently applying the findings from ongoing scientific advances, which evolve at a faster rate than they can process. Conversely, other respondents discussed how sometimes researchers take a long time to generate the required evidence. In such instances, the research evidence tends to be out of synch with decision-makers’ timeframe, hence forcing the policy-makers to make decisions that may not be informed by the most recent evidence:This section focuses on respondents’ discussions of how donor priorities, government priorities, and local priorities and interests can interfere with the use of evidence in priority setting. There was a feeling among the respondents that donor agendas can lead to a rearrangement of the priorities, even when priorities are based on sound evidence. Governments can also be guilty of using their authority to steer priority setting. Two respondents criticized politicians at both the national and sub-national levels for prioritizing their own interests:“…So sometimes the evidence is overlooked …. Budgeting and resource allocation are done from a perspective that looks at balance of delivery—vis-à-vis the perspective of other colleagues on the political scene…” (D1_3).To compound this problem, respondents pointed out that well-intentioned politicians who want to make use of evidence for the prioritization process may not always have the capacity to interpret and apply the information at their disposal.Personal-interests  can also hinder evidence-informed priority setting. National and district level respondents discussed how local partners- especially donors- sometimes push priorities based on their own beliefs, preferences, and needs, in spite of the evidence. Furthermore, they noted that individuals also tend to prioritize health issues that, in one way or the other, has or might affect them.“…I think people set priorities when they are endangered. For example, if you feel you are at risk for something then you tend to … put priority on that something …. Ideally, it should be according to the figures, the burden of disease and statistics...” (MOH_HS_5).Challenges can also occur in the transition from acquiring evidence to implementing solutions. In particular, respondents highlighted how a desire for quick solutions, and hence implementation, can override the use of evidence. When decision-makers think they know the solutions (possibly based on their experience), generating evidence seems a waste of time and resources. The tendency is to quickly implement the known solutions, even though they are not supported with credible evidence:“…People think why waste money doing a lot of appraisals, a lot of studies, a lot of analytical work. We want money for actions because we know what the problems are. But you know what the problems are in a small way, and you do not know how that same problem translates three, four, five years down the road, which is critical, otherwise after four years you’re coming back to the same problem. So, getting that across has been quite a challenge…” (DAP_HS_5).A secondary challenge related to the generation and use of evidence in healthcare prioritization is the failure to implement the identified priorities. As implied in the above discussion, policy-makers tend to weigh the costs and benefits of investing in gathering evidence; especially if the possibility that the evidence-informed priorities are not feasible. Logistics eg, limited human resources, can stand in the way of the resultant priorities’ deployment:“…For example, we may have … evidence that male circumcision works, and so you want to support the delivery of that service … but you find there are no surgeons. If there is a surgeon, there are no theatre materials. If there are theatre materials, there’s no post recovery management or supplies. You find that the entire system has gaps…” (DAP_HIV_2).Just as a lack of resources poses a problem at the start of the evidence-gathering process, a lack of resources or personnel can stand in the way of implementing the solution based on the priorities that have been established.Despite the obstacles to evidence-informed priority setting, respondents from both the national and sub-national levels reported successes that stemmed directly from using evidence. For example, some discussed how evidence helped priority setters overcome limitations, and how in some projects evidence had been used to successfully establish health priorities. Evidence-weighing processes resulted in the establishment of reproductive health related priorities. Furthermore, the use of evidence related to disease epidemiology, resources, local populations, and cultural practices resulted in successful evidence-informed health system prioritization:“…we did a lot of research … to come up with evidence-informed interventions. Starting from baseline in the countries where we were working, we saw, for example, malaria, pneumonia and diarrhea which were the main causes of child mortality, and the most important thing was to really have health workers who were not only specialized doctors but could train in a short period of time to diagnose these illnesses and treat them in an easy way. We found that also culturally, traditional attendants and midwives were very useful for these [purposes], so we said let’s study how effective it would be to prevent illnesses, cure diseases, and save lives by training these kind of health workers…” (DAP_MNCH_3).While the results are presented individually and sequentially, they are interrelated. For example, the lack of resources will impact the availability and quality of evidence, its completeness, quality and even the timeliness. Furthermore, how decision-makers define and attribute value to the different kinds of evidence may influence their willingness to invest in gathering either the qualitative or quantitative evidence. The lack of resources may introduce stakeholders with resources, who, by virtue of controlling the resources, also introduce self-interest in setting priorities at the expense of what the evidence dictates. The pressure to implement hampers evidence-informed decision-making. However, the pressure to implement could also contribute to the incompleteness of the evidence – whereby the producers of evidence are also under pressure to produce some kind of evidence for the decision-makers.We presented findings from an empirical study on how decision-makers at the national and sub-national levels conceptualize, access and perceive the utility of evidence when setting healthcare priorities. Respondents used different terminologies to define evidence. They access the evidence through documents, research and experts. Although there was interest in using evidence, respondents at the national and district levels reported varying challenges related to their access to and use of evidence when setting healthcare priorities.21 It was beyond the scope of the paper to establish if and how the decision-makers actually used evidence when setting healthcare priorities. Some of the literature on how decision-makers actually use evidence discusses the impact of the decision-making contexts and cases where evidence is used to support already made decisions as opposed to guiding the decision-making processes.40 However, if the decision-makers we interviewed are genuinely interested in using evidence to guide their prioritization process, the interest may present an opportunity for augmenting the use of evidence, if the identified barriers are addressed. The barrier of limited capacity  to collect evidence which results in generating of poor quality and incomplete information which in turn affects the use of evidence; have been discussed in the evidence-informed policy-making literature.43 The importance of good quality and timely evidence cannot be overemphasized. Poor quality of the evidence, timeliness of the evidence, the relevance of the evidence – with emphasis on the context where the decisions are actually made are some of the key limitations to the use of evidence in policy-making in many LICs.47 While lack of resources may account for the poor quality of evidence, exploring ways through which well-funded programs such as Vaccines—could be leveraged to strengthen the evidence base across the health sector would be beneficial.49 This is more so since program implementation at the district level is often under the same umbrella and vertical information gathering may prove to be resource intensive.The study findings that decision-makers have an interest in- and endeavor to use evidence when setting healthcare priorities, in healthcare programming, implementation, and in program evaluation are consistent with the literature on evidence-informed decision-making.47 At the national level, the existing resource center should be supported with the necessary human resources with the capacity to manage an integrated health information system for all programs.29 Focused and integrated capacity strengthening and support would alleviate some of the additional barriers of poor quality, completeness, and timeliness.Since all sub-national (districts) have a health information officer who is responsible for synthesizing district level information, it may be useful that these individuals are supported to effectively accomplish their duties. Capacity strengthening and supporting these individuals to assume leadership in all information gathering within the district, may contribute to more efficient use of the available resources to collect high quality evidence.However, there were subtle differences between responses from the national and sub-national levels and between respondents from the different programs, which are worth discussing.46 Most of the resource mobilization and allocation for the health sector occurs at the national level. At that level, measurable, and easily quantifiable evidence tends to be valued, it may be more difficult to make a case based on unquantifiable, let alone measurable problems.46 Furthermore, decision-makers at the national level tend to be appointed technocrats who are relatively removed from the populations they serve.50 Conversely, while some of the sub- national decision-makers are also technocrats, several are elected. Even the technocrats have to be vetted by the political appointees.50 There might be pressure for them to listen to, and consider the public’s interests—which tends to be more qualitative. Conversely the sub-national decision-makers are the legitimate implementers of the health programs.50 Their proximity to the population and their mandate may compel them to appreciate the qualitative evidence so as to respond to the needs of the populations they serve.First, relative to the district level respondents, national level respondents were more likely to conceptualize evidence in terms of quantitative measures and to value this kind of evidence relative to qualitative evidence. The reasons and uses of this kind of evidence (for example as an advocacy tool) may be more critical at the national level.17 This means that priority setters at the 2 levels are often using evidence to different ends. Where national-level priority setters rely on evidence to guide higher order decisions about what policies to fund, sub-national–level priority setters are using evidence to determine practical solutions for enacting these national-level policies. As such, there is a gulf separating the uses of evidence even if the evidence being used is the same. This can create multiple demands on a single set of evidence, and can even lead to conflicting ideas about what should be done depending on the ends to which the evidence is interpreted.Second, national-level respondents focused on evidence-informed priority setting with respect to policy formulation and program evaluation. By contrast, sub-national–level respondents focused on the use of evidence for the implementation of policy.48 Hence if the sub-national level evidence is of relatively poor quality, it should impact the quality and completeness of the national level evidence.49 Conversely, this may be a reflection of the differences in the quality of data, between the districts; and the sampling strategy used in the study. The country has over 100 districts and only 3 were included in the study. It is possible that the findings are a reflection on only a small sample which may not impact the overall quality of evidence at the national level—where data from all districts are aggregated. However, this may also be a true reflection of the shortcomings in resources, infrastructure, and personnel capacity at the sub- national level—which need to be addressed.43Third, there was a perception that the evidence at the sub-national level was of relatively poor quality while that at the national level—especially that collected through reports—was of relatively better quality. This perception was surprising given the ideal hierarchy of decision-making within the decentralization framework. Ideally, the sub-national evidence should be pulled together to form basis for the national level decision-making processes.52 In contexts where resources are limited, it is unreasonable to allocate resources according to personalist or organizations’ interests as opposed to the evidence. However, it is also important to note that evidence is just one of the considerations in priority setting. There are additional relevant considerations which may come into play. The literature on the criteria for setting priorities emphasizes the need for enlisting and using criteria that are relevant and acceptable to the priority setting context.53 It may be problematic if the priorities based on the agreed upon criteria are contrary to what the evidence points to. In some of these cases, unacceptable criteria such as personal interests, come into play. Transparency of the decision-making processes, including the evidence used, the type, and its sources  and how it is used in the prioritization process would enable decision-makers and all the other stakeholders to evaluate the evidence; against the decisions.54 Transparency may also deter those who may want to introduce self-interests in the prioritization process.The factors that compete and sometimes trump the use of evidence when setting healthcare priorities have been documented in the literature. Some of the literature alludes to the perceptions that factors such as politics, popularity; may, in some cases, be perceived as more critical – at least to the decision-makers – than what the evidence says.58 As discussed above, it would be efficient for the health system to leverage the resources and the health information structure set up for the funded programs to benefit the less well-funded. Furthermore, respondents from programs such as HIV and Vaccines- seemed more appreciative for qualitative information—which might be a reflection of their experiences with the various types of evidence.61 Qualitative evidence, which accesses personal experiences play a big role in the acceptability of interventions as were the case of HIV control, and new vaccines.The findings that there were differences in responses between the different programs that were sampled was interesting. For example, respondents from the HIV, Vaccines and MNCH cases; compared to respondents from for example the NCDs case; were more likely to report the availability of evidence for their prioritization. This finding may not be surprising since these programs have enjoyed external funding which requires stringent accountabilities. In many cases the accountability should be supported with evidence and the funding agencies support the gathering of this evidence.The findings of this study should be interpreted with caution. Since this was a qualitative study involving a sample of respondents from 6 programs at the national level, and 3 districts, we cannot claim that the findings are generalizable. However, the purpose of the study was to provide in-depth understanding of the use of evidence and not generalization. We however, observed the validity standards for qualitative data collection and reporting and hence believe the findings are credible and make meaningful contribution to the literature on the use of evidence within these programs’ priority setting.systematic collection of both qualitative and quantitative information right from the community, to the health units. Encouraging the use of this evidence at the level at which it is collected may increase the interest in collecting it. District level information should inform national level priority setting. Similarly, national level decision-makers should strengthen the collection and use of both qualitative and quantitative evidence gathered from all the districts within the country. In the mean time, decision-makers should be encouraged to use the based available evidence to improve the quality of the priority setting decisions.While there is interest in using evidence in priority setting presents an opportunity for augmenting evidence-informed priority setting, problems with collecting, accessing and interpreting data, and several barriers that make it difficult to translate evidence into meaningful, practical priorities. Hence, decision-makers in LICs need to invest in improving the availability of high quality evidence and committing to its use when setting healthcare priorities. At the district level, there is a need to invest in Since lack of quality evidence is more prevalent in some of the health programs (due to different funding models), decision-makers should leverage the resources available in the well-funded programs to harmonize the generation and use of evidence in priority setting across the HCS.The subtle differences in the availability and use of evidence in priority setting across the health programs within the HCS highlight the need for similar studies in other LIC HCSs to substantiate and understand the nature of these differences and how they may impact actual health system priorities in this era of evidence-informed decision-making and priority setting.This research was funded by the Canadian Institutes for Health Research, Ottawa, ON, Canada [Grant 10558616]. The author acknowledges the following people for their involvement in various aspects of the project: the study respondents without whose participation, the study would not have been possible, Drs. Ssengooba F., Kiwanuka S. and Kirunga-Tashobya C. for providing meaningful insights during the data collection and analysis; Sarah Roger, Emmy Arnold, and the various research assistants for their participation in the data collection, analysis and report writing.This study was approved by the McMaster Research Ethics Board and Makerere University School of Public Health Research Ethics Board and the Uganda National Council for Science and Technology. All respondents provided signed consent.Author declares that she has no competing interests.LK is the single author of the paper.[1] In this paper we adopt Lavis et al, 2009 understanding of evidence as “concerning experience or observations (facts) that are intended for use in support of a conclusion.”[2] In this paper priority setting is defined as: Process of ranking health interventions for resource allocation purposes. Some of the literature uses priority setting synonymously with resource allocation. It occurs at the micro-level (bedside); meso-level ; macro-level  and the global level.Supplementary file 1. Ministry of Health Departments and Programs.Click here for additional data file.Supplementary file 2. Interview Guide for the Retrospective Interviews With Policy-Makers (for Example HIV Theme Officer).Click here for additional data file.Supplementary file 3. In Vivo Node Tree (Expanding on the Evidence Node).Click here for additional data file.Supplementary file 4. Additional Quotes.Click here for additional data file.Since it improves the quality of priority setting decisions, evidence is critical when setting healthcare priorities.Although there are global efforts to strengthen the evidence base for priority setting, decision-makers in low-income countries (LICs) still experience constraints in accessing and using evidence in priority setting.The availability, accessibility and use of evidence varies between programs depending on whether or not the program is politicized or donor supported.The infrastructure and resources that are available to the programs that are well-supported should be leveraged to strengthen the use of evidence in priority setting within all programs in the whole healthcare system (HCS).Priority setting and resource allocation are critical in low-income countries (LICs) that have limited health resources. However, priority setting can easily be influenced by several unaccepted subjective factors which can be mitigated by using evidence. Many LICs have limited access to quality evidence. We found that since healthcare systems (HCSs) in LICs still have donor supported programs, these programs tend to have more accessible, quality evidence compared to the other programs. These differences imply a possibility that priority setting in the other programs is liable to being influenced by other factors. It would be beneficial and efficient to leverage the resources that are available to the well-resourced programs and use them to improve the evidence availability, quality and access for the entire HCS. The additional challenges to the use of evidence in priority setting should also be addressed so as to ensure that the generated evidence is used to improve health sector prioritization.


There are currently no pediatric ERAS® Society guidelines. We created an ERAS® guideline designed to enhance quality of care in neonatal intestinal resection surgery.Enhanced Recovery After Surgery  contains supplementary material, which is available to authorized users. Limited applications of ERAS® in children have demonstrated reduced surgical infections, readmissions, reoperations, LOS, and cost [Enhanced Recovery After Surgery  was assembled including surgeons, anesthesiologists, and neonatologists as well as subject matter experts. Parent representatives were consulted at multiple points during guideline development.A modified Delphi method was used to reach consensus regarding the target population and topics. The target population was determined to be term neonates  without major comorbidities undergoing intestinal resection surgery within the first 4 weeks of life. Complex surgical conditions were excluded including necrotizing enterocolitis (NEC), abdominal wall defects, and short bowel syndrome. Fourteen topics were identified for areas of recommendation development (Online resource 1) .GDC members were assigned topics based on expertise. For each topic, a systematic search strategy was performed in conjunction with a research librarian (Online resource 2). Screening followed standard Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) methods . GDC memSystematic reviews, randomized and non-randomized controlled trials, observational cohorts and case series were included. Case studies and expert opinion were excluded. Articles meeting eligibility criteria were reviewed in full text. One or more recommendations were drafted for each topic. Evidence was summarized, and the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach was used to assess the quality of evidence based on risk of bias, imprecision, inconsistency, indirectness, and publication bias .A two-round modified Delphi was used to review and select recommendations . In the Of 3514 total publications reviewed, 2909 were identified via initial systematic searches and 605 through additional searches, citation searches, and expert identification. Screening and data extraction was performed within each topic (Online resource 3). GDC members reviewed the evidence and submitted 36 preliminary recommendations.Based on consensus, recommendations were eliminated due to very poor-quality or conflicting evidence. Other recommendations were rephrased or combined.® guideline has 17 recommendations (Table n = 81) of the papers had a rating of very low  or low , 22% (n = 26) had a rating of moderate, and 10% (n = 12) had a rating of high.The final ERASns Table  Fig. 1)® guideliIn the setting of intestinal atresia, pediatric surgeons must decide between stoma creation or primary anastomosis. Very low quality evidence comparing primary and secondary anastomosis in neonates with intestinal atresia demonstrates that primary anastomosis is associated with a reduction in LOS, decreased readmissions, and decreased need for reoperations –22. GiveSurgical Site Infection (SSI) rates are high in neonatal intestinal surgery, and the consequences of SSIs are severe . The quaFew studies investigate the optimal duration of perioperative antibiotic prophylaxis after neonatal intestinal surgery. Low quality studies show no difference in neonatal SSI rates when prophylactic antibiotics were given for less than 24 h as compared to greater than 24 h , 27. AltNeonates are at high risk of surgical hypothermia (<36.5 °C), and temperature monitoring is frequently neglected . HypothePerioperative fluid management in neonates aims to maintain tissue perfusion, metabolic function, and acid–base-electrolyte status. Monitoring clinical response to fluids, blood glucose, blood gases and electrolytes is a key part of intraoperative care . IsotoniAcetaminophen is an important part of a multimodal regime to limit opioid exposure after neonatal surgery. Multiple high-quality studies indicate that IV acetaminophen reduces postoperative morphine consumption in neonates when compared with other regimens . When IVMorphine is effective in treating postoperative pain following neonatal surgery . HoweverWith neonatal morphine use, the therapeutic window between analgesia and respiratory depression is narrow . Other iModerate evidence supports the use of an opioid sparing, multimodal analgesia strategy in postoperative neonates. The lowest dose of opioid should be given for the shortest possible time. All neonates receiving opioids should be managed with continuous pulse oximetry, monitoring of other vital signs, and regular assessment of pain scores.High-quality evidence demonstrates that ultrasound-guided regional anesthetic techniques and regular acetaminophen reduces the exposure of infants to opioids and other anesthetic agents . EpiduraMany high-quality studies have demonstrated the efficacy of lingual sucrose/dextrose as an analgesic in neonates. A large systematic review demonstrated the benefits of oral sucrose in neonates undergoing heel lance, venipuncture, and intramuscular injections . A smallNeonates require a distinct set of hemoglobin thresholds and transfusion guidelines. Neonates have a limited ability to tolerate stress, and anemia is associated with a high risk of mortality . Due to Standardized perioperative communication and care processes can reduce adverse patient outcomes, ensure continuity of care and improve staff communication , 52. A sImproving communication with parents by providing information on communication, family centered rounds, and using technology  all have demonstrated improved patient outcomes and family satisfaction –58. PareThe discharge experience is frequently perceived by parents as confusing with inconsistent communication . To bettThere is high-quality evidence to support early enteral feeding in post-surgical neonates. Neonates that were fed early have a shorter LOS, and decreased SSIs with no increase in anastomotic leaks , 64. AddBreast milk is a resource-friendly feeding choice for term infants, with a high level of evidence for its benefits. In post-surgical patients, where feeding intolerances are common, breast milk is typically well tolerated and the presence of immunoglobulin, prebiotics, and growth factors improve intestinal adaptation , 68. TheInfants with stomas commonly suffer from sodium depletion , 72. InaIn neonates with stomas, mucous fistula refeeding can augment absorption of enteral nutrition required for healing and growth. Low to moderate quality studies of mucous fistula refeeding demonstrate improved weight gain, shorter duration of parenteral nutrition, decreased anastomotic leakage, and less cholestasis –75. CompIn some instances, good quality data were not available (urinary catheter use) or data were conflicting (chlorhexidine vs. povidone iodine based skin preparation) preventing the development of recommendations.Many recommendations within this guideline are supported by weak evidence. In some cases, supporting evidence is inferred from high-quality studies in other populations (preoperative antibiotics); in other cases, potential benefits were judged to outweigh harms (prevention of hypothermia). Finally, in some cases, weak evidence was bolstered by national standards (transfusions) .® guidelines integrate evidence-based practices into multimodal care pathways to optimize postoperative recovery [® guideline is necessarily different from adult guidelines. This article presents the evidence base for 17 ERAS® recommendations.ERASrecovery . Due to ® guidelines will continue to improve the care of neonatal surgical patients.Often the data supporting recommendations are of low to moderate quality. The guideline will undergo regular review, and higher quality data will be used to further improve recommendations . EffectiOnline Resource 1® recommendation development (DOCX 14 kb)Topics identified by guideline committee for ERASOnline Resource 2Systematic search strategies used for each topic (DOCX 43 kb)Online Resource 3PRISMA diagrams outlining evidence screening process (DOCX 245 kb)Online Resource 4Summary of evidence and quality of evidence used in support of each recommendation (DOCX 120 kb)Online Resource 5Complete guideline document with full reference list (DOCX 184 kb)Below is the link to the electronic supplementary material.