Effective alcohol policy measures conflict with the interests of the alcohol industry. In this study we addressed how various alcohol industry actors in Norway have responded to research findings and police data relating to the possible impacts of changes in on-premise trading hours on violent offending.A content analysis of documents was undertaken. The documents comprised i) hearing statements from policy processes on on-premise trading hours at the national level, and in 15 Norwegian cities, and ii) newspaper articles and other media coverage of this topic in Norway.Alcohol industry actors employed a range of strategies to shape the use of evidence regarding on-premise trading hours and violence. Nationally, the relevance of the international research literature was questioned before the publication of an unfavourable national study which was criticized directly. This led to commissioned attacks on the findings, constructing what were claimed to be disagreements between experts, emphasis on the complexity of violence and the role of confounding variables, and deflecting attention to alternative interventions. The handling of evidence at the local level was importantly different, where different industry actors and forms of evidence, notably police data, were involved in debates.Alcohol industry actors employed various strategies to shape perceptions and use of evidence to advance their interests. The particular strategies and arguments changed over time as new data and research became available, and also varied between the national and the local levels, and by categories of industry actors. Alcohol use is among the leading causes of disease burden globally . AlcoholExamining both alcohol producers and other industries, Jahiel and Babor  identifyWhether the alcohol industry has played a similar role with respect to research on effective alcohol policy measures is less well studied. Bakke and Endal  found inIn Norway, local authorities  decide on licenses for off-premise and on-premise alcohol sales and thus on outlet density and trading hours, within nationally determined parameters. Every 4 years, each city/municipality council is expected to review the local alcohol action plan, which includes regulations on alcohol outlet density and trading hours. This permits ‘the policy window’ for local alcohol policy changes to open regularly , 19.The A key feature of the public debates on on-premise trading hours in Norway is the possible impacts of trading hours on violence. In September 2009, the Ministry of Health and Care proposed, among a number of changes to the Alcohol Act, that the national on-premise latest permitted trading hours (hereafter “closing times”) were to be reduced by one hour, from 3.00 am to 2.00 am. This reflected evidence in the international research literature that earlier closing times led to less violence , though The abovementioned research project was initiated and led by the present first author and examined the impact of restricting or extending closing times between half an hour and one and a half hours on violence rates in 18 Norwegian cities (hereafter the RN study). Quarterly data on police reported violent assaults at nighttime on weekends in city centers over a 10 year period in these cities were analyzed . AssaultThe publication of this study in September 2011 was immediately relevant to both the on-going decision-making on national latest permitted trading hours and for decisions at local levels in the next few months, as most municipalities were about to revise their alcohol action plans and thereby decide on possible changes in local closing times. In addition to the formal research, the local police’s own violence statistics was another key data source for assessment of the possible impacts of changes in on-premise closing times that was prominent in media discussions of these issues.In this study, we examine how various alcohol industry actors at both the national and local level in Norway responded to these two contrasting forms of evidence; the research findings and the police statistics relating to the possible impacts of changes in on-premise trading hours on violent offending.Three data sources were used as follows; i) policy documents including formal hearings statements from policy making processes at the national and the local level; ii) Norwegian print and ether media articles; and iii) sources further identified from media articles.n = 107) and the main radio and TV channels (n = 7) were electronically searched in a media database (Retriever ©) using the following search terms; ‘On-premise trading hours*’ AND ‘Violence*’ AND various search terms for actors in the alcohol industry AND ‘Police*’, or ‘statistics*’, or ‘research*’, covering the period 01.01.2009 through 01.09.2012. This period was selected because the debates surrounding the decision making process over a possible national restriction on maximum trading hours started early in 2009 and the final decision was announced in June 2012. At the local level, on-premise trading hours were likely to be politically debated and possibly changed in the wake of newly elected municipality councils in most Norwegian municipalities (elections held in September 2011), i.e. within the data collection period.Documents from the national policy making process were obtained from the Norwegian government’s website (regjeringen.no), while those on local policy making processes were obtained from the municipality web-sites for 19 of the 30 largest cities in Norway. In all 19 cities, on-premise trading hours were on the political agenda in the local alcohol policy making processes in 2011–2012. Virtually all Norwegian newspapers  , the national and regional trade organizations in the hospitality industry, individual alcoholic beverage producers (e.g. breweries) and their national trade organizations, and the trade organization of wholesalers/importers. Other types of commercial actors that may be affected by on-premise trading hours changes (e.g. taxi services and travel agencies/event agencies) were not included.Only material of relevance to how alcohol industry actors dealt with data on the closing times – violence association was included. In the media articles, data on responses to scientific and non-scientific evidence (police data) comprised both direct and indirect quotations from alcohol industry actors and journalistic interpretation of these responses when this was supported in other parts of the article.The media search strategy identified a total of 2825 media articles, newspaper contributions/letters to the editor, and other media pieces. Headings and extracts of these were all screened and approximately half were read in full text. Altogether 56 unique media articles (including newspaper contributions) related to industry responses to evidence on closing times and violence. Similar versions to those included were identified in other media sources, and are not counted within the 56 selected for study. In addition, numerous media articles covered industry representatives’ views on on-premise trading hours as an issue, but did not express views on research evidence or police statistics specifically and were thus excluded.The media data were sorted by date and categorized by i) policy level ; ii) type of industry actor ; and iii) type of evidence (research vs police statistics); and iv) media reach . Initially, we applied a summative content analysis  to exploDifferent actors had different foci. In the media articles, there was primarily attention to research evidence at the national rather than the local level. The main industry organizations were more involved with research evidence and at the national level, whereas individual industry actors were more often concerned with the evidence at the local level, where police statistics were the predominant source of evidence discussed Fig. . Moreoven = 10), included eight statements referring to the evidence on trading hours and violence. At the local level, industry actors gave hearings statements in 9 of the 19 cities where on-premise trading hours were on the political agenda, and among these 9 hearing statements, a national hospitality industry trade organization gave 2 statements addressing evidence on closing times and violence. In the hearing statements, the trade organizations were typically involved at the national level paying attention to research evidence, whereas at the local level, there were few hearing statements from industry actors, concerned with research evidence (n = 1) and police statistics (n = 2) respectively.Hearing statements from industry actors regarding possible national restrictions , each commissioned reports on violence in relation to on-premise trading during the autumn 2011/winter 2012. These reports were both concerned with the possible national restrictions.“There is not a single research report in the entire world concluding that a small restriction in trading hours will lead to less violence” .“The international research  is in no way relevant in the debate of a one hour restriction.” At this time, these claims were, in one sense, largely in line with international reviews of the literature, which mainly showed harmful effects of large extensions of trading hours and few studies with inconsistent findings examining smaller changes in trading hours  does not believe trading hours and universal prevention measures are of any importance.” .Industry actors chose carefully which evidence to contest; although industry actors denied the existence of research evidence of an impact of trading hours on violence, the evidence of an overall association between alcohol consumption and violence was acknowledged. Industry actors were often rhetorically explicit in welcoming further research on this issue, thus not challenging the potential value of research evidence per se. Moreover, in several hearings statements, particularly those from the larger organizations, the actors portrayed themselves as serious and responsible, sharing the concern with the problem of violence in the night time economy, and being eager to find solutions.The proportion of total alcohol volume being consumed on-premise was emphasized in several statements, the argument being that any changes in this small volume was unlikely to impact on violence rates. Moreover, the complexity of the problem at hand was also noted by one actor, who claimed this meant a different approach was needed.“We have persistently noted the lack of sufficient evidence on the association between trading hours and violence. […] The Government deserves praise for having reached a responsible decision.” .In November 2010, when the Ministry of Health and Care announced their decision to delay national restrictions until they had Norwegian research evidence, industry actors expressed unanimous appreciation of the decision and repeated their views on the international evidence.omitted taking other factors, including police on the streets and public transportation, into consideration”.Less than 1 year later, the RN study  was publ“Now that large and important weaknesses of the [RN] study have been revealed, it is impossible for policy makers to adopt restricted trading hours.” .Three days later, NHO Reiseliv nuanced and strengthened their critique by referring to a short manuscript, which they had commissioned from a private consultancy firm (Menon Business Economics). The manuscript, assessing the RN study, was leaked to the media and its main points conveyed via media interviews with NHO Reiseliv. The headline of the 1.5 page manuscript was “Large and important weaknesses in the [RN study] report”, a phrase that was repeatedly echoed by NHO Reiseliv and other hospitality industry representatives in the media. The main critique pertained to the methods employed in the RN study and claimed lack of control for other factors impacting on violence. The manuscript was later included as an appendix in a report from Menon Business Economics, also commissioned by NHO Reiseliv, which reviewed literature and Norwegian data related to on-premise trading hours, alcohol consumption and violence including the RN study . NHO Rei“Useless report about on-premise trading hours” and “the findings are useless for politicians” were frequently repeated by hospitality industry actors. Holding a PhD in meta-analysis of British government support, this politician was regarded by several journalists, and claimed by industry actors, to be an expert. Some commentators noted that the politician likely had an underlying political agenda, as he and his party were strongly in favour of extended trading hours. Nevertheless his critique fueled the media’s attention to what was now being construed as a disagreement between experts, and resulted in newspaper headlines like “Strong fight between experts” , October 6, 2011). This framing was also conveyed by industry actors in media interviews. Referring to the local politician’s blog post and to the authors of the commissioned review of the RN study, NHO Reiseliv emphasised what was presented as a convergence of expert opinion as follows.“Two other experts, independent of each other, have now found that there are large and important weaknesses of the [RN] study.” .The industry commissioned critique of the RN study gained support from a local level politician, who offered similar critical remarks in newspaper interviews. He elaborated these remarks and wrote a blog post on a right wing liberal think tank website  . Statem“This confirms what we’ve been saying all the time, the issue is more complex than what the researchers claim. Now it seems that the researchers admit we were right in our critique” .The authors of the RN study (including the present first author) noted publicly that factors other than trading hours may also impact on violence rates, whilst defending the methods and study findings . In a ne“lucky to receive a conclusion she wanted” . In several debates on TV, this trade organization referred to the RN study in ways suggesting that the researchers served a political agenda. Thus not only was the integrity of the scientists attacked, the national politicians involved were also attacked.Beyond attacking the science, another kind of critique pertained to research integrity and possible political manipulation of the research process. NHO Reiseliv claimed that the Minister of Health, being in charge of the research institute where the RN study was conducted, had been “We do not believe the world is so simple that violence is reduced by almost 20% if closing hours are restricted by one hour. A number of other factors impact on violence.” .The issue of credibility of the science and the scientists was linked to that of complexity, which was repeatedly brought up by NHO Reiseliv and other hospitality industry representatives, suggesting that it was unlikely that there could exist policy measures with the effects observed in the research.“Both Norwegian and international research demonstrate large uncertainty regarding the association between on-premise trading hours and violence, and especially when the regulation is less than two hours.” .In February 2012, the other large national hospitality industry trade organization  and a regional trade union for servers published a report, which is likely commissioned work from a consultancy firm . This ren = 24). The arguments were no longer that evidence was lacking, but that the Norwegian research showing an impact of closing hours was methodologically flawed and that the researchers had a political agenda. Furthermore, it was asserted that Norwegian experts in the field disagreed, and that the problem of violence was much more complex than suggested by an implausibly simple solution of restricted closing hours “Up to early June 2012, just before the Norwegian Government was expected to finally decide whether or not to restrict the national latest permitted closing time from 3.00 am to 2.00 am, the wine and spirits importers’ association and NHO Reiseliv were involved in lobbying, as reported in the media. In one national newspaper article, representatives of both these organizations confirmed lobbying or “spreading information in society” by means of meetings with politicians, media reports, websites and multiple newspaper contributions and media interviews. Our data comprise only the latter two sources.In late June 2012, the Norwegian government launched a white paper on alcohol and drug policy, which stated that the national maximum trading hours (3.00 am) were to be kept as they were. The Minister of Health made little comment on this, beyond stating that over-serving was the big problem and that the requirements for local control of bars and pubs were to be stricter.In many Norwegian cities, the question of whether or not to restrict local closing times was being debated, alongside the national decision making process. Compared to the debate at the national level, as described above, a broader range of actors were involved in the local debates and local policy processes. These included individual bar owners/managers, local trade organizations and a local brewery, in addition to the national and regional trade organizations.iven the lack of scientific evidence for the effects of the proposed restriction, we ask that the current trading hours are continued.” In a hearing statement requesting extended trading hours in one city, the regional hospitality industry trade organization (NHO/NiT) used a similar argument: “there is no valid evidence that restricted trading hours has led to less violence”. They further argued that use of both available research and the police statistics had not taken into account other factors that may have impacted on violence rates. A similar argument was used by a local brewery in their hearing statement responding to a proposed restriction of closing time.In the local debates, bar and pub owners rejected research on trading hours and violence, but mainly by stating mere disbelief in the findings. However, a local brewery referred to one of the industry commissioned literature reviews  in their“Is it really a true decrease in violence in [city], and what is the true cause of the decrease? This can only be discussed when we have access to – and can perform independent analyses of – the real data. However, the police refuses to publish these data.” .This focus on police statistics on violence was much more prominent at the local than at the national level see Fig. . In many“There is not as much trouble around bars and pubs as the police claim.” .Several actors, both among the individual bar owners and among the hospitality industry trade organizations, claimed that the police exaggerated the violence problem.“The police only uses the data that support their own views, which leads to a biased presentation. I suspect that the police both over-report and under-report in order to obtain a desired basis for their recommendations to the policy makers.” .Some also accused the police of manipulating their data or their presentation of the police statistics. The similarities with tactics used in critiques of the research at the national level, thus extended also to attacking the integrity of those providing the evidence.Cherry picking of evidence was also seen among industry actors at the local level. Some referred to experiences in other cities or regions, suggesting that trading hours had no impact on violence, or even the opposite effect, thereby selecting anecdotal evidence to support their views.“Many of those who cause trouble, have something quite different than alcohol in their blood, and we do not have powder in our taps.” .“Are we [who work] in the night time economy business responsible for violence, given that only a small fraction of the alcohol is sold by us?  We experience that people drink a lot before going to the city centre.” .Others refuted any impact of restricted trading hours by dismissing alcohol serving as the source of the problem, and they claimed that violence and nuisance were often attributable to drugs rather than alcohol, or that drinking prior to going to bars was the cause of late night violence in the city centres rather than on-premise drinking.Overall, the industry trade organizations were less prominent in debates at the local level as compared to the debate on national restrictions. At the local level, individual bar and pub owners and their local associations were more prominent. The trade organizations’ arguments at the local level reflected those used in the national debate, although in this context, they emphasized to a larger extent the importance of the local authorities’ capacity to assess and decide on local matters. The individual bar and pub owners responded more often to local police statistics than to research on trading hours and violence and in doing so, they sometimes cherry picked examples to illustrate their points of view.The trade organizations had sufficient resources to prepare, nuance and substantiate their arguments on evidence and also to convey them through more numerous and more significant channels than local actors. Other actors did also use the phrases and arguments already produced by the trade organization actors. But, rather than providing a mere echo, they often presented their arguments along with their own personal testimonies of knowledge of the business and the problem at hand, which may have strengthened credibility. In the two cities where industry actors claimed lack of evidence on closing hours and violence in their hearing statements, the political decisions differed; in one city the proposed restriction was not adopted, and in the other city, restricted closing time was continued.Industry actors employed various strategies to shape perceptions of evidence by conveying their preferred interpretations regarding on-premise trading hours and violence. The relevance of the international research literature to the impacts of smaller changes to closing times was questioned first at the national level. After the publication of evidence demonstrating impacts of such changes in Norway, industry actors attacked the study and systematically cast doubt on the evidence, employing a range of strategies which are used by industry actors in other areas  (see TabIn line with the existing research literature on industry tactics to shape perceptions of available evidence (see  for tobaThere has been limited prior study of how alcohol industry actors shape evidence potentially harmful to the industry in the processes of policy making , 17. AlcStudy limitations are several. First, alcohol industry actors have clearly responded through a broader range of channels than those covered by our data set, including TV debates and interviews, and directly lobbying politicians and other policy actors. Second, many industry actors may have chosen, as a deliberate strategy, to ignore research/police statistics and this kind of strategy could not be revealed in our data set, which concerns what was done rather than what was not done. Thus, the full range of strategies in the handling of evidence employed by industry actors likely exceed those identified in the present study. Finally, the question as to whether, or to what extent, the shaping of evidence through industry activities made an impact in the policy making processes at the national and local levels, lies beyond the scope of the present study. The use of research based knowledge and other evidence of the effects of policy measures is far from simple and solely instrumental in policy making processes, it is also the subject of, and reflects the balance of forces in, political contestation –41.The present study contributes to the existing evidence base specifically on alcohol industry actors by drawing attention to the ways in which strategic approaches to evidence management vary flexibly over time as the evidence itself changes, and as do the policy circumstances. The circumstances of this study are of a policy controversy operating contemporaneously at both national and local levels where scientific evidence published by the first author was a high profile feature of the debate. Other policy debates may involve scientific and other forms of evidence in less prominent ways. The handling of evidence studied here is one component of higher level political strategies which frame the issues with which policy making contends in ways favourable to industry interests so as to defeat unfavoured proposals, such as this one . This frAs part of the research process, researchers get involved in research dissemination, knowledge exchange and various aspects of policy making. This study demonstrates that, notwithstanding reflexivity considerations, there is important value in not only observing and promoting the use of research evidence in policy making, but in also studying it. Alcohol policy researchers are well placed to identify misuses of evidence in the strategic management of research by industry actors and other vested interests. They are thus well placed to assist policy actors. Alcohol industry actors rhetorically distance themselves from tobacco industry actors in their efforts to influence policy, which have otherwise shared obvious similarities , 44, notAlcohol industry actors employed a range of strategies to shape the use of the various types of evidence on on-premise trading hours and violence to advance their own interests. The particular strategies and arguments changed over time as new data and research became available, and also varied between the national and the local levels and by categories of industry actors. There is a need to better understand how the handling of different forms of evidence is accommodated within the political strategies of a range of industry actors in respect of particular policy issues and forms of evidence, at different levels of policy making and in varying policy contexts.


Rare diseases are a global public health priority. Though each disease is rare, when taken together the thousands of known rare diseases cause significant morbidity and mortality, impact quality of life, and confer a social and economic burden on families and communities. These conditions are, by their nature, encountered very infrequently by individual clinicians, who may feel unprepared to address their diagnosis and treatment. Clinical practice guidelines are necessary to support clinical and policy decisions. However, creating guidelines for rare diseases presents specific challenges, including a paucity of high certainty evidence to inform panel recommendations.This paper draws from the authors’ experience in the development of clinical practice guidelines for three rare diseases: hemophilia, sickle cell disease, and catastrophic antiphospholipid syndrome.We have summarized a number of strategies for eliciting and synthesizing evidence that are compatible with the rigorous, internationally accepted standards for guideline development set out by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system. These strategies include: use of pre-existing and ad hoc qualitative research, use of systematic observation forms, use of registry data, and thoughtful use of indirect evidence. Their use in three real guideline development efforts, as well as their theoretical underpinnings, are discussed. Avenues for future research to improve clinical practice guideline creation for rare diseases – and any disease affected by a relative lack of evidence - are also identified.Rigorous clinical practice guidelines are needed to improve the care of the millions of people worldwide who suffer from rare diseases. Innovative evidence elicitation and synthesis methods will benefit not only the rare disease community, but also individuals with common diseases who have rare presentations, suffer rare complications, or require nascent therapies. Further refinement and improved uptake of these innovative methods should lead to higher quality clinical practice guidelines in rare diseases. The U.S. National Institutes of Health (NIH) Office of Rare Diseases Research defines a disease as rare if it affects less than 1 in 1500 people, while the European Union defines rare diseases as those affecting less than 1 in 2000 people . Many raMost rare diseases are not a focus for policymakers, funders, and researchers in many countries. Pharmaceutical companies may invest significant financial resources in bringing orphan drugs to the market, but often struggle to recover costs through sales . WithoutPerforming properly sized studies of these diseases, which minimize bias and confounding, is difficult when the number of affected individuals  for any one disease is low . AdequatEven if a researcher successfully designs and executes a trial, dissemination of results can be challenging. Journal editors may be hesitant to publish studies with a narrow audience, on the assumption that it will have little impact. And much like in common diseases, if a rare disease trial’s findings are negative, publication becomes even more difficult.are available, they are often heterogeneous. Patients may have widely varying baseline characteristics, interventions and comparators may be used differently, and outcomes and measurement techniques may differ. Patient reported outcomes, such as disease-specific quality of life instruments, may not exist, may not be reported at all, or may be reported with instruments that are neither properly validated nor straightforward to interpret; investigators often rely on an array of surrogate outcomes instead. Heterogeneity in studies can also make aggregating data challenging. The end result of all of these challenges is a slim and inconsistent body of evidence, much of which is low certainty and does not address critically important outcomes.When published studies The barriers to guideline development for rare diseases are considerable, spanning from evidence appraisal and synthesis, to issuing recommendations for care, to knowledge translation. Ideally, clinical practice guidelines summarize evidence for patient-important outcomes, and use clear criteria to generate recommendations , 17. In Our aim was to use a dedicated framework for guideline development in the field of rare diseases, as proposed by the RARE-Bestpractices Working Group . This frWe piloted the suggested framework in three specific guideline development processes undertaken in three different rare conditions. All three guidelines were coauthored and developed by authors of this report. All three guidelines also explicitly aimed to achieve a double goal: 1) produce a guideline to be used in clinical practice; and 2) implement our framework to overcome expected barriers in guideline issuing in rare diseases. The individuals most focused on the latter goal where the guideline methodologists, whilst the clinical panel members acted as end users trying out and experiencing the impact of our framework.The first guideline focused on hemophilia, an X-linked bleeding disorder caused by the hereditary lack of a coagulation factor. Blood clotting is severely impaired in hemophilia, resulting in serious bleeding with minimal provocation. Hemophilia A (deficiency of coagulation factor VIII) and hemophilia B (deficiency of coagulation factor IX) are both rare; hemophilia A affects fewer than 1 in 10,000 people and hemophilia B affects fewer than 1 in 50,000 people. In 2016, the National Hemophilia Foundation in the United States partnered with McMaster University in Canada to publish a guideline that explored the optimal model of care delivery, and the optimal composition of the care team, in this rare disease. The guideline used the GRADE  system and the GIN-McMaster guideline development checklist, and adhered to the principles set out by the Institute of Medicine in their 2011 document, “Guidelines We Can Trust.” –21 The gThe second guideline focused on catastrophic antiphospholipid syndrome (CAPS), also known as Asherson’s syndrome. CAPS is a clinical syndrome characterized by the rapid onset of multifocal thrombosis associated with multi-organ failure in patients meeting the serological criteria for antiphospholipid syndrome . The morThe third guideline focused on sickle cell disease (SCD), an inherited condition in which a mutated form of hemoglobin distorts red blood cells in low oxygen conditions. These crescent shaped “sickle” cells can obstruct the flow of blood in blood vessels, so that oxygen cannot reach nearby tissues. SCD results in anemia, acute painful crises, chronic pain, strokes, and damage to vital organs. The overall prevalence of SCD per 10,000 live births is approximately 30 in African Americans, 0.1 in Caucasian Americans, and 0.3 in Hispanic Americans . This prAll three guidelines were developed using an overarching approach proposed by the GRADE working group. GRADE is a methodological approach that helps guideline developers present summaries of evidence in a structured and standardized way, make transparent judgments about the quality of evidence, and then systematically move towards developing recommendations . GRADE hQualitative research methods can be used to generate evidence on patient values and preferences, equity, acceptability, feasibility, and implementability. These factors are essential to inform Guideline Panel recommendations when using the GRADE system. They are also critical in the guideline implementation phase as they support health policy decision making. However in both rare and common diseases, published evidence on these factors can be difficult to find; it may be necessary to generate it to support guideline development.2.Expert-based evidence can be systematically solicited through structured observation forms. Despite the paucity of published evidence, there is often a great deal of knowledge about rare diseases – usually from health care providers who cumulatively have many years of real world experience treating patients. Individually, they may not have enough data to publish a peer reviewed study. Furthermore, due to publication bias, data describing ineffective therapies may remain unpublished. However, when data are collected using a structured approach, this “expert-based evidence” becomes invaluable to inform recommendations for clinical practice.3.Patient registries can be used to supplement the published literature, particularly for questions where scant data were available. Registry data can be used to identify or explore relevant patient subgroups. Registries can also give a snapshot of the impact and natural history of a disease, and the variety and effects of treatments that are used. Registries have the benefit of prospectively capturing patient information; a robust registry, comprehensive in the depth of captured data and inclusive of all known patients, can be equivalent to a high-quality observational study.4.Finally, indirect evidence can be used. The concept of directness (or indirectness) of evidence is well-established in GRADE, and is used to assess whether the body of evidence used to inform guideline recommendations is directly applicable to the health questions of interest. Sometimes, guideline developers may not be able to find published evidence that directly addresses their clinical questions. In such cases, they can turn to evidence that differs to some degree in terms of population, intervention, comparator, and/or outcome of interest. Indirect evidence becomes particularly valuable in rare diseases. For example, indirect evidence may include extrapolation of data from a population affected by a more common disease that shares some features with the rare disease.  When the search for applicable/relevant indirect evidence is performed, a specific process to minimize bias can be adopted.In all three guidelines, elements of the RARE-Bestpractice Working Group’s pilot framework were successfully used. The specific elements, and our experience with their use, are described below.In this guideline, the following elements of the RARE-Bestpractice Working Group’s framework were piloted: indirect evidence; qualitative research; and expert-based evidence.The NHF-McMaster hemophilia guideline project started with a formal process of priority setting, question generation and prioritization, and identification of patient-important outcomes. Persons with hemophilia (PWH) and their families as well as health care providers were extensively involved in these initial steps. A modified Delphi process was used to finalize guideline questions and outcomes , 23.Due to the paucity of evidence on models of care in hemophilia, the methods group conducted parallel systematic searches for meta-analyses from other chronic diseases to generate indirect evidence . .UltimaThere was also scant published information on stakeholder (including patient) experiences and perspectives around health and psychosocial outcomes of importance, acceptability of different models of care, impact of different models of care on health inequities, and feasibility of implementing different models of care. An experienced qualitative researcher on the guideline development team designed and executed a pilot qualitative study to explore these areas . At the The NHF-McMaster hemophilia guideline adapted surveys developed for a guideline project in Saudi Arabia to systematically capture expert-based clinical observations from the Guideline Panel, as well as a selection of other individuals identified as experts in the field , 23, 33.Patient registries were not used in the NHF-McMaster hemophilia guideline, as the Panel had access to large longitudinal and cross sectional observational studies. However, the guideline ultimately called for Hemophilia Treatment Centers in the United States to reaffirm their commitment to collecting and sharing high quality patient-level data; this enhanced data collection capacity should strengthen the evidence base for future guidelines.In this guideline, the following elements of the RARE-Bestpractice Working Group’s framework were piloted: patient registry data; indirect evidence; and expert-based evidence.Similar to the NHF-McMaster hemophilia guideline, the initial development stages of the CAPS guideline revealed low certainty to no direct evidence for important outcomes identified by the Panel. Therefore, in addition to the systematic reviews conducted for each guideline question, further methods of data retrieval and evidence creation were used.The CAPS methods group and Guideline Panel identified the CAPS Registry as a potential source of data to inform the guideline. The CAPS Registry was initiated in 2000 by the European Forum on Antiphospholipid Antibodies to capture diagnosed cases worldwide . CurrentIn contrast to the NHF-McMaster hemophilia guideline, the CAPS methods group did not conduct novel research on patient values and preferences, acceptability, feasibility, or equity. Indirect evidence from other conditions was however used to inform the Panel’s seven outcomes of interest: mortality, permanent organ dysfunction, permanent neurologic deficit, complete recovery, thromboembolic event, and amputation. Recognizing that individuals with CAPS often suffer limb loss and neurologic deficits, the Panel considered patients with diabetes who had limb amputations, and patients with stroke who had neurologic deficits. Health utilities of stroke, acute thromboembolism, and major intracranial bleeds were also considered. Finally, indirect evidence from patients with acute arterial and venous thromboembolism and their bleeding complications  were considered.Expert-based evidence was systematically captured from CAPS Panel Members to provide clinical observations for questions around therapy and diagnosis. The collected expert-based evidence was used to inform two key areas. First, the responses around each treatment option provided data on their relative usage in practice. For example, systematically collected observations revealed that only one Panel expert had experience using rituximab. This reflected potential uncertainty around the use of rituximab in wider clinical practice for the Panel to consider. Second, systematically collected observations from Panel Members were discussed and compared to data from systematically reviewed published studies. Panel Members, both spontaneously during the panel meeting and when prompted during an exploratory interview at the end of the guideline development process, stated this was valuable in adopting a standardized and objective way of describing their own practice and experience. Many admitted that their perception of the relevance and direction of the evidence changed when they looked at their experience in a semi-quantitative way. A template of the standardized expert observation form used in the hemophilia and CAPS guidelines is in Appendix A.In this guideline, the following elements of the RARE-Bestpractice Working Group’s framework were piloted: qualitative research; and expert-based evidence.The SCD guideline process started with identifying a source guideline that could be used for the Adolopment process, including to identify subject areas of interest, prioritize questions, and identify relevant studies . After aA methods group, independent from the Guideline Panel, conducted the literature search in three sequential steps for each of the prioritized questions: (1) screening of the NIH guidelines reference list; (2) electronic database search for relevant systematic reviews; and (3) electronic database search for relevant primary studies. Systematic searches were also conducted for values attached to the outcomes of interest, preferences for the interventions under consideration, and resource use associated with the intervention options. The Panel was asked to review the list of identified studies and suggest if any were missed; however no studies reporting direct evidence were identified by the experts beyond what had been captured through the first two search steps. The SCD methods group did not conduct systematic searches for indirect evidence in related diseases.Panel Members were asked to complete online surveys looking at values and preferences, and resource use related to the interventions of interest. Panelists were also asked to suggest additional published and unpublished information of interest relating to benefits and harms, values and preferences, resource use, acceptability, feasibility, and equity. These additional pieces of information were provided in the form of unstructured communications from Panel Members to the methodology team, and were discussed during the face-to-face Panel Meeting. At their in-person meeting, Panelists were asked to complete a survey soliciting feedback about the guideline development process.The qualitative study conducted as part of the NHF-McMaster hemophilia guideline was submitted to and approved by the Hamilton Integrated Research Ethics Board in Hamilton, Ontario, Canada. Written informed consent was obtained from all participants in the qualitative study. Ethics approval for collection of expert based evidence in all the described guidelines was not deemed necessary, as it involved surveys of guideline panel members, whose ethical treatment and consent to participate in guideline related activities was considered implicit. All other work described in this study involved systematic reviews of anonymized and published data, and thus did not require ethics submission and approval as per the Hamilton Integrated Research Ethics Board in Hamilton, Ontario, Canada.Rare diseases pose a real challenge to patients, providers, payers and policy makers. Many of these conditions have no satisfactory therapeutic options, while others have very expensive therapeutic options with relatively little evidence to support them , 40. HeaOur study – in which we applied a framework for evidence synthesis, allowing for the flexible and transparent use of indirect, expert-based, and ad hoc newly generated evidence – successfully addressed some of the “evidence gaps” encountered when developing three target rare disease guidelines. These “gaps” are common to guideline development for many rare diseases. Guidelines in common diseases frequently rely on comparative evidence of treatment effects . This reliance on comparative studies is often not practical for rare disease guidelines. Indeed, there is often only one treatment for any given rare disease, or there is only very low certainty evidence for the effect of different treatments.A major strength of our study is that it shows how, in the absence of disease-specific comparative studies, valuable evidence can still be retrieved and successfully used. Systematic reviews of evidence from similar, more common diseases can provide valuable comparative information to Guideline Panels. Qualitative research has proven to be a viable data source for values and preferences, acceptability, feasibility, and equity . These iAnother strength of our study is its confirmation that unpublished non-experimental data  can be elicited and collected in such a way to be usable by Guideline Panels. Systematically collected evidence from field experts can provide this information; further, “going to the source” may overcome dissemination bias, albeit at the risk of introducing selection bias (which must be carefully monitored).Either of the two approaches indicated above can overcome the tendency of guideline developers to focus only on randomized controlled trials and large observational studies to inform their work, which may not be possible outside the field of common diseases. As users demand high-quality guidelines for rare diseases, it will be important to incorporate non-experimental and non-comparative data collected in a systematic and transparent fashion.There were clear limitations to our work, and to the RARE-Bestpractice Working Group’s framework. Qualitative research did provide valuable information in the NHF-McMaster guideline. However, in general, published qualitative data can be difficult to find in rare diseases – either because these studies are not conducted, or because they are not published. Guideline developers can conduct their own qualitative studies (as the NHF-McMaster group did), but this takes time and resources. Qualitative research may have complemented the limited quantitative evidence base in the CAPS guideline. In particular, it may have provided insight into patient perspectives, which are not reported in the literature. However, identifying patients to interview was a major limitation to conducting a qualitative study of CAPS; the number of patients diagnosed with this rare condition every year is low, the presentation is precipitous and often fulminant, and the mortality rate is close to 50%. In such circumstances, patient organizations may be able to assist with identifying patient perspectives. Partnering with a patient organization ultimately helped the CAPS guideline development team recruit a patient representative for the Guideline Panel. When published qualitative studies are available, there is still little consensus in how to rate the quality of evidence. GRADE-CERQual , a GRADE project group, has proposed that an assessment of confidence for individual review findings from qualitative evidence syntheses be based on four components: methodological limitations of the incorporated studies, relevance of the studies to the review question, coherence of the review finding, and adequacy of data supporting the review finding . This isFinally, our study identified limitations to the use of expert-based evidence. Guideline developers have been advised to help Panel Members identify the evidence underlying their opinions, and judge the quality of that evidence . HoweverFurther research into how novel methods of evidence creation would support the guideline update process is also needed. For example, if a guideline for a rare disease relies heavily on evidence from a more common disease with a rapidly evolving knowledge base, the rare disease guideline may require more frequent updating through reviews of indirect evidence. Guideline developers must also be able to efficiently respond to changes in the knowledge base that may impact guideline recommendations, but can only be detected by ongoing qualitative research . Finally, it is clear that non-experimental and non-comparative forms of evidence can play a valuable role in guideline creation. Several groups, including AID (Appraising and Including Different Knowledge in Guideline Development) and GRADE-CERQual are actively exploring this concept.After theoretically exploring the barriers to issuing guidelines on rare diseases, we proposed an operational approach consistent with GRADE . We haveAll three rare disease guidelines in this study maintained a clear link between quality of evidence and the strength of the ultimate recommendations, as specified by the GRADE Working Group framework. Rigorous clinical practice guidelines are needed to improve the care of the millions of people worldwide who suffer from rare diseases. We recommend that systematic processes must be used to create evidence-based practice guidelines that are useful to patients, clinicians, researchers, industry, and policy makers in the rare disease community. However, these processes must also be dynamic and responsive to the unique challenge presented by rare diseases: a dearth of high certainty evidence. Innovative evidence elicitation and synthesis methods will benefit not only the rare disease community, but also individuals with common diseases who have rare presentations, suffer rare complications, or require nascent therapies. Further refinement and improved uptake of these innovative methods should lead to higher quality clinical practice guidelines in rare diseases.


Kunu is a traditional fermented single or mixed cereals-based beverage popularly consumed in many parts of West Africa. Presently, the bacterial community and mycotoxin contamination profiles during processing of various kunu formulations have never been comprehensively studied. This study, therefore, investigated the bacterial community and multi-mycotoxin dynamics during the processing of three kunu formulations using high-throughput sequence analysis of partial 16S rRNA gene (hypervariable V3-V4 region) and liquid chromatography tandem mass spectrometry (LC-MS/MS), respectively. A total of 2,303 operational taxonomic units (OTUs) were obtained across six processing stages in all three kunu formulations. Principal coordinate analysis biplots of the Bray-Curtis dissimilarity between bacterial communities revealed the combined influences of formulations and processing steps. Taxonomically, OTUs spanned 13 phyla and 486 genera. Firmicutes (phylum) dominated (relative abundance) most of the processing stages, while Proteobacteria dominated the rest of the stages. Lactobacillus (genus taxa level) dominated most processing stages and the final product (kunu) of two formulations, whereas Clostridium sensu stricto (cluster 1) dominated kunu of one formulation, constituting a novel observation. We further identified Acetobacter, Propionibacterium, Gluconacetobacter, and Gluconobacter previously not associated with kunu processing. Shared phylotypes between all communities were dominated by lactic acid bacteria including species of Lactobacillus, Lactococcus, Leuconostoc, Pediococcus, and Weissella. Other shared phylotypes included notable acetic acid bacteria and potential human enteric pathogens. Ten mycotoxins  were quantified at varying concentrations in ingredients for kunu processing. Except for AOH, AME, and BEAU that were retained at minimal levels of < 2 μg/kg in the final product, most mycotoxins in the ingredients were not detectable after processing. In particular, mycotoxin levels were substantially reduced by fermentation, although simple dilution and sieving also contributed to mycotoxin reduction. This study reinforces the perception of kunu as a rich source of bacteria with beneficial attributes to consumer health, and provides in-depth understanding of the microbiology of kunu processing, as well as information on mycotoxin contamination and reduction during this process. These findings may aid the development of starter culture technology for safe and quality kunu production. Fermented beverages constitute a major part of the diets of traditional African homes . In NigeKunu is a traditional beverage produced principally from single or mixed cereals such as maize, millet, rice, or sorghum. In some cases, peanut is added to the cereals to make-up the raw material input. The grain  input could also be supplemented with additives such as cloves, pepper, ginger, sweet potato and tiger nut, which are added as homogenized mixtures just before the fermentation of the beverage. The sets of cereal and nut applied to the production of kunu determine its variety. For example, kunu-zaki comprises of millet, sorghum or maize; kunu-tsamiya, millet, sorghum or rice; kunu-gyada, rice, peanut, millet or sorghum; and kunu-gayamba, solely millet  (kunu processing and the proportion of mixture of milled grains and additives to the gelatinized gruel (The production process of ain kunu . The pH  3–5.46) . The ste 3–5.46) . Similared gruel .kunu-zaki are driven by consortia of bacteria (mostly lactic acid bacteria) (Saccharomyces cerevisiae)  may help unravel hitherto unidentified bacterial species associated with kunu processing and kunu products   and a feevisiae)  that con variety , most ofiversity . Currentproducts . In addikunu due to the use of diverse cereals and nuts that have been reported to be prone to several mycotoxins in the beverage formulation  for consumer benefits.Chemical food contaminants  may, however, distort the safety and quality of mulation . Thus, td levels . Howeverxposures . Furtherxposures . Thus, tkunu formulations by using HTS-based technology, and to evaluate the effect of processing on the mycotoxin profiles at various stages of processing by using liquid chromatography tandem mass spectrometry (LC-MS/MS).Therefore, this study aimed to determine the bacterial community diversity and dynamics during the processing of three Samples of millet, sorghum (red and white varieties), ginger, peanut, sweet potato, tiger nut and cloves were purchased in January 2017 from the local market in Ilishan Remo , Ogun state, Nigeria. Only ingredients without visible insect infestation, discolorations and rot were used for this study.kunu designated as A, B, and C were prepared using the ingredients. Formulation A comprised of millet, white sorghum, peanut, cloves, ginger and tiger nut while formulation B was made from millet, white sorghum, cloves, ginger and sweet potato. Formulation C consisted of millet, red sorghum, cloves, ginger and tiger nut. In the formulations, cloves, ginger, sweet potato and tiger nut served as additives to millet, sorghum (white or red variety) and peanut, which were used as grain bases for the beverage. Precisely 1 kg of each grain and 100 g of each additive were used in the formulations. Maize was excluded from the formulations in order to eliminate extremely high levels of diverse mycotoxins from the beverage.Three different formulations of kunu processing, rudimentary utensils and tap water were used in order to replicate as much as possible the traditional processing method. The exact process undertaken for the preparation of the kunu formulations in this study is outlined in Figure For kunu were collected, while samples of raw grains, milled additives, cooled gruel (after gelatinization) and pomace (after sieving) were taken for mycotoxin analysis, and samples of the fermenting substrate were collected at 0, 6, and 12 h of fermentation for microbial analysis. Approximately 20 mL and 20 g subsamples of liquid and semi-/solid samples, respectively, were randomly collected for analysis. All samples were immediately frozen at -20°C prior to shipment on dry ice and analyses at the Agricultural Research Council–Biotechnology Platform, South Africa for HTS analysis, and the Center for Analytical Chemistry, IFA–Tulln, Austria for multi-mycotoxin analysis.Samples were collected from various core processing steps based on relevance of each step to either the bacterial community profile study or mycotoxin reduction analysis. For both  analyses, samples of steep liquor, milled grains and kunu, samples were centrifuged at 12,000 rpm for 5 min and DNA was extracted from 0.25 g (wet weight) of the sediment. DNA integrity and concentration were verified by agarose (1% w/v) gel electrophoresis and fluorometric quantification , respectively.Genomic DNA was extracted from samples using the Quick-DNA Fecal/Soil microbe kit  according to the manufacturer protocol. For steep liquor, 20 ml of samples was vacuum filtered through a sterile 0.2 μm pore size membrane filter  and DNA extracted directly from the membrane filter whereas for milled grains and fermenting gruels, 0.25 g (wet weight) of sample homogenate were used for DNA extraction. For Partial 16S rRNA gene (hypervariable V3-V4) libraries were amplified using universal bacterial primers 341F (forward) and 805R (reverse) . Each foDemultiplexed sequence reads were checked for quality using FastQC software  and subsequently trimmed of low-quality regions (10 bp) at both 5′- and 3′-ends using Trimommatic . PANDAse1 of the National Centre for Biotechnology Information under the bioproject accession number PRJNA482055.Raw sequence reads generated in this study are available in the Sequence Read Archiveskunu were analyzed for the presence of over 295 microbial metabolites including the major mycotoxins  by a liquid chromatography tandem mass spectrometric (LC-MS/MS) method described by kunu), mycotoxins were extracted from 2.5 mL of the samples in 15 mL polypropylene tubes containing 7.5 mL of extraction solvent and centrifuged at 10,000 rpm for 3 min at ambient temperature. All extracts were diluted with acetonitrile/water/acetic acid  solvent and injected into the LC system as described in detail by ®C18-column, 150 × 4.6 mm i.d., 5 mm particle size, equipped with a C18 4 × 3 mm i.d. security guard cartridge . Confirmation of positive analyte identification was obtained by the acquisition of two scheduled multiple reaction monitoring (MRMs) which yielded 4.0 identification points according to the European Commission decision 2002/657. In addition, the LC retention time and the intensity ratio of the two MRM transitions agreed with the related values of an authentic standard within 0.1 min and 30% rel., respectively. Apparent recoveries of the metabolites were determined by spiking 0.25 mL of five different kunu samples. The spiked samples were stored overnight at ambient temperature to establish equilibrium between the metabolites and samples. The extraction (in 1 mL of solvent), dilution and analysis were as described earlier. The accuracy of the method was crosschecked by participation in inter-laboratory comparison studies organized by BIPEA . Only mycotoxins that were positive in the samples are reported in the results section.Samples of grains, by-products (steep liquor and pomace) and kunu formulations after quality filtering and binning into OTUs. Normalization (rarefaction) of data to a depth of 13,920 sequences per sample was sufficient to estimate community diversity in all samples  Bray-Curtis dissimilarity PCoA biplot for the absence/presence and relative abundance of OTUs  Bray-Curtis dissimilarity principal coordinate analysis (PCoA) biplot for OTUs distribution revealed that the bacterial community structure of the processing stages of formulation A were the most diverse compared to other formulations Figure . Howevers Figure , the simB Figure . Within s Figure . Other ckunu formulations taxonomically spanned at least 13 phyla and 486 genera, with several of the OTUs being unclassified at these taxonomic ranks. At the phylum taxonomic level, Firmicutes dominated most of the processing stages of all kunu formulations  was found to dominate kunu of formulations C , Sphingobacterium, Acinetobacter, and Pseudomonas each constituted at least 5% relative abundance in at least one processing stage , Enterobacter, Lactobacillus, Lactococcus, Pediococcus, Streptococcus, and Weissella each constituted at least 5% relative abundance in at least one processing stage  with at least 1% relative abundance in any of the processing stages of each formulation are presented in Figure  Figures . In contC Figure . In forme Figure . In forme Figure . In forme Figure . Across e Figure –C and whkunu formulations. For formulation A, Pediococcus (67.5%) dominated the initial fermentation (0 h)  and Enterobacter (33.8%) dominated at 6 h and Lactobacillus (68.9%) at 12 h fermentation. In contrast, Weissella dominated the initial fermentation (97.3%) and mid-stream (6 h) fermentation (32.9%) in formulation B while Acinetobacter (12.8%), Sphingobacterium (14.6%), Pseudomonas (11.5%) and Enterobacter (8.7%) were the relatively more abundant phylotypes in the final fermentation (12 h) stage of the same formulation  Figure , while Cn Figure . On the n Figure .Lactobacillus spp. (17%). Other shared phylotypes included species of Enterobacter, Gluconacetobacter Lactobacillus, Lactococcus, Leuconostoc, Pediococcus, and Weissella.A total of 35 OTUs, comprising 16 phylotypes (at the genus taxonomic rank) were common to all processing stages and formulations Figure . These “kunu formulations (Table 1 (AFB1) were higher in red sorghum (113 μg/kg), peanut (51.5 μg/kg), and tiger nut (26.1 μg/kg) compared to millet (0.67 μg/kg), while almost similar mean concentrations of AFB2, 6.56 and 9.27 μg/kg, were quantified in peanut and red sorghum, respectively. AFM1 was detected in peanut and red sorghum at mean concentrations of 1.7 and 4.22 μg/kg, respectively, while citrinin (CIT) was only quantified in millet at mean concentration of 5.64 μg/kg. The concentrations of aflatoxicol, alternariol (AOH), alternariolmethylether (AME), and beauvericin (BEAU) in grains did not exceed 5 μg/kg. In contrast, the concentrations of 3-Nitropropionic acid (3-NPA) were very high in peanut (1121 μg/kg) and red sorghum (936 μg/kg). Moniliformin (MON) levels were higher in millet (68.7 μg/kg) and white sorghum (44.2 μg/kg) than in red sorghum (2.93 μg/kg) and peanut (1.42 μg/kg).A total of 10 mycotoxins were found in the ingredients used for processing the three ns Table . Aflatox10) transformation. The value “1” was first added to the mycotoxin concentration in a sample, and then the resulting value was transformed to give the data reported in Figure kunu  were quantified in the steep liquor. About 9% (0.43 μg/kg) of the BEAU in the ingredients (4.97 μg/kg) and 40% of CIT in the milled grains (4.32 μg/kg) were carried over into kunu  of the BEAU in the ingredient  was carried over to kunu, with the sieving step removing the remainder portion of the toxin  irrespective of formulations.This study reports, for the first time, on the application of HTS in the delineation of bacterial community structure during processing of kunu formulations suggests a rich genetic diversity of bacteria in the grains. Although potable tap water used for the steeping was not evaluated independently for microbial diversity, it is also likely that potable water contributed to the species diversity at the steeping stage. Furthermore, this observation of high species richness may be due to the stepping environment and duration serving as a broth and permitting the proliferation, respectively, of the autochthonous microbial species. Similar observations of diverse bacteria occurring during the steeping of raw grains for kunu-zaki production were suggested in previous studies  with the soil environment (The high number of OTUs (species richness) obtained during the steeping stage of the various  studies . The higironment .kunu processing. The lower OTU richness observed in the milled grains in comparison to OTU richness in steep liquor could be attributed to the fact that during the overnight steeping, bacterial species associated with the grains and steep water proliferate, giving rise to a fermented broth—the steep liquor . Since this fermented broth (or steep liquor) are decanted and discarded before subsequent washing of the steeped grains, a large number of species present in the steep liquor are not carried over to the milling stage. Furthermore, since the steep liquor is not used in subsequent processing step, the OTUs detected in the milled grains are likely those that are within the grains, OTUs not completely eliminated during washing of the steeped grains, and/or any OTUs introduced by chance contamination during milling of the grains. Hence, the milled grain is expected to contain significantly lower species than the steep liquor as observed in all formulations. Between the milled grains and onset of fermentation, a reduction in OTUs was observed. This relatively low OTU diversity observed at the 0 h fermentation time compared to the pre-gelatinization stage in all the kunu formulations is attributable to the hot water (100°C) treatment during the preceding gelatinization of the milled grains. Certainly, such heat treatment will have at least a bacteriostatic  effect on most of the indigenous microbiota, consequently contributing to reduced species diversity  of processing stages and/or different formulations suggest that, indeed, ingredients and processing steps influence microbial community composition and dynamics.In view of the afore discussions on OTU diversity, the trend in numbers of OTUs obtained from the mid-stream (6 h) fermentation stage up to the final product stage for the formulations suggests the proliferation of adapted (after heat treatment) functional species  and development of a climax/streamlined bacterial community observed in the kunu processing stages and in the final product. Also, similar to observations of several culture-dependent microbiological studies on kunu  dominated bacterial community of kunu product in formulation C. Although species of Clostridium have been previously reported in some kunu variety  (Clostridium sensu stricto (cluster 1) group with kunu as well as the dominance of the cluster. Until recently, the phylogenetic differentiation (and hence taxonomic classification) of some species within the Clostridium sensu stricto from other Clostridium species has been unclear, making their functional role in human diet and health, as well as in food fermentations unclear  group. Subsequently, studies may explore the functional and metabolic traits of these species in kunu as well as their suitability as starter cultures for kunu fermentation or other cereal-based fermented beverages. Similarly, Acetobacter, Gluconobacter, Gluconacetobacter and Propionibacterium genera were present during kunu processing, especially in the steeped grains. To the best of our knowledge these species have not been previously reported in the microbiology of kunu processing. Acetobacter, Gluconobacter and Gluconacetobacter are acetic acid fermenters that have relevant industrial usefulness due to their ability to convert several sugars and alcohols into industrially important organic acids   , this is unclear . A few snnotated . In factnnotated  and potennotated . Based ovinegar) . Acetobased beer , while Gic beers . Conversproducts , with soproducts .kunu formulations elucidate functional roles of the dominant species at specific time points as well as their adaptation to the prevailing fermentation conditions. The observation of a streamlined and relatively dominant community of lactic acid bacteria, including species of Lactobacillus, Lactococcus, Leuconostoc, Pediococcus, Weissella and possibly species within the Clostridium sensu stricto clusters, toward the end of fermentation and in the final kunu product is related to their adaptation to the acidity pH of kunu fermentation which is expected as they are the drivers of the biochemical reactions and generators of lactic acid during the process  from Taiwan   in this traditional beverage. As earlier mentioned, Gluconacetobacter is an acetic acid bacterium; however, its role in food fermentation is not yet completely understood. It has been associated with traditional balsamic vinegar  phylotypes may constitute the core bacterial diversity of  in kunu , lactic nd akasa , cocoa fnd akasa , Spanishnd akasa  and suanm Taiwan . The role sugar)  and geneeriocins  that heleriocins . The posbic beer  and inyuan sauce . Also, srehalose . Howevern humans . Certain vinegar , howeverdic beer . Nonethetic acid .kunu were contaminated with different concentrations of several mycotoxins, albeit at levels that were lower than typical levels in maize batches applied to production of traditional fermented beverages in SSA  in the final product of the various formulations. The findings of this study agree with previous reports that suggested that traditional processing significantly reduced the levels of mycotoxins in fermented foods and the extent of reduction depends on the mycotoxin content of grain inputs  and sesame seed) that may be less prone to mycotoxins. The extract from tiger nut may be more useful compared to the whole tiger nut; this will further reduce mycotoxin exposure while enhancing nutritional content and imparting additional flavor. Furthermore, it is imperative to source high quality grains and apply simple first-line grain processing interventions  to keep mycotoxin levels in the starting materials at the barest minimum in order to encourage a further mycotoxin reduction to non-detectable/safe limits in the final kunu product. There is a need to explore the core phylotypes reported in this study by isolating and ascertaining their probiotic properties and suitability as starters for up-scaling of some of the cereal-based traditional fermented foods. This study has further laid the foundation for the potential discovery of novel mycotoxin detoxifiers from kunu. Overall, the data provided herein are highly relevant to experts in the food microbiology and safety, food processing and technology, microbial ecology and molecular biology horizons.In summary, our study indicates that the processing of CNE conceived the study. CNE, MS, OO, RA, CN, and RK designed the study. IC-O, KA, OO, OA, OE, CNE, and MS performed the experiments in Nigeria, South Africa, and Austria. MS, OE, DvW, CNE, and KA analyzed the data. CNE, CN, RA, JH, CTE, and RK supervised the overall study. KA, OE, DvW, and CNE drafted the manuscript. All authors reviewed and approved the manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Buruli ulcer can cause disfigurement and long-term loss of function. It is underdiagnosed and under-reported, and its current distribution is unclear. We aimed to synthesise and evaluate data on Buruli ulcer prevalence and distribution.Mycobacterium ulcerans detection, with no limits on study type, publication date, participant population, or location. English, French, and Spanish language publications were included. We included population-based surveys presenting Buruli ulcer prevalence estimates, or data that allowed prevalence to be estimated, in the systematic review. We extracted geographical data on the occurrence of Buruli ulcer cases and M ulcerans detection from studies of any type for the evidence consensus framework; articles that did not report original data were excluded. For the main analysis, we extracted prevalence estimates from included surveys and calculated 95% CIs using Byar’s method. We included occurrence records, reports to WHO and the Global Infectious Diseases and Epidemiology Network, and surveillance data from Buruli ulcer control programmes in the evidence consensus framework to grade the strength of evidence for Buruli ulcer endemicity. This study is registered with PROSPERO, number CRD42018116260.We did a systematic review of Buruli ulcer prevalence and used an evidence consensus framework to describe and evaluate evidence for Buruli ulcer distribution worldwide. We searched PubMed and Web of Science databases from inception to Aug 6, 2018, for records of Buruli ulcer and 2763 titles met the search criteria. We extracted prevalence estimates from ten studies and occurrence data from 208 studies and five unpublished surveillance datasets. Prevalence estimates within study areas ranged from 3·2 (95% CI 3·1–3·3) cases per 10000 population in Côte d’Ivoire to 26·9 (23·5–30·7) cases per 10000 population in Benin. There was evidence of Buruli ulcer in 32 countries and consensus on presence in 12.The global distribution of Buruli ulcer is uncertain and potentially wider than currently recognised. Our findings represent the strongest available evidence on Buruli ulcer distribution so far and have many potential applications, from directing surveillance activities to informing burden estimates.AIM Initiative. Mycobacterium ulcerans. This disease primarily occurs in west and central Africa, but also in parts of Asia, South America, the western Pacific, and Australasia.2 It is considered an important public health problem because of the characteristic necrotic ulcers it causes, and the scarring and deformity that can persist after treatment.3 Although the mode of transmission of M ulcerans is not fully understood, contact with slow-flowing, stagnant, or disturbed water bodies is an important risk factor.4Buruli ulcer is a neglected tropical disease caused by the environmental pathogen 4 but there is no consensus on its current distribution. Ten countries reported a total of 1864 cases to WHO in 2016,1 but this number is recognised to reflect a small proportion of the total burden. Cross-sectional surveys in endemic countries have demonstrated under-reporting of Buruli ulcer,7 for reasons including the chronic, stigmatising nature of the disease, its rural distribution, patients’ poor access to health care or preference for traditional healers, and lack of awareness or resources within health systems.8 Misdiagnosis might also contribute to underdetection: Buruli ulcer has a range of non-specific presentations that can be confused with other skin conditions, especially in the absence of confirmatory tests.10 Therefore, available data do not provide a full or accurate representation of Buruli ulcer burden and distribution. These measures are essential for targeting of active case detection, which is a key part of control,3 and for directing resources for case management.Buruli ulcer was reported in 34 countries between 1960 and 2015,M ulcerans, and to systematically review population-based studies reporting the prevalence of Buruli ulcer to provide a descriptive analysis of Buruli ulcer epidemiology in known endemic areas. We aimed to use an evidence consensus approach12 to delineate the overall distribution of previously reported cases and to quantify the strength of evidence for Buruli ulcer presence or absence in every country worldwide.Estimation of the global burden and population at risk of Buruli ulcer requires detailed information on the geographical limits and prevalence of the disease. We aimed to synthesise available data on prevalence and occurrence of Buruli ulcer and environmental occurrence of 1 data reported through the Global Infectious Diseases and Epidemiology Network (GIDEON);13 and surveillance datasets from national Buruli ulcer programmes in Cameroon, Ghana, Nigeria, and Togo. Peer-reviewed literature was identified from searches of PubMed and Web of Science databases from inception to Aug 6, 2018. Additional publications were identified from reference lists of identified papers.We did a systematic review of Buruli ulcer prevalence and used an evidence consensus framework to describe and evaluate evidence for Buruli ulcer distribution worldwide. Data sources included peer-reviewed scientific literature; conference proceedings, conference abstracts, and government reports (grey literature); data reported to WHO between 2007 and 2016;M ulcerans infection in humans or animals, or detection of M ulcerans in animal and environmental samples. Articles that did not report original data were excluded.We used the search terms “Buruli ulcer*” OR (“Mycob* AND ulcer*”) OR “Bairnsdale ulcer”. There were no limits on publication date, participant population, study type, or location  screened titles to exclude non-relevant publications and screened abstracts of selected records to identify papers that apparently fulfilled selection criteria. We read full texts of selected articles to identify studies meeting the selection criteria. Studies that recruited patients from health facilities or used strains of 14 The original spreadsheet was piloted on a subset of studies and then developed. Authors were contacted for additional data if community-level results were not presented. Data extraction was done by a single author (HS) and checked by a second one (JC). Data extracted included the number or prevalence of cases; the sample size and survey coverage (for population-based studies); the case detection method ; the recording date; the diagnostic procedure, including any confirmatory tests , and their results; and the location of origin (patient residence or endemic area visited if the case originated from a non-endemic area). Areas described as endemic, with no information on case detection, were not included.Data from surveillance datasets and selected publications were extracted into a bespoke Microsoft Excel spreadsheet used for the Global Atlas of Helminth Infections.M ulcerans included sample date and location; sample type ; taxonomic details for animal samples; confirmatory tests; and number of samples tested and number positive.Data extracted on environmental detection of Database of Global Administrative Areas.Geographical coordinates of occurrence locations were extracted if they were provided in the publication. Otherwise, point locations were georeferenced remotely . Point l15 adapted from a systematic review of podoconiosis prevalence16 , medium ($100–$499), or high (≥$500), following the approach of previous authors and supported by evidence that higher health expenditure is associated with better health system performance.20The major features for the national evidence framework were health reporting organisations ; occurrence data quality ; number of cases ; and evidence for absence. In countries with no cases reported, the consensus score was designed to quantify the evidence for Buruli ulcer absence, reflecting the possibility of under-reporting due to weak surveillance capacity or misdiagnosis as known endemic diseases with similar presentations21 leprosy,22 lymphatic filariasis,14 onchocerciasis,23 tropical ulcer,2 and yaws.24 Estimates of the frequencies of the common presentations of these diseases and Buruli ulcer were obtained from literature review and expert opinion .27 For each confounding disease, the frequency of each presentation shared with Buruli ulcer was multiplied by the frequency of the presentation among Buruli ulcer cases, and the products were summed to generate a symptom overlap score  and five unpublished surveillance datasets.The literature search identified 2763 records after deduplication . Another5 Seven surveys provided explicit details on the sampling frame. All surveys were community based and aimed to reach the entire population of chosen communities. Seven surveys covered the entire study area, one surveyed randomly selected communities within the study area, one surveyed a convenience sample of communities, and one used random and convenience sampling. Only one reported the survey coverage.8 Five reported laboratory confirmation of all or a subset of cases, and five used clinical case definitions. Only one study reported prevalence with 95% CIs.8Three surveys done in Cameroon, two in each of Benin, Côte d’Ivoire, and Ghana, and one in the Democratic Republic of the Congo were included . The lar32Overall prevalence estimates within the study area ranged from 3·2 (95% CI 3·1–3·3) cases per 10 000 population in Côte d’Ivoire to 26·9 (23·5–30·7) cases per 10 000 in Benin . The hig38 33 794 (94·9%) of 35 595 cases were from the African (AFRO) region, 1740 (4·9%) cases were from the Western Pacific (WPRO) region, 60 (0·2%) were from the American (AMRO) region, and one (<0·1%) was from the Eastern Mediterranean (EMRO) region. Evidence of M ulcerans in environmental and animal samples was reported from nine countries. A summary of data extracted from all publications is provided in the Human cases were recorded from 32 countries and inferred for two further countries  from which strains were reported to have been isolated.Laboratory confirmation of at least one case was reported by 134 (70·5%) of 190 selected studies including data on human cases, and 116 (61·1%) used PCR. However, most occurrence records (3165 [53·0%] of 5970) were categorised as clinically diagnosed only, because laboratory results were not disaggregated by unique locations.33 Buruli ulcer was considered less likely to be misdiagnosed as cutaneous leishmaniasis or yaws, which present a lower frequency of ulcerous forms.26 Onchocerciasis, leprosy, and lymphatic filariasis had symptom overlap scores of less than 6%.Symptom overlap scores for the confounding diseases are shown in Full results of the evidence consensus framework are provided at country level in the The African countries with evidence of Buruli ulcer were mostly clustered in a block covering much of central and west Africa. Countries around this block generally had weaker evidence for absence, with a higher number of endemic confounding diseases and lower health expenditure than did countries further from endemic areas. In the AMRO region, evidence of Buruli ulcer was strong in French Guiana and Peru, and moderate in Brazil, Mexico, and Suriname. Despite strong evidence of Buruli ulcer cases from French Guiana in literature reports, the disease has never been reported to WHO, so full consensus on endemicity was not reached through the framework. There was moderate evidence for Buruli ulcer in China. Endemicity status was indeterminate in Burkina Faso, Ethiopia, Honduras, Indonesia, Malawi, Malaysia, and Suriname. Niger, Eritrea, The Gambia, and Mauritania, all in the AFRO region, had the weakest evidence for absence, being endemic for cutaneous leishmaniasis and tropical ulcer, and having low healthcare expenditure.Subnational areas with evidence for endemicity were mostly clustered within equatorial, humid tropical, and tropical climate zones of west and central Africa . Areas wM ulcerans in animal and environmental samples are shown in M ulcerans DNA has been detected in faecal samples from animals in Australia (details and references in The areas with evidence of M ulcerans in animals and the environment. The evidence consensus framework applied has allowed us to expand on existing maps of Buruli ulcer distribution39 in several ways. The maps presented include evidence from a wider range of sources, provide finer resolution, and quantify the strength of evidence for Buruli ulcer presence, as well as the strength of evidence of absence where Buruli ulcer has not been reported.We have collated available data on Buruli ulcer prevalence and occurrence, and evidence of There have been few Buruli ulcer prevalence surveys, and most of those identified did not report detailed statistical analysis or indicators such as coverage. We did not undertake a meta-analysis because of the heterogeneous nature of compiled studies. Furthermore, most studies included were done in areas assumed to have a high local prevalence of Buruli ulcer, so a summary prevalence would probably overestimate the disease burden in the overall population.33 used convenience sampling as part of the study design, which implies a risk of bias in the estimated prevalence. Five studies reported clinical diagnosis according to WHO guidelines and five used laboratory testing to confirm all or a subset of cases. There was geographical bias across the studies included, representing only five countries of the 32 identified as having evidence for Buruli ulcer.Prevalence estimates reported by population-based studies were high relative to incidence data reported to WHO. This difference is likely to reflect under-reporting of Buruli ulcer through routine systems, but the population-based studies included might have overestimated Buruli ulcer prevalence as a result of sampling bias. Two of the ten studies includedOur investigation identified consensus on Buruli ulcer presence in 12 of 18 countries that reported Buruli ulcer cases to WHO from 2007 to 2016. However, the maps presented demonstrate remaining uncertainty on the global distribution of Buruli ulcer. There was indeterminate or moderate-quality evidence of Buruli ulcer in 15 countries that had not reported data to WHO from 2007 to 2016.The national and subnational evidence consensus maps demonstrate large contiguous areas of potential endemicity, both within and between countries, particularly in central and west Africa. Evidence for Buruli ulcer presence was generally strongest in these contiguous areas, which is likely to be partly due to environmental similarity in terms of suitability and partly due to increased emphasis on case detection in areas established as endemic.40 and the lower availability of data at the subnational level: in some countries, the only available data were those reported to WHO, with no information on subnational distribution. Given the recognised scale of Buruli ulcer under-reporting, it is likely that this map underestimates the scale of Buruli ulcer distribution.The area of Buruli ulcer presence defined by the subnational map of Buruli ulcer distribution in Africa  was moreCountries that had not reported Buruli ulcer cases, but were close to those that had, generally had weaker evidence for absence than countries located further from areas of Buruli ulcer endemicity. This trend was apparent in Africa, South America, and the southeast Asia and western Pacific regions, and reflects spatial clustering of countries with lower health expenditure and numerous co-endemic tropical diseases, irrespective of their evidence for Buruli ulcer. The proximity of Buruli ulcerendemic countries to those with lowest evidence for Buruli ulcer absence adds further weight to the possibility that Buruli ulcer might occur undetected in the latter group, as a result of cross-border transmission and environmental similarity of neighbouring countries.40 point-level data on disease occurrence are needed to support investigation into its spatial epidemiology. It is hoped that the maps and assembled geographical dataset will support such research in the future.Although the maps provide finer detail on the distribution of Buruli ulcer than do current official maps, they still mask the underlying epidemiology of Buruli ulcer. Areas identified as endemic might in fact contain only a few localised cases of Buruli ulcer and be mostly unsuitable for the disease. Because of the focal nature of Buruli ulcer,M ulcerans were limited in number, and many did not apply sufficiently specific tests to differentiate M ulcerans from other environmental mycobacteria. Therefore, the maps of evidence for environmental occurrence of M ulcerans do not provide a complete representation of environmental suitability for the bacterium. Although we assigned the maximum possible evidence quality score to clinical cases confirmed by PCR and environmental occurrences confirmed by quantitative PCR, these tests still entail a risk of false positives, as demonstrated by an external quality assessment including several reference laboratories that performed confirmatory testing in studies we included.41Studies on environmental occurrence of There was substantial geographical bias in the occurrence records, reflecting different levels of research and surveillance activity between countries. Further analysis of the data underlying this work should account for this bias. In the context of this study, this bias is expected to have affected areas where there were few studies, but not areas where there were many studies, since additional studies would not change the outcome measure unless they provided higher-quality data.39 demonstrating that the disease is likely to be more widely distributed than the official map suggests. This finding has important implications for understanding and communicating the global burden of Buruli ulcer. We have also expanded on the WHO map of Buruli ulcer distribution by qualitatively grading the strength of evidence for endemicity. In doing so, we have identified numerous countries with moderate or indeterminate evidence of Buruli ulcer, and those with weakest evidence for its absence, which might require further investigation to clarify the global distribution of Buruli ulcer. Active case finding in areas that have previously reported Buruli ulcer, and close to those currently reporting, should be prioritised. The assembled point-level dataset represents a novel resource for continent-wide exploration of environmental and biological predictors of Buruli ulcer, and estimation of the global burden and population at risk. The information provided by investigations such as these will help to target future control efforts and evaluate their impact.The areas with highest consensus for presence are presumably most suitable for Buruli ulcer transmission and would be targets for surveillance and research since they represent known disease foci. Some countries with strong evidence for Buruli ulcer are not shown in the current WHO map of Buruli ulcer,Supplementary appendix 1