How ‘evidence’ is conceptualised, generated and deployed in meso-level policy implementation on the ground is critical to health delivery. Using the case of a large-scale health service reconfiguration in northwest England, this study began as a narrative investigation into how different data types and sources are prioritised as NHS administrative structures change over time. During the research, one unpopular reconfiguration decision, the downgrading of a hospital, was challenged using judicial review. Suddenly, a key decision was being based not upon ‘facts and data’ type evidence but upon evidence of adherence to administrative procedure. This transferred focus away from the ever-shifting categories and hierarchies of data ‘types’ towards an emphasis on process. By comparing two deliberative contexts—committee and judicial review—this article proposes that evidence can be understood as simultaneously entity and process. As health service reconfigurations continue in response to austerity, integration agendas, evolving organisational landscapes, and demographic and political change, it is increasingly important to recognise the different meanings and uses of evidence. Evidence, a master-term, has quite different meanings across contexts. This article will establish the crucial importance of ‘evidence’ terminology. It rehearses the key argument, that evidence takes divergent forms, using a case study of health service reconfiguration in Greater Manchester, UK, culminating in a judicial review (JR). The background and political context of the case study is described, followed by a detailed analysis. The article concludes with a warning, that in the context of EBP, we must exercise caution in our use of evidence terminology, which carries various political-contextual baggage. A more sophisticated understanding can inform policymakers, especially around long-term planning, and researchers examining how different forms of evidence are assembled to yield different policy outcomes.The primacy of evidence in underpinning and justifying public policy decisions is broadly accepted as self-explanatory . As a large and well-loved public institution, how the NHS performs and is organised is a matter of great public interest. This section describes two significant political factors affecting the NHS: (a) the UK government’s austerity programme and (b) the devolution of local health decisions in Greater Manchester.Following the 2007/2008 global financial crash, the UK’s Conservative-Liberal Democrat coalition government initiated a programme of financial austerity in 2010. This involved reductions in public spending and selective tax increases.The widening funding gap, exacerbated by the UK’s growing and ageing population, resulted in the NHS having to make ‘efficiency savings’ of potentially up to £50bn by 2021/2022  and became responsible for the overall strategic direction of health and social care in Greater Manchester. This was seen as an opportunity to restructure services more in line with local needs and to pursue the NHS integration agenda as part of the broader narrative of health service reform. The present study began by examining the changing nature of evidence types used in various reconfigurations since 2005. During the research, one controversial decision, the downgrading of a hospital, was challenged using judicial review. This raised a different set of questions about the nature of evidence and its use, and became the new focus of the study.It is possible to describe a timeline of how different evidence types have been prioritised according to NHS organisational changes and wider political or economic drivers. The mid-2000s ‘Healthy Futures’ and ‘Making it Better’ reconfigurations across Greater Manchester were driven by four main arguments: a clinical case for the merits of concentrating services; a policy case—compliance with the European Working Time Directive; epidemiological evidence based on population projections; and a public consultation. These were approved by a joint committee of Primary Care Trusts . However, the evidence was strongly criticised and challenged by Ruane , who disSupplanting PCTs, Clinical Commissioning Groups (CCGs) were established following the Health and Social Care Act (2012), enabling clinicians with better knowledge of local needs to take meso-level decisions around resource allocation and service configurations. GP Clinical Directors were required to run their practices as businesses, so began to take more corporate approaches, giving increased prominence to accounting evidence. A more business-focused NHS emerged into an austerity context; configuration decisions increasingly hinged on ‘doing more with less’. This is reflected in the NHS’ gradual move away from employing CIPFA (Chartered Institute of Public Finance and Accountancy) accountants, who pursue more traditional ‘book balancing’ approaches, towards CIMA (Chartered Institute of Management Accountants) accountants, who pursue more strategic and profit-oriented methods. There is also a trust issue; medics are perceived as ‘heroic’ (in comparison to accountants or politicians), so the idea of CCGs shaping local health economies carries a great deal of intuitive value in convincing members of the public in consultation. However, clinician-originated evidence may also be aimed at an internal audience, who want to be reassured they are making the right decisions Byrne , p. 109.It is clear (to me at least) that the NHS is moving away from the legal framework embedded in the 2012 Health and Social Care Act which if my memory doesn’t fail me was based on competition between providers, guided by local commissioning led by local GP’s, and away from “political” decision making to one driven by market forces, subject to economic and quality regulation… Commissioning has moved (or is moving…) from a local base to one framed by the Sustainability and Transformation Partnership footprint Steer .These structural and perceptual changes corresponded with the launch of Healthier Together, which was more explicitly finance-oriented. The overarching Devo Health strategy was required to save £2bn over a five-year period so the austerity mantra of ‘do more with less’ therefore provided a strong justification for service integration. Financial reasoning continues to determine NHS organisational structure. For example, ‘Sustainability and Transformation Partnerships’ (from which Greater Manchester is currently exempt), introduced in 2017, have been denounced as being explicitly financially motivated.‘Healthier Together’, launched February 2012, sought to rationalise healthcare provision across Greater Manchester by moving treatment out of hospitals and into community settings, enabling the former to provide better services and the latter to benefit from ‘joined up care’ between local councils and the NHS , whose stated remit is to progress the integration agenda by ‘finding what works on a local level, and… responding to what people need across all ten boroughs’ . Having In this administrative tangle, different evidence types have different values. Healthier Together strongly emphasised being clinically-led and responsive to a public consultation. Its decision-making was informed by both clinical and consultation evidence.The Healthy Futures and Making it Better reconfigurations; the introduction of CCGs; Healthier Together and Devo Health all used different forms of entity evidence, determined by the prevailing NHS organisational structure at the time, to justify service reconfiguration decisions. While this is unsurprising, evidence can also be used in a different way…It was determined in March 2015 ‘that there must be a “specialist” hospital in the southern sector’ .All hospitals will improve and past does not indicate future performance Clinicians asserted that the CiC’s decision-making had ignored evidence from other hospitals’ past performances. Mr Justice Dove cited the ‘Decision Making Management Report’ (DMMR), which claimed ‘as current quality and safety standards will be improved in all sites [their emphasis], this should not be used as a criteria to determine the number of single services’ . The claimant also cited ‘Hospital Episode Statistic’ data, a historic record of the relationship between travel times and clinical outcomes. The defendants reiterated that historical data provides no basis for future predictions, as it is limited to the ‘moment in time’ the data was extracted. Accepting the simulation evidence, irrespective of the new road, over historical clinical data indicates a judicial decision that respects, but is not in thrall to, clinical evidence.Co-location refers to the planning of systems and pathways that cater for a range of conditions which rely on multiple interdependent services being available in the same place. Changing the availability of some services can affect other services outside the scope of the reconfiguration. It was claimed that the cardiovascular, vascular, burns and Cystic Fibrosis care provided at Wythenshawe would be potentially harmed by the loss of emergency and high-risk surgery on which they were co-dependent. However, a ‘post-consultation co-dependencies review’ .In (1) and (2) the judge deemed past performance clinical outcome data and computer-modelled projection data to be not necessary for rational decision-making; and in (3) he validated the CiC’s reading of co-location data as rational in its aim to reach a decision. Critically, the judge was not able, nor required, to appraise the data itself; his role was to determine that there was a rational process at work. The key words in these arguments are: ‘rational’  and ‘carried out correctly’ (in the case of co-dependencies). These are verdicts on process, as opposed to specific entity evidence . The judge based his final decision upon how specific pieces of evidence were handled, rather than on a simple hierarchy of evidence types.Clinicians from Wythenshawe Hospital mounted a legal challenge against the decision through judicial review (JR), a process of the administrative court that enables an individual or group to challenge decisions made by a public body. JR examines whether the process by which the original decision was made (a) followed the law and (b) was ‘rational’ . The rationality of a committee is to some extent determined by its wider organisational context but also by its adherence to ‘logic’ and ‘accepted moral standards’  expert in law and administrative procedure.The judge’s report, in its description, deliberation and treatment of evidence from both sides, and its focus on providing an independent rationale, is arguably more transparent than many of the documents produced by Healthier Together and the wider Devo Health project. Although a judge lacks expertise in the complexities of health service configuration, the decision being made in a judicial review is different in the following ways:This change in the treatment of evidence marks a significant turning point for the democratic deliberation of public policy. Despite being a well-established resource, designed to ensure the separation of powers in modern governance, some believe JR is becoming too heavily relied upon. Dyson , pp. 2–3nature of evidence—as entity, process, or some dual version of both—the developments in Greater Manchester suggest that it is best understood precisely in terms of the interaction between both in context. The positivist style of insistence that evidence is something known which should guide action, a key principle of those who argue for hierarchies of evidence with the most scientific (RCTs) having the most weight, never describes how evidence is actually deployed in any real social context. This is probably true of many apparently clinical deployments but it is necessarily true of any policy formation and implementation process that involves the interaction, or interweaving, of complex health, social care, political, and urban systems within a dynamic fiscal context in which resources are massively constrained. In their critique of an Institute for Public Policy Research (IPPR) report that purported to present evidence for hospital reconfiguration, Byrne and Ruane  and organised around a public consultation , whereas Devo Health involved a large group from a range of public services (a team of experts). Both processes relied on cooperation at various levels and on the deployment, weighing and organising of different types of entity evidence. Using a variety of heterogeneous evidence types to calculate the best course of action from a range of options leads to more robust conclusions  perspective reflects a view that those making legal decisions should pursue value-free approaches, and that the law does and should serve social ends, so legal decisions consider fairness and public policy—the Tamanaha . It has Tamanaha , p. 147.process through which facts and data are put—the basis for judicial reviews. The idea that evidence might exist simultaneously as both entity and process makes a useful contribution to discussions around evidence-based policy. This simultaneity is important; a more dynamic and process-oriented understanding of evidence improves on the somewhat more static concept of categorical and hierarchical evidence ‘types’, which are ‘profoundly unpragmatic’ ; it can also be understood as the c’ Byrne , p. 46. and as a process determining an outcome. While the distinct disciplinary forms of facts/data-type evidence move in and out of alignment with the shifting priorities of different NHS structures, the JR focused solely on the logic and administrative process of the original committee decision. The shift from using ‘disciplinary-based’ evidence, with fluctuating values linked to a prevailing culture, to being an instrumental tool in the service of a ‘higher’ decision, reflects this shift in focus from object to process. As judicial reviews become an ever more popular way to contest unpopular administrative decisions, the idea of evidence as a dynamic complex becomes increasingly relevant.The use of JR in health service configuration decisions mobilizes a different set of background understandings around evidence, in which evidence must be considered as both an entity This more sophisticated understanding can contribute to the thinking underpinning health service reconfigurations and other EBP-related decisions. First, we might consider those deploying evidence in committee-style decision-making contexts as skilled builders, assembling institutional-policy life from different levels and forms of evidence. Accounting for the wider context, the process of admitting and considering different evidence types, might enable decisions that are more resilient to organisational and contextual vagaries. At the very least, knowing that the administrative process itself might become evidence in a final decision should inform strategies at the committee stage. Second, for those studying related topics, such as knowledge aggregation, better recognising the relationship between facts and the process they are entered into might enable new insights in the more abstract debates around EBP. Either way, the core message remains: evidence terminology is slippery and more nuanced understandings of it would significantly benefit complex decision-making, especially in health and broader public policy.


The global COVID-19 pandemic has put everyone in an urgent need of accessing and comprehending health information online. Meanwhile, there has been vast amount of information/misinformation/disinformation generated over the Internet, particularly social media platforms, resulting in an infodemic. This public health crisis of COVID-19 pandemic has put each individual and the entire society in a test: what is the level of eHealth literacy is needed to seek accurate health information from online resources and to combat infodemic during a pandemic? This article aims to summarize the significances and challenges of improving eHealth literacy in both communicable  and non-communicable diseases . Also, this article will make our recommendations of a general framework of AI-based approaches to improving eHealth literacy and combating infodemic, including AI-augmented lifelong learning, AI-assisted translation, simplification, and summarization, and AI-based content filtering. This general framework of AI-based approaches to improving eHealth literacy and combating infodemic has the general advantage of matching the right online health information to the right people. Since its conceptualization in Norman et al.  in 2006,On February 15, 2020, the WHO formally raised the concern that the COVID-19 epidemic was accompanied by an infodemic , which iFrom an individual's eHealth literacy perspective, it is equally important to accurately comprehend correct health information and to disregard health misinformation/disinformation. However, recent research studies have revealed concerns on the eHealth literacy in various populations and called for major efforts in improving eHealth literacy. It was stated in Paakkari and Okan  that theThe eHealth literacy has a relatively short, but active research historyCancer healthcare is a very complex and demanding task that warrants close collaboration between cancer healthcare professionals and patients and their family members. In particular, the levels of eHealth literacy of patients and their family members play major roles in cancer diagnosis, treatment, and follow-up. There have been a vastly amount of research studies on the importance of eHealth literacy in cancer healthcare, as reviewed in a recent article . As a reWith the increased longevity and improved health at older ages worldwide, healthcare of the elder population and dementia, particularly Alzheimer's disease, presents significant challenges. It was pointed out in Efthymiou et al.  that 80%Patients with CVD are required to self-manage many aspects of their conditions, which requires eHealth literacy for various tasks. Richtering et al.  examinedIn short, due to complexity, heterogeneity, uncertainty, and chronic nature of diseases like cancer, Alzheimer's, and CVDs, patients and their caregivers need considerable level of eHealth literacy to seek, find, understand, and appraise health information from electronic sources and apply the gained knowledge to addressing or solving specific health problems.For instances, Hong et al.  examinedNotably, there are differences in the challenges and significances of eHealth literacy in communicable  and non-communicable  diseases. For instance, the levels of urgence in seeking healthcare-related information and patients' demographic characteristics and social determinants could be different in communicable and non-communicable diseases. Thus, our recommendations in the following section will consider such differences accordingly.In general, eHealth literacy means very differently to individuals with diverse backgrounds in traditional literacy, media literacy, information literacy, computer literacy, scientific literacy, and health literacy. Therefore, a key consideration when trying to improve eHealth literacy is to tailor general approaches to individual groups and scenarios. Here, we recommend a general framework of AI-based approaches originated from the methodologies in AI field, and these AI-based approaches have the general advantage of matching the right online health information to the right people.An individual person's eHealth literacy learning is a complex, lifelong process, which entails applying scientific insights about how people learn, leveraging emerging technologies such as social media and mobile devices, designing and using transformative learning activities, engaging teachers and other practitioners, and emphasizing continuous improvement. All of these components are integral parts of the cyberlearning, which is recently used by the NSF cyberlearning programNotably, mashup technologies have been advanced significantly in recent years because of the maturity of the service-oriented computing paradigm and service-oriented software development , 49. ForAlso, AI-enabled intelligent tutoring systems (ITS) of health information could be considered to support and guide individuals as they learn relevant health information and knowledge from online resources. As reviewed in Mousavinasab et al. , uses ofOverall, AI-augmented lifelong cyberlearning diagrams and approaches have great potential to transform current landscape of eHealth literacy education and significantly advance the eHealth literacy of the general public, including both healthcare professionals and average individuals. It is without doubt that significant efforts are needed from multidisciplinary domains, including governments, healthcare professionals, educators, computer and information systems, and AI experts, to move this field forward collaboratively and synergistically. It is equally important that individual citizens recognize and appreciate the importance of eHealth literacy for both personal life and public health, and accept the idea that eHealth literacy is a fundamental skill, like writing and reading, as advocated in literature articles . In thisIn governments' guidelines and checklists regarding online eHealth literacy education materialsIn the NLP AI field, another important relevant technology is text simplification, which aims to rewrite an input text so that the output is more readable and understandable . SimplifIn a recent work, Van et al.  pointed As pointed out in the wedding cake model of health information , knowledIn a recent study, Bala et al.  showed tImproving eHealth literacy for the whole society, e.g., via the approaches discussed in Sections AI-Augmented Lifelong Cyberlearning and AI-assisted Language Translation, Simplification, and Summarization, is an effective way to increase the efficiency of our healthcare systems, to reduce healthcare disparity, and to combat infodemic. Meanwhile, exploration of quicker approaches to combating infodemic, e.g., the one during the COVID-19 pandemic , 7, is uRecently, the topic of detecting fake news about COVID-19 on the Web has also received intense interest, partly due to the fact that many people still prefer to use search engines to find needed information on the Web . During In addition to the abovementioned advancements in information filtering technology on social networks and Web, it might be still important to have a mechanism to certify whether a health information source is reliable or not. For instance, the Health On the Net (HON) projectApparently, other than those 8,000+ NONcode certified websites, there are many more Websites and information sources that provide health information to the general public. From a technical perspective, it is a daunting task to monitor, evaluate, and filter the reliability and usefulness of those health information sources. Again, it will be a fundamental task to improve the general public's eHealth literacy so that each individual will be able to assess the level of reliability and usefulness of such health information sources, and AI-assisted language translation, simplification, and summarization and AI-based content filter can give them a hand whenever needed. In addition to AI-based content filtering, building a transparent, consistent, and trustworthy AI ecosystem is critical and should be considered. In short, our overall recommendations for improving eHealth literacy and combating infodemic include a combination of these three directions of efforts.Our recommendations made in Section Recommendations are from a technical perspective of AI approaches. Here, we will discuss social and ethical issues related to eHealth literacy and infodemic, including the possibility of integrating eHealth literacy education into the curriculum at all levels, a whole-society approach to eHealth literacy, and healthcare disparities. Finally, we will make concluding remarks on these topics in general.Existing literature has already argued that eHealth literacy is a fundamental skill for any individual, which should be trained and educated across the life span, including K-12, college, and the entire adulthood. For example, the CDC has its website for health literacy education for childcare, early childhood, K-12, and universitiesDespite the effort by the national action plan to improve health literacyHealthcare disparity has been recognized as a major concern in current healthcare systems around the world, and disparity in COVID-19 pandemic has shown to be significant . LiteratIn response to the COVID-19 pandemic and infodemic, we have discussed major concerns in eHealth literacy education, summarized the significances and challenges of improving eHealth literacy in both communicable  and non-communicable diseases , and made our recommendations of a general framework of AI-based approaches to improving eHealth literacy and combating infodemic, including AI-augmented lifelong learning, AI-assisted translation, simplification, and summarization, and AI-based content filtering. Finally, we discussed additional key issues including integrating eHealth literacy education into curriculum, a whole-society approach to improving eHealth literacy and combating infodemic, and healthcare disparity. We envision that there are huge opportunities in the near future to both improve eHealth literacy and combat infodemic because of the fast advancements of AI technologies and their wide and rapid adoption in every corner of our society.TL and XX conceptualized this work and prepared the manuscript draft. Both authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


This Framework provides a working model that can be updated as new approaches are advanced.Dietary bioactives are food substances that promote health but are not essential to prevent typical deficiency conditions. Examples include lutein and zeaxanthin, omega-3 fatty acids, and flavonoids. When quality evidence is available, quantified intake recommendations linking dietary bioactives with specific health benefits will enable health professionals to provide evidence-based information to consumers. Without evidence-based recommendations, consumers use information from available sources that often lack standards and rigor. This article describes a framework to develop guidance based on quality evidence fully vetted for efficacy and safety by qualified experts, and designed to communicate the amounts of specific dietary bioactive compounds with identified health benefits. The 4-step Framework described here can be adapted by credible health organizations to work within their guideline development process. Standards of practice used in clinical guidelines are adapted to quantify dietary bioactive intake recommendations from foods consumed by the general public, by taking into account that side effects and trade-offs are often needed for medical treatments but are not acceptable for dietary bioactives. In quantifying dietary bioactive recommendations, this Framework establishes 4 decision-making steps:  Statement of Significance: A credible step-by-step process for translating evidence into quantified dietary bioactive intake recommendations is provided; the approach is relevant to nutrition guidance developers globally. Many working definitions for bioactives exist, depending on the purpose and to some extent the regulatory constructs in which they are considered . The FraQuantifying intake recommendations for bioactives differs from doing this for nutrients with established DRIs, which typically are well defined chemically, with well-characterized metabolic roles in specific health outcomes. In contrast, most bioactives are chemically complex and diverse, and their role or effects on health can be partially met at times by other very similar chemical constituents, making their individual contributions to specific health outcomes or status often difficult to ascertain. Some bioactives can be rapidly converted into other active or nonactive constituents through the processes of digestion, absorption, and metabolism. Therefore causal inference associating a quantified intake of a bioactive with a benefit to normal structure/function or disease risk reduction can be more complex than that which occurs with nutrients.This Framework provides a step-by-step approach to quantify bioactive intake recommendations in food forms when the quality of evidence for benefit is determined to be sufficient and there is little or no evidence of harm. Although generally similar in approach, the Framework differs from medical treatment guideline development in which decisions need to be made, even when the quality of the evidence showing efficacy is relatively low or there are potentially negative treatment effects. In contrast, it is not imperative to recommend intakes for a dietary bioactive for which there is no overt medical condition requiring treatment, and thus intake recommendations for bioactives can be dichotomous: yes/no. This dichotomous approach eliminates the "weak" recommendations and "strong" designations used in the GRADE  approach.This Framework does not apply to bioactives consumed in supplement forms due to safety concerns associated with bioactives when isolated and consumed in concentrated forms . DiffereThis approach describes a 4-step process for evaluating the evidence about a dietary bioactive with demonstrated benefit to human health  and, where it is warranted, translating it into a quantified intake range for generally healthy populations or subpopulations. When recommended, intakes are issued in a formatted summary statement reflecting the quantified range of intake for a dietary bioactive consumed in food forms. The following principles apply when recommendations are made for dietary bioactives:Recommended dietary bioactive intakes are based on specific health outcomes using the best available evidence; multiple health measures of the outcome may be considered.Recommended ranges are expected to change as new data become available at differing levels of intake, or with new findings regarding benefit(s) to health.Recommended intake ranges are limited to consumption levels without known adverse effects that have the potential to harm an individual.Food forms include those in which the bioactive is naturally present or enhanced as long as it meets regulatory requirements. If data from isolated extracted forms are used to inform recommendations, chemical (species) and physical form (matrix) must be relevant to those consumed in food forms.Guiding Principles for Developing Dietary Reference Intakes Based on Chronic Disease, which outlines an approach to characterize a quantitative relation between a nutrient or other food substance (termed NOFS) and specifically chronic disease risk  only to populations similar to the studied populations.range rather than a single number to express CDRR recommendations.Provide an intake If changes in disease risk occur only at intake levels above the upper level (UL), CDRR is unnecessary, because the UL should not be exceeded.moderate using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) system.Ensure that the level of certainty for the intake-response relationship to chronic disease risk is at least Conduct a risk/benefit analysis when benefits and harms overlap for nutrients or other food substances (NOFS) and chronic disease risk. The method used to characterize and decide on the balance should be explicit and transparent.Evaluation of Biomarkers and Surrogate Endpoints in Chronic Disease , cognitive performance, BMI, blood lipids, and blood pressure. Valid and reliable measures of normal health status along with disease risk reduction biomarkers accepted by authoritative decision-makers can be used to quantify the health benefits of dietary bioactives. Such biomarkers are described in the IOM report on the  Disease .Double-blind, randomized clinical trial interventions evaluating a pure bioactive are the most direct means of establishing a causal relation between the bioactive and a specific health outcome. However, if the bioactive is part of a habitual diet, particularly if present in multiple foods, it can be impractical or impossible to design a study to test the bioactive independent of the baseline diet or matrix. In that case, the quantity consumed cannot be ascribed solely to the effect of that intervention unless the other nontested dietary sources are controlled for adequately. Also, it should be noted that when a semipurified extract or purified bioactive compound is evaluated, it might not behave the same when consumed in isolation rather than in a food matrix . Therefore, although evidence from blinded, randomized interventions with isolated extracts provides important information, it should be recognized that observational studies can also provide highly relevant supporting evidence.Slight changes in chemical and physical structure of a bioactive compound from one food matrix or type of food to another can affect its bioavailability or physiological effect. As an example, polyphenols occur in several chemical subclasses that differ by food source, including variations in glycosylation, esterification, and polymerization, which can affect bioavailabilty . TherefoRecommendations based on a body of evidence that is considered to be generalizable to the overall population will have broad reach. However, if evidence is not relevant to the general population, then the subpopulation of interest should be specified at the onset of the process to ensure recommendations are developed within the context of the specified subpopulation. This is consistent with the 2017 FNB guiding principles report for setting DRIs based on risk reduction for chronic disease, which stipulates that recommendations should only be applied to populations with the disorder or conditions that are similar to studied populations . OverallThis section describes a sequential decision-making Framework to be used in reviewing evidence and developing bioactive intake recommendations that quantify a range reflecting both efficacy and safety . The proCharacterize a single bioactive or group of bioactive compound(s) associated with a specific health outcome by chemical structure or isolation techniques.Ensure that sufficient food composition data are available to enable the translation of quantified intakes into dietary choices.Determine that intake of the bioactive is quantified by a reliable intake exposure or a validated biomarker of intake.In elucidating causal relations between bioactive dietary components and specific health outcomes it is important to know the variety of chemical forms and matrices in which they are found. The matrix, chemical form, and dose, as well as individual variation in metabolism based on genetic and gut microbial factors, can all affect the ADME of the parent compound or its metabolites. For example, a systematic review indicated that the flavan-3-ols present in tea (epicatechins) improve flow-mediated dilation in blood vessels, whereas the flavan-3-ols in cocoa/chocolate (catechins) improve flow-mediated dilation as well as blood pressure . This evThe unique chemical structure(s) related to the identified role in health must be measured in various foods with a high degree of confidence (accurate and reproducible), including under various conditions of preparation . Such information on the bioactive's composition in foods is necessary to quantify the relation between the intake of the bioactive with possible health outcomes as well as to quantify intakes with an established history of safe consumption. Additionally, food composition data are necessary for health professionals to translate recommended quantities into food-based dietary advice for consumers. If bioactive food composition data are lacking or weak, the process of developing recommendations must wait until sufficient food composition data are available.When bioactive intakes are calculated using dietary intake reports combined with food composition databases, substantial information on the methodology used to estimate intake should be made available. This includes describing how the approach corrects for random error, uses designs that decrease subject bias, and estimates usual intakes by repeated measures that take into account seasonal variation as well as under- or overreporting.Biomarkers of intake are necessary when evidence for the possible relationship between the bioactive and a health benefit is exclusively from observational or other human studies lacking measured, quantified total bioactive intake . Validated biomarkers are expected to show a precise and reproducible dose–response relationship with known amounts of the biomarker from an inclusive list of foods typically consumed by differing age groups under various metabolic conditions, and be applicable across ethnic groups and gender groups.In some cases, multiple chemical structures are represented in a class of bioactive components . It is necessary that the main components be chemically/structurally identified, and that analytical methods exist to quantify individual components. Where structural diversity exceeds current analytical capacity , quantitative estimates of their content in foods should be provided using the best analytical approaches available.If validated biomarkers of intake or exposure exist and are validated in similar population groups to those of interest, then such biomarkers of intake can be used to quantitatively describe its relation to a function or other health parameter. When intake biomarkers correlate with the group of bioactive components, the effects should be attributed only to the specific compounds, and not generalized to a group or class of compounds. For example, when blood concentrations of specific long-chain omega-3 fatty acids are related to health outcomes, the evidence cannot be extended to all dietary omega-3 fatty acids as a single combined category.There is also the possibility that the active component that produces a health benefit includes more than a single metabolite, rather than the bioactive itself. Therefore, it is useful to use established nomenclature, such as has been recommended for polyphenol catabolites . SimilarQuantify what a relatively high level of dietary exposure is for the bioactive.Document its history of safe consumption within the population.Ascertain the bioactive quantity with no known adverse health effects.Intake quantities recommended for dietary bioactives must be consistent with a history of safe consumption to ensure that the recommended range is not associated with known adverse effects that significantly impact health. Note that the criterion of no known adverse health effects excludes modest usual day-to-day general discomfort that occurs when eating a varied diet or making changes in one's diet . If long-term dietary intake data among typically “high” consumers are unavailable, standard toxicological testing should be used to quantify safe levels, particularly if it appears that benefits accrue when consumed at levels above historical safe use. Information on safety should take into account the nutritional, botanical, and toxicological peer-reviewed scientific literature, reports from authoritative bodies such as government expert groups or those of regulatory bodies, survey data of food or nutrient composition and consumption, and other peer-reviewed material describing the composition of the sources of the bioactive, as well as proprietary information where available from survey or consumption data, product sales data, and compositional analyses.Dietary Supplements: A Framework for Evaluating Safety provide a useful construct for the purposes of this Framework on dietary bioactives .only to bioactives in food forms, history of safe use of the bioactive is primarily from foods. If data are augmented with supporting data from bioactives in dietary supplement forms, it should be noted that history of bioactive use from supplements is probably of relatively much shorter duration and the conditions of use can be quite different (including consumption of bolus doses without other foods or beverages). Choice of a subpopulation for quantifying history of high use should reflect long-term high use relevant to the population for which the recommendation is being quantified. This promotes harmony in recommendations across similar populations. For example, tea consumption in populations with high intakes is useful in quantifying high intakes for geographic regions with comparable diets and populations, such as tea intake in Turkey for the Middle East, Ireland for Western Europe and the United States, and China for Asia.The highest quantified level of bioactive intake for which there is a history of consumption with no known significant adverse health effects needs to be determined so that the final recommended quantity reflects efficacious levels with demonstrated safe use. History of use must be relevant to the dietary bioactive forms and sources included in the evaluation. In this Framework, which pertains The method used by the US FDA can be followed to determine the quantity of a bioactive with a history of safe use . In briedose (intake per serving),total daily intake at the mean and high exposure (90th percentile) levels among those who ingest the bioactive,duration of use,conditions of use,frequency of use of the historically consumed food source, andcharacteristics of the populations consuming it, such as age, health status, etc.The US FDA considers 25 y of widespread use as the minimum for establishing a history of safe use, along with the number of consumers who consumed the ingredient (in this case the bioactive), and the frequency of consumption . This coBecause source affects bioactivity, the source should be specified when quantifying historically safe consumption, because processing can alter not only the food matrix, but also the compounds present and the bioactive's potency itself. If the dietary bioactive is present in a widely consumed food(s), the quantity of the bioactive historically consumed establishes a reasonable expectation of safety as long as there are minimal differences in dose, composition, processing, conditions of use, and the target population. However, safety assessment and testing apply even if there is a history of use when the food contains other compounds with known or suspected adverse effects, has reported or published adverse effects, or has a limited history of use.By definition “dietary bioactives” are present in the diet and therefore generally have a long and well-substantiated history of being consumed safely in food. However, experimental safety testing applies when:the dietary bioactive compound or food source has known or suspected adverse effects , orhistorical bioactive intake differs significantly from levels evaluated for health benefit, such as relatively higher intakes, longer duration, orthe recommended bioactive is targeted to a different population group than the population with long-term historical use.When toxicology testing is appropriate, it is useful to review all toxicological or clinical studies (published or proprietary) in which the test substance is identical to the bioactive under review, the matrix is the same, the route of consumption is the same, and the dosage and duration are similar to what is proposed. Established state-of-the-art toxicology testing guidelines from an authoritative body applicable in the region for which bioactive intakes are being recommended should be followed to establish safety. As an example relevant to the United States, current guidance from the FDA on new dietary ingredients defines specific genotoxicity, oral tolerance, reproductive effect, and teratogenicity tests . ToxicolExperienced food safety experts should evaluate the history of safe use , in order to prepare a concise statement indicating the highest quantity of the bioactive with no known serious adverse health effects, and provide supporting documentation. This level is used in Step 4 to ensure that the recommended range of bioactive intake based on demonstrated efficacy is at or below what is determined to be the safe level.Select a health outcome associated with the dietary bioactive and relevant population.Identify specific physiological or biochemical measures recognized as indicators of the health outcome.Characterize through systematic evidence review, relationships between quantified intakes of bioactive(s) with health outcomes in the target population.In Step 3, human data providing evidence of a causal relationship guide the decision on which health outcome to select, including measures related to normal structure or function, indicators of maintaining health, or reduced risk of disease. Animal and cell culture studies are appropriate for determining the potential mechanism and site of action, but are insufficient to quantify levels of dietary bioactives for specific human health outcomes. Identifying specific mechanism(s) of action strengthens confidence in the bioactive's relationship with a function/health outcome or chronic disease process, but is not necessary to make a recommendation where there is substantial evidence that the relation is causal. Primary evidence of an independent, causal relation between the bioactive(s) and health comes from human intervention studies. A descriptive summary of the evidence available in the scientific literature (sometimes referred to as knowledge mapping) is useful to determine if the evidence available from human intervention trials with supporting observational studies warrants proceeding with the process to develop recommended intakes . KnowledExperts in the relationships between the bioactive and health outcomes under evaluation are critical decision-makers when selecting meaningful health outcomes relevant to the bioactive. For most health outcomes, a specific validated and reliable physiological, structural, functional, or biochemical measure is used to establish a causal link between the dietary bioactive and the health outcome. An example of a biochemical indicator is macular pigment optical density (MPOD). MPOD is a noninvasive measure of basic eye structure associated with visual function, including photo stress recovery, glare disability, and contrast sensitivity . TherefoMeasuring chronic disease outcomes directly has limited feasibility because disease can take years to manifest. Therefore, qualified biomarkers of chronic disease risk are important outcome measures. They are defined as biomarkers that can impact the risk of chronic disease development, and for which the time course for the effect is known, as is the proportion of the population affected if intakes are inadequate or adequate .Surrogate end points  can be considered for the specific bioactive under review but only if they are in the causal pathway . For exaMultiple health measures related to the selected outcome should be included when evaluating evidence. For example, a recommended bioactive intake to support general cardiovascular health can be evaluated by multiple types of specific measures such as blood pressure and vascular plasticity, clinical measures, or disease incidence. Supporting normal vision can include more than a single indicator of visual function such as visual acuity and contrast sensitivity, or the status of specific structures of the eye. Furthermore, relevant outcome measures might differ by subpopulation .A summary of the evidence available in the scientific literature is a useful tool to inform decision-makers regarding the specific physiological measures by subgroups, before proceeding to a systematic evidence review . GRADE aIt is important to remember that because of the multifactorial nature of diseases, beneficial effects of a bioactive on intermediate markers are neither synonymous with nor a guarantee of disease prevention. For example, improved flow-mediated dilation can improve vascular health but cannot be assumed to be sufficient to prevent cardiovascular disease. Intakes of dietary bioactives should be communicated within the context of supporting normal structure/function and reducing risk of (not necessarily preventing) disease conditions.A systematic evidence review of the association between a bioactive and a health outcome is necessary . The evidence review should be published in a peer-reviewed publication and completed by experts in systematic reviews using current standards of practice in the field and experts in the bioactive–disease outcome relationship. An example of an approach to conducting nutrition systematic reviews is available from the USDA NESR website . The revIf the recommendation is intended for the general population, evidence should be based on research conducted in generally healthy people by excluding research designed to treat or reduce symptoms in persons with relevant medical conditions. For example, studies of subjects with diseases known to affect cognitive function (such as multi-infarct dementia and Alzheimer disease) are excluded when evaluating maintenance of normal cognitive function in older adults, and studies of persons with type 2 diabetes are excluded when evaluating maintenance of normal blood glucose after a meal, but not those with elevated glycated hemoglobin at levels deemed prediabetic. Inclusion and exclusion criteria for population, intervention, control, and outcome (PICO) define the scope of the final recommendation.In the process of conducting the systematic review, bioactive intakes from observational studies reported qualitatively as categories from low to high  need to be converted to specific quantities of intake by obtaining that information from the original research investigators. A meta-analysis or pooled study can help determine the amounts (grams per day) of bioactive intake related to the effect. The systematic evidence review of flavan-3-ols and cardiometabolic health by Raman et al.  illustraIn addition to quantifying the amount of bioactive intake associated with the health benefit, the evidence should be graded to reflect confidence in the estimated effect of the relationship. Credible up-to-date methods in the field of systematic evidence review practices include but are not limited to GRADE, which rates the quality of evidence based on study design factors, and “risk of bias, imprecisions, inconsistency, indirectness, and magnitude of effect” . Other eAlthough the quality of evidence represents a continuum, such processes typically result in assigning evidence into predefined categories. NESR uses 3 quality grades plus “not assignable.” The 2013 GRADE handbook offers a simple transparent way to communicate the quality of evidence according to 4 quality ratings  reflecting confidence that the “true effect is likely close to the estimate of the effect” . When muDetermine whether the quality of evidence supports making a quantified recommendation. If less than moderate quality, no recommendation is made.Develop the range of intakes with demonstrated efficacy and safe use.State the recommendation in this structured form:From specific foods if limitations are needed.Based on [insert moderate or high] quality evidence, the recommendation is made to consume between [range] grams daily of [dietary bioactive*] to [support structure/function or reduce risk associated with appropriate relationship] among [the general population or specific subpopulation]. *Using a systematic review with quality of evidence ratings, a panel of experts knowledgeable in both the bioactive and the health outcome quantifies the range of intakes for which there is sufficient quality evidence of a causal relation for normal structure or function, an indicator of health maintenance, or reduced disease risk. Decision-makers should consider that even small effect sizes can have a large impact at a population level. For example, Stamler  estimateless than moderate (as described in Step 3 of this Framework), the experts should state the decision is to NOT present a recommendation, and provide the rationale regarding data inadequacy with guidance for future research. This step differs from the procedure sometimes used for clinical practice guidelines  values derived from demonstrated experimental safety/toxicology testing. A clear-cut example is that when the range of intakes shown to achieve a benefit exceeds or mostly exceeds the demonstrated history of safe intake, then a quantified intake recommendation for that health outcome is not issued. However, undesirable effects could include factors identified by the expert panel other than “safety,” such as whether intake of energy, nutrients, or recommended food groups is likely to change in an undesirable direction. This could occur, for example, if consuming the specific bioactive in the range recommended would likely contribute excess energy intake in a population with a high prevalence of overweight.Although consumer values, preferences, and costs are among primary considerations for clinical practice guidelines as part of the GRADE approach, they are of relatively less importance for dietary bioactives because individual consumers make those choices on a discretionary basis as part of their everyday food choices. This is unlike medical treatments in which clinical practice guidelines determine the preferred treatment prescribed, taking into consideration important values and preference factors, such as treatment side effects.The FNB Guiding Principles report recommended that, rather than specific amounts, ranges be developed for chronic disease risk reduction . In 1989This Framework is intended to provide guidance to health organizations responsible for developing nutrition recommendations. This includes health professional societies, government agencies, and other nongovernmental authoritative bodies responsible for developing health and nutrition guidance. The Framework is flexible such that it can be adapted to each organization's individual guidance process.1) the purported benefit of the bioactive addresses an unmet need with respect to the major health issues in the population for which recommended intakes would apply , or 2) the bioactive is heavily marketed or promoted in the media  to the extent that guidance is needed from the scientific community regarding amounts, if any, that should be recommended.Given the expense and resources needed to properly develop a quantified bioactive intake recommendation, it is important to prioritize the candidate bioactives. Given that diets, risk factors, and health conditions differ greatly from country to country, recommendations developed for one region or country might or might not extrapolate to another specific population. Priorities for developing a quantified bioactive intake recommendation should be informed by experts knowledgeable about the specific country or region to which they will be applied. Two important determinants include: Integrity and transparency are of primary importance in developing recommended intakes for dietary bioactives. Box C:Role of sponsoring organization(s)Ensures that the evidence review(s) is conducted by experts using a transparent and credible recognized systematic approach, and that itOperates from a statement of task with a clearly defined scope of work.Registers evidence review prior to its conduct.Appoints an evidence review committee, which includes both systematic review methodology experts and subject matter experts experienced in the specific bioactive and potential health outcomes.Ensures evidence review is published in a peer-reviewed journal, along with detailed grading tables.Oversees recommendation review and developmentOperates expert recommendation panel review from a statement of task with a clearly defined scope of work.Appoints the recommendation panel of expertsWith demonstrated expertise  in the bioactive as it relates to the health outcomes under review and in biostatistics and systematic review methodology; andWho are free of significant financial, intellectual, and professional conflicts of interest; if such exist, manages and balances bias and conflict of interest according to published policies and procedures (such as is done by NASEM) .Identifies and makes publicly transparent a structured guideline development process that is defined before the review begins.Oversees an external blinded review by a representative group of scientists and potential users of the report; provides external review comments to the recommendation committee for decisions on final revisions.Publishes the recommendation report, with supplementary background information.Ensures financial support is transparent to external entities and in all publications.Establishes plans for periodic re-evaluation.The organization(s) responsible for issuing a recommended bioactive intake range and statement should proactively plan for updates with specific timing  with triggers in place for an earlier review, which could include new safety concerns, new data on bioactive consumption patterns, or new data on efficacy. Proactive steps include:Develop an analytical framework for conducting the update.Monitor evidence based on a literature scan using keywords from the analytic framework for manuscripts published since the last review. Review abstracts and publications for relevance to prespecified criteria.Consult with experts on the bioactive of interest and health outcome(s).Establish decision criteria for whether new evidence merits initiating a recommendation update review.Quantified ranges of intakes for dietary bioactives are a useful way to accurately relate dietary bioactive consumption with a specific health benefit(s) at levels found to be safe. The step-by-step Framework leading to the structured statement format ensures that all relevant context is communicated along with the recommended intake range, including the form of the bioactive, specific health outcome or indicator measures, and subpopulations affected. Approaches used in developing clinical practice guidelines for medical treatment  are modified when quantifying health-promoting dietary bioactive intakes that are consumed as a part of the everyday diet within the context of individual consumer values and preferences.Translating evidence into recommendations should be conducted using a structured and transparent process managed by credible health organizations experienced in and responsible for developing food and nutrition recommendations. The recommendation panel should rely on members and advisors with strong published scientific expertise in the bioactive and its metabolism, in safety/toxicology, and in the health outcome under review. It should also include input from multiple stakeholders, follow principles of scientific integrity, and be published in a peer-reviewed scientific journal. The resulting dietary bioactive intake recommendations can inform health professionals providing advice to the general public, as well as to scientists conducting research necessary to fill evidence gaps.


Gene therapies (GTs) are considered to be a paradigm-shifting class of treatments with the potential to treat previously incurable diseases or those with significant unmet treatment needs. However, considerable challenges remain in their health technology assessment (HTA), mainly stemming from the inability to perform robust clinical trials to convince decision-makers to pay the high prices for the potential long-term treatment benefits provided. This article aims to review the recommendations that have been published for evidence generation and economic analysis for GTs against the feasibility of their implementation within current HTA decision analysis frameworks. After reviewing the systematically identified literature, we found that questions remain on the appropriateness of GT evidence generation, considering that additional, broader values brought by GTs seem insufficiently incorporated within proposed analytic methods. In cases where innovative methods are proposed, HTA organizations remain highly conservative and resistant to change their reference case and decision analysis framework. Such resistances are largely attributed to the substantial evidence uncertainty, resource-consuming administration process, and the absence of consensus on the optimized methodology to balance all the advantages and potential pitfalls of GTs. Gene therapies (GTs) have shown extraordinary promise for the mitigation or potential cure of a broad spectrum of life-threatening or debilitating diseases, such as cancers and retinal disorders . Yet, de®), was approved in Europe in 2012 ; hence, specific considerations have been needed. Recommendations to mitigate such uncertainties in payer decision-making of GTs continue to be published since the first GT, alipogene tiparvovec  protocol, we identified articles in Medline and Embase published in English in the last 5 years that proposed any recommendations for the evidence generation for GTs as based on clinical evidence needs, economic analysis, and payer affordability considerations. The details for the search strategy and PRISMA diagram were provided in Randomized controlled parallel trials (RCTs) are inarguably accepted as the gold standard approach for establishing the safety and efficacy of a new intervention , 13. HowHowever, such limited data have created larger-than-usual gaps for GT evidence needs , and strHowever, reducing the bias of historical control groups to complement single-arm trials must also be minimized, and the following aspects could be considered , 26, 27:DAF considerations: Although numerous approaches to reduce payer uncertainties with single-arm studies have been published, HTA guidance on assessing evidence from single-arm trials remains scant  biological plausibility , 25; (2)DAF considerations: Few HTA agencies currently provide guidelines on statistical methods to validate surrogate endpoints, nor provide explicit criteria to decide whether specific surrogate endpoints are more valid than others  to confirm treatment benefits and bridge the evidence gaps in the initial regulatory and HTA submissions for GTs , 36–38. DAF considerations: Depending on country-specific restrictions on reimbursement and local clinical practices, the transferability of data may be challenged. Moreover, while VOI analysis can support decision-making on the need to perform additional studies  gains, and indirect benefits, such as the continuation of normal education, improvement of productivity, and reduction in the caring burdens of family members. In addition, GTs can bring other broader forms of value, such as scientific spillover, adherence improving factors, health equity issues, and value of cure , 37, 42.However, these broader values are not or insufficiently captured in the QALY and incremental cost-effectiveness ratio (ICER) calculation in current HTA reference cases. Hence, it has been argued that cost-effectiveness should provide references cases analyses from both healthcare payer and societal perspectives , 25, 26.DAF considerations: While numerous publications propose novel elements of value, these have had limited influence on HTA bodies which focus predominantly on clinical effectiveness and cost-effectiveness. Some broader disease-related value elements  are already incorporated in some agencies such as the Haute Autorité de Santé , Agenzia Italiana del Farmaco , and the Spanish Agency for Evaluation of Medicines and Healthcare and the Products Interministerial Committee for Pricing (AEMPS and CIPM). Other agencies including the National Institute for Health and Care Excellence  and the Scottish Medical Consortium  tend to incorporate modifiers to the ICER threshold and social cost when applicable, while Tandvårds-och läkemedelsförmånsverket  and Zorginstituut Nederland  also consider indirect benefits . However, broader societal values  are rarely considered  than normally applied has been explored for products with long-term benefits or studies with longer time horizon, such as 2.5 % when time horizon <30 years, and 1.5% thereafter in France  is accounted for on the balance sheet within the year of acquisition. However, the key accounting principle of amortization may offer a new approach for healthcare payers to unlock access to GTs while spreading the budget impact over several years (al years .Our review has summarized several approaches for HTA evidence generation to meet the idiosyncrasies of GTs. While these theoretical perspectives are rational and necessary, from a pragmatic perspective, much flexibility is necessary from HTA bodies to put recommendations into practice. Yet, the flexibility of tailoring current DAFs within HTA organizations is not a likely reality in the near term.While a new class of product with a novel mechanism of action may stimulate arguments that a specific reference case is necessary, mainstream opinions indicate that adjustments could be considered for GTs, although a new reference case is not needed . For exaHence, given all the evidence needed for GTs, to ensure fair and timely evolution of DAF of HTA proactive interactions with relevant stakeholders could be powerful and well-warranted , 38, andPublished recommendations for evidence generation and economic assessment of GTs currently appear to have limited impact on HTA DAF due to either them being more theoretical in orientation rather than being pragmatic or because of conservative attitudes of HTA on DAF. Multi-stakeholders dialogue is warranted to enhance communications that will allow more certain and rapid assessments of GTs, thus enabling optimized patient access at a reasonable budget impact.The original contributions presented in the study are included in the article/MT conceived the design of this review, drafted the table of contents, and provided expert insights. TQ and GS wrote the entire manuscript. MP, SL, EP, and JJ contributed to the literature searches and abstract drafting. All authors contributed to the article and approved the submitted version.MP, EP, JJ, and MT was employed by the company Creativ-Ceutical. GS was employed by company Apothecom. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.