In 2008, a conceptual model explaining the role of motor competence (MC) in children’s physical activity (PA), weight status, perceived MC and health-related fitness was published.The purpose of the current review was to systematically compile mediation, longitudinal and experimental evidence in support of this conceptual model.N = 585) then screened further and combined into one review (n = 152), with 43 articles identified for extraction. Studies needed to be original and peer reviewed, include typically developing children and adolescents first assessed between 2 and 18 years and objective assessment of gross MC and at least one other variable . PA included sport participation, but sport-specific samples were excluded. Longitudinal or experimental designs and cross-sectional mediated models were sought. Strength of evidence was calculated for each pathway in both directions for each domain  by dividing the proportion of studies indicating a significantly positive pathway in the hypothesised direction by the total associations examined for that pathway. Classifications were no association (0–33%), indeterminate/inconsistent (34–59%), or a positive ‘+’ or negative ‘ − ’ association (≥ 60%). The latter category was classified as strong evidence  when four or more studies found an association. If the total number of studies in a domain of interest was three or fewer, this was considered insufficient evidence to make a determination.This systematic review (registered with PROSPERO on 28 April 2020) was conducted in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) statement. Separate searches were undertaken for each pathway of interest  using CINAHL Complete, ERIC, Medline (OVID), PsycINFO, Web of Science Core Collection, Scopus and SportDiscus. Potential articles were initially identified through abstract and title checking :290–306, 2008). However, to test the whole model, the field needs robust longitudinal studies across childhood and adolescence that include all variables in the model, have multiple time points and account for potential confounding factors. Furthermore, experimental studies that examine change in MC relative to change in the other constructs are needed.PROSPERO ID# CRD42020155799.The online version contains supplementary material available at 10.1007/s40279-021-01516-8. Motor development research has recently increased its focus on public health , largelyIn 2015, a narrative review was published to synthesise evidence related to the original model, and a figure was produced to summarise the available evidence at this time  and considers all variables in the model that relate to MC. Therefore, the purpose of the current review was to systematically compile longitudinal, experimental and mediation  evidence from late 2014 relevant to the Stodden et al.  review. The review was submitted to PROSPERO on 22 January 2020 (ID CRD42020155799) and registered on 28 April 2020. The searches were performed on 30/31 October and 8 November 2019. Searches focused on articles published from 2014 to 2019  as separate reviews . Four authors carried out the initial abstract and title screening for each of the four reviews independently as ‘the first screener’. An additional two authors (with particular subject expertise) assisted in each review as the ‘second screener’. The purpose of this first screening was to identify any potentially relevant article. A combined total of 585 articles were located for full-text screening. Two authors then independently screened each full-text article. Any conflicts were resolved in meetings with at least three authors to reach consensus. A total of 336 articles proceeded to stage 2 (249 excluded). These 336 articles were the result of included studies for each review . These were then combined into one file, and duplicates (n = 177) and any references that had been included in the narrative review ) reporting a significant association in the hypothesised direction, rated as no evidence (‘0’). When studies with many comparisons were excluded , 48, theCheng et al.  reportedFindings for locomotor/coordination/stability skills showed strong support for this pathway  when considered without the two studies with multiple comparisons , 48. SixIn slightly older Flemish children, Coppens et al.  reportedFive studies investigated the pathway from MC to weight status, with only 11% (5/45) of the analyses significant in the hypothesised direction, which means there was no evidence for this pathway (‘0’). The level of evidence was rated as strong negative (‘−−’), i.e., 80% (4/5), when one study with many comparisons was excluded . SimilarFor total MC, Cheng et al.  reportedIn summary, there is strong evidence of a negative association for weight status to MC and the reverse pathway, so long as studies with many multiple comparisons , 48 wereOnly one study assessed the impact of an MC intervention on weight status . WhereasThere was insufficient evidence to support a longitudinal relationship between perceived MC and actual MC as we identified no studies that investigated this pathway direction.Evidence was also insufficient to support a longitudinal relationship between actual MC and perceived MC. One study reported positive associations in two of six analyses (33%), rated as not enough information (‘I’). This Canadian longitudinal study investigated the pathway from actual MC (process assessment) at age 5–7 years to perceived MC  at 16 and 26 years . Total aTwo intervention studies investigated whether a motor skill intervention increased perceived MC , 67. LanOverall, there was no evidence for a pathway linking MC and perceived MC in either direction as evidence was inconsistent. This was likely because of the small number of longitudinal/intervention studies conducted in the past 5 years.Nine studies investigated perceived MC as a mediating variable, two using longitudinal data , 38, andBoth longitudinal mediational studies , 38 and Of the five studies that considered the path from PA to perceived MC to MC , 72–74, Of the eight studies that considered the path from MC to perceived MC to PA, six –71, 74 fStudies that found a mediating pathway used a mixture of process , 68–72 aSix studies investigated the path from health-related fitness to MC , 62, 64,All six studies used a product assessment of MC, with five studies using the body coordination test for children (Korperkoordinationtest fur Kinder [KTK]) , 62, 64.Cardiovascular endurance was investigated in five studies, with four studies reporting it was a predictor for motor coordination. Cardiovascular endurance  at 6 years old predicted motor coordination across 3 years in the same sample of Portuguese children , 40 and Flexibility was assessed using the ‘sit and reach’ test in two studies. Flexibility at 8 years old predicted motor coordination for Portuguese girls for hopping, jumping sideways and moving sideways . Yet, flFive studies addressed muscular strength and endurance as a predictor of MC , 42, 64.Handgrip was reported as a predictor of motor coordination by three Portuguese studies: children aged 6 years followed up over 3 years (same child sample) , 40 and One study reported that curl-ups at 6 years old did not predict motor coordination across 3 years in Portuguese children . When thLower body muscular strength and power (standing long jump) predicted motor coordination in two Portuguese samples , 40, 42.In summary, overall, there was indeterminate evidence for the pathway from fitness to MC (without the study with multiple comparisons). There was insufficient evidence for the pathway from fitness to total MC and to object control skills. There was no evidence for the pathway from fitness to locomotor/coordination/stability skills (27/97 [28%]); however, when the study with multiple comparisons was removed, there was strong positive evidence (‘++’) (17/25 [68%]).Five longitudinal studies , 62, 75 Two of the five studies investigated object control skills as a predictor of cardiorespiratory endurance , 59 and Only one study investigated flexibility. Fransen et al.  reportedIn summary, overall, there was strong positive evidence for the pathway from MC to fitness but insufficient evidence for the pathway from total MC to fitness and from object control skills to fitness. There was strong positive evidence (‘++’) (22/31 [71%]) for the pathway from locomotor/coordination/stability skills to fitness.Four studies  investigated mediation from PA to fitness to MC. This was classified as strong positive (‘++’) evidence. Britton et al. , JaakkolTwo longitudinal studies , 37 and The level of evidence for each pathway is summarised for each skill domain according to the information provided above and represented in Fig. The unique contribution of this review is that it examined all pathways of the Stodden et al.  model thn = 28) in this review investigated the path to or from PA. The studies varied widely in terms of measure of PA  and MC , age and sex of the study sample and the length of follow-up, making it challenging to compare study findings. Overall, we found no evidence for the pathway from PA to MC. This result was unexpected and contradicts past reviews, which highlighted the association between MC and PA [The bulk of studies  investigated the path from weight status to MC than the reverse (n = 5). There was strong evidence of a negative association for weight status to MC and the reverse pathway, so long as studies with many multiple comparisons [More studies  have either provided evidence that an unhealthy weight status is associated with poorer motor coordination (but not object control skill) because of the need to propel the body through space  or have Most studies in the current review assessed BMI , 63, 64 There was strong positive evidence for a pathway from MC to fitness and indeterminate evidence for the reverse. In terms of domains, there were strong positive relationships in both directions for locomotor/coordination skills. There was not enough evidence in either pathway direction to associate object control skills with any aspect of health-related fitness. There were not enough studies to judge total skills as a predictor or outcome of fitness or for stability or flexibility as predictors or outcomes of MC. There was strong evidence that locomotor/coordination predicted muscular strength/endurance and some evidence for cardiovascular endurance as an outcome of MC. For the pathway from fitness to MC, there was strong evidence that cardiovascular endurance predicted locomotor/motor coordination. There was mixed evidence concerning MC and muscular strength/endurance.These findings are confirmed by previous literature , 18, 25.r = 0.25), locomotor (r = 0.19), object control (r = 0.22) and stability/balance (r = 0.21) [Overall, evidence linking MC and perceived MC in either direction was insufficient and inconsistent. This was likely because of the small number of longitudinal/intervention studies conducted in the past 5 years. A recent meta-analysis regarding the association between actual and perceived MC reported significant small pooled effects for overall MC  . HoweverIn terms of fitness, seven analyses from four studies reported mediation from PA to MC, and seven of eight analyses from four studies found evidence of the reverse direction, providing strong positive evidence. For perceived MC, the path from PA to MC was classified as indeterminate, and the reverse was classified as no evidence. Whilst several studies since 2014 have assessed mediation, few used longitudinal designs , 37, 38.Very few intervention studies met our inclusion criteria . Whilst many interventions assessed both PA (and or fitness) and MC with the intention to change both, analyses and/or designs were mostly targeted to detect eventual parallel gains, so it was not possible to state whether change in MC explained change in the other variable (or vice versa). Only one intervention assessed whether change in MC affected change in PA and fitness, finding that total skills mediated the effect of the 12-month intervention on moderate to vigorous PA and fitness in Australian 8-year-olds . The othNo longitudinal studies included all the variables and pathways of the model conceptualised by Stodden et al. . Thus, tThe current review considerably expands the evidence base by synthesising mediation, longitudinal and experimental studies within the last 5 years, including studies published in languages other than English. Further strengths include using PRISMA guidelines, clear inclusion and exclusion criteria, careful analysis and consideration of null results, calculation of effect sizes where possible and multiple authors double checking each other’s categorisation. The broad scope of this review meant that we could not assess how other relevant variables  related to the core variables in the model. For instance, an important consideration in understanding the influences on health-related fitness and MC is biological maturation and associated allometric growth, given the differential impact maturational status and growth has on MC and health-related fitness . Furthersample size ,subgroups ,means ,standard deviations ,intervention group detail ,all variables in a model ,non-significant values  and/orthe regression coefficient in mediator studies .Meta-analyses were not permitted because of the limited number of studies with homogenous measures within a variety of domains regarding MC, PA, fitness and weight status. Future researchers should carefully consider the design of, and the measures used in, a study. Applying standard and robust measures and reporting fully on these parameters will facilitate comparing and pooling data from different studies. Our results syntheses were conducted based on the number of significant associations versus the number of non-significant associations. As stated in the discussion on the PA pathway, this helped to provide a balanced picture of the literature. However, some studies included multiple analyses within one study, and this way of calculating results does not take into account the weighting of results from one single study. In an effort to combat this ‘bias,’ we decided to present the results with and without studies with multiple (more than eight) analyses in the one study. If it had been possible to perform meta-analyses, we could have considered the weighting of the data from different studies. Future reviews need to consider the weighting of evidence from single studies and ensure this is considered in results analyses and interpretations. What this current review has highlighted is that these single studies can shift the weight of evidence, which shows the importance of the process of result synthesis. We did calculate effect sizes, where possible, in an attempt to better understand the strengths of associations across pathways. This served to highlight the few studies in which sufficient information was reported to enable calculation of effect sizes. Various study aspects were not reported, which then precluded calculation of effect sizes, for instance, a lack of reporting regarding:Future analyses need to report effect sizes, or sufficient information for effect sizes to be calculated, especially when null results are reported.Alice in Wonderland [Overall, there was evidence of a strong negative association for a pathway from weight status to MC and the reverse. There was strong positive evidence for the path from MC to health-related fitness and indeterminate evidence from fitness to MC. There was strong evidence of a positive path from locomotor/coordination skills to fitness in both directions. There was indeterminate evidence for a pathway from MC to PA and no evidence for the reverse. There was insufficient evidence between MC and perceived MC. Conclusions on mediation outcomes are weakened by the predominantly cross-sectional nature of the available evidence and the limited studies, with indeterminate evidence for the PA to MC to perceived MC mediated pathway (and insufficient evidence for the reverse) but strong positive evidence for the fitness-mediating pathway. This review has gone “through the looking glass”, as described in nderland  when thinderland , the fieSupplementary file1 (DOCX 18 kb)Supplementary file2 (DOCX 63 kb)Supplementary file3 (DOCX 37 kb)Supplementary file4 (DOCX 22 kb)Supplementary file5 (DOCX 37 kb)Supplementary file6 (DOCX 44 kb)Supplementary file7 (DOCX 33 kb)Supplementary file8 (DOCX 32 kb)Below is the link to the electronic supplementary material.


This is a protocol for a Campbell Evidence and Gap Map. The objectives are to identify and assess the available evidence on health, social care and technological interventions to improve functional ability among older adults. Low‐ anThe accumulation of exposures and environmental influences throughout the life course can influence the development of different risk factors that lead to chronic diseases, injuries, or other age‐related issues that contribute to declines in intrinsic capacities. Without a supportive environment, whether social or built, this will result in diminished functional ability. The gradual decline in intrinsic capacities as some people age can require increased health and social care services from informal  and formal caregivers . Increased care needs lead to increased burden on families, stress for older adults, and costs to society. For this reason, efforts to deliver cost efficient, effective interventions that optimize functional ability at any level of intrinsic capacity, is critical for older adults. Health and social care interventions , and related systems, services and policies may include technological tools and devices and provision of health and social supports in the home.While it is important to offer home‐based supports that promote functional ability, we need to be mindful that existing health inequities may be worsened  involved in supporting functional ability among ageing adults.We have a broad range of intended user groups including practitioners, researchers, policy/decision‐makers, and the public. We have also included three stakeholder groups as authors and advisory members: the WHO Department of Ageing and Life Course (RS), Cochrane Global Ageing (JTC) and the Campbell Ageing group (JTC). Each of these intended users are included as authors on the EGM and have participated in defining the intervention and outcome framework. The EGM will be used to identify the best quality evidence to guide decision making in this area. It will also help to identify gaps in the evidence base both in terms of evidence syntheses and primary research and thereby facilitate the prioritization of topics for further research.Currently, no EGMs exist that identify and assess the available evidence on health, social care, and technological interventions to improve functional ability among older adults.2Identify available systematic reviews and randomized trials.Identify areas where systematic reviews are needed.Identify gaps in evidence where further primary research is needed.Assess equity considerations in available systematic reviews and randomized trials.Assess gaps and evidence related to health equity.The objectives are to:33.1Define a framework.Identify the available evidence.Appraise the quality of the evidence.Extract, code and summarize the data that relate to the objectives.Visualization and presentation of the findings in a user‐friendly manner.We will adapt evidence gap map methods from various key papers  framework to define the intervention focus for this EGM as well as the outcomes. We further defined the scope of the framework in consultation with our research team which includes input from the public (AL), practitioners , information scientist (MR), policy‐makers  and researchers .The EGM framework will inform the inclusion and exclusion criteria. Key dimensions of the framework are listed below. We will follow the standard EGM framework as a matrix where the rows are intervention domains, and the columns are outcome categories. The interventions and outcomes will have sub‐categories.Maintaining autonomy and independence, especially being able to make their own choices and decisions, are important for older adults in all settings , social care services  and health care services  delivered in the home setting.3.3We have created an Advisory Group comprised of methodologists, physicians , consumers and researchers with expertize in assistive health technology, healthy ageing, long‐term care, rehabilitation, disability, memory, and cognitive impairment. We held an exploratory meeting to invite feedback on the development of our EGM framework at the Global Evidence Summit in Cape Town, September 2017. The participants included family practitioners, geriatricians, social workers, and methodologists. We also held a seminar at the Bruyère Research Institute Grand Rounds  with family practitioner‐researchers, where participants provided feedback on the intervention‐outcome framework. Our decision to focus on the selected intervention categories was also informed by public engagement through our public representative (AL). Our central team  plans to meet at least once a month to discuss the direction and scope of the EGM.3.4Figure We will review and update our conceptual framework as we develop the map.44.1We will include randomized trials and systematic reviews of both randomized and nonrandomized trials that meet our inclusion criteria. We define a systematic review according to the PRISMA definition, which encompasses articles that specifically stated methods used to identify studies , strategies for study selection  and explicitly detailed methods of synthesis ; sensory functions and pain ; neuro‐musculoskeletal function ; voice and speech ; cardiovascular, hematological, immune, respiratory system function ; digestive, endocrine, metabolic functions ; genitourinary and reproductive function ; and integumentary system function .Functional ability will consist of the following constructs: basic needs ; learning and applying knowledge ; contribution ; mobility ; relationships ; and communication .Process and other outcomes will include cost (out of pocket), cost‐effectiveness, falls, satisfaction of older adult, safety, caregiver outcomes, adherence, health service utilization, quality of life, financial security, access, and stigma. Access is a multi‐faceted concept and can be understood as the opportunity or ease with which consumers or communities are able to use appropriate services in proportion to their needs Daniels, . The conWe will use the PROGRESS framework to identify studies that measured effects of interventions by gender or other health inequalities.4.54.5.1We will include interventions in the home setting for older adults. We will define home as an individual’s place of residence that can include residential homes, apartments, long‐term care facilities, hospices, nursing homes and any nonacute care places of residence. Studies of mixed settings will be included as long as the intervention takes place in the home setting at least 50% of the time. We will code the settings so that the evidence can be filtered according to setting.Acute and sub‐acute hospital and convalescent care settings will be excluded .4.6We will develop and pilot a search strategy (with a selection of studies that met our inclusion criteria) with the guidance of an information scientist (MR). This search will comprise of the medical and health databases ( MEDLINE (via OvidSp), EMBASE (via OvidSp), Cochrane Database of Systematic Reviews, CENTRAL, CINAHL (Via EBSCOhost), PsychINFO (via OvidSp) and AgeLine (via EBSCOhost) and databases relevant to social care and social policy , Social Science Citation Index (via Web of Science) and Social Policy and Practive (via OvidSp). See Table We will search for relevant trials and systematic reviews in the gray literature via ProQuest Theses and Dissertation Global and via Conference Proceedings Citation Index. We will also search for relevant unpublished studies via relevant international organizations .https://osf.io/). Ongoing randomized trials will be searched in ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform.Ongoing systematic reviews will be identified by searching for protocols in PROSPERO and the Cochrane and Campbell libraries as well as on the open science framework .Description of the methods will include a definition of the data sources and methods of searching, the inclusion and exclusion criteria, study selection, quality appraisal, and data extraction methods and the approach to presentation/visualization. This section will provide a table in‐text showing one full search from a database as well as a PRISMA flow chart. An appendix will provide full search strategies used for each database, including any restrictions and filters used.The results section will present the number, type, and quality of studies retrieved for the main intervention categories, namely, products and technology related to mobility, and health and social services at home. We will also present information about how health equity has been considered in studies, provide an overview of major evidence gaps based on our framework, and list any limitations. The conclusion will provide implications for researchers, decision‐makers and policymakers on the evidence base in this area and the key areas for the commissioning of future research. We will also include implications for researchers considering conducting an EGM.Figure: Conceptual map/theory of change.Figure: PRISMA flow chartTable: number of studies by study design.Table: Number of studies by intervention and subcategories.Table: Number of studies by population.Tables and figures we will include:5.2We will present results as a matrix of interventions (rows) and outcomes (columns) and assess the availability of evidence across the additional filters of age groups, conditions, equity categories  and setting.5.3When there are multiple reports for a single study, we will treat them as one study. We acknowledge that SRs are likely to include the RCTs in the map and there may be more than one SR which includes the same RCT. All relevant randomized trials will be included regardless of whether or not it is included in a systematic review.66.1Two reviewers will independently screen the titles and abstracts of all retrieved articles. Title and abstracts will be screened on the basis of intervention, study design, and population, and not on the basis of outcome. Full‐texts of potentially eligible studies will then be retrieved and screened. The reviewers will compare the results, and any conflicts will be resolved through discussion or by a third reviewer. We will not contact authors of studies or reviews for missing information.6.2Two reviewers will independently extract data on published and ongoing systematic reviews and randomized trials related to the population, intervention, comparison, and outcomes. Our coding categories (Appendix) for data extraction will be based on our intervention/outcomes framework. In addition, we will collect details on characteristics that may be of interest to decision makers as filters for the evidence ‐ the country of the study (using WHO regions as well as the World Bank country classifications by income), age group , health conditions , study design  and setting .We will also collect details, if reported, on health equity as defined according to the PROGRESS framework ‐ place of residence, race/ethnicity/language/culture, occupation, gender/sex, religion, education, socioeconomic status, social capital and other characteristics associated with disadvantage and vulnerability such as sexual orientation, age and disability .


Campbell Systematic Reviews has continued to grow and diversify in its coverage of social science content areas, while maintaining its productivity in publishing high quality, policy‐relevant reviews and evidence and gap maps. In reflecting on this past year, I am reminded of my vegetable and fruit gardens, which I expanded during increased time at home during the COVID‐19 lockdowns over the past two summers.In 2021, In terms of growth, the Campbell Board approved a new Coordinating Group on Child Welfare and a new Sub‐Group of the Social Welfare Coordinating Group, Sexual Orientation and Gender Identity. Our Methods Coordinating Group defined seven types of empirical methods research which it considers, including innovative methods papers, systematic reviews and guidance  enabling public comments and debate for all of our articles, (2) developing a policy on equity, diversity and inclusion for our editorial teams and our content, (3) updating our conflict of interest policy, (4) screening for adherence to methodological expectations, and (5) adopting a transparent peer review policy. For each of these, as an evidence‐based organization, we will monitor implementation and assess effects on production timeliness and quality.We have built on the enthusiasm of members of the Campbell community, with a number of projects underway led by our members. These include, but are not limited to, the development of a Campbell Wikipedia Project (Campbell Collaboration Wikipedia Project, Campbell Systematic Reviews was accepted by Clarivate in the Emerging Sources Citation Index, which is a big step forward on our pathway to seeking a journal impact factor. We have also almost doubled the number of stories of policy influence in the last year. We track these influence stories using Altmetric, available on all of our articles since January 2019. Campbell systematic reviews have influenced international guideline development, intervention development and implementation, and informed policy discussions and research agendas.As a measure of our impact, in May 2021, Campbell Systematic Reviews. This evidence‐base needs to be transparent, rigorous and comprehensive. It needs to identify not only the policy implications, but also the uncertainties in the evidence‐base which require further research. Some recent examples are reviews on homelessness (Keenan et al., The ongoing COVID‐19 pandemic and its social and economic consequences require evidence to inform policy both now and in the post‐COVID recovery in all of the areas covered by The Campbell Collaboration is at an exciting crossroads. We are currently exploring transition to a new host institution to codevelop our long‐term sustainability as a member‐driven, open‐access organization which is committed to providing the best evidence to inform social policy decisions. This will be the fifth change in host organization of Campbell, and is happening at a time when we are at our strongest in history in terms of production, impact and membership, thanks to the current leadership.Returning to my gardens, I plan my crops and planting locations based on evidence about complementary crops to minimize pests, soil composition, spacing, orientation, sun exposure, fertilizer and water to maximize production. Despite this planning, some crops under‐delivered due to unforeseen pest invasions or weather extremes. Others outperformed expectations. Monitoring of these variations in outcome, and exploring possible causes gives insight into how to plan in future years.At Campbell, we welcome your submissions of high quality evidence synthesis and methodological research focused on evidence synthesis. As with my gardens, we will continue to monitor our growth and influence so we can make any needed corrections to achieve our vision of better evidence for a better world.


The complexity lies in the nature of evidence as a basic concept that circulates in both expert and non-expert spheres of communication, supports different interpretations in different contexts, and is inherently open to contestation. We outline a new methodology that combines a social epistemological perspective with advanced methods of corpus linguistics and elements of conceptual history to investigate this and other basic concepts that underpin the practice and ethos of modern medicine. The potential of this methodology to offer new insights into controversies such as those surrounding EBM is demonstrated through a case study of the various meanings supported by evidence and based, as attested in a large electronic corpus of online material written by non-experts as well as a variety of experts in different fields, including medicine.Evidence-based medicine has been the subject of much controversy within and outside the field of medicine, with its detractors characterizing it as reductionist and authoritarian, and its proponents rejecting such characterization as a caricature of the actual practice. At the heart of this controversy is a complex linguistic and social process that cannot be illuminated by appealing to the semantics of the modifier  This has been particularly true since the emergence of the evidence-based medicine (EBM) movement in the early 1990s. The basic idea in EBM is that clinical and health policy decisions should not be based merely on intuition, expert opinion, or pathophysiological reasoning. Such sources are considered potentially biased and unreliable. Evidence-based decisions should integrate expertise and patient preferences with the “best available external evidence from systematic research” , as attested in the Genealogies of Knowledge corpus of Internet English  literature on the meanings of ractice “” sectionevidence-based in a corpus of Internet English, and what this patterning reveals about the potential for this widely used term to generate different meanings in different contexts, in part because of the numerous meanings supported by each of its constituent elements (evidence and based). The analysis is rooted in the theoretical assumptions and draws primarily on the methodological innovations of two areas of scholarship: corpus linguistics (CL) and critical discourse analysis (CDA). Both insist on the primacy of attested uses of language, focus on identifying repeated patterns, and offer generalizations based on close analysis of such patterns. Corpus linguistics offers theoretical notions that assist in the analysis of the semantic and affective dimensions of key concepts such as evidence and evidence-based, as revealed in actual use across many texts, authors and historical periods, depending on the design of the corpora to which the analyst has access. These include, for example, the notion of collocation , semantic preference (the tendency of a certain word to collocate with other words from a specific semantic set), and semantic prosody   provides all data discussed below and includes a total of 3476 texts from over 35 outlets, including ScienceBlogs, Discover Society, openDemocracy, and UCSUSA (Union of Concerned Scientists).The data on which the analysis offered in “evidence and evidence-based have been raised in relation to other areas of evidence-based practice, where analogous controversies have arisen. For instance, Kvernbekk ”, and resorts to philosophy to unpack the meanings of the two components of the term, namely evidence and based. Rather than pursue this discussion in the realm of philosophical debate, our proposed methodology involves analysing attested uses of the two concepts that feed into different interpretations of evidence-based medicine—evidence and based—in a large electronic corpus of texts written by non-specialists using evidence, -based, and evidence-based in arguments about everyday topics such as climate change and human rights, as well as specialists on various areas of Evidence-based Practice, including EBM. Understanding the variety of meanings that basic concepts such as evidence assume in general discourse and in different contexts may afford us some insight into how they came to be condensed and streamlined to support a specific conception of medical practice associated with the EBM model and dominant among some, though by no means all members of the medical community.In assessing the controversy around the adequacy of EBM, Martini  asserts vernbekk , p. 515 evidence, placing this basic concept in a very high (118th) position in the general frequency list. Evidence occurs with higher frequency in the corpus than lexical items that are very common throughout the English language, such as same and since, and is also more frequent than many other terms that encode basic concepts characteristic of our thematically designed corpus, such as policy and society. At N − 1, the position one word to the left of the keyword, the most frequent collocates are the (719), of (359) and no (229). The and of are the two most frequent words in the overall corpus, so their high frequency in relation to evidence is not immediately informative. In contrast, no generally occurs much less frequently, and thus the pair no evidence makes for an interesting collocation, as can be seen in Fig. In the GoK Internet corpus, there are 4402 occurrences of the keyword no evidence form part of the larger pattern no evidence of (64), a phrase that in turn occurs in several longer sequences, as in the following examples:There is no evidence of a link between autisms and vaccines.  and the evidence is clear (9).The second example intervenes in the glyphosate controversy, the debate about whether the use of this herbicide is harmful to humans. Glyphosate is domestically known for being the main active ingredient in Roundup, but is also used on a vast scale in agriculture. Studies on its effects are so far inconclusive. Chemical interactions produce complex effects, and while studies declaring the herbicide safe are often considered questionable for being produced by commercial stakeholders, several studies condemning the use of glyphosate are criticized for being “unsupported by evidence”  or enough (17) evidence, but also insufficient (17) or little (38) of it. The fact that the quantifiers expressing sufficiency are often negated, as in not much evidence or not enough evidence, while the main negative ones tend not to be reversed, suggests that the quantification of evidence in most cases implies that it falls short of confirming or supporting a given claim  identifies a particular object of study, whereas empirical evidence (38) identifies a certain method of observation. New evidence (51) is uncovered in relation to earlier information on the same topic, and supporting evidence (11) is expected to back up claims to truth. An interesting category of qualifying adjectives concerns the specification of an affective response, as in compelling (29) and convincing evidence (13). Convincing is often modified by negation, as in no convincing evidence, whereas compelling and other intensifying qualifiers such as ample (14) and damning (7) are usually employed affirmatively, as in:compelling evidence that children’s perceptions of cigarettes are influenced by branding and that branding detracts from the impact of health warnings on packs. There is Not only can evidence be quantified, but it can also be qualified in a variety of ways: evidence as compelling is presented as an objective characteristic, thus fabricating a tacit agreement between the author, their sources, and the implied reader. The implication is that the evidence ‘speaks for itself’. The representation of actors and participants is a central issue in critical discourse analysis (e.g. van Leeuwen evidence: the evidence shows (40), indicates (15) and supports (11). Together, these two patterns (the absence of a subject and the proliferation of verbs of persuasion in position N + 1) suggest that perspectives and positions are seemingly automatically validated by the presence of evidence, with no need for a human mediator to be specified as the agent responsible for interpreting the information at hand. The most common collocate on the right-hand side of evidence, namely suggests (59), forms part of this widespread grammatical pattern. Often, the broader pattern includes strong markers of emphasis that reinstate the yes–no dichotomy at the far ends of quantification: there are several variants of all the evidence suggests as well as no evidence suggests. In these cases, too, the majority of instances do not specify potentially relevant participants in the situation of evidence assessment.Note that the indefinite ‘there is’ does not specify where the evidence originates. While such specification may occur elsewhere in the article, a more impactful omission is that of the supposedly compelled subject—in the absence of a subject, the qualification of the dicates 1 and suppabundant and compelling frequently precede evidence, but by far the most common qualifier modifying evidence is the seemingly more neutral adjective scientific (197). What exactly makes evidence scientific is seldom specified, but in some cases a number of expected characteristics are explicated:scientific evidence, published in Environmental Research Letters, that confirms the numerous anecdotal accounts. This is the first Rhetorically charged quantifications and qualifications such as anecdotal evidence (14 instances) is generally understood to be gathered unsystematically and in limited amounts. Furthermore, anecdotes are told rather than observed, meaning that the scientific mode is presented in opposition to the narrative mode of exposition. While such schemas can in some cases be derived from the concordance lines, often the collocation scientific evidence is simply used as a means of emphasis, as further illustrated by the fact that the collocation is frequently preceded by additional adjectives of persuasion. Indeed, the most frequent pattern associated with scientific evidence is overwhelming scientific evidence (10 occurrences). Overwhelming encodes a rhetorical mixture of quantity and suggested affective response, and thus a merger of the features of adjectives such as much and compelling, discussed earlier. In addition, the phrase overwhelming scientific evidence is in the majority of cases used to set up a contrast, as in:overwhelming scientific evidence linking fine soot particles to premature death, Honeycutt testified before Congress that “some studies even suggest PM makes you live longer.” Despite the Here, scientific evidence is partly defined by the communicative setting: an academic outlet is required to present it, and it is contrasted to anecdotal accounts. The latter contrast implies, once again, demands of quantity and quality, as despite the overwhelming scientific evidence, which occurs three times in the corpus, the function is not primarily to qualify the evidence in question but to suggest that one should be surprised at the existence of an opinion opposed to the author’s. Arguably, the various patterns of linguistic modification characteristic of evidence do not merely, or even primarily, serve to affirm, negate, qualify or quantify, but mainly to convince the implied reader of a given stance by rhetorical measures that conflate persuasive and epistemic modes of expression.Collocational patterns do not merely report a state of affairs; they express a communicative purpose which is of pragmatic rather than semantic importance , rather than as a verb (as in based on evidence). However, the two variants are often closely related, and where relevant attested verbal uses will contribute to the argument.One of the central arguments put forward by corpus linguists is that meaning does not reside in individual words, but is realised in extended lexical elements. This means, as already shown in the previous section, that based, the Oxford English Dictionary (OED London-based (36 instances) or UK-based (33). As these examples illustrate, such locations tend to be nations or cities. A second general use indicates a relation to a particular material, often described in chemical terms, and exemplified in the corpus by compounds such as carbon-based (10) and chlorine-based (1). The OED  and science-based (95), but also because of the widespread use of based in relation to social categorizations, as in class-based (36), community-based (30), or faith-based (25). The final main use listed by the OED concerns a specification of the base in question, as in broad-based (20). The fact that instances of all main categories of use can be easily retrieved from the concordance output suggests that the corpus provides an adequate general overview of the variety of meanings expressed by based.For compound adjectives containing nary OED  lists fo The OED  lists asbased cannot always be intuitively grasped by restricting one’s view to the various compounds in which it occurs, given that these compounds themselves form an adjectival unit that typically modifies a noun that follows. The co-text to the right of the compound has to be taken into account to arrive at an interpretation. A first observation is that when the adjectival compound itself contains a concrete material substance, the full phrase may still establish an abstract relation, as in oil-based economy (1) or carbon-based capitalism (1). The meaning of based is not equivalent in carbon-based capitalism and carbon-based vegetation (1), with the former expressing a relation of dependency and the latter one of composition. Thus, the interpretation of each adjective remains highly context dependent. An observation related to this tendency towards the abstract commonly expressed by based is the adjective’s frequent involvement in the description of large, intangible systems, which may be more or less regulated. The words system and systems immediately follow based fifteen times, and more specific examples of systems attested in this position include taxation (7), politics (8) and organi[zs]ation(s) (17). Social systems are composed of regular patterns of practice, and there is a marked tendency in the corpus for adjectival compounds containing based to be followed by nominal compounds containing making, such as rulemaking (follows based 7 times), policymaking (19) and decision-making . This is a strong collocational pattern, as the gerundial form making seldom occurs in the corpus without the preceding words policy or decision. In fact, the expression evidence-based decision-making has become so cliché that, as the following example from our corpus indicates, it can be toyed with for humorous purposes:decision-based evidence making, tailored to the needs of policy elites and politicians on the lookout for accessible catchphrases to woo a jaded electorate.” Tom Slater has written that in order to give ‘scientific’ credibility to extending the neo-liberal state and deepening social inequality, think-tanks such as the CSJ have “mastered the craft of The meaning of policymaking can also follow based; there are, for instance, no fewer than 55 occurrences of policy or policies immediately following based, the bulk of which are part of the larger pattern evidence-based policy and science-based policy. Alternatives to policy that occur within similar patterns include approach(es) (21) and practice(s) (8). Thus, based very frequently, and often quite explicitly, functions to express the relation between a productive field of practice and the principles that guide it. This interpretation also holds for evidence-based medicine, yet what linguistically separates this particular phrase from its more general variants is the frequent use of an acronym, namely EBM . There is, for instance, no acronym EBA, which could hypothetically refer to evidence-based approaches. The acronym EBP occurs only 17 times in the corpus, and its use is restricted to a single article . The use of acronyms is extremely common in scientific writing across various disciplines, even though in the health sciences it has been criticised as a source of “irritation, misunderstanding, and even alienation” One example is the implementation of a new procedure to treat asthma at the Royal Children’s Hospital in Victoria, Australia. The Typically, nouns referring to the outcome of regulatory procedures such as n” Moris , p. 1274based. Consequently, the example contains both a pleonastic and a tautological element, with the full formula EBM taking up the position normally reserved for evidence. The entire framework comes to serve as a substitute for the raw material: in other words, the wording may suggest that EBM occupies a singular, unambiguous conceptual space. This recalls our discussion of phrasings such as the evidence suggests in the previous section. The evidence suggests implies access to a fully transparent, univocal dataset, while the EBM-based approach implies access to a fully coherent, set framework, but in both cases the assumption of a rigid outline may be evaluated either positively or negatively. The author of the example above favours EBM and sees it as a model practice that should guide developments in other sectors. At the other end of the spectrum, some critics of the model characterise EBM’s rigid univocity as distinctly malignant, as the following example illustrates:EBM is not] medicine based on evidence, but the equivalent in the field of medicine of a cult with its unique dogma, high priest … and fervent disciples,” says Dr. John Service, editor-in-chief of Endocrine Practice. “[In the above example, the abbreviation EBM itself contains the word EBM as an oppressive and disciplinary apparatus. Similar critiques were raised in other publications, including scholarly journals, around the same time the Denialism blog post was written. Holmes et al. His book left scope for alternative views of meritocracy including positive associations of “merit based” migration of “high skill” and professional workers, while cutting humanitarian and family-reunification programs. The corruption of visa work programs might soon expand under Trump’s immigration-reform agenda: His administration seeks to expand The critique presented in this quote characterises s et al. , for insbased connects a practice to its justification, while in the second it sceptically connects a practice to its limitations. The phrases are formally equivalent while their evaluative purposes—partially signalled through the use of scare quotes—are functionally opposed, as they are in different interpretations of evidence-based medicine. Thus, researchers seeking to untangle controversies in the health sciences may benefit from examining discursive patterns that cut across a broad range of disciplines and fields of practice. The continued rejection of the term evidence-based in the context of healthcare has of course not gone unnoticed, and in recent years alternative terms have come to circulate widely. The World Health Organization (n.d.), for instance, explicitly states that evidence-based implies a restricted viewpoint, and that the institution now prefers the term evidence-informed. It is not only based that in some contexts has fallen into disrepute. In Norwegian, the term evidence-based is often translated using kunnskap, usually an equivalent of knowledge rather than evidence, in part precisely to dissociate the discourse from the “positivist connotations that are tied to the English expression” Ending AIDS requires In the first example, evidence-based to evidence-informed, indicative of a correspondent shift in practice? Positions on the subject will vary, but our corpus suggests that while medicine may attract a variety of modifiers such as alternative (27 occurrences before medicine), conventional (8), modern (8) and real (8), none offer rival paradigms to evidence-based. They may refer to a competing outlook, but not to a competing set of principles and correspondent practices. The rigid view associated with EBM thus partly consolidates its continued dominance, at least as the standard against which to argue.Yet are shifts in wording, such as the one from BMJ and The Lancet to official WHO and CDC reports, medical textbooks, as well as blog posts and Wikipedia articles on medical topics. We hope to begin compiling these corpora in 2021, with a view to making them and an accompanying software interface available to the research community as soon as practicable. Creating such new resources can afford us novel insights into some of the continued controversies and points of contention in the field.We began this article with a plea for scholars of the medical humanities to engage with a new research methodology we believe can shed light on important aspects of the evolution and contestation of a core constellation of concepts that underpin the practice and ethos of modern medicine. While the basic methodology of corpus-based CDA is well established in various areas of the social and human sciences, and to a much lesser extent in medicine, we set out to demonstrate that combining this methodology with theoretical assumptions drawn from conceptual history and social epistemology provides a powerful framework for elaborating an ambitious research agenda. We acknowledge the largely non-expert and diverse nature of our corpus and the need to supplement it with texts drawn specifically from the field of medicine. Our analysis is offered as a preliminary, pilot study that prepares the ground for the creation and analysis of large thematic corpora of expert and non-expert discourses on medicine, ranging from scientific papers published in journals such as evidence as a basic concept with wide currency in both specialist and general language to support a variety of often conflicting meanings, and therefore to continue to invite contestation. We supported our argument with an analysis of the lexical patterning of evidence, followed by an analysis of based, the second component of EBM and a modifier that may activate a range of different, and conflicting associations. Drawing on a large corpus of Internet English featuring articles and blogposts written by medical practitioners, scientists, journalists and activists of various political hues, we unpacked some of the many meanings associated with each lexical item, highlighting the futility of attempting to impose a single, rigid interpretation of what evidence-based means in the context of modern medicine. The lexical patterns we identified had implications beyond their use in a particular blogpost or by a particular medical specialist or layperson. For example, one of the patterns we identified across texts and authors is there is no evidence of. The same pattern was documented decades ago in an early corpus-based study as prevalent in medical case reports, more specifically in the context of “eliminating unsubstantiated interpretations of a symptom or disease in order to narrow down the potential number of causes” (Baker Focusing on one such point of contention, the controversy surrounding different interpretations of EBM, we argued, firstly, that it is in the nature of s” Baker , p. 103.evidence, but to elucidate why heated discussion may arise around ways of speaking about evidence and other basic concepts: when the same means of expression are used to discuss different modes of practice, interlocutors’ epistemic expectations may be violated. As both specialists and non-specialists move back and forth across different domains and are exposed to different interpretations of a basic concept, it is possible that those who reject whole paradigms of knowledge such as EBM may do so because they see them as presenting either too rigid or an overly flexible interpretation of the concept in question in relation to their own experience of its active use in a variety of contexts. Academics and professionals concerned with the promotion or contestation of medical paradigms therefore cannot ignore existing notions of evidence that circulate among different groups in society.The point is not to suggest that scholarly articles are fundamentally more transparent than blogposts in their use of evidence as a basic concept, we need to draw on a theoretical approach that incorporates both scientific and lay processes of conceptualizing and evaluating evidence. Second, examining concordance patterns means that individual knowledge becomes secondary to discursive manifestations of knowledge as a social, linguistically negotiated endeavour. Third, we have found that knowledge claims can rest, as in the case of evidence, on attributions of sufficient quality or quantity, but that such assessments are always subject to rhetorical techniques of persuasion. We have also shown that elements such as based, which may appear to be neutral or insignificant, in fact express a wide range of divergent relations. Kvernbekk (based may indeed be unclear across various notions of evidence-based practice. Our analysis has shown that conflicts of interpretation may arise between social settings and knowledge domains that alternatively understand the relation expressed in evidence-based as one of disciplinary ties, one of institutional embedding, one of fundamental principles, or one of shared practices and goals. This suggests that a robust theory of evidence-based medicine needs to account for the values and principles that are specific to the discourse within which claims and beliefs about evidence are articulated. As also pointed out by other authors (Engebretsen et al. evidence incorporating various interpretations and nuances of meaning that circulate in the everyday linguistic environment. To develop such a theory will be one of the aims of our future research.Our analysis also supports some of the basic presuppositions underpinning the social epistemology approach, as outlined in our introduction. First, the lexical patterns we identified had implications beyond their use in particular communicative settings and across the entire social fabric. Considering the nature of


Recommendations within guidelines are developed by synthesising the best available evidence; when limited evidence is identified recommendations are generally based on informal consensus. However, there are potential biases in group decision making, and formal consensus methods may help reduce these.We conducted a case study using formal consensus, to develop one set of recommendations within the Neonatal Parenteral Nutrition guideline being produced for the National Institute for Health and Care Excellence. Statements were generated through identification of published guidelines on several topics relating to neonatal parenteral nutrition. Ten high quality guidelines were included, and 28 statements were generated; these statements were rated by the committee via two rounds of voting. The statements which resulted in agreement were then used to develop the recommendations.The approach was systematic and provided transparency. Additionally, a number of lessons were learnt; including the value of selecting the appropriate topic, giving adequate time to the process, and ensuring methodologies are understood by the committee for their value and relevance.Formal consensus is a valuable option for use within guideline development when specific criteria are met. The approach provides transparent methodology, ensuring clarity on how recommendations are developed.The online version contains supplementary material available at 10.1186/s12874-021-01267-0. Guidelines are developed by the National Institute for Health and Care Excellence (NICE) to improve health and social care. The guidelines are based on the best available evidence and are produced following a detailed manual . The manThe effectiveness of group decision making using informal consensus may be open to bias . DespiteHandbook for Guideline Development’, which explicitly describes the process of formal evidence synthesis using systematic literature reviews, but does not give guidance on informal or formal consensus methods  2020.Additional file 2. Statements: Tables to show the generated statements for both round 1 and round 2 of consensus, plus example excluded statements. Used for the evidence review “General Principles” Neonatal parenteral nutrition NICE Guideline [NG154] 2020.


The absence of quantified results from digital forensic investigations, unlike those of conventional forensics, is highlighted.A number of approaches towards quantitative evaluation of the results of digital forensic investigations are reviewed.The significant potential benefits accruing from such approaches are discussed.Unlike conventional forensics, digital forensics does not at present generally quantify the results of its investigations. It is suggested that digital forensics should aim to catch up with other forensic disciplines by using Bayesian and other numerical methodologies to quantify its investigations’ results. Assessing the plausibility of alternative hypotheses  which explain how recovered digital evidence came to exist on a device could assist both the prosecution and the defence sides in criminal proceedings: helping the prosecution to decide whether to proceed to trial and helping defence lawyers to advise a defendant how to plead. This paper reviews some numerical approaches to the goal of quantifying the relative weights of individual items of digital evidence and the plausibility of hypotheses based on that evidence. The potential advantages enabling the construction of cost-effective digital forensic triage schemas are also outlined. To illustrate, the random match probability (RMP) of two matching DNA profiles not belonging to the same person  is ca.10Such quantitative measures, generally derived the results of statistical analyses or laboratory experiments, are valuable since they enable both defence and prosecution sides to evaluate the strength (or weight) of an individual item of recovered evidence (e.g. a DNA profile) and, by extension, to estimate the strength (or plausibility) of a case built from many such evidential items.This state of affairs is at least in part related to the relative maturity of conventional forensic science in comparison with digital forensics. We might tentatively trace the origin of systematic forensic science investigations to be ca.1900 with the publication of Edward Henry’s fundamental work on fingerprints , followeFred Cohen, in particular, has made significant efforts to specify the rigorous scientific and engineering principles and practices upon which the requirements for such metrics should be based . His worWhile there has been significant progress in specifying models and processes for the systematisation of digital forensic investigations, which are aimed at improving the consistency and reliability of the conclusions reached , it is iAs a point of clarification, we should note here that the great majority of the work described in this review article refers to the quantification of hypotheses (propositions or claims) based on the digital evidence, rather than the quantification of the digital evidence itself; this is an important distinction. The only instances cited here involving the quantification of digital evidence itself occur in the examples where conditional probabilities (likelihoods) are assigned to the nodes of Bayesian networks based on surveys of experienced domain experts.The formal relationship between plausibility and probability can be conveniently expressed as follows: probabilities signify the quantities that define a particular monotonic scale on which degrees of plausibility can conveniently be measured .H, with a single mutually exclusive and exhaustive alternative H̅, and recovered evidence E, Bayes' Theorem can be conveniently expressed as:Bayesian methods, based on the conditional probability theorem of Revd. Thomas Bayes in his renowned posthumously published essay , have reJudea Pearl showed how networks can be defined and constructed which permit the propagation of probabilities from initial priors to final posteriors, based on the values of intervening conditional probabilities ,20. One From 20 typical cases of internet auction fraud prosecuted in Hong Kong, China, Bayesian networks for both the prosecution and the defence cases were created and the LR of these two alternative explanations for the existence of the recovered digital evidence was computed to be 164 000 in favour of the prosecution hypothesis . This fiA third example of Bayesian network analysis involves an actual case from Hong Kong, China of a leaked confidential (Yahoo!) email; the prior probabilities were taken to be strictly noninformative and the conditional probabilities (likelihoods) were elicited by questioning a domain expert. With every anticipated item of digital evidence successfully recovered the posterior probability in favour of the prosecution hypothesis was ca.97.2%. While a credible alternative hypothesis was not available for this case against which to compare the result, it corresponds to an LR of ca.34.7 in favour of the hypothesis; however, both single-parameter and multi-parameter sensitivity analyses resulted in minimal perturbations to that value ,27.While Bayesian networks deal mainly with conditional probabilities, these quantities can on some occasions be difficult to obtain in a reliable manner. In such situations it may instead be possible to apply conventional (frequentist) probability theory to the evaluation of the plausibility of a hypothesis put forward by either the prosecution or the defence side. For example, in cases where a seized computer is found to contain a relatively small number of illicit images or videos (e.g. of child pornography) amongst a much larger number of non-illicit material (e.g. of adult pornography) it is possible to use an Urn Model ,29 and tceteris paribus) a simpler mechanism is more likely than a more intricate mechanism, then enables the odds ratio for the two alternatives to be computed as the inverse of the ratio of their complexities. In a particular scenario involving the deposition of a single 1MB image the odds against the THD were calculated to be just 2.979:1; these odds were lengthened to 197.9:1 if an up-to-date malware scanner was known to be operational at the material time [In cases where very large quantities of illicit materials are recovered from a seized computer, the Trojan horse defence (THD) ,32 is soial time . A similial time , where rial time .Knowing the relative plausibility of alternative hypotheses explaining the existence of the recovered digital evidence can be a valuable tool for aiding the investigating authority in deciding whether or not to refer a case to the prosecuting authority, and equally in assisting the prosecution authority in deciding whether or not to proceed to trial. Conversely, such quantitative information could also be made use of by the defence side in deciding how to plead: if the prosecution’s case is overwhelmingly plausible then the defendant may be advised to plead guilty whereas if its plausibility is only marginal then a not guilty plea might be entered. In some jurisdictions it is also possible that an expert witness might be permitted to bring forward such data as part of their testimony at trial.P) and without (Q) that item of evidence:The probative value  of an individual item of digital evidence in the context of a particular criminal case reflects the degree to which the presence of that item of evidence, if recovered, contributes to the overall plausibility of the hypothesis concerning the processes that created all of the recovered digital evidence. Perhaps the simplest method to achieve this is to take the difference in the posterior probabilities of the Bayesian network for the hypothesis in the presence and in the absence of that item of evidence . A seconThese three approaches lead to somewhat different orderings for the evidential weights of the BitTorrent case mentioned above .In digital forensic investigations it is commonly the case that the individual items of digital evidence may, at least to a first approximation, be considered conditionally independent of one another. Knowing the relative importance of each individual item of digital evidence in an investigation can then enable an efficient digital investigation scheme to be devised in which the most highly probative evidential items are searched for first, in order, relegating the items of lesser importance until later. If one or more of the anticipated items of high importance are not recovered the search may be de-prioritised or even abandoned; conversely, if all of the anticipated items of high importance are found then it may not be considered necessary to search for those items of lowest importance as the overall plausibility of the investigative hypothesis would not be sensibly improved by doing so.In addition, such priority-driven investigation schemes can be termed cost-effective if they are extended to utilise economic criteria such as Return on Investment (RoI) or Cost–Benefit Ratio (CBR), through a knowledge of the resources  required to recover and analyse each anticipated item of digital evidence ,40.We have endeavoured to make the case that digital forensics should aim to catch up with conventional forensics in providing quantified results from its investigations, despite the intrinsically sensitive and volatile nature of much digital evidence. A number of methodologies, such as Bayesian networks, complexity theory, probability theory and statistics, and information theory, by which this may be accomplished have been outlined and some typical results have been summarised. The uses to which such quantitative results could be put in both investigative and juridical contexts have also been briefly explored. In particular it should be emphasised that by bringing digital forensics into line with conventional forensic science, expert witness testimony and examiners’ investigation reports can offer the courts a transparent rationale for the degree of certainty associated with their conclusions, rather than relying on previous relevant experience as the sole determinant of their expert opinion.Ongoing and future lines of research in this area are likely to involve comparative studies of different measures of complexity  as appli


When the history of the COVID-19 pandemic is written, it is likely to show that the mental models held by scientists sometimes facilitated their thinking, thereby leading to lives saved, and at other times constrained their thinking, thereby leading to lives lost. This paper explores some competing mental models of how infectious diseases spread and shows how these models influenced the scientific process and the kinds of facts that were generated, legitimized and used to support policy. A central theme in the paper is the relative weight given by dominant scientific voices to probabilistic arguments based on experimental measurements versus mechanistic arguments based on theory. Two examples are explored: the cholera epidemic in nineteenth century London—in which the story of John Snow and the Broad Street pump is retold—and the unfolding of the COVID-19 pandemic in 2020 and early 2021—in which the evidence-based medicine movement and its hierarchy of evidence features prominently. In each case, it is shown that prevailing mental models—which were assumed by some to transcend theory but were actually heavily theory-laden—powerfully shaped both science and policy, with fatal consequences for some. After careful enquiry, we see no reason to adopt this belief. We do not find it established that the water was contaminated in the manner alleged; nor is there before us any sufficient evidence to show, whether inhabitants of the district, drinking from that well, suffered in proportion more than other inhabitants of the district who drank from other sources .The authorities concluded that there was no evidence cholera was waterborne. As quoted in a relatively recent reprint :it has bThis excerpt illustrates the powerful impact of an assumed theoretical model of causality on scientific thinking (and thence to policy decisions). The local public health authorities already ‘knew’ the mechanism of spread, so they found ‘no evidence’ to support an alternative theory—partly because they failed to look for it and partly because they overlaid empirical evidence with their existing mental model. It is not entirely clear why they believed that their own theory fitted the data obtained—but as Barnes & Bloor  observedLancet commented that ‘in riding his hobby [horse] very hard [Dr Snow] has fallen down through a gully-hole and has never since been able to get out again’ [Lancet in 1858 did not even mention his contribution to cholera [Using the rhetoric of scholarship (“after careful inquiry…”), but without actually rebutting Snow's arguments, the authorities of the day depicted Snow as unrigorous and mistaken. Academic peers were equally scathing in their reviews. While removal of the Broad Street pump handle is now seen as a landmark intervention in the history of public health, at the time the editor of the t again’ . The two cholera . It was  cholera ,16.. 3Those tempted to view the snail's pace march of science in nineteenth-century public health as a phenomenon of yesteryear should first consider how empiricist-dominated mental models continue to both shape and constrain science. One such model—evidence-based medicine (EBM)—has had particular influence in the COVID-19 pandemic. In the early 1990s, a group of epidemiologically trained doctors—the forerunners of the EBM movement—challenged the traditional way clinical decisions were made  and introduced decision-making based on empirical evidence from randomized controlled trials (RCTs)—carefully controlled experiments in which participants are randomly allocated to ‘intervention’ and ‘control’ groups with systematic follow-up and measurement of predefined quantitative outcomes .bias—that is, influences that could ‘deviate the results or conclusions … systematically away from the truth’ [correctly by following agreed methodological procedures. There was little or no recognition that facts are theory-laden and truth perspectival—for example, as scientists’ analytic gaze, and the questions they choose to ask, are shaped by the mental models with which they approach and communicate about the world [Initially seen as bold and heretical, EBM quickly became the new orthodoxy. Its declared mission was to strengthen medicine's empirical foundations while reducing its reliance on theoretical reasoning . Centrale truth’ . ‘Truth’he world .hierarchy of evidence with RCTs and meta-analyses of RCTs at the top and the so-called lesser forms of evidence below it [EBM has long promoted a below it . This isbelow it .explanations when selecting and interpreting empirical evidence, is a rule that is honoured more in the breach than in the observance. The World Health Organisation's published summary of its guideline development process, for example, emphasizes EBM's hierarchy of evidence, RCTs and the over-riding need to eliminate bias, but makes no reference to explanatory theory [Not everyone who aligns broadly with the EBM movement agrees that method should always and necessarily be privileged over theory. Indeed, there is a rich seam of philosophical writing on the need for EBM's ‘gold standard’ RCTs to be informed by causal explanations –25, and y theory .The hierarchy of evidence, and the meticulous methodological rules and procedures developed to extend it, saved many lives during the pandemic. They facilitated the generation, at impressive speed, of definitive evidence from RCTs and meta-analyses on the efficacy and safety of drugs and vaccines ,31. GiveBut as the next section describes, the same rules and procedures—and the empiricist assumptions which underpinned them—were used to justify a more questionable approach to research on the effectiveness of preventive measures such as facemasks in controlling spread of the virus.. 4Physical interventions to interrupt or reduce the spread of respiratory viruses, originally published in 2011 [Along with other non-pharmaceutical interventions, facemasks were included in a Cochrane systematic review,  in 2011 . When th in 2011 . Unlike The style and tone of the Cochrane review was what Bourdieu , followiA group of Danish researchers, also hospital doctors trained in the EBM tradition and Cochrane Collaboration members, sought to fill the evidence gap flagged by Jefferson and Heneghan in their Cochrane review. They designed a masks-on versus masks-off RCT, which—after some months’ delay—was published in a leading medical journal . The triBoth the Jefferson systematic review and the Heneghan–Jefferson commentary seem to reflect a mental model of extreme caution when introducing new treatments. But compared to a new drug or vaccine, the risk of serious harm from facemasks is extremely low, and the potential for benefit at population level could be high. Hence, it could be argued that the usual reasons for advocating caution in clinical trial research do not hold. Indeed, because of the very different balance of probabilities, there are strong arguments for reversing the usual assumption that avoiding harm is more important than striving for benefit. We should, perhaps, adopt the precautionary principle and recommend this intervention ‘just in case’ . In phil. 5The assumptions which placed the RCT in a bias-free class of its own have rightly been challenged . Particuto the exclusion of other kinds of evidence, and that is the assumption, based on what might be called naive empiricism, that data can be identified, collected, analysed and summarized without the need for theory. Academics in many other scientific disciplines emphatically reject the assumption that controlled experiments should always and necessarily over-ride mechanistic evidence, defined as evidence produced by multiple different methods which help illuminate and explain phenomena at a theoretical level [But there is a more fundamental—i.e. philosophical rather than methodological or practical—objection to the emphasis on RCTs al level ,41.inferior but complementary to evidence from RCTs (Medawar's ‘bride’ and ‘groom’). In order to refine an intervention to maximize its potential impact, we need to understand, at a theoretical level, the chain of causality linking intervention to outcome. While well-conducted RCTs may have high internal validity (i.e. they can produce strong evidence for the population from which the sample was drawn), their external validity may be low (i.e. their findings may not apply to other populations or settings). Mechanistic evidence allows scientists to elucidate the different steps in the causal pathways that help us anticipate why an intervention which worked in one setting is also likely work in a different setting—and also to reason why an intervention that did not work in one setting could still have an important contribution to a programme of interventions somewhere else.Mechanistic evidence is, arguably, not summate results, as in the Cochrane Collaboration's empirically driven systematic reviews [explain mechanisms and enrich our understanding, as in more theory-driven narrative reviews [et al. depict the synthesis of mechanistic studies (the more heterogeneous the better) to produce a sense-making narrative review as the building of a dry stone wall—an artisan craft in which each stone is carefully selected to make a unique contribution to fill a particular shaped gap.For all these reasons, to build a robust knowledge base about interventions, and depending on the precise circumstances, it is sometimes necessary to draw on a wide range of evidence, both mechanistic and experimental, and use review methods that do not merely  reviews , but als reviews ,43. OgilI undertook, with colleagues, a literature review on the benefits and possible harms of facemasks . Our rev. 6During 2020, in an ironic reversal of the paradigmatic battle between airborne (miasma) and waterborne explanations for the spread of cholera in the nineteenth century, key advisory groups including the WHO, US Centers for Disease Control and Prevention, European Centre for Disease Control and Public Health England all assumed that the dominant mode of transmission of SARS-CoV-2 was respiratory droplets. Just as in the 1850s, policymakers assumed the mode of transmission rather than seeking empirical demonstration of it. They dismissed claims from people who argued that the virus was—or could be—significantly spread through the air. And they ignored what philosophers of science might call ‘black swan’ evidence  such as super-spreader events. The ill-fated performance of Bach's St John Passion in Amsterdam's Concertgebouw auditorium in March 2020, for example, soon after which 102 of 130 choir members developed symptoms of COVID-19 and four people died, simply cannot be explained by an exclusively droplet mode of transmission .Droplets emitted in coughs and sneezes are relatively heavy and fall to the ground or onto surfaces quickly. A droplet mode of spread means that transmission of the virus will occur only when in close proximity to others—1 m according to the WHO; 1.5 or 2 m according to other bodies—and also via our hands  and fomites . ‘Contact and droplet mitigation strategies’ (preventive efforts built on droplet theory) include the physical distancing of shoppers in queues plus frequent and assiduous washing of hands and wiping-down of surfaces. Masks, according to the droplet hypothesis, need only be worn when physical distancing is impossible, and are viewed as a relatively minor component of the prevention package.both at close range and at longer distances (via ‘shared air’), and that the chemistry of air composition, the physics of air flow and the architecture of the built environment are all influential in the model of spread. Using this mental model, masks, worn by everyone at all times when inside a building or vehicle, were a key factor that could keep the air virus-free—which is why, even before 2020, many people in Asian countries wore masks outside the home. Japan's highly successful ‘3Cs’ prevention policy—avoid crowded places, closed spaces and close contact—was based on an assumed airborne route of transmission [Despite the WHO's firm insistence on the droplet theory of transmission, this mental model was not universally accepted. Indeed, in China where the disease had emerged, the prevailing theory was based on recent historical experience with the SARS and MERS viruses, both of which had been shown to be airborne . An airbsmission .But in the West, and throughout most of 2020, policy thinking was dominated by the empiricist conclusion that there was ‘no evidence’ for the efficacy of masks and by speculation, based on a droplet model of spread, that both risk compensation and touching the mask could cause harm . TellingBecause different mental models informed very different policies in different countries, a vast natural experiment resulted. The findings were striking. Countries which had recommended facemasks for the public in the first 30 days from the first documented case  had, on average, orders of magnitude fewer deaths from COVID-19 than countries which delayed such recommendations beyond the first 100 days . But jus. 7from surfaces has been unsuccessful and the virus can remain viable in air for several hours and that under certain environmental conditions  it can travel many metres and persist for hours. They also argued that airborne transmission is strongly suggested by well-documented super-spreader events (such as singing performances) and nosocomial outbreaks . The conclusions of these early reviews have been affirmed and strengthened by more recent summaries of the evidence [The WHO's position on prevention of COVID-19 is based largely on advice from its Infection Prevention and Control Research and Development Expert Group for COVID-19. Most of its members are clinicians with a background in hospital-based infectious diseases and training in EBM; they are experts in topics such as wound management—for which droplet spread is predominant and handwashing is an effective intervention . Throughevidence ,54.lack of evidence in favour of aerosol spread with evidence refuting aerosol spread.Back in June 2020, over 200 aerosol scientists from around the world published an open letter addressed to international policymaking bodies summarizing studies undertaken by its signatories which had demonstrated ‘beyond any reasonable doubt’—so long, one might add, as one accepts the validity of mechanistic evidence—that the SARS-CoV-2 virus is released in tiny microdroplets small enough to be carried long distances in the air when people talk, cough and even just exhale . Yet a f. 8possibility of truth that distinguishes the scientific imagination from the fanciful. The notion of the purity of German blood, for example, is a theory of sorts but not a scientific one, whereas waterborne transmission of cholera is a scientifically testable theory.While the Vienna Circle of logical positivists did not reject theoretical reasoning altogether, they placed little emphasis on it and identified some kinds of theoretical reasoning as potentially sinister and anti-science. They sought, for example, to purge science of German romanticism, a mental model which speculated that there was something pure and good about ‘German blood’, as they saw it serving the cause of political and ideological groups  . But theDuring the COVID-19 pandemic, libertarian groups drew heavily on what they took to be objective empirical data  and rejected mechanistic explanations based on indirect—and, they felt, low-quality—evidence. To a greater or lesser extent, people who aligned with the libertarian movement took the view that recommendations to stay home, maintain 2 m distance, wear facemasks and even get vaccinated were unwelcome intrusions of the state. They believed that segmentation should be practised instead of lockdown , that facemasks were harmful and an unacceptable infringement of personal freedom, and that this essentially mild disease should be allowed to wash over the population to achieve what was termed ‘herd immunity’ . This coOne explanation for why ideology and mental models became entangled in this example is that the Danmask RCT represented the closest to gold-standard evidence in the mental models that prevailed among the EBM community—namely, it used a study design from the top of the hierarchy of evidence and was ‘unbiased’. The libertarians were simply ‘following the science’ when they seized on the Danmask trial. But this does not explain why the trial's evident flaws were not acknowledged by many senior members of the EBM community. The Danmask trial, for example, was published without a CONSORT statement—the internationally agreed methodological checklist required by many journals as a condition for accepting a RCT for publication . UnusualA second explanation, then, is that the flawed RCT evidence on facemasks was either wilfully or unconsciously misinterpreted. Because one possible interpretation of its findings is that facemasks have no effect, ideological mental models came to align with scientific hypotheses in the minds of some libertarians (both lay and academic), who quickly deemed those hypotheses to be supported by ‘robust’ (i.e. RCT) evidence. While Medawar's writings imply that a scientific hypothesis can be readily distinguished from the fanciful, the facemask example suggests that this is not universally the case.A reviewer of an earlier draft of this paper suggested that opposition by some individuals to the evidence on the efficacy of facemasks might be explained more in terms of sheer stubbornness than by recourse to mental models. This comment raises the question of whether the tendency to mistake ideological hypotheses for scientific ones may stem from intellectual vices. As Quassim Cassam has observed, academic effort can be understood in terms of intellectual virtues—defined as aspects of mind that promote effective and responsible intellectual inquiry, such as carefulness, flexibility, open-mindedness, conscientiousness and creativity—and intellectual vices—defined as aspects of mind that inhibit effective and responsible intellectual inquiry, such as excessive conformity, carelessness, rigidity, prejudice, closed-mindedness, dogmatism, complacency and arrogance . The urg. 9In this paper, I have argued that both the long delays in replacing flawed, miasma-driven approaches to cholera prevention in the nineteenth century and long delays in replacing an exclusively contact-and-droplet model of SARS-CoV-2 prevention with one that includes airborne transmission in the twenty-first both had a philosophical explanation in terms of which mental models of reality prevailed and the extent to which scientists and policymakers favoured data over theory. In the more recent example, ideological movements in the West drew—eclectically—on statements made by scientists, especially the confident rejection by some members of the EBM movement of the hypothesis that facemasks reduce transmission.The WHO changed its stance to recommend the more extensive use of facemasks by the lay public in December 2020 . By earlWhile I disagree with the scientists who reject the airborne theory of SARS-CoV-2 transmission and the evidence for the efficacy of facemasks, they should not be dismissed as ideologically motivated cranks. On the contrary, I believe their views are—for the most part—sincerely held and based on adherence to a particular set of principles and quality standards which make sense within a narrow but by no means discredited scientific paradigm. That acknowledged, scientists of all creeds and tribes should beware, in these fast-moving and troubled times, of the intellectual vices that tempt us to elide ideology with scientific hypothesis.