This rapid evidence review identifies and integrates evidence from epidemiology, microbiology and fluid dynamics on the transmission of SARS-CoV-2 in indoor environments.Searches were conducted in May 2020 in PubMed, medRxiv, arXiv, Scopus, WHO COVID-19 database, Compendex & Inspec. We included studies reporting data on any indoor setting except schools, any indoor activities and any potential means of transmission. Articles were screened by a single reviewer, with rejections assessed by a second reviewer. We used Joanna Briggs Institute and Critical Appraisal Skills Programme tools for evaluating epidemiological studies and developed bespoke tools for the evaluation of study types not covered by these instruments. Data extraction and quality assessment were conducted by a single reviewer. We conducted a meta-analysis of secondary attack rates in household transmission. Otherwise, data were synthesised narratively.We identified 1573 unique articles. After screening and quality assessment, fifty-eight articles were retained for analysis. Experimental evidence from fluid mechanics and microbiological studies demonstrates that aerosolised transmission is theoretically possible; however, we found no conclusive epidemiological evidence of this occurring. The evidence suggests that ventilation systems have the potential to decrease virus transmission near the source through dilution but to increase transmission further away from the source through dispersal. We found no evidence for faecal-oral transmission. Laboratory studies suggest that the virus survives for longer on smooth surfaces and at lower temperatures. Environmental sampling studies have recovered small amounts of viral RNA from a wide range of frequently touched objects and surfaces; however, epidemiological studies are inconclusive on the extent of fomite transmission. We found many examples of transmission in settings characterised by close and prolonged indoor contact. We estimate a pooled secondary attack rate within households of 11%  = 9, 13). There were insufficient data to evaluate the transmission risks associated with specific activities. Workplace challenges related to poverty warrant further investigation as potential risk factors for workplace transmission. Fluid mechanics evidence on the physical properties of droplets generated by coughing, speaking and breathing reinforce the importance of maintaining 2 m social distance to reduce droplet transmission.This review provides a snap-shot of evidence on the transmission of SARS-CoV-2 in indoor environments from the early months of the pandemic. The overall quality of the evidence was low. As the quality and quantity of available evidence grows, it will be possible to reach firmer conclusions on the risk factors for and mechanisms of indoor transmission. It is well established that SARS-CoV-2 is readily transmitted in indoor environments; however, questions remain about the relative importance of different transmission mechanisms, the risks associated with non-clinical indoor environments and activities and the role of ventilation and plumbing systems in mitigating or amplifying transmission. Although other reviews address aspects of these questions -3 there The purpose of this review is to integrate evidence from epidemiological, microbiological and fluid mechanics studies on the transmission of SARS-CoV-2 in indoor, non-clinical settings in order to answer ten specific questions:What evidence is there for aerosolised transmission?What evidence is there for faecal-oral transmission?What evidence is there regarding the role of ventilation systems in indoor transmission?What evidence is there regarding the role of plumbing systems in indoor transmission?What evidence is there regarding transmission via different indoor surfaces ?What evidence is there for the transmission in indoor residential settings?What evidence is there for transmission in indoor workplace settings?What evidence is there for transmission in other indoor settings ?Do particular activities convey greater risk ?What evidence is there for the appropriate length of distancing between people?We designed two separate search strategies: one to identify epidemiological and microbiological papers and the other focused on fluid mechanics papers (mechanistic studies). We searched PubMed, medRxiv, arXiv, Scopus, WHO COVID-19 database, Compendex & Inspec. Searches were collaboratively developed by two reviewers (MD and LG) and results exported on 20 and 21 May 2020. Full search details are in Appendix S1 in the We included studies reporting data on any indoor setting except schools, which is addressed elsewhere in a living systematic review , any indEpidemiological studies: We excluded studies of transmission within clinical settings and studies focusing purely on the clinical characteristics of cases. We also excluded statistical modelling studies aiming to predict future outcomes, as opposed to descriptive studies characterising past events.Microbiological studies: We included studies involving the testing of swabs taken from “real world” settings for the presence of SARS-CoV-2. As most of these were conducted in hospital settings, we included studies from both clinical and non-clinical settings. However, to maximise the transferability and generalisability of these findings to non-clinical indoor settings, we excluded microbiological studies of samples collected in areas of the hospital such as operating theatres and Intensive Care Units (ICU) where aerosol-generating procedures are routinely carried out. We also included laboratory studies investigating the persistence and viability of the virus under different controlled conditions.Mechanistic studies: We included articles reporting data on any respiratory virus, numerical simulation studies focusing on the mechanisms of transmission and studies investigating any mechanisms with potential to influence transmission in indoor environments, such as ventilation, air conditioning or plumbing systems.Articles were screened by two separate teams . Title and abstract and full text screening were conducted by one reviewer within each team, with rejections assessed by a second reviewer.Data extraction and quality assessment for each article was conducted by a single reviewer, as above. A range of critical appraisal tools was employed, according to study design: case series and case reports were evaluated using Joanna Briggs Institute checklists . We adaprma.uni function in the metafor package [2 and Cochrane’s Q were calculated to assess heterogeneity. For consistency, the same function was used to estimate confidence intervals for SAR in individual studies that were not included in pooled estimates. A fixed effects analysis was chosen because the number of studies was relatively small, and thus a simpler underlying model (fewer assumptions/parameters required) was likely to be better estimated: in addition, there is reasonably good theoretical and simulation evidence that fixed effects models are relatively robust to moderate heterogeneity [Data heterogeneity was such that results were synthesised narratively, except for the results on secondary attack rates within households, which were meta-analysed using a fixed effect model in R 3.6.3  using th package . I2 and ogeneity . There wAfter the removal of duplicates, a total of 1573 articles were identified. A total of 1447 were rejected through title and abstract screening and a further 68 were rejected at the full-text screening stage and quality assessment stage. Forty-one did not provide data relevant to study questions, 26 were poor quality and one article could not be retrieved. Fifty-eight articles were retained for analysis  and in the care home (P < 0.00001). We found two workplace studies which did not present sufficient data to estimate SARs but nevertheless provide insight into workplace transmission  difference in the numbers of droplets emitted between each of these levels and the next -79. TherThe four studies we found which report on transmission in communal contexts are consistent with the conveyance of risk through close contact daily living activities. It is striking that the SAR reported in the care home  is an orOur findings are consistent with the hypothesis that the main route of CoV-2 transmission is through person-to-person short-range transmission, which occurs through large respiratory droplets ejected while speaking, coughing and sneezing. The distance that these respiratory droplets travel before falling to the ground depends on (among other factors) how they were generated. The physical behaviour of droplets is well characterised: those generated by speaking fall to the ground within 1 m or closer to the speaker ; dropletThis rapid evidence review integrates evidence from epidemiological, microbiological and fluid mechanics perspectives on the transmission of covid-19 in indoor settings. We found epidemiological, mechanical and microbiological evidence consistent with person-to-person, short-range spread via mostly respiratory droplets that directly reach recipients either through the air or through touching contaminated surfaces and then transferring the virus on the hands to mucosal membranes. Evidence from numerical simulation and fluid mechanics studies, microbiological laboratory studies and environmental sampling studies suggest that aerosol transmission is theoretically possible and is another potential source of transmission but we did not find conclusive epidemiological evidence to confirm this. However, evidence from fluid mechanics experiments and numerical simulations indicate that ventilation can play an important role in reducing disease transmission through diluting and dispersing the concentrations of viral particles in the air. Although viral RNA can be detected in faeces of affected individuals, we found no evidence for the presence of live virus in faecal samples nor for transmission through infected faeces.Evidence from household, communal residential, community and workplace settings suggests that close and prolonged physical contact is important in transmission dynamics. Within households, the risk of transmission was higher between spouses than between other types of relative. Community and social settings associated with a higher risk of transmission are also those where people gather in close proximity indoors for prolonged periods. Churches and religious gatherings, sharing meals and bathing facilities, close physical contact and activities such as singing together have all been reported in conjunction with outbreaks. In contrast, there have been fewer reports of transmission in relation to more casual, short term social contact, although this may be because such contacts are subject to recall bias and harder to track and trace. Many of the workplace settings where outbreaks have occurred are characterised by close physical contact and prolonged time spent in crowded indoor spaces. Evidence from the study on outbreaks in meat and poultry processing plants  also higTo our knowledge, this is the only review focusing on indoor transmission across different indoor contexts and combining evidence from epidemiological, microbiological and fluid mechanics studies. The key advantage of bringing together evidence from different disciplines in this way is that it enables practical issues that are of direct and immediate importance to decision-makers to be addressed. Three recently published systematic reviews address similar questions: Koh and colleagues estimated a pooled secondary attack rate in household settings of 18.1%  – somewhat higher than our estimate, potentially reflecting the small number and poor quality of the primary studies we found on this topic. . This stOur findings have several implications for researchers, policymakers and the general public. First, we highlight important gaps in the evidence base: although aerosolised transmission is theoretically possible, whether this actually occurs in non-clinical indoor settings remains uncertain and many questions remain unanswered. It is still not known what quantity of live virus is required to present an infection risk or whether live virus is present in sufficient quantities in aerosolised particles to present a risk. Further research on these questions is urgently warranted. Although we excluded evidence from animal studies in this review, such studies should be included in future reviews, as they can potentially provide direct experimental evidence on airborne transmission . Second,This review has a number of limitations. Although the focus of this study is transmission of SARS-CoV-2 in indoor, non-clinical settings, most of the microbiological and environmental evidence was generated in clinical contexts because this is where most of this type of study have been conducted to date. Clearly such settings are very different from non-clinical, community contexts: for example, there is a higher risk of transmission via aerosol generating procedures (AGP) and greater numbers of individuals infected with SARS-CoV-2, so virus detection in these settings is likely higher than in non-clinical indoor settings. To maximise the transferability and generalisability of these findings to community settings, we attempted to extract and report only on samples taken from areas of hospitals accessible to visitors and the general public; however, this was not always possible, as the studies did not provide information on the extent to which AGPs were carried out in patient rooms. Therefore, these results must be treated with caution in applying them to non-clinical settings.The quality of the available epidemiological evidence was poor, so this makes any conclusions uncertain. In particular, there is significant variability in contact tracing approaches across different countries and even different regions within countries. Contact tracing of rapidly evolving infectious diseases inevitably contains case ascertainment biases, non-homogenous sampling over time and location, and uncontrolled correlation . There m


With our work we want to support better, evidence‐based decisions by: (a) supporting rigorous synthesis of evidence on key questions; (b) further improving the quality of climate change research through critical appraisal of primary research; and (c) improving the efficiency of the scientific process through enhanced oversight of, and less duplication in, secondary research efforts. We further believe that our work strengthens the IPCC and other climate change assessments by supporting the completion of its evidence pyramid  in resource constrained contexts like the Global South. Our aim is, therefore, to provide accessible and functional training in climate evidence synthesis that works across settings .3It is an exciting and important task to build a thriving  community that tries to learn what climate solutions work under what conditions, for whom, and why, using rigorous systematic review and evidence (gap) map methodologies. Beyond the core remit of the group to promote rigorous evidence synthesis as an important pathway to more evidence‐informed climate policy, CSCG will further try to advance evidence synthesis methodology more generally in four priority areas.Promote the adaptation and application of qualitative and mixed methods evidence synthesis methodologies: The social science of climate change provides a very diverse body of evidence. Only a small part of this literature is directly on impact evaluations of climate solutions based on experimental or quasiexperimental research designs IPCC, . In partEncourage the development and application of novel evidence synthesis technology: Climate change is not only one of the most pressing policy issues of our time, but also a very prolific area of research with tens of thousands of studies being published every year. In such an age of big literature with a vast and fast‐growing evidence base, traditional manual tools of evidence synthesis are being pushed to their limits (Callaghan et al., Integrating the evidence synthesis traditions of the climate and the systematic review communities: It is clear that comprehensive learning on climate solutions must be rooted in a systematic exploration of alternative future pathways as well as systematic evaluation of past policies. The CSCG brings together two evidence synthesis communities with complementary strengths in ex‐ante and ex‐post assessments of climate solutions. We are convinced that there is great scope for mutual learning and that sharing and integration of these distinct traditions will ultimately lift evidence synthesis and the resulting scientific policy advice to a higher level.Promote the application of formal assessment of confidence in evidence within syntheses of climate solutions: Assessment of uncertainty and confidence in evidence is a critical component of climate change assessments (Janzwood, Climate change is one of the most pressing problems that humankind faces. We need to urgently and rapidly learn from the best available evidence on climate solutions. The CSCG warmly invites registrations and submissions to Campbell Systematic Reviews from climate experts across diverse disciplines and evidence synthesis practitioners.


Literature-based gene ontology (GO) annotation is a process where expert curators use uniform expressions to describe gene functions reported in research papers, creating computable representations of information about biological systems. Manual assurance of consistency between GO annotations and the associated evidence texts identified by expert curators is reliable but time-consuming, and is infeasible in the context of rapidly growing biological literature. A key challenge is maintaining consistency of existing GO annotations as new studies are published and the GO vocabulary is updated.In this work, we introduce a formalisation of biological database annotation inconsistencies, identifying four distinct types of inconsistency. We propose a novel and efficient method using state-of-the-art text mining models to automatically distinguish between consistent GO annotation and the different types of inconsistent GO annotation. We evaluate this method using a synthetic dataset generated by directed manipulation of instances in an existing corpus, BC4GO. We provide detailed error analysis for demonstrating that the method achieves high precision on more confident predictions.Two models built using our method for distinct annotation consistency identification tasks achieved high precision and were robust to updates in the GO vocabulary. Our approach demonstrates clear value for human-in-the-loop curation scenarios. GO information, which includes GO terms and their definitions or descriptions, and supportive evidence, which includes the coding regions on a genomic sequence, or a reference to a document describing experimental findings relating to gene product function. Sequence-based GO annotations are produced by comparing different sequences drawn from coding regions; this process can transfer GO terms from an old sequence to a new sequence because two genomic sequences with structurally similar coding regions tend to produce gene products with similar functions  evidence text [SEP] GO term + GO definition [SEP]” or “[CLS] evidence text [SEP] evidence code [SEP]” as input to BioPubMedBERT. We model the different types of (in)consistencies in two ways: (1) in a multi-class setting (Model-Term) that aims to distinguish between consistent, Type A, Type B and Type C inconsistencies by comparing evidence information with GO information; (2) and in a binary setting (Model-Code) for distinguishing between consistent and Type D inconsistencies by comparing evidence information with evidence code. In the basic system, two models are fine-tuned on training and development sets generated using random GO descendant selection based on the strategy mentioned in “We model each generated instance as a paired sequence of “In the system with training set optimization, alternative Model-Term and Model-Code variants are fine-tuned on a collection of training and development set using the term overlap weighting strategy introduced in “consistent and unsupportive instances is illustrated in Table contradictory, over-specific, and erroneous-code instances are generated without manipulating the evidence text from the original consistent GOA instances. Thus, they retain the canonical distribution of evidence section information with Consistent GOA. The unsupportive instances are generated by replacing the original evidence text with unsupportive evidence sentences and therefore have different distribution of section information from consistent GOA. We concatenate the 1-dimension section encoding with 768-dimension [CLS] encoding in BioPubMedBERT’s last hidden layer and forward to a linear layer in Pytorch consistent GOA using the baseline setting and basic system. We then run additional experiments using training set optimisation and evaluate the impact of the addition of evidence section information using he AdamW  optimisehe AdamW  frameworPrecision for consistent GOA recognition is 1 at the article level but 0.5 at the evidence level. Precision is appropriate for interpreting model’s performance because the same GO term may be assigned to multiple evidence sentences in the same article. Once a single inconsistent GOA is detected in an article, all GOA that are linked to that article will be forwarded to curators for further inspection. Although there may be other inconsistent GOA linked to that article that were not detected by the system, they will still be manually reviewed. Thus, the detection of only one inconsistent GOA effectively achieves a perfect Recall at article level. A low volume of false negatives—missed (in)consistent GOA within a single article—can be tolerated in a real-world human-in-the-loop curation scenario.In contrast to the existing literature-based GOA resources in GO consortium database where GO annotation is linked to evidence at the article level, we grounded annotation evidence to sentences via the BC4GO corpus. This strategy can better reflect the model’s ability to detect (in)consistent GOA. Considering an article that has two evidence sentences supporting the annotation of the same GO term, where the first evidence sentence is correctly identified as being consistent to the GO term and the second evidence sentence is not, the Recall and We also use H) of each predicted label (i) using Shannon entropy; see Eq. ith (in)consistency type.To support further analysis of Model-Term’s performance, we calculate the uncertainty , or in aggregate across a collection (Per-Collection). Some metrics only support (Per-Collection) analysis as the aggregated uncertainty value of each sampled collection equals to the value of hyper-parameter The Pearson correlation coefficient between the scores on these metrics and prediction uncertainty can then be investigated to provide insight into how uncertainty varies with linguistic characteristics. This analysis can be done either by taking the sum of all the instances metric scores and divided by the number of instances within a collection (Either per-instance or per-collectionGOLen: The count of tokens in a GO term split by the blank sign.#GOLen-2 | feeding behavior (GO:0007631)AlignRatio: The count of tokens in evidence text divided by the count of tokens in GO definition text.#AlignRatio-0.32 | Evidence: CeCDC-14 and ZEN-4 are interdependent in their localization | GO definition: Any process that modulates the frequency, rate or extent of any process in which a protein is transported to, or maintained in, a specific location.GEORatio: The count of word overlaps between GO term and evidence text divided by the GOLen.#GEORatio-0.25 | GO term: regulation of protein localization | Evidence: CeCDC-14 and ZEN-4 are interdependent in their localization Per-Collection%ContainRoman: The percentage of instances that has the occurrence of Roman numerals in the GO term in each sampled collection.#%ContainRoman-0.5 | feeding behavior (GO:0007631) | photosynthesis, light harvesting in photosystem(GO:0009769)%ContainDigit: The percentage of instances that has the occurrence of digital numbers 09 in the GO term in each sampled collection.#%ContainDigit-0.5 | cellular response to interleukin-(GO:0071347) | cellular response to peptide (GO:1901653)%ContainStop-OF: The percentage of instances that has the occurrence of stopword “of” in the GO term in each sampled collection.5#%ContainStop-OF-0.5 | feeding behavior (GO:0007631) | regulationfeeding behavior (GO:0060259)Precision for Model-Term and 0.82 Precision for Model-Code. The training set optimisation and the addition of evidence section information further contribute to improving the Precision  and Type C (evidence unsupportive) GOA Table . Howeverconsistent and over-specific GOA during evaluation on the test set  are paired with the same piece of evidence text, the model has difficulty discriminating between them. This is reflected in the fact that most of the error cases are caused by mis-categorisation between et Table .Table 7Tconsistent GOA from inconsistent GOA  inconsistencies can appear anywhere in the document.The evidence section information is a strong indicator for discriminating Consistent GOA from other types of inconsistencies. It outperforms the basic system but can also cause biases as the mixture of od Table . This isod Table . The conThe model is effective in assisting GOA inconsistency detection although it is not effective in distinguishing different types of inconsistencies Table . In realErroneous Code (Type D) inconsistencies. This is because the relationship between an evidence code and evidence text is more complex than the relationship between a GO definition and evidence text. For example, in Type A inconsistencies, there are many lexical alignments from “negative regulation” within GO terms to “decrease”, “prevent”, “deactivate” within evidence texts. In Type B instances, over-specific GO terms often have large term overlap with the correct GO term. The pairwise semantic relation patterns of GO definition and evidence text in Model-Term are restricted. However, the Model-Code input of pairwise relations between evidence code and evidence texts do not have any text alignments or term overlaps. The assessment of consistency between the two requires a comprehensive knowledge inference process which relies on both the identified evidence text and prior knowledge or other text in the article. For example, the decision of whether an evidence text such as “Here we show that a knock-out of the ybeB gene causes a dramatic adaptation block during a shift from rich to poor media and seriously deteriorates the viability during stationary phase. YbeB of six different species binds to ribosomal protein L14. This interaction blocks the association of the two ribosomal subunits and, as a consequence, translation” should be labelled as “IDA” [PubMed Central article PMC3400551] is decided based on information such as whether or not there is a genetic mutation or allele variation, a 1-on-1 physical interaction, or the expression pattern of gene product. It requires that the result be determined through direct assay for the function, process, or component of the gene product. This requires a sophisticated process of considering the evidence text in relation to several decision rules rather than a direct association between the text and the evidence code.Model-Code was not successful at identifying consistent and Type C; the differences of text length between GO definition and evidence text potentially influence the model’s uncertainty in recognising Type-C GOA; the occurrence of Roman numerals and digits in the GO term demonstrate possibilities in influencing the prediction uncertainty of consistent and Type-C GOA as well.Model-Term demonstrated strong potential for feasibility for real-world GO curation, particularly for the more confident predictions, as shown in Fig. Catalysis of the reaction: geranyl diphosphate + isopentenyl diphosphate = 2-trans,6-trans-farnesyl diphosphate + diphosphate.” We found that over We found a small number of error cases that were caused by the presence of biological or chemical formulas within GO definitions. These are not particular to any type of (in)consistency. For example, the definition for “geranyltranstransferase activity (GO:0004337)” is “At present, not every piece of information in the generated GOA instance is exploited by our modelling: for example, GO synonyms or larger context (such as the full paragraph) from where the evidence text was extracted were not used. Some researchers have developed methods to identify hypothesis statements or new knowledge from scientific literature using language meta-knowledge . Accordivertical consistency estimation between the GO definition and evidence text as two texts are at different levels of specificity in expressing a gene product function. The GO definition describes the gene function more abstractly while the evidence expresses more detailed information. A previous related benchmark called GOCat [horizontal sentence pair similarity estimation [micro-averaged Precision on the test instances with new GO terms, compared to 0.74 micro-averaged Precision on test instances with seen GO terms. The results demonstrate that our model is effective at processing new knowledge.Our proposed method uses ed GOCat  uses hortimation  where twtimation . Our sysContinual monitoring of the consistency of GO annotation records in modern organism databases is important to maintain currency and quality of the information in these resources. We formally identify five major types of (in)consistent GO annotations that reflect the major GO annotation quality concerns for GO curation community. We propose a novel and efficient method to apply state-of-the-art text mining models to automatically detect these five major types of (in)consistent GO annotations, evaluated using an automatically generated data set. Our method satisfies the open-world assumption and is therefore robust to changes in the GO terminology.Precision for Model-Term and 0.82 Precision for Model-Code in distinguishing consistent from inconsistent GOA. This method can improve the efficiency of human curators by enabling curators to focus their efforts on correcting identified inconsistencies and by categorising these inconsistencies, therefore reducing the number of records that need to be manually reviewed.We have demonstrated a novel method that can be adopted for real-world human-in-the-loop curation. Our implementation achieved 0.74 Another strength is that the model has achieved competitive performance among predicted results with less prediction uncertainty, which can be used by human curators to further focus their efforts. We were able to further improve performance through training set optimisation and the addition of evidence section information. Through a detailed performance analysis using a linguistic test suite, we identified superficial linguistic features that may impact the model’s prediction uncertainty.In future work, we aim to produce a more comprehensive evidence-based GO annotation corpus focusing on inconsistencies. We will seek assistance from expert curators to test and extend the proposed methods on real-world database records with broader gene function perspective, and will specifically seek to improve the identification of evidence code inconsistencies. We will also examine the use of meta-knowledge analysis to improve the model’s performance in identification of instances that lack supportive evidence. We will refine the modelling of semantic hierarchical relationship between parent and children GO terms.


In malignant lymphoma an early and accurate diagnosis is essential for therapy initiation and patient outcome. Within the diagnostic process, imaging plays a crucial role in disease staging. However, an invasive biopsy is required for subtype classification. Involvement of the spleen, a major lymphoid organ, is frequent in malignant lymphoma; this may be reactive or due to infiltration by malignant cells. Using radiomics features of the spleen in a machine learning approach, we investigated the possibility of distinguishing malignant lymphoma patients from other cancer patients and to classify lymphoma subtypes in the case of disease presence. Recent studies have proven the value of radiomics analysis in differentiating lymphoma from non-lymphoma groups on involved sites. Supported by machine learning, imaging could gain importance as a noninvasive diagnostic tool for future lymphoma classification, offering more precise radiological information for an interdisciplinary approach regarding treatment planning.p < 0.01); HL vs. NHL AUC = 0.75 (p < 0.01); DLBCL vs. other NHL AUC = 0.65 (p < 0.01); MCL vs. FL AUC = 0.67 (p < 0.01). Classifying malignant lymphoma vs. non-lymphoma was also possible using only shape features AUC = 0.77 (p < 0.01), with the most important feature being sphericity. Based on only shape features, a significant AUC could be achieved for all tasks, however, best results were achieved combining shape and textural features. This study demonstrates the value of splenic imaging and radiomic analysis in the diagnostic process in malignant lymphoma detection and subtype classification.The spleen is often involved in malignant lymphoma, which manifests on CT as either splenomegaly or focal, hypodense lymphoma lesions. This study aimed to investigate the diagnostic value of radiomics features of the spleen in classifying malignant lymphoma against non-lymphoma as well as the determination of malignant lymphoma subtypes in the case of disease presence—in particular Hodgkin lymphoma (HL), diffuse large B-cell lymphoma (DLBCL), mantle-cell lymphoma (MCL), and follicular lymphoma (FL). Spleen segmentations of 326 patients  were generated and 1317 radiomics features per patient were extracted. For subtype classification, we created four different binary differentiation tasks and addressed them with a Random Forest classifier using 10-fold cross-validation. To detect the most relevant features, permutation importance was analyzed. Classifier results using all features were: malignant lymphoma vs. non-lymphoma AUC = 0.86 ( Malignant lymphomas represent a heterogenous group of neoplasms of lymphatic tissue and are divided into Hodgkin lymphoma (HL) and non-Hodgkin lymphoma (NHL). While many patients with malignant lymphoma present with rapid, indolent swelling of lymph nodes, the spleen as a major lymphoid organ is also often involved in the course of disease of these malignant processes. Splenic involvement is seen in approximately one third of all HL  at preseApart from histopathological classification according to the WHO classification of tumors of hematopoietic and lymphoid tissues, clinical imaging plays an important role in pretherapeutic staging and risk assessment of malignant lymphoma . While PThis study evaluates whether quantitative imaging biomarkers (QIB) of the spleen can be used as a surrogate for predictions in the diagnostic process of malignant lymphoma. Therefore, we applied advanced machine learning techniques by using radiomics features of the spleen to explore the predictive value of the spleen for distinguishing malignant lymphoma patients from other cancer patients and to differentiate between malignant lymphoma subtypes, in particular, in HL and NHL subtypes: diffuse large B-cell lymphoma (DLBCL), follicular lymphoma (FL), and mantle-cell lymphoma (MCL). Recent studies on malignant lymphoma patients have shown the value of QIB analysis in distinguishing malignant lymphoma from other cancer entities ,16,17,18While a differentiation of subtypes via noninvasive imaging will surely not replace histopathological analysis, it helps to better understand imaging characteristics of malignant lymphoma as complementary biomarkers and may offer more precise radiological information prior to, during, and after therapy.The purpose of this study was to investigate the diagnostic value of radiomic features of the spleen in distinguishing malignant lymphoma from non-lymphoma as well as the determination of malignant lymphoma subtypes in the case of disease presence—in particular HL, DLBCL, MCL, and FL.This study was approved by the Ethics Committee of the Ludwig Maximilians University of Munich and performed according to current guidelines for retrospective studies.We collected data from 385 patients with malignant lymphoma, who were treated in the University hospital Munich between March 2010 and March 2018. Data were retrieved as part of our inter-institutional study collaboration, which closed the study in March 2020. To be included in the study, patients had to be diagnosed with a histologically proven HL, DLBCL, FL, or MCL. Patients with a prior lymphoma or a primary CNS lymphoma were excluded. Furthermore, patients were excluded, if no ceCT staging was available, or no full image of the spleen was acquired. The final cohort consisted of 326 patients . For nonThe standard protocol for initial CT staging included a contrast-enhanced image series of the neck, chest, and abdomen. Images were acquired in the portal venous phase using 27 different multidetector-row CT scanners from 4 different vendors, of which the vast majority was acquired at the university hospital, nearly 40% of all scans were taken on a SOMATOM Definition Flash  in-house scanner. CeCT images were obtained by using a weight-adapted intravenous contrast agent. Chest images were obtained after 30 s; abdomen images were obtained after 70 s. For reconstruction, standard soft tissue kernels  were used. Slice thickness varied between 0.5 and 5 mm. To reflect the reality of lymphoma diagnosis in major lymphoma treatment centers, a heterogeneous image acquisition cohort was used to produce generalizable results and identify stable features. In daily clinical routine, patients either present themselves with preclinical imaging or receive a clinical and radiological workup during a hospital stay. All CT scans were reviewed by board-certified radiologists during clinical routine for stage of disease and splenic involvement. Radiological information about splenic involvement was retrieved from the radiology and nuclear medicine report whenever applicable, otherwise the images were reviewed again as part of this project to retrieve missing data. Screening of splenic involvement was performed by two board-certified radiologists in a blinded manner.Images of the non-lymphoma cohort were acquired in portal venous phase using 17 different CT scanners from 3 different companies and included a ceCT of the abdomen in portal venous phase.As a prerequisite for quantitative image analysis, the spleen was segmented on all CT scans. For 190 patients, this was done semi-automatically by board-certified radiologists in a custom software based on MeVisLab  that offers automatic interpolation and refinement of contours to organ boundaries. A 3D neural network based on the U-Net  was thenTo summarize the data, descriptive statistics were used: dichotomous variables were stated in absolute frequency and percentage and were compared with the chi-squared test. Continuous data were tested for normal distribution with the Kolmogorov–Smirnov test and presented in median and lower and upper quartiles. To evaluate not normally distributed data the Mann–Whitney U-test was used.To characterize the radiological appearance of the spleen, we computed all features available in the Python package PyRadiomics 3.0 , which pTo make the features from scans with different slice thicknesses comparable, all images were resampled to an isotropic voxel size of 1 mm prior to feature computation. To account for the heterogeneous CT scanners in our cohort, we investigated the effect of an additional feature harmonization using the ComBat method . BatchesTo enable a better interpretability of the relevant features, we also investigated predictive models on subsets of these features: only 448 features on the original CT image, only 14 shape features (14), which are independent of variations in image acquisition, and, as baseline models reflecting clinical knowledge, CCD only and spleen volume only. Shape features including CCD and spleen volume are computed directly on the segmentations and not on the underlying image data. Therefore, these features are particularly suitable for a real-world data set like ours that is heterogeneous in terms of scanners and imaging settings.For investigating associations between radiomics features of the spleen and the presence of lymphoma or the distinction of different lymphoma subtypes, we trained Random Forest classifiers in a 10-times repeated stratified 10-fold cross-validation. We used the Python package scikit-learn 0.24  with defWe applied this methodology to four different classification tasks, creating a cascade of binary classifiers to distinguish the five classes: (i) non-lymphoma, (ii) Hodgkin lymphoma, (iii) DLBCL, (iv) follicular lymphoma, and (v) mantle-cell lymphoma . To accoBaseline characteristics of the included patients are shown in In A Random Forest classifier for distinguishing all malignant lymphoma patients from controls based on features of the spleen achieved an AUC of 0.86 . The most important feature according to permutation importance is gray level non-uniformity on an image filtered with a LoG of σ = 3 mm, with an AUC decrease of 0.10. The AUC is almost the same when only using original features, but although sphericity is the most important feature, restricting the model to shape features leads to a drop of the AUC to 0.77 . Using CCD of the spleen as the only feature, malignant lymphoma patients are distinguished from controls with an AUC of 0.68 . For splenic volume in all patients, the AUC is 0.67 . After feature harmonization with ComBat, similar AUCs are achieved. The AUC when using original features is 0.81 vs. 0.85 but the CIs overlap and therefore the difference is not significant. The Random Forest classifiers were also applied to the subgroup of malignant lymphoma patients without splenic involvement, which produced similar AUCs as the classifiers on all patients. All results are summarized in The results for the cascade of classifiers predicting subtypes of malignant lymphoma are shown in When using all features, the most important feature according to permutation importance was always a feature on a LoG-filtered image. In two cases, as well as in the lymphoma vs. non-lymphoma case, it is a texture feature. When using only features from the original image, the most important feature is always a shape feature, again consistent with lymphoma vs. non-lymphoma. Furthermore, shape features always perform comparably to original features.The goal of this study was to explore whether potentially robust predictors of the spleen for lymphoma detection and subtype classification could be identified based on quantitative imaging, which may serve as an imaging biomarker for noninvasive, early diagnostic directions in lymphoma patients. The results of this study suggest that (i) shape features of the spleen have more predictive value than the craniocaudal diameter of the spleen and spleen volume in detecting malignant lymphoma, (ii) radiomics features and subsets, like shape features, allow the differentiation of different malignant lymphoma subtypes, and (iii) the sphericity of the spleen is a key characteristic that has not gained a lot of scientific attention in malignant lymphoma classification until now. While there are several studies that used QIB for diagnostic classifications and predictions of patient outcome and prognosis in different cancer entities ,13,28,29Only few studies have focused on differentiating between lymphoma subgroups with imaging techniques, mostly focusing on involved lymph node sites. A recent study showed promising results in categorizing DLBCL and FL patients on MRI using statistical analysis correlating texture features and subtype . FeaturiTo evaluate the diagnostic value of the spleen in lymphoma patients for differentiating malignant lymphoma from non-lymphoma patients and classifying subtypes, segmentations of the spleen on baseline ceCT imaging were used to derive different QIBs. Extracted imaging features of the spleen included those from which Aerts et al. derived their established radiomics signature . MeanwhiIn this study, we followed a classification tree  to assesThe CCD of the spleen was used as a baseline to evaluate the binary classification tasks, as it represents a daily used splenic parameter and is easy to retrieve during clinical routine. Spleen volume on the other hand is unlikely to be retrieved in clinical routine but corresponds well with the CCD of the spleen for assessing splenomegaly ,38. By uClassifiers based on shape features could also be used to significantly differentiate between the varying subgroups of lymphoma, indicating that shape is not only different in malignant lymphoma vs. non-lymphoma patients but also varying between malignant lymphoma subtypes. Accordingly, the most important features, when using only features from the original image, were again shape features in all classification tasks. The best outcome for differentiating malignant lymphoma subtypes could be achieved in most of the binary classifiers using all available radiomic features, combining shape and texture features. It must be noted that numerically superior classifying results in differentiating the indolent subtypes MCL and FL were on the other hand achieved using only shape features. While the AUC is numerically higher than other classifiers, the CIs overlap and therefore the difference is not significant.Direct splenic involvement of malignant lymphoma was observed in 12.6% of all patients in our cohort. In order to account for a potential bias in our analysis due to splenic involvement, we additionally analyzed a subgroup where we excluded patients with splenic involvement. In this subgroup, similar results as in the complete cohort were observed. These results suggest that other anatomical characteristics of the spleen are more relevant for the classification task. Overall, our results indicate that radiomic features of the spleen can serve as surrogates for CT-based lymphoma diagnosis and subtype differentiation. In order to extract this valuable information in CT scans, segmentations of the spleen are required. Currently, clinicians usually either do not have enough time or do not have the right tools to perform segmentations in their daily routine. In this study, a U-Net for automatic spleen segmentation was successfully trained with high accuracy. The U-Net is currently the state of the art for automatic image segmentation and was also used by Humpire-Mamani et al. for spleen segmentation . When a Our study has some potential limitations. As a retrospective, single-center study, we were able to retrieve a large sample size, however, when dividing our cohort into subgroups sample sizes varied between entities with DLBCL being the largest one and MCL patients being the smallest one with 48 patients. While there are many different subtypes of lymphoma, we only concentrated on four of the largest malignant lymphoma entities. As the non-lymphoma group, we used imaging data of patients with colorectal cancer, which of course does not necessarily indicate a healthy spleen, although splenic metastases in colorectal cancer are uncommon . HoweverWith 326 patients and images retrieved from 27 different CT scanners, images were acquired with a high variability in image acquisition. Especially textural features can be sensitive to image acquisition and reconstruction settings . As the As this study focuses solely on the spleen in differentiating lymphoma subtypes, additional information could be gained by analyzing other sites of disease such as enlarged lymph nodes. Further prospective studies are needed that combine the prognostic value of the spleen and other involved sites in malignant lymphoma patients.Being able to differentiate malignant lymphoma subtypes from imaging data using radiomics analysis may be useful in the future for creating diagnostic support systems, benefiting the patient with a fast and interdisciplinary approach to a final diagnosis. While it will not replace the histopathological analysis of lymphoma, the diagnostic value of the spleen and other involved sites could become more important in noninvasive examination of lymphoma, offering more precise imaging information, and therefore assisting the pathologist in immunohistochemical decisions. In addition, it could also offer early valuable information on lymphoma progress and intratumoral mutations of genetic markers in lymphoma, which would have to be evaluated in further studies.Imaging plays an important role in the diagnosis of lymphoma for defining the stage of disease, while the immunohistopathological workup of extracted lymph nodes defines the subtype of malignant lymphoma. The results of this study suggest that (i) shape features of the spleen have more predictive value than CCD and spleen volume in detecting malignant lymphoma, (ii) radiomics features allow the differentiation of different malignant lymphoma subtypes , and (iii) the sphericity of the spleen is a key characteristic that has not gained a lot of scientific attention in malignant lymphoma until now. The best results were achieved using all features to build classifiers. Classifiers using only shape features performed significantly as well, indicating that the shape of the spleen changes depending on lymphoma subtypes. The results of this study conclusively demonstrate that the spleen offers potential diagnostic value in lymphoma patients and should be evaluated in further studies.


Standards for Educational and Psychological Testing) and methodology  from education and psychology can be applied to validation practices for patient-reported outcomes that are measured by instruments that assess theoretical constructs in health.Contrary to common usage in the health sciences, the term “valid” refers not to the properties of a measurement instrument but to the extent to which data-derived inferences are appropriate, meaningful, and useful for intended decision making. The aim of this study was to determine how validity testing theory (the Standards recommendation) through a literature review and mapped to the five inferences. Evaluation of the evidence was not within the scope of this study.The Health Literacy Questionnaire (HLQ) was used as an example of a theory-based self-report assessment for the purposes of this study. Kane’s five inferences  for theoretical constructs were applied to the general interpretive argument for the HLQ. Existing validity evidence for the HLQ was identified and collated  for theoretical constructs, and which connect raw data to the intended interpretation and use of the data. The literature review identified 11 HLQ articles from which 57 sources of validity evidence were extracted and mapped to the general interpretive argument.Kane’s five inferences and associated warrants and assumptions were demonstrated in relation to the HLQ. However, the process developed in this study is likely to be suitable for validation planning for other measurement instruments. Systematic and transparent validation planning and the generation  of relevant validity evidence supports developers and users of PRO instruments to determine the extent to which inferences about data are appropriate, meaningful and useful  for intended decisions about the health and care of individuals, groups and populations. Contrary to common usage in the health sciences, the term “valid” refers not to the properties of a measurement instrument but to the extent to which data-derived inferences are appropriate, meaningful, and useful for intended decision making , 99. DatStandards for Educational and Psychological Testing and Michael T Kane’s argument-based approach to validation are the authorities in validity testing theory and methodology, and these draw on a long history of thinking and debating about validity in education and psychology [The ychology , 101. Unychology . Contentychology . In consychology , 100. Thychology . Howeverychology . Criteriychology , 45, 100ychology .Validity in the 3rd edition of Educational Measurement [Educational Researcher [Standards, from the first technical recommendations in 1954 through to the 2014 Standards [Standards for Educational and Psychological Tests to the Standards for Educational and Psychological Testing, which recognised that the focus of validity is on the process of testing rather than on the test itself [Standards [Messick built on the work of previous theorists , 33, 34 surement , and hissearcher , that brsearcher . The idetandards . Worthy t itself . Messickt itself , 82. Mestandards , 59, 62.Standards [test content; response processes of respondents and users; internal structure; relations to other variables; and the consequences of testing as related to a source of invalidity such as construct-irrelevant variance or construct underrepresentation.The tandards  describeStandards in a practical way, a systematic methodological approach is needed [Depending on the intended measurement purpose, a validity argument will require evidence from one or more of these sources. To apply the validity testing theory of the s needed –27.An Argument-based Approach to Validation [Standards [interpretive argument : Clearly and coherently state the intended interpretation and use of scores, including the chain of inferences that will lead from observed scores through to decisions based on those scores.The validity argument: Evaluate evidence and the construct theory to determine the extent to which these support the interpretive argument.The Kane formalised his practical validity testing methodology in his 1990 essay lidation , 62, 65.lidation , the 198tandards , construEducational Measurement  [practical argument model [Kane’s methodology places validation as a process of establishing empirical evidence to build an argument for the validity of the interpretive argument . Interpr4; 2006) , Kane dint model , 105: 1)nt model , 105. ThStandards and Kane to clinical medicine, research and education but they do not develop full interpretive and validity arguments [Standards) and discusses the importance of the accumulation of evidence to support the inferences, actions and decisions that arise from the interpretation of PRO scores. But, again, there is no development of interpretive or validity arguments for PRO scores.Most of the interpretive and validity argument research has been done in language testing and in education. One example is the comprehensive development of a validity argument for the new Test of English as a Foreign Language (TOEFL) –27. Thisrguments . The 201rguments  is an imrguments   and methodology  from education and psychology can be applied to validation practices for patient-reported outcomes that are measured by instruments that assess theoretical constructs in health. The aim will be addressed by developing an interpretive argument and collating relevant validity evidence for a representative theory-based self-report assessment in health. It is expected that the study outcomes will inform validation planning and collation of evidence for the inferences, warrants and assumptions inherent in an interpretive argument. Further analysis, not addressed in this study, is needed to evaluate the merit and strength of the collated evidence and to synthesise it in relation to the inferences in the interpretive argument so as to substantiate the interpretive argument .The aim of this study was to determine how validity testing theory  . The HLQdescribing the health literacy of populationsinforming foci for health literacy interventionsmeasuring outcomes of public health and clinical interventions designed to improve health literacy.In our expository study, Kane’s five inferences for theoretical constructs were applied to the existing general interpretive argument for the HLQ . The HLQThe indicators of the nine-domain theoretical construct of the HLQ Table  are the Standards. Reliability evidence and general study characteristics were also extracted from the articles. A descriptive and frequency analysis approach was used to identify, categorise and count the sources of evidence [Standards. The collated sources of validity evidence were then aligned with the five inferences (and underlying warrants and assumptions) of the HLQ interpretive argument, as specified by the Standards.After establishing a general interpretive argument, a literature review was conducted to find existing validity evidence for the HLQ. HLQ validity testing studies were collated by the first author (MH) through a search of the peer-reviewed literature in EBSCOhost databases: Academic Search Complete; CINAHL Complete; ERIC; Global Health; MEDLINE Complete; PsycINFO. No limiters were applied. Articles with statements in the title or abstract about undertaking validity testing of the HLQ were included for data extraction. Reference lists of pertinent systematic reviews were scanned for relevant articles, as were article reference lists and the authors’ personal reference lists. This study categorised HLQ data only as sources of validity evidence, as defined by the evidence , 50. TheFigure scoring inference that scores are calculated as per the HLQ scoring instructions. Next, the generalisation inference assumes the warrant that scale scores are representative of a universe of scores from other similar respondents and measures. The extrapolation inference links data to theory and assumes the warrant that the nine scale scores are representative of the nine health literacy domains . The theory-based interpretation inference assumes the warrant that a relationship exists between the nine-domain theory and the items and scales such that the HLQ scores support profiles of health literacy strengths and needs. Finally, the implications (or utilisation) inference assumes the warrant that the nine theoretical health literacy domains embody factors that affect health outcomes. Evidence to support each of these inferences implies that HLQ data  can be used appropriately and meaningfully for the intended purposes listed in the top row of Fig. 1Figure The review resulted in 11 articles in which validity testing of the HLQ had been conducted , 92, 94.test content reported the most frequently . There were no studies that reported evidence about the consequences of testing.Table Standards [Standards through Hawkins et al. (2018) [A theoretical validity testing framework was referenced by four of the 11 HLQ papers , 51, 94,tandards , 94 indi. (2018) , 49 and . (2018) , 94.Table Standards places information about scoring in test content [The  content . All 11  content . The stu content  conducten = 9 studies) [In the HLQ studies, reliability was calculated mainly through Cronbach’s alpha with or without composite reliability (studies) , 92, 94.studies)  also calstudies)  used thestudies)  used a qStandards: test content, response processes, internal structure, and relations to other variables.Evidence for extrapolation from observed data to a theoretical construct can be extensive and is based on four of the five sources of evidence in the test content included information about item content and difficulty (as described by the HLQ item intent descriptions) [Evidence based on iptions) , 92, 94,iptions) ; examinaiptions) , 74, 94.response processes provided analyses about what the items and scales mean to respondents and users of the HLQ  [response processes because the interviews were conducted with both users of HLQ data (clinicians) and respondents (patients) [response processes was the Maindal et al. study, which conducted cognitive interviews.Evidence based on riences) , and wheriences) . Three sriences) , 74, 94.atients) . The onlinternal structure described the interrelationships of items within scales, and the extent to which the items conform to the construct of that scale [internal structure. Confirmatory factor analysis (CFA) [n = 9 studies). The Morris et al. study also used principal component analysis (PCA) and Rasch analysis [Evidence based on at scale . Some stis (CFA) , 92, 94 is (CFA) , 86 studrelations to other variables is usually derived from studies that analyse the relationships between an instrument and other criteria [Standards places evidence for discriminant validity within relations to other variables (p.16) [n = 7 studies) [Evidence based on criteria . Howevercriteria . The nins (p.16) , so thisstudies) , 92, 94.studies)  using listudies) , 109 to studies)  did corrstudies)  using thstudies) , 68, 74.test content, response processes, internal structure, and relations to other variables. However, instead of the focus being on whether or not the scale scores account for the scope of the health literacy construct (extrapolation inference), the focus for the theory-based interpretation inference is on evidence to support scale scores as estimates of the theory. That is, the focus is on evidence for the extent to which a scale score estimates a person’s or population’s level of attributes, resources or competencies associated with that HLQ domain. Evidence, therefore, relies heavily on test content and response processes to understand the cognitive connection between the HLQ domains and the items and scales. Often these are qualitative studies undertaken during development  [Evidence for the theory-based inference can also rely on evidence based on r users) , 92, 94.r users) , 74, 92.consequences of using the data to inform decisions and actions [Evidence to support this assumption is based on the  actions , 63, 72. actions , 66, 67 response processes  was found in only one HLQ translation study [Validation is an ongoing process of checking that the interpretation and use of a PRO assessment’s data are valid for each new context, and so generation of validity evidence for the use of HLQ data in different contexts is ongoing. There is incomplete evidence for the generalisation (except for reliability) and implications inferences. Also, the HLQ has been translated to more than 30 languages since 2013 but evidence based on on study . AlthougStandards’ five sources of validity evidence and mapped to the five inferences in the HLQ interpretive argument. Further research is required to synthesise and evaluate the evidence to determine a validity argument for the HLQ general interpretive argument, and for each stated purpose of HLQ data. Table This study integrates validity testing theory and methodology from education and psychology with measurement research in health. Kane’s five inferences and associated warrants and assumptions were demonstrated in relation to the HLQ, a PROM based on a theoretical construct. Existing HLQ validity evidence was collated within the Collation and categorisation of existing sources of validity evidence for the HLQ shows that some sources of evidence have been published for HLQ scoring, extrapolation and theory-based interpretation inferences but minimal evidence is available for the generalisation and implications inferences. The HLQ is a relatively new measurement instrument that has been quickly taken up for use in a wide range of studies in more than 60 countries. The development and publication of validity evidence has not kept pace with the application of the HLQ in practice. However, since the literature review for this study was conducted, more articles presenting evidence for the HLQ have been published , 108.response processes. Many HLQ translation teams did conduct cognitive interviews but the processes and outcomes of this testing have not been published. Evidence based on internal structure has been generated for translated HLQs through CFA to confirm that the relationships between HLQ items and scales in different contexts behave in comparable ways to the English-language HLQ items and scales. However, full analyses of measurement invariance comparing data from translated and original versions of the HLQ using, for example, multiple-group confirmatory factor analysis are lacking. This type of analysis is essential if cross-cultural data comparisons are to take place [relations to other variables is common for the HLQ because of the decision in this study to categorise calculations of discrimination between each of the nine scales under discriminant validity: that is, although the nine scales represent the overall construct of health literacy, analyses should show that each scale is distinct from the other scales. However, it should be noted that there are limited studies of convergent and discriminant validity with other constructs in the nomological network of health literacy.For translations of the HLQ, gaps mainly exist for evidence based on ke place . EvidencThe findings of this study are important because they demonstrate that validity testing theory and methodology from education and psychology can be applied to the development, testing, and retrospective evaluation of a theory-based health literacy measure, and can potentially be applied to other self-report PROMs , 85. Thitest content to another language and to understand the response processes of the target population when they engage with the translated instrument’s items [Quantitative analyses can confirm the psychometric properties of an instrument, but these data do not explain how people interpret and cognitively respond to items or the consequences of data-based decisions , 58, 70.’s items , 95, 102’s items , 111.A limitation of this study is that it does not evaluate existing HLQ validity evidence to develop a validity argument for the HLQ or to determine gaps in evidence to support HLQ data for decision making in different contexts and for different purposes. Future research for the HLQ will involve more detailed analyses of the outcomes of the studies reviewed. The predominant strength of this study is that it demonstrates that well-developed theory and methodology from education and psychology can be applied to validation practice for PROMs. This study was retrospective in nature, but the process can also be used prospectively for validation planning and testing of newly developed PROMs.PROM validation practice needs to evolve from referring to validity as a property of a measurement instrument to understanding that validity refers to the data-derived inferences that influence decision making. This study demonstrated a process for systematic and transparent validation planning  and collation of relevant existing validity evidence, using the HLQ as an example PROM. Such a process enables PROM developers and users to then synthesise evidence and evaluate the extent to which the evidence supports the appropriateness, meaningfulness and usefulness  of the data for making decisions about the health and care of individuals, groups and populations in different contexts.


Cancer is a non-communicable disease and is the number 2 leading cause of death globally. Among all cancers, cervical cancer is the number 1 killer of women in low-income countries (LICs). Cervical cancer is a well understood preventable cancer. The rates of cervical cancer are very varied and inversely proportional to the effectiveness of disease management policies. Management of cervical cancer includes prevention, screening, diagnosis and treatment. The main objective of this scoping review is to map the evidence on cervical cancer management in sub-Saharan Africa (SSA) to understand the coverage of cervical cancer prevention and treatment services and provide an opportunity to generate knowledge on the risk factors, attitudes and practices extendable globally.This review will be guided by Arksey and O’Malley’s framework recommended for conducting scoping review studies. The Preferred Reporting Items for Systematic Review and Meta-Analysis extension for Scoping Reviews (PRISMA-Scr) checklist will also be completed to ensure that the review adheres to the sound methodological rigour acceptable for scoping review studies. The following electronic databases will be searched for potentially eligible articles: PubMed, Ebsco Host, Scopus and Cochrane Database of Systematic Reviews. Study screening procedures recommended by Higgins and Deeks will be followed. A narrative synthesis will be used, with data synthesised and interpreted using sifting, charting and sorting based on themes and key issues.Cervical cancer can become a disease of the past with a proper control strategy in place. It is therefore imperative to map available evidence on the management of cervical cancer to inform policy and advocacy action. More knowledge on the status quo will guide policymakers in ensuring cancer management guiding policies are formulated/updated/revised accordingly.Not registered with PROSPERO (not needed).This scoping review was not registered. Cancer of the cervix is caused by persistent infection with high-risk types of human papillomavirus (HPV) . PersistCC is a preventable and treatable disease . HPV canApproximately 570,000 new CC cases and 311,000 deaths were recorded worldwide in 2018, and globally, CC is at number 4 of all cancers . In sub-In some Western countries, CC rates decreased by 65% . In NorwLICs recognise the importance of HPV vaccination, but getting the resources for the vaccine has been a challenge. Gardasil4 and Cervarix vaccines are available for protection against two types of HPV that cause approximately 70% of CC cases . GardasiOn the other hand, HPV vaccines do not protect against already existing infections and do not protect against all the types of HPV that cause CC ; therefoCC is the most screened cancer worldwide . CervicaIncidence, survival and death rates can be used to measure progress in CC control with death rates being the best indicator . ApproxiThe 5-year survival rate is 91% for localised CC at diagnosis which falls to 57% for distant stage CC . Stages HIV-positive people are a high-risk group for HPV infections , yet SSACervical cancer control is defined as activities to reduce the CC burden through dissemination and implementation of evidence-based interventions . The intWhat are the factors associated with high HPV vaccine coverage in SSA?What are the barriers to CC screening uptake in SSA?What factors are associated with late-stage CC presentation at diagnosis in SSA?What is the status of radiotherapy and chemotherapy in CC management in SSA?The proposed hypothesis for this review is CC management needs improvement in SSA. The main research question is: What evidence is there on CC management in SSA? The sub-research questions include the following:To explore the factors that are associated with high HPV vaccine coverage in SSATo determine the barriers to CC screening uptake in SSATo find out the factors that are associated with late-stage CC presentation at diagnosis in SSATo establish the status of radiotherapy and chemotherapy in CC management in SSAThe aim of this review is to map evidence on CC management in SSA. The objectives of the study are as follows:The proposed scoping review will be conducted in accordance with Arksey and O’Malley’s (2005) scoping review framework. Arksey and O’Malley  indicateAn initial search was performed to determine whether a previous review addressing this topic in SSA was conducted or was in progress. There were no complete or in-process reviews focusing on the coverage of cervical cancer prevention and treatment services in SSA. Based on the review question, the search strategy was developed by identifying the key concepts using the PICO (Problem/Intervention/Comparison/Outcome) approach  and furtTwo independent reviewers will conduct the abstract and full article screening. The literature will include published peer-reviewed journal articles with evidence of empirical design utilising either qualitative, quantitative or mixed method research approach addressing the research questions. The screening procedure will be guided by Higgins and Deeks’ framework . All artStudies presenting evidence on cervical cancer.Studies presenting evidence conducted in SSA.No limits will be applied for the publication date of included studies.All study designs will be considered.The inclusion criteria were guided by the following principles to determine the articles relevant for this review:Studies that do not focus on humans, as well as those written in languages other than English, will be excluded. Non-empirical material like book chapters, opinion papers, commentaries and editorials will also not be included.A data collection instrument Table  was deveA narrative synthesis will be used, with data synthesised and interpreted using sifting, charting and sorting based on themes, key issues and type of study. Citation tracking will be done using the Reference Manager Software in Endnote version X7. Data analysis and tabulation of the findings will be done using Review Manager (RevMan) . The narStudies that will be published between the research and report writing will be obtained by subscribing to updates to databases using the search domains used during the literature search. Data will be extracted by the principal investigator, and accuracy will be checked by a second reviewer. Studies with uncertainties about their inclusion will be discussed with a third reviewer.The quality of evidence will be assessed based on guidance in the National Institute for Health and Care Excellence single technology appraisal Specification for Manufacturer/Sponsor Submission of Evidence adapted from the Centre for Reviews and Dissemination’s guidance for undertaking reviews in healthcare .Mixed Method Quality Appraisal Tool will be used for quality assurance , with asCC deaths and incidence rates are still high in SSA . The ratGlobal research has shown that among the ways to prevent CC, CC screening and follow-up has the greatest or second greatest impact after vaccination; hence, lives can be saved by the comprehensive application of available evidence-based interventions to all females who could be affected by CC. The results of the scoping review will be used to inform health policy and knowledge to end-users regarding strategies that can be used to facilitate CC control. The review will be used to identify research gaps that need to be addressed in CC management. Furthermore, this review will provide a complete and reliable picture of how CC control is being managed in the region, the challenges and opportunities of CC management will be highlighted . The resThe outlook for CC control is bright since the knowledge of what to do is there and the tools to do prophylactic interventions which include, vaccination of the girls, screening and preventive treatment for adult women are well known. Treatment of precancerous lesions is of low cost compared to the cost of invasive cervical cancer treatment (which is not readily accessible to many women) . It is d


The Scottish Intercollegiate Guidelines Network (SIGN) is the leading national clinical guideline producer in Scotland. Improved design and dissemination of guidelines produced for the public can empower people to take an active role in self-management and shared decision-making. The public version of the guideline examined covered getting assessed and diagnosed with autism, and approaches that can help. The aim of this study was to test a public version of a guideline for the parents of children and young people with autism, implement improvements, and identify what works in making it usable and accessible.We recruited mothers from across Scotland. User testing involved formal ‘think aloud’ semi-structured interviews that guided users through the booklet. Interviews took place individually and were recorded and transcribed. Key findings were identified and themed using the honeycomb user experience model.Fourteen user-testing interviews were conducted. Facilitators for usability and desirability of the guideline included the chunking of text, consistent use of colour and boxes to highlight important information. Simple language, written in a tone of partnership, helped to engage mothers. Value arose from the guidelines ability to explain the process of diagnosis and make mothers feel empowered in their relationships with healthcare professionals. There was a lack of consensus on the usefulness of rating the strength of evidence and recommendations.There was a marked similarity between what was important to the mothers and what has been found to be important to other groups. The involvement of service users and carers in the guidelines development was key to its credibility. One size does not fit all in presenting evidence-based recommendations to the public and it is a challenge to provide sufficient information while avoiding information overload. Recommendations and evidence levels are suitable for use in public versions, but these should be kept as simple as possible. Clinical guidelines are tools that provide evidence based recommendations and direct appropriate healthcare –3. TradiAutism spectrum disorders are a highly prevalent condition. Prevalence in children is around 1% , 10. SwiUser testing has been used successfully to evaluate and improve systematic reviews, guideline creation tools  and dissParticipants were recruited through the SIGN patient network , NHS Greater Glasgow and Clyde Child and Adolescent Mental Health Service and from third sector organisations including the National Autistic Society, Autism Network Scotland, and local carers’ centres. Snowball sampling was also used. The SIGN research analyst (LW) contacted organisations, and potential participants were approached by email and telephone. Written information about the project and what participation would involve was provided to all participants prior to the user testing interview. Interviews took place at a location convenient to participants and travel expenses were provided. The names used in quotations are pseudonyms. We aimed to recruit 12 to 15 participants because our previous user testing suggested that this would be sufficient to reach a saturation of views.Interviews took place in November 2016 in locations across Scotland including Glasgow, Edinburgh, Perth, Dundee and Arbroath.User testing was based on a method of data collection and analysis, which was developed by Rosenbaum . This meOne interviewer (LW) conducted all the interviews, and a SIGN public partner (a volunteer who works with SIGN to provide a public perspective) acted as an observer and took structured notes. The observer was not available for the final three interviews. A training session was held with the observer prior to the start of data collection. A pilot interview was held to examine the feasibility of the interview guide, and resulted in minor amendments, including the order of the questions, and the print quality of materials.There are three public versions of SIGN guideline 145: Assessment, diagnosis and interventions for autism spectrum disorders , a booklSIGN’s public versions used three pre-defined levels of evidence, to explain the evidence quality underlying our advice to the public. We tested the use of an additional level of evidence, for a strong recommendation based on good quality evidence see Fig. . The newSIGN public versions aim to be an accurate translation of the clinical guideline and only those interventions recommended in the guideline are included. However, further information is included to help the public to understand the recommendations.phase one identified barriers to the use of the public version and necessary amendments to the guideline and was published on SIGNs website ,phase two identified issues that may be generalisable beyond this specific guideline, and is reported on here.The method of analysis is described in detail in our previous paper . It incoIn phase two, the findings were refined and analysed thematically  using thThe researchers met to discuss the results, their discussions were focused on identifying areas of agreement and disagreement and on reaching consensus on their interpretation of participants’ interview transcripts. Agreement was reached on key themes and findings, no statistical form of inter-rater reliability was used .The lower order themes were then grouped under the overarching themes from the honeycomb model. An iterative process of writing up the findings and refining the themes was then carried out, which involved both researchers , 30The researchers recognise the need for substantial analytical and interpretative work on the part of qualitative researchers and acknowledge that coding is a subjective process. However, the research values, experience, skills and training of qualitative researchers and the application of rigorous research and analytical methods should ensure that the themes and findings generated are authentic and trustworthy This project is classified as a NHS service evaluation. Information on the planned data collection was submitted to the East of Scotland Research Ethics Committee, who confirmed that full ethical approval was not required. The project was carried out in accordance with the principles laid out in the declaration of Helsinki. Participants were sent an information sheet in advance, and gave informed written consent before their interview began. Participants were informed of their right to withdraw at any point of the study, all data was held securely and no personally identifiable information or quotes have been published.Fourteen mothers of people with autism took part in the study. Sixteen people agreed to take part but two people were unable to make the available dates. All of the participants were female, with a mean age of 42 and a range of 10 years, they were largely well educated, and involved in a range of activities (such as volunteering) in relation to autism, see Table 1There was agreement from most mothers that the guideline would be a valuable resource for families of children with autism, particularly at the time of assessment and diagnosis. The parts of the guideline that mothers found most useful included, information on their child’s health condition, tools and tests used for assessment, information on interventions, what to expect from healthcare professionals and services at each stage of diagnosis, and signposting to sources of information and support.Mothers suggested that the public version empowered them to ask informed question of healthcare professionals and to participate in shared decision-making. Information was valued, even if it was not linked to a recommendation, because it could be used in discussions with healthcare professionals and to highlight specific services or interventions see Fig. . This ty“… it Fig.  tells yoThe ‘Where can you find out more?’ section provided a list of trusted sources of help and information. Mothers wanted local support information, but this could not be provided in a national guideline. Signposting to national and third sector organisations may be the best way that a national body can meet this need.The number of text boxes see Fig.  used in Mothers highlighted the importance of the consistent and meaningful use of colour. In SIGN public versions, recommendations are green, which is associated with safety or ‘go’. Information boxes are highlighted in blue see Fig. , which mSIGN uses a system of icons with text to flag recommendations and their evidence level see Fig. . Mothers“I think, I suppose it’s saying ‘its ok’, - based on research, based on clinical. I would say probably your Joe Public will just think it’s one in the same thing, they’re not going to be differentiating it at all.” Dianewhy it is necessary to have these different levels of recommendation. Similarly, mothers found the use of the not enough evidence icon  the need for different levels of recommendation. When involving the public in guideline development, training can be offered about evidence quality  to explaWork in the field is ongoing on developing quality criteria for public versions of clinical guidelines and other decision aids –40, and On balance, our user testing supported the use of multiple levels of recommendation. This group understood the meaning of the hierarchy of strength, and recent research supports this in other groups . HoweverInterestingly the mothers of children with autism placed a greater emphasis on the active inclusion of the public in developing the clinical guideline and on the inclusion of up-to-date evidence than people with glaucoma . MothersUnsurprisingly the use of a tone of partnership and collaboration in the public version was important for these mothers. Guideline developers should tailor the language used in public versions to the specified user group as far as possible, and user testing or some other form of consultation is certainly helpful. Acknowledging any challenges with terminology and describing how chosen terms have been selected in the public version, may also reassure the user group that the guideline maker is aware of the sensitivity and controversy that may be associated with them.SIGN uses consultation with service users and the public to select their recommendations for inclusion in their public versions; however, the public may still struggle to understand how to make practical use of all recommendations included. For example, recommendations that use the question mark symbol to reflect uncertainty in the evidence base. The inclusion of qualitative evidence examining public perspectives in the guideline development process may help to make recommendations more feasible, acceptable and actionable. , 47 SIGNSIGN’s public versions of clinical guidelines were  redesigned based on the work of the DECIDE project between 2011 and 2015 . This stThe sample was limited to women, all of whom were mothers of a child with autism. The sample consisted mainly of highly educated women, many of whom were active in the autistic community and half of the sample was recruited from SIGNs patient network or via SIGNs Facebook page. This group was more aware of clinical guidelines than the general public and may have held stronger views on aspects of the management of autism than a more diverse sample would. Recruiting from other sources, such as online communities independent from SIGN, or diverse third sector organisation might help to broaden SIGNS samples in future research.This guideline did not contain numerical information on the risks and benefits associated with intervention options . Future Many aspects of usability, usefulness, desirability and credibility were the same for the mothers of children with autism as for other public groups , 4. The