Obtaining evidence-based answers from biomedical literature is difficult. Analogical reasoning with embeddings from deep learning  may extract such biomedical facts, although the state-of-the-art focuses on pair-based proportional (pairwise) analogies such as man:woman::king:queen  approach underpinned by prior knowledge and another type of 4-term analogy (other than pairwise).person and death: “dagger = −Romeo +die +died” (search query: −Romeo +die +died). Our SemDeep approach worked with pre-existing items of knowledge (what is known) to make inferences sanctioned by a 4-term analogy (search query −x +z1 +z2) from CBOW and Skip-gram embeddings created with a PubMed systematic reviews subset (PMSB dataset). Stage1: Knowledge acquisition. Obtaining a set of terms, candidate y, from embeddings using vector arithmetic. Some n-gram pairs from the cosine and validated with evidence (prior knowledge) are the input for the 3cosAdd, seeking a type of 4-term analogy relating the semantic fields disease and treatment. Stage 2: Knowledge organization. Identification of candidates sanctioned by the analogy belonging to the semantic field treatment and mapping these candidates to unified medical language system Metathesaurus concepts with MetaMap. A concept pair is a brief disease treatment statement . Stage 3: Knowledge validation. An evidence-based evaluation followed by human validation of biomedical facts potentially useful for clinicians.As preliminaries, we investigated Continuous Bag-of-Words (CBOW) embedding analogies in a common-English corpus with five lines of text and observed a type of 4-term analogy (not pairwise) applying the 3CosAdd formula and relating the semantic fields y belonging to the semantic field treatment was F-measure=80.00% . We developed an empirical heuristic with some predictive power for clinical winners, that is, search queries bringing candidate y with evidence of a therapeutic intent for target disease x. The search queries -asthma +inhaled_corticosteroids +inhaled_corticosteroid and -epilepsy +valproate +antiepileptic_drug were clinical winners, finding eight evidence-based beneficial treatments.We obtained 5352 n-gram pairs from 446 search queries by applying the 3CosAdd. The microaveraging performance of MetaMap for candidate Extracting treatments with therapeutic intent by analogical reasoning from embeddings (423K n-grams from the PMSB dataset) is an ambitious goal. Our SemDeep approach is knowledge-based, underpinned by embedding analogies that exploit prior knowledge. Biomedical facts from embedding analogies  are potentially useful for clinicians. The heuristic offers a practical way to discover beneficial treatments for well-known diseases. Learning from deep learning models does not require a massive amount of data. Embedding analogies are not limited to pairwise analogies; hence, analogical reasoning with embeddings is underexploited. How to treat a disease or condition remains to be the most common type of clinical question . It is dtheir ability to capture relational meanings”  is  is 16] iThis study used a subset of PubMed systematic reviews  of 301,2z reused in this study  is from our previous study [From the PMSB dataset, a total of 423K n-grams with a frequency count >5 have vector representations in both models, that is CBOW and Skip-gram. We considered “good” the Skip-gram and CBOW embeddings created in our previous study  as they us study .x,treatment z) from our previous study (prior knowledge) [y for the 3CosAdd formula, that is, the 12-candidate y with CBOW and Skip-gram embeddings yielding the highest 3CosAdd values. We limited the list of candidates to 12, similar to Arguello Casteleiro et al [To address Q3 and apply the 3CosAdd formula 2, 2 n-gram pairs (disease owledge)  were neero et al , and folro et al .This stage accomplishes a named entity recognition (NER)  task invy is difficult to interpret for being truncated strings of characters or containing short forms . String searches in the PMSB dataset and the web search the sense inventory, Allie [First, disambiguation of n-grams y, Allie , enabliny as to whether it belongs to the semantic field Tx . Following Artstein R and Poesio M [Second, the manual binary classification of candidate n-gram Poesio M , we repoPoesio M .Txy to UMLS Metathesaurus concepts TxY. MetaMap performance is calculated using precision, recall, and F-measure [Third, entity normalization (grounding)  with Met-measure ,52.z1 and z2 in the search query. Once the NER task was completed, we obtained the NER winners. An NER winner was a search query -x +z1 +z2 with the maximum observed number for n2 or n3:We took n1 as the number of different UMLS Metathesaurus concepts (represented as Z1 and Z2) mapped as y belonging to Tx, that is, the number of Txy.n2 is the number of different 12 top-ranked candidate n-grams TxY excluding Z1 and Z2.n3 is the number of different UMLS Metathesaurus concepts TxX,Y) acquired previously to determine the therapeutic intent of candidate TxY for target disease X, where X was the UMLS Metathesaurus concept mapped to n-gram x.We sought evidence for the Metathesaurus concept pairs . TxY related to the target concept disease X=“C024302|Sepsis” with CUI=C024302. The rationale for the 7 evidence-based categories introduced is as follows:The 3 domain experts introduced 6 evidence-based categories to further refine the correlations between the semantic field The name of 4 of the evidence-based categories .NER winners (maximum observed number for n2 or n3 in Stage 2) and clinical winners. A clinical winner is a search query -x +z1 +z2  for target disease x with a maximum observed value for n4, that is, the number of different concepts TxY (excluding Z1 and Z2) assigned to the evidence-based category “Tx with therapeutic effect.”This study distinguishes between TxX,Y), 2 more observers  were asked to express agreement or disagreement with the evidence for the concept pairs . To audit the evidence-based categories assigned along with the evidence collected (quotes and references) for the concept pairs  from our previous study [z was mapped to the UMLS concept Z with the UMLS Semantic Type “T061|Therapeutic or Preventive Procedure” or “T121|Pharmacologic Substance.” X,Z).To apply the 3CosAdd formula (and systematic creation of search queries), we reused 63 unique n-gram pairs (us study  (open-acus study ). Every x,z), we built 223 search queries -x +z1 +z2 for the 3CosAdd formula. x,y) n-gram pairs for 10 target diseases x, that is, the 12 top-ranked n-grams  obtained per search query from the CBOW and Skip-gram embeddings. An n-gram pair with y as a non-ASCII character was discarded.With 63 n-gram pairs  n-gram pairs from applying the 3CosAdd formula. x,y) n-gram pairs from the 5352 n-gram pairs. Among the 1935 unique  n-gram pairs, there were 954 n-gram pairs  with candidate y belonging to Tx. The Krippendorff alpha [Txy mapped to TxY for the 10 diseases (microaveraging) [TxY.Different search queries brought the same  , MetaMapNER winners, that is, the search query -x +z1 +z2 for the 3CosAdd formula per model and disease target x having the maximum observed values for n2 or n3.The maximum observed value for n2 was the highest possible value, that is, n2=12, for both CBOW and Skip-gram.−epilepsy +valproate +AED. However, the number of different TxY (excluding Z1 and Z2) differed, that is, n3=11 for Skip-gram and n3=10 for CBOW.The maximum observed value for n3 was for the search query, TxX,Y) mapped to the unique 954 n-gram pairs . Although the UMLS related concepts table (file=MRREL) [e=MRREL)  containsManual searches in the literature proved to be time-consuming and labor-intensive; thus, not all the concept pairs for the target disease anemia and hypertension had evidence. Hence, we limited the study to 408 UMLS CUI pairs , and onlTxX,Y) with more than 1 evidence-based category, such as the concept pair . The evidence-based category “Tx with therapeutic effect” has the highest number of CUI pairs, with 190 pairs , where 117 pairs have evidence (quotes) taken from BMJ Best Practice. The evidence-based category “correlation” has the highest number of evidence-based information sources with 108 uniform resource identifiers of the total 238. clinical winners, that is, search query -x +z1 +z2  with the maximum observed number for n4 per target disease x. NER winner is not necessarily a clinical winner, that is, the maximum observed value for n4 does not always correspond to the maximum observed value for n3 or n2.Table 4 shows the clinical winners according to the observer O2. All rows except two are clinical winners according to the 3 domain experts and both observers.In TxX,Y) with evidence, observer O1 disagrees with 25 of them, and observer O2 disagrees with 26 of them. The Cohen kappa of −0.023 is paradoxical [Considering the 408 concept pairs .clinical winner, which is an NER winner. clinical winner that is not an NER winner. y with domain knowledge meaning  and biomedical evidence ratified with an audit (Stage 3). The evidence provided for the evidence-based categories  is presented in clinical winners found  not appearing among the 20 top-ranked candidates for target x with the highest value for cosine with Skip-gram embeddingsFavor n-grams z2 with frequency counts in the corpus >100Favor n-gram TxY with the evidence-based category “Tx with therapeutic effect,” that is, TxY with therapeutic intent.The heuristic selects 93 of the 304 search queries, which brings 126 of the 190 UMLS Metathesaurus concepts clinical winners obtained is possible.Considering the precision and recall values for the empirical heuristic , the ansperson {you; Romeo} is related to the semantic field death {die; died; dagger} in the context of Shakespeare’s Romeo. Hence, we answer Q1 and Q2 with a “yes”; therefore, analogical reasoning with CBOW embeddings seems feasible with a small common-English corpus. This challenges the current assumption that “learning in current deep learning models relies on massive data” [Humans can agree that the semantic field ve data” .z1 and z2 among clinical winners  and n4=7 for Skip-gram, where the relation between z2 and z1 was inflectional morphology singular:plural. This query resembled the search query, −Romeo +die +died.Category membership. The search query, −epilepsy +valproate +antiepileptic_drug, with n4=8 for CBOW. The search query, −hypertension +antihypertensive_drug_classes +antihypertensive, with n4=8 for Skip-gram. Both search queries were the best clinical winners .Commonalities in structural features. All search queries focused on the therapeutic intent of z1 and z2 for target disease x. However, some queries did not have the above abstract semantic relationships between z1 and z2. For example, the search queries −osteoarthritis +knee_arthroplasty +hyaluronic_acid with n4=5 for CBOW and −heart_failure +beta-blockers +aldosterone_antagonists with n4=5 for Skip-gram.clinical winners; therefore, a tactic preference was latent promising systematicity [We answered Q4 by demonstrating that it is feasible to gain some predictive power for the maticity . Textboxclinical winners) are selected by the heuristic. The precision was 4.30% .With a threshold n4 >7, the recall is 100%. All search queries with n4 >7 (the best TxY with therapeutic intent), the precision was 97.85%  and the recall was 32.50%.With a threshold n4 >0  theThe lists may not comprise mutually exclusive concepts in meaning. For example, “C0060657|formoterol” and “C1276807|Budesonide/formoterol” are both treatments with evidence of therapeutic intent for asthma .TxY for asthma.The lists were incomplete. For example, “C0772501|Levalbuterol” and “C0907850|ciclesonide” are both treatments with evidence of therapeutic intent for asthma  and not We did not use Skip-gram with negative sampling ; therefore, it can be argued that we did not use the best configuration of a word2vec model . The effx, treatment z) [x could suffer the same limitation. For example, in Appendix B in the study by Pakhomov et al [For Stage 1, the 3CosAdd formula needed at least two n-gram pairs (disease tment z) . Only onov et al , among tov et al ).For Stage 2, the MetaMap version was 2016v2 (with a 2016 UMLS release), and few n-grams were considered as clear terminological gaps. The n-gram “anti-VEGF_agents” was manually mapped to CUI=C4727875, which exists in the 2019AA UMLS release. Five n-grams were mapped to very broad CUIs as they had the character “*” in The NER task (Stage 2) and the searchers in the literature seeking evidence for concept pairs (Stage 3) were time-consuming and required highly trained domain experts. The appraisal of the literature was not performed by a review team as proficient as the ones conducting Cochrane systematic reviews.The heuristic developed by visual inspection lacked finesse, and its improvement calls for further investigation.The UMLS CUIs were mapped to SNOMED CT identifiers . From a There are reusable datasets for evaluating relatedness made of UMLS CUI pairs:Medical coders set : 101 CUIMedical Residents Relatedness Set : 588 CUIUMLS MRREL table : It has All reusable datasets mentioned above lack evidence (quotes with references) from the biomedical literature. Extracting clinically useful information automatically from free text in PubMed/MEDLINE may require a natural language understanding of statements containing relevant relations for health care. Hence, extracting treatments with therapeutic intent by analogical reasoning from embeddings (423K n-grams from the PMSB dataset) is an ambitious goal. Our SemDeep approach is knowledge-based, underpinned by embedding analogies that exploit prior knowledge. Biomedical facts from embedding analogies  are potentially useful for clinicians. The heuristic offers a practical way to discover beneficial treatments for well-known diseases.Learning from deep learning models does not require a massive amount of data. Embedding analogies are not limited to pairwise analogies; hence, analogical reasoning with embeddings is underexploited.


It is publicly available and the underlying code is open source. Since our last update two years ago, we have had 10 releases to maintain and continuously improve evidence for target–disease relationships from 20 different data sources. In addition, we have integrated new evidence from key datasets, including prioritised targets identified from genome-wide CRISPR knockout screens in 300 cancer models (Project Score), and GWAS/UK BioBank statistical genetic analysis evidence from the Open Targets Genetics Portal. We have evolved our evidence scoring framework to improve target identification. To aid the prioritisation of targets and inform on the potential impact of modulating a given target, we have added evaluation of post-marketing adverse drug reactions and new curated information on target tractability and safety. We have also developed the user interface and backend technologies to improve performance and usability. In this article, we describe the latest enhancements to the Platform, to address the fundamental challenge that developing effective and safe drugs is difficult and expensive.The Open Targets Platform ( Other recent additions, such as the chemical probes or the target enabling packages require a consistent manual curation effort as data is scattered across different resources . A minimum target priority score of 40 is used , providing a dataset of synthetic-lethality evidence for the association between 623 genes and 19 cancer types  provide a rich source of disease-associated genomic loci. Nevertheless, it remains a long-standing challenge to link these loci to targetable causal genes. The Open Targets Genetics Portal addresses this problem by interpreting manually curated associations from the GWAS catalog, as well as independent signals from GWAS with publicly available summary statistics, most importantly the UK Biobank GWAS data ,24,25. TThe inclusion of the Genetics Portal evidence in the Open Targets Platform supersedes the previous GWAS catalog evidence, removed in release 20.02 Figure . The incA few challenges remain after evidence is appropriately collected and normalised Figure :Annotation sparsity. Evidence can sometimes be informative to discriminate targets in similar diseases or phenotypes. For example, a target associated with Crohn's disease could also be associated, albeit indirectly, with the more general term ‘inflammatory bowel disease’. To systematically benefit from this evidence, we take advantage of the ontological properties of EFO by propagating evidence from any child node to its parent nodes all the way up to its corresponding therapeutic area(s)  Figure , i. In t) Figure , EFO v3 Target–disease evidence scoring. Deciding what constitutes a strong evidence source and how it compares with similar data sources remains an ongoing question and is open for interpretation. The lack of appropriate gold standards across therapeutic areas limits the effectiveness of appropriate benchmarks. Despite these limitations, we score all our evidence in the range between 0 and 1, providing an informed estimation on the strength of the association between the target and the disease , frameworks  and technologies . Partially released during 2020, these technologies will enable new functionalities in the Open Targets Platform.https://www.targetvalidation.org/ and there are five releases each year that include updates to existing annotation and evidence data and integration of new datasets and features based on user requests and scientific advancements. Details of each release are available from the Open Targets blog and the Platform release and technical notes. The output data from our pipelines remains accessible in different formats depending on the user's individual requirements. It is available through the user interface, various REST API endpoints, our Python client, and our Data Downloads page. All data, which includes the input files used for each release pipeline run, can also be downloaded from EMBL-EBI’s FTP service , Elixir (https://elixir-europe.org/services/tag/interoperability-and-standards), and the Gene Curation Coalition https://thegencc.org/), harmonizing different biomedical datasets still constitutes one of the key bottlenecks when it comes to interpreting the available knowledge. Several other resources including Pharos  and the statistical genetics analyses from the Open Targets Genetics Portal. This has included clinical trial and drug target information for COVID-19 to aid in the effort to identify targets for the repurposing of existing drugs or development of novel treatments to help fight the pandemic. We have reviewed and reiterated our scoring of evidence, and enhanced the prioritisation of targets for particular diseases with the addition of tractability and safety information generated through Open Targets informatics projects. We are implementing a redesign of the Portal infrastructure, data pipelines and frontend to enhance usability and allow for more complexity in the data for the future.The emergence of new large-scale technologies with increased evidence granularity (e.g. scRNAseq or CRISPR), as well as the increasing interest to stratify diseases based on these or other phenotypic readouts, introduces challenges that the Platform will seek to address in the near future through adaptation of our data model and new ways to represent data. Determining the relative importance of different pieces of evidence when combining the available information to suggest potentially successful targets for drug discovery is a further challenge. The inclusion of new systematic data sources, such as the state-of-the-art GWAS data from Open Targets Genetics Portal, revealed the need to appropriately weight and benchmark scored evidence against other orthogonal data sources, a challenging task due to the lack of appropriate gold standards. Moreover, recent studies have pinpointed to the usefulness of expanding experimentally determined evidence using protein interaction networks ,40. NetwTo address a diverse set of challenges, and to ensure that the data within the Platform remains at the cutting-edge to inform drug discovery decision-making, we will work alongside our data providers and the Open Targets consortium members to introduce innovative solutions for the systematic identification and prioritisation of targets based on diverse and complex publicly available data.gkaa1027_Supplemental_FileClick here for additional data file.


Information regarding the distribution of evidence grades in psychiatry and psychotherapy guidelines is lacking. Based on the German evidence- and consensus- based (S3) psychiatry and psychotherapy and the Scottish Intercollegiate Guidelines Network (SIGN) treatment guidelines, we aimed to specify how guideline recommendations are composed and to what extent recommendations are evidence-based.Data was collected from all published evidence- and consensus-based S3-classified psychiatry and psychotherapy guidelines. As control conditions, data from German neurology S3-classified guidelines as well as data from recent SIGN guidelines of mental health were extracted. Two investigators reviewed the selected guidelines independently, extracted and analysed the numbers and levels of recommendations.On average, 45.1% of all recommendations are not based on strong scientific evidence in German guidelines of psychiatry and psychotherapy. A related pattern can be confirmed for SIGN guidelines, where the mean average of recommendations with lacking evidence is 33.9%. By contrast, in the German guidelines of neurology the average of such recommendations is 16.5%. A total of 24.5% of all recommendations in the guidelines of psychiatry and psychotherapy are classified as level A recommendations, compared to 31.6% in the field of neurology and 31.1% in the SIGN guidelines. Related patterns were observed for B and 0 level recommendations.Guidelines should be practical tools to simplify the decision-making process based on scientific evidence. Up to 45% of all recommendations in the investigated guidelines of psychiatry and psychotherapy are not based on strong scientific evidence. The reasons for this high number remain unclear. Possibly, only a limited number of studies answer clinically relevant questions. Our findings thereby question whether guidelines should include non-evidence-based recommendations to be methodologically stringent and whether specific processes to develop expert-opinion statements must be implemented. Evidence-based medicine aims to integrate the best research evidence with clinical expertise and patient values . In thisIn Germany, the Association of the Scientific Medical Societies (AWMF) started coordinating the development of guidelines in 1995. Since 1997 guidelines are cost-free and publicly available online . In contRegardless of any financial consideration, the development process of German guidelines follows standard operation procedures, developed by the AWMF aiming at providing the same level of quality across guidelines. More than 20 years ago clinical guidelines in Germany were not subject to standardisation, so the AWMF started an extensive program in 1998 to improve their quality . For thiIn that regard, three major classifications for guidelines can be distinguished in Germany. Groups of experts develop in an informal consensus (‘expert recommendations’), which results in so called S1-classified guidelines. By comparison S2-classified guidelines need a formal multidisciplinary consensus. Since 2004, the differentiation between S2e-classified (‘evidence-based’) and S2k-classified (‘consensus-based’) exists. S2k-guidelines are not evidence-based and recommendations are not graded, but the consensus process is standardised and has to follow standard operating procedures. For the development of S2e-guidelines, a systematic research process and the selection and evaluation of evidence are required. S3-classified guidelines are evidence- and consensus-based and have the highest methodological quality as well as the most expensive development process. After a structured literature research and rating of the evidence, they have to fulfil different DELBI criteria [Since 1998 the total number of published medical guidelines in Germany has been growing continuously, but over the last years the proportion of S1 to S3 guidelines has changed. In 2002, 445 S1-guidelines, 121 S2-guidelines and 17 S3-guidelines were available. In 2016, 108 S3-guidelines were available and 374 guidelines were newly registered [High methodological quality is essential to guarantee recommendations based on scientific evidence with practical relevance and to minimise possible conflict of interest biases. Moreover, the amount of recommendations and their respective grading levels are important from the view of usability. However, the way of how to grade evidence and to develop guidelines differs between countries. Nonetheless, most guidelines use the original or the adapted version of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) process .Despite important efforts to harmonise and improve the guideline development processes, guidelines do have a relevant proportion of non-evidence-based recommendations that are presumably expert opinions. In 2014 one study of the Scottish Intercollegiate Guidelines Network (SIGN)  evaluated the 42 national guidelines with respect to the quantity and quality of the given recommendations. The authors observed that only 24% of all recommendations were based on high scientific evidence (denoted as level A recommendations). Remarkably, more than a third of all recommendations were expert-based recommendations and more than a half of the recommendations were based on limited scientific evidence . MoreoveThe challenge to develop high-quality guidelines in psychiatry and psychotherapy is all the greater compared to somatic disciplines due to the implementation of patient reported outcomes. So called “soft endpoints”, which are composed of patient and observer reported outcomes collected via self- or observer-rated clinical rating scales, questionnaires, or interviews . MoreoveThis study focuses on all available German S3-guidelines in psychiatry and psychotherapy in their respective most recent versions (status July 2019). The S3-guidelines of neurology were investigated to serve as a first control group to allow a comparison to a medical field with an overlap in diagnoses or treatments. The recent SIGN guidelines of mental health (status July 2019) were investigated as second control group taking into account the study from Baird and colleagues published in 2014 . MoreoveTo identify guidelines of interest, we screened the AWMF database, the webpages of the publishing medical societies and the SIGN webpage for all guidelines covering psychiatric or neurological disorders. Two independent investigators (MH and LL) selected and evaluated all AWMF S3-guidelines published by the German Society for Psychiatry and Psychotherapy (DGPPN) and the German association for Neurology (DGN) and the respective SIGN guidelines. If investigators had different results, these results were discussed with a third investigator (AH) until a final consensus-decision was reached.n = 18), we included expired guidelines and guidelines under review as well (n = 4). In contrast to the SIGN guidelines, where expired guidelines are automatically removed after 10 years, the German guidelines are still available and used in clinical practice years after their expiration . The dementia guideline as a joint publication of DGPPN and DGN was listed twice but was analysed as psychiatric guideline. 5 of 48 currently available SIGN guidelines were guidelines covering the domain of mental health. Taking the information of the SIGN 50 developer’s handbook into consideration, these five guidelines were methodologically comparable to German S3-guidelines [15 from 17 guidelines, published by the DGPPN were S3-guidelines. The two excluded guidelines were guidelines classified as S1 and S2: the S1-guideline for disorders of sexual preferences and the S2-guideline for personality disorders. We included the two content related S3-guidelines “Functional body complaints” and “Post-traumatic stress disorder”, published only by the AWMF, resulting in a total of 17 S3-guidelines of psychiatry and psychotherapy. The DGN published a total of 96 guidelines with only five S3-guidelines. Apart from recent guidelines , “Expertenkonsens” (expert consensus) and “GCP” . The recommendation levels A, B and 0 were consistently used in all analysed guidelines . An analysis of the size of the SIGN consensus groups was not performed due to the mentioned differences in the development process and in the composition of these groups. Since guidelines sometimes applied different wording for grading non-evidence-based recommendations, we summarised those recommendations as “clinical consensus points” (CCP). This label included the terms “We based all analyses on the long versions of the guidelines. The rational for using the long version was, that we initially expected that the long and short versions would include the same number of recommendations. However, as detailed below, we detected inconsistencies regarding the number of recommendations in seven of the German S3-guidelines. All analyses were computed using IBM SPSS 25  with a significance level of α = 0.05. Descriptive statistics include means, standard deviations and maximum and minimum counts. Mean relative frequencies of the four recommendation grades  for each of the three study groups separately  were calculated based on the individual relative frequencies of recommendation grades within each guideline (absolute count of a given recommendation grade (e.g. A or CCP)/absolute total count of all recommendations). Explorative Pearson correlation analyses were conducted to investigate correlations between the size of the consensus group and the relative frequencies of a given recommendation grade and between the amount of CCP recommendations and the total of recommendations in German S3-guidelines.In July 2019, 18 out of the 22 German guidelines were available as a recent or updated version: 14 of them were guidelines of psychiatry and psychotherapy and four guidelines of neurology. The four remaining guidelines were under revision. 5 of the available and current 48 SIGN guidelines were guidelines covering the domain of mental health. For 14 of the German S3-guidelines a short version was available. In 7 out of 11  of the gIn the AWMF psychiatry and psychotherapy guidelines, 354 level A recommendations with a mean relative frequency of 24.5%  and a mean absolute frequency of 20.8  could be detected. In contrast, the AWMF guidelines of neurology had a total of 76 level A recommendations with a mean relative frequency of 31.6%  and a mean absolute frequency of 15.2 . The SIGN guidelines of mental health had a total of 46 level A recommendations with a mean relative frequency of 31.1%  and a mean absolute frequency of 9.2 .The mean relative frequency of level B recommendations of the AWMF guidelines of psychiatry and psychotherapy was 18.5%  with a total of 320  level B recommendations. In the AWMF neurology guidelines, the mean relative frequency was 29.4%  based on a total of 88 recommendations . In the SIGN guidelines, a total of 45 level B recommendations with a mean relative frequency of 29%  and an absolute mean frequency of 9  were counted.A related pattern could be observed for level 0 recommendations, with a total of 210, a mean relative frequency of 11.8%  and a mean absolute frequency of 12.4  in the AWMF guidelines of psychiatry and psychotherapy. In the guidelines of neurology the mean relative frequency of level 0 recommendations was 22.5% , with a total of 54 , while the SIGN guidelines had a mean relative frequency of 6%  with a total of 12 .The AWMF guidelines of psychiatry and psychotherapy had a higher relative mean frequency  of CCP recommendations based on a total number of 752  compared to the AWMF guidelines of neurology with a relative mean frequency of 16.5% , based on a total of 55  of CCP recommendations. The SIGN guidelines had a mean relative frequency of 33.9%  based on a total of 92 CCP recommendations   for the German guidelines of psychiatry and psychotherapy, but no statistically significant correlation for the neurology guidelines . For the SIGN guidelines of mental health, the analyses showed also a statistically significant positive correlation . Analysing the German S3 guidelines of psychiatry and neurology together, as well as all guidelines included in this study, the number of CCP recommendations correlated significantly with the total number of recommendations. .Exploratory correlation analyses between the number CCP recommendations and the total amount of recommendations showed a statistically significant positive correlation . All other correlation analyses for both AWMF groups separately, did not show significant correlations between the relative frequency of a given recommendation grade and the size of the guideline group. Analysing all AWMF guidelines (N = 22), a statistically significant negative correlation for grades B (r = − 0.498 and p = 0.018) and 0  and a statistically positive correlation for CCP recommendations  with the size of the guideline group were detected. These correlations indicate that in guidelines with increasing size of the consensus groups, the number of CCP recommendations increases as well. For B and 0 recommendations, the reverse relationship was observed. For grade A recommendations, no significant correlation was detected .For the AWMF guidelines of psychiatry and psychotherapy only the exploratory correlation analyses between the size of the consensus group and the relative frequencies of grade 0 recommendations showed a statistically significant negative correlation  could indicate that in neurology the responsible society tries to avoid the development of such time- and money-consuming guidelines. Concerning the SIGN guidelines, the mean frequency of 33.9% of CCP recommendations shows that the observation of a high number of non-evidence-based recommendations in guidelines of psychiatry and psychotherapy seems to be an international phenomenon. Based on the additional analyses of the correlation between the total of recommendations and CCP recommendations, the findings of Baird et al. seem to affect only the German guidelines of psychiatry and psychotherapy. Analysing all SIGN guidelines Baird et al., detected a positive correlation between the increasing number of recommendations and the proportion of CCP recommendations. In our analyses we could confirm this relationship showing that this is an international phenomenon.This overall effect and the limited scientific evidence in a magnitude of psychiatry and psychotherapy guidelines are critical, because such effects may support the self-confidence and interests of expert groups . The impAdditionally, it appears of importance to differentiate between “expert opinion” and “CCP” recommendation: “Expert opinion” describes a level of evidence, not a grade of recommendation. Even a CCP recommendation does not automatically mean, “lack of scientific evidence” nor it is only “eminence based” . Though In this context, it may be possible that small groups of experts and stakeholders will drive the practice of a specialty in one direction. At the same time, respective high-quality clinical trials or meta-analyses in psychiatry and psychotherapy beyond studies regarding efficacy and effectiveness of pharmacological and psychotherapeutic interventions are lacking. In our view, there seems to be two possible options to address the outlined issue of conflicts of interest: The first option would be, not to provide any recommendations for clinical scenarios where no evidence is available yet. However and as discussed before, treatment guidelines without any non-evidence-based recommendations while having a higher methodological standard, would potentially ignore many clinically important scenarios and questions. The latter may result in a guideline that might be less useful in clinical practice due to the lack of importance and applicability for a variety of questions. Moreover, one should bear in mind that the absence of evidence does not equal the evidence of absence. For example, until today no randomised-controlled trial has been conducted to show that using a parachute is more effective to save lives than not using parachutes . The secAnother aspect is that the composition of the guideline-developing group could influence the amount of CCP recommendations. Following Baird and Lawrence , a high Our analysis showed also that in some guidelines the number of recommendations differs between the short and the long version. Furthermore, in some guidelines the grading differed between both versions (e.g. in the guideline of Obsessive Compulsive Disorder recommendation number 4–10 is graded as CCP in the short version and as a statement in the long version ). Such fAs a limitation, we only included guidelines of neurology as control group from Germany, which consisted only of five S3-guidelines. By comparison, analyses such as the one presented by Baird and Lawrence for the SIGN guidelines are not available for Germany. Thus, we cannot derive statements with respect to other medical disciplines. Moreover, with a total of 96 the number of neurology guidelines is very high and we assume that the only available five S3 guidelines may have had a stricter methodology approach than the remaining 91 resulting in an overestimation of the frequency of evidence-based recommendations. Secondly, our correlative findings are only significant when both disciplines are analysed. Another limitation is, that we solely analysed the current published SIGN guidelines of mental health for an international comparison, which consisted also only in five guidelines. Moreover, four of the German guidelines were out of date, while we used only current SIGN guidelines. This must be considered also as a potential limitation.In summary, we were able to show a high frequency of non-evidence-based recommendations in S3-guidelines of psychiatry and psychotherapy. Furthermore, we could demonstrate that the size of the involved group seems to affect and promote the development of such recommendations. As guidelines should be practical tools to simplify the clinical decision-making processes based on scientific evidence in times of multiple treatment options, our findings support a view that new strategies to deal with clinically relevant questions where no evidence is available are needed. For reasons of practical applicability, guideline developers in general should be careful about the included amount of recommendations and the scientific evidence on which the recommendations are based.


Inverted papilloma (IP) is an unusual type of benign tumor that has high recurrence rates and the potential to transform into squamous cell carcinomas (SCC). The mechanism of the transformation process from IP to IP-SCC is uncertain and there is no consensus regarding the best practice for IP-SCC detection. The goal of this study is to identify the best clinical methods to detect for IP-SCC.An evidence-based review was performed using Medline and Ovid to obtain all articles up to October 10th, 2019 pertaining to identification of IP malignant transformation. All manuscripts discussing clinical methods or biomarkers were included.Based on clinical research studies, convoluted cerebriform pattern and apparent diffusion coefficient values on Magnetic Resonance Imaging (MRI) can help differentiate benign IP from SCC and increased SUVmax on PET/CT is associated with higher probability of malignancy although not as specific. No consensus about the best biomarker for IP-SCC has been reached among researchers and continues to be exploratory.Endoscopy with biopsy is the gold standard practice to identify IP-SCC; however, MRI is the preferred imaging modality to recognize malignant transformation in cases where biopsy is difficult. Multiple biomarkers have shown positive results, but no single indicator with clinical significance for monitoring malignant transformation process has been found. Inverting papilloma (IP) is a challenging condition to manage in the sinonasal cavity. A recent literature review has shown that the best surgical approach to the management of IP is endoscopic . HoweverThe reported incidence of malignancy in IP varies greatly in the literature, and has been reported as high as 27%, but most commonly is seen to be between 5 and 10% –8. The mWhile the most common surveillance technique is endoscopy  followedThis article was written following the evidence-based-review method established by Rudmik et al.  A systemThe manuscripts of all included studies were reviewed. For clinical surveillance techniques, the level of evidence for each paper was given, followed by a grading system Table  based onAfter reviewing 249 papers, the results were summarized together under ‘clinical surveillance techniques’, ‘computed tomography’, ‘magnetic resonance imaging’, ‘PET-CT with FDG’, and ‘biological markers’.Certain general demographics might influence how clinicians schedule surveillance plans for patients. Therefore, in order to guide clinical decisions, All studies that discussed the location, follow-up and clinical risk factors associated with IP-SCC were included in this review.Most IP-SCC originated from the ethmoid region, maxillary sinus and lateral nasal wall ; howeverIP requires follow-up of more than 3 years, due to the high rate of delayed recurrences . The folIt is reasonable to frequently follow up with patients who are at a higher risk of malignancy. Unfortunately, there is not enough high-level evidence about the risk factors leading to malignant transformation. In a grade C study, it was found that factors associated with significantly poor prognosis were advanced-stage, metachronous tumors, and cranial base or orbital invasion . SeveralAggregate grade of evidence: NABenefit: direct visual inspection, easily accessible, biopsy is the gold standard of confirming presence of IP-SCC.Harm: bleeding and infection risk, chance of missing the tumor.Cost: lowBenefit harm assessment: Benefit outweighs harm.Value judgement: Endoscopy followed by a biopsy is the first line method for IP-SCC identification.Recommendation level: Recommended.Intervention: First line method for long term surveillance.Clinical examination, particularly endoscopic exams, are crucial for surveillance. Currently the confirmation diagnosis of IP-SCC depends on histopathological results. Clinicians use endoscopic sinonasal examination to monitor for a recurrence of an IP. However, it is often difficult to detect a malignant transformation of a benign IP from visual inspection, particularly if the tumors are not easily visible by nasal endoscopy. Moreover, IP-SCC often do not involve the entire tumor, hence the biopsies can be non-representative and are not highly reliable .AggregaAggregate grade of evidence: NA (only 1 Level 4 study)Benefit: No evidence differentiating IP from IP-SCC on CT scan.Harm: Concerns regarding radiation exposure.Cost: ModerateBenefit harm assessment: No benefit for IP-SCC identification.Value judgement: Require more researchRecommendation level: No recommendationIntervention: None.There are no studies that support the use of CT to identify the presence of IP-SCC Table . A levelAggregate grade of evidence: Level 2: 2 studies; Level 3: 3 studies;Benefit: MRI imaging feature can help differentiate IP from IP-SCC.Harm: Repeated exposure to gadolinium, claustrophobia, and heating/dislodgement of implanted metal.Cost: Moderate/High.Benefit harm assessment: Benefit outweighs harm.Value judgement: Require more research.Recommendation level: Recommended.Intervention: Using CCP and ADC value, MRI has capacity to identify IP-SCC. MRI is recommended if biopsy is not achievable or there are concerns of the accessible part of the tumor to biopsy inaccurately representing the entirety of the tumor.Five studies were included in the review Table . ConvoluAggregate grade of evidence: Level 2: 2 studies; Level 3: 2 studies; Level 4: 2 studies.Benefit: SUV max can increase suspicion of IP-SCC.Harm: Radiation exposure.Cost: High.Benefit harm assessment: Equal balance of benefit to harm.Value judgement: Challenging to recommend PET-CT use in the identification of IP-SCC due to contradicted research results and costs.Recommendation level: None.Intervention: None.PET-CT studies have shown that IP patients have higher SUV max than non IP patients, and IP-SCC patients have higher SUVs than IP patients Table  37]. Ho. Ho37]. Multiple genetic factors have been identified as potential malignant transformation markers for IP.P53 is a transcription factor controlling cellular proliferation and apoptosis via the regulation of genes involved in cell-cycle arrest, DNA repair, and apoptosis . Ten retP21 cyclin-dependent kinase inhibitor is a cell cycle progression control factor. It promotes cell cycle arrest in response to a variety of stimuli. The inhibitory effect of P21 on cell cycle progression correlates with its nuclear localization. P21 can be induced by both p53-dependent and p53-independent mechanisms , 41. TwoMuscle segment homeobox gene MSX2 is implicated in cellular differentiation and survival. Zhang et al.  noted siPDCD4 is a tumor suppressor gene that has been previously implicated in the progression of many types of tumours. Compared to controls, Wang et al. noted a significant decrease in PDCD4 expression in IP . FurtherTargeted amplicon sequencing by Yasukawa et al. identified significant genetic mutation of the KRAS gene in SCC-associated and dysplastic IP as compared with nondysplastic IP . The usePhosphatase and tensin homologue, PTEN, is a tumor suppressor gene. Zhang et al. found that decreased expression of PTEN was associated with both IP dysplasia and malignant transformation . While KSquamous cell carcinoma antigen (SCCa) is one of the most reliable tumor markers for squamous cell carcinoma. Three studies associated a high level of serum SCCa with IP progression, growth, and recurrence –56. ThisFascin is an actin cross-link binding protein required for the formation of actin-based cell-surface protrusions and cell motility. Fascin up-regulation in lung, gastric, breast and hepatobiliary carcinomas correlates with aggressiveness and decreased survival. Two retrospective studies examined the association between fascin and IP-SCC by using immunohistochemical staining of specimens taken from different groups: IP, IP with high dysplasia, IP with SCC and normal control tissue. It was found that fascin expression increased gradually and significantly with the progression and severity of dysplasia –59.Survivin, an inhibitor of apoptosis, is also found to be potentially correlated with SCC. Nuclear survivin expression was significantly higher in SCCs than in IPs in one study .Two studies found that increased expression of COX-2, a rate-limiting enzyme in prostaglandin synthesis, might be linked with malignant transformation , 61, 62.E-cadherin and β-catenin are cell adhesion molecules commonly used to mark the epithelial to mesenchymal transition . CorrespKi-67 is a marker of cell proliferation, in immunohistochemical studies of protein expression, Tsou et al. observed significantly greater Ki-67 staining in IP-SCC as compared with non-recurrent IP [Sahnane et al. identified that occupation exposure increases LINE-1 hypomethylation which might be an epigenetic marker .Because IP shares certain clinical characteristics with recurrent respiratory papillomatosis, namely high recurrence, confined aggressiveness, and malignant transformation potential, Jalilvand et al. contend that IP likely also has an infectious etiology . IdentifIn IP specimens, HPV positivity ranged between 9 and 60%. Comparatively, controls ranged between 0 and 7.6% –77. The Although the relationship of HPV infection to malignant transformation remains somewhat controversial , 82, a rUsing evidence-based and systematic scientific review methods, this study aimed to elucidate the usefulness of numerous clinical investigative options for characterising IP-SCC. IP-SCC can either be found at the initial presentation, or during follow-ups when recurrences are identified. The literature does not provide findings that differentiate between the two scenarios. The senior authors recommend that the endoscopic exam followed by a biopsy be the gold standard in identifying IP-SCC. However, there are clinical scenarios where the endoscopic exam may not be possible, or concern that the biopsies are non-representative. We recommend clinicians follow patients with MRI. Given the high recurrence rate in the first 2 years following the original resection, and the mean interval between first resection and malignancy is about 40 months, our clinical practice is to do serial MRI scans for 3 years and then a repeat MRI at 5 years when the endoscopic exam may not be possible, or concern that the biopsies are non-representative. Also, in these scenarios, a complete head and neck examination is recommended on a yearly basis due to the sporadic sites of IP-SCC can be found.The senior authors do not use CT in elucidating the presence of IP-SCC due to lack of strong evidence. While there is evidence to support the use of PET-CT, the senior authors do not use PET-CT to determine the presence of IP-SCC due to costs and access. If the MRI is suspicious, the authors will take the patient to the operating room to get a frozen section biopsy. If it is positive for SCC, a PET-CT is done post-operatively to determine if there is metastatic spread of the tumor.The future of surveillance will include biological markers. While this study clearly illustrates there is no utility in using biomarkers at the present time, there are several biological markers that have shown great potential as prognostic indicators. Genetic factors such as MSX2, PDCD4, KRAS, PTEN are thought to play an important role in malignant transformation. Several proteins, including SCCa, fascin, survivin, COX-2, cell adhesion molecules and Ki-67 are found in higher levels in IP-SCC than in IP. Further research in this area may allow for an accessible test for clinicians to use to help risk stratify IP patients.Based on the evidence level of various methods to identify IP-SCC, while histological diagnosis is the gold standard for the diagnosis of malignant transformation, clinicians should keep in mind of the sampling error of endoscopic biopsy. MRI as a standalone technique is the preferred modality in recognizing malignant transformation of inverting papilloma. Further clinical trials are required to reinforce these findings. This study also identified a number of biological markers that may become key markers in the future for identifying patients that require close follow-up.


Several conversion factors can be integrated into the development of an eHealth tool to make it inclusive: (1) providing physical, technical, and financial access to eHealth; (2) enabling the integration of people at risk of SHIs into the research and development of digital projects targeting such populations (co-design or participatory research); (3) promoting consistency between the digital health literacy level of future users (FUs) and the eHealth tool; (4) developing an eHealth tool that is consistent with the technological skills of FUs; (5) ensuring that the eHealth tool is consistent with the help-seeking process of FUs; (6) respecting the learning capacities of FUs; and (7) being sensitive to FUs’ cultural context. However, only little empirical evidence pointing out how these conversion factors can be integrated into an effective eHealth tool is available.On the basis of Amartya Sen’s theoretical framework of social justice, the objective of this study was to explore how these 7 conversion factors can be integrated into an eHealth tool for caregivers of functionally dependent older persons.This study was based on a social justice design and participant observation as part of a large-scale research project funded by the Ministère de la Famille through the Quebec Ami des Aînés Program. Data were collected by recording the preparation sessions, the co-design and advisory committee sessions, as well as the debriefing sessions. The results were analyzed using Miles and Huberman’s method.A total of 78 co-designers participated in 11 co-design sessions, 24 preparation sessions, and 11 debriefing sessions. Of the 7 conversion factors, 5 could be explored in this experiment. The integration of conversion factors has been uneven. The participation of FUs in the development of the tool supports other conversion factors. Respecting the eHealth literacy level of FUs means that their learning abilities and technological skills are also respected because they are closely related to one another and are therefore practically difficult to be distinguished.Conversion factors can be integrated into the development of eHealth tools that are intended to be inclusive and contribute to curbing SHIs by integrating FU participation into the tool design process. There are a few, but not many. Of course, caregivers today and caregivers in 20 years' time will not be the same.Community worker CoD8Several verbatim extracts support more than one conversion factor. As discussed earlier, less tech-savvy people may have difficulty using a digital tool. In this sense, several verbatim extracts have been categorized into these 2 conversion factors. The same applies to the process of help-seeking, which may be hampered by limited access to technology and poor technological skills. eHealth literacy is defined as the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions. Therefore, technological competence and the help-seeking process should be considered as literacy skills, as reflected in the categorization of verbatim extracts.It should be noted that there is a co-occurrence of the conversion factor related to FU participation with the other conversion factors. It appears that understanding conversion factors such as eHealth literacy, technological skills, and the help-seeking process requires FU participation in the development of the tool.The objective of this study was to explore how 7 conversion factors can be integrated into the design of an eHealth tool.The results suggest that conversion factors can be integrated into the development of an eHealth tool at different levels. The technological skills of FUs can be taken into account when choosing the functionalities to be integrated into the tool and the format of the technology used (computer vs tablet). However, these decisions are not easy to make. In this project, the technological skills of caregivers ranged from poor to advanced. Some co-designers did not use technologies because they said they lacked the skills, whereas others indicated that they mastered them very well. One might think that co-designers’ perception was biased by stereotypes related to poor technological competence among older persons. However, statistics show that 19% of adults believe that they have poor skills and they do not use the internet, suggesting that 81% of adults feel that they have moderate to good skills . In addithird-party assistance.The conversion factor related to eHealth access was not addressed by co-designers in terms of access to efficient bandwidth, for example, in rural areas, or in financial terms, but rather from the angle of technological competence and eHealth literacy level. This may reflect the fact that in Quebec, financial or material access to eHealth is not perceived as a major problem. Nevertheless, we must remain cautious and vigilant to ensure that access to eHealth becomes universal because there is concrete evidence that internet access is not globally available . Discussions with co-designers led to the exclusion of the use of technology for the so-called less competent individuals rather than adapting the eHealth tool or training them to use it. There seems to be a consensus among caregivers that they are not interested in using technology and that it will be necessary to supplement the tool to support them in their help-seeking process. One of the avenues discussed in this study is to have a third party use the eHealth tool to search for services. This possibility is of particular interest in the context of caregiving because one of the triggers for seeking formal services is also the intervention of a third party (paper in preparation). The third party would act as a mobilizer both to identify the services available through the eHealth tool and to encourage the use of formal services. The inclusiveness of the tool would be further enhanced by its availability to third parties, who can be friends, neighbors, pharmacists, and so on. Instead of referring to access, this conversion factor could be renamed The conversion factor related to the cultural context was minimally integrated by explicitly addressing the language. The integration of this factor goes further than simply making the tool available in several languages. Can it be argued that this conversion factor was implicitly integrated by involving the FU? Can we assume that the choices made by co-designers were necessarily in accordance with their beliefs, values, and habits? In the case of this experiment, there was little cultural variation among the co-designers as the majority were French Canadian, although this was not what was initially desired. The only variabilities present were related to the particularities of each region, which were integrated by selecting services by sector rather than by region, for example. However, as help-seeking is a process that varies from one cultural community to another, caregivers from an ethnic minority may not feel concerned by the tool . Sen argSimilar to the cultural context, knowledge of the caregivers’ help-seeking process was also made possible through the discourses of the co-designers, especially the caregivers themselves. Unlike other design methodologies where the FU is questioned punctually, co-design has allowed us to reflect on the process of help-seeking, which we may not have thought to address in an interview or questionnaire. Co-design allowed the tool to ensure that each of its development stages was consistent and adapted to the caregivers’ help-seeking process.For eHealth literacy, the team used literature to help developers ensure that their tool would require no more than basic literacy. Elements include language, cognitive overload, and visual exploration by people with low literacy skills, among others, with cross-references to conversion factors such as technological skills and learning abilities. In concrete terms, it is through the content of the site (the choice of words and sentences) that we were able to keep the literacy level requirement to a specific level. However, the simple and refined nature of the tool intended by co-designers, especially in relation to the conversion factor related to technological skills, is also consistent with the principles found in the reference documents related to literacy.In summary, conversion factors, particularly compliance with the desired eHealth literacy level, the help-seeking process, and cultural context, were integrated into the eHealth tool by the co-designers’ discourse and, more importantly, by the caregivers themselves.In future research on conversion factors, it does not seem useful to continue to focus on the 7 factors. A range of population-based measures are underway to ensure physical and financial access to technology, and a number of alternatives , although imperfect, are now available. Efforts should be streamlined to pressuring governments to guarantee access to technologies for all citizens in the same way that it ensures access to hydroelectricity, for example. However, it seems essential to continue to look for solutions to the access problems related to technological skills. Nevertheless, this issue can be addressed under the conversion factor related to eHealth literacy. According to some authors, eHealth literacy is composed of 2 types of skills: general skills that include traditional literacy , media literacy , and information literacy (information seeking and understanding) as well as specific skills that include computer literacy (IT skills), health literacy , and science literacy (scientific processes and outcomes) ,55. LearIf we had to map the conversion factors to be considered in the development of an eHealth tool to date, here is what it would look like .The participation of the FU would be the central conversion factor that allows the integration of the other conversion factors. Learning abilities would likely be an integral part of the concept of eHealth literacy. However, the risk associated with this integration would be to leave out the findings generated by the project from the perspective of education and cognitive science about learning in the digital context. The interconnectedness of access to technology, technological skills, and eHealth literacy may raise questions about whether these conversion factors are all necessary or whether they could be functionally grouped under a single concept. However, similar to learning abilities, there would be a risk of losing all the knowledge related to these concepts, each of which could be the subject of future research.The rigorous approach of this study is based on the scientific quality criteria identified by Guba and Lincoln  for the To ensure the credibility and dependability of the study, data collection was spread over a period of 1.5 years and it involved a variety of participants  having various profiles . In addition, various data were collected from the recordings of the preparation, co-design, and debriefing sessions. The consistency between data and results is supported by the supervision of 2 researchers (the author's thesis supervisors), one of whom was not involved with the project. The reader was invited to judge the consistency between the verbatim excerpts and the results presented. In addition, the links between the data (synthesis documents) and the coding (deductive and inductive) were also made available to the supervisors. Finally, the synthesis documents were verified by theoretical triangulation and by researchers (KL and MC).Transferability was ensured by producing as complete a description as possible of the contexts related to the research process, including the profile of the participants. The reader will, thus, be able to determine the degree of transferability of the results of this research in other contexts.The first author (KL) used a reflective approach by highlighting her preconceived ideas, participating in each debriefing meeting, and adding comments to the documents.This study has some limitations. As already mentioned, cultural diversity could not be represented through co-designers, which limited the possibility of studying the conversion factor in relation to the cultural context. In addition, the caregivers who participated as co-designers were mostly retirees and hence not representative of caregivers among the active population. These caregivers could have influenced the study of conversion factors related to the help-seeking process, technological competence, and eHealth literacy. Similarly, most of the caregivers recruited were already service users. Caregivers at the beginning of the help-seeking process could also have contributed to the study of the process-related conversion factor. Although in line with Quebec statistics, where the majority of caregivers are women (approximately 58%) , as is tFinally, the evolving nature of the project related to the development of an eHealth tool and the inherent chronology meant that data saturation was not obtained for each of the conversion factors. However, this was anticipated given the exploratory nature of this project.This project contributed to the empirical exploration of 7 conversion factors and to the modeling of the relationship among them. Bonvin and Farvaque  argued tConversion factors can be integrated into the development of eHealth tools that are intended to be inclusive and contribute to the reduction of SHIs by integrating the participation of FUs into the design of the tool. However, there is currently no way for the developers of the eHealth tool to rapidly and effectively ascertain whether these conversion factors are well integrated into the development of their tool. The growing development of eHealth around the world, especially in this time of a pandemic, and the governments’ commitment to combating SHIs provide a unique opportunity to reflect on good practices for an inclusive and healthy digital society. To pursue this reflection, it will be important to identify empirical indicators that can measure these integration factors during and after eHealth tool development and guide developers in the designing of inclusive eHealth tools and educational technology that genuinely contribute to reducing SHIs.


Brown adipose tissue (BAT) is the primary non-shivering thermogenesis organ in mammals, which plays essential roles in maintaining the body temperature of infants. Although the development of BAT during embryogenesis has been well addressed in rodents, how BAT grows after birth remains unknown. Using mouse interscapular BAT (iBAT) as an example, we studied the cellular and molecular mechanisms that regulate postnatal BAT growth. By analyzing the developmental dynamics of brown adipocytes (BAs), we found that BAs size enlargement partially accounts for iBAT growth. By investigating the BAs cell cycle activities, we confirmed the presence of proliferative BAs in the neonatal mice. Two weeks after birth, most of the BAs exit cell cycle, and the further expansion of the BAT was mainly due to lipogenesis-mediated BAs volume increase. Microscopy and fluorescence-activated cell sorting analyses suggest that most BAs are mononuclear and diploid. Based on the developmental dynamics of brown adipocytes, we propose that the murine iBAT has two different growth phases between birth and weaning: increase of BAs size and number in the first two weeks, and BAs size enlargement thereafter. In summary, our data demonstrate that both lipogenesis and proliferation of BAs contribute to postnatal iBAT growth in mice. Childhood obesity can cause severe medical and socio-emotional consequences3, and is associated with increased risk of adulthood obesity4. Nevertheless, the pathogenesis of childhood obesity is not fully understood, and therapeutic ways to reduce or prevent childhood obesity are limited.Childhood obesity is a major public health concern5. In mammals, there are two types of adipose tissues: white and brown 6. WAT stores triglycerides in adipocytes, and BAT dissipates chemical energy as heat7. Based on its developmental origin, BAT is further divided into classical BAT  and white adipocyte derived BAs 8. Interscapular BAT (iBAT) depot is one of the major cBAT depots in small mammals9. In humans, iBAT is consistently present in the children under 10 years old and dissipates in adults10. Recently, two independent studies have shown that the BAT activity of pediatric obese patients is significantly lower than that of the lean cohorts12. Furthermore, higher brown-like BAT composition at birth is associated with less body fat gain in early life13. These clinical observations have highlighted the importance of BAT in the regulation of pediatric obesity.Obesity is a result of chronic energy imbalance: energy uptake surpasses energy expenditure, and it is marked by excessive expansion of white adipose tissue14. From birth to pubertal maturation, the iBAT weight increases around nine times15. The growth of BAT is essential for keeping the whole body energy homeostasis during postnatal development, because genetically ablation of BAs causes obesity as early as 16 days after birth16. Although the BAT is crucial for maintaining body temperature and nutritional homeostasis, the underlying mechanisms controlling postnatal iBAT growth have not been clearly addressed17. There is an urgent need to understand how the BAT grows after birth, which will shed light on pediatric obesity related investigations.In mice, the iBAT depot appears at embryonic day 15.5 (E15.5) and expands rapidly through the rest of the developmental stages, likely by increasing the number of BAsIn this study, we investigated the growth of iBAT in postnatal mice by dissecting the developmental dynamics of the BAs. Our data indicate that the iBAT has two different growth phases between birth and weaning: the iBAT grows by increasing brown adipocytes (BAs) number and size in the first two weeks after birth (first growth phase), and expands through BAs hypertrophic growth thereafter (second growth phase). Furthermore, our data suggest that a small fraction of BAs are proliferative in the first growth phase. In summary, this study extends our understanding of postnatal iBAT growth, and suggests the presence of proliferating BAs in newborns.18. We checked the postpartum iBAT growth dynamics, which showed that the iBAT grew rapidly in the first 30 days after birth and its weight increased around 11 times from postnatal day 1 (P1) to P30 . Compared to P4, the BAs cross-sectional area did not change significantly at P8 (103.2 ± 8.64) and P12 (109.1 ± 15.39 µm2). From P12 to P15 and from P15 to P30 (135.1 ± 9.24 vs 200 ± 25.02 µm2), the BAs cross-sectional area increased significantly in both stages 3/2. Vn indicates the BA volume at different growth stages; V1 indicates the BA volume at P1. An represents the average BA cross-sectional area at different postnatal days; A1 represents the average BA cross-sectional area at P1. Compared to P1, the BA volume and iBAT weight had similar fold increases at P4 and P8  is a protein enriched in the BAs plasma membrane22, indicating that lipogenesis is highly associated with BAT growth. We then asked the question whether lipogenesis was the main driver of BAs hypertrophic growth. BODIPY 493/503 (BODI) is a well-defined lipophilic fluorescent probe for staining neutral lipids23. By double staining the iBAT sections with BODI and MCT1, we were able to assess the individual BA's lipid contents: BODI staining visualized the typical multilocular lipid droplets of BAs, and MCT1 immunofluorescence signals outlined the cell borders  is a transmembrane protein that transports fatty acids from the extracellular environment into the cell. Inside the cell, fatty acid binding proteins (FABPs), such as FABP327 and FABP4 28, bind and transport the free fatty acids to the mitochondria for oxidation or to the lipid droplets for storage. From P1 to P8, the expression of Lpl and Cd36 did not change much; from P8 to P12, both genes decreased significantly; from P15 to P30, Lpl increased substantially, and Cd36 showed out a mild increase  and fatty acid synthetase (Fasn)29. As crucial de novo fatty acids synthesis genes, both Acc1 and Fasn had low expression from P1 to P15 but increased robustly from P15 to P30  is a key enzyme that hydrolyses the triacylglycerol components of circulating lipoproteinsPatatin like Phospholipase Domain Containing 2 , Acyl-CoA Oxidase 1 (Acox1), and Carnitine palmitoyltransferase 1b (Cp1tb). Pnpla2 encoded protein catalyzes the initial step of triglyceride hydrolysis30. The protein products of Acox1 and Cpt1b are rate limiting enzymes of fatty acid (FA) β-oxidation, which catalyzes the first step of FA β-oxidation and facilitates the transfer of FA from cytoplasm to mitochondria, respectively32. During the postnatal iBAT growth, the expression of Pnpla2 did not change substantially; Acox1 and Cpt1 decreased with age , a nucleotide analog that labels cells passing through S phase. Six time points were included in this study: P1, P4, P8, P12, P15 and P30. Uncoupling protein 1 (UCP1) is a well-known marker of BAs. We used UCP1 antibody to label brown adipocytes. In the iBAT, EdU uptake was detected at P1, P4, P8, P12, but not at P15 and P30 , suggesting that the BAs might not undergo dedifferentiation at this age. These gene expression data suggest that the high expression of cell cycle genes in the neonatal mice is neither due to enrichment of adipocyte progenitor cells nor due to BAs dedifferentiation, and further indicate the presence of proliferative BAs.In the neonatal pups, besides originating from the preexisting BAs, the newly formed BAs could originate from two other cell populations: the adipocyte progenitor cells+) BAs were detectable  is a marker of M phase of the cell cycle. To further confirm the presence of proliferative BAs in neonatal mouse pups, we examined the iBAT for pH3 signals. In both P4 and P8 iBAT, pH3 positive  fluorescence protein following Cre-mediated recombination39. Ucp1::Cre transgenic mice express Cre recombinase in the BAs40. We crossed the Ucp1::Cre mouse line with the Rosa 26tdTomato mice to genetically label BAs with tdTom  to analyze the cell cycle activity of BAs at P8. Rosa 26Together, these data suggest that a subset of BAs are proliferative in the neonatal mice.Karyokinesis and cytokinesis are two related but different processes, and karyokinesis without cytokinesis may lead to polyploidy (more than two copies of DNA) or multinuclearity (more than one nucleus). Although we confirmed the presence of mitotic BAs in the neonatal mice, it was not clear whether these cells would complete the cell cycle and yield out new daughter cells. If the mitotic BAs in the neonates fail to complete karyokinesis or cytokinesis, we should be able to detect polyploidy or multinuclearity in the cell-cycle-quiescent BAs of young mice. In the iBAT of P15 or P30 mice, no more DNA synthesis activity was detected  naive growth phase, and 2) maturation phase. In the naive growth phase, increase of BAs number and size coincides; in the maturation phase, lipogenesis-driven BAs hypertrophic growth contributes to the iBAT mass expansion Fig. E. Our da35. The other category of data supports the view that differentiated BAs are proliferative. For example, overexpression of a cell cycle inhibitor p27 (encoded by Cdkn1b) in adipocytes largely reduces the iBAT depot size without affecting BAs size43. The controversy of these two tires observations can now be reconciled with our new hypothesis: the overexpression of p27 suppresses the proliferation of BAs in the naive growth stage, which results in the paucity of iBAT in the adult mice. In the adult mice, embryo-derived BAs are no longer proliferative and newly formed BAs are exclusively derived from activated adipocyte progenitor cells.Based on the published data, two distinct opinions exist regarding to the proliferation capability of terminal differentiated BAs. One category of data suggests that terminal differentiated BAs do not proliferate, and that the newly formed BAs in the adult iBAT mostly originate from PDGFRα + progenitor cells44. Our data show that BA lipogenesis happens throughout the two postnatal growth phases, which is evidenced by the increase of multilocular lipid droplets in each individual BAs and the increase of TG levels in the iBAT. Nevertheless, the underlying mechanism may be different. Compared with the genes at P30, the expression levels of lipid synthesis genes (such as Acc1 and Fasn) are constant and much lower in the naive growth phase  axis after a long gestation period, and therefore are able to rapidly turn on non-shivering thermogenesis after birth48; however, because the HPA axis is not mature in the new born mouse pups, it is unlikely that the proliferation of classical brown adipocytes is driven by beta-adrenergic signaling in the iBAT naive growth phase. Growth hormone (GH) and Insulin growth factor (IGF) signaling pathways are essential for postnatal development50, and disruption of IGF but not GH signaling in the adipose tissue results in severe iBAT paucity52, suggesting that IGF signaling may regulate the classical brown adipocytes proliferation during the iBAT naive growth phase. Future studies are secured to assess whether manipulating the IGF signaling affects the proliferation of classical brown adipocytes.One of the major findings of this study is that a fraction of brown adipocytes are proliferative in the iBAT naive growth phase. It will be interesting to decipher the molecular mechanisms that control classical brown adipocytes proliferation during this growth period, which will be insightful for developing new therapeutic strategies of generating more thermogenic brown adipocytes. Beta-adrenergic signaling is one of the pathways that regulates brown adipocyte proliferation40 and Rosa 26tdTomato mouse lines39 were reported previously, and were purchased from the Jackson Laboratory. Otherwise mentioned, C57BL/6 J background wild type mice were used throughout this study. All mice were kept at room temperature ( 22 ± 1 °C ) with a 12 h light/dark cycle. Pups were weaned at 28 days of age.All animal procedures were approved by the Institutional Animal Care and Use Committee of Masonic Medical Research Institute. All experiments were performed in accordance with relevant guidelines and regulations. C57BL/6 J background UCP1::Cre53. Imaging was performed on a Zeiss Confocal LSM 700.Interscapular brown adipose tissue (iBAT) was fixed in 4% PFA. iBAT was cryoprotected with 30% sucrose and embedded in OCT. 10 μm sections were used for immunofluorescence staining. Antibody sources were listed in Supplementary Table 1. EdU was administered subcutaneously at 5 µg/gram body weight 24 h before iBAT collection. EdU was detected with Click-iT chemistry (Invitrogen). After primary and secondary antibody staining, sections were treated with Sudan Black B (0.5% in 70% ethanol) for 5 min to reduce background autofluorescencehttps://theolb.readthedocs.io/en/latest/imaging/measuring-cell-fluorescence-using-imagej.html).The BAs's borders were visualized with MCT1 antibody, and the BAs' sizes were measured with Fiji software. For measuring BA's lipid contents, iBAT sections were double stained with MCT1 antibody and BODIPY 493/503. The cellular BODIPY fluorescence intensity (CBFI) was quantified based on an online protocol  buffer at a raito of 25 µl PBS/mg tissue, and homogenized with a glass dounce homogenizer. After centrifuge, 100 µl supernate solution was transfered into a new tube. To extract the lipid, 300 µl chloroform and 150 µl methanol was added into the supernate. The mixed solution was centrifuged to separate the aqueous phase from the organic phase (containing lipid), and the latter was transfered into a new ependorf tube for air dry. The TG concentration was determined with L-type triglyceride M Assay kit (Wako diagnostic), and normalized to the iBAT weight.Lipid from iBAT of different ages mice were extracted following a published protocolFixed and Hoechst stained BAs were filtered through 40-μm cell strainers. BAs not labeled with tdTom reporter and Hoechst were used as negative control to establish the flow cytometer settings. FACS was performed on FACSAria III (BD). The flow cytometric data were analyzed with DIVA software (BD).Total RNA was isolated using the Trizol reagent. For quantitative reverse transcription PCR (qRT-PCR), RNA was reverse transcribed (Superscript III) and specific transcripts were measured using SYBR Green chemistry and normalized to 36B4. Primer sequences were provided in Supplementary Table 2.Values were expressed as mean ± SEM. Student’s t-test or ANOVA with Tukey’s honestly significant difference post-hoc test was used to test for statistical significance involving two or more than two groups, respectively.Supplementary information.


There has perhaps been no issue as contentious in Covid-19 as face masks. The most contentious scientific debate has been between those who argue that “there is no scientific evidence”, by which they mean that there are no randomized controlled trials (RCTs), versus those who argue that when the evidence is considered together, “the science supports that face coverings save lives”. It used to be a ‘given’ that to decide whether a particular factor, either exogenous or endogenous, can cause a particular disease, and in what order of magnitude, one should consider all reasonably cogent evidence. This approach is being increasingly challenged, both scientifically and politically. The scientific challenge has come from methodologic views that focus on the randomized controlled trial (RCT) as the scientific gold standard, with priority being given, either to evidence from RCTs or to observational studies which closely mimic RCTs. The political challenge has come from various interests calling for the exclusion of epidemiological evidence from consideration by regulatory and advisory committees. There has perhaps been no issue as contentious in Covid-19 as face masks. Their benefits and drawbacks are endlessly debated politically, and scientifically. The most contentious scientific debate has been between those who argue that “there is no scientific evidence”, by which they actually mean that there are no randomized controlled trails (RCTs), vs. those who argue that, admittedly, there is no single definitive study, but when the evidence is considered together, “the science supports that face coverings save lives” , but thaIt used to be a ‘given’ that to decide whether a particular factor, either exogenous or endogenous, can cause a particular disease, and in what order of magnitude, one should consider all reasonably cogent evidence: several types of epidemiologic studies, clinical studies, laboratory studies, and sometimes socio-economic studies. This approach is being increasingly challenged. The scientific challenge has come from methodologic views that focus on the randomized controlled trial (RCT) as the scientific gold standard. However, the form of the argument has changed over time. In particular, we can distinguish between an earlier phase when the main distinction  Monographs provide a model for a systematic approach to combine evidence from human, animal and mechanistic studies to classify various exposures with regards to carcinogenicity.Evidence synthesis may include systematic reviews or formal meta-analyses of the findings of observational studies and/or RCTs, but it usually also incorporates other types of evidence, e.g. animal studies and laboratory science. Several key considerations for adjudicating causality were proposed by Bradford Hill in 1965 : StrengtIn such approaches, inferring causality involves the synthesis of a variety of evidence, and no single study is definitive , not evedifferent and in different directions, so that the likely strength of the hypothesized biases can be assessed.An integrative view of evidence synthesis that has come under renewed attention recently is named ‘triangulation’ . It explTriangulation strategies include cross-context comparisons, use of different control groups, natural experiments, within-sibling comparisons, natural experiments, instrumental variable (IV) analyses, Mendelian Randomization (MR), exposure-negative controls, and outcome-negative controls. Other non-RCT-based methods include population comparisons, difference in difference, and regression discontinuity studies . Of courThus, traditional approaches to evidence synthesis, including triangulation of evidence, are pluralistic and inclusive, with all of the relevant evidence being considered. For example, in the IARC Monographs on carcinogenicity, laboratory and animal studies are playing an increasingly important role, next to human data .Both phases in the debates about the RCT being the gold standard, have led to the development of scoring systems for evidence.The older view from the Evidence-Based Medicine movement prioritized RCTs for medical decision making, in an iconic ‘pyramid of evidence’ with the RCT on top, followed at some large distance by observational cohort studies and at even further distance for other types of observational studies. This thinking was crystallised in scoring systems like GRADE that in practice are completely geared towards RCTs; they will only ‘upgrade’ observational studies if there is evidence of ‘reverse confounding’, or if the effect is huge . This pyOne recent controversy  involvedIn recent years a ‘causal inference’ theory has increasingly promoted a distinction within observational studies, based on the idea an RCT is the gold standard for research involving exposures and outcomes in humans. Thus, when we cannot perform an RCT, the next best option is to perform an observational study which closely mimics the RCT model . This haNevertheless, the RCT-paradigm has become increasingly dominant in epidemiological theory in recent years . The zenTraditional approaches to evidence synthesis, much like any science, from astronomy to physics, inevitably involve subjective judgements at the forefront of scientific developments, e.g. when comparing the findings of studies which used different designs, which were conducted in different populations, or when comparing and synthesizing epidemiological, clinical, animal, and mechanistic evidence. It should be emphasized that equally subjective judgments can play a role in accepting or not accepting the results of RCTs—think of all debates about the validity of RCTs organized by the pharmaceutical industry which makes the medications that they put themselves on trial.Integrative considerations such as exemplified by those of Bradford Hill, as well as those used on a regular basis nowadays by IARC, provide a powerful guide of how to synthesize different types of evidence. Of course, decision-making processes may vary depending on the urgency of the issue (e.g. Covid-19), and the consequences to individuals and to the public health of an incorrect decision . NeverthTo placate legal and vested interests, it looks as if an easy solution is at hand, in that there is now a ready-made methodological view which explicitly argues that RCT-type evidence is the gold standard, and other observational evidence is of doubtful, if any, validity. Moreover, there are now readily available tools  which can be used to score individual studies on humans, clinical as well as epidemiological, on this basis. Such scoring systems have a veneer of objectivity, or at least provide a clear ‘paper trail’ so that regulatory authorities can more transparently report how decisions were made.The quote from the recent appointee to the EPA Clean Air Scientific Advisory Committee that we mentioned speaks for itself. It prefers a theory that restricts numerical evidence to intervention studies or analyses with the use of causal inference statistical methods .We want to emphasize that, despite the disturbing aspects of these developments, we are not arguing that there is any direct connection between modern causal inference theory and attempts to influence evidence synthesis by vested interests. However, both reinforce each other’s tendency to restrict the ‘acceptable evidence’ to studies which are RCTs, or observational studies which closely mimic RCTs. All other relevant evidence from individual studies  does not fit this paradigm, and such studies are either rejected or scored so low that they are ignored .Covid-19 has, within nine months, provided a ‘fast forward’ version of ‘normal science’ in which decisions need to be made quickly, lives are at stake, and one cannot wait for the perfect study. This does not mean that standard RCTs are not important—on the contrary, the trials of dexamethasone, chloroquine, and vaccines illustrate their crucial role, but it means that there are many important issues about curtailing the epidemic for which trials are neither possible, nor required. One example is the acceptance of the cumulative effect of barrier methods and screening to break the chain of transmission, whether they are mouth masks or rapid tests—based on insight and modelling, rather than RCTs or RCT-like observational studies [The way ahead is advocated by Savitz et al.  who arguAlgorithm-based methods such as GRADE and ROBINS-I that rely on the RCT paradigm are just a part of the toolkit of methods that can be used for evidence synthesis. When used carefully, they may assist the assessment of possible biases in studies of some particular exposure-outcome associations. For example, being able to group studies as to the likely occurrence, direction and strength of residual confounding, may provide a useful knowledge base for triangulation, and for the approach advocated by Savitz et al. . However