This study is not only relevant for precision medicine but also paves the way for introducing decision theory into OMICS data science.Correctly estimating the hormone receptor status for estrogen (ER) and progesterone (PGR) is crucial for precision therapy of breast cancer. It is known that conventional diagnostics  yields a significant rate of wrongly diagnosed receptor status. Here we demonstrate how Dempster Shafer decision Theory (DST) enhances diagnostic precision by adding information from gene expression. We downloaded data of 3753 breast cancer patients from Gene Expression Omnibus. Information from IHC and gene expression was fused according to DST, and the clinical criterion for receptor positivity was re-modelled along DST. Receptor status predicted according to DST was compared with conventional assessment via IHC and gene-expression, and deviations were flagged as questionable. The survival of questionable cases turned out significantly worse (Kaplan Meier  In particular, individually optimized treatment of breast cancer patients has widely been studied, drawing on molecular subtyping7, networks and pathways12 or more general gene expression signatures20. Among such signatures, PAM5022, PREDICT23, the Gene expression Grade Index24 and the Genomic Grade Index25 have been widely recognized. For a survey see Huang, C. C. et al.26.Precision medicine relies on biomarkers for selecting the most adequate therapy for each single patient28 and are estimated by immunohistochemistry (IHC) in clinical practice36. If at least one of both receptors (ER or PGR) is found positive (i.e. receptors are present), hormone therapy will suffice for an effective treatment, avoiding the much more severe side effects of chemotherapy. However, if the receptor status should accidentally be estimated falsely positive, hormone treatment will not work and the patient may be deprived from life-saving chemotherapy.Hormone receptor status  as well as the human epidermal growth factor 2 receptor (HER2) are considered most important38, suggesting up to 20% misclassifications41. Guidelines have been issued43 but improving assessment precision remains a key issue44.Due to its focal importance, receptor assessment quality has been scrutinized in multicenter studies48 as well as for gene expression analysis51. A survey was given by Bolstad, Irizarry, Astrand and Speed52. Chen modelled the probability distributions for gene-expressions of ER, PGR and HER2 and set cutoff values at a posterior probability of 0.5 to discriminate receptor positive from negative patients53. Likewise, Laas set cutoffs for receptor positivity within the frequency distributions of gene-expression of ER and HER241. Gong used random sampling to estimate optimum thresholds for ER and HER2 and verified these in a test cohort54. Bergqvist set thresholds for HER2 expression according to visual inspection55. Lopez introduced fuzzy rules to identify breast cancer biomarkers56. Concluding, prediction algorithms for low dimensional input space (such as hormone receptors), yield fairly similar results.A most natural way to increase reliability is to enhance IHC-estimates by additional information, e.g. from gene expression. Methodological advances were reported for data normalization58 we have worked on such approaches, applying standard statistical means (odds-products). In this work we expand the approach by drawing on Dempster Shafer decision Theory (DST)59.In previous papers62, driver’s vigilance monitoring61, aircraft technology64 and also in medical settings, e.g. image based decisions65, diagnosis of prostate66 and breast cancer67. In all these applications, unclear or even contradicting information from several sources are fused to arrive at decisions of optimized precision.DST has been widely applied in self driving carsp, characterizes the event as such, including its circumstances, but disregards the measuring process itself. The performance of the measuring process is characterized by sensitivity and specificity, in short by its receiver operating characteristics (ROC). In contrast, DST includes the measuring process and characterizes expected outcomes  by two numbers, the belief bel(Rez+) and the plausibility, pl(Rez+), together called ‘evidence’ of the outcome. Note that this definition of evidence specifically relates to the framework of decision theory and differs from more general meanings of the term, e.g. in ‘evidence based medicine’. The belief indicates the chance to correctly measure ‘positive’ by virtue and quality of the measuring process. The plausibility indicates the chance that the reading ‘positive’  could also result (a) by chance or (b) represent a false positive outcome (for a receptor status truly ‘negative’). For each measurement with dichotomic outcome θ, defined as Classical probability theory characterizes the chance for an event to occur by a single number, its probability. A probability, The main parts of this paper address readers familiar with DST and demonstrate how to combine several evidences for single hormone receptors and further combine several receptor estimates towards an overall hormone receptor status. Readers not yet familiar with DST may first see the ‘supplementary methods’ and then resume to read the following chapters.68, we decided to re-use published gene expression data69 and screened the Gene Expression Omnibus (GEO)71 for breast cancer studies using the Affymetrix chip U133A + 2.072. We found and curated 38 studies with 3753 samples.In order to establish a comprehensive database76 as compared by Bolstad52 and evaluated in our previous work77, we used RMA (MATLAB affyrma)78 and standardized data for each sample. Further batch corrections have been scrutinized in our previous work77 and were found ambiguous. We therefore refrained from performing them.Out of numerous methods available for normalization57. Whereas in multiple logistic regression all genes would be incorporated simultaneously in the predictor (multiple logistic regression), the ODDS method performs logistic regressions on single genes and then combines the odds. In other words, in this former work, probabilities from IHC, gene and co-gene were joined according to conventional statistics, i.e. by multiplying odds. Hence we refer to this method as ‘ODDS’ in the following. Like most binary classifiers79, ODDS builds a score out of input variables (i.e. gene expression). This score is either compared to a threshold and the corresponding class assigned . Or else, the sharp threshold is replaced by an interval, within which the score is considered ‘uncertain’. We chose the latter possibility in the setup of ODDS and excluded samples classified ‘uncertain’. This lets the remaining samples  gain certainty in their assessment, which is appropriate for a data basis on which our new discrimination method (DST) is evaluated.We adopted receptor genes and selected co-genes (see Table safeOdds).When performing ODDS like in our previous paper, results may contradict IHC—for a few samples. As opposed to this, in the current work we took IHC-estimates for granted and applied ODDS only to samples where no IHC estimate was available. Hence, values imputed by ODDS could never contradict IHC-estimates—just to be on the save side. Moreover, ‘uncertain’ results from ODDS were also discarded. All in all, to establish the database for the present work, we retained results either from IHC or safely imputed by ODDS and call this procedure ‘sODDS’  or safely imputed via ODDS (1010 samples). For imputation we used default parameters and the threshold 57. Finally, 1798 + 1010 = 2808 samples remained as HER2-, according to the sODDS method, see Fig. As first step of data cleansing we ruled out crosstalk by HER2: We dismissed HER257 with sufficient score (see above), samples were also included (513 samples). If IHC was missing for both, ER as well as PGR, samples were taken only if both status could be imputed safely via ODDS (332 samples). Thus, 1714 + 513 + 332 = 2559 samples were retained with receptor status established most accurately by state of the art (IHC plus ODDS ≙ sODDS), see Fig. These 2808 samples were then checked regarding ER and PGR as follows: If IHC-estimates were available for ER as well as PGR, samples were taken as is (1714 samples). If IHC was only available for ER and the status of PGR could be safely imputed via ODDSp, the overall average rate of differences, computed from Supp. Table p = 159 / 2227 ~ 0.071. Markers within bars represent expectation values /(NIHC + 2) of discordance rates derived from the beta-distribution, see the mathematical formalism in section ‘Concordance between IHC and gene expression’ in supplementary materials. Bars denote 95% confidence intervals of the respective mean values, allowing the qualification of studies against each other: If an upper bound lies below the dotted line in Fig. A study-wise evaluation of receptor status assessment via IHC and ODDS was performed, see Supp. Table Similarly, a control chart for progesterone is shown in Supp. Fig. 80) for estrogen only, since scmgene does not give PGR estimates. Note that scmgene was evaluated for all (3753) samples but results were only compared for those 2559 with HER2− and reliable hormone receptor status, see Supp. Table Additionally, we evaluated the scmgene-marker  → evidence.In the main part of this paper for brevity we skip the concept of masses (hence above shown in parenthesis) and present formulae for evidence directly based on responsibility functions. Masses are relegated to supplementary methods.one gene, labelled ‘Expr', to keep notation slim. It applies analogously to the other genes considered.Responsibility functions are exemplified now for the expression of +, for receptor positivity from logistic regression  of IHC receptor status versus gene expression, xExpr, see Eq. , estrogen co-gene (AGR3), progesterone receptor gene (PGR) and its co-gene (ESR1). Results are consolidated in Table , estrogeWe define DST masses (see supplementary methods) and evidences by drawing on responsibility functions as follows: We observe in Fig. θ+, of uncertain predictions from gene expression will result positive merely by chance, not by virtue of the method. Both parts together represent all true positive IHC-measurements: FP), although being truly negative samples. Based on this nomenclature we proceed as follows:Out of all positive IHC-measurements ), a certain fraction has been confirmed positive by gene expression, due to virtue of the method. This fraction of measurements is represented by the interval  is assessed via three sources of information, each yielding separate evidence (in our case of dichotomous data: 2 numbers):83  boil down to Eqs.  and 9 inThe above procedure is carried out similarly to obtain the combined evidence for the PGR-status  if either ER or PGR (or both) are positive. Classically, this is a crisp, logical decision as stated. DST however, allows more elaborate combination rules to combine evidences for estrogen  yields the overall evidence for ‘hormone receptor status’ As always we have to accept an amount of uncertainty given by In our simple case with only two outcomes these criteria reduce to 57, we considered receptors  separately. For each of them routine estimates . Note that out of 1432 patients considered positive by sODDS, only for 651 survival data were available. Among these, 26 were questioned by DST. Nonetheless the difference was statistically significant at the 1%-level.Figure It is interesting to see what DST achieves as compared to ODDS when starting from the same patient cohort. We therefore took the same sample of patients 2559) as in Table 559 as inMost strikingly, DST had flagged almost double as many patients ‘uncertain’  and also evidences for several different items, interacting in a biological setting (here exemplified by estrogen and progesterone regarding precision therapy of breast cancer).57, and here we demonstrate that further improvement is possible by application of DST. Since absolute truth of receptor status is unknown, the usefulness of our approach is demonstrated by disease free survival being severely degraded in patients flagged by DST as possibly wrongly diagnosed and treated. Explicit data on treatment are rare in the breast cancer studies downloaded from GEO. Hence we performed our calculations on the reasonable assumption that treatment was given according to the best knowledge available about receptor status.We have started from some best methodology available up to now (sODDS), i.e. receptor status assessment via IHC enhanced by gene expression. ODDS had already been shown to improve precision therapyThe potency of DST is further underpinned by directly comparing both methods applied to the same dataset: DST flags about twice as many patients as ‘uncertain’ (153) as ODDS (82). Additional patients subjected to a re-evaluation of hormone receptor status will improve precision of therapy.Formulae and results derived therefrom incorporate three assumptions which we think were reasonable:58.To compute basic belief assignments for each patient, we obtained DST mass functions by logistic regression. It proved more stable than other possible methods, e.g. fitting bimodal distributions such as Gaussian mixtures, as done in a previous studyFor joining evidences from 2 genes for the same receptor we chose the Dempster ECR. For adding IHC evidence to those from gene expression, we opted for the Yager ECR.To obtain upper limits ives Eq. . Likewisentclass1pt{minimatwo numbers (belief and plausibility) in each decision, instead of just a single number as conventional probability theory does. In addition, information from different sources is joined according to mathematical rules rather than ‘clinical intuition’.It is interesting to note that up to now clinical decisions follow the conventional (Cantor) non-exclusive OR-logics: ‘If ER or PGR receptor estimates is/are positive, the patient is considered positive’. In part, the strength of DST originates from elaborate evidence joining: Simple and/or combinations (such as estrogen-or-progesterone receptor positivity) are fundamentally refined, since DST draws on θH = 1−αH−βH follows αH + βH + θH = 1 and, accordingly, DST beliefs  of all samples lie in one single plane in 3-dimensions, cutting through the axes at αH = 1, βH = 1 and θH = 1, see panel A. Decision boundaries  and confine tetrahedrons, each containing exclusively positive (red) or negative (blue) samples, respectively. These tetrahedrons appear as equilateral triangles in the view shown in panel B. Uncertain samples (shown in ochre) appear within the pentahedron (panel A), which reduces to a kite-shaped area in the view shown in panel B.Joining evidences from both receptors in a more intricate way according to DST, allows for an intuitively convincing view on the classification, see Fig. While in this paper we have presented a very simple example to demonstrate feasibility and benefit of the method, possible applications are wide in scope. Pathology and laboratory medicine in many cases have to deliver crisp decisions even if measurements by complementary methods lack agreement. DST is the method of choice to handle such situations.The final evidence obtained by DST also allows assigning weights (loss functions) to certain outcomes in order to model clinical consequences Eq. . SupposeSeveral different methods are available for joining: The ‘Dempster additivity rule’, OMICS data often comprise a considerable number of estimates, say counts for multiple fragments out of a gene, needing to be condensed to characterize the gene (activity) as a whole. A plethora of similar situations can rigorously be handled by DST. Even if some of these estimates should be (partly) contradictive, DST is an appropriate tool for consolidating. Formulae need to be adapted to the particular situation in question.We think that this work may help pave the way for decision theory entering OMICS data science.Supplementary Information.


Sorghum bicolor), maize (Zea mays), and foxtail millet  with RNA-seq transcriptome profiling. These data are the cornerstone of a larger project that examined the conservation and divergence of gene expression networks within these crop plants. This data set focuses on temporal changes in gene expression to identify the network architecture responsible for daily regulation of plant growth and metabolic activities. The power of this data set is fine temporal resolution combined with continuous sampling over multiple days.The purpose of this data set is to capture the complete diurnal  transcriptome of fully expanded third leaves from the C4 panacoid grasses sorghum (The data set is 72 individual RNA-seq samples representing 24 time course samples each for sorghum, maize, and foxtail millet plants cultivated in a growth chamber under equal intervals of light and darkness. The 24 samples are separated by 3-h intervals so that the data set is a fine scale 72-h analysis of gene expression in the leaves of each plant type. FASTQ files from Illumina sequencing are available at the National Center for Biotechnology Information Sequence Read Archive. Sorghum bicolor), maize (Zea mays), and foxtail millet  with RNA-seq transcriptome profiling. These data are the cornerstone of a larger project that examined the conservation and divergence between sorghum, maize and foxtail millet of circadian clock and diurnal gene expression networks and their local DNA regulatory elements [The purpose of this data set is to identify transcripts with diurnal (i.e. daily) patterns of regulation within the C4 panacoid grasses sorghum  and 12 h darkness for 15 days  or 20 days (foxtail millet). Light periods were 26–28 °C and dark periods were 22 °C. Samples were taken at 3-h intervals for a total of 72-h. Sampling began at 3 h from when lights were turned on at the beginning of the first collection day. Quality of total RNA extracted from samples was confirmed by Bioanalyzer (Agilent). Strand-specific RNA-seq libraries were constructed from mRNA, isolated from total RNA by poly(A) selection, as described previously [The data set is 72 individual RNA-seq samples Table  represenThis is RNA-seq analysis of leaf blade tissue from young plants (15–20 days after planting). Care should be taken in directly extrapolating the transcriptome profiles to other tissues and/or older plants.The plants were grown in a growth chamber under cool white fluorescent lighting and under well-controlled temperatures. These conditions should be taken into account when making comparisons to plants grown in the field under natural sunlight and temperature.The goal of the RNA-seq design was to capture transcript levels from expressed genes, not to describe the full range of transcript splice variants.


Here, we present a search-based strategy for disease detection and classification, which detects diseased samples via their outlier novelty versus a database of samples from healthy subjects and then compares them to databases of samples from patients. This approach enables the identification of microbiome states associated with disease even in the presence of different cohorts, multiple sequencing platforms, or significant contamination. a priori organismal markers. These are lacking for many diseases. Here, we present an alternative, search-based strategy for disease detection and classification, which detects diseased samples via their outlier novelty versus a database of samples from healthy subjects and then compares these to databases of samples from patients. Our strategy’s precision, sensitivity, and speed outperform model-based approaches. In addition, it is more robust to platform heterogeneity and to contamination in 16S rRNA gene amplicon data sets. This search-based strategy shows promise as an important first step in microbiome big-data-based diagnosis.Microbiome-based disease classification depends on well-validated disease-specific models or IMPORTANCE Here, we present a search-based strategy for disease detection and classification, which detects diseased samples via their outlier novelty versus a database of samples from healthy subjects and then compares them to databases of samples from patients. This approach enables the identification of microbiome states associated with disease even in the presence of different cohorts, multiple sequencing platforms, or significant contamination. In addition, extending a model to other studies, even of the same disease, can be challenging, since selection of organismal biomarkers generally requires careful consideration of the effects of a plethora of factors, including host metadata  applications in many diseases . Currentge, etc. ) and seqHere, we present an alternative, search-based strategy for disease detection and classification. Using a database of samples from healthy subjects as a reference distribution we employ an outlier detection strategy to identify disease status. Then, disease classification is achieved by subsequent comparison to databases of samples from patients. Using standard performance measures, the precision, sensitivity, and speed of our search-based method outperform the widely used model-based approaches of random forest (RF), support vector machine (SVM), and eXtreme Gradient Boosting (XGBoost). In addition, we demonstrate that our approach is more robust to platform heterogeneity and contamination in 16S rRNA gene amplicon data sets.We recently developed Microbiome Search Engine (MSE), which rapidly and precisely identifies matches to a query sample from hundreds of thousands of known microbiomes based on their phylogeny-based compositional similarity . We hypoStep I: The disease status of a sample is determined via an outlier microbiome novelty score (MNS) against the baseline database. Taxonomic profiles are available for tens of thousands of human-associated microbiomes on the operating taxonomy unit (OTU) level that have been generated by a large number of studies via sequencing 16S rRNA gene amplicons. Because the diversity of human microbiomes from healthy subjects in 16S studies is nearly saturated as defined by the MNS . The novel samples are identified based on the threshold of mean MNS in 2010. The novel sample rate was developed in a descending trend and kept in a quite low level that was less than 5%. Source data are provided as Copyright © 2020 Su et al.2020Su et al.Creative Commons Attribution 4.0 International license.This content is distributed under the terms of the 10.1128/mSystems.00150-20.4TABLE S1Table S1, DOCX file, 0.1 MB.The fecal samples of the baseline database for search-based diagnosis. Download Copyright © 2020 Su et al.2020Su et al.Creative Commons Attribution 4.0 International license.This content is distributed under the terms of the 10.1128/mSystems.00150-20.5TABLE S2Table S2, DOCX file, 0.04 MB.Data Set Gut for evaluation of search-based diagnosis. Download Copyright © 2020 Su et al.2020Su et al.Creative Commons Attribution 4.0 International license.This content is distributed under the terms of the P value < 0.01, corrected by removing longitudinal replicates; Each of the 3,113 samples in Data Set Gut was searched against the baseline database to calculate its MNS, quantifying the degree of structural dissimilarity between a query microbiome and those in the healthy baseline database   to compare the overall performance of MSE with these three model-based approaches on multicohort classification which measures the rates of both correctly classified queries and misclassified ones. (The AUC performance measure is for binary discrimination and is not well suited to more than two categories.) Among the five cohorts  of Data Set Gut, MSE achieved a k of 0.85 ± 0.03, indicating a striking agreement with the experimental design  also supports excellent performance of MSE in the simultaneous presence of the two sequencing platforms . These represent the most likely source of contamination for human microbiome samples in the context of disease diagnosis. The rate of contamination was set to 5%, 10%, 15%, and 20%. Even at a high contamination rate of 20%, MSE still offers a reasonable performance, in both disease status detection . In step I, the MNS of patient samples were significantly higher than those of controls , based on which they were detected with AUC = 0.71 . This compared favorably to an average k of 0.52  for model-based approaches using Data Set Gut as training , Sweden (n = 695), the United Kingdom (n = 185), and Australia (n = 26), with the remaining being from 61 other countries/regions (n = 375). We split the baseline database into two subsets, those from the same regions as Data Set Gut  and those from regions different from Data Set Gut . Each of the two subsets were then separately used as a baseline database for detecting the unhealthy microbiome in Data Set Gut. To avoid the bias of uneven sample amount, the subset from the original regions was rarefied to n = 2,000 samples (with 10 replicates) to avoid the random bias. The baseline from the same regions as the test set yielded higher precision (AUC = 0.68 ± 0.004) than that from different regions (AUC = 0.60 ± 0.004) in detecting diseased samples -based search  using PICRUSt  or Euclidean distance  to that of cosine distance and Euclidean distance on the performance of step I and step II. In step I, MSE achieved an AUC of 0.81 for disease detection, which is higher than results based on cosine distance (AUC = 0.74) or Euclidean distance  was evaluated for MNS calculation -like classification strategy. The resulting predictions provide direction and hypotheses for clinical decision-making. The two-stage approach provides considerable computing performance advantages for continuous updating of the microbiome databases to incorporate new studies compared to alternate model-based approaches. Moreover, MSE does not require curation of disease-specific models or biomarkers. MSE also provides a new approach to disease prediagnosis, i.e., deciding whether the microbiome is indicative of a disease status.In light of the general shift of microbiome-sequencing focus from healthy to diseased hosts –15, the –On the other hand, despite its performance advantages when tested on human gut microbiomes for several disease cohorts, this type of search-based classification strategy will not offer better performance than other machine learning methods under all circumstances (in accordance with the “no free lunch” theorem in machine learning). For example, for classification of human body sites from microbiome data, the accuracy of MSE is equivalent to or lower than several machine learning approaches we tested . TherefoData Set Gut  consistsAll microbiome samples were processed from their published sequence data using Parallel-META 3 version 3.4.3  with GreMSE aims to rapidly identify the structurally “similar” microbiomes of a given query microbiome from a large-scale depository of known microbiomes . The seasimilarityT as the time taken to calculate the phylogeny-based similarity between two microbiomes in a database with N samples, the time consumed for a single exhaustive search isDefining indexingT, the time taken for a single dynamic index-based search by MSE isIn comparison, defining the dynamic index-fetching time as indexingT « 200 × similarityT, even for searches in a database with one million samples. Therefore, the search time can be theoretically approximated asActually, We found that for a single search in the database with 10,000 samples, the exhaustive search took 1.15 s, and MSE used 0.14 s . When thq and its top N matches, for its match i, whose microbiome similarity is iS (see q) was calculated from the top N ordered matches (we used N = 10)The microbiome novelty score (MNS) is proposed to evaluate the compositional uniqueness of a microbiome sample compared to a reference microbiome database . With a s Si see , the MNSFor each microbiome sample, its MNS was derived by searching it against all samples in the reference database. Thus, a higher MNS means lower similarity to those microbiomes in the database, suggesting higher novelty and uniqueness.r = 0.915 [We used a microbiome similarity measure that has a strong correlation with the UniFrac metric (Spearman  = 0.915 ) and is j. Thus, for node i, with descendants j and k, the node weights arewhere i is a terminating edge, then Ψ is the unordered set of abundance of OTU i in the two samples. The similarity, S, of the two samples is thenIf node n is the number of nodes on the tree.where N matches (we used N = 10). For a status metadata, G = {g1, g2, …., gm} for healthy status , and with a given query sample q and its top N matches, the microbiome similarity to match i is iS, and then the score of query sample q for status gk isThe disease status of a new microbiome is predicted based on the metadata of its top Thus, the final classification is determined by the status with the maximum score across all cohorts.We constructed the machine learning model for multiple disease diagnosis using three metrics of random forest (RF), support vector machine (SVM), and eXtreme Gradient Boosting (XGBoost). All models were trained and verified using the taxonomic relative abundances on OTU level after 16S rRNA gene normalization. The RF model was trained by the randomForest package in R; the SVM model was trained by the svm function in R; the XGBoost model was trained by the xgboost package in R. The machine learning training processes were performed with optimized parameters for multicategory classification for microbiome data using the strategy introduced by Statnikov et al.  Table 2Table 2.area under the receiver operating characteristic curve (AUC) is usually used to measure the performance of a model for discrimination between binary states . Because AUC is not available with a status of ≥3, we used the Kappa coefficient (k) to evaluate the performance of MSE in multistate classification. The Kappa coefficient quantifies the consistency of the sample distribution in multiple cohorts between the classification results of MSE and the known distribution. k is always between 0 and 1, and the following intervals provide guidance for interpretation: 0.00 to 0.20, slight consistency; 0.21 to 0.40, fair consistency; 0.41 to 0.60, moderate consistency; 0.61 to 0.80, substantial consistency; and 0.81 to 1.00, almost perfect consistency. The recall  and precision  were also calculated to assess the performance of diagnosing each disease.The k to quantify the impact of changed sample conditions; specifically, Δk quantifies the impact of the condition in classification performance.We also used reductions in To avoid the statistical bias due to longitudinal sampling from the same host, we removed the replicated microbiomes in statistical analysis of both step I and step II. In step I, among replicates sampled from the same person in each cohort, only one sample was randomly selected and kept in calculating the results of the Wilcoxon rank-sum test for MNS. Likewise, during the cross-validation of step II, replicates from the same person in each cohort were also removed by random selection as in step I before calculating the Kappa coefficient/recall/precision for MSE and the machine learning-based approaches.http://americangut.org/). The indoor environmental contamination OTUs were from the indoor environment microbiomes of the Qiita public database  and a given contamination rate r%, a contamination source microbiome sample was randomly selected, and m × r% OTUs were randomly extracted from this sample and mixed into the OTU table of the target sample. The baseline databases of step I and the disease databases (training samples) of step II were kept as they were , while contaminations to Data Set Gut were simulated as described above to test the robustness of the MSE and the three model-based methods to contaminations.The reagent blank contamination OTUs were from the reagent blank microbiomes of the American Gut Project  as training data for model-based approaches  and another 100 samples as testing data . For model-based machine learning methods, the total running time includes both training time and testing time. For MSE, the total running time is for a complete two-step diagnosis with database construction and search . All teshttp://mse.ac.cn. The source code is also posted to the GitHub repository at http://github.com/qibebt-bioinfo/meta-storms. We have also developed a QIIME 2 plugin, which can be found at http://github.com/qibebt-bioinfo/q2-metastorms. In addition, all tests in this work were implemented as Linux shell scripts under Linux/Mac OS X to facilitate reproduction of the results. Scripts and test data sets are available at the download page of MSE (http://mse.ac.cn).MSE is developed and implemented in C/C++. It comes with a full automatic installer for cross-platform installation and setup in Linux/Mac OS X. The indexing and searching algorithm is optimized for parallel computing based on multiple CPUs using the OpenMP library. MSE accepts OTU tables for both database construction and search and thus is compatible with QIIME/QIIME 2, Parallel-META 3, and many other microbiome profiling tools. Both the source code and executive binary application packages are available at http://qiita.ucsd.edu) using the study IDs listed in The source data underlying


Digital tissue deconvolution (DTD) estimates the cellular composition of a tissue from its bulk gene-expression profile. For this, DTD approximates the bulk as a mixture of cell-specific expression profiles. Different tissues have different cellular compositions, with cells in different activation states, and embedded in different environments. Consequently, DTD can profit from tailoring the deconvolution model to a specific tissue context.Loss-function learning adapts DTD to a specific tissue context, such as the deconvolution of blood, or a specific type of tumor tissue. We provide software for loss-function learning, for its validation and visualization, and for applying the DTD models to new data. The cellular composition of a tumor specimen is a prognostic factor  if we want to estimate contributions from small cell populations, and (3) if we want to disentangle highly similar cell types. Genes weights can be introduced by replacing the residual sum of squares byAn important observation is that genes can be weighted to obtain better estimates  with known cellular compositions C . The rational behind loss-function learning is to obtain the vector g by minimizing a loss function L on the training data.Loss-function learning uses training data to optimize DTD models. These training data are bulk gene-expression measurements The package DTD incorporates a correlation-based loss function:Here, The R package DTD has the basic function callX is a matrix with reference profiles in its columns, and train.data is a list containing “mixtures” Y and “quantities” C, defined as earlier. train_deconvolution_model returns a deconvolution model that can be applied through the function estimate_c on new data y, which is a vector or matrix with bulk gene-expression levels. The function estimate_c returns estimated cellular proportions for the bulk profiles y. The workflow is summarized in Here, Y and their corresponding compositions C. An example for such a function call is mix_samples. This function uses, for example, raw counts from scRNA-Seq (sc.counts), where columns correspond to cells and rows to genes. Furthermore, the vector sc.pheno ascribes cell types to the respective columns in sc.counts. The function returns a list with the components Y and C, which correspond to artificial mixtures and their underlying cellular compositions, respectively. A similar function is provided to generate a reference matrix X.Loss-function learning requires training data. These data can be generated experimentally, for example, through fluorescence-activated cell sorting (FACS)-based cell counting combined with bulk RNA sequencing. Alternatively, these data can be generated artificially from scRNA-Seq experiments. For this purpose, we implemented functions that automatically generate training mixtures We provide software for the digital deconvolution of bulk gene-expression data. We take into account that DTD needs to be adapted to a specific type of tissue. It is not likely that there is a universal deconvolution formula, which performs consistently well in all settings. The most reliable results might be obtained by optimized models that have been trained within a machine learning framework.In this study, we provide software for such a framework, called loss-function learning, and for the application of the learned deconvolution models to bulk data. It is computationally tractable and easy to use.A DTD tutorial is available as In summary, we provide the R package DTD that contains software to systematically improve the performance of DTD algorithms.


Growing evidence suggests that the efficacy of immunotherapy in non-small cell lung cancers (NSCLCs) is associated with the immune microenvironment within the tumor. We aimed to explore radiologic phenotyping using a radiomics approach to assess the immune microenvironment in NSCLC. Two independent NSCLC cohorts (training and test sets) were included. Single-sample gene set enrichment analysis was used to determine the tumor microenvironment, where type 1 helper T (Th1) cells, type 2 helper T (Th2) cells, and cytotoxic T cells were the targets for prediction with computed tomographic (CT) radiomic features. Multiple algorithms were in the modeling followed by final model selection. The training dataset comprised 89 NSCLCs and the test set included 60 cases of lung squamous cell carcinoma and adenocarcinoma. A total of 239 CT radiomic features were used. A linear discriminant analysis model was selected for the final model of Th2 cell group prediction. The area under the curve value of the final model on the test set was 0.684. Predictors of the linear discriminant analysis model were skewness , kurtosis, variance (subsampled from delta [subtraction inner pixels from outer pixels]), and informational measure of correlation. The performances of radiomics on test set of Th1 and cytotoxic T cell were not accurate enough to be predictable. A radiomics approach can be used to interrogate an entire tumor in a noninvasive manner and provide added diagnostic value to identify the immune microenvironment of NSCLC, in particular, Th2 cell signatures. PD-L1) gene in the tumor has been associated with a response to checkpoint blockade therapy and prognosis, to some extent [Immune checkpoint blockade therapy, an anti-cancer treatment that potentiates the ability of the immune system to recognize and destroy cancer cells, has become a standard in the care of patients with non-small cell lung cancers (NSCLCs) –5. The ee extent , 7, but e extent –11. Genee extent , wherease extent . Recent e extent .Immunohistochemistry after tumor resection is the only technique that yields quantitative information on the immune microenvironment and immune response type assessment. Therefore, efficient and reliable noninvasive biomarkers are urgently needed. Radiomics has recently been identified as a way to use accurate quantitative imaging descriptors in line with advances in image processing techniques , 13. HigAccordingly, we have conducted a study to demonstrate the feasibility of radiomic prediction of the immune microenvironment by using computed tomographic (CT) imaging in NSCLC. Our ultimate goal was to identify useful predictive radiomic characteristics of the immune microenvironment and to further develop treatment strategies.Two independent datasets were used for the training and test sets. The training dataset labeled “Lung3”, contains data from patients treated at the MAASTRO Clinic, the Netherlands . The teshttp://cancerimagingarchive.net/) [CT images were gathered from The Cancer Imaging Archive  . All imave.net/)  and in-hve.net/) , 20. DetThe training set gene expression data were generated using the Rosetta/Merck Human RSTA Custom Affymetrix 2.0 microarray (GPL15048) and stored in the Gene Expression Omnibus (GEO accession: GSE58661). The test set gene expression data were generated using RNAseq and downloaded from the Broad GDAC Firehose. The immune cell signature was estimated from gene expression data using single-sample gene set enrichment analysis (ssGSEA) to determine the immune microenvironment of lung cancer. The GSVA R package was used for the ssGSEA . A set oThe Th1, Th2, and cytotoxic T cell groups were predicted using machine learning algorithms, which included penalized logistic regression, penalized discriminant analysis, sparse discriminant analysis, linear discriminant analysis, naive Bayes, classification and regression tree (CART), bagged CART, and random forest. R (version 3.6.0) and the caret R package were used for the prediction modeling . Data prDemographic information and tumor characteristics for both the training and test sets are presented in Classification trees and rule-based models and recursive feature elimination methods performed better than penalized linear classification models. Random forest, linear discriminant analysis, and penalized logistic regression models performed well in the training set of Th2 cell prediction  . Random We selected a linear discriminant analysis model for Th2 cell prediction and penalized discriminant analysis model for Th1 cell prediction because both models are simpler than bagged CART or random forest. Although the tree based ensemble methods including random forest and bagged CART showed best performance, but the performance was not significantly different between the ensemble methods and linear models including linear discriminant analysis and penalized discriminant analysis models. The prediction performance on the test set of Th2 cells was good (AUC = 0.684)  but not Several previous reports suggest that malignant phenotypes and efficacy of immunotherapy in cancers are defined not only by the intrinsic activities of cancer cells but also by components in the tumor microenvironment, especially tumor-infiltrating immune cells –30. DespIn this study, based on the deconvolution of gene expression data from 89 patients in the “Lung3” cohort using ssGSEA and immune microenvironment profiling with the use of 28 immune cell signatures from gene expression, we found associations with certain radiomic features for subsets of Th1, Th2, and cytotoxic T cells in NSCLCs. Through a prediction performance test using a TCGA test set, we also demonstrated that the radiomic prediction for Th2 cell signatures of NSCLCs was feasible (AUC = 0.684), even though the performances of radiomics on the test set of Th1 and cytotoxic T cells were not accurate enough to be predictable. Our results indicate that skewness , kurtosis, variance (subsampled from delta), and IMC correlated with Th2 cells. Our study is in line with but distinguishable from previous reports –33. The Most past studies dealing with antitumor immunity have focused on CD8+ cytotoxic T cells, as their cytotoxic activity was deemed to be critical for tumor eradication. Recent work indicates that specific subsets of CD4+ T cells , B cells, macrophages as well as dendritic cells provide an important contribution to antitumor immune responses , 29. TheWe found that skewness , kurtosis, variance (subsampled from delta), and IMC were correlated with Th2 cells. Among these features, outer pixel skewness refers to skewness of the peripheral portion of the tumor. More interestingly, given that this radiomic feature reflects the peritumoral radiologic phenotype, this finding may indicate that the immune microenvironment of NSCLCs is closely related with certain interactions between the tumors and surrounding normal lung tissues. In other words, these edges or peritumoral radiologic findings offer information that can help predict tumor immune activity in the development of immunotherapy strategies. There has been no previous research regarding the relationship between the five radiomic features, including skewness of outer pixels and subsets of immune cells, therefore further verification of this finding in larger cohorts is needed.Despite the advantages of utilizing a different cohort for external validation, this analysis has several limitations. First, the training data are retrospective and limited to 89 patients from a single institution. This deficiency could be addressed in future work by using a larger patient cohort. Second, CT scans we obtained for radiomic feature selection were partly contrast-enhanced and partly non-contrast and there are differences in image resolution and even may be in acquisition protocol between training and test data (Scan protocol including tube voltage and tube current was not provided). Contrast agent may obscure the radiomic textural features and variable acquisition protocol might lead to be unstable in features because textural- and intensity-based features may be affected by intensity and scanner variability. However, it is rather difficult to have the same acquisition protocol between multi-center data in practice. Testing using independent data with some differences in acquisition reflects the real clinical practice and the performance measured under such circumstance is a good measure of generalization of the proposed method. The data are obtained from the public database and many studies using the data still used the data as a whole despite the heterogeneity , 34, 35.In conclusion, radiomic prediction for the immune microenvironment of NSCLC was feasible and found a relationship between radiomic features and Th2 cell signatures. This study demonstrated the potential of radiomic features as noninvasive biomarkers to capture the tumor microenvironment properties and to predict the tumor-suppressive or tumor-permissive status of NSCLC. The results of this study may help define categories of tumor immune activity for patients with NSCLC and develop immunotherapy strategies.S1 File(DOCX)Click here for additional data file.S1 Table(DOCX)Click here for additional data file.S2 Table(DOCX)Click here for additional data file.S3 Table(DOCX)Click here for additional data file.


Arabidopsis thaliana. MOG executable and code: http://metnetweb.gdcb.iastate.edu/ and https://github.com/urmi-21/MetaOmGraph/.The diverse and growing omics data in public domains provide researchers with tremendous opportunity to extract hidden, yet undiscovered, knowledge. However, the vast majority of archived data remain unused. Here, we present MetaOmGraph (MOG), a free, open-source, standalone software for exploratory analysis of massive datasets. Researchers, without coding, can interactively visualize and evaluate data in the context of its metadata, honing-in on groups of samples or genes based on attributes such as expression values, statistical associations, metadata terms and ontology annotations. Interaction with data is easy via interactive visualizations such as line charts, box plots, scatter plots, histograms and volcano plots. Statistical analyses include co-expression analysis, differential expression analysis and differential correlation analysis, with significance tests. Researchers can send data subsets to R for additional analyses. Multithreading and indexing enable efficient big data analysis. A researcher can create new MOG projects from any numerical data; or explore an existing MOG project. MOG projects, with history of explorations, can be saved and shared. We illustrate MOG by case studies of large curated datasets from human cancer RNA-Seq, where we identify novel putative biomarker genes in different tumors, and microarray and metabolomics data from  Petabytes of raw and processed data generated with microarray, RNA-Seq (bulk and single-cell) and mass spectrometry for small molecules and proteins are available through public data repositories . These dA common goal of analyzing omics data is to infer functional roles of particular features  by investigating co-expression and differential expression patterns. A wide variety of R-based  tools caIncreasing the usability of the vast data resources by enabling efficient exploratory analysis would provide a tremendous opportunity to probe the expression of transcripts, genes, proteins, metabolites and other features across a variety of different conditions. Such exploration can generate novel hypotheses for experimentation, and hence improving the fundamental understanding of the function of genes, proteins and their roles in complex biological networks ,21–25.At present, there are very limited options for researchers to interact with expression datasets using the fundamental principles of exploratory data analysis . ExploraIn this paper, we present MetaOmGraph (MOG), a Java software, to interactively explore and visualize large expression datasets. MOG overcomes the challenges posed by the size and complexity of big datasets by efficient handling of the data files. Further, by incorporating metadata, MOG adds extra dimensions to the analyses and provides flexibility in data exploration. At any stage of the analysis, a researcher can save her/his progress. Saved MOG projects can be shared, reused and included in publications. MOG is user-centered software, designed for exploring diverse types of numerical data and their metadata, but specialized for expression data.MOG is an interactive software that can run on any operating system capable of running Java . MOG’s Graphical User Interface (GUI) is the central component through which all the functionality is accessed Figure . Access interactive visualizations. The researcher can visualize data via line charts, histograms, box plots, volcano plots, scatter plots and bar charts, each of which is programmed to allow real-time interaction with the data and the metadata. Users can group, sort, filter, change colors and shapes, zoom and pan interactively, via the GUI. At any point in the exploration, the researcher can look-up external databases: GeneCards (http://metnetweb.gdcb.iastate.edu/MetNet_atGeneSearch.htm) for additional information about the genomic features in the dataset. Researchers can also easily access SRA and GEO databases using the accessions present in the study metadata.MOG displays all the data in interactive tables and trees, providing a flexible and structured view of the data. The user can interactively filter or select data for analysis. This ability is particularly important for aggregated datasets, as users may wish to split data into groups of studies, treatments or organs. A novel aspect of MOG is its capability of producing eneCards , EnsembleneCards , EnsembleneCards , RefSeq eneCards , TAIR  that can occur when handling diverse data types. Bug reports can be submitted with a single click, if encountered.Although specifically created for the analysis of omics data, which is the focus of this paper, MOG is designed to be flexible enough to generally handle numerical data. A user can supplement a MOG project with any type of metadata about the features, and about the studies. Thus, a MOG user can interactively analyze and visualize voluminous data on any topic. For example, a user could create a project on: transmission of mosquito-borne infectious diseases world-wide; public tax return data for world leaders over the past 40 years; daily sales at Dimo’s Pizza over 5 years; player statistics across all Women’s National Basketball Association (WNBA) teams; climate history and projections since 1900.https://github.com/urmi-21/MetaOmGraph/blob/master/CONTRIBUTING.md.In addition to the functionality we have programmed into MOG, MOG borrows some functionality from freely available and extensively tested third-party Java libraries . We have combined these to create a highly modular system that is amendable to changes and extensions and developers can easily implement new statistical analyses and visualizations in the future. MOG is an open source project and we plan to expand and develop it further through community driven efforts. Information on how to contribute to MOG, and who to contact with further questions, is provided at https://github.com/urmi-21/MetaOmGraph/tree/master/manual).Based on the utility and popularity of R for data analysis, we have implemented a GUI to facilitate execution of R scripts through MOG. MOG’s GUI enables a user to interactively select or filter data using MOG; these data are then passed to R. This avoids the need to constantly write new R code to specify different genes and samples for analyses. For example, a user can write an R script for hierarchical clustering of genes based on the expression levels, interactively select or filter data using MOG, and execute the R script. More details on how to use MOG for executing R scripts are provided in the user manual available from (A user can quickly create a new MOG project using two delimited files: (i) a file with unique identifiers (IDs) for each feature (e.g. gene), metadata about that feature and numerical data quantifying each feature across multiple conditions (e.g. multiple samples and studies), and (ii) a file containing unique identifiers for each sample and metadata about the samples and studies in the datafile. These are virtually combined by MOG, using the unique identifier in each file . SelectiArabidopsis thaliana datasets described herein, can be re-opened, analyzed, modified or shared. Ongoing exploration results, such as correlations, lists and other interactive analyses, can be saved in any MOG project, regardless of whether it was obtained from our website or created from custom data.New MOG projects, as well as those from well-vetted datasets, including the human and Measures of statistical association between a pair of features in a dataset quantify the similarity in their expression patterns across the samples that comprise that dataset . Genes wMOG provides the researcher with several statistical measures to estimate associations/co-expression among the features. It can also compute association between samples, which reflects similarity between the samples. Choosing appropriate statistical measures and interpretations for each dataset is left to the user.We have incorporated four key methods that measure association among pairs of features. Each has its own advantages and disadvantages, depending on the types of relationships the researcher wishes to detect, and the characteristics of the dataset being explored.X and Y, whereas, the Spearman correlation coefficient measures monotonic relationships between the two variables. Both excel at detecting linear relationships, however, Spearman is less sensitive to outliers  be a pair of discrete random variables over the space X and Y is defined as:p is the joint probability mass function of X and Y, and p(x) and p(y) are the marginal probability mass functions for X and Y, respectively. Compared to correlation measures, MI is a more general approach that can detect complex, non-linear associations. The interpretation of the MI value is different than that of correlation values: an MI value of zero, I = 0, implies statistical independence of X and Y, whereas a correlation value of zero need not imply statistical independence  between selected features across samples. MI quantifies the amount of information shared between two random variables. Let  . CLR detA. thaliana), but may not control ancillary sources of variation (e.g. coverage variation in RNA-Seq data). MOG provides a choice between a fixed effects model (FEM) or a random effects model (REM)  ,46 for tel (REM) ,46. The t-test ; Welch’s t-test ; and a permutation test . For analysis of groups with paired samples, we have implemented: a Paired t-test ; a Wilcoxon signed-rank test ; and a permutation test for paired samples .Determining differentially expressed features from aggregated datasets provides direction for further data exploration. In MOG, we have incorporated several popular statistical methods to evaluate differential expression between two groups of samples. For analysis of groups with independent samples, we have implemented: Mann–Whitney U test (a non-parametric test that makes no assumptions about data distribution); Student’s MOG’s methods to identify differentially expressed genes are general statistical methods which are designed for large sample sizes (30 or more samples for gene expression data). Computation of these methods via MOG permits interactivity, which promotes data exploration. A limitation of the interactive differential expression analysis methods implemented in MOG is that they are designed for large sample sizes and use normalized data as input. For smaller sample sizes, a user can apply specialized model-based methods, accessible through R, to infer differentially expressed genes in RNA-Seq or microarray datasets. For example, methods like edgeR , DESeq2 shifting biological interactions among these genes or their regulators  and parametric options  to estimate P-values are calculated by a permutation test that estimates the null distribution of a test statistic by randomly permuting the labels of the observed data points (assuming that the labels are exchangeable under the null hypothesis)  . Becauseparallel .P-values for multiple comparisons: the Bonferroni method  and Genotype Tissue Expression   .To create the MOG project, we excluded from the dataset any organ types in which the number of tumor or non-tumor samples was <30. To ensure statistical independence among the samples, we removed all non-tumor samples from TCGA and included only one TCGA tumor-sample per patient . We alsohttps://gtexportal.org/home)). We were unable to locate metadata for 17 of the TCGA samples and excluded these samples from the dataset (https://www.genenames.org/), NCBI Gene (https://www.ncbi.nlm.nih.gov/gene), Ensembl  from the Hu-cancer-RNASeq-dataset and its metadata. The MOG project contains expression values for 18 212 genes, 30 fields of metadata detailing each gene, across 7142 samples representing 14 different cancer types and associated non-tumor tissues  from Mentzen and Wurtele . We downloaded these data from the Plant/Eukaryotic and Microbial Resource (PMR)  data that we used to create a MOG project were from 656 GC-MS samples describing the effect of 50 knock out (or knock down) mutations of genes of mostly unknown functions on the accumulation of metabolites in thaliana   . We creain silico results. In other cases, the exploration led us to completely novel predictions that could be tested experimentally.We illustrate MOG’s usability and flexibility by exploring three diverse datasets from different perspectives. The statistical analyses and visualizations shown were generated exclusively using MOG. Often, our exploration led us to conclusions consistent with prior experimental or Determining that a dataset is valid, properly normalized and free of batch effects is a critical preliminary step in the analysis. To verify that samples from similar biological conditions exhibit similar expression patterns for all the genes, we used MOG to compute pairwise Pearson correlations among samples from the same biological condition (tumor/non-tumor and organ type). All the samples had high Pearson correlations (>0.70) with others taken from the same organ and tumor status, except one sample from lung adenocarcinoma (LUAD), which we removed from the dataset .We visualized the distribution of Pearson correlation values for non-tumor samples. For homogeneous samples, such distributions should appear unimodal. However, several organs show multimodal distributions . This fikey genes that are regulated by, or implicated in, the molecular and cellular processes driving cancer, and to further explore the processes in which these genes are involved. For this task, we used MOG first to identify the differentially expressed genes in samples from each tumor type versus corresponding non-tumor samples, and then to examine the expression patterns of these genes. We define a gene as differentially expressed between two groups if it meets each of the following criteria:logFC| ≥ 1 where logFC is calculated as in limma 2 transformed data, is significant between the two groups Mann–Whitney U test, on the logWe wanted to identify Hu-cancer-RNASeq-dataset, between 2000–5000 of the 18 212 genes are differentially expressed (Table In each type of cancer in the ed Table . Thirty-Hu-cancer-RNAseq-dataset).Several genes that are deeply implicated in cancer are not differentially expressed in any of the tumor types we analyzed. One example is tumor suppressor protein 53 (TP53) Figure  A and B.Fifteen of the 16 genes upregulated across all tumor types are co-expressed across the tumor samples, across the non-tumor samples and across the combined tumor plus non-tumor samples Figure . Cyclin In contrast, there is no co-expression cluster among the 19 genes that are downregulated across all cancer types; 62 individual gene pairs are correlated across all the samples (Spearman correlation ≥ 0.60) . ExpressTo determine whether the genes that are differentially expressed in cancers are involved in known biological processes, we performed gene ontology (GO) enrichment analysis using GO::TermFinder  and ReviWith the aim to use MOG from the vantage point of an individual gene, we selected the glypican 3 (GPC3) gene as an interesting candidate for a case study. GPC3, encoding a glycosylphosphatidylinositol-linked heparan sulfate proteoglycan, is located on the X chromosome and has been implicated as a critical regulator of tissue growth and morphogenesis . GPC3 inWe explored expression patterns of GPC3 with regards to the 14 tumor types. Differential expression of GPC3 in non-tumor versus tumor samples varies by organ. GPC3 expression is 30-fold higher in the LIHC samples than in the non-tumor liver samples, and 8-fold higher in the UCS samples compared to the non-tumor uterus samples . In contThese results are consistent with targeted studies of liver, breast and lung tumors. GPC3 expression is upregulated in liver cancer ,71–72, aWe then investigated co-expression patterns of GPC3 in the tumor and non-tumor tissues from different organs . GPC3 co-expression patterns differ between tumor and non-tumor samples according to the organ sampled Figure , moreoveWe analyzed GO term enrichment for those organs with more than 10 GPC3-co-expressed genes: colon, esophagus, kidney and liver. The term cell adhesion is enriched in GPC3-co-expressed genes from colon, esophagus, kidney and liver. The terms cell development, extracellular matrix organization and multicellular organism development are enriched among GPC3-co-expressed genes in colon, esophagus and kidney. Other GO terms are over-represented in a organ-specific manner .To further explore potential interactions of GPC3 with other genes, we used MOG to build two gene co-expression networks from the 3012 genes that are differentially expressed in LIHC—one network from non-tumor liver samples, and a second from LIHC samples . Then, we imported each network into Cytoscape  and idenIn the network built from non-tumor liver samples, MCODE ranked the GPC3-containing cluster second most significant (73 nodes (genes); MCODE score 30.7). GPC3 was directly connected with 21 genes in this cluster , which iIn contrast, in the LIHC samples, GPC3 was not significantly co-expressed with any other genes, and thus was absent from the entire LIHC network. However, the LIHC network does contain a module with 114 genes (MCODE score 94.3), 33 of which are in the GPC3-containing cluster identified from the non-tumor network (17 of these genes are directly connected with GPC3 in the non-tumor network) . This clP-value < 0.05). These genes have increasing expression with cancer progression. We similarly identified the genes that have a decreasing pattern of expression with cancer progression.To identify potential biomarkers for tumors, we used MOG to distinguish genes whose expression is associated with the disease progression. We used MOG to separate samples by organ, and then by early stage (stage I or stage II) and late stage (stage III and later), based on the study metadata. At last, we performed a Mann–Whitney U test on those genes that are upregulated in tumor versus non-tumor samples  to reveaESCA, KIRP, KIRC THCA all included metadata and had sufficient numbers per stage to detect differentally expressed genes.  MOG reveals 221 genes that increase expression during tumor progression , and 227 genes that decrease expression  . For exanot labeled as prognostic in THPA  gene is flagged by THPA and high CD70 expression is prognostic unfavorable for renal cancer. MOG analysis shows CD70 expression is higher in two types of renal tumors, KIRC and KIRP, and increases with disease progression Figure , but CD7exceptions can be extremely important, because specific tumor sub-types might respond differently to a particular treatment. By using MOG to explore RNA-Seq from large numbers of conditions and organs, a researcher can visualize data for individual samples or groups of that show changed expression of a prognostic marker or sets of markers, and compare these to those that do not.For prognosis and personalized medicine ,90 excepSuch exploration could suggest statistical analyses to try out in other, independent datasets to determine whether subsets of non-canonical samples might have a biologically distinct signature, revealing a different modality for a particular cancer. This in turn could be followed up by targeted experimental approaches or clinical studies.AT-microarray-dataset was to explore patterns of expression of genes with little or nothing known about them. The well-vetted dataset we used .We sought to identify genes of unknown or partially known function that might be involved in photosynthesis, the process that gave rise to the earth’s oxygenated atmosphere and the associated evolution of extant complex eukaryotic species. We focused particularly on regulation of the assembly and disassembly of the photosystem I and II light harvesting complexes; these dynamic processes respond sensitively to shifts in light and other environmental factors . In partP-value < 10−5) are integral to the light reactions of photosynthesis: generation of precursor metabolites and energy; photosynthetic electron transport in photosystem I (PSI); reductive pentose-phosphate cycle; response to cytokinin; and PS2 assembly  Biological Functional terms most highly over-represented  ,96–97, asing CDD ). A Pearsing CDD ; these gP-value < 10−3) . The GO terms most highly enriched  among genes upregulated in pollen are processes of cell cycle, mitosis, organellar fission, chromosome organization and DNA repair . This reflects and emphasizes the critical role of these processes in male gametophyte development, particularly sperm biogenesis. Each angiosperm pollen grain must produce two viable sperm each used in the double fertilization of the ovule. Above all else, proper mitogenesis is essential to the function of a pollen grain. We visualized the leaf versus pollen differential analysis by volcano plot  among the genes are: defense response, response to stress, response to external biotic stimulus and response to other organism  if the available memory is not sufficiently large. For example, a dataset of 100 000 human transcripts over 5000 samples  requires at least 4GB  of free memory to be loaded into memory at once. To circumvent this problem, R developers can use the new DelayedArray  frameworIn contrast, MOG uses an indexing strategy to read data only when it is needed, which drastically reduces the total memory consumption of the system. Table Hu-cancer-RNASeq-dataset (18 212 genes over 7142 samples) using a laptop with 64 bit Windows 10, 8 GB RAM and Intel(R) Core(TM) i5-7300HQ CPU; the system’s resource utilization was monitored by Windows Performance Monitor tool (WPMT) (We benchmarked MOG’s performance with the l (WPMT) . During l (WPMT)  , but was able to load a subset of data consisting only of 410 tumor and non-tumor liver samples. We measured the execution time (time taken to compute and display output) of the Mann–Whitney U test for differentially expressed genes in tumor versus non-tumor samples. The test completed in 21 min with PIVOT, but only seven seconds with MOG . For each dataset, we performed a Mann–Whitney U test on all the genes for tumor versus non-tumor groups. MOG took only 31 s to compute a Mann–Whitney U test on 18 212 genes over 1054 samples , an A. thaliana microarray dataset (AT-microarray-dataset) and an A. thaliana metabolomics dataset (AT-metab-dataset). In each case, known information was recapitulated in the MOG analysis, and new potential relationships became apparent.We demonstrated MOG’s functionalities by exploring three different well-validated datasets: a human RNA-Seq dataset from non-tumor and tumor samples  into MOG for two reasons. First, the selection of appropriate statistical and computational methods depends on the data structure and the biological questions to be asked. Different types of data have different characteristics , and if Analyses performed while exploring and statistically analyzing datasets on MOG can be saved; by clicking ‘save’, all the analyses that have been performed are added as objects to the MOG project file. Results obtained with MOG can be shared by sharing the saved MOG project file. If a user wishes to document the information to reproduce the analysis, she/he needs to manually specify the parameters and methods used. In the future, we plan to implement automated report generation for each analysis.MOG is a novel Java software for interactive exploratory analysis of big ’omics datasets or other datasets. By using an indexing strategy to read data only when it is needed, the total memory consumption of the system is minimized, enabling MOG to perform much more efficiently than the available R-based software. Visualizations produced by MOG are fully interactive, and enable researchers to detect and mine interesting data points and probe the relationships among them. The statistical methods implemented in MOG help to guide exploration of hidden patterns in a user-friendly manner. By integrating metadata, MOG affords an opportunity to extract new insights into the relationships between gene expression and gene structure, gene location, or any of the diverse information entered by scientists about the biology and experimental conditions.Taken together these features can aid a researcher in developing new, experimentally testable hypotheses.http://metnetweb.gdcb.iastate.edu/MetNet_MetaOmGraph.htm. MOG’s source code and user guide is available at https://github.com/urmi-21/MetaOmGraph/. MOG’s source code (version 1.8.0) at the time of submission is archived and can be accessed using the DOI:10.5281/zenodo.3520986. Additional files are available at https://github.com/urmi-21/MetaOmGraph/tree/master/MOG_SupportingData.We subscribe to FAIR data and software practices . MOG is gkz1209_Supplemental_FilesClick here for additional data file.


The increasing availability of molecular data provided by next-generation sequencing (NGS) techniques is allowing improvement in the possibilities of diagnosis and prognosis in renal cancer. Reliable and accurate predictors based on selected gene panels are urgently needed for better stratification of renal cell carcinoma (RCC) patients in order to define a personalized treatment plan. Artificial intelligence (AI) algorithms are currently in development for this purpose. Here, we reviewed studies that developed predictors based on AI algorithms for diagnosis and prognosis in renal cancer and we compared them with non-AI-based predictors. Comparing study results, it emerges that the AI prediction performance is good and slightly better than non-AI-based ones. However, there have been only minor improvements in AI predictors in terms of accuracy and the area under the receiver operating curve (AUC) over the last decade and the number of genes used had little influence on these indices. Furthermore, we highlight that different studies having the same goal obtain similar performance despite the fact they use different discriminating genes. This is surprising because genes related to the diagnosis or prognosis are expected to be tumor-specific and independent of selection methods and algorithms. The performance of these predictors will be better with the improvement in the learning methods, as the number of cases increases and by using different types of input data . This will allow for more precise identification, classification and staging of cancerous lesions which will be less affected by interpathologist variability. Renal cell carcinoma (RCC) is not a single entity but rather a heterogeneous set of tumors classified in about 40 subtypes, of which clear cell (ccRCC), papillary and chromophobe RCC account for 70%, 10–15% and 5%, respectively . TherefoIn fact, tumor staging is not only a valuable prognostic factor, but it is also used to determine the right treatment strategy for patients and to predict the risk of metastasis development. The currently adopted prognostic factors for RCC include the TNM staging system, the four-tiered WHO/ISUP  grading system, histologic subtype, presence of the sarcomatoid component, microvascular invasion, tumor necrosis and invasion of the collecting system . In caseAt the same time, the use of molecular data has been explored to improve the reliability of diagnosis and prognosis in RCC. Currently, next-generation sequencing (NGS) techniques offer the opportunity to extract a multitude of new features, genomic and transcriptomic, potentially related to the phenotype. For example, gene variations, RNA expression, alternative RNA splicing events and gene fusions data from a biopsy are relatively easily obtainable . HoweverIn this section, the main algorithms used to implement diagnostic and prognostic predictors starting from molecular variables in RCC will be summarized. However, extensive and recent reviews of the history of AI and its applications in medicine and oncology were carried out by Hamamoto et al.  and HameMachine learning, the algorithms to implement AI, can be divided into two main types: supervised and unsupervised learning . The forThe best-known artificial intelligence algorithms are artificial neural networks (ANNs), structures that mimic the neuronal topology of the human brain. They can have several artificial neurons (or nodes) organized in layers, and neurons of each layer can implement different transfer functions. All of this ensures a great flexibility in dealing with different tasks. Generally, the available cases are divided into a training set and a test set  but sometimes into training, validation and test sets . These algorithms perform well if the training data are numerous, representative of reality and not contradictory. Many training attempts must be made before the desired performance is achieved, and during these attempts, it may be necessary to modify the network structure, i.e., number of nodes and the type of transfer function.While an advantage of ANNs is their ability to discover and model complex relationships among data, they present two weaknesses. Firstly, since ANNs are non-linear algorithms, training usually results in a relative minimum of the error function (between obtained and expected outputs) and not in the absolute minimum. The second is the overfitting of the data, which is a condition that occurs when the algorithm does not generalize the data profile but tries to follow them so precisely that it ends up chasing noise. Fortunately, this drawback can be easily noticed; in fact, too high performance on training data and very poor performance on test data indicate overfitting. Overfitting can also be prevented by applying methods that interrupt the learning cycles of the algorithm when the performance of the two datasets (training and test) starts to diverge .Support vector machines (SVM) are also algorithms used for classification. During the learning process, these algorithms look for a hyperplane separating the two datasets . SVMs do not use all the data for the learning but rather only one datum representing the closest point between the two sets . Usually, these algorithms are linear; therefore, they reach the absolute minimum of the error function. Moreover, they are particularly suitable when there is a clear separation between the data groups to be classified. Instead, they have poor performance in the case of noisy data .The random forests algorithm (RFs) combines the predictions of many decision trees (forests) into a single model. Each decision tree learns from a subset of elements chosen randomly from all available training data (bootstrap). Subsequently, the average of the predictions of each decision tree (bagging) is calculated, allowing obtaining the final predictions .Finally, Lasso regression (least absolute shrinkage and selection operator regression) is an algorithm that performs independent variable selection (feature selection) and regularization (to reduce variance). It can select important predictors of a model ,25.However, there are several challenges in the application of machine learning to large amounts of data, such as genomic data. The first problem, which we have already mentioned, concerns data overfitting that occurs because the dimension of the input is much larger, at least one order of magnitude, than the sample number, and this is also known as the “large p, small n problem”. Second, the influence of each input variable on the prediction is difficult to assess because of the multiple non-linear operations. The third problem is called the “black box problem”, that is, it is not possible to understand the reason that generates the results and to predict the behavior of the system due to the complexity of the machine learning techniques. Since the European Union’s General Data Protection Regulation (GDPR) of 2018 requires the transparency of AI, it will be necessary to address the black box issue .In this review, we investigated and reported the state of art of the application of AI for diagnosis and prognosis in renal cancer using only molecular data as input, the most commonly selected genes, and we analyzed the non-AI-based predictors making a comparison between the two approaches. A Pubmed search was conducted using the keywords “artificial intelligence”, “machine learning algorithm”, “renal cell carcinoma”, “renal cancer”, “kidney cancer” and excluding the keywords “radiomic”, “imaging”, “histopathology images”, “CT-based”, “tomography” and “MRI”.Most of published studies are focused on the prediction of prognosis and diagnosis in clear cell RCC since this is the most dominant type of renal cancer; therefore, more data are available and, consequently, the algorithms can be better trained. Only two publications, of the same authors, concern predictions on the papillary cancer type ,28.Usually, the RNA-seq gene expression and, in some cases, methylation data were obtained from The Cancer Genome Atlas (TGCA) resource, thus ensuring the uniformity of the starting data. Various authors used level 3 TGCA data ,29,30, iThe first phase of the analyses consists in identifying the most differentially expressed genes (DEGs) between the control and treated condition. This objective is not trivial; in fact, there are still limitations and biases that do not allow capturing all the DEGs due to the preparation of the library, the different representativeness as a function of the RNA length, the alignment of the reads and the tests for expression ,35,36,37The second phase of the analyses consists in a reduction in the starting data (feature selection) to only the most important ones for discriminating cases, and this represents a challenge. In several studies, the feature selection was obtained through various methods, for example, using clustering algorithms, principal component analysis or random forests. In particular, as selection criteria, we start from simple methods, statistical P-values and fold changes . More elFurthermore, in order to distinguish cancer from non-cancer samples, RNA-seq data of 537 ccRCC patients collected in TCGA were used to train a supervised learning classifier based on a support vector machine (linear kernel), and the receiver operating characteristic (ROC) curve was adopted to measure the performance of this classifier . The algn = 523) to train SVM, random forests, SMO, naïve Bayes and J48 algorithms [A study published in 2015 focused on prediction of kidney cancer survival (< or ≥5 years) using TCGA RNA-seq data of 220 patients . It was gorithms . The SVMgorithms . At the gorithms .In 2018 and 2020, the same authors dealt with papillary renal cell carcinoma (PRCC) to discriminate between early and late stages of the disease. In particular, 104 genes were selected from gene expression profiles derived from 161 patients and used in a shrunken classifier . A test A very recent study adopted different algorithms  to predict stages in ccRCC . UnfortuWe did not include two studies in It is difficult to compare the performance of these different prediction algorithms, since they concern different RCC subtypes and each predictor is fed with different types and numbers of genes. Furthermore, among the prognosis predictors in ccRCC, some algorithms have been trained to predict survival, and others to distinguish early from late stages, which, although they are two strongly correlated variables, are not identical. It should be taken into account that the specific parameters used by programmers to implement each specific algorithm are not known in detail. However, the best performance was achieved in the most recent study on ccRCC prognosis (AUC 0.86) which also used the fewest genes, by employing an SVM that performed better than logistic regression, multi-layer perception (MLP), random forests and naïve Bayes.We performed features comparisons to identify which selected genes were in common among studies regarding ccRCC prognosis in The HUS1B (Checkpoint protein HUS1B) gene forms a complex with Rad9 and Rad1 which are involved in response to damaged DNA, triggering cell cycle checkpoint signaling and DNA repair mechanisms . CACNA1DFrom this comparison, it emerges that different studies with the same goal, for example, prognosis prediction in ccRCC, selected different gene lists, but all their algorithms performed well. On the other hand, other studies with the specific purpose of selecting prognostic genes in ccRCC, which also used the TCGA source, obtained lists of genes that are different from each other and from the above-mentioned genes ,54,55,56Despite joining all genes used by studies for ccRCC prognosis predictions, none of them were in common with the genes used for the papillary-type RCC predictions, confirming that the two cancer variants are very different at the molecular level.We then analyzed the published studies which used non-AI-based predictors for diagnosis or prognosis in RCC to compare their performance with that of the predictors reported in Although these studies are not directly comparable to the above-mentioned ones because the AUC was not calculated, they generated lists of genes different from the above-mentioned studies but significantly associated with survival or tumor grade.All studies demonstrated to have selected genes highly correlated to the overall survival, and one study reported a successful discrimination of patients from healthy individuals . RegardiRegarding studies about ccRCC prognosis, we joined the genes selected from these previous studies , NNMT (Nicotinamide N-methyltransferase), S100A4 (Protein S100-A4), SPINT2 (Kunitz-type protease inhibitor 2), TNFAIP6 (Tumor necrosis factor-inducible gene 6 protein), VCAM1 (vascular cell adhesion molecule 1) and VEGFA°  is already known. In particular, the expression of GPX3 is decreased in ccRCC ,65. NNMTThe use of the patient’s clinical and molecular variables is very useful for obtaining new information important for personalized therapy development. Today, we have much more molecular information available thanks to next-generation sequencing techniques and therefore more possibilities to identify the truly discriminating molecular features. There are several bioinformatic methods for the identification of discriminating variables, which subsequently will be used by the various artificial intelligence algorithms. These variables, such as transcripts, proteins or metabolites, could also represent new therapeutic targets.Artificial intelligence systems are able to learn the relationships among data only by looking at the examples and are able to capture and reproduce non-linear relationships among the data. These algorithms are constantly being improved to ensure that they can learn better and faster and be more robust to the noise in the data. Another issue is that most machine learning algorithms are so-called “black boxes”, that is, they derive an internal model of the functioning of reality, but this cannot be explained.These machine learning methods can also stratify patients more accurately, identifying those who present a low-stage but high-risk expression profile tumor and therefore should receive adjuvant therapies and major attention. On the other hand, patients with a high-stage but low-risk expression profile could receive less aggressive treatments under close observation. However, these objectives remain challenging, especially when considering the great molecular heterogeneity of kidney tumors.In this study, firstly, we show that artificial intelligence algorithms yield fairly accurate predictions, even with a low number of variables , but theIn colon and breast cancers, AI predictors reached an accuracy of 0.767 and 0.807, respectively, for disease recurrence . Better To increase the accuracy of predictions in prognosis, data on mutations have been integrated with those of gene expression ,92,93. HIn the future, thanks to the greater availability of data in TCGA, it will be possible to realize gender-, ethnic- and RCC variant-specific predictors.AI-based predictors are powerful tools that can be continuously trained as new data become available. In this review, we summarized recent studies that adopted these predictors for diagnosis and prognosis in RCC. We show the good performances obtained so far, but also the need for improvement in order to achieve real clinical usefulness.