Distinguishing ductal carcinoma in situ (DCIS) from invasive ductal carcinoma (IDC) regions in clinical biopsies constitutes a diagnostic challenge. Spatial transcriptomics (ST) is an in situ capturing method, which allows quantification and visualization of transcriptomes in individual tissue sections. In the past, studies have shown that breast cancer samples can be used to study their transcriptomes with spatial resolution in individual tissue sections. Previously, supervised machine learning methods were used in clinical studies to predict the clinical outcomes for cancer types.We used four publicly available ST breast cancer datasets from breast tissue sections annotated by pathologists as non-malignant, DCIS, or IDC. We trained and tested a machine learning method (support vector machine) based on the expert annotation as well as based on automatic selection of cell types by their transcriptome profiles.We identified expression signatures for expert annotated regions  and build machine learning models. Classification results for 798 expression signature transcripts showed high coincidence with the expert pathologist annotation for DCIS (100%) and IDC (96%). Extending our analysis to include all 25,179 expressed transcripts resulted in an accuracy of 99% for DCIS and 98% for IDC. Further, classification based on an automatically identified expression signature covering all ST spots of tissue sections resulted in prediction accuracy of 95% for DCIS and 91% for IDC.This concept study suggest that the ST signatures learned from expert selected breast cancer tissue sections can be used to identify breast cancer regions in whole tissue sections including regions not trained on. Furthermore, the identified expression signatures can classify cancer regions in tissue sections not used for training with high accuracy. Expert-generated but even automatically generated cancer signatures from ST data might be able to classify breast cancer regions and provide clinical decision support for pathologists in the future.The online version of this article (10.1186/s13058-019-1242-9) contains supplementary material, which is available to authorized users. Breast cancer is the most common cancer and the highest incidence of all cancers in women with an incidence rate of over 1.6 million cases per year . The morSpatial transcriptomics (ST) is an in situ capturing technique, which uses a glass slide containing oligonucleotides to capture mRNAs, while maintaing the spatial information of histological tissue sections. Following cDNA synthesis, the resulting barcoded cDNA libraries are sequenced using standard RNA-seq technology . SpecifiST allows for a wide range of applications. The transcriptome is measured for the whole tissue section by sequencing that allows to compare different areas within that tissue section , 21. ForFor the study, we used four publicly available ST breast cancer datasets .The ST sequencing results in paired-end reads. Read one R1) contains the spatial barcode and the unique molecular identifier (UMI). Read two (R2) contains the transcript sequence information. All the datasets were processed using the ST Pipeline version 1.3.5 with default settings , 25. The containsThe ST Pipeline generates a BED file containing the sequence reads mapped to genomic positions together with the spatial locations (the ST spots) of all the reads. All the transcripts annotated to the gene Malat1 were removed due to its overexpression and internal priming. We developed an open-source computational pipeline that uses the BED file to compute ST tag clusters (ST-TCs) for all the ST spots. Based on the transcription termination site (TTS) profiling, we computed TTS regions (ST-TCs) by peak calling of the transcripts genomic positions using the parametric clustering method paraclu , which wWe developed a Python open-source tool for unsupervised classification of ST spots based on the ST-TC expression profiles using the sklearn framework . DatasetDimensionality reduction was performed using principal component analysis (PCA). PCA is a dimensionality reduction technique that simplifies the complexity in high-dimensional data (such as gene expression data) by transforming the data into fewer dimensions while retaining important trends and patterns . Furtherclasses in machine learning terminology) using differential expression analysis .We classified (annotated) ST spots based on the morphology observed in H&E-stained histological tissue sections. The ST spots covering a minimum of 20 cells in the tissue section were selected. We identified subtype-specific ST-TCs (gene expression signatures) for non-malignant, ductal carcinoma in situ (DCIS), or invasive ductal carcinoma (IDC) regions (the three supervised expert annotation of the H&E images and independently in an unsupervised way by PCA. The identified expression signatures were used to characterize ST spots in ST experiments (testing of the model) which were withheld during model training.We used machine learning technique to learn expression signatures (train a model) for non-malignant, DCIS, or IDC regions. The regions were identified in two ways, by ground truth) and provided as separate files. The training and test datasets are pre-processed and normalized as described in the unsupervised classification. The training datasets together with the ST spot labels were used to train a model using multi-class support vector machine (MC-SVM) method [For this, we developed a Python open-source tool (see references to tools) for supervised classification of ST spots using a multi-class support vector machine with the sklearn framework . The too) method . We chosThe spatial transcriptomics (ST) technology places histological tissue sections on ST glass slides composed of 1007 ST spots covering the slide. Following tissue permeabilization, polyadenylated transcripts are captured on the slide and 3′ end sequencing libraries are produced containing spatial barcodes, determining where on the slide each transcript was captured .Here, we developed a data processing and data analysis workflow for ST data, which extends the previously employed gene model centric expression analysis . In brieFrom four publicly available ST datasets of breast cancer tissue  processeclasses. We conducted differential expression analysis -stained histological tissue sections are routinely examined and classified by pathologists. We manually annotated the four breast cancer ST experiments and selected a total of 194 ST spots consisting of non-malignant regions, ductal carcinoma in situ (DCIS) regions, and invasive ductal carcinoma (IDC) regions Fig. . These tThe differentially expressed transcripts can be regarded as marker signature for the three classes and in turn each class specifically expressed some of the marker transcripts Fig. b. SeveraMachine learning has been frequently used for cancer prediction and prognosis. The method employs statistical, probabilistic, and optimization techniques to learn from known examples in order to recognize patterns in large complex datasets. Application examples range from general disease diagnosis to precision medicine  and inclWe first trained a machine to learn expression signatures based on ST-TC expression data for ST spots manually annotated to the classes non-malignant, DCIS, or IDC. For this, we used a multi-class (three classes) support vector machine (MC-SVM) which performs classification by constructing hyperplanes in a multidimensional space. Our workflow for the four breast cancer ST datasets used three datasets to train the machine (the machine generates a model) and the fourth dataset was used for validating or testing the model. This process of training on three datasets and validating on the fourth dataset was conducted four times in total and is often referred to as cross-validation technique . The preFrom the manually selected ST spots of the four breast cancer datasets Table A, we useWe further tested the performance of the MC-SVM classifier for the same 133 ST spots without performing prior differential expression analysis. Here, we used all 25,179 ST-TCs for model training of the selected ST spots. The classifier identified ST spots in dataset 1 with an accuracy of 0.95, 0.98, and 0.97 for non-malignant, DCIS, and IDC spots, respectively , HCA groups the ST spots on the 2-dimensional PCA plot such that each ST spot belongs to one cluster. Mapping these three groups of ST spots onto the four stained tissue images . We applied principal component analysis (PCA) on the data matrix consisting of 25,179 ST-TCs and 979 ST spots to place ST spots with similar expression close to each other in a 2-dimensional representation  of the 25,179 dimensional expression space Fig. ”. Given We used the three groups of ST spots resulted from the unsupervised identification and performed the cross-validation workflow. The machine was trained on the combined three breast cancer datasets 2, 3, and 4 and validated on the dataset 1 resulting in an average F1-score of 0.94  data, trained with one machine learning algorithm (MC-SVM) and evaluated the performance of the model to identify cancer regions. Four independent ST datasets were available and used in this study. Expert pathologist annotation of the H&E-stained tissue images provided classification of 194 ST spots of ductal carcinoma in situ (DCIS), invasive ductal carcinoma (IDC), and non-malignant tissue regions for all four experiments. We derived ST expression signatures for each of the three classes consisting of distinct sets of transcripts (798 ST-TCs) Fig. a. In addIn the past, various supervised machine learning algorithms were used in clinical studies to predict clinical outcomes based on expression signatures from bulk cancer samples, distinguishing DCIS from IDC or for extracting cell type-specific information from gene expression profiling from heterogeneous samples using deconvolution techniques , 48–58. Our further analysis employed unsupervised classification of ST spots for identifying cancer regions. The resulting three distinct classes corresponded to the three expert annotated regions. The classification performance of the machine learning model (93.7%) was less accurate compared to the expert selected ST spots  are grouped into ST tag clusters (ST-TCs) by peak calling (B).Additional file 3:Figure S2. Volcano plot representation of differentially expressed ST-TCs. The tag clusters. Expression profiles of a) Non-malignant versus DCIS, b) DCIS versus IDC, c) Non-malignant versus IDC. The x-axis represents log2 expression fold change and the y-axis represents log10 p-value. The pirate plot of normalized log expression values for the differentially expressed tag clusters highlighted (examples) in volcano plot.Additional file 4:Figure S3. Top 25 Enriched GO terms represented in dot plot. The size of the dots represent the number of genes associated with the given GO term and the color of the dots represent the P-adjusted values.


MYCN amplification was more frequently occurred in the ultra-high-risk subtype, in accordance with the overexpression of MYC/MYCN targets in this subtype. In summary, prognostic subtypes identified by deep learning-based multi-omics integration could not only improve our understanding of molecular mechanism, but also help the clinicians make decisions.High-risk neuroblastoma is a very aggressive disease, with excessive tumor growth and poor outcomes. A proper stratification of the high-risk patients by prognostic outcome is important for treatment. However, there is still a lack of survival stratification for the high-risk neuroblastoma. To fill the gap, we adopt a deep learning algorithm, Autoencoder, to integrate multi-omics data, and combine it with K-means clustering to identify two subtypes with significant survival differences. By comparing the Autoencoder with PCA, iCluster, and DGscore about the classification based on multi-omics data integration, Autoencoder-based classification outperforms the alternative approaches. Furthermore, we also validated the classification in two independent datasets by training machine-learning classification models, and confirmed its robustness. Functional analysis revealed that  MYCN amplification  and accounts for approximately 15% of childhood cancer mortality  and Therapeutically Applicable Research to Generate Effective Treatments (TARGET) projects, multi-omics integration is much needed in cancer researchers. Recently, Suo et al.  have proIn this study, we used multi-omics-based unsupervised learning to stratify the high-risk neuroblastoma based on the new features re-encoded by Autoencoder algorithm, and compared the stratification with those identified by iCluster or PCA. The stratification of high-risk neuroblastoma by Autoencoder was also validated in two independent datasets, which may not only help the clinicians make rational and efficacious chemotherapeutic protocols, but also demonstrate that the deep learning-based algorithm is very efficient in multi-omics integration.https://ocg.cancer.gov/programs/target/data-matrix), and GEO database with accession number GSE49711 for SEQC cohort.We used multi-omics data from two projects in this study: Therapeutically Applicable Research to Generate Effective Treatments (TARGET) project  for further analysis.To integrate gene expression and CNA data, we first stacked these two datasets by the190 overlapping samples from TARGET cohort to form a new one. Then we selected the initial prognostic features (genes or CNAs) with Cox regression , obtaining labels for different survival-risk groups through K-means clustering from transformed new features by Autoencoder, and training classifiers with models such as SVM, Naïve Bayes, and logistic regression according to the class labels.Also, to demonstrate the robustness of the classification at predicting prognosis, these supervised classification methods mentioned above, along with XGBoost, were trained on gene expression data and CNA data respectively by different machine learning methods.There were two datasets used for demonstrating the robustness of the classification for predicting prognosis, one was the remaining 190 samples which had CNA data only , and the other was 176 high-risk samples with gene expression data in SEQC project . The class labels for samples from TARGET internal validation set and SEQC external validation set were predicted by CNA-based XGBoost and gene expression-based SVM models, respectively.https://ocg.cancer.gov/programs/target/data-matrix).The gene expression data from TARGET project were profiled by Affymetrix Exon ST platform, and normalized by Robust Multi-array Average (RMA) procedure, which could be downloaded from the website , of which the 100-node bottleneck layer was used to represent new features for further analysis. We then selected 35 survival-associated features  from the 100 new features.Autoencoder is a dimensionality reduction method based on artificial neural network, which consists of input, hidden, and output layers. The data integration analysis by Autoencoder was implemented in R programming software with package For a given layer, a specific activation function was assigned, and the output x' was given by a composite function of x, which was composed of all these activation functions from each layer, and could be expressed as:where k represents to the number of layers.the Pseudo-Huber loss function, which ensures that derivatives are continuous for all degrees, that is:We measured the error with function where n stands for the dimension of the input data. As can be seen in Supplementary Figure The Autoencoder was trained using the gradient descent algorithm with 10 epochs, a batch size of 32, and a learning rate of 1e-6. The parameters of L1 and L2 regularization were set to 0.0001 and 0.001.For the RMA-based gene expression data by microarray platform, we transformed the expression value as Z-score for each gene. For the gene expression data of RNA-seq, we firstly calculated the fractions of the genes that had a FPKM value over the threshold we set, which can be seen in the Supplementary Table https://ocg.cancer.gov/programs/target/data-matrix). We merged the segmented CNAs from the 380 samples, and annotated the genes in the CNAs by GISTIC2.0 . The rows and columns of the CNA matrix represent the genes and samples, respectively. Each element of the CNA matrix was normalized as log2 (segmented copy number)−1.The segmented copy number regions with segment means were available at the TARGET website . The feature combinations with highest average predictive accuracy were selected. The features for XGBoost were selected by its internal algorithm. Given the features, receiver operating characteristic (ROC) curve was plotted for each model, and the one with highest area under the curve (AUC) was selected as the prediction model.Features for the models, including Naïve Bayes, logistic regression and SVM, were selected by a backward elimination manner. For each gene or CNA, the importance was evaluated by the ANOVA t test were implemented in R programming software with version 3.5.0. In addition, we determined the optimal number of clusters on three metrics: C index for the prognostic differences, Silhouette index and Calinski–Harabasz criterion  with a functional database named Hallmark50  analysis, principal component analysis, K-means clustering, integrative clustering and student-We collected 407 high-risk neuroblastoma samples from TARGET project  or overall survival (OS)], which were used for integrative analysis later on.As illustrated in Figure P < 0.05) associated with EFS or OS. Subsequently, K-means clustering analysis was performed on the 35 new features with clustering number ranging from 2 to 6 . The two-omics data were integrated and represented by the 100 new features obtained from the bottleneck layer of the autoencoder. We then conducted a univariate Cox-PH regression on each of the 100 new features and identified 35 features significantly  and integrative clustering analysis (iCluster) were also incorporated to evaluate the performance of multi-omics integration approaches . In addition, as compared with the DGscore method , Autoencoder-based classification also achieved higher statistical significance (P = 5.66e-6 for EFS and P = 1.28e-5 for OS). The result indicated that Autoencoder-based multi-omics integration outperformed these alternative approaches.In the training data, we found that the classification by Autoencoder had better performance than the other two approaches Figures –F, amongTo demonstrate the robustness of the classification at predicting prognosis, we built two supervised classification models based on gene expression and CNA data separately to predict the classification labels for samples from both internal and external validation datasets, respectively.After obtaining the labels from K-means clustering, we first built two supervised models based on the gene expression and CNA data, respectively. Each omics data was normalized as Z-score to avoid platform differences. For the internal validation, we used the remaining 190 samples with only CNA data from the TARGET project which didn't overlap with the samples with gene expression data. Meanwhile, the 176 SEQC high-risk neuroblastoma samples with gene expression data was used as external validation.P values (< 0.0001) between the two subtypes  was then performed on the two gene sets . Interestingly, MYCN amplification (MNA) was observed to present higher frequency in G1 than G2 , indicating that our classification was associated with MNA to a certain extent, but some samples without MNA also had poor survival. However, we did not identify significant down-regulated pathways in G1 subtype. Alternatively, interferon-alpha response pathway was down-regulated in G1 , which is a common defect in human cancers . The Autoencoder-based classification also generates high C-index (0.74 ± 0.08 for EFS and 0.71 ± 0.08 for OS). The performance comparison of Autoencoder with PCA and iCluster demonstrates that our Autoencoder-based classification is superior to the two alternative approaches. Moreover, the result of Autoencoder-based classification is also more significant than DGscore method. To demonstrate the robustness of the classification, we build two supervised classifiers for the independent CNA and gene expression datasets, respectively. For both of the datasets, we achieve good C-indices and significant log-rank P-values (P < 0.05). We thus conclude that Autoencoder-based classification outperforms other approaches, and we speculate that the unique advantage of the Autoencoder, which can capture the core features relevant to the prognosis, have contributed to this.In the present study, we have adopted a deep learning-based algorithm, Autoencoder, to integrate copy number alterations and gene expression data to identify two prognostic subtypes, defined as G1 and G2, in high-risk neuroblastoma. The subtype G1 exhibits worse prognosis than G2 in both EFS and OS (High-risk neuroblastoma is an aggressive disease. To our knowledge, the present study is the first to apply deep learning approach to distinguish ultra-high-risk subgroup from the high-risk neuroblastoma, with validation in independent datasets. The integrative classification of the high-risk neuroblastoma may help clinicians develop personalized treatment programs, and better predict patients' prognosis.Prognostic subtypes identified by deep learning-based multi-omics integration could not only improve our understanding of molecular mechanism, but also help the clinicians make decisions.In this study, TS designed the study. LZ and CL adapted algorithms and software for building model and data analysis. LZ, CL, and YJ interpreted data in context of NB biology. LZ, YJ, CL, and GC drafted the manuscript. YF, DY, and YT collected the data. TS, XN, and YG revised and finalized the manuscript. All authors read and approved the final manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Neuroblastoma is a major cause of cancer death in early childhood, and its timely and correct diagnosis is critical. Gene expression datasets have recently been considered as a powerful tool for cancer diagnosis and subtype classification. However, no attempts have yet been made to apply deep learning using gene expression to neuroblastoma classification, although deep learning has been applied to cancer diagnosis using image data. Taking the International Neuroblastoma Staging System stages as multiple classes, we designed a deep neural network using the gene expression patterns and stages of neuroblastoma patients. Despite a small patient population (n = 280), stage 1 and 4 patients were well distinguished. If it is possible to replicate this approach in a larger population, deep learning could play an important role in neuroblastoma staging. In addition to careful analyses of clinical symptoms, numerous diagnostic methods have been used to diagnose cancer . In partWith the increasing amount of available data from visual images over recent years, numerous diagnostic techniques for cancer have been developed through machine learning methods such as convolutional neural networks (CNNs) ,3. MoreoIn addition to image data, basic classifications, such as a diagnosis of cancerous versus healthy tissue, can be performed through gene expression data, and models have been developed using traditional machine learning methods. However, AI-based deep neural networks (DNNs) can be developed using classification models with data matrices of continuous values such as expression data. Unlike image data, genomic data can be used as a proxy for the early diagnosis of cancer, meaning that models based on gene expression data can also be useful for identifying or predicting the diagnosis or progression of cancer and for providing timely and appropriate cancer treatment .However, to construct a DNN model, a sufficient dataset is required . AlthougIn addition to furnishing genomic data, these sources also provide data indicating patients’ medical status, allowing us to examine the correlations between clinical variables and specific genomic data of interest ,9.Neuroblastoma is an extracranial solid tumor that most commonly occurs in childhood ,11. The Neuroblastoma is staged using the International Neuroblastoma Staging System (INSS) . This syHowever, as increasing amounts of data on neuroblastoma have become available, and suitable genomic data can be obtained from public databases , it is now possible to explore whether a correlation exists between INSS stages and genomic traits such as the mutation profile or gene expression data .In this study, in order to identify such correlations, we developed a simple DNN model using a data set of neuroblastoma patients including gene expression data and clinical data . We investigated whether our DNN model with gene expression data could classify the INSS stages.https://www.ncbi.nlm.nih.gov/geo/). GSE85047 contains 280 samples of neuroblastoma clinical data and the data matrix includes INSS stage and expression array data. An expression array was performed using an Affymetrix Human Exon 1.0 ST Array   . The INSIn order to convert the array ID into the HUGO gene symbol, we used the GPL5175 probe-gene symbol mapping annotation file. Next, we edited the matrix containing only INSS stage and gene expression array data for data feeding into the DNN architecture. The data matrix was 280 patients by 13,091 gene symbols. We split this data matrix into a training set and test set at a ratio of 8:2, using scikit-learn train_test_split .To construct our DNN model, we utilized TensorFlow 1.13.1 as our machine learning library with Python 3.6.0 .We chose tf.contrib.learn.DNNClassifier for model construction. For the hyperparameters of our model, we set the dropout rate at 0.15, we chose the Adam optimizer, and we fixed the learning rate at 1e-5. The activation function was leaky_relu and the number of layers was 4. The numbers of neurons of the layers were 512, 256, 128, and 16, respectively . When weIn order to obtain measurements for the performance of our model, accuracy was calculated using the predicted values from the training set and the test set; then, receiver operating characteristic curves and the area under the curve (AUC) were obtained by the roc-curve function in the scikit-learn package.After 5,000 iterations with the training set, the accuracy was calculated from each training set and test set, with values of 100% and 55.56%, respectively.In the training set, the macro-average AUC, micro-average AUC, and all the AUC values for one-versus-rest (OVR) decisions were all 1 . In the From these results, we could distinguish stages 1 and 4 in neuroblastoma patients. Considering the poor prediction of the other stages in the test set, it is likely that overfitting occurred for stages 2, 3, and 4S. Alternatively, there may be no distinguishable genes between stages 2, 3, and 4S in terms of gene expression.Our study was performed using data from a relatively small number of patients (280 cases). Increasing the number of patients to the order of 103 or 104 would be appropriate, since successful DNN construction requires several thousand labeled cases in biological problems ,17.


Current laboratory tests are less than 50% accurate in distinguishing between people who have food allergies (FA) and those who are merely sensitized to foods, resulting in the use of expensive and potentially dangerous Oral Food Challenges. This study presents a purely-computational machine learning approach, conducted using DNA Methylation (DNAm) data, to accurately diagnose food allergies and potentially find epigenetic targets for the disease.An unbiased feature-selection pipeline was created that narrowed down 405,000+ potential CpG biomarkers to 18. Machine-learning models that utilized subsets of this 18-feature aggregate achieved perfect classification accuracy on completely hidden test cohorts (on an 8-fold hidden dataset). Ensemble classification was also shown to be effective for this High Dimension Low Sample Size (HDLSS) DNA methylation dataset. The efficacy of these machine learning classifiers and the 18 CpGs was further validated by their high accuracy on a large number of hidden data permutations, where the samples in the training, cross-validation, and hidden sets were repeatedly randomly allocated. The 18-CpG signature mapped to 13 genes, on which biological insights were collected. Notably, many of the FA-discriminating genes found in this study were strongly associated with the immune system, and seven of the 13 genes were previously associated with FA.a priori biological insights to construct a 96-CpG signature. This research builds on previous work because it uses a completely computational approach to obtain a perfect classification accuracy while using only 18 highly discriminating CpGs . In machine learning, simpler models, as used in this study, are generally preferred over more complex ones . Lastly, the completely data-driven methodology presented in this research eliminates the need for a priori biological information and allows for generalizability to other DNAm classification problems.Previous studies have also created highly-accurate classifiers for this dataset, using both data-driven and  Food Allergy (FA) is a specific immune response that occurs upon exposure to a particular food . FA affeCurrent laboratory procedures used to detect food allergies, such as allergen-specific IgE (sIgE) testing and skin prick tests (SPT), are effective predictors of sensitivity to a specific allergen but not whether a patient will have allergy symptoms upon laboratory exposure . In facti.e. true food allergy) are needed to avoid OFCs and reduce false positive diagnoses of FA.The lack of effective IgE-mediated biomarkers for clinical FA  leads toEpigenetic factors have been noted to be a possible means of diagnosing food allergy 13]. Mar. Mar13].a priori biological information, identifying overlapping protein-coding genes and enriching for immunologic genes.Martino et al. 2015  pioneereThis study analyzes the same dataset used by Martino et al. 2015  with theceteris paribus ..ceteris i.e. one that does not use any a priori information), so that the methodology will be applicable to other diseases involving DNAm.Using a purely data-driven method to build a perfect classifier .Weka , a Java-The research was conducted using a dataset found in the Gene Expression Omnibus (GEO)  under acSimilar to Martino et al. 2015 , the 13 Each sample consisted of normalized Methylation levels taken from mononuclear blood cells at 405,658 CpG islands across the genome. These Beta values were features for the machine-learning classifiers in this study.The 58 samples were randomly split into three cohorts: 40 samples for training, 10 samples for cross-validation, and 8 completely hidden samples for testing. Half of the samples in each of the three cohorts were allergic subjects, while the other half were sensitized. To avoid potential bias, eight random folds   were crelimma  . To prevAnalysis ) R packaAnalysis . One or The GEO2R tool produced eight ranked lists (one for each of the eight folds) of differentially methylated CpGs. The top 99 CpGs from each list were combined for an aggregate ranked list of 636 unique CpGs, the count being less than 792 since some of the CpGs overlapped across the eight folds. For each fold, and for each of the 636 unique CpGs from the combined GEO2R list, four different machine learning classifiers were built: a Decision Tree (DT), Logistic Regression Model (LR), Radial Basis Function (RBF), and a Multi-Layer Perceptron (MLP). The perceptron was a deep learning network with an architecture of two hidden layers with ten nodes each. MLPs of other architectures can theoretically be added to the aforementioned four classifiers without changing the methodology, as model selection is not predetermined and instead stipulated by the cross-validation data.Each predictive classifier used only one CpG and was built on the training data. For each fold and each CpG-feature, the classifier  with the highest cross-validation accuracy was selected. Finally, the average hidden test set accuracy across the eight independent folds was computed. This quantity is referred to as the “accuracy score” throughout the rest of the paper. 636 of these accuracy scores (one for each CpG) and 20,352 classifiers (8 independent folds x 636 features x 4 classifiers) were created in total using this process for the single feature case.Sequential forward feature selection (SFS)  was usedThis process was stopped at twelve input-features, as perfect classification across the eight folds was achieved with 12-feature combinations. Adding additional features would increase the complexity of the classifiers, deviating from the objective of finding the simplest models to explain the data.Ensemble methods construct a set of machine learning classifiers and then classify new samples by taking a vote of the predictions from these models . EnsemblWhile there are many ways to create an ensemble of classifiers, a simple majority-scheme method was used in this research. For simple-majority ensembles, odd numbers of classifiers are generally used to avoid the cases of ties  and to eThe final set of CpG signatures were re-validated through testing on a large number of datasets where the samples were repeatedly randomly allocated to the training, cross-validation, and hidden test sample cohorts. As earlier, the number of samples in train-validation-test datasets was kept at 40-10-8, with an equal number of allergy and sensitized samples in each cohort.Gene set analysis can provide biological context as well as insights into disease mechanisms and possible treatments 33]. Bio. Bio33].Gene ontology concepts were used to identify functionally-related gene sets.  The GeneRNF213;LOC100294362, achieved an average accuracy of 84.375% and an average AUROC of 0.84, on the hidden test data across the eight independent folds. This CpG was also identified via the GEO2R analysis process. When 29 or more independent single-feature classifiers were combined through the simple voting scheme, as shown in Dietterich 2000  providesAs shown in For each given number of features, from one to twelve, As shown in As discussed earlier and shown in Interestingly, cg06410630 appeared in the best classifier for each of the feature-sizes from one to twelve. cg10461264, cg06116095, and cg26963090 were the next three most frequently appearing CpGs.a priori information was used in this completely data-driven approach.Taking the top 26 classifiers with 12 features each, including the two with the feature-lists enumerated in RNF213, is the most discriminative CpG found in this study and a strong biomarker of FA. RNF213 has previously been connected to immune response and virus defense 64][[64][63][[[64][63]The 18-CpG list mapped to a novel 13-gene signature that is a strong biomarker of FA. Out of these 13 genes, seven genes overlapped with the genes found by Martino et al. 2015 , while tThe methods used in this study, being completely data-driven, are applicable to other problems that use High Dimension Low Sample Size (HDLSS) data. This methodology can be used with DNAm data to gain new biological insights and create highly-accurate classifiers for diseases such as certain cancers, Asthma, Crohn’s disease, and HIV . The appa priori information, the classifiers used in this study can computationally evolve as new data are added, increasing in accuracy as time progresses. Additionally, the ensemble approach using single-feature classifiers could provide computationally efficient diagnostic classifiers for DNAm data.Because they do not use Since not all of the possible CpG feature combinations have been exhaustively evaluated, it is possible that there is a CpG signature with a smaller number of features that can perfectly classify the examples for this dataset. However, that approach is computationally expensive and may not yield significant additional biological insights.A greater limitation for both this research and future food allergy-related work is the lack of publicly-available datasets and the low number of samples associated with FA. Classifiers generally improve with data, especially when the number of features is large . Having Furthermore, as the data used in this study contained both peanut-allergic and egg-allergic patients, future work should analyze the differences between the DNAm underpinnings for the two allergens. Arasi et al. 2018  also calFuture work should be focused on creating clinical tests for distinguishing between FA and sensitized patients, thus helping avoid misdiagnosis and dangerous OFCs. The genes and pathways highlighted by this research should also be further studied to elucidate the mechanisms and possible treatments of food allergies. This data-driven machine-learning approach opens the door to the computational analysis of other diseases, which may lead to enhanced research and understanding of those ailments.S1 Table(PDF)Click here for additional data file.S2 Table(PDF)Click here for additional data file.S1 Fig(PDF)Click here for additional data file.


Data from discovery proteomic and phosphoproteomic experiments typically include missing values that correspond to proteins that have not been identified in the analyzed sample. Replacing the missing values with random numbers, a process known as “imputation”, avoids apparent infinite fold-change values. However, the procedure comes at a cost: Imputing a large number of missing values has the potential to significantly impact the results of the subsequent differential expression analysis.We propose a method that identifies differentially expressed proteins by ranking their observed changes with respect to the changes observed for other proteins. Missing values are taken into account by this method directly, without the need to impute them. We illustrate the performance of the new method on two distinct datasets and show that it is robust to missing values and, at the same time, provides results that are otherwise similar to those obtained with edgeR which is a state-of-art differential expression analysis method.The new method for the differential expression analysis of proteomic data is available as an easy to use Python package. The recent availability of high-resolution omic measurements has called for the creation of statistical methods and tools to analyze the resulting data –4. Proteimputation: All missing values are replaced by samples from a given distribution , we provide here only a general evaluation of the identified significantly differentially expressed phosphopeptides. Without the loss of generality, we use the data corresponding to four samples measured one hour after irradiation  and four corresponding controls . As explained in “Description of the phosphoproteomic data” section, SA, MT, and YH are three respective mutations that the sample cells can have; labels 1 and 2 mark two biological duplicates that were available for each mutation combination. In the measurement data, there are 6201 peptides that have at least one positive count in the eight aforementioned samples. The dataset contains 31The numbers of DE phosphopeptides identified by respective approaches using the FDR threshold 0.20 are: 1278 for edgeR without imputation, 93 for edgeR with imputation, and 45 for ProtRank. The result obtained with edgeR without imputation is clearly excessive with more than 20% of all peptides being identified as significantly differentially expressed. This is due to the irregular zeros that distort the results; this is shown by 63% of the DE phosphopeptides’ counts being zeros, which is more than double of the overall fraction of zero counts in the data. EdgeR with imputation does not have a similar problem and yields a similar number of differentially expressed phosphopeptides as ProtRank.b as b as b. When the relative representation is more than one, the given count bin b is over-represented among the identified DE phosphopeptides. When the relative representation is less than one, the given count bin b is under-represented among the identified DE phosphopeptides.To gain further insights, we evaluate median counts (computed from the positive counts only) of the identified DE phosphopeptides, in particular in comparison with median counts of all phosphopeptides. Denoting the fraction of the identified DE phosphopeptides in the count bin The result is shown in Fig. To summarize the results obtained on the phosphoproteomic data: ProtRank yields a plausible number of DE phosphopeptides which furthermore show no systemic biases. By contrast, edgeR without imputation produces an excess number of DE phosphopeptides and edgeR with imputation is strongly skewed toward phosphopeptides that have either low or high counts.The complex experimental setup of this dataset allows us to illustrate another ProtRank’s asset: The possibility to simultaneously address all possible differential expression patterns (see “ProtRank description” section for details). In the case of the given dataset, for example, it is possible that the two types of samples (SAMT and SAYH) react to irradiation in a different way: Some phosphopeptides can increase upon irradiation in SAMT samples and decrease upon irradiation in SAYH samples, for example. Besides the baseline comparison of all irradiated samples with their counterpart controls, ProtRank makes it possible to consider two separate groups—SAMT samples and SAYH samples, in this case. The rank score of each gene is then computed in such a way that the genes that consistently increase upon irradiation in both SAMT and SAYH samples, the genes that consistently decrease upon irradiation in both SAMT and SAYH samples, as well as the genes that increase upon irradiation in SAMT and decrease in SAYH samples (and vice versa), are assigned a high rank score. From the 45 genes identified by ProtRank as differentially expressed, 18 increase upon irradiation in both SAMT and SAYH, 18 decrease in both SAMT and SAYH, 6 increase in SAMT and decrease in SAYH, and 3 decrease in SAMT and increase in SAYH. The increase in SAMT and decrease in SAYH combination produces a particularly robust signal with two of the identified genes having FDR<0.01.Similar multi-directional analyses are also possible using other tools—such as edgeR that we use here for comparison—but they need to be manually done separately for each combination of directions, and the obtained results have to be compiled whilst explicitly taking into account that they come from multiple comparisons. ProtRank allows the same analysis to be carried out in two lines .We have shown so far that ProtRank overcomes the problem of missing values without the need to impute them and, at the same time, produces results that are in the absence of irregular zeros to a large extent comparable with results produced by the state-of-art differential analysis tool edgeR . We now With Perseus, we used the transcriptomics dataset which, as we have seen, is essentially free of irregular zeros and EdgeR is therefore expected to produce a meaningful differential expression analysis result. We first log-transformed the counts and then imputed the missing values . While tProteus is designed to use an evidence file from MaxQuant as input and currently lacks the possibility to use a simple table of peptide intensities instead; to apply Proteus on the datasets studied here so far is therefore not possible. Instead, we use the peptide intensities produced by the example described in  in “MissWe have shown that the presence of irregular zeros—missing values that in the differential expression (DE) analysis occur in comparisons with substantial measured values—in proteomics data importantly influences the resulting lists of DE genes produced by common statistical tools such as edgeR. We stress that for other kinds of input data, such as transcriptomics data, edgeR is a good statistical tool that produces high-quality results.There are two main directions that we see for the future development of ProtRank. Firstly, the computationally intensive bootstrap step  could be replaced by an approximate analytical procedure. The first motivation for such an approach is provided by Fig. Secondly, a different recent method, Proteus, addressing the problem of missing values in proteomics  came to We propose here a novel method for differential expression analysis of proteomic and phosphoproteomic data. The main advantage of this new method is that it is robust to the missing values that are common for proteomic and phosphoproteomic measurements. As a result, it does not require the imputation step which is commonly used to eliminate the missing values , yet we https://github.com/8medom/ProtRank as an easy to use Python package.ProtRank requires no parameters to be fine-tuned for the analysis. It also does not employ any normalization of counts in individual samples as this would not change the gene ranking that is taken into account by the method . Importantly, ProtRank makes it possible to automatically address more complex differential expression patterns such as the case discussed in “Description of the transcriptomic data The transcriptomic data from head and neck squamous cell carcinoma patient-derived cell lines have been originally analyzed in . plasmid  . Disulfide bonds were reduced with tris(2-carboxyethyl)phosphine at a final concentration of 5 mM at 37 ∘C for 30 min and alkylation of free thiols was performed with 10 mM iodoacetamide at room temperature for 30 min in the dark. The solution was subsequently diluted with 0.1 M ammonium bicarbonate to a final concentration of 1.5 M urea and digestion was performed overnight at 37 ∘C by sequencing-grade modified trypsin (Promega) at a protein-to-enzyme ratio of 50:1. Acidification was performed by adding formic acid to a final pH<3 in order to stop protein digestion. Peptides were desalted on a C18 Sep-Pak cartridge (Waters) and one-tenth of the resulting eluate was processed individually for total proteome analysis. Phosphopeptides were enriched from 1 mg of initial peptide mass with TiO 2 as previously described  (the lowest rank score is as far from zero as the highest is from one). The rank score σ0 which is the method’s parameter. We set σ0=0.1 which corresponds to assigning the change from zero to a positive count the same score as assigned to a pair of two positive counts with the 10th percentile logarithmic fold change. In general, lower values of σ0 result in a higher fraction of zero counts among the identified differentially expressed genes. The precise choice of σ0 is made less important by Eq.  of the compared samples. Since the change from a zero count to a positive count corresponds to a large positive logarithmic fold change, we assign those pairs uniform rank score t by Eq.  which loTo rank the genes based on all comparisons, we now have to aggregate individual rank scores into a final rank score. Similarly to , 16, thiN=5000 genes in total, the respective rank scores are 0.5/N=10−4 and 1−0.5/N=1−10−4, respectively. Their direct multiplication then yields approximately 10−4 which is the same value as a gene whose logarithmic fold change is 50th largest in both cases . This is obviously an undesired outcome as the two mentioned genes are far from being similarly differentially expressed. The problem is overcome by aggregating according to Eq.  and dn=1 corresponds to searching for genes with large positive logarithmic fold change in group n . The final rank score is then computed as the minimal rank achieved out of all possible group directions,Given the final score obtained with Eq. , genes wdn=1 (seeking for genes upregulated in group n) and to dn=−1 (seeking for genes downregulated in group n). Terms corresponding to comparisons involving two zero counts are again ignored from computing the aggregate score. Genes are then ranked by their final rank score from the highest to the lowest.where the inner-most term simplifies to g that is ranked rg by Eq. .In addition, basic statistical properties of the dataset can be displayed using data_stats function, and the logarithmic fold changes computed for selected rows  can be visualized using plot_lfc function.https://github.com/8medom/ProtRankOperating system: Platform independentProgramming language: PythonOther requirements: Python packages Numpy, Scipy, Pandas, MatplotlibLicense: GNU General Public License v3.0Any restrictions to use by non-academics: NoneProject name: ProtRankProject home page:


A deep learning approach refines the state-of-the-art subtypes of colorectal cancer and evaluates the fidelity of cell lines that model cancer. Colorectal cancer (CRC) is a common cancer with a high mortality rate and a rising incidence rate in the developed world. Molecular profiling techniques have been used to better understand the variability between tumors and disease models such as cell lines. To maximize the translatability and clinical relevance of in vitro studies, the selection of optimal cancer models is imperative. We have developed a deep learning–based method to measure the similarity between CRC tumors and disease models such as cancer cell lines. Our method efficiently leverages multiomics data sets containing copy number alterations, gene expression, and point mutations and learns latent factors that describe data in lower dimensions. These latent factors represent the patterns that are clinically relevant and explain the variability of molecular profiles across tumors and cell lines. Using these, we propose refined CRC subtypes and provide best-matching cell lines to different subtypes. These findings are relevant to patient stratification and selection of cell lines for early-stage drug discovery pipelines, biomarker discovery, and target identification. WNT signaling pathway are thought to initiate the transformation to cancer  accounts for 10% of cancer-related deaths . CRC is o cancer , 5. Thisutations . Since tutations . Based oAF genes . A furthAF genes . The divAF genes , designaAF genes . This leAF genes  Fig S6)WNT signaWe propose a multiomics method that incorporates data on gene expression, copy number, and mutations in identifying CRC subtypes, in a manner that has important implications for patient stratification. Our method is able to match cell lines to each subtype, and in the future, it will be useful in assigning best-matching xenografts or organoid models to the study of each subtype. Patient-derived tumor models have been shown to predict in patients’ response to treatments , and so Genomic assays are high dimensional (tens of thousands of genes), and high-dimensional spaces are challenging to analyze. This problem is further exacerbated when seeking to integrate multiple types of assays from diverse omics platforms. It is, therefore, beneficial to introduce methods which are able to recognize patterns in the molecular signatures spanning different omics data types. Latent factor analysis is an unsupervised learning technique well suited to the task at hand. This type of analysis is sometimes referred to as dimensionality reduction; it seeks to learn (infer) a lower dimension representation of data, which preserves the important structure/pattern therein. The method describes data using a handful of latent factors, rather than tens of thousands of genes, simplifying downstream analysis such as distance calculations and clustering . In the The patterns represented by latent factors should be interpretable in the biological context, that is, it is desirable that the patterns correspond to cellular processes. Matrix factorization has become the workhorse for latent factor analysis for multiomics and general data analysis . Latent K − 1, where K is the number of clusters one expects to find. Recently, a large-scale study of latent factor methods (Preprint) has demonstrated the desirability of specifying more latent factors than are expected to exist in the data. Specifically, it was shown that using more latent factors than are needed can compensate for shortcomings in the training algorithm and improve log-likelihood and recovery of ground truth latent factors. Hence, it is desirable to have a latent factor method that is able to learn a large number of latent factors efficiently from data.Although it is reasonable to expect that the latent factors describing the data be of much lower dimensionality than the genome-scale input data, it is an open question just how low the dimensionality should be. Heuristics to pick the number of latent factors have been proposed by method designers. PCA and MFA typically suggest an elbow method, where latent factors are ordered by their variance explained, and the user determines an elbow point in the graph, discarding latent factors with a low variance explained. MOFA formalizes this heuristic by starting the fitting process with a high number of latent factors, and during training, discarding ones with a variance explained below a preset threshold . iCluster+’s heuristic comes from its k-means roots, that is, one tends to set the number of latent factors to  methods PreprintPreprint), a flexible framework for nonlinear latent factor inference. Our implementation is inspired by methods implemented by ladder-VAE (Preprint) and disentangled autoencoders (β-VAE) (Preprint). A similar method proved successful at stratifying cancers by their tissue type based on gene expression profiles (Preprint), and other autoencoder architectures have been used to integrate multimodal data in robotics  Preprintdder-VAE Preprint (β-VAE) . Using t (β-VAE) Preprintprofiles Preprintrobotics , as wellrobotics .Here, we used a multimodal, stacked VAE to extract latent factors that can be used to identify CRC subtypes and predict patient survival. We call the method, “multiomics autoencoder integration,” or maui. We compared maui performance with state-of-the-art multiomics analysis methods and showed that it outperforms the matrix factorization methods, showing improved clinical relevance. Computationally, maui's is in orders of magnitude more efficient than the state-of-the-art methods we compared it with. In a further step, we map commonly used CRC cell lines to the latent factor space defined by maui. This allows us to use patterns recognized by maui as significant in CRC to rank the suitability of in vitro models for the study of specific tumors. We, then, determined the cancer cell lines most appropriate for drug target studies aimed at distinct CRC subtypes.The CRC cohort in the TCGA data set  has been extensively studied. Statistical analysis supports the use of clinical follow-up data , and a sWe used maui to extract latent factors from data on gene expression, point mutations, and copy number alterations and did the same using MOFA  and iCluing maui., and S2.K to vary from 2 to 9, and for each K, we computed adjusted mutual information (AMI) of the clustering with the CMS labels. k-means clustering only reproduces the CMS subtype to a significant degree for K values of 4–6 only using maui . Supervised learning algorithms such as SVM may benefit from a larger number of input features . We would like to demonstrate that maui’s ability to learn more latent factors  is beneficial in a fair benchmark. To assess which of the methods is best able to capture the CMS labels, regardless of the number of latent factors, we repeated the previous exercise—predicting the CMS from the latent factors—using an unsupervised learning algorithm. We clustered the samples with a well-defined CMS (some CRC samples do not have a CMS) using k-means clustering on the latent factors . We let ing maui  and S1C.ing maui .K’s 4–6 ; this consists mainly of a subset of tumors designated as CMS2   . K-meansnonical)  and S5.P < 0.05 . Using those clinically relevant latent factors, we fitted a multivariate Cox proportional hazards regression and computed Harrell’s c-index  achieves a lower score than a maui model with multiomics data (c = 0.72), even when the former is provided with more genes as input features . This inOne advantage of maui over other methods such as iCluster+ and MOFA is that it is able to learn orders of magnitude for more latent factors, at a fraction of the computation time . To demonetSmooth . While most latent factors are active in the gene expression domain, mutation data do not significantly affect most of them. Some latent factors capture interactions between two or more omics types  latent factors with gene ids, we observed an enrichment of pathways such as Wnt signaling and other APC-mediated processes known to play a role in CRC . One of To check the robustness of maui and ensure we were not overfitting, we ran maui and iCluster+ with a range of parameters, such as the number of latent factors. MOFA uses a heuristic to pick the number of latent factors, starting with a large number, and during training, removing latent factors which explain the variance in the data below a threshold of 2%.P < 0.001, one-sided t test) and MOFA  . We computed the compound benchmark score for each model using the different hyperparameter sets . maui robustly outperforms iCluster+ ( t test) .COLO741, which has been shown to derive from melanoma and not from CRC . This finding indicates the merit of using this method to flag cell lines as poor models for tumors.The molecular profiles of cancer cell lines often differ significantly from those of tumors because of the differences in selective pressures faced by cells in culture and natural tumor microenvironments; adaptation requires distinct genomic alterations . This meK = 5 for the number of nearest neighbors is immaterial, as the method is insensitive to the choice of K , we found that more of the cell lines could be assigned to a “cell line cluster” in which most of their neighbors are other cell lines . For neaice of K . UMAP emice of K . Althougice of K . Rejectiice of K . Using te origin , 37. ThiThe CMS scheme  is incomCRC is a heterogeneous disease, with subtypes that are driven by different types of genomic alterations: hypermutated tumors, tumors showing CIN, etc. Multiomics data analysis has the potential to clarify questions regarding disease subtypes. Taking advantage of this will require new methods which scale computationally as the amount of available data increases. In addition to stratifying patients into clinically relevant subgroups, it is necessary to find potential drug targets specific to each subtype. Most drug target discovery studies use cancer models such as cell lines, organoids, or xenografts, and it is thus necessary to match these cancer models to the appropriate subtype in each study, or if a cancer model is inappropriate for the study of any subtype, to be able to flag it as such.We have developed an autoencoder-based method, called maui, that integrates data from multiomics experiments. We used it to infer latent factors that summarize molecular patterns, from data sets made up of RNA-seq, single nucleotide polymorphisms, and copy number variants (CNVs). The latent factors capture patterns which explain the variation across the different data modalities, capturing important aspects of cancer biology including different gene expression programs, mutational profiles, copy number profiles, and their interactions. We showed that these latent factors are predictive of previously described CRC subtypes (CMS). maui outperforms the methods MOFA and iCluster+, which we benchmarked against in the task of CMS label prediction and in predicting patient survival regardless of the CMS subtypes involved. From the standpoint of computational performance, maui can extract more latent factors from larger data sets at a fraction of the computational cost of both iCluster+ and MOFA, making it better suited for the analysis of increasingly larger data sets. It can leverage its computational efficiency to learn from larger data sets, containing more genes, to produce latent factors which are more predictive of patient survival; using 6,000 genes, maui produces more clinically relevant representations than using 1,300 genes. We also used these latent factors to produce a novel system of classification for CRC. In addition to reproducing the CMS nearly perfectly, it revealed that one of the CMS subtypes, CMS2, is in fact two distinct tumor subtypes. These have different characteristics in terms of the underlying gene expression programs and distinct survival rates. This shows that the prognostic value is improved by an unbiased selection of more input genes, rather than restriction to known markers or driver mutations. Our results support the idea that passenger mutations and driver mutations could have an effect on cancer outcomes .Preprint). By example, ranking latent factors by their clinical relevance , and in the future, with even more data types , we expect the input spaces to grow far beyond 6,000 that were used here. We still recommend that users of maui who are interested in using, for example, DNA methylation data perform some feature selection on the 450,000 or so CpG’s, in a similar way to the feature selection we performed on gene expression, mutation, and copy number data. Hence, the computational efficiency of maui is not a mere academic exercise; at today’s scale, this increase in computational efficiency is the difference between a model that can be fit at all and one which cannot. In addition to using more input features, the computational efficiency allows maui to learn more latent factors than we might believe to truly exist in the data. This is desirable in latent factor models, as fitting more latent factors increases the amount of ground truth factors which are recovered by a method. This effect tends to outweigh any harm that may come from overparameterizing the model Preprintce as in , we haveKRAS mutant tumors  consists mainly of hypermutated tumors with low CIN, and the cell lines we matched with that cluster using maui share their characteristics; such matches have been a standard method in finding disease-specific cell lines , and thiThe CRC subtypes we used as a starting point for this study had previously been defined based solely on gene expression profiles. Our aim to refine these subtype definitions using multiomics data restricted us to a subset of the tumors used in the CMS study. We used only samples from the TCGA which had measurements for gene expression, mutations, and copy numbers (n = 519). The original CMS study used a larger cohort . It is unclear whether our proposal of splitting the CMS2 subtype into two clusters would hold for a larger data set. Only when multiomics data are available, this question will be possible to address.Although the autoencoder architecture of maui is able to perform inference in larger data at a fraction of the time required by matrix factorization methods such as MOFA and iCluster+, it is more challenging to provide a biological interpretation of the model it produces. Linking genes to latent factors is not as straightforward as in matrix factorization. The solution we propose is based on correlations between input genes and latent factor values, heuristically picking those that are most significant. Although we were able to show that such latent factors—gene relationships capture meaningful information about cancer biology and recapitulate known associations between the dysregulation of certain pathways and patient survival, this method is potentially less robust in establishing these associations and might require more user involvement in the analysis pipeline.In this study, we have developed a deep learning–based multiomics integration method (maui) and shown that it can be used to define clinically relevant subtypes of CRC, as well as predict the fitness of cancer cell lines as models for the study of tumors. We found new associations between cell lines and particular CRC subtypes. The latent factors inferred by maui are also interpretable in a biological context and predictive of patient survival, which permits associations to be made between underlying oncogenic processes and patient survival. We benchmarked maui against two state-of-the-art methods for multiomics data integration and showed that not only is it more effective in defining clinically meaningful subtypes but also it does so with superior computational efficiency. An increase in speed of orders of magnitude will permit maui to be used in studies involving larger cohorts and more types of omics data, an experimental trend which will continue to increase in the future. maui’s suitability as a general tool for multiomics integration should also make it useful outside of the context of cancer to explore issues related to in basic biology in studies using multiple genomic assays.https://portal.gdc.cancer.gov) using the TCGAbiolinks R package . mij = 1 if and only if gene i carries a nonsynonymous mutation in sample j. Copy number alterations are GISTIC calls by gene, represented as a real-valued matrix where cij is the GISTIC segment mean for the segment containing gene i in sample j.We obtained data for tumors from the TCGA-COAD (n = 389) and TCGA-READ (n = 130) project designations of the Genomic Data Commons  and are the same data types as the TCGA data, with the exception that transcriptome profiles are reads per kilo base per million mapped reads (RPKM)-normalized and not fragments per kilo base per million mapped reads (FPKM). We considered 54 cancer cell lines originating from the colon.CCLE data were obtained from the CCLE portal (We considered only tumors (from TCGA) and cancer cell lines (from CCLE) which have “complete data”, that is, available measurements in all three assays: gene expression, SNVs, and CNVs.We used gene-wise median absolute deviation statistic, computed directly on the raw data described above, to select the most informative genes. For the comparisons with MOFA and iCluster+, we used the 1,000 genes with the highest median absolute deviation for gene expression, 200 for mutations, and 100 for copy number alterations, for a total of 1,300 input features. We selected the features so strongly to make a comparison against iCluster+ viable, and with a larger feature space, the runtime would become untenable .For the final clustering analysis, we used a larger feature space, with 5,000 gene expression values, 500 mutations, and 500 CNVs for a total of 6,000 features, taking advantage of maui’s neural network architecture which allows for larger feature spaces to undergo feature selection as part of the training.We fit the autoencoder using all TCGA samples, both with and without a CMS label n = 519,  as well All input features were scaled and centered before feeding to the neural network, using batch normalization. Before this scaling, mutation data were binary, CNV data GISTIC calls, and gene expression counts were RPKM/FPKM values which were scaled and centered. TCGA and CCLE gene expression matrices were first scaled and centered individually and, then, concatenated and scaled jointly to filter out the “batch effect” of CCLE versus TCGA data enable mapping of tumors and cell lines to the same space. This means that for a trained maui model, when new, unseen samples are to be mapped onto the latent factor space, they must first be normalized in this way to fit the distribution of the training data.netSmooth , A is the degree-normalized adjacency matrix of the PPI, and (1 − α) is the restart rate. The process is guaranteed to converge and has a closed-form solutionThe algorithm is a simple To pick the optimal α value, we performed a grid search over a range between 0 and 1, and picked the lowest α value within 1 SD of the highest score on the Harrell’s c-index benchmark .xi from different modalities, we call the full multiomics data set Starting from different data matrices x are explained by the variation in a smaller set of latent factors, z. To infer the latent variables We define a generative model er bound :ELBO = EIn the equation above, Preprint) and reparameterize We follow Preprintx to z (the “encoder”), is a neural network which will be trained to compute z, the path from x to The first half of the autoencoder, leading from z’s to their prior distribution.Setting the loss function of the neural network to the negative evidence lower boundThe VAE described above is for a one-layer Bayesian framework, that is, Using the reparametrization trick as above and specifying the full loss function, inference in the generative model .Deep neural networks have many parameters, making them very flexible. This flexibility, however, comes at a cost—deep models are prone to overfitting: the generation of models which explain the training data well, but generalize poorly to new data. In addition, deep nets are prone to producing complex relationships between many variables. In the case of a latent variable model, that means latent factors that change with the variation of any of a large number of input features, a property which makes the task of interpreting the biological meaning of those latent factors difficult. In technical terms, we wish to enforce sparsity in To address the first issue of potential overfitting, we used batch normalization . When fiz’s which are far from the Gaussian prior. Disentangled representations, where latent factors depend on complementary input feature sets, can support this kind of sparsity. This holds when latent factor representations are non-negative, which we achieve by passing them through a rectified linear unit (ReLU). When each non-negative latent factor depends on a different set of inputs, the relationships will be sparser.The second mode of regularization, encouraging representations, where latent factors depend only on a few input features, is partially achieved by the KL term in the loss function, as that penalizes distributions of We, therefore, add a multiplier to the loss function similar to β-VAE  and, allPreprint), where β is initially 0 and is gradually increased by β = β + κ until its value reaches 1.To ensure the network finds a good representation before it starts regularizing, we used the “warm-up” method proposed by Preprinth), Nhidden, and the number of latent factors, Nlatent.The stacked VAE presented above is a class of models which are parameterized by the number of hidden units  and computed a compound benchmark score at each point. The compound benchmark score is the average of the scores of the AUC in the supervised CMS prediction task, the AMI in the unsupervised CMS subtype prediction task, the −log10p of the multivariate log-rank test for differential survival statistics, and the c-index , for Nlatent > 30 ; the same reasoning for iCluster+ resulted in five latent factors. We allowed MOFA to use its own heuristic, discarding latent factors with variance explained below 2%, yielding a 20-component model. We used the MOFA default threshold when picking the number of components to keep in the PCA comparison, which yielded five components.For the final analyses shown in the results section, to avoid leakage of benchmarks into the unsupervised learning algorithms, we ran maui with parameters corresponding the mean of the distribution of compound benchmarks (κ = 0.01. We used the Adam optimizer (We implemented the model using Keras (v2.1.5) using a TensorFlow (v1.6.0) backend. We used rectified linear units for all activations except for the last layer which is sigmoids, for all features. We trained our network for 600 epochs using minibatches of size 100 and ptimizer .To quantify the correspondence between latent factors learned by using different methods and the CMS label, we used SVMs , a superK’s in the range of 2–9. For the cluster assignments for each K, we computed AMI of the clustering with the CMS labels. The AMI is an information-theoretic measure of the concordance between two labelings (clusterings and CMS), which accounts for chance. Higher values indicate closer relationships between labelings.We benchmarked maui against MOFA and iCluster+ in the power of latent factors to predict CMS labels in an unsupervised fashion to present a fair comparison between maui (70 latent factors) and MOFA (20 latent factors) and iCluster+ (10 latent factors). We used k-means clustering, as clustering based on distance metrics suffers from the “curse of dimensionality,” and does not, in general, benefit from a larger number of input dimensions (unlike supervised learning methods). To assess the ability of k-means clusters to capture the CMS labels, we ran k-means with 1,000 starts, picking the best (lowest variance) solution for each run. In addition, we applied the algorithm with The subtyping scheme presented in the results section is based on k-means clustering using maui latent factors learned from multiomics data. We did this using a maui model trained on 6,000 input features , as it is more predictive of patient survival than the one using 1,300 features  and prodz = f(x), where f(x) is a nonlinear function which may not necessarily be well approximated by a linear z ≈ Wx, as in models such as MOFA or iCluster+. The architecture and depth of the neural network also makes it nontrivial to associate the input genomic features  with the different latent factors. However, to make biological sense of the latent factors, it is necessary to make that association. To do that, we computed Spearman’s ρ for each latent factor with each input feature and called a latent factor associated with an input feature if P < 0.001.The stacked variational autoencoder model described above computes latent factors t tests and Benjamini–Hochberg correction for multiple hypothesis testing. Genes with adjusted P value below 0.05 were called differentially expressed. To find out if the genes associated with latent factors . We used pathways (gene sets) defined by the Kyoto Encyclopedia of Genes and Genomes .All survival analysis was performed using the python package P-value 0.05) nonzero coefficients in the individual Cox models. Having selected clinically relevant latent factors from each model , we fitted a full Cox regression using those 2and ran a cross-validated out-of-sample c-index calculation using regularized Cox PH regression, searching for the optimal result among the regularizers 1, 10, 100, 1000, 10000. The results reported in To compare maui to MOFA and iCluster+ (as well as to a gene expression only–based maui model), we used Harrell’s C  in a CoxK computing the true positive rate , that is, K’s.To assess the fitness of different cancer cell lines as models for tumors, we computed the pairwise Euclidean distance between each of the samples (TCGA and CCLE), in the space of the latent factors derived from maui. Then, we computed, for each cell line, the proportion of its five nearest neighbors which are also cell lines, the working hypothesis being that cell lines that form “cell line clusters” are more cell-line like than tumor like, and likely less fit as models for tumors. We repeated the exercise considering other numbers of nearest neighbors from 1 to 20, at each https://github.com/BIMSBbioinfo/maui.maui is available as a general purpose python package for the study of multiomics data. For more information, visit


We highlight the use of ML Repo in several use cases to demonstrate its wide application, and we expect it to be an important resource for algorithm developers.The use of machine learning in high-dimensional biological applications, such as the human microbiome, has grown exponentially in recent years, but algorithm developers often lack the domain expertise required for interpretation and curation of the heterogeneous microbiome datasets. We present Microbiome Learning Repo (ML Repo, available at  Machine learning is widely used as a method for classification and prediction, with a growing number of applications in human health . The usemlrepo-source branch of the GitHub repository, under preprocessing/make.mappings.r. As a result, for each dataset we generated RefSeq-based OTU and taxa abundance counts, and Greengenes-based OTU and taxa abundance counts. These counts are presented in tables that are organized as follows: OTUs or taxa as rows, and samples as columns. OTUs are represented as either NCBI genome identifiers or Greengenes identifiers. Taxa are represented as “kingdom; phylum; class; order; family; genus; species; strain," with highest taxonomic specificity where possible. We excluded additional post-processing filtering and normalization steps so that these parameters can be included in future benchmarking use cases as needed. We also limit our data to OTU and taxa tables because other metrics such as α and β diversity can be subsequently generated as needed.We present the Microbiome Learning Repo (ML Repo), a repository of 33 curated classification and regression tasks involving human microbiome data. Our 33 tasks are derived from 15 publicly available human microbiome datasets, which include 12 amplicon-based and 3 shotgun sequencing datasets Table . We downmlrepo-source branch of the GitHub repository, under preprocessing/make.mappings.r. Studies that were cross-sectional by design but contained several samples per subject were filtered to contain 1 sample per subject. In study designs with paired diseased-healthy or pre- and post-intervention samples, samples were reduced to 2 samples per subject with subject identifiers provided as confounder variables. Hence, each prediction task is made available as an individual, compartmentalized metadata file that contains sample identifiers, responses to predict, and optionally, confounder variables that are inherent to the research study design such as paired healthy and diseased samples from the same subject (see Methods for more details). As a result, we generated 33 distinct tasks for predicting human-associated responses.Sample metadata from individual studies were manually curated to generate viable prediction tasks. When available, published study exclusion criteria, such as reported use of antibiotics, were applied accordingly and confounders were removed by dropping samples or stratification. Well-known confounders were accounted for when constructing prediction tasks for other human-associated conditions; e.g., predicting age using the Yatsunenko 2012 dataset is restricted to samples from the USA owing to the known variation in gut microbiomes across different geographical locations. Details of how samples were subset for each prediction task can be found in the We expect 2 types of users: (i) machine learning algorithm developers with limited knowledge of microbiome study designs and (ii) microbiome researchers interested in obtaining additional datasets for meta-analysis. Generally, we expect that method developers will be most interested in sweeping through the full set of prediction tasks for benchmarking, and hence would prefer to download a single compressed file containing all tasks and data. On the other hand, we expect microbiome researchers to be more selective in downloading specific datasets and tasks depending on their research domain. Hence, researchers may prefer to browse specific details about tasks and datasets prior to downloading.On the basis of these expected use cases, we created a publicly available web-based interface for ML Repo hosted by GitHub Pages . Tasks aWe expect ML Repo to be beneficial for both the machine learning community as well as the microbiome research community. ML Repo will be a powerful complement to the University of California Irvine's Machine Learning Repository because it will allow for benchmarking curated classification tasks with high-dimensional data and hence enable the subsequent development of novel algorithms for these complex datasets. Our streamlined approach in generating OTU and taxa tables offers a rich set of 15 datasets that microbiome researchers can use directly for further comparison with their own studies, for teaching and learning purposes, or for large meta-analyses. We expect that our provided OTU and taxa tables will also be beneficial for researchers with limited access to high-performance computing resources or bioinformatics skills necessary for processing raw sequencing data. In addition, we also expect microbiome-specific methods development to benefit from our repository. The subset of samples found in each prediction task metadata file replace the work of rigorously deciphering metadata and understanding the subtle differences of individual research studies. New methods, such as OTU-picking algorithms, can be evaluated not only on metrics such as speed and accuracy but also based on overall impact to study findings.Although a number of microbiome repositories exist, many are intended as data archival repositories , 19 or fP = 0.0014) and with a radial kernel (P = 0.00032)  and SVM-Radial (P = 0.03)  Fig. . We foun03) Fig. , which mTo assess the impact of reference database choice on classification accuracies, we also used the classification tasks to compare random forest using OTUs picked with the Greengenes 97 database or the NCBI RefSeq Targeted Loci Project 16s project. We found that there was limited impact of database choice on overall classification accuracies Figs  and 5. TWe expect and hope that the broader microbiome research community will add new datasets and prediction tasks to ML Repo. We provide instructions  on our GWhen available, preprocessed FASTA files were downloaded from QIITA . For all other datasets, raw FASTQ files were downloaded from sources listed in Supplemental Table 1. Adaptors and barcodes were removed and sequences were quality filtered (at Phred score ≥ Q20) using SHI7  or QIIMEClassification tasks were selected on the basis of reported study results, biologically relevant high-level phenotypes, and sufficient sample sizes. Original metadata files and research methods were rigorously and manually curated in order to subset samples with minimal confounders. For confounders that were inherent to the study, we include an additional variable to control for in the task metadata files. The presence of control variables can be found by examining “control_vars” in the Tasks table.Website templating was developed using Jinja2  and custexample folder in the ML Repo Github repository. To compare machine learning models, we iterated through tasks with binary responses. OTU counts were converted to relative abundances, filtered at a minimum of 10% prevalence across samples, and collapsed at a complete-linkage correlation of 95% . We then constructed a 5-fold cross-validation for tasks containing >100 samples, or a leave-one-out cross-validation for tasks with fewer samples. For n-fold cross validation, samples were assigned to folds such that classes were equally balanced within each fold . For tasks that contained control variables, we selected folds such that samples with the same control variable value were contained within the same fold. For example, for a task dataset containing matching stool and oral samples from subjects, the Subject Identifier would be listed as the control variable and we should assign samples to folds such that all samples from a specific subject were contained within a fold. This step is crucial to avoid biasing or overfitting the training model; test folds should contain not only new samples but also samples that are independent from those in the training set. Models were constructed using the “caret” package [p), where p is the number of features. This entire process was bootstrapped 100 times, and the mean class probabilities were used to calculate the resulting AUCs and ROCs. To compare classification accuracies using different reference databases, we used a similar procedure but held the model constant and predicted using different base OTU tables. This framework enables comparison of a myriad of machine learning models available in the “caret” package and can be easily expanded to compare different OTU-picking algorithms, or normalization and filtering techniques.Case study results were generated with custom R  scripts, package . ControlGigaScience database, GigaDB [All test datasets are available in the Microbiome Learning Repo site . Snapsho, GigaDB .Project name: Microbiome Learning Repohttps://knights-lab.github.io/MLRepo/Project home page: Operating system: platform independentProgramming language: Python, RLicense: MIT LicenseRestrictions: NoneRRID:SCR_017079GIGA-D-18-00316_Original_Submission.pdfClick here for additional data file.GIGA-D-18-00316_Revision_1.pdfClick here for additional data file.Response_to_Reviewer_Comments_Original_Submission.pdfClick here for additional data file.Reviewer_1_Report_Original_Submission -- Pierre Mahe9/20/2018 ReviewedClick here for additional data file.Reviewer_2_Report_Original_Submission -- Edoardo Pasolli9/21/2018 ReviewedClick here for additional data file.Reviewer_2_Report_Revision_1 -- Edoardo Pasolli3/5/2019 ReviewedClick here for additional data file.Reviewer_3_Report_Original_Submission -- Robert Rentzsch, Ph.D.9/21/2018 ReviewedClick here for additional data file.Reviewer_3_Report_Revision_1 -- Robert Rentzsch, Ph.D.2/28/2019 ReviewedClick here for additional data file.AUC: area under the curve; IBD: inflammatory bowel disease; ML Repo: Microbiome Learning Repo; NCBI: National Center for Biotechnology Information; OTU: operational taxonomic unit; ROC: receiver operating curve; SVM: support vector machine.D.K. serves as CEO and holds equity in CoreBiome, a company involved in the commercialization of microbiome analysis. The University of Minnesota also has financial interests in CoreBiome under the terms of a license agreement with CoreBiome. These interests have been reviewed and managed by the University of Minnesota in accordance with its Conflict-of-Interest policies.This work is supported by funds from National Institutes of Health grant R01AI121383.Conceptualization: P.V. and D.K.; data curation: P.V.; formal analyses: P.V.; methodology: P.V., B.M.H., D.K.; software: P.V.; writing—original draft: P.V.; writing—review and editing: B.M.H. and D.K.