Cancer is a complex disease that deregulates cellular functions at various molecular levels . Integrated multi-omics analysis of data from these levels is necessary to understand the aberrant cellular functions accountable for cancer and its development. In recent years, Deep Learning (DL) approaches have become a useful tool in integrated multi-omics analysis of cancer data. However, high dimensional multi-omics data are generally imbalanced with too many molecular features and relatively few patient samples. This imbalance makes a DL based integrated multi-omics analysis difficult. DL-based dimensionality reduction technique, including variational autoencoder (VAE), is a potential solution to balance high dimensional multi-omics data. However, there are few VAE-based integrated multi-omics analyses, and they are limited to pancancer. In this work, we did an integrated multi-omics analysis of ovarian cancer using the compressed features learned through VAE and an improved version of VAE, namely Maximum Mean Discrepancy VAE (MMD-VAE). First, we designed and developed a DL architecture for VAE and MMD-VAE. Then we used the architecture for mono-omics, integrated di-omics and tri-omics data analysis of ovarian cancer through cancer samples identification, molecular subtypes clustering and classification, and survival analysis. The results show that MMD-VAE and VAE-based compressed features can respectively classify the transcriptional subtypes of the TCGA datasets with an accuracy in the range of 93.2-95.5% and 87.1-95.7%. Also, survival analysis results show that VAE and MMD-VAE based compressed representation of omics data can be used in cancer prognosis. Based on the results, we can conclude that (i) VAE and MMD-VAE outperform existing dimensionality reduction techniques, (ii) integrated multi-omics analyses perform better or similar compared to their mono-omics counterparts, and (iii) MMD-VAE performs better than VAE in most omics dataset. Ovarian cancers are generally diagnosed at an advanced age as the early-stage disease is usually asymptomatic, and symptoms of the late-stage disease are nonspecific3. The anatomical location and the ovaries’ position are mainly responsible for asymptomatic and nonspecific nature of the disease. Since the ovaries have limited interference with the surrounding structures, ovarian cancer is hard to detect until the ovarian mass is significant, or metastatic disease supervenes. Due to the symptoms’ nonspecific nature, it often requires multiple consultations with a primary care physician and several investigations in finding the disease and for an appropriate therapy/treatment. In this context, the development of useful tools to better understand the complex pathogenesis is needed for effective cancer management and prognosis5.Ovarian cancer is a common and deadly gynaecological cancer with a high mortality rate in developed countries. It accounts for 5% of all cancer deaths in females in the UK8. Any study limited to any of these levels is insufficient to understand the complex pathogenesis of cancer. Integrated multi-omics analysis of data from these levels is essential to understand cellular malfunctions responsible for cancer and its progression holistically. Importantly, integrated multi-omics analysis, taking the advantage of various omic technologies , can identify reliable and precise biomarkers for diagnosis, treatment stratification and prognosis9. Recent advancements of omics and computational technologies, including deep learning, have boosted the research in integrated multi-omics analysis for precision medicine and cancer. In recent years, many research works12 have been published on integrated multi-omics analysis of cancer. These works are either on individual cancer18 or pancancer22. Most of these works integrated di-omics16, few of them integrated tri-omics23, and very few of them integrated tetra-omics24 data. There are a few integrated multi-omics analyses of ovarian cancer, including di-omics25 and tri-omics27 based analyses.Cancer is a complex and heterogeneous disease that deregulates cellular functions in different molecular levels, including DNA, RNA, proteins and metabolites. Importantly, molecules from different levels are mutually associated in reprogramming the cellular functions29 are becoming very useful and effective in integrated multi-omics analysis of cancer data. However, high dimensional omics data are normally imbalanced with a large number of molecular features and a relatively small number of available samples with clinical labels29. For example, DNA methylation and mRNA integrated dataset (TCGA) for ovarian cancer has 39,622 features  for common 481 samples with clinical labels. This imbalanced dimensionality of cancer datasets makes it challenging to use a machine learning (ML) or DL in integrated multi-omics analysis, especially in individual cancers as they have few samples. For instance, TCGA ovarian cancer RNAseq dataset has only 308 samples, whereas pancancer RNAseq dataset has 9081 samples 29. Algorithms for dimensionality reduction, such as autoencoder-based DL algorithms18, together with conventional solutions, such as principal component analysis (PCA)30, are possible solutions to the dimensionality problem. Importantly, considering the discontinuous and non-generative nature of traditional autoencoders, VAEs31 have emerged as DL-based generative models for compressed features learning or dimensionality reduction. There are many works which have used VAE in their studies. However, they are mostly mono-omics studies of individual cancer33 or pancancer35. OmiVAE29 is the only work that considered VAE for integrated multi-omics (di-omics) analysis of pancancer. Also, VAE may suffer in representing the input features due to uninformative compressed features and variance over-estimation in feature space36. None of the existing works has used MMD-VAE, and also their analyses are limited to cancer molecular subtypes classification only.Due to the increasing availability of large-scale multi-omics cancer data, machine learning approaches, especially DL approaches35 use unsupervised dimensionality reduction methods, separating the downstream analysis from the reduction method. However, dimensionality reduction in cancer multi-omics analysis is an intermediate step toward the downstream analysis, like classification . Separating the dimensionality reduction and model  learning may not be optimal for classification as datasets are not always suitable for a classification task. For example, a DNA methylation dataset that includes cancer and normal samples is readily applicable for classification due to their discriminative features. However, a DNA methylation dataset that only includes cancer samples may not be useful in classification. Due to lack of supervision during a dimensionality reduction process, some key features can be filtered before training the classifier, affecting the final performance38. In this context, supervised dimensionality reduction methods can be more useful in balancing multi-omics datasets and their integrated downstream analysis.Moreover, most of the existing works36. To the best of our knowledge, this is the first work that does a comprehensive mono-omics and integrated multi-omics  analysis of individual cancer (ovarian) using VAE or MMD-VAE, or both. The objectives of this work are three-fold. First, we have designed and developed a DL architecture of VAE and MMD-VAE that supports unsupervised and supervised learning of latent features from mono-omics, di-omics and tri-omics data. Second, we did a dimensionality reduction performance analysis of the developed DL architecture on ovarian cancer by cancer samples identification, molecular subtypes clustering and classification. As a dimensionality reduction technique, the performance of MMD-VAE or VAE depends on input features and sample size, not on the cancer type. Hence, we have tested the developed MMD-VAE and VAE using datasets with three different sample sizes  to demonstrate our findings’ robustness. Finally, a survival analysis of an existing ovarian cancer dataset has been carried out using the reduced or latent features sets.In this work, we did an integrated multi-omics analysis of ovarian cancer using VAE and MMD-VAEIn the following, we briefly discuss the datasets used, data preprocessing, VAE/MMD-VAE architecture, dimensionality reduction and survival analysis methods.39, one for mRNA, CNV/CNA and RNAseq, and two for DNA methylation. All the mono-omics datasets except the second DNA methylation dataset are from the TCGA Ovarian Cancer (OV) cohort43. The second DNA methylation dataset is from GDC TCGA Ovarian Cancer (OV) cohort44. It includes cancer and normal samples. We have concatenated these mono-omics data to form the di-omics and tri-omics datasets. The table’s ’Feature dimension’ and ’Sample size’ columns demonstrate that all the datasets are imbalanced with too many input features and relatively too few numbers of samples with clinical labels. For example, one tri-omics (CNV + DNA methylation + RNAseq) dataset has 72,885 input features with only 292 samples.We used mono-omics and multi-omics  data for the study. We generated multi-omics data using different combinations of high dimensional mono-omics data. Table Step 1 First, we intersected the mono-omics datasets to find the common and same size samples. We did two different intersections of the datasets  in Supplementary documents) using common sample IDs  to keep the maximum number of samples for the study. The intersection of CNVs, mRNA and DNA methylation datasets has found 481 samples, and the intersection of CNVs, DNA methylation and RNAseq has found 292 samples in common within the datasets.Step 2 We identified and removed the missing/zero/NA values in the four downloaded omics files. All the data files, except the RNAseq, had no missing/zero/NA values, and 212 input features or genes  with zero expression values were removed from the RNAseq dataset.Step 3 Non-normalised datasets, such as CNVs and RNAseq datasets, were normalised using the min-max technique. We used the min-max normalisation as unlike other techniques  it guarantees multi-omics features will have the same scale45. Thus, all the features will have equal importance in the multi-omics analysis.Step 4 Finally, we concatenated the normalised mono-omics datasets to form the di- and tri-omics datasets. Concatenations involving RNAseq, such as The second DNA methylation44 dataset is highly imbalanced  as it has only 10 normal samples compared to the 603 cancer samples (class ratio: 1.36:98.64). We used the Borderline-SMOTE SVM46 to reduce the class imbalance  by re-sampling of normal samples (10 to 283). After the re-sampling, we have 886 samples compared to the original 613 samples. This has increased class ratio (31.94:68.06) between the normal and cancer samples.The downloaded datasets are not ready  to be used in dimensionality reduction and integrated multi-omics analysis. They need to be preprocessed, such as the datasets’ sample sizes need to be same to integrate and generate di- and tri-omics datasets. We have preprocessed the downloaded datasets (TCGA Ovarian Cancer cohort) using a four steps method .31 is a deep generative model, which can learn meaningful data manifold from high dimensional input data. Unlike, standard autoencoders , a VAE encodes an input : an encoder encodes or generate each latent variable Step 2 (sampling): a sampler samples points from the latent space by sampling from the encoded or encoding distribution 47 for the regularisation term and jointly optimises the encoder and decoder using the following loss function that rely on the traditional evidence lower bound (ELBO) criterion: Step 3 (decoding) : a decoder decodes the sampled points from a conditional distribution Step 4 (backpropagation): finally, the calculated loss is backpropagated through the network to update the model accordingly.A VAE36:49. KL divergence naturally encourages the latent code x, making the code uninformative/unaware about the input. In this context, the encoder could fail to learn any meaningful latent representation of the input.Uninformative latent code/feature: The regularisation term  in loss function instead of the 50 can address the above issues. According to the MMD, two distributions are identical if and only if all of their moments are same. Unlike p(z) and q(z) are. We can use the kernel embedding trick to estimate MMD for two distributions as Eq.  with an input layer with nodes equal to the LFs (32/64/128), a hidden layer with nodes equal to the half of LFs, and an output layer with nodes equal to the class numbers .The encoder network comprised of an input layer and three hidden layers. The decoder network structure is the mirror image of the encoder structure. Notably, the encoder and decoder share the necessary bottleneck layers. We used the architecture for mono-omics, di- and tri-omics data, and the size of the bottleneck layers is same for all the datasets. However, the other layers’ sizes varied according to the omic count  and omic data. For example, for mono-omics data, such as mRNA data, the input and output layers are 12,043, and hidden layers sizes are 2048 and 1024. As shown in Fig. z given input sample x or simply N) in the bottleneck layer to make the sampling process differentiable and suitable for backpropagation. The sampled latent features vector (z/LFs) is the compressed lower-dimensional representation of omics or integrated multi-omics data.Encoder: The encoder network using two hidden layers encodes mono-omics data into a 1024 dimensional vector, di-omics data into two 1024 dimensional vectors and tri-omics data into three 1024 dimensional vectors. The encoding network for the DNA methylation data is different from the other omics data. For example, in the first hidden layer, each chromosome related DNA methylation data are encoded into corresponding vectors with 256 dimensions wheres for the others, input data are encoded into a 2048 dimensional vector. This encoding is to capture the intra-chromosome relationships, and second hidden layer for the DNA methylation data captures the inter-chromosome relationships. For di- and tri-omics data, the second hidden layer respectively concatenates two and three 1024 dimensional vectors and produces an encoded 512-dimensional vector. The encoder’s final hidden layer fully connects to two output layers. These two layers of the size of latent code or features (32/64/128) are part of the bottleneck layers and represent the mean z as the input and passes through three hidden layers, and finally outputs the reconstructed vector M is the number of chomosomes, CE is the binary cross-entropy between input data , which is: nll- negative log likelihood which can be calculated as mean of N, which is: 29. The binary cross-entropy based classification loss (Decoder: The decoder network takes the latent feature vector ng Eqs.   and 6)  z as theentclass1pt{minimaWe used a batch normalisation technique in each fully connected block to implement the VAE/MMD-VAE DL architecture. This is to address the internal covariate shift   by normalising layer inputs52. Thus, it stabilises the learning process and significantly improves the learning speed. As the activation function, we used the rectified linear units (ReLU) for the hidden layers, the sigmoid for the decoder’s output layer and the softmax for classifiers’ output layer. We built the model using PyTorch (version 1.5.0). The implementations of the models used in this paper are available on GitHub (https://github.com/hiraz/MMD-VAE4Omics).The VAE/MMD-VAE architecture does all the activities illustrated in Fig. 53. Hence, correct clustering and classification of ovarian cancer samples and molecular subtypes are important for improved disease management. Authors in53 have identified four ovarian cancer transcriptional ) subtypes, which may have clinical significance. These four subtypes of high grade serous ovarian cancer (HGS-OvCa) are named as Immunoreactive, Differentiated, Proliferative and Mesenchymal54. The datasets used in this work are about HGS-OvCa, and the clinical data include these molecular subtypes for most of the samples. Although these molecular subtypes are transcriptional , they can be used for other omics data analysis due to their correlation or association with transcriptional data58. For example, authors in55 have reported that DNA methylation is often negatively associated with gene expression in promoter regions, while DNA methylation is often positively associated with gene expression in gene bodies.Cancer samples identification and molecular subtypes are useful in prognostic and therapeutic stratification of patients and improved management of cancersz or LFs) can be used to cluster and classify cancer samples, subtypes, including existing transcriptional or molecular subtypes of ovarian cancer. The performance of clustering and classification exploiting z can demonstrate the dimensionality reduction capability of VAE or MMD-VAE. We demonstrated the dimensionality reduction capability of VAE and MMD-VAE using the latent features learned from the mono-omics, integrated di-and tri-omics data of ovarian cancer, and used for the followings:Clustering We can use the LFs learned (unsupervised and supervised) by the VAE/MMD-VAE models to cluster samples into cancer vs normal and molecular subtypes. We used a two- and three-dimensional embedding of the mono-omics, di- and tri-omics features for the selected samples, and visualised the clustered samples using scatter plots. Two dimensional (2D) and three dimensional (3D) embedding of the omic(s) features for the selected samples are accomplished by selecting first 2 and 3 LFs from the learned LFs  and MMD-VAE/VAE performs in molecular subtypes clustering.VAE or MMD-VAE generated latent and compressed features  LFs inferring and predicting survival subgroups of HGS-OvCa. We followed a 3-step process . NbClust can calculate up to 30 indices or metrics to determine the optimal number of clusters in a data set. It also identifies the best value for K by the majority rule. In our all 11 datasets  model for each of the LFs produced by the VAE/MMD-VAE  and identify the best model for a training dataset. In each round of the validation, 60% data were used for training and rest 40% were left out from the training as the test dataset. Finally, we used the test dataset to predict the survival subgroup or risk labels. We used the Cox-PH model and Kaplan-Meier (KM) survival curves to evaluate survival prediction performance. We used the following three metrics for the evaluation:Predicting survival group labels for new samples: After having the survival subgroups labels from K-means clustering, we used an SVM-based classifier . We then predicted survival using the labels of the test set. We then computed the C-index using concordance function of R’s survival package64. Similarly, we calculated the C-index only considering the clinical features .P value of Cox-PH regression The Cox-PH models built on training datasets compute log-rank p values for the models. Also, we plotted the Kaplan-Meier survival curves of the two survival subgroups (predicted) and calculated the log-rank p-value of the survival difference between them.Brier score Brier score function measures the accuracy of probabilistic prediction65. In survival analysis, it measures the mean of the difference between the observed and the estimated survival beyond a certain time66. The score ranges between 0 and 1, and a smaller score indicates better accuracy of the prediction. We used the R Package survAUC67 to compute the Brier score.Identifying prognostic biomarkers from LFs: We did all the above analyses using the LFs as their compressed representations simplify the survival analysis and molecular subtyping. However, we need to map these LFs back to their corresponding input features to identify potential molecular biomarkers. We mapped the associated input features for each clinically relevant LF using a linear model and filtered the features with zero or insignificant input feature values. Next, we estimated the correlation between the CRLFs and their corresponding input features. Finally, we used the filtered correlation data for hierarchical clustering (colour map) of LFs and their input features.Identification of robust survival subgroups of ovarian cancer (HGS-OvCa) can significantly improve patient care. Existing molecular subtypes of HGS-OvCa, such as transcriptional molecular subtypesWe used the developed DL architecture of VAE/MMD-VAE for cancer samples identification, molecular subtypes clustering and classification, and survival analysis using the TCGA ovarian cancer datasets. The results demonstrate the performance of the VAE and MMD-VAE in dimensionality reduction and survival analysis.68. We reported the results only for We trained and tested the developed VAE and MMD-VAE models with three different bottleneck layers . We have selected the first 2 LFs of the 128 LFs to cluster the samples into two groups . The two-dimensional embedding of the DNA methylation dataset’s input features was plotted on scatter plots for PCA, t-SNE, VAE and MMD-VAE. As illustrated in Fig. Molecular subtypes clustering: We clustered the transcriptional subtypes using the LFs learned through unsupervised and supervised VAE and MMD-VAE models. For the LFs learned via unsupervised model, we have selected the first 2 LFs of the learned 128 LFs to cluster the molecular subtypes. The two-dimensional embedding of the mono omic, di-omics and tri-omics datasets’ input features were plotted on scatter plots for PCA, t-SNE, VAE and MMD-VAE. Figure 9 in Supplementary documents presents the results of 2 LFs-based molecular subtypes clustering. As seen in Fig. 9b–i in Supplementary documents, all the dimensionality reduction methods poorly clustered the samples into four subtypes using the mono- and tri-omics datasets. This result is expected as the original omics datasets are not discriminative or well representative of the transcriptional subtypes. As Fig. 9a in Supplementary documents illustrates, even the most relevant transcriptional dataset (mRNA) do not represent the transcriptional subtypes. Hence, the unsupervised PCA, t-SNE, VAE and MMD-VAE models struggle to cluster the transcriptional subtypes. In this context, we can use the supervised versions of these models, especially VAE and MMD-VAE. We used the supervised VAE and MMD-VAE models to learn the task-oriented  or guided LFs from the mono-, di- and tri-omics datasets. We have selected the first 2 LFs of the learned 128 LFs to cluster the molecular subtypes. Figure 58. For the same reason, the use of integrated non-transcriptional and transcriptional data helps to maintain a similar performance or improve the performances of the transcriptional subtypes clustering and classification. For example, the accuracy of MMD-VAE using mRNA is 93.8%, which has been maintained (93.7%) in case of the integrated CNV-mRNA, and improved to 95.5% in case of the integrated CNV-mRNA-methylation datasets. The confusion matrices in Fig. 10b,d,f in Supplementary documents for MMD-VAE illustrate similar results. As a dimensionality reduction algorithm, in majority datasets, MMD-VAE shows better performance compared to VAE. This performance difference could be due to the MMD-based loss function. We tested the supervised MMD-VAE and VAE on omics datasets with three different sample sizes  to demonstrate our findings’ robustness. We used the learned LFs to classify the transcriptional subtypes and presented the classification accuracies in Table 5 in Supplementary documents. As seen in the Table, both VAE and MMD-VAE show robust classification accuracy for the same omics data with different sample sizes.Cancer samples identification: We used an SVM-based classifier to identify the cancer samples from the normal samples using the LFs learned through the unsupervised PCA, t-SNE, VAE and MMD-VAE. Table 3 in Supplementary documents presents the classification performances for the DNA methylation dataset of 886 samples (GDC cohort). All the models except t-SNE have more than 99% classification accuracy with very high precision (0.99), recall (0.99) and f1 score (0.99). The discriminative features  of the DNA methylation data is the main reason for this classification performance. Molecular subtypes classification: Like the transcriptional subtypes clustering, we used the LFs learned through the unsupervised and supervised VAE and MMD-VAE models in molecular subtypes classification. For the unsupervised setting, we also compared the classification performance of the LFs learned through VAE and MMD-VAE with of the LFs learned through PCA and t-SNE. Table 4 in Supplementary documents presents the classification performance of an ANN-based classifier utilising the LFs learned via these unsupervised models from mono-, di- and tri-omics datasets. As we can see from the table, the classifier using the PCA and t-SNE generated LFs poorly classify the existing transcriptional subtypes in all omics datasets. On the other hand, the classifier using the VAE and MMD-VAE generated LFs can classify the transcriptional subtypes for mono-omics (mainly mRNA), di- and tri-omics data with higher accuracies in the range of 73.2–81.44%. However, the performances may not be acceptable in many real-life applications. Lack of discriminative features within the omics datasets for the transcriptional subtypes is the main reason for the low accuracies. Supervised learning of the LFs can improve the classification performance. In the supervised setting, VAE or MMD-VAE and the classifier jointly learn the LFs using the transcriptional subtypes as the supervisory guidance. We trained the joint models on the mono-, di- and tri-omics datasets and tested the models separately. Table The dimensionality reduction performance results of VAE and MMD-VAE in clustering and classification demonstrate the followings:in any downstream analysis  unsupervised dimensionality reduction is useful if the input dataset is discriminative , otherwise supervised dimensionality reduction is necessary, andintegrated dimensionality reduction and multi-omics analysis of data may improve or maintain the similar performance of their mono-omics counterparts exploiting their association, without confounding each other.We did a comprehensive survival analysis using eleven datasets, including mono-omics and multi-omics data, particularly for the samples with existing transcriptional subtypes and inferred survival/risk groups. Considering the space limitation, we presented a subset but enough of the results . We did a two-stage survival analysis of the samples (481 and 292) using the two inferred subgroups. In the first stage, we plotted Kaplan-Meier survival curves for all the samples. As seen in the Kaplan-Meier survival curves . Potential reasons for the higher p values or insignificant differences between the predicted survival groups for the datasets could be (i) the smaller sample size with few events to identify the differences and (ii) too much compression may have obscured the clinically relevant features.In the second stage, we predicted survival subgroup labels using an SVM-based classifier splitting the samples into training and test data using a 60/40 split ratio. After predicting survival groups for the test datasets, we ran two multivariate Cox-PH regressions  on the training samples, then predicted survival using the labels of the test datasets. For the clinical co-variates, we considered three clinicopathological characteristics of the considered patients: (i) age at diagnosis, (ii) clinical or FIGO stage, and (iii) grade. We calculated C-indexes, Brier scores and models’ p-values for the training and held-out test samples for the multivariate Cox-PH regressions. As seen in Table 7 in Supplementary documents, the training samples generated moderately high C-indexes in between Impact of inferred subgroups on survival: In case of mono-omics and multi-omics data, inferred subgroups combined with the clinical covariates  improve the survival prediction. For example, subgroups learned using the MMD-VAE on CNV mono-omics data combined with clinical features shows higher C-index value (0.63) than the C-index value (0.62) for the clinical features. However, the improvement is not that significant. In summary, inferred subgroups, combined with the clinical features, demonstrate survival prediction performance higher than or similar to the clinical features.mono-omics vs multi-omics based LFs in survival: Similar to molecular subtypes classification, multi-omics based LFs may incrementally  improve the survival subgroups classification accuracy compared to mono-omics based LFs (shown in Table 6 in Supplementary documents). However, multi-omics  based LFs and subgroups predicted based on them do not improve the survival prediction performances than their mono-omics counterparts. Potential reasons for the lower performance using multi-omics based LFs could be due to (i) smaller sample size with few events  to identify the differences, (ii) too much compression  may have obscured the clinically relevant features.Finally, we presented a simple method to identify potential prognostic biomarkers from the clinically relevant LFs using a linear model. Figure 69. Thus, we can identify the other responsible gene or cgp island that could jointly work as a multi-omics prognostic biomarker. However, the linear mapping is not the best solution to identify a set of biomarkers from the learned CRLFs as it is only identifying correlations values within From Fig. Integrated multi-omics data analysis of cancer is vital for a comprehensive understanding of cellular functions, responsible for cancer and its development. DL algorithms have become a useful and popular tool for integrated multi-omics analysis of cancer data in recent years. However, imbalanced high dimensional multi-omics data makes a DL based integrated multi-omics analysis difficult, especially in individual cancer. DL based dimensionality reduction techniques, such as variational autoencoder is a potential solution to this dimensionality issue. In this work, we have designed and developed VAE and MMD-VAE based dimensionality reduction techniques for mono-omics and multi-omics data analysis of cancer. These techniques can learn latent features from any omics data using unsupervised and supervised training process. Then, we used the developed techniques to learn latent features from the mono-omics, di-omics and tri-omics datasets of ovarian cancer. Finally, we used the learned latent features to analyse ovarian cancer through cancer samples identification, molecular subtypes clustering and classification and survival analysis. The results, especially subtype clustering and classification and survival subgroups prediction demonstrate that integrated di- and tri-omics based LFs can perform better or similar to their mono-omics counterparts based LFs. Also, molecular subtypes clustering and classification results show that MMD-VAE is outperforming VAE in most datasets. Notably, the results demonstrate that unsupervised dimensionality reduction is useful in the downstream analysis  when the input dataset is discriminatory, otherwise supervised dimensionality reduction is needed. Finally, inferred and predicted survival subgroups results show a significant survival difference between the two subgroups. However, multi-omics based LFs and subgroups predicted based on them do not improve the survival prediction performances than their mono-omics counterparts. One of the potential reasons for this could be a sub-optimal number of LFs or dimension of the selected dimension space. Moreover, the straightforward integration (concatenation based) strategy, treating omics measurements from different platforms equally and performing integration in a parallel fashion may not always be useful. The linear model used for mapping between CRLFs and input features may not be the best solution for biomarkers identification. There is significant scope for future work in these areas.The residual disease is one of the strongest predictors for outcome estimation after any cancer therapy or treatment. Hence, VAE or MMD-VAE based integrated multi-omics analysis on predicting residual disease for ovarian cancer will be one of our future works.Supplementary Information.


Over the past decade, neural networks have become one of the cutting-edge methods in various research fields, outshining specifically in complex classification problems. In this paper, we propose two main contributions: first, we conduct a methodological study of neural network modeling for classifying biological traits based on structured gene expression data. Then, we suggest an innovative approach for utilizing deep learning visualization techniques in order to reveal the specific genes important for the correct classification of each trait within the trained models. Our data suggests that this approach have great potential for becoming a standard feature importance tool used in complex medical research problems, and that it can further be generalized to various structured data classification problems outside the biological domain. Formally, each biological trait corresponds to a multiclass classification problem according to the number of distinct classes related to this trait .With the rapid rising of deep learning research over the past years , neural Naturally, the latest turmoil of deep learning studies has also brought attention to studies that focus on basic questions regarding the application of neural networks; whether neural networks modeling improves on gold-standard methods, whether it worth the cost of adding complexity to model interpretability, or how an appropriate network architecture should be chosen . As our The second contribution of this paper relates to the estimation of feature importance through deep learning visualization techniques. Here we aim to shed light on patterns within the input that are important for the prediction of a given model. Particularly, for each of the biological traits we strive to identify the specific genes important for its classification. We demonstrate how feature importance analysis can be conducted using two visualization techniques commonly used in computer vision. We start by utilizing Saliency Maps  to addreAs the first step of our work, we decided to focus on some basic questions regarding neural networks – whether we can use them as a simple tool for classifying biological traits, what is the level of accuracy that can be obtained, and which architectures should be used for this purpose. We have utilized a previously published dataset  of 2441 As the most basic neural network architecture we started by examining shallow neural networks with one non-linear hidden layer (ReLU activation), tested across an increasing number of hidden layer neurons , top. AsNext, we turned to examine the effect of data normalization on the performance of the neural network classification models. We applied standard Z-score normalization to the dataset across all samples to normalize the expression of each input gene. This pre-processing step is a common practice in gene expression analyses as distinct genes may vary by the shape of their expression distribution. We tested the model architectures described above on the normalized dataset and found that introducing gene normalization to the data strongly improved the accuracy of all classification models . In partEncouraged by the observation that neural networks can appropriately be used to classify biological traits based on molecular input data, we next aimed to identify the specific genes contributing to the classification model of each biological trait. For this purpose, we examined two commonly used deep learning visualization techniques – Saliency Maps and Activation Maximization.Saliency Maps are utilized to highlight the specific input patterns relevant for the process of assigning a sample to a particular class during model prediction . More formally, we average the saliency scores of all samples in a certain class to form one saliency pattern that quantifies the contribution of each gene to the classification process of this class. Similarly, to the saliency maps per samples  that are essential for classification. Activation maximization is commonly used to generate a synthetic input that best fits a trained model assignment to a particular class  as the average of absolute regression coefficients across all classes of a trait under study. We found that using the top K trait-specific genes for the classification of each trait, either selected based on the activation maximization or the logistic regression model, results in similar performance . In addileading trait-specific genes as the minimal set of K genes that allow accuracy that is comparable to that of the full set of genes. For instance, in the case of classifying the extracellular pathogenic stimulation, 10 top genes is the minimal set whose accuracy is similar to the accuracy obtained by 414 genes  consisting of the expression levels of 414 immune-related genes in peripheral blood monocyte-derived dendritic cells (DC) extracted from healthy individuals. In total, 2441 samples are included in the dataset – each sample is treated in our analysis as a 414-genes expression panel. Each sample is accompanied with information regarding the gender and the ethnicity of the individual from which the immune cells were derived and information regarding an extracellular stimulation following which the genes expression levels were measured (or indicated that it was measured without prior extracellular stimulation). The distribution of the 2441 samples across the different classes within the three biological traits – gender, ethnicity, and extracellular stimulation – is summarized in Z-score normalization applied on each gene across all 2441 samples.We utilized a published dataset  , and second, we aimed to highlight the fact that for many datasets, conventional methods may be sufficient for the construction of an accurate classifier.We tested two different neural network architectures for the multiclass classification problem for each of the three biological traits – a neural network with a single hidden layer . Here, the averaging is only possible due to the tabular organization of the molecular data across multiple samples. We use this concept of averaged saliency maps to move from per-sample characteristics to class-specific characteristics.Activation Maximization is a deep learning visualization technique that is commonly used for generating an input instance that maximizes the activation of a particular filter within a trained model . SpecifiGiven the contribution scores of each gene per class , averaging these scores across the classes allows the detection of genes that are important for the classification of the entire biological trait. We refer to such averaged scores as “trait-specific gene scores,” and top-ranked genes based on these scores are referred to as “trait-specific genes.” Finally, the “leading trait-specific gene” are the minimal group of trait-specific genes whose classification accuracy is similar to that obtained by the full input set of genes.During the past decade, neural networks have emerged as a promising tool widely used in complex classification analysis, standing at the frontline of various deep learning fields . Recent We focused on three biological traits – an extracellular pathogenic-like stimulation following which gene expression levels were measured, the gender of the person from which a sample was derived and the person’s ethnicity. We started by exploring different neural network architectures for the prediction of each of the three biological traits and found that in all three cases, simple architectures were sufficient to obtain highly accurate predictions. We further demonstrated that data normalization greatly improves the performance of the network models, enabling the use of smaller, hence more efficient, neural networks. We also found that when using the normalized dataset, a simple logistic regression obtained accuracy scores that are similar to those obtained by neural network models. The tradeoff between model simplicity versus model accuracy should be therefore taken into consideration before turning to the use of a more complex neural network model.Next, we explored the use of two deep learning visualization techniques – Saliency Maps  and ActiGSE53166.Data was downloaded from GEO, accession number https://github.com/roniwile/neural-network-visualization-methods-for-gene-expression-data.The full analysis – including code implementation of the saliency maps and activation maximization methods as well as the datasets used – is available on Both authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


It is meaningful to assess the risk of cancer incidence among patients with precancerous colorectal lesions. Comparing the within-sample relative expression orderings (REOs) of colorectal cancer patients measured by multiple platforms with that of normal colorectal tissues, a qualitative transcriptional signature consisting of 1,840 gene pairs was identified in the training data. Within an evaluation dataset of 16 active and 18 inactive (remissive) ulcerative colitis subjects, the median incidence risk score of colorectal carcinoma was 0.6402 in active ulcerative colitis subjects, significantly higher than that in remissive subjects (0.3114). Evaluation of two other independent datasets yielded similar results. Moreover, we found that the score significantly positively correlated with the degree of dysplasia in the case of colorectal adenomas. In the merged dataset, the median incidence risk score was 0.9027 among high-grade adenoma samples, significantly higher than that among low-grade adenomas (0.8565). In summary, the developed incidence risk score could well predict the incidence risk of precancerous colorectal lesions and has value in clinical application. The conto 95.6% . Moreoveto 95.6% . Howeverto 95.6% . None coOur recent studies  demonstrBased on the unique advantages of the qualitative transcriptional signature, we selected gene pairs with stable but reversal REO patterns among CRC and normal colorectal tissues as the signature for calculating incidence risk scores of precancerous lesions. These scores, in turn, were used to predict the incidence risk of malignant transformation among non-cancer patients suffering such precancerous colorectal lesions. Score performance was evaluated using multiple independent datasets via the comparison of CRC incidence risk scores among non-cancer patients with precancerous lesions  at different disease stages. Results revealed that the incidence risk scores of high-grade precancerous lesions were significantly higher than those of low-grade lesions, suggesting that the signature could well predict the incidence risk of CRC in patients suffering precancerous colorectal lesions.1) and The Cancer Genome Atlas (2).The gene expression profiles used in this study were shown in me Atlas  (TCGA2).For array-based data measured using the Affymetrix platform, raw mRNA expression data (.CEL files) were downloaded and the Robust Multi-array Average (RMA) algorithm was used for background adjustment without quantile normalization . For arr3). For sequence-based data from TCGA, Ensembl gene IDs were mapped to unique Entrez gene IDs of protein coding genes.For array-based data, the probe ID was mapped to the Entrez gene ID with the corresponding platform file. If a probe was mapped to zero or multiple genes, then data of this probe were discarded. If multiple probes were mapped to the same gene, the expression value of this gene was defined as the arithmetic mean of the values of those probes. For sequence-based data from ArrayExpress, gene symbols were mapped to Entrez gene ID with the biological database network   if the rank of gene i was higher (or lower) than that of gene j. If gene pair  had the same REO pattern as did a majority of samples , it was considered a stable REO pattern and the gene pair was defined as a stable gene pair. For two groups of samples, a gene pair with a stable but reversal REO pattern in the two groups was defined as a reversal gene pair. For the reversal gene pairs identified in the above process, REOs of gene pairs in CRC tissues were defined as the CRC signature that was applied for defining the incidence risk score of precancerous colorectal lesions.In the training datasets, the gene measurement of each gene was converted to the corresponding rank within each sample . Then, pair wise comparisons were performed for all within-sample genes to identify stable gene pairs for a specific tissue type. For one sample, the REO pattern of two genes, iG > Gj and iG < Gj to characterize CRC and normal tissues, respectively. For each non-cancer patient with a precancerous lesion, the risk score was simply calculated as the percentage of REOs characterizing CRC. The incidence risk score for a particular sample was calculated as:In this study, we defined the REO patters n/m,Score = m was the number of gene pairs included in the signature, n was the number of gene pairs with the same REO patterns characterizing CRC. The higher the incidence risk score was, the greater the cancer incidence risk that the patient had.where The performance of the risk score was then evaluated using samples from patients with precancerous colorectal lesions (including UC and adenomas) at different disease stages from multiple datasets .4  database4 . The hyp4 . And p vThe analysis procedure of this study is described in For the 614 CRC and 55 normal colorectal tissue samples from the 11 datasets measured using the Affymetrix microarray platform see , 356,573For each sample, the risk score was simply calculated as the percentage of REOs characterizing CRC, which was close to 0 and 1 in normal and CRC tissues, respectively. Among training data with all merged samples, score medians were 0.0174 and 0.9891 for normal and CRC tissues, respectively. In the GSE22619 dataset of 10 normal colorectal tissues, we found the median score to be 0.1139. With RNA-seq, we studied 13 CRC tissue samples obtained via surgical resection and 33 CRC tissues obtained from biopsy , findingp < 0.05, To elucidate the functions and pathways that were associated with the CRC incidence risk signature, KEGG pathway analysis were performed. Functional enrichment analysis of 1,580 genes included in the signature showed that 10 pathways were significantly enriched . Similar findings were also obtained in the GSE53306 dataset consisting of 16 active and 12 remissive UC samples . Detailed results are shown in p = 1.2176e-07). Similar results were also obtained when analyzing active and remissive UC samples from datasets GSE53306 and GSE13367 , respectively.Then, in the GSE13367 dataset consisting of 16 active and 18 inactive (remissive) UC samples, the median CRC incidence risk score of active UC samples was 0.6402, significantly higher than that in remissive UC samples , as shown in We also evaluated the applicability of our score to the GSE9452 dataset, which included 8 UC samples with and 13 UC samples without  macroscopic signs of inflammation. We found that CRC incidence risk scores in UC_inflammation samples (median = 0.5682) were significantly higher than those in UC_without_inflammation samples (median = 0.1228) . In the GSE8671 dataset, the CRC incidence risk scores in the 10 high-grade dysplasia samples (median = 0.8837) were only slightly higher than those in the 14 low-grade dysplasia samples , as shown in p = 0.0191). The CRC incidence risk scores of the 13 high-grade dysplasia samples from the GSE37364 dataset were also significantly higher than those of the 14 low-grade dysplasia samples from the GSE8671 dataset . Similar results were also obtained for high-grade dysplasia samples from the GSE8671 dataset and low-grade dysplasia samples from the GSE37364 dataset . We also evaluated our score using the GSE10714 dataset consisting of 2 high-grade and 3 low-grade dysplasia colorectal adenoma samples. The median incidence risk score in the high-grade dysplasia samples was 0.9345, also higher than that in the low-grade dysplasia samples (median = 0.8853), as shown in In the GSE37364 dataset consisting of 13 high-grade and 16 low-grade dysplasia colorectal adenoma samples, the median incidence risk score in the high-grade dysplasia samples was 0.9076, significantly higher than that in the low-grade dysplasia samples (median = 0.8543)  in feces, one of the key membrane transporters of bile acids, is positively correlated with the incidence of CRC  can be found in the article/QG, QZ, and WJ conceived the study, analyzed the data, made figures, performed the statistical analysis, and drafted the manuscript. JX, JC, HY, JH, and LA searched the data and participated in the statistical analysis. YX participated in discussing and revising the manuscript. ZG and GG conceived of the study, participated in its design and coordination, helped to draft the manuscript, and supervised the work. QG, QZ, and LA revised the manuscript. All authors read and approved the final manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Arabidopsis thaliana gene datasets, SMOTE algorithms were then combined with traditional and advanced ensemble classified algorithms respectively, using Support Vector Machine, Random Forest (RF), AdaBoost (adaptive boosting), GBDT (gradient boosting decision tree), and XGBoost (extreme gradient boosting). After comparing the performance of these ensemble models, SMOTE algorithms with XGBoost achieved an F1 score of 0.94 with the balanced A. thaliana gene datasets, but a lower score with the unbalanced datasets. The proposed ensemble method combines different balanced data algorithms including Borderline SMOTE (BSMOTE), Adaptive Synthetic Sampling (ADSYN), SMOTE-Tomek, and SMOTE-ENN with the XGBoost model separately. The performances of the SMOTE-ENN-XGBoost model, which combined over-sampling and under-sampling algorithms with XGBoost, achieved higher predictive accuracy than the other balanced algorithms with XGBoost models. Thus, SMOTE-ENN-XGBoost provides a theoretical basis for developing evaluation criteria for identifying orphan genes in unbalanced and biological datasets.Orphan genes are associated with regulatory patterns, but experimental methods for identifying orphan genes are both time-consuming and expensive. Designing an accurate and robust classification model to detect orphan and non-orphan genes in unbalanced distribution datasets poses a particularly huge challenge. Synthetic minority over-sampling algorithms (SMOTE) are selected in a preliminary step to deal with unbalanced gene datasets. To identify orphan genes in balanced and unbalanced  Arabidopsis thaliana  alter the carbon and nitrogen content of the plant, increasing the protein content and decreasing the starch content (TaFROG (Triticum aestivum fusarium resistance orphan gene) contributes to disease resistance genes for crop-breeding programs (GN2 (GRAINS NO. 2) can affect plant height and rice yield  algorithms are widely used in the detection of orphan genes in big datasets. The method of ML can be used to make two kinds of field classification from an enormous genome dataset . Orphan Additionally, the number of orphan genes is much less than the numbers of non-orphan gene datasets, therefore unbalanced datasets pose significant problems for developers of classifiers. The original method of over-sampling and under-sampling  can helpThe bagging and boosting methods are two important approaches to ensemble learning  that canA. thaliana were designed as a model for the identification and prediction of orphan genes. The result could show that balancing algorithms play a more effective guiding role in identifying the orphan genes in a species.In the study described in this manuscript, over-sampling and under-sampling algorithms were introduced to clean up unbalanced data . RepreseData preprocessing is the first step for data mining and affects the result. Preprocessing includes data discretization, missing values, attribute coding, and data standard regularization. In practice, each industry has unique data characteristics, so different methods are used to analyze the data and perform preprocessing.The processing of unbalanced data describes classes with obviously uneven distribution. The traditional method used random over-sampling to increase the number of small-class samples to achieve a consistent number. Because this method achieves balance by a single random over-sampling strategy of copying data, the added repeated data will increase the complexity of data training and induce over-fitting.To deal with the problem of unbalanced data classification, some algorithms have been used effectively to improve the performance of classification. Common methods for processing datasets included mainly: over-sampling and under-sampling, or a combination of under-sampling and over-sampling.K  neighboring samples of the same category, which is used as an auxiliary sample to synthesize a new sample and repeated n times. Finally, linear interpolation is performed between the sample and each auxiliary sample to generate n synthesized samples. The basic flow of the algorithm is:To solve the problem of over-fitting associated with unbalanced data when the learning information is not generalized, (i)K samples of the nearest neighbor for each sample xi, whose label is “1”;Find (ii)j belonging with few categories is selected randomly from K;A sample x(iii)i and xj to construct a new minority sample.Linearly interpolate randomly between xK values of the neighbor samples themselves, so it is difficult to determine the optimal value. Additionally, the newly synthesized samples may fall into the sample area labeled “0,” which confuses the boundaries between them and interferes with the correct classification of the data.The SMOTE algorithm effectively solves the problem of over-fitting caused by the blind replication of random over-sampling techniques. However, the selection of the nearest neighbor sample in step 1 exits is purposeless. Users need to determine the number of Therefore, to address these two problems, Adaptive Synthetic Sampling adaptively generates different numbers of sampling samples according to data distribution . The bas(i)ml−ms)×β, where ml is the number of majority samples, andms is the number of minority samples. If β = 1, the number of positive and negative samples is the same after sampling, indicating that the data is balanced at this time.Calculate the number of samples to be synthesized, as follows:G = ((ii)ri = Δi/K, where Δi is the number of majority samples in K nearest neighbors, i = 1,2,3…….,msCalculate the number of K nearest neighbor value of each minority sample, Δ is the number of majority samples in the K neighbors, the formula is as follows: (iii)ri, the formula is To normalize (iv)According to the sample weights, calculate the number of new samples that need to be generated for every minority sample. The formula is si = xi + (xzi−xi)×λ, where si is the synthetic sample, xi is the ith minority samples, and xzi is a random number of the minority sample λ∈  selected from the K nearest neighbors of xi.Select one sample from the K neighbors around each data with the label “1” to be synthesized, calculate the number to be generated according to g the formulaApart from using a single under-sampling or over-sampling method, two resampling methods can be combined. For example, SMOTE-ENN , ENN is x to y as d . If there is not a sample z, such as d  < d  or d  < d , A  pair is called a Tome link.SMOTE-Tomek  also comThe main idea of the ensemble learning algorithm is to construct multiple classifiers with weak performance and use a certain strategy to combine them into a classifier with strong generalization performance. Consequently, the performance of the ensemble is better than that of a single classifier.This study created two classification models for unbalanced datasets and used Python to build five integrated learning models of SVM, RF, AdaBoost, GBDT, and XGBoost and conducted comparative experiments to find the optimal model. XGBoost performed best in the classification, Five kinds of balanced data learning methods of resampling: SMOTE, BSMOTE, ADASYN, SMOTE-ENN, and SMOTE-Tomek, were then combined with XGBoost to build an ensemble model that produced excellent classification results .XGBoost was modified by adding regular items to the GBDT algorithm that can predict the orphan gene binary classification problem and increase the calculation speed. XGBoost uses the gradient boosting algorithm of the based learner classification and regression tree (CART) to calculate the complexity of the leaf nodes of each tree and uses the gradient descent algorithm to minimize the loss for finding the optimal prediction score, thus avoiding over-fitting the learned model and effectively controlling the complexity of the model .The derivation process is as follows:(i)Objective function: (ii)Using the first and second derivatives, the Taylor formula expands:(iii)Measuring the complexity of the decision tree as:(iv)Substituting the above two steps into the objective function (1), it is organized as:(v)Ij = {i|q(xi) = j}, represents the sample set belonging to the j-th leaf node.Then, (vi)To minimize the objective function, let the derivative be 0 and find the optimal prediction score for each leaf node:(vii)Substitute the objective function again to get its minimum value:(viii)Find the optimization goal of each layer of the build tree through obj to find the optimal tree structure, and split the left and right subtrees as:The confusion matrix (error matrix) is a matrix table shown in  that is (i)True positive (TP): the actual value of the model is the orphan genes, so the model predicts the number of orphan genes.(ii)False positive (FP): the actual value of the model is the orphan gene, but the model predicts the number of non-orphan genes.(iii)False negative (FN): the true value of the model is non-orphan genes, so the model predicts the number of orphan genes.(iv)True negative (TN): the true value of the model is non-orphan genes, but the model predicts the number of non-orphan genes.A large number of confusion matrix statistics make it difficult to measure the pros and cons of a model. Therefore, we added using Recall, Precision, and F1-score, as performance indicators to better evaluate the performance of the model:(i)Recall rate (accuracy rate of positive samples):(ii)Precision (precision rate of positive samples):(iii)F1-score value:The receiver operating characteristic (ROC) curve reflects the probability of identifying correct and wrong results according to different thresholds. The curve passes  and , and the validity of the model is generally determined by the diagonal of the curve in the upper left section of the graph.The AUC value is the value of the area under the ROC curve, which is generally between 0.5 and 1. The quantized index value can better compare the performance of the classifiers: a high performance classifier AUC value is close to 1.A. thaliana were obtained from The Arabidopsis Information Resource (TAIR8) dataset ftp://ftp.arabidopsis.org/home/tair/Genes/TAIR8_genome_release, which contained a total of 32825 gene sequences. The known orphan genes of A. thaliana downloaded from the public website https://www.biomedcentral.com/content/supplementary/1471-2148-10-%2041-S2.TXT . In particular, the F1 value indicates that the XGBoost model can distinguish orphan genes and non-orphan genes precisely.On the balanced We found that the ROC curve of SMOTE-XGBoost completely wrapped the ROC curves of the other methods, and the Precision-Recall (PR) curve confirmed that XGBoost produced the best performance among the five balancing algorithm methods .The PR curve  indicateWe also tested five different models, XGBoost combined with a balanced algorithm including SMOTE, BSMOTE, ADASYN, SMOTE-Tomek, SMOTE-ENN, to further explore the result of the unbalanced datasets. The results of the confusion matrices of five models are shown in The performance indices of the five balanced algorithms with ensemble XGBoost classifiers models are shown in Therefore, the SMOTE-ENN-XGBoost model is used to classify and analyze the orphan genes in unbalanced datasets and applied to the actual predictions.A. thaliana dataset to build training and testing models  can be found in the article/QG and XJ: development of methodology. HY and YX: sample collection. QG, XJ, EX, and XW: analysis and interpretation of data. QG, XJ, LG, and SL: writing, review, and revision of the manuscript. All authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


KiMONo performs network inference by using statistical models for combining omics measurements coupled to a powerful knowledge-guided strategy exploiting prior information from existing biological sources. Within the resulting multimodal network, nodes represent features of all input types e.g. variants and genes while edges refer to knowledge-supported and statistically derived associations. In a comprehensive evaluation, we show that our method is robust to noise and exemplify the general applicability to the full spectrum of multi-omics data, demonstrating that KiMONo is a powerful approach towards leveraging the full potential of data sets for detecting biomarker candidates.Constantly decreasing costs of high-throughput profiling on many molecular levels generate vast amounts of multi-omics data. Studying one biomedical question on two or more omic levels provides deeper insights into underlying molecular processes or disease pathophysiology. For the majority of multi-omics data projects, the data analysis is performed level-wise, followed by a combined interpretation of results. Hence the full potential of integrated data analysis is not leveraged yet, presumably due to the complexity of the data and the lacking toolsets. We propose a versatile approach, to perform a multi-level fully integrated analysis: The Knowledge guIded Multi-Omics Network inference approach, KiMONo ( However, integrating multiple levels simultaneously is still an ongoing challenge. To account for this, most studies use a level-by-level approach where each omic level is analysed independently. Disregarding the complex ‘cross-omic’ interplay simplifies the analysis problem but comes at the cost of potential overlooked and misinterpreted results3.Over the past decade, inventions of high throughput techniques enabled the measurement of multiple omic levels on a large scale and in a cost-efficient manner. The resulting data provides deep insights into the network of molecules orchestrating biological mechanisms. Hence, it is no surprise that many studies underpin their findings using multi-omic analysis when identifying disease or condition specific key molecules. So far, there exist several different multi-omic analysis strategies. Probably one of the first and well-established combinations of two omic levels, are expression quantitative trait loci (eQTL) analyses5. These methods infer lower-dimensional representations (latent factors) of the original high dimensional multi-omic data space. Even though the latent factors represent certain patterns of the data, it is often difficult to find the biological meaning of these factors5. Improved interpretability is one of the big advantages of network based approaches. These identify condition specific key molecules via inferring and analysing a network representation of the processes7. Such networks often consist of thousands of nodes, representing the different omic input features, such as genes and proteins which are linked condition-specifically.Recently, more sophisticated latent factor-based models have been introduced, capable of analysing multiple omic levels simultaneously8. To increase the specificity one can use more advanced machine learning approaches, instead of correlation, to identify associations between nodes10. These methods are only applicable to high dimensional multi-omic data with large amounts of samples.The most common and straightforward inference approaches generate correlation based networks. Here network links are generated by calculating pairwise correlation between all features. Unfortunately, this strategy often results in low-specific and highly connected networks which are hard to interpret11. MiRlastic facilitates prior knowledge to increase the performance for high dimensional and low sample size data analysis. MiRlastic creates a multivariate penalized regression model for each mRNA with the multitude of predicted miRNA regulators per mRNA, thereby filtering predicted regulations in the context of real data. Next, aggregating miRNA and mRNA species from all regression models assembling a mRNA-miRNA interaction network.To overcome the limitation of relying on pairwise correlation we previously developed miRlastic12 penalization in the multivariate regression approach to model each gene's expression level. The chosen penalty term allows for a bi-level selection, penalizing each omic level as well as penalizing within each level. Aggregating these models, all features are linked to their gene and assembled in the final heterogeneous multi-level network.We now developed a method capable of using any kind of omic levels inferring a multi-level network. Our novel Knowledge guIded Multi-Omic Network inference method—KiMONo, vastly expands miRlastic. KiMONo allows a simultaneous integration of multiple omics levels, for example, genomic variants, methylation, gene expression, proteomics and biological information. KiMONo leverages various prior information, reduces the high dimensional input space, and uses sparse group LASSO (SGL)In our benchmark were able to showcase KiMONo’s performance by applying it to one of the most comprehensive multi-omic data sets available—the PanCancer data collection of The Cancer Genome Atlas. It consists of almost 5000 samples describing 12 different cancer types across ~ 50,000 features on five omic levels. As a validation we also applied the same benchmark setup to a data set of the highly complex disease major depressive disorder (MDD). This challenging data set contained over 107 samples measured on four different omic levels with in sum over 4.5 million features. Finally, we successfully identified key molecules driving the conditions within these data sets. KiMONo retrieved previously reported genes that matched the underlying disease setting, in addition to yet unknown potentially interesting genes. We present KiMONo as a versatile method to derive fully integrated and holistic multi-level networks capturing the data-supported interplay between omics levels.Our novel method KiMONo infers a condition specific multi-level network from any mix of multi-omic data sets see Fig. . The net11. To be more specific,Once the prior and input data are submitted, KiMONo optimizes a penalized regression model for nodes individually to establish its outgoing network links. The feature’s value, i.e. gene expression, represents the criterion variable KiMONo further uses the fitted models, of all nodes to assemble a multi-level omic network. Within this network, nodes represent features of the input data, like genes, proteins methylation sites or SNPs, and connections between them are weighted via the We used the breast invasive carcinoma data, a subset of PanCancer collection, as a benchmark set. It is the largest set containing 604 patients with ~ 50,000 measured features across the 5 omic levels described in the previous section.First, we evaluated the performance using all features at once without any prior pre-selection. Since, neither conventional LASSO or elastic net nor SGL were able to infer networks we asked next whether incorporating first- and second-order links would increase KiMONos inference performance?An evaluation of the inferred networks using only the first-order prior showed that 5,349 models were inferred beyond intercept-only modes having a mean Next we evaluated if the amount of different omic layers also impacts the performance. Therefore, we selected models with second-order prior and entclass1pt{minimaentclass1pt{minimaentclass1pt{minimaPanCancers’s breast invasive carcinoma data. Using KiMONo we inferred a network for each test case where 5–95% of the samples were removed  and Clinical (0.02) layer, maintained the general low performance of 0.02 and 0.01, respectively.We observed a strong effect of noise on the coverage of the network Fig. B. LookinTo exemplify the data analysis power of KiMONo on multi-layer data, we inferred networks on the PanCancer data consisting of 11 cancer types. As a post-processing step, we excluded all models for which 13. Here the top enriched KEGG pathway is the cancer-related Chronic myeloid leukemia (FDR = 1.45e−37) pathway followed by Pathways in cancer (FDR = 6.3e−35). Furthermore, we were able to identify 345 features which were uniquely identified to each cancer type. For instance, the methylation site cg00103783 (chr17:7.583.931), mapping to MPDU1 gene, was only detected as important within the head & neck squamous cell carcinoma network. Interestingly14, introduced MPDU1 as a potential biomarker for HNSC.The final networks had on average 26.343 links and 3158.2 nodes  was the top enriched pathway, followed by Hepatitis B (FDR = 2.51e−39) and Cell cycle (FDR = 2.3e−38). Both, Cell cycle and Hepatitis B, were known breast cancer-related pathways17. However, the Breast Cancer-specific KEGG pathway ranked on place 14 (FDR = 5.0395e−33) among all enriched pathways. Another interesting result was the inferred Glioblastoma multiforme (GBM) network, Fig. GBM is one of the rarest cancer types, it is also one of the most lethal ones having a survival time of 14–15 months after diagnosis18. The GBM data set was relatively small including only paired data for 61 patients with 58,051 features across 5 omic layers. Nevertheless, KiMONo inferred 112,945 links between 9341 nodes. Even though the top 20 features were not as densely connected as in the previous example, we were able to link CTNNB1, HIF1A, HDAC1 and EWSR1 to survival time. Beside EWSR1 all have been reported as survival time related in GBM  and multi-level omics are relatively new26. Therefore, making successful inference of a multi-omic cross-talk regulatory network is of importance to better understand the depression phenotype.Even though the PanCancer is one of the most comprehensive multi-omic datasets available, we further wanted to evaluate our method on a more complex type of disease, like MDD. While progress has been made in understanding the pathomechanisms of MDD, success in translating findings into clinical practice has been limitedFor this purpose, we applied KiMONo on a cohort, consisting of 107 healthy individuals and patients. There were 4,247,909 imputed SNPs, 12,418 transcripts and 320,481 methylation sites available for the evaluation of our method, after filtering for the 25% of methylation sites with the least variance. Biological information such as BMI, age, sex and status of the diagnosis and cell type composition were also taken into account for network inference.To ensure a higher quality of selected features we filtered for To compare with state-of-the-art methods, we identified eQTL and eQTM genes using pairwise models and set them into context to the findings of KiMONo. Using the same proximity restrictions for the MatrixEQTL and KiMONo, we found 873 and 660 eQTL-genes, respectively, overlapping in 301 Fig. C. FurtheThe top 20 genes identified with the highest betweenness measure were found to exhibit higher performances compared to the average model. The potential of our method becomes apparent when looking at connections found through KiMONo but not pairwise models of MatrixEQTL. After correcting for residual effects of all other features in multilevel models, the connection between the expression of SLC39A11  and SNP rs1493550 and methylation site cg26124719 located both in an intron became clearly resolved  an essential pathway for the central nervous system and studies have shown the effects of antidepressant treatments on autophagy29. Interestingly, among the top 10 pathways was Axon guidance (FDR = 1.054e−3), which has been shown to be a strong risk factor for depression, as stress may affect brain structure and function31.Half of the top 10 hits have been previously linked to depression or pathways involved in the pathogenesis of the disease  that were uncovered by MatrixEQTL using pair and level-wise tests. In addition we found further associations, complementing MatrixEQTL, when deriving regulatory networks in context with all features from all omic levels. It is possible that these features can only be detected when taken into account the context of the underlying omic-crosstalk. Across all top hits in the MDD dataset Table , we obseIn addition to incorporating second-order links, we also showed the advantage of multivariate models derived from various omic-layers by uncovering relationships that were not found in pairwise models. After correcting for residual effects of every feature except for the one of interest, the connection became clear Fig. C,D. Our Applying KiMONo on both TCGA cancer types and MDD, we were able to find previously reported genes that matched well with the underlying disease setting see Fig. B/Table 1In summary, we showed KiMONo is a versatile method to derive fully integrated and holistic multi-level networks capturing the data-supported interplay between omics levels. Comprehensive benchmarks demonstrated that KiMONo is more sensitive to noise than to the reduction of samples. Further, application to two human disease settings showed that key nodes of the inferred multi-omics disease networks also play key roles in disease pathophysiologies. Ultimately, the holistic networks inferred using KiMONo may serve as tools to easily uncover key regulatory features, no matter the disease setting or complexity of the data.TCGA) data portal41. This is one of the most comprehensive multi-omic data sources. This collection contains multi-omic data sets of 4926 samples describing 11 different cancer types—acute myeloid leukemia (191 samples), bladder urothelial carcinoma (135 samples), breast invasive carcinoma (871 samples), colon adenocarcinoma (421 samples), glioblastoma multiforme (580 samples), head & neck squamous cell carcinoma (309 samples), kidney clear cell carcinoma (496 samples), lung adenocarcinoma, lung squamous cell carcinoma (344 samples), ovarian serous cystadenocarcinoma (563 samples), rectum adenocarcinoma (164 samples) and uterine corpus endometrioid carcinoma (495 samples). The portal provides open access to highly preprocessed ‘level 3′ data of five omic characterizations, Proteome (~ 130 proteins), Transcriptome , Copy Number Variation (~ 84 CNV), Mutation , Methylation (~ 2043 sites) but also phenotype information in the form of Clinical data (4 variables). In our analysis we only included samples which were measured across all 5 omic levels, restricting the data sets to 2036 patients across 11 cancer types  data set as a second real data example. This cohort consisted of 289 caucasian individuals, 160 healthy controls and 129 patients diagnosed with major depressive disorder. Recruitment strategies and further characterization of the MDD cohort have been described previously in45. Three levels of omic information, comprising the transcriptome, methylome and genotype, as well as biological information, were measured for 107 out of 289 individuals, consisting of 33 females and 74 males, distributed over 64 controls and 43 patients. Details on the omic preprocessing can be found in45.In addition to the TCGA data we also used a Major Depressive Disorder (46 based on GRCh37 (hg19) RefSeq. Additionally, we annotated methylome, the CpG site probe, and the transcriptomes gene symbol to sequence positions by performing a re-alignment using Bismark47. Furthermore, we connected the genes to SNPs and methylation sites within a distance of 10 kbp and 500 kbp, respectively. Second-order links were created between genes via a ‘guilt-by-association’ approach using the BioGrid database. Furthermore, we connected genes with their associated genes methylation site generating, introducing second-order linked methylation sites.For generating the prior knowledge first-order links, we annotated gene expression probes and gene symbols using the Re-Annotator pipelineTo assess goodness of fit on every gene-level model, we use the r-squared metric measuring how much of the variance of the expression can be explained by the model. We calculate for each model the explained sum of squares l omics/clinical level. This is done by calculating the PanCancer data set, to dissect the goodness of fit for contribution of the proteomics level, we corrected the gene expression measurements by the contribution of the other omics layers but not the proteomics level. Then, l. Finally, l to 0.In order to approximate each information level contribution to the 44 resource. We not only interconnected the transcriptome but also all other layers.We used the PanCancer breast invasive carcinoma network models as test scenarios, investigating the impact of the different prior strategies. To evaluate how well each strategy performed, we compared the performance of the models, which explained at least 10% of the variance within the data and the coverage of the inferred networks.Overall, we compared two different prior strategies. On the one hand, a prior solely based on the genomic location and annotation databases. Here we annotated protein, methylation, mutation and clinical information to the transcriptome level. On the other hand we generated a prior also including second-order links using the BioGridTCGA breast invasive carcinoma data and randomly reduced the amount of samples. We repeated each simulation 20 times (except for the case where 100% of the data was available). The final test cases included 10%, 30%, 50%, 70% and 100% of the data. Note, for each generated dataset, KiMONo excludes features with To benchmark the performance on small data, we simulate data sets with shrinking sample size. Therefore we used the PanCancer breast invasive carcinoma data. Random noise was generated using Gaussian noise, We followed a similar strategy for benchmarking the robustness of the method with respect to noise. Here we simulated test sets by decreasing the signal to noise ratio. All simulated sets were generated using a subset of the In the MDD dataset, we implicitly computed multivariate expression quantitative trait loci in the KiMONo approach, as we impose a genomic proximity prior to link variants and gene expression measurements. Thus, we compared the quantitative trait analysis results of KiMONo to the state-of-the-art pairwise analysis tool, matrixEQTL. Here we used both methods to detect expression quantitative trait loci (eQTL) and expression quantitative trait methylation sites (eQTM) genes within the MDD data set. For the matrixEQTL calculation, we focused on cis-eQTL and cis-eQTM windows of 10 kbp and 500 kbp distance, respectively. Further, we corrected the expressed genes for the covariates, BMI, age, sex and status of the diagnosis, with significance threshold set to 48. Furthermore, we used the betweenness centrality to estimate the importance of nodes within the single-layer network. The betweenness centrality is defined as We treated all links between the multiple levels after KiMONo inference as undirected edges, generalizing the multi-layer directed network to a simple single-layer association network representation. To show that the generalized network structure is, like most biological networks, scale-free, we tested goodness of fit to evaluate if the node degree follows a gamma distributionPanCancer data is publicly available via the TCGA data portal . A list of the sampleIDs and cancer types which contained all 5 omic levels can be found in Data The https://github.com/cellmapslab/KiMONo. We used the Bioconductor’s R packages Homo.sapiens, GenomicRanges and FDb.InfiniumMethylation.hg19 to generate the annotation between various omic types within TCGA’s PanCancer data. Furthermore we used the Re-Annotator pipeline46 based on GRCh37 (hg19) RefSeq and Bismark47 to annotate the MDD data. For state-of-the art eQTL analysis we applied the open source tool matrixEQTL (version 2.3). Pathway annotations were performed via the open source tool pathwaX13.KiMONo is freely available as an R package on No ethics approval was required for the study.Supplementary Information