As of today, cancer is still one of the most prevalent and high-mortality diseases, summing more than 9 million deaths in 2018. This has motivated researchers to study the application of machine learning-based solutions for cancer detection to accelerate its diagnosis and help its prevention. Among several approaches, one is to automatically classify tumor samples through their gene expression analysis.In this work, we aim to distinguish five different types of cancer through RNA-Seq datasets: thyroid, skin, stomach, breast, and lung. To do so, we have adopted a previously described methodology, with which we compare the performance of 3 different autoencoders (AEs) used as a deep neural network weight initialization technique. Our experiments consist in assessing two different approaches when training the classification model — fixing the weights after pre-training the AEs, or allowing fine-tuning of the entire network — and two different strategies for embedding the AEs into the classification network, namely by only importing the encoding layers, or by inserting the complete AE. We then study how varying the number of layers in the first strategy, the AEs latent vector dimension, and the imputation technique in the data preprocessing step impacts the network’s overall classification performance. Finally, with the goal of assessing how well does this pipeline generalize, we apply the same methodology to two additional datasets that include features extracted from images of malaria thin blood smears, and breast masses cell nuclei. We also discard the possibility of overfitting by using held-out test sets in the images datasets.1 score of 99.03, 89.95, and 98.84 and an MCC of 0.99, 0.84, and 0.98, for the RNA-Seq (when detecting thyroid cancer), the Malaria, and the Wisconsin Breast Cancer data, respectively.The methodology attained good overall results for both RNA-Seq and image extracted data. We outperformed the established baseline for all the considered datasets, achieving an average FWe observed that the approach of fine-tuning the weights of the top layers imported from the AE reached higher results, for all the presented experiences, and all the considered datasets. We outperformed all the previous reported results when comparing to the established baselines. Cancer is a label for a group of diseases that is characterized by abnormal and continuous cell growth, with the potential to spread through its surrounding tissues and other body parts . During When automatically classifying tumor samples, one approach is to analyze the samples derived molecular information, which is its gene expression signatures. Gene expression is the phenotypic manifestation of a gene or genes by the processes of genetic transcription and translation . By studDeepCC, a supervised deep cancer subtype classification framework based on deep learning of functional spectra quantifying activities of biological pathways, robust to missing data. The authors conducted two studies, each with a different cancer detection . The authors claimed that the described method achieved overall higher sensitivity, specificity, and accuracy compared with other classical machine learning methods widely used for this kind of task, namely random forests, support vector machine (SVM), gradient boosting machine, and multinomial logistic regression algorithms, with an accuracy higher than 90%.Several authors have chosen the previously mentioned approach of analyzing the gene expression of tumor samples. Many of the developed methodologies in this scope use straightforward supervised training, especially when using deep neural networks (DNNs), relying on their depth to produce the best results. Gao et al.  proposedGenome Deep Learning (GDL), a methodology aiming to study the relationship between genomic variations and traits based on DNNs. This study analyzed over six thousand samples of Whole Exon Sequencing (WES) mutations files from 12 different cancer types from TCGA, and nearly two thousand healthy WES samples from the one thousand genomes projects. The main goal of GDL was to distinguish cancerous from healthy samples. The authors built: 12 models to identify each type of cancer separately, a total-specific model able to detect healthy and cancerous samples, and a mixed model to distinguish between all 12 types of cancer-based on GDL. All the experiments were evaluated through: (a) three performance metrics — accuracy, sensitivity, and specificity — and (b) Receiver Operating Characteristic curves, with the respective Area Under the Curve (ROC-AUC). This methodology achieved a mean accuracy of 97.47% on the specific models, 70.08% on mixture models, and 94.70% on total specific models, for cancer identification.Sun et al.  proposedIn , Kim et However, many DNNs, besides the known open challenges regarding their training setting , have a versus 0.2912 and 0.2967, respectively), and (2) was able to capture the change in gene expression quantification.To support our motivation and choices, we present some works that include unsupervised training in their methodologies. In , the autshallow artificial network trained to distinguish papillary thyroid carcinoma from healthy samples. The authors highlighted two different results, in their 5-fold cross validation experiment: the combination of a SMOTE [1 score of 98.12, while the usage of a DAE achieved a mean F1 score of 94.83.The authors in  describe a SMOTE  with Tom1 score. The results show that the semi-supervised deep learning approach achieved superior performance over the other considered methods, with an average F1 score of 98.97% across the three used datasets.In  presente1 score. The authors reported that the overall best result was achieved through a combination of Denoising AE, followed by its complete import into the classification network, and by allowing subsequent fine-tuning through supervised training, yielding an F1 score of 99.61.The authors in  developeIn , the aut1 score of 98.04, when identifying thyroid cancer.In , FerreirWe extend the previously described work in  by assemThis section is organized as follows: “In our experiments, we use two different types of data, which are described in the subsections that follow.PanCancer, sorting the obtained results from highest to lowest RNA-Seq examples, and finally selecting the thyroid, skin, stomach, breast, and lung datasets.We used five different RNA-Seq datasets, from The Cancer Genomes Atlas (TCGA) , each reNA, Null or NaN, which are substituted by NA [All five datasets are composed of approximately 20 thousand features. Each column feature in these datasets represents a specific gene, and the cell values for each column are the expression of that gene in a particular sample. All the RNA-Seq data were normalized according to the distribution based on all samples. The expression distribution of a gene is estimated by calculating the mean and variance of all samples with expression values, and discarding zero’s and non-numeric values such as ed by NA . With thentropic value; with no value variation, one cannot infer any information. In total, 2,056, 2,072, 1,993, 457, and 591 columns were removed on the thyroid, skin, stomach, breast, and lung datasets, respectively. By default, we attributed the remaining missing values   is an unf and g, parameterized on θe and θd respectively, where θ=θe∪θd, L being the loss function, and J the cost function to be minimized. When learning, the AE aims to find value θ that:Let the encoding and decoding functions of the AE be penalizing the reconstruction of the input, given by n (ReLU)  for all n (ReLU) . Similarn (ReLU) , 35–37.“unsupervised pre-training guides the learning towards basins of attraction of minima that support better generalization from the training dataset” [We use the AEs as a weight initialization technique  since evdataset” . Thus, wThe simplest AE has only one hidden layer. This type of AE learns through the optimization cost function presented in Eq. A Denoising AE (DAE)  aims notX, intentionally corrupted by a sort of noise [Dropout layer, immediately after the input layer, where 10 of the connections are randomly cut.where of noise . To simuof noise , we applsparsity penatly  learning process also has two main goals: (1) minimizing the reconstruction error when aiming to copy the input data, and (2) applying a X, an SAE can address unique statistical features of the dataset it has been trained on [λ of 10−5.Although it also tries to reproduce ained on , 41. To The top layers, where we use three different AEs as weight initialization;The dimension of the latent vector of the AEs, that means the encoding layer size;The imputation technique, to replace missing data when preprocessing the datasets.We have adopted the methodology described in , which wFor all these, we follow the same pipeline see Fig. . For eacThe RNA-Seq input data as one of 5 cancers, namely thyroid, skin, stomach, breast, and lung;The malaria input data as Malaria or Non-Malaria;The breast masses input data as Malign or Benign.With the complete architectures (AE as the top part of the classification network) assembled, we train each one to distinguish:Besides the top layers imported from the AE, the classification part of the full architecture is composed of a Batch Normalization layer , followeTo assess the following experiments, we decided to only use the AE that achieved the best results in the first experiments. For points (2) and (3), we try three different dimensions: 64, 32, and 16. For the data imputation study, we use three strategies: replacing the data with (a) the mean column value (used as default), a constant value , and (b) with the most frequent value.decoder layer, with all its weights randomized, guaranteeing that there are no discrepancies concerning the network’s topological complexity, for both strategies.Furthermore, we want to study if when using Strategy 2 (importing the complete AE into the classification network) the model yields better results just because it has one more layer and, therefore, more parameters to train. To observe if the classifier is better only by being deeper, we pre-trained the AE and, at the embedding step for Strategy 1, we add a Finally, we want to assess how the pipeline behaves when dealing with different data types, besides RNA-seq entries. Hence, we apply the same methodology to the image extracted features datasets described in “1 score, and provide the Receiving Operator Curve, with the respective Area Under the Curve (ROC-AUC), and the Precision-Recall Curve.We use stratified 10-fold cross-validation, to ensure and provide statistical evidence. The AEs are trained during 300 epochs, and the classifier during 500 with a batch size of 100. The classification model is trained with the binary cross-entropy loss function  and withThe Malaria train and test sets are composed of 3957 and 1949 examples, respectively;The Wisconsin Breast Cancer train and test sets are composed of 381 and 188 examples, respectively.Furthermore, to study how the model generalizes to unseen data during the training phase, we evaluate the performance of the best architecture combination on a held-out test set, for the Malaria and the Wisconsin Breast Cancer datasets. For both, and separately, we use a ratio of one third to create two new splits. Therefore:We performed a stratified split, meaning that we preserve the distribution of the label in both the train and test sets. With the training set, we followed the same stratified cross-validation strategy described above. The performance on the held-out set was assessed through the same metrics as well.To support our claim that using AEs as weight initialization improves a DNN performance, we defined three different baselines, for each of the used datasets.1 score of 51.15%.For the RNA-Seq data, we established as baseline the results from the classification part of our methodology, without the top layers of the AEs. The baseline model was trained under the circumstances described in the previous section. The results of such experiment can be found in Table We further added another baseline for the RNA-Seq datasets, where we use a simple AE with random and fixed weights, with the intent of discarding the possibility of our pipeline yielding better only because its classification architecture is slightly deeper. These baseline results are presented in Table 1 score, this classifier performance ranged from 18.8% to 87.4%, considering all the malaria parasite species-stage combinations. Secondly, in [1 score of 53% when detecting parasites from artifacts.For the malaria dataset, we consider two results of two different approaches, applied to the same domain. Firstly, in , the autndly, in  a 5-clas1 score of approximately 97%.For the Wisconsin Breast Cancer dataset, we chose as baseline the work presented in , where tAutoencoders as weight initialization can efficiently predict diseases when applied to different biological and feature-extracted data. Given the results, one tends to assume that the methodology originally presented in [fine-tuning (Approach B), with an F1 score of 99.03 and an MCC of 0.99, when distinguishing thyroid from the other cancer types . The various networks combinations also achieved very high results for each cancer type, as observable in Table fine-tuning (Approach B), with an F1 score of 89.95% and an MCC of 0.84, on the Malaria dataset (as highlighted in Table The combination of the pre-trained DAE encoding layers import to the upper layers (Strategy 1) of the deep classification network and allowing subsequent fine-tuning (Approach B), with an F1 score of 98.84% and an MCC of 0.98, on the Wisconsin Breast Cancer dataset, (as shown in Table The combination of the pre-trained AE encoding layers import to the upper layers (Strategy 1) of the deep classification network and allowing subsequent ented in  generaliented in  and the With these results, there is evidence that this methodology can generalize to other types of data and tasks.Subsequent model fine-tuning (Approach B) leads to better results than fixing the weights (Approach A). Similarly to [larly to , it was There is high evidence supporting that importing only the encoding part of the AE leads to good results. According to the results in Table There is no evidence of overfitting, considering the additional experiments with held-out test sets. According to the results in Table Changing the AE latent vector dimension has more impact when fixing the AE pre-trained weights (Approach A) than when fine-tuning all the weights (Approach B). When varying the latent vector dimension, the 3 tested dimensions achieved lower results than the encoding layer size used as default when analyzing Approach A (Fixed Weights). However, when analyzing Approach B (Fine-Tuning Weights), the results in Table 1 score variation of 1% to 3%, comparing with the default size experiment. In Approach A (Fixing Weights), the performance difference was more significant, with the F1 score decreasing nearly 20% with a latent vector dimension of 64, and approximately 60% with a dimension of 16, for Strategy 1.There is no evidence supporting a conclusion on which is the best data imputation strategy. After the imputation strategy experiment, the results pointed out that the mean strategy led to the highest performance in the classification task when considering Approach B. However, one can observe in Table It is possible to observe the complete results for all the experiments in Tables We compared the performance of a deep neural network (DNN) when using three different autoencoders (AEs) to initialize its weights. To do so, each AE was pre-trained and then attached to the top layers of our classifier. In the importation phase, two different strategies were studied: (1) just importing the AE’s encoding layer, and (2) importing all the AE’s layers. Each of the three built architectures was then trained to classify the input data as one of the five types of cancer in this study. Two different approaches were analyzed, in the training process: (A) fixing the imported weights, and (B) by allowing them to be fine-tuned during supervised training. Additionally, we studied (1) how changing the encoding space dimension impacts the AEs and DNN performances, and (2) how the missing data replacement strategy influences the performance in the classification task. We also assessed the impact that the number of AE imported layers has on the DNN overall performance.Furthermore, we extended the generalization study of this methodology by applying it to two different datasets: the MalariaScope thin blood smears data and the Wisconsin Breast Cancer tumors datasets.1 score, but to all the other evaluation metrics as well. After a 10-fold cross-validation training process, a full embedding of a pre-trained Basic AE to the top layers of the DNN (Strategy 2), followed by fine-tuning, achieved the best overall performance, with an F1 score of 99.03 ±1.21. Moreover, we outperformed as well other established baselines, for the MalariaScope and Wisconsin Breast Cancer datasets, supporting the claim that this methodology generalizes well, including when dealing with other data types. After performing two distinct held-out datasets, we could conclude that our models generalize well to unseen and different data, not overfitting during the training phase. Allowing fine-tune (Approach B) on the imported weights of the AEs led undeniably to better results than fixing the weights of the top layers (Approach A), as can be observed in the results. Approach A is more sensitive to latent vector dimension variations, in comparison with a more stable Approach B. Finally, the results showed no evidence on which imputation strategy is the best, considering the RNA-Seq data.We outperformed the best result reported in , accordiIn conclusion, this methodology led to state-of-the-art performance in cancer classification from gene expression, strongly supporting that using AE as weight initialization can help DNNs achieving better performances. We believe that it also has high potential of generalizing well to other data and problems, as shown in the results using datasets of features extracted from images.In the long term, and although some of the data is considered a toy dataset, we expect that this work will lead to a more efficient and robust automated system for the diagnosis of diseases, in particular cancer, providing a faster diagnostic, and improving the expected treatment outcome.


Ovarian cancer causes 151,900 deaths per year worldwide. Treatment and prognosis are primarily determined by the histopathologic interpretation in combination with molecular diagnosis. However, the relationship between histopathology patterns and molecular alterations is not fully understood, and it is difficult to predict patients’ chemotherapy response using the known clinical and histological variables.We analyzed the whole-slide histopathology images, RNA-Seq, and proteomics data from 587 primary serous ovarian adenocarcinoma patients and developed a systematic algorithm to integrate histopathology and functional omics findings and to predict patients’ response to platinum-based chemotherapy.P = 0.003).Our convolutional neural networks identified the cancerous regions with areas under the receiver operating characteristic curve (AUCs) > 0.95 and classified tumor grade with AUCs > 0.80. Functional omics analysis revealed that expression levels of proteins participated in innate immune responses and catabolic pathways are associated with tumor grade. Quantitative histopathology analysis successfully stratified patients with different response to platinum-based chemotherapy (These results indicated the potential clinical utility of quantitative histopathology evaluation in tumor cell detection and chemotherapy response prediction. The developed algorithm is easily extensible to other tumor types and treatment modalities. Ovarian cancer is one of the deadliest cancers in women worldwide, causing 151,900 deaths per year , when tuκ = 0.25–0.58) [Histopathology evaluation is the gold standard for diagnosing ovarian cancer and identifying the histological types . Interpr25–0.58) –8. This 25–0.58) .Platinum-based chemotherapy is the standard treatment for patients with advanced stages of serous ovarian carcinoma , 11. PlaWith the recent advances in the reliability of whole-slide histopathology scanners and high-throughput omics profiling  coupled In this study, we developed convolutional neural network models to analyze cellular patterns and morphology in a series of patients with serous ovarian carcinoma. Our models successfully identified ovarian cancer cells, classified histology grade and transcriptomic subtypes, and predicted patients’ response to platinum-based chemotherapy. We further conducted differential expression and enrichment analyses to connect findings from our quantitative histopathology studies with the underpinning molecular pathways. Importantly, our approaches are completely data-driven and can accommodate new categories of cancers or the response to other novel treatment strategies. The development of these prediction algorithms will contribute invaluable information to precision cancer care .Five hundred eighty-seven serous ovarian carcinoma patients participated in The Cancer Genome Atlas (TCGA)  were incn = 553) [Although the pathology samples were collected at different hospitals, all samples underwent multi-omics profiling passed the required quality control requirements (including having a board-certified pathologist to review the histologic sections of the top and bottom portions of the samples to confirm the diagnoses and to make sure that the samples contained an average of 70% tumor cell nuclei with less than 20% necrosis) . RNA-seqn = 553) . Four din = 553) .AlexNet , GoogLeNn = 16) or with anaplastic grade (n = 1) were excluded from the grade classification task. The models were developed using only the training set, and all hyperparameters  were optimized by fivefold cross-validation on the training set. This approach ensured that there was no information leakage in our model training and optimization process. Thus, the test set performance served as an objective measure of the external validity of our models. The optimal baseline learning rate was 0.001 for GoogLeNet and AlexNet, 0.0005 for VGGNet. The optimal weight decay was 0.0005 for AlexNet, 0.0002 for GoogLeNet, and 0.0002 for VGGNet. The optimal momentum was 0.9 for all models, and L2 regularization was used in all convolutional neural network architectures. The final cancer detection and grade classification models were evaluated by the untouched test set. Fivefold cross-validation was used in transcriptomic subtype classification due to the smaller number of samples in the four subtypes. To ensure the reproducibility of the results, the training-test set partition and the model optimization processes were repeated three times for each classification task. The areas under the receiver operating characteristic curves were computed and compared.The TCGA dataset was randomly divided into distinct training  and test sets , in order to evaluate the classification performance objectively. Patients without grade annotation   were empn = 277), fivefold cross-validation was used to evaluate the performance of the neural networks. This cross-validation design allowed each patient to serve in the test set once and ensured that the data from the same patients were not included in both the training and test sets simultaneously, which enabled objective evaluation of the model performance. The machine learning model computed a predicted response index for each patient, and the median index observed in the training set was used to divide the patients in the test set into an early-relapse group and a late-relapse group. The log-rank test was used to examine the differences in platinum response between the two predicted groups.Platinum-free interval (PFI) is often used as a quantitative estimate of the efficacy of platinum-based chemotherapy . It is dTo connect the histopathology patterns with molecular aberrations, differential expression, enrichment, and pathway analyses were conducted using the transcriptomics and proteomics data to reveal the differences in functional omics between patients with different grades and response to platinum-based chemotherapy.P value corrected by the Benjamini-Hochberg procedure. Gene Ontology (GO) [In the tumor grade analyses, the fold change of each gene and protein between the high-grade group and the low-to-moderate-grade group was computed. Exploratory analyses were conducted using the OASISPRO tool . Since nogy (GO)  and KEGGogy (GO)  enrichmeogy (GO)  was emplIn the platinum response analyses, Spearman’s correlation coefficient between the PFI groups and the expression level of each gene or protein was calculated, and a correlation test was performed and corrected by the Benjamini-Hochberg procedure. GO and KEGG pathway analyses were conducted, and protein-protein interactions among the most relevant proteins were examined using the same tools. Similar multi-layer neural network approaches were employed to evaluate the predictive values of transcriptomics and proteomics data for the PFI groups. All statistical analyses were performed using R version 3.6.Five hundred eighty-seven ovarian cancer patients were identified from The Cancer Genome Atlas . The majWe used convolutional neural networks to detect histopathology images with cancer cells. Results showed that our method accurately identified the images with cancer cells from those without tumor, with area under the receiver operating characteristics curve (AUC) > 0.95 Fig. c. These Histopathology grade characterizes the differentiation level of serous ovarian adenocarcinoma tissue, and it is associated with the prognoses of ovarian cancer patients . HoweverP value = 6.66 × 10−16) protein-protein interactions . Interestingly, many of these differentially expressed transcripts are non-coding RNAs (such as MIR199A1 and MIR3681), suggesting a role of post-transcriptional regulations in determining the levels of tumor differentiation. We further conducted machine learning analyses that employed proteomics and RNA-seq data to predict tumor grade. Results showed that these molecular profiles only have a weak predictive value for tumor grade , indicating a cogent relation between histologic morphology and the molecular patterns underpinning the subtypes .Due to the fact that many samples received moderate-to-high scores in more than one subtype, the definition of transcriptomic subtypes is not clear-cut. To further examine the correlations between transcriptomic subtype scores and histopathology patterns, we computed the correlation coefficients between transcriptomics-based subtype scores and the histopathology-predicted subtype scores. Results showed that there are moderate correlations between the transcriptomics-based and the histopathology-predicted scores . These results suggested that subtle histological changes in the cancer cells, which are not captured by the standard histopathology annotations or pathological staging, are predictive of patients’ chemotherapy response.Platinum-based chemotherapy is the standard treatment for advanced-stage ovarian cancer patients , 11. HowP value < 0.05). These proteins form a tight protein-protein interaction network  and are significantly enriched in purine ribonucleoside metabolic processes, ATP metabolic process, and respiratory electron transport chain . However, the vast majority (99.8%) of the differentially expressed genes do not have significant differential expressions at the protein level, suggesting substantial post-transcriptional regulations of the identified transcripts. In addition, machine learning analyses showed that cancer proteomic profiles possess weak signals for predicting the platinum response groups (AUC = 0.638 ± 0.014), and the predictive power of the RNA-seq data is even weaker  are significantly correlated with patients’ PFI  in grade classification among pathologists has been reported for ovarian carcinoma [It is worth noting that modest inter-rater agreement  Transcriptomics analysis uncovered the 12 transcripts whose expression levels are associated with tumor grade. Sidebar: red indicates high-grade tumors; blue indicates low-to-moderate-grade tumors. (B) Proteomics and RNA-seq data have weak predictive power for tumor grade. Cross-validation AUC using proteomics data = 0.566 ± 0.016. Cross-validation AUC using RNA-seq data = 0.516 ± 0.005. Figure S2. Convolutional neural networks associated histopathology image patterns with the transcriptomic subtypes of serous ovarian carcinoma. (A) Features extracted by a convolutional neural network (16-layer VGGNet) are associated with transcriptomic subtypes . Triangular dots represent the mean PC1 and PC2 of the four subtypes. (B) The histopathology-predicted subtype scores are moderately correlated with the subtype scores defined by the transcriptomics data . The red line in each figure panel shows the regression line of the RNA-seq-defined transcriptomic subtype scores and the histopathology-predicted scores. Figure S3. Relations between platinum response and functional omics profiles. (A) Transcriptomic profiles of 1148 transcripts are significantly associated with the PFI of serous ovarian cancer patients. (B) Proteomics and RNA-seq data have weak predictive power for platinum response groups. Cross-validation AUC using proteomics data = 0.638 ± 0.014. Cross-validation AUC using RNA-seq data = 0.519 ± 0.003.Additional file 2: Table S1. Gene Ontology (GO) enrichment analysis results of proteins associated with the grade of serous ovarian adenocarcinoma patients. Table S2. KEGG pathway enrichment analysis results of proteins associated with the grade of serous ovarian adenocarcinoma patients. Table S3. Gene Ontology (GO) enrichment analysis results of proteins associated with platinum-free interval of serous ovarian adenocarcinoma patients. Table S4. KEGG pathway enrichment analysis results of proteins associated with platinum-free interval of serous ovarian adenocarcinoma patients.


Non-small cell lung cancer (NSCLC) is one of the most common lung cancers worldwide. Accurate prognostic stratification of NSCLC can become an important clinical reference when designing therapeutic strategies for cancer patients. With this clinical application in mind, we developed a deep neural network (DNN) combining heterogeneous data sources of gene expression and clinical data to accurately predict the overall survival of NSCLC patients. Based on microarray data from a cohort set (614 patients), seven well-known NSCLC biomarkers were used to group patients into biomarker- and biomarker+ subgroups. Then, by using a systems biology approach, prognosis relevance values (PRV) were then calculated to select eight additional novel prognostic gene biomarkers. Finally, the combined 15 biomarkers along with clinical data were then used to develop an integrative DNN via bimodal learning to predict the 5-year survival status of NSCLC patients with tremendously high accuracy . Using the capability of deep learning, we believe that our prediction can be a promising index that helps oncologists and physicians develop personalized therapy and build the foundation of precision medicine in the future. The most common NSCLC subtypes are adenocarcinoma (ADC), squamous cell carcinoma (SQC), and large cell carcinoma. Although the overall 5-year survival rate of patients diagnosed with stage I ADC was 63%, nearly 35% of patients relapsed after surgery with a poor prognosis2. Adjuvant treatments have been considered ideal for ADC patients with the highest risk of recurrence or death to increase survival rates3. Therefore, prognostic stratification is crucial for categorizing patients to help doctors make decisions on therapeutic strategies.Lung cancer is the worldwide leading cause of cancer-related mortality, with non-small cell lung cancer (NSCLC) accounting for approximately 85% of all lung cancer patients4. Previous studies have shown the importance of biomarkers for NSCLC, such as EPCAM, HIF1A, PKM, PTK7, ALCAM, CADM1, and SLC2A1, which were used as a single biomarker for predicting prognostic condition or metastasis11. However, cancer is a systemic disease with complicated and illusive mechanisms that often involves multiple genes and cross-talk between pathways. Therefore, extending our understanding of NSCLC via the single gene biomarkers by studying the interactions between genes is essential for more accurate prognostic prediction.Recently, researchers have developed predictive methods based on gene expression profiles to classify lung cancer patients with distinct clinical outcomes, including relapse and overall survival12. In addition, predicting cancer prognosis can be improved by appropriately modeling the interactions between biomarkers compared with the single biomarker approach13. Deep learning has seen unprecedented success in many fields, such as image recognition14, speech recognition15, and biology16. A deep neural network (DNN) is composed of non-linear modules, which represent multiple levels of abstraction17. Each representation can be transformed into a slightly more abstract level, leading to even more involved interactions among features. As a result, deep learning algorithms can extract high-level abstractions from different types of data sources and provide superior performance compared with traditional machine learning methods18. Thanks to the representation of features in hidden layers, a DNN can easily combine the networks for different modalities. Therefore, we aimed to propose an integrative DNN that combines both gene expression and clinical data to improve prognostic prediction for NSCLC patients.Machine learning algorithms are powerful tools that apply input features (biomarkers) to capture the complicated interdependencies between these features to accurately predict clinical outcomeset al. trained their convolutional neural network (CNN) with the whole-slide images for NSCLC patients obtained from The Cancer Genome Atlas (TCGA) to classify them into adenocarcinoma (LUAD), squamous cell carcinoma (LUSC), or normal lung tissues. They suggested that deep learning models can help in detecting cancer subtypes and gene mutation19. Bo He et al. performed feature extraction on computed tomography (CT) images of NSCLC patients and built a random forest classifier for predicting the survival status20. Note that both their prediction models dealt with image data while we focused on gene expression and clinical information. We believe that their works and ours are complementary to each other. From the gene expression data, we selected a set of 15 prognostic biomarkers that can identify patients with adverse prognosis in the early stages. We can draw biological insights from the molecular mechanisms among them for a better understanding of the formation and metastasis of cancer cells. We motivate our systems biology feature selection process in the paragraphs below.During the flourishing development of deep learning algorithms in biomedical applications, two important works have drawn our attention. Coudray 21. However, while traditional feature selection methods are often based on the statistical or predictive performance of the patient dataset, biological concepts were rarely considered when isolating potential gene features. Therefore, the predicted gene features to apply and improve therapeutic strategies for cancer patients are limited.Due to the relative small sample size of patient data compared to the large number of genetic features, scientists have assiduously focused on feature selection algorithms that aim to obtain a subset of significantly representative features22. There is increasing interest in applying systems biology approaches to identify cancer-associated genes as feature selection strategies23. In this study, we established a systems biology approach for NSCLC patients, which identified eight novel survival-related genes based on seven previously well-known biomarkers. The combined 15 biomarkers were used in the following DNN model to predict the survival status of patients. Through a bimodal deep learning approach, we combined gene expression profiles and clinical data sources to predict the 5-year overall survival of NSCLC patients. We believe our significant improvements to predicting prognostic outcomes for lung cancer patients may help oncologists and physicians make accurate and precise decisions on appropriate treatment for individual patients, which may build the foundation for future personalized therapeutic strategies.Systems biology is computational and mathematical modeling of complex biological systems that has been widely appliedWe integrated systems biology and deep learning approaches to predict the survival status of NSCLC patients. In addition, the systems biology approach was specifically used to identify prognostic biomarkers (gene features). The selected prognostic biomarkers were used as input features for our DNN prediction based on the 5-year survival of lung cancer patients. Moreover, we further integrated their clinical background via an integrative DNN model to improve the performance of our prediction. The schematic of our strategy is shown in Fig. EPCAM, HIF1A, PKM, PTK7, ALCAM, CADM1, and SLC2A1)25. Based on each of these seven biomarkers, we separated the 614 ADC patients into biomarker- and biomarker+ subgroups according to expressions by StepMiner26 . We selected the top 30 genes as candidates for each well-known NSCLC biomarker. The details of the PRV for each biomarker are listed in the COPS5, CUL1, CUL3, EGFR, ELAVL1, GRB2, HSP90AA1, NRF1, PPP1CA, RNF2, RPA2, SIRT7, and SUMO1 were overlapped in all seven lists. By filtering these genes based on significance for survival (p < 0.01), the final eight prognostic biomarkers, CUL1, CUL3, EGFR, ELAVL1, GRB2, NRF1, RNF2, and RPA2, were identified. Combined with seven well-known biomarkers, a set of 15 prognostic gene biomarkers were adopted in the following analyses.We constructed interaction networks for both biomarker- and biomarker+ subgroups, resulting in a pair of gene interaction networks for each NSCLC biomarker. From each pair of interaction networks, genes were ranked by PRV 30. Due to the imbalance of labels across the entire dataset , the classifiers tended to classify the patients as alive. In this case, if a naive classifier classified all patients as alive, it still reached an accuracy of 0.6953. On the other hand, AUC is a much better performance metric than accuracy that takes class imbalance into account and is a thresholdless metric . In a previous study, a clinical prognostic index (CPI) risk score was defined by using clinical data30. We again noted that the DNN  achieved a significantly higher AUC than CPI  and RF   as the activation function and Nadam as the optimizer. To verify the effectiveness of the DNN on survival classification, we compared the performance with other well-known classifiers, such as K-Nearest Neighbors (KNN), Random Forest (RF), and Support Vector Machine (SVM) by using the same prognostic biomarkers as input features31, it was difficult to integrate two heterogeneous data sources. The DNN has the flexibility to integrate heterogeneous data sources by merging the hidden layers of the neural networks. The application of a DNN for integrating different types of data sources has been successful in handling audio and video data sources32; however, it is relatively new to integrate gene expression and clinical data sources. Therefore, we proposed using a DNN for data integration  and clinical data (CPI) to define a composite risk model (CRM). The threshold was chosen from the median of training sets for the CRM. We compared the performances of our proposed integrative DNN with RF and CRM, as shown in Fig. Gentles 33. To compute these metrics, we need to find suitable cut-off points for reclassification. We used the Youden index34 to select cut-off points for reclassification. Such reclassification was conducted on both DNN with only microarray data and the integrative DNN with both microarray and clinical data. We calculated the cut-off points as 0.4396 and 0.4008 for the two DNNs, respectively. Similarly, we also calculated new cut-off points as 0.32 and 0.34 for the microarray RF and integrative RF, respectively. It is to note that these new cut-off points are smaller than 0.5 , indicating that the number of patients classified as deaths increased after reclassification.To obtain an overall picture of the performance comparison, we also considered precision, recall, and F1-score. The F1-score is also called the F1 measure and it considers both the precision and the recall by computing the harmonic mean of precision and recallTo confirm the effectiveness of reclassification, we evaluated their performances in terms of accuracy, precision, recall, and F1-score for the microarray DNN and RF and the integrative DNN and RF Fig. . Note thMethods) and proportional-hazards model were used to evaluate the results for both DNN models with/without reclassification  and low risk (which was predicted as survived) by our proposed microarray and integrative DNNs, respectively. KM analysis . The patients for E-MTAB-923 (n = 90) are more balanced  than the original cohort. We compared our proposed DNN models with RF in terms of AUC and accuracy Fig. . InteresWe divided the independent validation set into two risk groups and used the proportional-hazards model Fig.  and KM e35. The strategy for the filter method was based on ranking the performance or mutual information of each feature, which was widely used in many domains because of their simplicity. The wrapper method worked as a black box by selecting features based on the performance of the classifier, which often showed great computational complexity and only worked well in limited classifiers. The embedded method was similar to the wrapper method; however, it focuses on reducing the amount of computational time required. All of the methods above aimed to select the top-ranked features purely based on statistical or predictive performance and disregarded the biological meaning of the gene features. Therefore, the selected gene features often lack biological insights and cannot be applied to further experimental validation.Traditionally, feature selection methods can be categorized into three different types: the filter, wrapper, and embedded methods36. Overexpression of nuclear ELAVL1 in NSCLC patients was correlated with lymph node metastasis and may serve as a potential diagnosis biomarker37. In addition, while nuclear ELAVL1 was important in cancer progression, cytoplasmic ELAVL1 was also identified as an independent prognostic factor for survival in NSCLC38. EGFR is a well-known transmembrane protein involved in controlling cell survival and tumorigenesis in many malignancies, including NSCLC39. Moreover, mutated and overexpressed EGFR has been reported in a myriad of NSCLC case studies40. Interestingly, while EGFR was recognized as a potential therapeutic target of NSCLC, its binding adaptor, growth factor receptor bound protein 2 (GRB2), was shown to be a stabilized EGFR and co-activated downstream to the MAPK/ERK pathway40. Among the Cullion family, Cullin 1 (CUL1) is one of the scaffold proteins in E3 ubiquitin ligase involved in cancer progression. High expression was correlated to patient survival rate, which was identified as an independent prognostic factor in NSCLC41. There are eight members in the Cullin family. In addition to CUL1, CUL3 is known as a scaffold protein in the ubiquitin-proteasome system and contributes to cellular regulation, such as cell cycles, protein trafficking, and stress response, which are common tumorigenesis phenomenon when mutated. Furthermore, one substrate adaptor of CUL3, kelch-like ECH-associated protein (Keap), was first identified as an inhibitor of transcriptional factor Nf-E2-related factor 2 (Nrf2) and played important roles in anti-oxidation stress and cell defense during cancer suppression42. Although there is only limited evidence addressing the function of Nrf1 and Nrf2 in prostate cancer43, the genetic and functional conservation between them identifies their active roles in lung cancer progression. Ring finger protein 2 (RNF2), a member of the group II polycomb group (PcG) protein, was highly expressed in many types of human malignancies44. Despite the role RNF2 plays in biological processes in cancers via its diverse mechanisms, it highlights the potential oncogenic activity of RNF2 on NSCLC. Replication protein A 2 (RPA2), a single-strand DNA binding protein, processes DNA metabolism in response to DNA damage-associated replication arrest45. It has also been considered a potential therapeutic target and prognostic indicator for colon cancer, as shown by differences in its immunohistochemical expression between cancer patients and controls46. To summarize, each biomarker we identified showed great potential for being a prognostic biomarker based on its biological background.To address this issue, we demonstrated a systems biology approach to select biologically meaningful gene features and identify eight prognostic biomarkers for NSCLC patients. These prognostic biomarkers  were identified by overlapping seven computed PRV lists. Among the eight prognostic biomarkers, most of them have been reported and directly relate to NSCLC in previous studies. ELAVL1 is a well-known RNA-binding protein associated with multi-carcinogenesis, such as large cell lymphoma and glioma, by modulating RNA stability47. Using GO biological process enrichment analysis, we found that CUL1, CUL3, RNF2, and RPA2 overlap in their mitotic cell cycles, which has important biological implications in tumor development. We also analyzed these prognostic biomarkers by investigating their interdependencies using STRING (https://string-db.org/)48 . In addition, the interactions among GRB2, EGFR, HIF1A, and SLC2A1 were also highly correlated with cancer in the KEGG “Pathways in cancer” (pathway #5200).We further analyzed eight prognostic biomarkers using GSEA )48 Fig. . The intOne reason our proposed PRV feature selection method performs well in not only the test set, but also the independent validation set, is due to the robustness emphasized in our systems biology approach. Several steps were taken to ensure the robustness of our feature selection method. Firstly, we used gene expression data from six different datasets and selected only genes with the same probe number. Secondly, our PRV feature selection method was applied to each of the seven well-known NSCLC biomarkers in our data cohort. We then constructed seven different paired interaction networks to obtain seven PRV lists. The eight prognostic biomarker genes were selected via overlapping these seven PRV lists. In other words, we only selected the prognostic biomarkers that appeared in all seven PRV lists. This selection process guarantees robustness in our method based on the superior predictive performance of our proposed integrative DNN. In the future, we can not only choose well-known biomarkers from the literature, but use other clinical outcomes to group patients for selecting features.21. In recent years, some researchers have begun to consider combining gene expression and clinical data to make such predictions31. In this study, we applied the concept of bimodal learning to construct an integrative DNN where two heterogeneous modalities  were integrated for predicting ADC patient overall survival. By using two modalities, the integrative DNN approach is capable of providing the missing information left by the other observed modality. Compared with our microarray DNN, we observed an increase in AUC and accuracy from the integrative DNN. We also demonstrated improved prognostic performance for survival analysis. This highlights the benefit of integrating microarray and clinical data via our integrative DNN approach. A good bimodal learning model possesses certain properties. The joint representation should be similar in its feature space, implying that the two heterogeneous data sources correspond to similar concepts. We use two DNNs to extract features from each data source and jointly train them for the complete bimodal learning model in the combined layers. From this, the combined model better integrates two data modalities into a joint representation. Furthermore, image data such as CT-scans and whole-slide images were utilized for training deep learning models to provide tissue classification or patient overall survival prediction20. We believe that our work and theirs are perfect complements to each other. For future work, we could use more than two types of data sources to construct a multimodal learning model for even more accurate prediction.In most of the existing work, only gene expressions were used as features for training classifiers, with the aim of predicting various disease outcomeshttp://www.ncbi.nlm.nih.gov/geo) with accession numbers GSE19188, GSE29013, GSE30219, GSE31210, GSE37745, GSE50081 with the same platform: the human Affymetrix HG-U133 Plus 2.0 platform (GPL570). Six independent GEO datasets were merged into one cohort with 614 patients. We separated 256 patients as the training set, 85 patients as the validation set, and 171 patients as the test set, all of which have complete clinical data. E-MTAB-923 was taken as an independent validation set with the same platform. Table The open access microarray data were downloaded from the National Center for Biotechnology Information (NCBI) Gene Expression Omnibus (GEO) database 50. According to the constructed interaction networks, we defined prognosis relevance values (PRV) to measure the difference between biomarker+ and biomarker- interaction networks for each gene  via the StepMiner algorithmA DNN is composed of one input and one output layer, with many hidden layers in between representing multiple levels of abstraction. Each hidden layer is composed of many neurons. Deep learning has been successfully applied to supervised learning for combining different modalities. For our dataset, we not only used microarray data, but also clinical data. Here, we combined two DNNs  by merging their output layers and further concatenating several hidden layers before reaching the final prediction. The integrative DNN can be expected to benefit from combining the two separate data sources.We performed 10-fold cross-validation to select hyper-parameters for the benchmark models. For KNN, we selected Euclidean distance as the distance measure and considered at most 30 nearest neighbors. We utilized RBF kernel for SVM with different levels of L2-regularization and different gamma values for the kernel. For RF, we restricted the maximum depth as one-third of the dimension of the input data dimension and varied the number of trees used. The detailed searched optimal parameters were summarized in Table For bimodal DNN, we varied the number of neurons in the two DNNs for microarray and clinical data and observed their cross-entropy losses in the validation set Table , as illu51 curve is a graphical plot that illustrates the diagnostic ability of a binary classifier system created by plotting the true positive rates against the false positive rates at various threshold settings. The area under the ROC curve has been used for evaluating how well the model performs51. Under some conditions, we can achieve a better reclassification performance by adjusting the cut-off points. In this study, the cut-off points were determined with the help of Youden index34, which is a frequently-used summary measure of the ROC curve. In previous classification tasks, we classified a patient by comparing the probabilities of patient survival outcomes with 0.5 for all classifiers. To further improve predictive performance, new cut-off points determined with Youden index were used for reclassification.The receiver operating characteristic52 . We applied the cox proportional-hazards model to analyze the relationship between the prognostic genes for survival53. The hazard ratios (HR) and confidence intervals (CI) were reported.In our study, overall survival time was calculated from the date of surgery to the date of death. We predicted the survival status of patients within 5-year. Therefore, patients were treated as alive patients when survival time was greater than five years. Survival curves were demonstrated based on a Kaplan-Meier estimation for five years and compared using a log-rank testSupplementary Materials.


Patient-Net (P-Net) in which biomolecular profiles of patients are modeled in a graph-structured space that represents gene expression relationships between patients. Then a kernel-based semi-supervised transductive algorithm is applied to the graph to explore the overall topology of the graph and to predict the phenotype/clinical outcome of patients. Experimental tests involving several publicly available datasets of patients afflicted with pancreatic, breast, colon and colorectal cancer show that our proposed method is competitive with state-of-the-art supervised and semi-supervised predictive systems. Importantly, P-Net also provides interpretable models that can be easily visualized to gain clues about the relationships between patients, and to formulate hypotheses about their stratification.Methods for phenotype and outcome prediction are largely based on inductive supervised models that use selected biomarkers to make predictions, without explicitly considering the functional relationships between individuals. We introduce a novel network-based approach named  State-of-the-art methods for these problems are largely based on inductive supervised models that use sets of selected biomarkers, usually represented as vectors, to predict the phenotype or outcome of interest , without taking into account the relationships between individuals.Phenotype and outcome prediction using sets of selected biomarkers are well-established prediction tasks in the context of computational biology, including different prediction problems ranging from the response to a specific drug17. These methods mainly used unsupervised approaches and hence have been not specifically designed and are not appropriate for phenotype/outcome prediction problems. Recently a few works proposed semi-supervised “network-based” approaches for the prediction of the phenotype/outcome of patients, on the basis of their bio-molecular profiles (e.g. gene expression of genotypic profiles)19, including also methods able to integrate multiple sources of omics data20, and methods based on Supervised Random Walks21, specifically modified for the classification of tumors22.Several works proposed “network-based” methods by constructing graphs of patients, in order to discover the underlying structure of the data n-dimensional set of biomarker values , and edges represent similarities between the biomarkers of a pair of patients. Hence, this “patient-space” differs from the classical “biomarker-space”, where nodes represent biomarkers and edges similarities between biomarkers and not between patients24. More precisely, we construct networks of patients on the basis of their gene expression similarities (e.g. by considering their expression profiles), and then we apply a semi-supervised transductive method to predict their phenotype or clinical outcome. The algorithm leverages local learning strategies, by considering the direct neighbors of each node in the “patient network”, as well as global topological characteristics of the net through the adoption of appropriate graph-kernels25.In this work, we introduce a novel network-based method for modeling in the “patient space”. In this context the nodes of the network represent patients through an P-Net (i.e. Patient-Net), uses the available a priori knowledge about the phenotype/outcome of patients and their biomolecular similarity, to assign a score and to rank or to classify patients according to the phenotype/outcome under study.Our method, that we named 24. In fact these methods, by exploiting the topological relationships between nodes in gene and protein networks, can select markers for specific diseases32, but cannot be directly applied to predict the outcome or phenotype of patients, which is instead the main aim of P-Net.Importantly, P-Net, differently from classical inductive supervised models, is not a mere “black box”, since the visual inspection of the network can unravel further characteristics of the patients under study. Indeed, by exploiting a cytoscape.js interface, and some rendering options available in the package, it is possible to explore and graphically analyze the obtained graph. It is worth noting that P-Net is also completely different from semi-supervised network-based methods where genes or proteins represent the main object to be studiedP-Net is competitive with supervised and semi-supervised state-of-the-art methods for phenotype/outcome prediction. A fast and efficient implementation of P-Net is publicly available from GitHub (https://github.com/GliozzoJ/P-Net). Moreover, by using a cytoscape.js interface, our method offers an intuitive way to explore and graphically analyze the patient network.A large set of experiments with real biomolecular data, including Pancreatic, Breast, Colorectal and Colon cancer patients, shows that P-Net) is a semi-supervised algorithm able to assign to each patient a score related to its odds to show a specific phenotype . The predictor is constructed from patients’ molecular profiles using a graph G = <V, E>, where the set of vertices V corresponds to patients and the set of edges E to relationships between them . From this similarity network among patients, a graph kernel  is applied to obtain weighted edges aware of the global topology of the network34, while edges with low weight are removed through a cross-validation procedure. Finally, a scoring system, by exploiting a subset VC ⊂ V of labelled patients belonging to the subgroup C of interest (e.g. patients having poor prognosis or responsive to a specific treatment), assigns a score to each patient on the basis of the labeling of its neighborhood. The resulting ranking of patients can be used to rank or classify them by their likelihood to show a specific C phenotype.Network-based ranking of patients with respect to a given phenotype  is obtained from the columns (patients) of M , but other measures  can be used instead. We applied filtered Pearson correlation because other correlation metrics achieved similar or worse results. W can be seen as the adjacency matrix of the weighted graph G, where the edge weights represent the bio-molecular similarity between patients.Computation of a kernel matrix from the patient similarity matrix The kernel matrix K is derived from the similarity matrix W , in the original graph. This means that two nodes may be directly connected in the new graph even if they were not in the original graph , but for our experiments we chose the random walk kernel38, since it showed on average better results: I is the identity matrix, D is the “degree” diagonal matrix with elements dii = ∑jwij and a is a value larger than 2. A p-step random walk kernel can be obtained by simply multiplying K by itself p times. Intuitively, we can think at graph-kernels as functions able to enrich the similarity between nodes in the transformed kernel space, since the novel edge weights of K is determined by the overall topology of the network 39. It is worth noting that a graph kernel implicitly induces a new non linear similarity measure between patients, since the novel weights of the computed Gram matrix (i.e. the similarity patient matrix resulting from the application of the kernel) take into account both the topology and the metric distance between the nodes/patients to construct a novel “enriched” similarity graph  on the training set (see Section “Implementation of P-Net” for more details).Ranking of patients with score functions The score functions associate a score to each node of the network on the basis of the labeling of its neighborhood and the weights of its incoming edges , derived as described in34, compute their score respectively considering the average, the nearest and the k nearest labeled neighbours. While these scores exploit only weights coming from the labeled neighbours, the remaining score functions STOT, SDiff and SDnorm  allow us to exploit the information that comes from the whole neighborhood of the investigated node i, including both nodes i ∈ VC and i ∉ VC. In this way the labeling of each node depends on both “positive” and “negative” annotations of the neighborhood nodes.es Table . SAV, SN Figure P-Net provides a score to rank patients, but by setting a threshold τ we can obtain a classifier: patients having a score larger than τ are classified as positive and the others as negative. In our experiments, we set the threshold according to the predicted best accuracy by a fast internal leave-one-out procedure on the training set (see Section “Implementation of P-Net”), and then we applied the selected optimal threshold on the test set. Additional information about P-Net and its implementation are available in the next Section and in the Section “Efficient implementation of P-Net” in the Supplementary P-Net and the selection of the network threshold τ (see “Filtering of the kernel matrix” in the “Methods” Section), can be performed through cross-validation or leave-one-out (LOO) procedures. In particular, to obtain an unbiased evaluation of the generalization performance and an unbiased network filtering, we performed a double leave-one-out procedure: (a) an internal LOO to select the network threshold τ; (b) an external LOO to estimate the generalization capabilities of the algorithm.The analysis of the generalization performances of n2 times, where n is the number of patients. To avoid this computational burden, we propose an efficient implementation of the P-Net LOO, that requires only a unique run across patients.Unfortunately, the classical implementation of the double LOO requires to run the algorithm Theorem 1: Having a kernel matrixKobtained from the weighted adjacency matrixWof a graphG = <V, E >, with verticesv ∈ Vdenoted withi ∈ { 1, …, |V|}, andN(i) = {j | j ∈ V, kij > 0} the neighbourhood of nodeiin the graphGtransformed according to the kernelK, when a leave-one-out procedure is applied with theP-Netalgorithm, the following fact holds: Proof:kii = 0 ⇒ node i is left out.I. i, kii = 0, one of two possible conditions holds:i ∉ N(i), kii is not included in the computation of the score function S(i), where S is one of the scores functions described in Section “Methods”.if i ∈ N(i), kii is considered in the score function, but by hypothesis kii = 0if If ∀ i is computed independently of the labeling of the node i, that is, in other words, the node i is left out.In both cases the score of the node kii = 0 ⇐ node i is left out.II. i is left out, even if i ∈ N(i), kii is not used in the computation of the score S. Equivalently we can set kii = 0 and then we can use kii in the computation of S(i), since it has no effect on the computation of the score.□If i is used only in step 5 of the algorithm , and by setting kii = 0 we automatically exclude that node by the computation of its score, i.e. we hold-out that node from the training set. As a consequence of the above theorem, to perform the LOO it is sufficient to set to 0 the diagonal of K and then run one time P-Net across all the nodes of the graph G. For instance, to select the optimal threshold τ, we can simply set to zero the diagonal of the kernel matrix K and then apply a unique run of P-Net . The Pancreatic cancer dataset is available from ArrayExpress Archive (accession number E-MEXP-2780).P-Net with both supervised inductive methods and semi-supervised network-based algorithms for outcome/phenotype prediction. Among supervised inductive methods we selected Support Vector Machines (SVMs) and Random Forests since they showed superior performances for these tasks42.We compared 43), as well as other univariate feature selection methods, to select the most relevant features, and as a second step a supervised algorithm to predict the outcome using the previously selected features26. Net-Rank enhances the Google Pagerank-learning approach44 by using Laplacian regularization and an additive margin: for PageRank the nodes are web pages and the edges are hyperlinks between them; for NetRank the vertices of the network are genes and the edges are relationships between them (e.g. protein-protein or transcription factor-target interactions). Then a Support Vector Machine is trained with the selected features/biomarkers to predict the outcome of patients26.We also considered an approach that applies as a first step a network-based method  and the discrepancy between predicted labels of connected nodes in the graph are jointly minimized. Instead, in our semi-supervised approach, a graph kernel is firstly applied to capture the overall topological characteristics of the graph, and then a score function is applied over the transformed graph to rank patients.Finally, we compared our method with an approach developed by Park 26 by using the same experimental set-up, metrics and datasets used by each of them. We also applied the same filtering approach adopted in the first step of the P-Net algorithm  to select the genes significantly associated with the patients’ phenotype/outcome, and we performed gene set enrichment analysis45 to uncover pathways and Gene Ontology terms overrepresented in the set of selected genes, to characterize the underlying disease mechanisms.We compared P-Net with each of the above cited methodsThe data sets used in the experiments are publicly available and are schematically described in the next section.P-Net method on several microarray gene expression cancer data sets.We applied our novel et al.26 includes 30 patients, split into two groups (good prognosis and poor prognosis) according to their survival time . The Breast, Colorectal and Colon cancer patients were originally analyzed by Park et al.19. Breast cancer data are composed of 189 invasive breast cancer patients, divided into the classes “high” and “low” risk of recurrence; Colorectal cancer data includes 177 patients and Colon cancer 213, both divided into the groups “recurrence”, “no recurrence”. All these data sets include also “unlabeled” patients, i.e. patients having no prognostic or phenotype associated information. Table The Pancreatic ductal adenocarcinoma data set from Winter P-Net, using Breast, Colorectal and Colon cancer data, with Park et al. semi-supervised graph-based method that minimizes both the classification error and the “internal consistency” of the predicted labeling19. Moreover we compared P-Net on the same data sets with three other supervised inductive methods, i.e. SVM, Random Forest and Naive Bayes. In the above mentioned experiments we employed the same experimental set-up, based on 10-fold cross-validation, as proposed in19. With these data sets P-Net achieved the best results with a Random Walk Kernel 1-step and the differential or the differential normalized score function   Fig. . MoreoveOC) Fig. .Figure 4P-Net are competitive with the other compared algorithms  and other simpler univariate feature selection methods26. We did not use non-linear SVMs since in26 the authors reported that more complex kernels did not lead to better results.We compared et al.26 used NetRank and a series of univariate feature selection methods: fold change, t-statistic, Pearson and Spearman rank correlation coefficients, SAM  method46, and random selection of genes as control, to select the genes most correlated with the survival time of a patient. Then the selected top ranked genes were used to train a Support Vector Machine (SVM) in order to predict patients with poor (PP) or good (GP) prognosis.Winter The generalization performances were evaluated through a Monte Carlo cross-validation technique (MCCV). More precisely we randomly split the data in training and test set 1000 times. At each round of the splitting procedure, we performed feature selection through the t-test using only the data of the training set. Then we constructed a network of patients using only the selected genes and we filtered the edges according to the P-Net accuracy on the training set by the efficient leave-one-procedure described in the Section “Implementation of P-Net”. The score threshold to optimally separate poor and good prognosis patients is computed on the training set and the generalization performances are finally evaluated on the separated test. A detailed step by step description of the experimental set-up is available in the Section “Experimental set-up with Pancreatic cancer data” in Supplementary P-Net used the Random Walk Kernel, and the model with p = 8 steps, the Nearest Neighbour score function and a set of genes selected according to the t-test achieved the best results. Fig. P-Net achieves better results than the other methods for each training set size, except with 24 patients, where it obtains the second best result. In all cases the difference is statistically significant (t-test at 0.05 significance level). Detailed results showing average accuracy and standard error of the mean for all assessed methods are available in Table P-Net. Results show that, even when selecting a relatively large number of biomarkers associated with the poor prognosis (1000 genes), P-Net significantly outperforms the SVM for any size of the training set  and the standard hypergeometric test to discover overrepresented KEGG pathways , as implemented in the R package GOstats45. Since the lists of overrepresented GO terms for breast and colorectal cancer were very large, we used REVIGO49, a clustering algorithm based on semantic similarity measures, to obtain a representative subset of more interpretable terms.Linking predictions to the underlying disease mechanisms is of paramount importance, and to this end we applied classical gene set enrichment analysis to uncover pathways associated with patients’ phenotype and outcome, starting from the set of genes selected as differentially expressed at the first step of the https://www.qiagenbioinformatics.com/products/ingenuity-pathway-analysis) software50  in order to further investigate enriched signaling and metabolic pathways related to our selected genes.Additionally, the obtained gene lists were analyzed by Ingenuity Pathway Analysis , i.e. the accumulation of unfolded/misfolded proteins in the reticulum, may arise due to glucose deficiency, hypoxia, calcium imbalance and oxidative stress. ERS is associated with tumor development and metastasis in breast cancer since the high cell proliferation rate in tumors causes hypoxia, nutrients starvation and higher ROS production53. In particular, increased ROS production in cancer cells arises from oxidative phosphorylation, oxygen metabolism and NADPH oxidase functions leading to oxidative stress, described also in breast cancer54.364 genes were selected by t-test and used for further functional analysis. Among overrepresented KEGG pathways we found “Proteasome” which is a complex dedicated to protein catabolism involved in several cellular functions . Recent studies suggested that deregulation of the ubiquitin-proteasome pathway may have a permissive role in breast cancer development, and drugs which target this pathway are currently in clinical trial55 while apoptosis escape is one of the hallmarks of cancer56. p53 is a tumor suppressor able to avoid cancer development through block of the cell cycle, programmed cell death, repair of damaged DNA and senescence. p53 is mutated in 20-30% of breast cancers and it is generally silenced by loss of upstream/downstream mechanisms57. Another interesting term is “cellular oxidant detoxification” which is consistent with the presence of oxidative stress in breast cancer cells58.Considering GO:BP terms, we found some common biological processes known to be involved in tumor development, such as “cell cycle phase transition”, “apoptotic signaling pathway” and “signal transduction by p53 class mediator”. Defects in the cell cycle checkpoints are associated with breast cancer molecular subtypes59. The analysis also revealed that “Cell Death and Survival” and “Cellular Growth and Proliferation” are the molecular and cellular functions most significantly overrepresented in the recurrence-associated genes . These findings are consistent with GO:BP hypergeometric test and the selected genes encoding for cyclins , enzymes involved in ubiquitination  and the transcription factor STAT1. These results are in line with those obtained by other groups which analyzed GSE2990 dataset62. In particular, Sotiriou and coworkers reported that genes predicting high risk of recurrence are mainly involved in cell cycle regulation and proliferation60. Of note, STAT1 was identified as a breast cancer recurrence gene also by Park and colleagues62 and it was associated to resistance to endocrine therapy and to distant metastasis-free survival by Huang and others61.Finally, from IPA analysis mTOR-p70S6K signaling resulted enriched in the Top Canonical Pathways, in agreement with its frequent deregulation found in breast cancer, often associated to drug resistanceβ-oxidation, where lipids are used by fast growing cells to support various activities like membrane formation and signaling. Abnormal expression of genes involved in fatty acid metabolism was found by different studies in correlation with metastasis, drug resistance and relapse63. Notably, many GO:BP terms found in our analysis are consistent with aberrant lipid metabolism . Considering “Propanoate metabolism” and “Pyruvate metabolism”, most tumor cells highly depend on aerobic glycolysis rather than mitochondrial oxidative phosphorylation to obtain energy, known as “Warburg effect”, and this disregulates the above mentioned metabolic paths64. Moreover, GO:BP analysis revealed multiple terms related to “interleukin mediated signaling pathway” . For instance, Kuniyasu et al.65 associated the production of IL-15 in colon cancer cells with proliferation, resistance to apoptosis, metastasis and angiogenesis.A total of 55 genes were found associated with recurrence in colon and colorectal cancer. In colon cancer, KEGG pathway analysis highlighted dysregulations in few metabolic pathways, such as “Fatty acid biosynthesis”, “Propanoate metabolism” and “Pyruvate metabolism”. Lipid accumulation was observed in many cancer types , caused by an imbalance between fatty acid biosynthesis and in vivo and contributes to the development and progression of this tumor66. Among biological processes consistently overrepresented in our analysis, we found GO:BPs related to the β-catenin destruction complex , nitric-oxide biosynthesis , inflammasome . The β-catenin destruction complex was recently found disrupted in colorectal cancer67, possibly enabling the migration of β-catenin to the nucleus and the subsequent transcription of target genes. Aberrant stabilization of the β-catenin due to mutations in the destruction complex was associated with various cancers68. Regarding the biosynthesis of nitric-oxide, colorectal cancer is strongly associated with chronic inflammation and nitric-oxide produced by NOS2 is associated with the initiation and progression of the disease. Moreover, over-expression of NOS2 is correlated with poor outcome69. Finally, the activation of NLRP3 inflammasome, with consequent production of IL-1β, IL-18 and procaspase-1 activation which induces pyroptosis (i.e. a form of programmed cell death), showed an anti-tumoral effect preventing/inhibiting colorectal cancer development70.In colorectal cancer, pathway analysis showed that “Pathogenic Escherichia coli infection” is associated with the disease. Indeed, in literature some strains of E. coli are known to produce a genotoxic metabolite (colibactin) that alkylates DNA 71. Among the deregulated genes involved in the EMT signaling, CTNNB1, as well as JAK1 play key roles in colon cancer progression: CTNNB1 through the activation of Wnt/β-catenin signaling, and JAK/STAT pathway by regulating cell survival and proliferation, differentiation and migration72. Our findings are in line with previous studies showing that CTNNB1 and JAK family members are colon cancer recurrence-specific genes and prognostic biomarkers74. Notably, we also identified NLRC4 gene, a member of NOD-like receptors in the “Inflammasome Pathway”, as a novel gene related to colon cancer recurrence. Previous studies have shown that NLRC4 takes part in inflammation-induced colon cancer tumorigenesis and participates in anti-apoptotic pathways downstream of p5375. In addition, a recent study showed that NLRC4 mediates M2 TAM infiltration and angiogenesis through VEGF production in metastatic colon cancer76.Functional annotation of selected genes by IPA revealed that 13 out of 54 genes are closely associated to “Cell Growth and Proliferation” and “Cell Death and Survival” . The top canonical pathways enriched for these genes were “Epithelial-Mesenchymal Transition (EMT)”, “Inflammasome” and “iNOS signaling” , or might suggest stratifications of patients on the basis of their similarity or predicted phenotype.Supervised inductive models in most cases are “black boxes” not suitable for interpreting and explaining the obtained results. On the contrary P-Net with the Pancreatic ductal carcinoma data set obtained by adapting to our context a graph visualization tool developed in our laboratory for visualizing biomolecular interactions . In the figure the patient ground truth is shown by the node shapes  while the node color shows the prognosis predicted by P-Net. The graphical representation shows that the group of good prognosis patients (highlighted in green) are all correctly predicted by P-Net. A second group (highlighted by a red ellipse) includes poor prognosis patients predicted by our method, some of which are wrongly classified. However, though being labeled as “good prognosis”, the “misclassified patients” are really borderline patients: their survival time is very close to the cut-off of 17.5 months selected in26 to separate good from poor prognosis patients. As another example, Fig. As an example of this opportunity, Fig. 78 and different visualization perspectives and interactions, can allow domain experts to identify interesting visual explanations of the achieved classification, which would not be so evident in a textual representation of the graph. Different approaches have been proposed along this direction, such as77 in the context of biomolecular interactions and79 for the visualization of phenotype similarities. These tools offer different functionalities for clustering graph nodes according to different similarity measures, for changing the color and the shape of the nodes according to different parameters, for changing the layout perspectives and pointing out different graph properties. All these functionalities aid experts to uncover hidden information represented in the graph and to obtain further clues about different patients.Thus, the graph visualization, eventually coupled with graph clustering algorithmsP-Net is a semi-supervised transductive approach where the genetic similarities between patients guide the process of outcome prediction and provide a graph representation of the biomolecular similarity between patients. P-Net analyses the biomolecular profiles of patients by firstly constructing a similarity graph between them and then applying a graph kernel, which extends the notion of similarity between patients and exploits the global topology of the graph to discover novel relationships between them. Finally, P-Net ranks patients with respect to the phenotype/outcome under study.P-Net leave-one-out procedure, together with the low spatial and time computational complexity of the method  proposed simple measures such as the normalized and average normalized similarity20. For patient profiles characterized by a medium size, such as mRNA data, protein expression, miRNA, the same authors proposed correlation-based measures  with or without an initial pre-filtering (using e.g. univariate statistics or machine learning based feature selection methods). For very high dimensional data (such as genotypic or epigenomic data), a pre-filtering step is mandatory to reduce profiles having a very large number of features. In this context we can adopt a two-steps pre-filtering, e.g. using a fast univariate feature selection method to drop the less significant features and then using a second level multi-variate feature selection method to take into account feature interactions and refine the set of selected features 85. This framework can be integrated with P-Net by substituting its first two steps (data collection and construction of the patient similarity network) with appropriate procedure to construct the network with the different data. Then we have two options to accomplish the integration of the different networks: a) direct integration from the similarity matrices; b) integration after the application of a graph kernel . In both cases we can apply suitable network-based data combination methods, such as simple averaging of the weights of the different networks, or weighted averaging according to the informativeness of each source of data86 or the SNF methods proposed in83. Considering the large range of possible clinical or omics data that can be used in this context87, and the different similarity measures and integration techniques that can be considered88, we believe that these problems need specific future studies and works that could expand the applicability and the effectiveness of the P-Net learning framework.It has been shown that the integration of different kinds of data produces synergies between data that can improve the prediction performancesP-Net requires potentially expensive feature selection algorithms to reduce the high dimensionality that usually characterizes genomic data, in order to remove irrelevent features that could add noise to the network of patients. For computational complexity reasons we applied simple univariate methods, but more refined multi-variate methods could lead to better results at the cost of a higher computational time complexity89.A limitation of the proposed approach is that Supplementary Information.