Despite recent improvements in cancer diagnostics, 2%-5% of all malignancies are still cancers of unknown primary (CUP), for which the tissue-of-origin (TOO) cannot be determined at the time of presentation. Since the primary site of cancer leads to the choice of optimal treatment, CUP patients pose a significant clinical challenge with limited treatment options. Data produced by large-scale cancer genomics initiatives, which aim to determine the genomic, epigenomic, and transcriptomic characteristics of a large number of individual patients of multiple cancer types, have led to the introduction of various methods that use machine learning to predict the TOO of cancer patients. In this review, we assess the reproducibility, interpretability, and robustness of results obtained by 20 recent studies that utilize different machine learning methods for TOO prediction based on RNA sequencing data, including their reported performance on independent data sets and identification of important features. Our review investigates the strengths and weaknesses of different methods, checks the correspondence of their results, and identifies potential issues with datasets used for model training and testing, assessing their potential usefulness in a clinical setting and suggesting future improvements. Cancer is the leading cause of death worldwide, and the overall burden of cancer incidence and mortality is expected to increase due to the growing population, aging inhabitants, and changes in the prevalence of risk factors . Key facThe development of array- and next-generation sequencing (NGS)-based whole-genome profiling techniques has enabled the rapid molecular characterization of cells or tissues, inspiring the establishment of several large-scale cancer genomics initiatives, such as ICGC and TCGA . These iVarious methods that utilize machine learning to predict the TOO based on NGS data have been developed recently, although the clinical use of such methods is still limited . AdvantaIn this minireview, we aim to assess the reproducibility, interpretability, and robustness of different NGS-based TOO prediction methods, including reported performance on independent data sets and identification of important features. Since the accuracy of the models depends on the input data type , we haveWe have conducted a literature search to identify studies that used RNA-Seq data to train machine learning models for cancer classification and TOO prediction. Studies that relied exclusively on miRNA sequencing were excluded from further analysis. Based on the aforementioned criteria, we selected 20 recently published studies for analysis.The majority of studies were based on deep learning, using neural networks of different architectures . SeveralThe models were trained on datasets comprising 9 to 40 cancer types (with a median number of 32 cancer types), and the number of sample points used for training ranged from 1,960 to 20,918 . All of the selected studies trained their models on either TCGA or ICGC data, with some studies including cancer sequencing data produced in-house  or data We have compared the classification accuracy of various models, which we defined as the number of correct predictions divided by the number of total predictions. In the cases where the classification accuracy was not reported, we have calculated it from the results described in the original publication. Cases where no prediction could be made were counted as incorrect predictions. Accuracy was the most commonly used measure of predictive performance in the studies surveyed in this minireview. While this measure can be influenced by class-imbalanced data, the influence of training dataset composition on measures of predictive performance is outside of the scope of this minireview.ρ = 0.0591, p-value = 0.7989). The prediction accuracy varies by tumor type, with some tumor types being more frequently mispredicted. Patterns of more frequent misclassifications among groups of cancers arising from the same organ , and/or among cancers represented by a small number of samples in the training set , as noted in multiple studies (In general, the analyzed models achieve high cross-validation prediction accuracy in the range of 73%–99% (with a median cross-validation accuracy of 95.5%;  studies . All of Out of the studies included in this minireview, only seven tested the performance of the developed models on an independent test set. Overall, the predictive accuracy of models on independent test data was lower than the cross-validation accuracy calculated on the data used for training the model, with only two studies attaining a prediction accuracy on an independent test set that was comparable to or higher than the one obtained with cross-validation . Jones eInterestingly, when testing the accuracy of the same model on test sets composed of both primary and metastatic samples, metastatic samples showed a lower prediction accuracy. For example, Furthermore, Several methods employed feature selection approaches to identify the most informative sets of genes for TOO prediction using different strategies to choose relevant genes. Two studies utilized various subsets of top-selected genes based on the Gini index from Random Forest models . SimilarTo estimate the correspondence of informative gene sets identified by various studies, we extracted gene lists provided by the studies that employed feature selection on the primary dataset. Studies selected between 36 and 4,861 unique genes, with the highest number of overlapping genes found between gene lists identified by We mapped the extracted gene lists to hallmarks of cancer  and GeneKLK3, a serine protease used as a serum marker in prostate cancer screening and disease monitoring, and PRAC, a highly expressed gene in prostate cancer (NAPSA, an aspartic proteinase expressed in type II pneumocytes. Its expression can be used to distinguish pulmonary lesions originating from primary lung adenocarcinoma or other primaries (IGFBP1, a hepatocyte-derived secreted protein required for normal liver regeneration by inhibiting proapoptotic signals (We further examined genes proposed by different studies as potential cancer signatures. Well-known prostate cancer signatures, including e cancer , have berimaries . Additio signals , as an iDPYS, were also identified by three more recent studies (BEST1 and new prostate and gastric biomarker SI, were either not found by other studies or were only identified by  studies  as imporDEFB1 gene was detected by four different studies ( studies , howeverIn this minireview, we analyzed 20 recent studies that employed machine learning to predict the TOO of cancers based on NGS data. Our goal was to assess their performance, reproducibility, interpretability, and robustness. We found that all of the analyzed methods exhibited very high prediction accuracy, ranging from 73% to 99%. This performance represents an improvement over currently used microarray-based methods, which have a prediction accuracy of 54%–100% . These fHowever, while the overall prediction accuracy of the models is high, it varies by tumor type, with tumors originating from the same organ or tumors that are underrepresented in the training set being more frequently mispredicted. This suggests that the accuracy depends more on the composition of the training set than on the method used for training the model. Researchers should, therefore, aim to assemble balanced datasets for model training and include as many samples of rare and underrepresented cancers as possible.Furthermore, most of the analyzed studies did not employ an independent test set, and the ones that did mostly showed a reduction in accuracy, especially for test sets obtained from metastatic patients or FFPE samples. Since CUP patients are, by definition, metastatic patients and FFPE tissues are still the most commonly available sample type for RNA-Seq, due to the cost-effectiveness of storage , both meIdentification of features important for prediction, implemented by several of the analyzed studies, could lead to novel biomarker discovery and discovery of genes whose expression is dysregulated in cancer, expand our current knowledge of mechanisms of cancer development and progression, identify potential actionable targets, and inspire novel treatment strategies. Indeed, most of the studies that employed feature selection identified at least 25% of actionable targets among their set of selected genes and showed that some of those genes are already known cancer signature genes. However, the overlap of gene lists provided by different studies is quite low, indicating that these results should be interpreted with caution. It is important to note that the majority of the analyzed studies employed filter methods, which select relevant features based on their intrinsic characteristics. Some used wrapper methods, where features are added or removed iteratively and scored based on their impact on the machine learning model’s performance, or embedded approaches, which combine properties of both filter and wrapper methods.Filter methods, while computationally less demanding, typically do not consider the subsequent classification model, often resulting in inferior performance compared to wrappers. In contrast, wrappers can be susceptible to overfitting and sensitive to parameter adjustments . RecentlFor example, in a study using ten microarray datasets for cancer classification, Gene Selection Programming , a metho


Finally, the single cell is determined as malignant or non-malignant by the majority vote of the five k-NN classification results. We tested the predictive performance of PreCanCell in 19 single-cell datasets, and reported classification accuracy, sensitivity, specificity, balanced accuracy (the average of sensitivity and specificity) and the area under the receiver operating characteristic curve (AUROC). In all these datasets, PreCanCell achieved above 0.8 accuracy, sensitivity, specificity, balanced accuracy and AUROC. Finally, we compared the predictive performance of PreCanCell with that of seven other algorithms, including CHETAH, SciBet, SCINA, scmap-cell, scmap-cluster, SingleR, and ikarus. Compared to these algorithms, PreCanCell displays the advantages of higher accuracy and simpler implementation. We have developed an R package for the PreCanCell algorithm, which is available at https://github.com/WangX-Lab/PreCanCell.We propose PreCanCell, a novel algorithm for predicting malignant and non-malignant cells from single-cell transcriptomes. PreCanCell first identifies the differentially expressed genes (DEGs) between malignant and non-malignant cells commonly in five common cancer types-associated single-cell transcriptome datasets. The five common cancer types include renal cell carcinoma (RCC), head and neck squamous cell carcinoma (HNSCC), melanoma, lung adenocarcinoma (LUAD), and breast cancer (BC). With each of the five datasets as the training set and the DEGs as the features, a single cell is classified as malignant or non-malignant by   •PreCanCell is an algorithm for predicting cancer and non-cancer cells from single-cell transcriptomes.•PreCanCell is a simple and effective ensemble learning algorithm.•PreCanCell uses well-annotated and high-quality cancer single-cell transcriptomes as the training set.•PreCanCell does not require users to provide gene markers and training or reference datasets.•PreCanCell displays better performance in predicting cancer and non-cancer cells than most established algorithms. We compared PreCanCell with seven other algorithms in several predictive performance, including accuracy, sensitivity, specificality, balanced accuracy (the average of sensitivity and specificity) and the area under the receiver operating characteristic curve (AUROC). We also analyzed correlations of these DEGs with various cancer-related molecular and clinical features in pan-cancer. This study provides a simple but effective method for identifying malignant and non-malignant cells, and a set of reliable cancer and non-cancer marker genes inferred from single cell transcriptomes.With a dramatic increase of well-annotated cancer single-cell transcriptomes to date, using the supervised method to infer malignant and non-malignant cells by learning these data may greatly improve the predictive performance. Based on this idea, this study proposed a novel algorithm, termed PreCanCell, for predicting malignant and non-malignant cells from single-cell transcriptomes. We first identified the differentially expressed genes (DEGs) between malignant and non-malignant cells commonly in five cancer single-cell transcriptome datasets. These datasets were associated with several common cancer types, including renal cell carcinoma (RCC), head and neck squamous cell carcinoma (HNSCC), melanoma, lung adenocarcinoma (LUAD), and breast cancer (BC). The number of malignant cells in each of the five datasets is relatively large, with the proportion > 10 %. Using these DEGs as features and the five cancer single-cell transcriptome datasets as the training set, PreCanCell predicts a single cell as malignant or non-malignant based on its gene expression profiles with the 22.1GSE157220) and a scRNA-Seq dataset for immune cells across health human tissues (GSE126030) from the NCBI gene expression omnibus (https://www.ncbi.nlm.nih.gov/geo/). Furthermore, we generated a synthetic scRNA-seq dataset by merging GSE157220 and GSE126030. For cell type annotation, we adopted original methods provided by these datasets-associated publications as ground truth. For all datasets, we performed quality control of reads and cells following the quality control steps described in the original publications. After data quality control, we performed normalization of gene expression values using the following methods. For full-length transcriptome data, gene expression levels were quantified as: Ei,j= log2, where TPMi,j refers to transcript-per-million of gene i in cell j. For unique molecular identifier (UMI) data, we normalized gene expression values using the “NormalizeData” function in the R package “Seurat” (v4.0.6) x + 1) transformation.We collected 16 publicly available scRNA-seq datasets, which were gene expression profiles of 663,760 cells. These datasets covered 11 cancer types, including RCC (T1_RCC), HNSCC (T2_HNSCC), melanoma (T3_Melanoma and P11_Melanoma), LUAD (T4_LUAD), BC (T5_BC and P1_BC), basal cell carcinoma (BCC) (P2_BCC), hepatocellular carcinoma (HCC) (P3_HCC and P6_HCC), synovial sarcoma (SyS) (P4_SyS and P5_SyS), glioblastoma (GBM) (P7_GBM and P10_GBM), pancreatic ductal adenocarcinoma (PDAC) (P8_PDAC), and metastatic castration-resistant prostate cancer (mCRPC) (P9_mCRPC). Among the 16 datasets, T1_RCC, T2_HNSCC, T3_Melanoma, T4_LUAD, and T5_BC were the training set, and the others were test sets. In addition, we obtained a scRNA-seq dataset for pan-cancer cell lines . We downloaded microarray (Affymetrix Human Genome U219 array) gene expression profiling data for 962 human cancer cell lines and their drug sensitivities  to 265 compounds from the Genomics of Drug Sensitivity in Cancer (GDSC) project (https://www.cancerrxgene.org/downloads). In addition, we downloaded data of RNA-seq gene expression profiling  in 1033 cancer cell lines and 51 normal cell lines from the Cancer Cell Line Encyclopedia (CCLE) project . All RNA-seq gene expression values (x) were transformed by log2(x + 1) before subsequent analyses. A summary of the datasets used in this study is shown in To analyze correlations of the DEGs with cancer-related molecular and clinical features in pan-cancer, we downloaded RNA-seq gene expression profiling  and clinical data for 33 TCGA cancer types from the genomic data commons data portal , it was defined as an upregulated gene in malignant cells; if it was expressed in at least 10 % of non-malignant cells and had a mean expression fold change of > 0.25 , it was defined as an upregulated gene in non-malignant cells. Finally, to credibly identify marker genes for malignant and non-malignant cells, we obtained the gene set, which was the intersection of the upregulated genes in malignant cells identified in each of the five training datasets; this gene set was defined as tumor marker genes (TMGs). The intersection of the upregulated genes in non-malignant cells identified in each of the five training datasets was defined as non-tumor marker genes (NMGs). The DEGs were the set of genes in TMGs or NMGs. We performed the differential gene expression analysis with the function “FindMarkers” in the R package “Seurat” (v4.0.6).For each training dataset, we identified DEGs between malignant and non-malignant cells. We obtained the DEGs based on the following criteria. First, the gene expression difference between malignant and non-malignant cells was statistically significant if the adjusted Wilcoxon rank-sum test 2.3x, we scale it as follows:min(x) and max(x) denote the minimum value and maximum value of gene expression across all single cells, respectively. With the DEGs as features and k-NN (k = 5) as the classifier, PreCanCell first predicted malignant and non-malignant cells in each training dataset and reported 10-fold cross-validation results. Next, PreCanCell predicted malignant and non-malignant cells in each test set. In the test set prediction, PreCanCell employs the ensemble learning algorithm. That is, PreCanCell predicts a single cell as malignant or non-malignant with the DEGs as features and k-NN (k = 5) as the classifier based on each of the five training datasets, and finally assigns the cell by the majority vote of five classification results. The classification accuracy, sensitivity, specificity, balanced accuracy and AUROC were reported.The prediction model PreCanCell utilizes the DEGs as features to predict malignant and non-malignant cells. Before the development and evaluation of classifier, all gene expression values are scaled to the range  by min-max normalization in each dataset. That is, for each original gene expression value 2.4We utilized the ESTIMATE algorithm 2.5For each cancer type from the TCGA project, we used the median expression levels of TMGs and NMGs as the cut-off value to subgroup samples into low-expression (< median) and high-expression (> median) classes. We employed the Kaplan–Meier (KM) model 2.6t tests. We used the false discovery rate (FDR), which was evaluated by the Benjamini-Hochberg method P-values in multiple tests. In evaluating correlations between the expression levels of TMGs and NMGs and other variables, we utilized Spearman correlations and reported correlation coefficients (ρ) and P-values.In comparisons of the expression levels of TMGs and NMGs between two classes of samples, we used two-tailed Student’s 2.7We compared the predictive performance  between PreCanCell and seven other algorithms, including CHETAH 3k-NN (k = 5) classifier. Finally, the single cell is classified by the majority vote of the five k-NN classification results. https://github.com/WangX-Lab/PreCanCell). Here we first analyzed correlations between these DEGs’ expression and cancer-related molecular and clinical features in pan-cancer and in individual cancer types. Next, we reported the predictive performance of PreCanCell in 16 datasets. Finally, we compared the predictive performance of PreCanCell with that of seven other algorithms.PreCanCell is a method for distinguishing between malignant and non-malignant cells based on their gene expression profiles. This method first identifies the DEGs between malignant and non-malignant cells in common in five single-cell transcriptome datasets . With ea3.1P = 2.22 × 10−39) and in 25 of the 30 individual cancer types with normal sample’s size not less than five (FDR < 0.05) (P < 0.05) (P < 0.05) (P < 0.05) , disease-specific survival (DSS), progression-free interval (PFI) and disease-free interval (DFI), respectively) (P < 0.05)  A. Moreov < 0.05) B. In add < 0.05) C. The ex < 0.05) D. Furthe < 0.05) E. These ctively) F. In add < 0.05) . Again, ρ < − 0.70) (ρ ≥ 0.89)  (P < 0.05) (P < 0.05) (P < 0.05)  A. The ex − 0.70) B, while  ≥ 0.89) . Moreove > 0.27) . These r < 0.05) C. The ex < 0.05) D. Taken  < 0.05) E.Fig. 3Chttps://www.cancerrxgene.org). Interestingly, among 265 compounds tested in cancer cell lines, the expression of TMGs showed significant positive correlations with IC50 values in 212 (80 %) compounds, and the expression of NMGs had significant negative correlations with IC50 values in 232 (88 %) compounds (FDR < 0.05)  in cancer cell lines using the data from the Genomics of Drug Sensitivity in Cancer (GDSC) project ( < 0.05) . These r3.2k-NN classifier with the DEGs as features in the five training datasets. In each of the five training datasets, the 10-fold cross-validation accuracy, sensitivity, specificality, balanced accuracy  of PreCanCell with seven other algorithms in predicting malignant and non-malignant cells in the 11 test sets A. These k-NN is a lazy learning algorithm that focuses on prediction rather than training.We also compared the computational resource and running time required by these algorithms B. Of not4k-NN, both of which were breast cancer single-cell transcriptomes. We obtained 0.835, 0.777, 0.930, 0.853 and 0.834 of accuracy, sensitivity, specificality, balanced accuracy and AUROC, respectively, compared to 0.919, 0.956, 0.859, 0.908 and 0.92 of them by PreCanCell. It proves that PreCanCell has significantly better performance than the single model. The second advantage of PreCanCell is its simplicity. PreCanCell does not require users to provide gene markers and training or reference datasets. In addition, it is implemented with a single R package. Furthermore, as k-NN is a lazy learning algorithm that focuses on prediction rather than learning or training, the implementation of PreCanCell is simple and straightforward. Users can even add additional training datasets to improve PreCanCell’s prediction performance. The third advantage of PreCanCell is that it needs less computational resources, such as CPU and memory usage, than most existing algorithms. Finally, PreCanCell is set up with parallel operations so that it can be implemented with multiple workers to increase running speed. It is particularly useful in predicting large datasets for which k-NN is relatively time-consuming.This study proposes a novel algorithm: PreCanCell, to predict cancer and non-cancer cells from single-cell transcriptomes. Compared to the established algorithms for cell type annotation, PreCanCell has several advantages. First, it is more accurate than most algorithms in predicting malignant and non-malignant cells. Its excellent predictive performance is mainly attributed to: (1) the use of well-annotated and high-quality cancer single-cell transcriptomes as the training set; and (2) the use of an ensemble learning algorithm for class prediction. PreCanCell has overcome the drawback of many methods developed earlier using poorly-annotated training sets or reference sets. In addition, PreCanCell employs ensemble learning that has been shown to be more accurate and robust than a single model k-NN as the base classifier for its some advantages over other algorithms. First, k-NN is a non-parametric method that does not make any assumption about data distribution; thus, k-NN is suitable for various types of data, including complex and non-linear data. Unlike bulk transcriptomes, single-cell transcriptomes are not normally distributed for which k-NN is a viable method. Second, k-NN is robust to noise and outliers since it uses a majority vote to reduce the impact of mislabeled samples. Again, it is quite suitable for single-cell transcriptomes which are full of noise. Finally, as a lazy learning algorithm, taking high-quality training sets as background, k-NN often can achieve a high prediction performance. Here we set k = 5 because: (1) for binary classification, the k of k-NN should be an odd number; (2) the k-NN’s prediction performance was the best in the training datasets when k ranged from 1 to 15; and (3) when k = 5, the algorithm’s computational complexity is lower than that with k > 5.Here we chose In some datasets, some of the compared algorithms have achieved inferior performance A. A mainIn addition to develop the novel algorithm for cancer cell identification, this study uncovered cancer marker genes by analyzing single-cell transcriptome datasets for five common cancer types: RCC, HNSCC, BC, LUAD and melanoma. We further demonstrated that the cancer marker genes had significant associations with malignancy-related characteristics in tumor bulk and cancer cell lines, such as their upregulation correlated with heightened tumor proliferation capacity, ITH and drug resistance, and poor prognosis in pan-cancer and in diverse individual cancer types.There are several limitations in this study. First, the reported predictive performance in the training set could be overestimated due to the predicted sets being involved in feature selection. Nevertheless, the reported predictive performance in the test sets should be warranted. Second, this algorithm is limited to annotating cancer and non-cancer cells, while it is not designed for identifying subpopulations of non-cancer cells, such as immune cells, stromal cells and epithelial cells. This is a direction for us to extend the algorithm in the future.Ethical approval and consent to participate were waived since we used only publicly available data and materials in this study.Tao Yang: Data curation, Formal analysis, Investigation, Software, Validation, Writing - review & editing, Visualization. Qiyu Yan: Visualization, Data curation, Formal analysis, Validation. Rongzhuo Long: Visualization, Formal analysis, Software. Zhixian Liu: Project administration, Funding acquisition, Investigation. Xiaosheng Wang: Conceptualization, Methodology, Resources, Investigation, Writing - original draft, Supervision, Project administration.10.13039/501100004608Natural Science Foundation of Jiangsu Province (BK20201090) and 10.13039/501100002858China Postdoctoral Science Foundation (2021M691338) to ZL.This work was supported by The authors declare that they have no competing interests.


RNA sequencing (RNA-Seq) is a technique that utilises the capabilities of next-generation sequencing to study a cellular transcriptome i.e., to determine the amount of RNA at a given time for a given biological sample. The advancement of RNA-Seq technology has resulted in a large volume of gene expression data for analysis.Our computational model (built on top of TabNet) is first pretrained on an unlabelled dataset of multiple types of adenomas and adenocarcinomas and later fine-tuned on the labelled dataset, showing promising results in the context of the estimation of the vital status of colorectal cancer patients. We achieve a final cross-validated (ROC-AUC) Score of 0.88 by using multiple modalities of data.The results of this study demonstrate that self-supervised learning methods pretrained on a vast corpus of unlabelled data outperform traditional supervised learning methods such as XGBoost, Neural Networks, and Decision Trees that have been prevalent in the tabular domain. The results of this study are further boosted by the inclusion of multiple modalities of data pertaining to the patients in question. We find that genes such as RBM3, GSPT1, MAD2L1, and others important to the computation model’s prediction task obtained through model interpretability corroborate with pathological evidence in current literature.The online version contains supplementary material available at 10.1186/s12859-023-05347-4. Cancer is a leading cause of deaths worldwide, accounting for nearly 10 million deaths in 2020, or nearly one in six deaths. In a multi-stage process that often goes from a pre-cancerous lesion to a malignant tumour, cancer develops when normal cells undergo a transition into tumour cells. According to research, it is presently possible to prevent between 30 and 50% percent of cancers by avoiding risk factors and using proven evidence-based preventative methods. Many cancers have a high chance of cure if diagnosed early and treated appropriately . FortunaOwing to the tabular nature of the gene expression dataset, applications of self-supervised learning techniques on gene expression data (tabular data here) have not made it to the limelight as much as they have in the domains of Computer Vision and Natural Language Processing. This is mainly because tabular models must be able to accommodate features from different discrete and continuous distributions and uncover correlations without relying on the positional information.So why is self-supervised learning worth exploring in the context of gene expression tabular data? Apart from the increase in performance that comes with deep learning models, it also enables fusion with multiple other modalities of data  – eliminating the need for feature engineering, representation learning, and end-to-end compositional multi-task models.Initially, RNA Sequencing gene expression data pertaining to 519 entries is procured from The Cancer Genome Atlas (TCGA). Owing to the high dimensional nature of the dataset, we have proposed a variety of feature selection techniques to mitigate the effects of the curse of dimensionality. The features chosen are used to train classical machine learning models  without the self-supervised learning objective.Subsequently, the efficacy of self-supervised learning methods in this domain is explored via TabNet. This requires the procurement of additional unlabelled data to be used for pretraining, also obtained from TCGA across various cancer projects, with the unifying factor being the type of cancer, which was selected to be “Adenomas and Adenocarcinomas”.The TabNet model is first pretrained on the unlabelled corpus, after which the pretrained model is fine-tuned on the labelled dataset.The explainability of the best-performing computational model is demonstrated by plotting its feature importance mapping. Additionally, majority of the genes deemed as important by the prediction task are in agreement with existing literature.In the interest of further improvement in score, additional modalities of data, namely Copy Number Variation (CNV) and Clinical data pertaining to the same patients are obtained in the same manner as above. These modalities of data then undergo preprocessing in parallel and all the modalities are joined using Submitter ID as the join key.These different modalities of data are trained using the TabNet model with a self-supervised learning objective and encouraging results are obtained in the context of vital status estimation.With this line of thought, our work can be summarised as follows:The earliest work on the estimation of vital status using RNA-Seq data serves as the baseline model for this study . They usIn the domain of deep learning, the work of  is notewOn the multimodal front, the work of  proposesTo combat the curse of dimensionality inherent in the data, there are a variety of useful analyses of feature selection techniques that can be used. A comparison of 10 dimensionality reduction techniques using 30 simulation datasets and 5 real datasets has been performed . These dSelf-supervised learning (SSL) methods have predominantly been focused on natural language processing , 14 and In this study, we analyse the vital status  of those patients diagnosed with colon cancer. This study is performed in two settings – the unimodal data setting  and the multimodal data setting .For the study of RNA-Seq data, the problem can be formulated as predicting the vital status For the multimodal data setting, the problem can be formulated as predicting the vital status The pipeline for the study of RNA-Seq data using supervised learning methods can be found in Fig. The labelled data used in this study pertains to the TCGA-COAD project, focusing on Colon Adenocarcinoma. This data was downloaded from the GDC data portal , with fiAs first defined by , TranscrDue to technical biases such as depth of sequencing and gene length, the normalization of the abundance values quantified in the previous section needs to occur in order to make gene expression values comparable within and across samples.N denotes the number of protein-coding genes, The fragments per kilobase of transcript per million mapped reads (FPKM) calculation aims to control for transcript length and overall sequencing quantity. The upper quartile FPKM (FPKM-UQ) is a modified FPKM calculation in which the protein-coding gene in the 75th percentile position is substituted for the sequencing quantity. This is thought to provide a more stable value than including the noisier genes at the extremes. They are computed as follows:Upon experimentation with the above measures, this study proceeds to use FPKM-UQ as its preferred unit due to better model performance with data.The COAD dataset initially has a total of 519 data points with 60,660 features. However, this data is reduced to 445 data points to match submitter IDs with other modalities, in order to obtain a fair comparison with the multimodal workflow. This is clearly detailed in this section.The most glaring inference that can be made is that the dataset suffers massively from the curse of dimensionality. Table A multitude of different gene types that include coding, non-coding, and pseudo-coding exist. Based on the literature, a subset of genes known as the protein-coding genes, which are present in the human genome, are essential to the study of human biology and medicine and, as a result, have a significant impact on cancer vital status prediction. The original TCGA patient dataset had 60,660 gene ensemble IDs (features), therefore the first phase of feature reduction is decreasing that number to 19,962 protein-coding genes.L1 / Lasso Statistical t-test Principal Component Analysis (PCA) The number of features filtered as important by each feature reduction technique can be seen in Table  The second step of feature reduction is the usage of one of three different feature reduction techniques, namely -p-value of 0.01 was used as a cutoff for a feature to be selected. These values were determined experimentally as the best parameter values for the respective feature reduction methods.For Lasso, an inverse regularisation strength of 0.001 was used, with the genes with non-zero weights then selected as a filter. For PCA, the number of features captured corresponded to 99.9% of the variance across genes. For the t-test, a labelled dataset, with the same features used as a filter  on the unlabelled dataset.This feature reduction is performed on the A variety of predictive analyses have been conducted on the data. These, in combination with the different feature reduction techniques used, resulted in a set of methods used from start to finish to model the vital status of the patient. The workflow for the same has been elucidated in Fig.  Logistic regression is one of the most common and useful classification algorithms in machine learning. It is a supervised machine learning algorithm that can be used to evaluate the probability of a certain target value or class. In this case, it is used to model the probability of the vital status of the patient as either 0 or 1. Logistic Regression is thought to be a baseline in this work owing to its simplicity . The mod ANNs are nonlinear models which discover complex relationships between the input and targets to discover inherent patterns. An artificial neural network has three or more layers; the first layer consists of input neurons; which send data into the intermediate layers, which in turn consolidate and send the final output data to the output layer. While more complicated than most other methods, the capturing of complex relationships between features is better done by the neural network. A network of 4 layers with hidden layer sizes as  respectively, and ReLU activation performed the best on this data. KNN is a non-parametric and lazy learning algorithm. In short, the algorithm entails a  weighted distance calculated between the target and the learned data. The best k points are the basis for the class chosen. In the classification phase, k is a user-defined constant, and an unlabelled data point is classified by assigning the label which is most frequent among the k training samples nearest to that query point.The inferior results obtained using PCA as a feature reduction technique was an indication that the eigenspace was a bad representation of the feature set which implies the lack of correlation within the data. This in turn spurred us to consider using KNN in order to obtain better results. Training with the 25 nearest neighbours yielded the best results. A Generalized Additive Model with automated interaction detection, Explainable Boosting Machine (EBM) is tree-based and boosts gradients in a cyclic fashion. The boosting process is carefully constrained to train on a single feature at a time in round-robin form with a very low learning rate, making the order of the features irrelevant. Second, pairwise interaction is automatically found and included by EBM, increasing accuracy while maintaining intelligibility. Apart from the default parameter set, tweaking the parameters max leaves (3), and max bins (40) yielded better results. This has been implemented using the Interpret ML package. XGBoost is a decision-tree-based ensemble Machine Learning algorithm that uses a gradient-boosting framework.In gradient boosting, each predictor improves on its predecessor’s error; It relies on the fact that the best possible model, when combined with previously trained models, minimizes the overall prediction error. The key idea is to set the target values for the coming model in order to minimize the error. XGBoost has been used extensively with RNA-Seq data due to its tabular nature . The modSelf-supervised learning (SSL) is a machine learning technique developed to solve the challenges posed by the over-dependence of labelled data, either due to the complexities involved in finding it or simply due to the unavailability of labelled data.The aim of this technique is to capture subtle nuances in data that do not require the presence of labels. This allows the subsequent training done on the labelled data to be more useful in the prediction task itself, as opposed to learning structures within data. The procurement of the labelled dataset follows the same workflow as detailed in the section above. Additionally, an unlabelled dataset consisting of the same gene IDs, with a total size of 4801 samples from various types of adenomas and adenocarcinomas is also procured by the same means for self-supervised learning purposes. TabNet is a neural network designed to handle tabular data well as it also allows for pretraining on unlabelled data. TabNet uses sequential attention to choose features at each decision step, enabling interpretability and better learning as the learning capacity is used for the most useful features.The TabNet architecture has an encoder and decoder module wherein the former is inspired by top-down attention using sparse instance-wise feature selection constructed on top of a sequential multi-step architecture. For the self-supervised learning objective, a decoder architecture has been proposed for reconstructing the masked features from the encoded representation. To do this a binary mask S has been chosen as Finally, TabNet also has a built-in interpretability mechanism that quantifies the contribution of each feature to the trained model across the dataset.In the first step, feature reduction is performed on the labelled dataset as we assume that this captures the important features.The same subset of features selected from the labelled dataset is filtered in the unlabelled dataset as well.The TabNet pretrainer is then trained on the unlabelled dataset in order to capture the embedding space of the gene expression data.The pretrained model is then trained on a training split of the labelled subset in a 5-fold cross-validation, the average receiver operating characteristic area under the curve (ROC-AUC) score of which is reported in the end.  and Clinical data for the same patients were downloaded from TCGA. The manifest files for the above were ned from , and wer Copy Number Variation captures the variation in the number of copies of a certain DNA sequence across different patients. While this is also done on a per-gene basis, this is different from gene expression . The CNV dataset consists of only categorical variables.The first step of preprocessing pertaining to CNV data involves filtering only the high-variance genes. The genes that seem to vary more across the set of patients can be assumed to be more important and contain more information. The top 2000 highly varying genes are selected. Subsequently, Chi-Squared Analysis, which is known to be effective against categorical data, is used as the final step of feature reduction. The 256 genes  deemed most important by Chi-Squared Analysis are selected in this stage.  and the final labelled dataset is of the size .In order to eliminate any bias, the RNA-Seq data obtained post submitter ID matching was used in all the previous experiments.As discussed above, the pipeline for the study of multimodal data follows the same workflow as the self-supervised workflow in unimodal data setting due to its superior results as compared to the supervised models.The self-supervised pretraining step involves the TabNet model mapping the embedding space of the unlabelled multimodal data from which the pretrained weights are then obtained. This is then followed by a fine-tuning stage, where the pretrained weights are used to train a TabNet Classifier that is then trained on the labelled multimodal data. The Optuna framework is employed for hyperparameter search for these stages. Finally, the optimal model is used to report the average ROC-AUC score over a 5-fold cross-validation.All experiments carried out with the models used in this study have their scores reported after a 5-fold cross-validation over the TCGA-COAD (Labelled) dataset, using Google Colab with an NVIDIA Tesla K80 GPU.This study provides a strong and thorough base on the efficacy of supervised learning approaches by combining each model with each feature reduction technique. Although we have tried a host of computational models, only the ones that show the highest scores have been enlisted below.0.742 among the various supervised learning approaches. The scores obtained by this model is also better than that obtained by [It can be seen in Table ained by  which sep-value threshold of 0.001. In this setup, the best-performing model in particular did not see a stark change in the mean ROC-AUC but did see a rise in standard deviation in the results  and self-supervised approaches to clearly demonstrate the performance gain obtained via pretraining. Our results, as can be seen, in Table 0.842 obtained with RNA-Seq data . Optuna was used to tune hyperparameters such as weight decay, learning rate, patience (for early stopping), and gamma (feature reuse in masks).A pretrained TabNet model with the employment of t-test as a feature selection technique shows the highest score of 0.88 ± 0.019 in the context of vital status prediction.As can be seen in Table This study makes use of model interpretability, which helps quantify and visualise the effect of features in the dataset, i.e., the gene expressions to the vital status predicted by the models used. As elucidated in the previous sections, TabNet is the best-performing model in our study with a final ROC-AUC score of 0.842 on RNA-Seq data  and 0.88 on multimodal data. The genes deemed important by TabNet are verified in literature to bring confidence in their predictions.TabNet consists of an encoder which is comprised of an attentive transformer, a feature transformer and a feature mask, and a decoder which is comprised of just the feature transformer. The decoder employs a sparse feature selector at each step. Differing from SHAP, which uses cooperative game theory, TabNet  computestivation .Using the values in the aggregate feature mask of the best performing TabNet model (using the t-test feature selection technique), the feature importances as described in Fig. Similar to the approach used with supervised techniques, using the genes (features) selected in Fig. As observed from Table To further understand the results, functional enrichment analysis was performed on full list of genes, ordered by their feature importances as determined by the TabNet model. The list of pathways obtained are below:In this section, we look into the associated pathways with the important genes as inferred from the TabNet approach as presented above. The list of important genes with non-zero importances, in order, was selected as the input for GProfiler which yielded the results as observed in Table Patients’ survival (or time to event) statistics are frequently gathered and evaluated in clinical studies. The information may be used to compare two or more groups in terms of both the overall number of occurrences and the frequency at which a particular event happens. The distribution of many other forms of data does not apply to survival data; they are non-negative and frequently skewed depending on how quickly events happen. More significantly, they frequently experience censoring (missing or incomplete data), which can happen for a number of different reasons. The work of  details Additional file 1. SM1. Data Procurement and Processing. SM2. Other Feature Reduction Techniques. SM3. Other Results. SM4. Feature Importances. SM5. Hyperparameter Tuning.


The increasing generation of population-level single-cell atlases has the potential to link sample metadata with cellular data. Constructing such references requires integration of heterogeneous cohorts with varying metadata. Here we present single-cell population level integration (scPoli), an open-world learner that incorporates generative models to learn sample and cell representations for data integration, label transfer and reference mapping. We applied scPoli on population-level atlases of lung and peripheral blood mononuclear cells, the latter consisting of 7.8 million cells across 2,375 samples. We demonstrate that scPoli can explain sample-level biological and technical variations using sample embeddings revealing genes associated with batch effects and biological effects. scPoli is further applicable to single-cell sequencing assay for transposase-accessible chromatin and cross-species datasets, offering insights into chromatin accessibility and comparative genomics. We envision scPoli becoming an important tool for population-level single-cell data integration facilitating atlas use but also interpretation by means of multi-scale analyses. By learning representations for both cells and various condition covariates, scPoli facilitates atlas-level integration and analysis of single-cell genomics datasets with improved interpretability. A possibility opened by these atlases is that of meta-analyses relating cell types and states with biological conditions or demographics metadata4.The advancements in single-cell technologies have enabled the generation of datasets comprising information from millions of cells. These datasets, also called ‘atlases’, include data from different conditions and individuals and offer precious insight into cellular processes and states in different scenarios. Consortia such as the Human Cell Atlas7. Tremendous efforts have been made to solve the data integration problem for single-cell RNA sequencing datasets using approaches ranging from statistical11 and graph-based14 methods to deep learning models17. Nonetheless, single-cell data integration remains challenging18, especially in the case of many datasets with a variety of technical and biological properties.Performing meta-analysis on an atlas requires learning a joint representation of all datasets correcting batch effects between them21, which build upon data integration algorithms to update an existing atlas by integrating a query dataset. Transferring information from reference to query enables efficient annotation of the query cells23.Many analyses can be accelerated by mapping data on top of an atlas. Algorithms for efficient use of reference atlases are known as reference mapping methods6 rely on one-hot-encoded (OHE) vectors to represent conditions24. This encoding does not allow a downstream interpretation of the effect of each sample on the mapping. Additionally, in the presence of many unique condition categories, the number of conditional inputs can become close or equal to the number of gene expression measurements leading the model to produce inaccurate data representation25. Among current reference-building methods26 only scANVI and Seurat v3 offer cell type classification coupled with a reference mapping algorithm21. Yet, while they can integrate annotated data to extend the reference, this requires retraining, which is time-consuming and can sometimes be not possible due to data sharing restrictions.Existing deep learning integration methods27 combined with advances in meta-learning28 that is able to learn representations for both cells and samples (or other batch covariates). scPoli offers a cell-level and a sample-level view of the dataset enabling multi-scale analysis: the simultaneous exploration of sample and cell representations.In this Article, we introduce single-cell population level integration (scPoli), a semi-supervised conditional generative model and open-world learnerscPoli uses prototype-based cell label transfer and is augmented with an uncertainty estimation mechanism. We demonstrate that scPoli is competitive in data integration and cell annotation with other methods across six datasets. We further showcase the features of our model by integrating a lung atlas and performing reference mapping for two queries. We show potential use cases of condition embeddings such as sample classification and data integration workflow guidance. Finally, we build a reference of 7.8 million peripheral blood mononuclear cells (PBMCs) from 2,375 samples and explore the sample-level representation obtained with scPoli.xi) in a dataset can be ascribed to batch effects and biological signals. Similarly to other conditional models29, scPoli aims to regress out batch effects in a nonlinear fashion by means of a conditional variable (si) representing batch while retaining biological information. Moreover, scPoli posits that cell identities (ki) can be represented with learnable cell type prototypes28 modeled using latent cell representations (zi)  the replacement of OHE vectors with continuous vectors of fixed dimensionality to represent the conditional term, and (2) the usage of cell type prototypes to enable label transfer.The variation of gene expression (zi) Fig. . scPoli,N is the number of conditions. These are concatenated to the input, and an additional neuron for each condition is added to the first layer of the encoder neural network. In the case of thousands of conditions to be integrated, this approach leads to an increase in the number of total trainable parameters that can slow down training. scPoli uses learnable condition embeddings M embeddings to accommodate the query data conditions  and other types of methods ). Out of these models, only our method, scANVI and Seurat v3 tackle both data integration and label transfer, while some exclusively do integration (scVI and Symphony), or classification (MARS and SVM). All of these models, except for MARS and Symphony were part of the Luecken et al.6 data integration benchmark, where they came out as top performers.To understand how well scPoli integrates single-cell datasets, and how accurately it transfers cell type annotations, we benchmarked our model against other methods for data integration and label transfer. We included in this comparison both deep learning models  by 5.06% in data integration Fig. . When we31.To assess the quality of the classification achieved during reference mapping we used two metrics: the weighted averaged and the macro-averaged F1 score, with the latter being more sensitive to underrepresented cell types. We observed that scPoli outperformed all methods except for the linear SVM on the weighted F1 metric Fig. . Out of Furthermore, scPoli’s integration performance and label transfer accuracy were stable across runs and different dataset sizes ies Fig.  while maies Fig. . We furties Fig. .Fig. 3scWhen looking at the first two principal components (PCs) of the sample embeddings we found that samples from the same studies grouped together Fig. . We addiOne of the main hurdles in atlas building comes from discrepancies in annotation terms across datasets. scPoli and other methods that leverage these annotations require prior label harmonization labels before usage, which requires expert knowledge. Nonetheless, scPoli can work on multiple levels of annotation  and can propagate labels to underclustered datasets during reference building. scPoli is able to model multiple sets of prototypes for each level of annotations. This prior information is then leveraged by optimizing the prototype loss on each set of labeled prototypes. To simulate such a scenario, we integrated the HLCA, but this time using both a coarse and a fine annotation  graph-based uncertainty was used, we noticed that cells that lay in regions of transitions between cell types displayed the highest levels of uncertainty, as well as cells whose identities were not present in the reference data 32 Fig. . These data Fig. . We consata Fig.  and inspata Fig. . A subseTo see how scPoli would perform when mapping a query dataset from a different condition than the one in reference, we mapped data collected from cancer patients. These data contain two cell identities not present in the reference (cancer and erythrocytes). We observed that scPoli successfully mapped the query dataset  PBMC dataset by Su et al.90% Fig. .Fig. 4sc2PC1 0.41, adjR2PC2 0.16 obtained with an ordinal regression; the adjusted R2 is reported to take into account the different numbers of available predictors)  Fig. . We, thers) Fig. .35, with 99 samples from 65 patients in two different cohorts    rts Fig.  obtained97) Fig.  and the 73) Fig. , rather 00) Fig. . We coulThese analyses suggest that, while in more focused studies where technical factors are controlled and kept consistent across samples, biological signals represent the main source of sample-level heterogeneity; in bigger-scale collections of data with a variety of technical factors, these variations will dominate. This led us to speculate that, since sample embeddings identify major sources of variations in the data, they could also guide the choice of the conditions to integrate in a data integration workflow. We proceeded to integrate the data using other two models conditioned on the covariates that showed association with the first PC of the embedding space (experiment and cohort).Additionally, we trained a model with a dummy batch covariate, which was equal for all cells. In this case, scPoli will leverage exclusively cell type annotations and the prototype loss to perform integration. We observed that the integration yielded by the cohort- and experiment-level models displayed a similar quality of integration despite the reduction in the number of conditions to integrate   samples by modeling the likelihood of the input data using a Poisson distribution, as proposed in ref. R2 of 0.97. We also observed a strong association with sequencing assay (adjR2 0.86)   Fig.  and mode59) Fig. . When we59) Fig. . We comp59) Fig. .Fig. 6scP < 0.01) and filtered them for coefficients of determination larger than 0.3 in absolute value. We found the biggest number of correlated genes with the first PC. This number did not decrease regularly going further through the PCs, and we found PC2, PC4 and PC5 to also have a substantial number of correlated genes , which is a ribosomal gene involved in the cellular response to stress; the second top positively correlated gene, on the other hand, was RHOA (r = 0.85), a gene involved in the immune response and that we observed to be more highly expressed in disease samples  and PC4 (TSLPY2) in Supplementary Fig. To understand whether the sample embeddings reflected any gene expression patterns across samples, we computed the Pearson correlation between the mean expression of each gene in the various samples and the PC scores of the embeddings. We thus obtained lists of significantly correlated genes with each PC θ and ϕ are respectively the encoder and decoder networks parameters, DKL indicates the Kullback–Leibler divergence.The resulting loss function, also known as evidence lower bound, is formulated asThe first term of the loss formulated above is also known as reconstruction loss, and it might take different forms depending on the generative process of the input data. In the case of count data, we assume a negative binomial distribution as input; thus the likelihood will follow:In the case of scATAC-seq data we assume a Poisson distribution:The negative log-likelihood of the appropriate distribution is used as reconstruction loss during training.E to represent conditions. The learning objective for this network is akin to that of a standard CVAE, but the embeddings s are optimized during training as parameters of the model using backpropagation:The architectural backbone of scPoli builds upon CVAEs, but with an important modification. While in a standard CVAE different conditions are represented by means of fixed OHE vectors Condition embeddings are implemented using the torch.nn.Embedding class, which takes as input an index indicating the condition and outputs the learned embedding. These embeddings are randomly initialized and optimized together with the rest of the trainable parameters of the network by minimizing the loss function used to train the model.D is the dimensionality of the latent space of the CVAE model and k represents the cell type label. Prototypes are computed by averaging the latent representation of the data points belonging to each particular cell type:zi is the latent representation of cell i, k is the cell type label and k:scPoli posits that cell identities can be represented using prototypes. For each labeled cell type in the data, a prototype j, unlabeled prototypes are computed using:zi is the latent representation of cell i, j is the unlabeled cluster label and j:Additionally, when the model is trained on partially labeled data, a set of unlabeled prototypes are initialized after clustering the unlabeled data using the Louvain algorithm implementation in Scanpy. Assuming that each unlabeled cell is now assigned a cluster Unlabeled prototypes offer good reference points for downstream analyses and novel cell type annotation but are not used for the prototype loss computation.K is the number of cell types in the data, ai is the cell type annotation of cell i and p = 2 and to the Manhattan distance when p = 1. Nonetheless, any distance metric could potentially be used.L is the number of cell type annotations provided to the model and i and annotation l.scPoli’s training objective includes a supervised term we call prototype loss. This term has the objective of pulling together cells belonging to the same cell type towards their correspondent prototype in latent space.N condition embeddings, where N is the number of samples present in the reference. The model receives gene expression G is the number of input features), which is concatenated to the sample embedding of the corresponding sample E is the embedding dimensionality). When multiple condition labels are passed to the model, the embeddings for each condition k is the number of condition covariates passed. The objective function Pretraining: initialize η is a hyperparameter that tunes the strength of the prototype loss; store learned prototypes with the model.Fine-tuning: initialize cell type prototypes using (6) and optimize Reference buildingM additional sample embeddings where M is the number of samples in the query. These embeddings constitute the only trainable parameters when training on query data. The model receives gene expression Pretraining: freeze the weights of the encoder and decoder networks from the reference model, and initialize Fine-tuning: initialize labeled prototypes in the query dataset if any labeled cells are present, the unlabeled data are clustered using the Louvain algorithm and unlabeled prototypes are initialized for the detected clusters using (8). Optimize Reference mappingWe describe the procedure used to train an scPoli model when building a reference of performing query-to-reference mapping below:scPoli is optimized using Adam and a default learning rate of 0.001. The ratio of pretraining/fine-tuning epochs we use is 0.9, but the optimal value might vary depending on the input dataset.i with latent representation zi the cell type identity k of the closest prototype p in latent space:scPoli assigns to unlabeled cell The minimum distance between the latent representation and any reference prototype is used as a proxy for uncertainty for unknown cell type detection.4. The authors held out a few labeled datasets as query and then after mapping them they looked for the most optimal uncertainty threshold by generating a receiver operating characteristic (ROC) curve tracking correct label transfer.We do not fix a default value for this uncertainty above which a cell should be classified as unknown. In our experiments, we observed the distributions of uncertainties and picked a quantile as the cutoff value. When we expected more unknown cells in the unlabeled data, for example in the cancer query, we picked lower quantiles. Another possible approach could be the one proposed by ref. This uncertainty does not have an upper bound, but we offer the option to scale and normalize it to have values between 0 and 1.G be the dimensionality of the gene expression input, E the dimensionality of condition embeddings in scPoli, N the number of conditions, D the latent space dimensionality, and Henc and Hdec the widths of the input layers of the encoder and decoder, respectively. When comparing these two models with the same number and width of hidden layers and latent dimensionality and ignoring bias terms of the fully connected layers, scPoli introduces E × N parameters in the embedding matrix, The introduction of learnable condition embeddings that replace OHE vectors to represent conditions leads to a difference in the number of trainable parameters between the scPoli CVAE and a standard CVAE network. Let This inequation results inE, we can see that scPoli results in a lower number of parameters in the case of a relatively high number of conditions N  on the same dataset ) under- or oversampled to reach sizes of 1K, 10K, 100K and 1M cells. The model was trained with either a sparse or a dense input matrix to test the difference in memory consumption and training time between the two. We tracked the time needed to train the model for 100 epochs and max memory consumption during training net of the memory needed to store the data. The experiment was run on a server with two Intel Xeon Platinum 8280L 2.70 GHz central processing units and an NVIDIA v100 graphics processing unit. The results of this analysis are available in Supplementary Fig. 37. We ran PeakVI using comparable parameters to obtain a comparison.We adapted scPoli to work on scATAC-seq data by using a Poisson likelihood for modeling fragment counts. Fragment counts are obtained by aggregating odd and neighboring even reads. This approach was proposed by ref. R2, which is defined as:n is the number of samples and p is the number of predictors.We compute the association between a covariate and a PC by fitting a linear model with the covariate as the predictor and the values of the PC as dependent variable. We fit this model by using the lm function in R. After fitting the model, we report the adjusted scArches scVI (v0.5.3): we ran the model using default parameters;scArches scANVI (v0.5.3): we ran the model using default parameters;43 and used supervised PCA for reducing the data dimensionality to 50;Seurat v3 (v4.0.3): we followed the tutorial44 and ran the model using default parameters.Symphony (v0.1.0): we followed the vignetteWe benchmarked the data integration performance of our model against various state-of-the-art methods. These include:scArches scANVI (v0.5.3): we ran the model using default parameters;MARS: we ran the model using default parameters;43 and ran the model using default parameters;Seurat v3 (v4.0.3): we followed the vignetteSVM: we fit a LinearSVC object from scikit-learn (v0.24.2) on the reference data.We benchmarked the performance on label transfer and cell type classification against the following methods:6. To quantify label transfer accuracy we used the weighted averaged and macro-averaged F1 score.We quantified the quality of the data integration using the following metrics from the scIB (v 1.0.0) package and Luecken et al. Cell type ASW (average silhouette width), isolated label F1, isolated label silhouette, NMI  and ARI (adjusted Rand index) were used as biological conservation metrics. To quantify batch mixing we used PC regression, graph connectivity and batch ASW. The overall integration score is a weighted average of the average batch mixing score and the average biological conservation score, with weight 0.4 and 0.6 respectively. Descriptions for these metrics can be found in ref. Weighted F1 score: average of the F1 scores obtained for each class, weighted by their support.Macro F1 score: average of the F1 scores obtained for each class, without any weighting.An overview of the used metrics can be found in Supplementary Table 4. The dataset consists of 584,884 lung cells from 166 samples and 107 subjects. Gene expression is subset to 2,000 selected genes, which we used for model training. Different levels of cell type annotation and sample and patient metadata are curated and available.We obtained the HLCA core dataset from the authors of the study61.The atlas contains 7,800,850 PBMC cells from 2,375 samples, representing cells from 25 datasets, 1,977 healthy or diseased donors. For model training and analysis, 10,000 HVGs were selected. A coarse annotation consisting of 14 cell types was used for initializing scPoli’s prototypes. The data and metadata were curated by scientists at the Chan-Zuckerberg Initiative and collected from refs. 35 PBMC dataset of 65 patients with COVID-19 and healthy controls. The dataset contains 99 samples and 222,003 cells, and was downloaded as part of the Fredhutch COVID-19 collection available at ref. 62. For model training and analysis 4,000 HVGs were used.This is a published34 PBMC dataset of 129 COVID-19 patients and 16 healthy controls. The dataset contains 270 samples and 559,517 cells, and was downloaded as part of the Fredhutch COVID-19 collection available at ref. 62. For model training and analysis 2,000 HVGs were used.This is a published36. The dataset was downloaded from ref. 63, which contains expression profiles based on the one-to-one orthologs  defined in the three species. A total of 2,000 HVGs were selected on the basis of the reference datasets.These data consist of the transcriptomics profiling of the primary motor cortex of 305,638 single nuclei in humans, marmoset monkeys and mice64. The data were published for the NeurIPS 2021 competition on multimodal single-cell integration. We isolated only the ATAC features and trained the model on those. We filtered peaks that were detected on less than 5% of cells. After these preprocessing steps the data consisted of 69,249 cells and 16,134 features.We obtained the scATAC-seq data from ref. 65 unless specified otherwise. Count data were used for all datasets.All datasets used for benchmarking were obtained from ref. 66. The dataset contains 32,484 cells from 4 studies and 16 cell types. The data was subset to the 4,000 most highly variable genes before further analysis.The immune PBMC dataset used for benchmarking was obtained from refs. The data contain 16,382 pancreas cells from 8 different batches. The cells are annotated and assigned to 14 cell types. A total of 4,000 HVGs were used for model training and analysis.The mouse brain dataset consists of 332,129 cells and 4 batches. Ten cell types are present. The data were subset to the 4,000 most highly variable genes before further analysis.The data consist of 25,919 cells in 4 batches and 14 cell types. A total of 4,000 highly variable genes were selected for downstream analyses.68. The dataset is a collection of 14 studies on various types of cancer. It contains 317,111 cells annotated in 25 cell types. We selected 4,000 HVGs for model training.The tumor atlas was obtained from refs. 66. These data consist of 32,472 lung cells from 3 batches and 17 cell types. A total of 4,000 HVGs were used for model training.The lung data were obtained from refs. Further information on research design is available in the Any methods, additional references, Nature Portfolio reporting summaries, source data, extended data, supplementary information, acknowledgements, peer review information; details of author contributions and competing interests; and statements of data and code availability are available at 10.1038/s41592-023-02035-2.Supplementary InformationSupplementary Figs. 1–15 and Tables 1–3.Reporting Summary


Each cluster was then molecularly characterized to identify specific molecular characteristics. We found that cluster C3 showed minimal genetic aberration with a high prognosis. Next, classification models were developed using data from each omic level to predict the subgroup of unseen patients. Decision‑level fused classification models were then built using these classifiers, which were used to classify unseen patients into five novel clusters. We also showed that the multi-omics-based classification model outperformed single-omic-based models, and the combination of classifiers proved to be a more accurate prediction model than the individual classifiers. In summary, we have used ML models to develop a classification method and identified five novel NSCLC clusters with different genetic and clinical characteristics.Non-small Cell Lung Cancer (NSCLC) is a heterogeneous disease with a poor prognosis. Identifying novel subtypes in cancer can help classify patients with similar molecular and clinical phenotypes. This work proposes an end-to-end pipeline for subgroup identification in NSCLC. Here, we used a machine learning (ML) based approach to compress the multi-omics NSCLC data to a lower dimensional space. This data is subjected to consensus K-means clustering to identify the five novel clusters (C1–C5). Survival analysis of the resulting clusters revealed a significant difference in the overall survival of clusters ( It is projected that in the US alone, for the year 2022, there will be 1,918,030 new cancer cases1. Lung cancer alone will contribute to 236,740 new cases (both sexes combined) and will be a leading cause of cancer related deaths1. The first line of treatment for lung cancer is determined based on the histopathological stage and includes chemotherapy, surgery, radiation, targeted therapy, and their combinations2. Even with the advancements in therapies, the 5-year survival rate for lung cancer remains minimal1. The poor survival rate can be attributed to the ineffectiveness of the first line of therapy due to the lack of understanding of underlying tumor heterogeneity at the molecular level5. The heterogeneity of the tumor is largely determined by the genetic and epigenetic makeup of the tumors7. Therefore, precise identification of the molecular subtypes (subgroups) using molecular data is essential in order to effectively use the existing treatment strategies and improve the patient care3.Non-small cell lung cancer (NSCLC) with three subtypes, namely, squamous-cell carcinoma (LUSC), adenocarcinoma (LUAD), and large-cell carcinoma contributes to the majority of the lung cancer-related deaths every year9. Projects like The Cancer Genome Atlas (TCGA) have successfully used the HTS technologies to generate genomic, epigenomic, transcriptomic, and proteomic data to characterize cancer and normal samples across 33 cancer types10. Several studies have attempted subgroup identification using the TCGA data. The initial studies used statistical methods to develop models for subgroup identification and prognosis13. As these studies are based on single-omic, they do not take into account the inter-dependencies between different omics.With the rapid development of high-throughput sequencing (HTS) technologies, massive amounts of molecular data are being generated at various levels of evidence (single-omic level)15. Besides providing additional information, adding multiple levels of evidence will increase the dimension of the data. In the case of machine learning (ML) models, the large dimension of the data may lead to overfitting due to the relatively small number of samples16. To overcome this, first, the large-dimension data needs to be converted into a lower dimension. This can be done using linear projection approaches like principal component analysis (PCA). However, disease phenotype is the resultant of a combination of genetic and epigenetic factors which may not be linear18. Therefore, ML techniques can be used to integrate different levels of evidence and project it to a lower dimension in a non-linear manner using models like autoencoders (AE)19.It is necessary to consider information from multiple levels of evidence while subgrouping to model complex biological phenomena21. Chaudray et al. made one of the early attempts in the direction of early data integration using ML in cancer to predict the survival in hepatocellular carcinoma (HCC) samples using mRNA, miRNA, and methylation data20. The authors identified prognostic subgroups with a significant difference in survival by explicitly applying Cox-regression as the loss function to retain the features contributing to survival. Baek et al. carried out their work in the same direction on pancreatic cancer (PAAD) using mRNA, miRNA, and methylation data to cluster the patients16. Here, mutation data along with multi-omics data and clinical data is used to build a classification model to predict the five-year recurrence and survival. Recently, Zhan et al. combined the information from histopathology images (H and E) and transcriptomic data to predict the survival in HCC patients22. They proved that imaging based predictions are more accurate than Cox-PH based predictions alone.Several attempts have been made to use multi-omics data for various applications, including patient stratification24. Expression patterns of proteins carry vital diagnostic and prognostic information25.All these works demonstrated that multi-omics data conveys more information than single-omic. We hypothesize that addition and non-linear processing of distinct levels of information will further improve the discriminative ability. In this work, in addition to mRNA, miRNA, and DNA methylation data, protein expression data is also integrated. Proteins have a crucial role to play in cellular signaling and phenotype determination22, multi-omics data integration strategy can also be used for subgroup identification. Several studies have discussed the significance of subgroup identification from the point of view of precision therapy3. One of the important directions in the application of ML to multi-omics data is to use it for the identification of the subgroup to which the samples belong. This will help the clinicians decide on the treatment regimen. Our aim in this work is to identify the novel molecular subgroups in NSCLC to convey additional information, besides the existing histopathological grades. This additional information about subgroups will help in the effective utilization of the existing treatment strategies. Also, we aim to build classification models to predict the class labels for new samples. The final classification label will be obtained in two steps. In the first step, the most widely used classification models, support vector machine (SVM), Random forest (RF), and feed-forward neural network (FFNN)  with three hidden layers, each with 2000, 1000, and 500 nodes, and bottleneck layer with 100 nodes was used Fig. . This ar30.To visualize the distribution of samples in these five clusters, both before and after dimensionality reduction by AE, t-SNE plots were generated. It was evident from the t-SNE plots that there was a significant overlap between the samples in the original feature space Fig. d. Also, The PAC value obtained by clustering the multi-omics data without dimensionality reduction by AE PAC = 0.31) was higher as compared to the case of dimensionality reduction by AE (PAC = 0.14) . This indicated that none of the clusters obtained were consistent . The top 100 principal components (PCs) were obtained by applying PCA on the multi-omics data matrix (standardized by mean and standard deviation). These PCs were then clustered using consensus We also carried out the clustering of the subset of selected features from individual levels of evidence (single-omic) and their combinations. Clustering was carried out on these chosen features with and without dimensionality reduction by AE and PCA Table . The PACiClusterPlus32, an existing and widely used statistical multi-omics data integration technique35. iClusterPlus was applied to multi-omics data, and the parameters were tuned using tune.iClusterPlus as recommended by the authors. The clusters obtained using our technique, and iClusterPlus were compared using two cluster evaluation methods, Silhouette coefficient, and Calinski-Harabasz index. The closer the value of the Silhouette coefficient to one and the higher the Calinski-Harabasz index, the better is the clustering. Both these scores indicated that the clusters obtained using the proposed algorithm were better separated than iClusterPlus , Gaussian mixture models (GMM), and regular K-means clustering algorithm. The clustering scores obtained for consensus K-means and regular K-means were comparable in this case , and the complete pipeline was repeated for different architectures of AE’s. The performance was compared using the PAC values for To understand the clinical significance of the different clusters obtained, we compared the survival times among the five clusters Fig. d. The co37, patient stage, tumor purity38, smoking status  and mutation rate were obtained from Chen et al. study33  and Chr 1   Fig. .Figure 441. The infiltration of different immune cells also defines the clinical and biological nature of the cancers. Hence, we performed ESTIMATE analysis in the newly identified subgroups of the NSCLC patients42. The ESTIMATE analysis showed the highest infiltration of immune cells in C3 29. Hence, the classification models were trained using each molecular level of evidence. Based on the classification accuracy obtained on the test data set, it was observed that F3 (DNA methylation) had the highest classification accuracy for both base classifiers  Table .Figure 5As each level of evidence conveys complementary information, classification models were also obtained for the feature representation obtained by fusing features from different levels of evidence. F3 was combined with other levels as it had the highest classification accuracy at the single-omic level. It can be observed from Table To further validate the classification models, we used those samples for which only the methylation data was available. These samples were not used for cluster identification or classification as other levels of evidence were not available . We obtained the subgroup label for these samples using the single-omic methylation non-linear decision-level fused model, as this model had the highest classification accuracy for single-omic data. The overall molecular characteristics of these samples, as expected, followed a similar trend as other samples. The samples in cluster 3 had the least copy number and mutational changes, and the highest immune cell infiltration Fig. . This hi5. The availability of various molecular features as a consequence of the advancements in high-throughput genomic technologies has enabled the better subgrouping of cancer patients. We know that the phenotype of a patient is the resultant of various molecular features interacting non-linearly. To exploit this non-linear relation of molecular features, we used machine learning (ML) based methods. We used mRNA (F1), miRNA (F2), methylation (F3), and protein expression (F4) data from NSCLC samples. The latent representation of this multi-omics data was obtained using AE, a non-linear dimensionality reduction technique. This hidden representation was then clustered using consensus K-means clustering to identify five clusters. The clusters obtained with autoencoder (AE) based clustering were better than those obtained by clustering the preprocessed molecular features directly  were built using each level of evidence to predict the class labels. Linear and non-linear decision-level fused models were used to integrate the prediction probabilities from different classifiers and obtain the final subgroup label. DNA methylation (F3) based model had the best predictive ability among all Table . DNA metentclass1pt{minimacy Table . One of To summarise, this work proposed an end-to-end pipeline for machine learning-based subgroup identification in non-small cell lung cancer (NSCLC). We also proposed and validated the fusion-based classification models for the identification of subgroups in new samples. Since the classification models were built for individual levels of evidence, they can be used in the presence of single omic data as well. The generalizability of our model is yet to be validated due to the limitation in terms of the availability of an independent dataset. Also, exposure to more samples both in terms of heterogeneity and the number of samples, might provide better insights into the resulting subgroups. Therefore, the future work would include validating the proposed method in an independent cohort of data.K-means and GMM with consensus K-means in terms of Silhouette coefficient and Calinski Harabasz index needs further analysis and will be considered for future studies. Further, including additional information from whole slide histopathological (H and E) images as an additional level of evidence can provide better insights.The performance in the current work is based on several assumptions made at different levels. These include preprocessing of the data to reduce dimensionality, using the most well-known ML models, and using cluster labels for subgroup identification. All these need independent evaluation, which may further help to better understand the non-linear processing happening in ML. Also, the better unearthing of biological knowledge using ML models. The comparable performance of regular TCGAbiolinks (v 2.18.0) package in R46 was used to obtain this data for samples from LUAD and LUSC tumor types. Protein expression (RPPA level - 4) data was downloaded from the TCPA data portal48. Further, cBioPortal49 was used to obtain the clinical data. In this study, each level of evidence (single-omic) is referred to as a factor. The mapping from omic levels to the factors is shown in Supplementary Table The proposed pipeline was applied on the TCGA NSCLC (LUAD and LUSC) samples. TCGA multi-omics data comprising mRNA, miRNA, methylation, mutation, and copy number variation were downloaded from the GDC data portal. p) was high compared to the number of samples (n). Hence, the preprocessing of data was carried out to ensure reliability besides reducing the dimension of the data50. Preprocessing of raw data which included, selecting a subset of features, imputing the missing values, and data transformation, was carried out as outlined in Supplementary Figure 51.It can be observed from Supplementary Table 16. Genes in F1 were then sorted based on the standard deviation, and the top 2000 most variable genes were considered for further analysis33. Features retained in both the cases were scaled by min-max normalization to ensure that the data ranged between the values of 0 and 1. In the case of F3 (DNA methylation), beta values were used for analysis. The CpG probes on X and Y chromosomes, those mapping to SNPs or cross hybridized were dropped. The preprocessing was carried out using the DMRCrate (v 2.4.0) package52 in R. Samples and probes with more than 50. Further, the NAs in the retained probes were imputed using K-nearest neighbors (KNN) (K = 5)50. The selected probes were then sorted in the decreasing order based on their standard deviation and the top 2000 probes were considered for further analysis33. As beta values range from 0 to 1, further normalization was not required. For F4 (protein expression level-4), proteins whose expression was missing in more than Briefly, in the case of F1  and F2 , genes with zero expression in more than The preprocessed features corresponding to the feature-vectors (samples) common across all the four different levels of evidence F1–F4) were stacked to obtain the multi-omics data matrix  of the various factors was still high compared to the sample size (n). This   library in Python 3 in Google Colab.Multi-omics data was split with the train-validation split of 90–10% and used to train the AE model. The AE model was trained for 100 epochs with early stopping criteria, i.e., the model training was stopped if the validation error did not reduce for five subsequent epochs. The input data was fed in batches of 24 samples each. Rectified linear unit (ReLU) was used as the activation function, mean-squared error (MSE) as the loss function, and adaptive moment estimation (Adam) as an optimizer, as the input data was continuous. The AE model was built using the 19. Hence, the model which had the smallest difference between the training and validation loss was considered for subsequent analysis.Different architectures of AEs were obtained by varying the number of layers, and the number of nodes in each layer. The performance of AE model was measured in terms of training and validation loss  by varying K from 2 to 10. The process of clustering was repeated 1000 times using 33. The most consistent cluster was identified based on the proportion of ambiguously clustered pairs (PAC). This metric is quantified with the aid of the cumulative distribution function (CDF) curve54. The section lying in between the two extremes of the CDF curve  package.To determine if there exists any difference in the survival between the clusters obtained, Kaplan-Meier (KM) survival curves and log-rank test were used Fig. d. The en33. To summarize, the features with zero expression in more than 56. The cytobands with To identify the features specific to each cluster in each level of evidence, feature selection was carried out by statistical tests as described in Supplementary Figure S542 and subjected to ANOVA. CIBERSORT analysis was carried out using the LM22 signature gene set43. ANOVA with Tukey’s post-hoc test was carried out on these immune cells, and those with 45. The expression data from all the protein-coding genes were used as input for GSEA analysis.Immune, stromal, and estimate score for each sample was obtained from ESTIMATE analysisK-means clustering as output labels. The input to the models were the molecular features specific to each subgroup (DE features) selected from individual omic levels  and L1 weight regularizer (1e-05). The models were trained with different learning rates , and the one with the best accuracy was chosen.As the data was non-linearly separable, a radial kernel was used for SVM. The hyperparameters for SVM and RF were obtained by 5-fold cross-validation (CV) repeated ten times. For the FFNN, appropriate number of layers and neurons were chosen based on the dimension of the input vector. Categorical cross-entropy was used as the loss function with Adam optimizer while training the FFNN. To avoid overfitting, each fully connected layer was followed by a dropout layer (0.1), and To obtain an unambiguous prediction model, the prediction probabilities from each of these classifiers (29. The final classification probability (57.In the case of linear decision-level fused model, the prediction probabilities obtained from entclass1pt{minima58. Here, two non-linear decision-level fused models with different train-test splits were trained. In the first model, both In the case of the non-linear decision level fused model, the concatenated prediction probabilities (29. Here, features from different molecular levels were concatenated to obtain a new feature representation. This fused representation was then used to train each of the ML classifiers.As different levels of evidence carry complementary information, the combination of features from different omic levels will provide additional insights. Hence, the technique of feature-level fusion can help in better classificationSupplementary Information 2.Supplementary Information 2.Supplementary Information 3.