Carcinoma of unknown primary (CUP) is a type of metastatic cancer, the primary tumor site of which cannot be identified. CUP occupies approximately 5% of cancer incidences in the United States with usually unfavorable prognosis, making it a big threat to public health. Traditional methods to identify the tissue-of-origin (TOO) of CUP like immunohistochemistry can only deal with around 20% CUP patients. In recent years, more and more studies suggest that it is promising to solve the problem by integrating machine learning techniques with big biomedical data involving multiple types of biomarkers including epigenetic, genetic, and gene expression profiles, such as DNA methylation. Different biomarkers play different roles in cancer research; for example, genomic mutations in a patient’s tumor could lead to specific anticancer drugs for treatment; DNA methylation and copy number variation could reveal tumor tissue of origin and molecular classification. However, there is no systematic comparison on which biomarker is better at identifying the cancer type and site of origin. In addition, it might also be possible to further improve the inference accuracy by integrating multiple types of biomarkers. In this study, we used primary tumor data rather than metastatic tumor data. Although the use of primary tumors may lead to some biases in our classification model, their tumor-of-origins are known. In addition, previous studies have suggested that the CUP prediction model built from primary tumors could efficiently predict TOO of metastatic cancers . We syst Carcinoma of unknown primary (CUP) is a type of metastatic carcinoma whose primary tumor site is unknown. CUP accounts for approximately 3–5% of all human malignancies . Since tIn order to improve the treatment of CUP patients and prolong their survival time, an accurate identification of tumor tissue-of-origin (TOO) is essential. Currently, there is a clinical practice for tracing the tissue origin of CUP, which involves physical examination, laboratory testing, immunohistochemistry, pathological imaging, and endoscopic examination. However, the results could be highly subjective and only the TOO of about 20–30% CUP patients could be revealed . For theWith the increasing availability of high-throughput genomic and transcriptional data, there are several molecular biomarkers in The Cancer Genome Atlas (TCGA) including somatic mutation, copy number variation (CNV), gene expression, microRNA expression, and DNA methylation, which were used to trace cancer TOO . The mosFinally, there are also a few methods to trace tumor TOO by integrating multiple biomarkers. For example, Hoadley et al. proposed a method to trace the origin of 12 cancer types based on methylation and CNV . MarquarAlthough many previous studies have used molecular profiles such as DNA methylation, somatic mutation, gene expression, and their combinations to predict the tissue origin of CUP, there is still no systematic comparison among them to our best knowledge. In addition, the epigenetic studies on CUP are more or less ignored. To address this need, we aim to compare the predictive ability of these biomarkers and combinations in a unified background. In addition, we aim to investigate whether multi-biomarkers can significantly improve prediction accuracy compared to single biomarkers.1), somatic mutation , and 450 k DNA methylation array data (HumanMethylation450_after_2011_08_02) of the 21 different tumor types were collected from the ICGC data portal3. We used samples from 21 primary tumors as training and validation datasets to construct and validate models for inferring CUP. The data from raw TSV files were pre-processed by extracting and deduplicating, respectively, generating three feature matrices with “p” rows of the tumor samples numbers and “q” columns of gene numbers across aforementioned three categories of biomarkers. Each sample with histologically confirmed origins was tagged for its type of cancer. In particular, the somatic mutation data was extracted and deduplicated to form a feature matrix according to information of icgc_donor_id, chromosome, chromosome_start, and gene_affected before the feature value divided by the length of the gene. In total, 7,224 TCGA samples originating from 21 cancer types were downloaded in our work. Detailed information on the number of samples of each cancer type can be found in The publicly available datasets for gene expression profile  was a learning framework based on boosting tree models for solving supervised learning problems. In this study, all genes obtained from the above step were used to train the classification model based on XGBoost because of its excellent scalability and operation . XGBoostTo compare the accuracy and robustness of different biomarkers in terms of cancer type prediction, publicly available gene expression profile, somatic mutation, and DNA methylation data from 7,224 samples were collected from TCGA for this study. The complete workflow is shown in Due to the intra-tumor heterogeneity, it was critical to identify the most informative genes from the high-dimensional datasets in order to better distinguish true mutation from background noise. Pearson correlation-based feature selection was characterized by fastness in operation speed and simple in complex calculation, which made it a successful multi-variable filtering method for high-dimensional data analysis. It was used to assess correlations between cancer types and corresponding gene features. Hence, we adopted Pearson correlation to select the most informative genes from the generated seven different feature matrices for classification detection. This process consisted of the following four steps: First, we created an array and binarized each row and column of 7,224 tumor samples. If the samples belonged to the tumor type, they would be labeled as “true;” otherwise, they would be labeled as “false.” Second, we calculated the correlation of the feature with samples labeled “true” for each cancer type and then sorted in decreasing order according to their correlation. Third, we took the most important signature, which appeared in the first N genes of the list for each cancer type, where N was an integer. Fourth, we combined the first N genes in lists of 21 cancer genes and removed the redundant genes. Using a series of integers, we generated a corresponding number of gene sets for further classification.To evaluate the performance of the biomarker genes of gene expression profiling, somatic mutation, DNA methylation, and different combinations of them, a 10-fold cross-validation method was used to train XGBoost classification model. Especially to avoid overfitting of XGBoost algorithm, we achieved relatively stable and reliable results through 10 times 10-fold cross-validation, and minimized the percentage of false positives and false negatives as much as possible. The accuracies are shown in We compared the classification performance of various combinations of biomarkers to get through the evaluation index of recall rate, precision, and f1 score. We plotted the heat map of mean value of recall, precision, and f1 score on the 14 top-ranked genes for each cancer type. In We then looked at the performance of XGBoost algorithm using only gene expression values as the train features and fitting the cancer type as labels. We used 10 times of the 10-fold cross-validation method to evaluate the classification performance of each cancer on the 14 top-ranked genes. In two cancer types (PRAD and THCA), the precision was 100%. However, the precision obtained by READ and COAD was lower, at 79.20 and 59.15%, respectively. The precisions corresponding to each cancer were plotted as Data of gene expression profiling, somatic mutation, and DNA methylation can be used to identify the primary site of tumors. However, for the first time, the three biomarkers and their combinations have been used to identify the origin of tumor tissues, and their ability to trace the origin of primary tumors has been compared and analyzed. We carried out a large number of experiments by using a shared sample of 7,224 and combined data from 21 cancer types. By comparing their performance, we found that the gene expression profile data obtained the highest accuracy, while the combined data could not obtain better classification performance. The comparison results are shown in XGBoost has been proven to have better performance than other more traditional models in many machine learning tasks, so we used the XGBoost algorithm to construct the classification model and tested it on seven different biomarker combinations. The results showed that the XGBoost algorithms can predict the cancer type of unknown primary tissue with an efficient accuracy. We chose the 14 top-ranked genes from each cancer and put them together for classification. Our results indicated that the gene expression data obtained an accuracy of 94.63%, which is the highest prediction accuracy. However, the prediction accuracy of the combination with gene expression data with other data was slightly lower than that of using gene expression. The same thing happened with DNA methylation data, which alone had a slightly higher prediction accuracy of 87.59% than both data of DNA methylation and somatic mutation on prediction of cancer tissue origin. Somatic mutation had the worst classification of cancers with a terrible prediction accuracy of 43.33%.Due to the optimal classification performance of gene expression profile, we further functionally annotate the union of 14 top-ranked genes of each cancer type in the gene expression data. The enrichment results are shown in In our study, all data came from the TCGA dataset, and the batch effects of RNA-seq and methylation data may have had a negative effect to our results. However, it is unclear whether the batch correction methods will bring some additional bias and which batch correction method is correct. In TCGA, each sample was divided normalized such that the total number of transcripts is 1,000,000, which actually performs a very rough batch correction. Finally, we added There are some limitations to our study. First, we constructed and assessed the models based on TCGA primary tumor data rather than metastatic tumor data, because it is extremely difficult to collect metastatic samples with a known primary tumor site. In the future, we will try to collect metastatic cancer samples to construct CUP prediction models or test known models. Second, we did not supply an independent dataset for validation since we could not find a database other than TCGA, which has data on gene expression, DNA methylation, and somatic mutation simultaneously. Finally, we only simply concatenated the features of different biomarkers. It might be better to test the effects of interaction terms since the biomarkers are not independent biologically.In summary, this is the first study to compare the power of different biomarkers in inferring cancer TOO under the same condition, including the same dataset, the same preprocessing scheme, and the same classification algorithm. In the future, we will try to include metastasis tumor samples into our study, incorporate independent testing samples, and add interaction terms and novel classification models for improving prediction accuracy.The identification of the origin of tumor tissue was a challenging task. With a large number of molecular profiling, we can use them alone or combine some of them to improve the identification of primary tumor sites. Although we used primary tumor data, the primary information they provided were the most important to pinpoint the exact TOO for CUP. Machine learning algorithms were also effective tools to help classify cancers. The number of features used can greatly affect predictive performance. In this study, we used gene expression profiles, somatic mutation, and DNA methylation data to generate the feature matrix. Then, the optimal number of genes was obtained according to Pearson correlation algorithm, and the classification model was established using XGBoost algorithm. The same approach was used to compare the performance among a combination of some of the aforementioned biomarkers. The experimental results showed that the highest accuracy can be achieved by using gene expression profiling, but combining multiple biomarkers could not achieve better prediction performance.The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.JY and BH designed the study. HL, CQ, BW, and PB collected the data, analyzed the data, interpreted the data, and wrote the manuscript. HL, GT, and JM performed the experiment. GT, XZ, and JM reviewed the manuscript. All authors contributed to the article and approved the submitted version.BW, GT, and JY were employed by the company Genesis Beijing Co., Ltd. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Purpose: Our aim was to construct a signature that accurately predicted the prognostic and immune response of melanoma.Methods: First, the weighted co-expression network analysis (WGCNA) algorithm was used to identify the hub genes related to clinical phenotypes of melanoma in the cancer genome atlas (TCGA) database. Nest, the least absolute shrinkage and selection operator (LASSO) analysis was used to dimensionality reduction of these hub genes and constructed a prognostic signature to predict the prognosis and immunosuppressive response of melanoma.Result: Through in-depth analysis, we constructed a 5-mRNA prognostic signature and verified its prognostic value in internal  and external independent datasets . Based on this signature, the tumor immune microenvironment (TME) of melanoma was characterized, and the result was found that patients in the high-risk group had lower CD8 T cell infiltration and immune checkpoint expression , as well as higher M0/M2 macrophage infiltration. Our results also found the risk score based on a 5-mRNA signature was significantly associated with tumor mutational burden (TMB) and tumor stem cell markers . Lastly, we built a nomogram for clinician prediction for the prognosis of patients with melanoma.Conclusion: Our findings indicated that the 5-mRNA signature has an important predictive value for the overall survival of melanoma. By analyzing the tumor immune microenvironment and tumor stem cell marker between different groups, a new method is provided for the stratified diagnosis and treatment of melanoma. Melanoma is one of the most aggressive and fatal forms of skin tumor, although it accounts for about 4% of all skin cancers, its mortality rate is as high as 80% . MelanomIn this study, by constructing a WGCNA network, we grouped some genes with the same characteristics into the same cluster. Next, we calculated the correlations of all clusters with some important clinical phenotype data. Then, after a battery of screening tests, we developed and used independent external data to validate a 5-mRNA signature that predicted melanoma prognosis. Univariate and multivariate Cox regression analyses combined with key clinical characteristics were used to construct a graph that clinicians can use to predict the prognosis of melanoma. Finally, we found significant associations between risk scores and melanoma immune microenvironment and tumor stem cell markers.n = 472) and clinical feature data (n = 477) were downloaded from the UCSC dataset.1 The RNA-seq data included 471 melanoma samples and 1 normal skin sample. The clinical information of melanoma samples included survival events, tumor staging, grade, Breslow depth value, and Clark level value. We selected samples (n = 472) with both RNA-seq data and clinical phenotype data for subsequent analysis. Later, when building the Lasso regression model, we only retained samples (n = 452) containing both RNA-seq data and survival data for analysis. We installed a 2:1 ratio to randomly group these samples as the training set (n = 301) and internal validation set (n = 151), respectively. All the data used for survival analysis comes from an attachment to an article  of the variance for subsequent analysis. For the processing of externally validated data from the GEO database, we download public data from the GEO database, then RMA corrected, standardized, and then logarithmic.Transcriptome  was carried out with R package “ClusterProfiler.”We uploaded all the genes of the blue module to DAVID’s websiteThe mutation frequency less than 0.05 and synonymous mutations were removed. We used CIBERSORT  online st-test, which used the two-sample t-test and the Welch two-sample t-test, respectively, according to whether the variance was homogeneous. The statistical method used for all survival analyses was the Log-rank test.We use R package “survival” for univariate and multivariate analysis, R package “forestplot” for forest map, and then R package “rms” for the nomogram. The calibration curve and the decision curve analysis performed by R package “rmda” were used to measure the accuracy of the nomogram. The statistical test method of the two groups of data involved in this paper was the R2 = 0.91, The R package of “WGCNA” is used to perform co-expression network analysis. The Pearson correlation analysis was used to detect the outlier samples and we found 8 samples were outlier samples, then we removed them . The 4,9r = −0.32, p = 3e-12). It also showed a strong negative correlation with the Clark level  and the T stage  and the external validation set , the prognostic model showed high predictive power. The risk score = −0.08 × SRGN expression value—0.04 × IDO1 expression value—0.04 × RARRES3 expression value—0.03 × CCL4 expression value—0.007 × HLA-DPB1 expression value. We grouped the population according to the median risk score. We found that the risk of death in the training set and the validation set increased significantly with the increase of the risk score , Breslow depth , Clark level , T stage , N stage , and pathological stage  showed good predictive power of prognosis. Next, we put this factor to perform the multivariate analysis. The results showed that risk score was an independent predictor of melanoma risk  was a monomeric heme-containing oxidoreductase that catalyzes the rate-limiting step of tryptophan (Trp) metabolism to kynurenine (Kyn) . IDO1 su T cells . In gene T cells , therefoIn conclusion, through deep mining, we have built a 5-mRNA prognostic signature that not only predicts the overall survival rate of melanoma but also according to its classification of immune microenvironments, it is possible to predict the effect of immunotherapy.The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found in the article/YZ, JP, and HD performed the data management and analysis procedures. YZ, HD, and XF analyzed the results. YZ, NZ, and JP contributed to the writing of the manuscript. All authors contributed to the study conception and design, read and approved the final manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


Triple-negative breast cancer (TNBC) is a heterogeneous disease and we have previously shown that rapid relapse of TNBC is associated with distinct sociodemographic features. We hypothesized that rapid versus late relapse in TNBC is also defined by distinct clinical and genomic features of primary tumors.n = 453), and whole genome copy number and mutation data for 171 cancer-related genes (n = 317). Association of rapid relapse with clinical and genomic features were assessed using Pearson chi-squared tests, t-tests, ANOVA, and Fisher exact tests. We evaluated logistic regression models of clinical features with subtype versus two models that integrated significant genomic features.Using three publicly-available datasets, we identified 453 patients diagnosed with primary TNBC with adequate follow-up to be characterized as ‘rapid relapse’ , ‘late relapse’  or ‘no relapse’ (nrTNBC: > 5 years no relapse/death). We explored basic clinical and primary tumor multi-omic data, including whole transcriptome (n = 63 patients), testing (n = 63) and independent validation (n = 34) cohorts, although performance of all models were overall modest.Relative to nrTNBC, both rrTNBC and lrTNBC had significantly lower immune signatures and immune signatures were highly correlated to anti-tumor CD8 T-cell, M1 macrophage, and gamma-delta T-cell CIBERSORT inferred immune subsets. Intriguingly, lrTNBCs were enriched for luminal signatures. There was no difference in tumor mutation burden or percent genome altered across groups. Logistic regression mModels that incorporate genomic features significantly outperformed standard clinical/subtype models in training (We identify clinical and genomic features associated with rapid relapse TNBC for further study of this aggressive TNBC subset.The online version contains supplementary material available at 10.1186/s12885-021-08320-7. Triple negative breast cancer (TNBC) is an aggressive breast cancer subtype defined by lack of targetable estrogen receptor (ER), progesterone receptor (PR), and HER2 . TNBC acTP53 (~ 75% of TNBCs) and PIK3CA ~ 25% [Advances in sequencing technology have facilitated comprehensive molecular profiling of breast cancers, including subsets of TNBC , 8. Two CA ~ 25% –13 whileCA ~ 25% –13. The CA ~ 25% .To more accurately understand the differences in patient outcome in TNBC, we sought to understand distinct clinical and genomic features among primary TNBCs categorized based on outcome: rapid (rrTNBC), late (lrTNBC) and no relapse (nrTNBC). In several large TNBC cohort studies, the median time to distant metastasis was around 2 years, ranging from 19.7 to 31.2 months, –16 thus As an initial investigation of genomic features associated with rrTNBC, we aggregated data across multiple cohorts then utilized a train/test split and an independent validation cohort to model predictors of rapid versus late relapse.Patient-specific data were obtained from The Cancer Genome Atlas (TCGA) , MoleculFor data from the METABRIC, normalized gene expression data, copy number data, and somatic mutation data for 171 cancer-related genes were obtained from the publicly available European Genome-Phenome Archive (IDs EGAD00010000210 and EGAD0001000021) and associated publications , 21. Copn = 287, TCGA n = 160, neoadjuvant dataset n = 446) were extracted, quantile normalized within TNBCs from each study, and subsequently median centered. We evaluated summary expression metrics . One hundred twenty-five published gene expression signatures were calculated as we have previously described [Given gene expression data from multiple studies and disparate platforms, gene expression data for all TNBCs for each dataset  AUC of the 25 runs, and 95% confidence interval was calculated using the standard deviation of the sample of means.We compared the performance of three logistic regression models in predicting rapid relapse versus late relapse. The “null model” contained only clinical variables (age/stage at diagnosis and PAM50/TNBC subtype). The “null plus significant genomic features”, adds any feature significantly different between rrTNBC and lrTNBC with a nominal p-values comparing rapid vs. late and relapse vs. no relapse using ANOVA and logistic regression. For count variables (e.g. mutated vs. not) we used Fisher exact tests to evaluate relapse vs. not and rapid vs. late relapse. P-values for CIBERSORT and mutation signatures were evaluated using logistic regression, while CNAs, and mutations were evaluated using Fisher exact tests. Data visualization was made using ggplot2 [Differences in patient and tumor characteristics were evaluated using Pearson chi-squared tests. The association of gene signatures with neoadjuvant chemotherapy response was evaluated using simple linear regression and t-tests. All calculations of association were multiple-testing corrected using the Benjamini–Hochberg procedure for false discovery rate. For continuous variables, we calculated  ggplot2 . All staFrom three large cohorts with primary breast cancer genomic data – TCGA,  METABRICp = 0.01), but not PAM50, was associated with significant differences in DMFS. The strikingly different visualized outcomes suggests that our relapse categorization does, in fact, identify truly distinct subsets based on outcome when compared to approaches that focus on intrinsic features.We assessed the percentage of total DMFS events each year Fig. b. In thip = 1.9e-10). The majority of patients were basal-like PAM50 subtype (78%), but, lrTNBCs were significantly more likely to be non-basal . Lehmann/Pietenpol TNBC subtype also reflected significant differences across groups (Chi-square p = 0.02). The immunomodulatory phenotype was highest in nrTNBC , luminal androgen receptor was highest in lrTNBC , and basal-like 2 was highest in rrTNBC . A subset of patients in this cohort  had data on response to neoadjuvant chemotherapy (NAC). As anticipated, those patients with rrTNBC or lrTNBC were significantly more likely to have residual disease (RD) after neoadjuvuant chemotherapy , relative to those with nrTNBC . Intriguingly, the rate of residual disease was similar among rrTNBC and lrTNBC despite markedly different timing of relapse.We evaluated the association of clinical, pathologic, and intrinsic expression subtype with rapid vs. late vs. no relapse status Table . There wp < 0.001). The patients with data on response to NAC all had whole transcriptome data but no available mutation or copy number data, so we calculated a score for 125 published gene expression signatures and evaluated the association of each signature with NAC response (pCR vs. RD) and DMFS. Signatures were grouped by phenotype as previously described [n = 127 patients; Fig. Response to NAC is known to be a robust prognostic biomarker in TNBC . In thisescribed  n = 127p < 0.001p < 1.2e-8) and anti-correlated with immune suppressive cell types including M2 macrophages, memory resting CD4 T-cells, resting NK cells, and resting mast cells. ER/HER2 signatures reflected an almost opposite pattern to immune signatures, with positive correlation to immune suppressive cell types and anti-correlation with anti-tumor immune cell type. Metabolic signatures appeared to have a strong correlation specifically with M0 macrophages . As a sensitivity analysis, we evaluated the association of three representative signatures from each group with 7 immune cell-type specific signatures from MSigDB [To understand what immune cell types in the tumor microenvironment may be reflected by the immune signatures, we visualized the association of three representative signatures from each group  with the relative proportion of 22 inferred immune cell subsets via CIBERSORT . Most CIBERSORT immune subsets were not statistically significant . To more comprehensively investigate inferred immune subsets, we evaluated the association of summed protumorigenic subsets  and summed antitumorigenic subsets  with rapid versus late versus no relapse , highest in ‘no relapse’, while there was no difference in the protumorigenic subsets (ANOVA p = 0.62), although the absolute differences were small.To assess pathways and phenotypes associated with rapid vs. late vs. no relapse, a score was calculated for 125 published gene expression signatures across the entire dataset  of patients had data on single nucleotide variant/mutation data including 171 cancer-related genes and whole genome CNAs . Only a 53 of patPIK3CA mutations were more frequent in relapse relative to nrTNBC. In addition, PTEN, ARID1A, and RYR2 mutations were enriched in nrTNBC relative to rrTNBC . We then compared rrTNBC vs. lrTNBC . Among CNAs, the copy number landscape was similar across the rapid vs. late vs. no relapse groups  for relapse (rrTNBC + lrTNBC) vs. nrTNBC Fig. c becauseNBC Fig. d and foup < 0.005). The genomic features that contributed most included clinical features , mutations , copy number alterations , and expression signatures . We evaWe previously demonstrated in two large cohorts that disparities in sociodemographic features are strongly associated with rrTNBC, –19 and iwhat genomic features contribute most. More complex modeling approaches  did not improve model performance and led to challenges with overfitting. Our models were evaluated in over two times the number of TNBC patients available in TCGA [Our goal was to identify distinguishing features and determine if predictive models incorporating clinical, expression-based subtype, and/or multi-‘omic models could identify patients at high risk of rapid relapse. lrTNBCs are more likely to be non-basal  and our data identify eight luminal signatures are associated with late relapse. Multiple groups have identified a ‘luminal androgen receptor’ subset of TNBC based on molecular classifications, , 11 and  in TCGA  alone –  in TCGA . CollectStage at diagnosis was strongly associated with rrTNBC in univariate analyses and in logistic regression models. One hypothesis is that stage at diagnosis captures non-biological features including socioeconomic or demographics features –45. RaceResponse to neoadjuvant chemotherapy remains the best prognostic biomarker for TNBC,  but therClinically, it is clear that a subset of patients with TNBC have highly aggressive, largely treatment-refractory disease , 2, 14. While this study presents promising methods to categorize TNBC relapse it does possess significant limitations. Categorization of tumors depends on study-reported estrogen receptor (ER) status; variability and changes in standard determination of estrogen receptor positivity since 2010 guidelines  may haveIn conclusion, we provide evidence that rrTNBC reflects a distinct clinical entity characterized by unique genomic features. Predictive modeling using clinical and genomic features in these datasets revealed modest results, but with improved data may identify patients at high risk for ‘rapid relapse.’ Multi-level analyses of the interaction between clinical, multi-‘omic, and sociodemographic features and timing of relapse are warranted.Additional file 1: Figure S1. Additional Analyses of Gene Expression Signatures. (A) Sensitivity analyses of correlation between three representative signatures from each group  with the immune cell-specific signatures [n = 453), visualized using CorrPlot [(B) Heatmap with hierarchical clustering of the gene expression signatures with the greatest variance (top 25%) across the dataset. Figure S2. Variation of Expression Signatures Across Rapid vs. Late vs. No Relapse Groups. The calculated score for 16 published gene expression signatures that demonstrated statistical significance (ANOVA FDR p < 0.05) comparing rapid vs. late vs. no relapse. The score value is presented for immune signatures (A) and estrogen/luminal signatures (B). Each boxplot represents the 25th to 75th percentile with the median indicated as the central line and whiskers indicating 1.5 x interquartile range. (C) Immune cell subset proportion from CIBERSORT, visualized as relative values (Z-score) with rapid relapse (red), late relapse (green), and no relapse (blue). Figure S3. Mutation and Modeling Sensitivity Analyses. (A) CoMut plot of gene-level mutation for the entire cohort, with mutation indicated in blue, visualized with ‘GenVisR’ package [(B) Frequency of gene-level copy number gains (red) or losses (blue) across the genomegnatures , 31 acroCorrPlot , 29. (B) package . (B) Fre


GDCquery function of the TCGAbiolinks package in the R software. We used least absolute shrinkage and selection operator (LASSO) as feature selection method. We compared the results of the new proposed model with and without LASSO with the results of the single 1D-CNN and machine learning methods which include support vector machines with radial basis function, linear, and polynomial kernels; artificial neural networks; k-nearest neighbors; bagging trees. The results show that the proposed model with and without LASSO has a better performance compared to other classifiers. Also, the results show that the machine learning methods  with under-sampling have better performance than with over-sampling techniques. This is supported by the statistical significance test of accuracy where the p-values for differences between the SVM-R and SVM-P, SVM-R and ANN, SVM-R and KNN are found to be p = 0.003, p =  < 0.001, and p =  < 0.001, respectively. Also, SVM-L had a significant difference compared to ANN p = 0.009. Moreover, SVM-P and ANN, SVM-P and KNN are found to be significantly different with p-values p =  < 0.001 and p =  < 0.001, respectively. In addition, ANN and bagging trees, ANN and KNN were found to be significantly different with p-values p =  < 0.001 and p = 0.004, respectively. Thus, the proposed model can help in the early detection and diagnosis of cancer in women, and hence aid in designing early treatment strategies to improve survival.Cancer tumor classification based on morphological characteristics alone has been shown to have serious limitations. Breast, lung, colorectal, thyroid, and ovarian are the most commonly diagnosed cancers among women. Precise classification of cancers into their types is considered a vital problem for cancer diagnosis and therapy. In this paper, we proposed a stacking ensemble deep learning model based on one-dimensional convolutional neural network (1D-CNN) to perform a multi-class classification on the five common cancers among women based on RNASeq data. The RNASeq gene expression data was downloaded from Pan-Cancer Atlas using  The global cancer statistics estimated about 19.3 million new cancer cases in 2020 alone, and close to 10 million deaths of 36 cancers in 185 countries3. Breast cancer (with estimated 2.3 million new cases) is the most common diagnosed among women, followed by lung, colorectal, thyroid, and ovarian cancers. Moreover, the most leading cause of death is the lung cancer (with estimated 1.8 million deaths). The cancer burden is expected to increase to 28.4 million cases by 20403.Recent global public health research shows an epidemiological paradigm shift from infectious to non-communicable diseases, the latter including different types of cancers. The incidence and prevalence of cancer are on the increase worldwide, both in the developing and developed countries6. Recently, RNASeq gene expression data8 has emerged as the preferred technology for the simultaneous quantification of gene expression compared to the DNA microarray10. The classification of cancer using gene expression data from RNASeq technology provides the opportunity to discriminate healthy and diseased samples or among different types and subtypes of cancer more accurately11. RNASeq gene expression data have had a profound impact on disease diagnoses and prognoses through accurate disease classification, which has helped clinicians to choose the appropriate treatment plans for patients12. There exists striking disparities in the global cancers among women13. Correct classification of these cancers is among the essential strategies to inform clinical decisions and reduce morbidity and mortality from cancers among women.Cancer tumor classification based on morphological characteristics alone has serious limitations in differentiating among cancer tumors and may cause a strong bias in identifying the tumor by experts15. In addition, the samples also contain several genes that are uninformative and degrade the classification performance16. As a way to mitigate this problem, it has been suggested to first perform filtration and feature selection through methods such as the two-sample t-test at a given stringent significance threshold before going further with model building17. This procedure ensures that only informative and sufficiently differentially expressed genes between the outcome classes are used in building the classifiers. This process of feature selection motivates the evaluation of methods for the classification of different cancer tumors and disease stages, to improve early detection and the design of targeted treatment strategies that may reduce mortality. The two-sample t-test as a method for feature selection is easy to use but comes with the problem of multiple testing that the user has to deal with. Other methods or approaches that are model based, such as regularized regression methods, have recently become popularly used.Although the use of gene expression data from RNASeq technology has improved cancer classification, it has its own limitations due to it being characterized by small samples sizes, with each sample having a large number of genes 20. These studies, however, differ in the methods used for feature (gene) selection. In particular, Castillo et al.18 used differential expression analysis and minimum-redundancy maximum-relevance method for feature selection in the microarray and RNASeq data. García-Díaz et al.11 applied a grouping genetic algorithm for feature selection in five different cancers using RNASeq data.There are many supervised and unsupervised machine learning as well as deep learning methods developed for cancer classification using gene expression data. Several studies reported a higher predictive performance of the machine learning methods on the multi-class cancer classification problem19, on the other hand, used support vector machines (SVM) and a recursive feature elimination method to remove the uninformative genes. These studies concentrated on the application of machine learning methods on a multi-class classification problem. Several methods developed by other authors for multi-class cancer classification are reported to have a higher predictive performance compared to existing methods21. Lee et al.22 proposed a new ensemble classifier called cancer predictor using an ensemble model (CPEM), for classification of over 31 different cancer tumors downloaded from TCGA repository. In addition, they assessed different input features such as mutation profiles, mutations rates, mutation spectra, and signature. Thereafter, they investigated different machine learning and feature selection models in order to find the best model which achieved 84% of accuracy using 10 folds cross-validation. Furthermore, they used the six most common cancers out of 31 types and the model achieved 94% of classification accuracy. However, some of the statistical methods achieved results that are better than machine learning algorithms.Ramaswamy et al.23 compared machine learning and deep learning methods in classifying 11 different tumor classes using microarray gene expression data. They implemented eight supervised machine learning methods including KNN, support vector classifier (SVC), logistics regression (LR), linear discriminant analysis (LDA), naïve Bayesian classifier (NB), multi-layer perceptron (MLP), decision trees, and random forest (RF) as well as one unsupervised method such as k-means. In addition, they applied two deep neuronal networks (DNN) methods. Their results showed that the deep learning methods outperformed the other machine learning methods.Tabares-Soto et al.24. Findings from this study might help in the early detection and accurate classification of these cancer types and contribute to efforts of finding therapies that may increase survival for women at risk.In this study, we propose a stacking ensemble deep learning model that uses five 1D-CNN as base models. The results of these models are combined using NN, which is used as a meta model to classify the most common types of cancers among women using RNASeq data. We compared the performance of our new proposed model when using the full list of genes as input with its performance when using a reduced selection of genes using LASSO. Also, we consider comparing the performance of our current proposed model with other machine learning methods since there are limited studies that compare the performance of deep learning and machine learning methods to classify different types of cancer. LASSO is used as a feature selection technique, since it has been shown to improve prediction accuracy, especially when there is a small number of observations and a large number of featureshttps://portal.gdc.cancer.gov/), using R statistical software version 3.6.3 via the TCGAbiolinks package27. The data contains 2166 samples from the top five common cancers between women. We applied eight multi-class classification methods to find the best classifier that discriminates among five common cancers among women. The machine learning methods were implemented in the R software, while the deep learning method (1D-CNN) was implemented using TensorFlow with Keras.In this paper, we downloaded the RNASeq gene expression data from Pan-Cancer Atlas  from RNASeq gene expression datasets. The cancer tumors were breast, colon adenocarcinoma, ovarian, lung adenocarcinoma, and thyroid cancer. The datasets were downloaded from Pan-Cancer Atlas using “Gene expression” and “Gene expression quantification” are passed to the data.category and data.type arguments, respectively, to filter the data files to be downloaded. The platform “Illumina HiSeq” was used to download the data. We used “results” for file.type argument to filter the legacy database, and “RNA-Seq” was chosen as experimental.strategy argument to produce the expression profiles. Moreover, we selected the tumor samples to be downloaded using the “Primary solid Tumor” value as sample.type argument. The downloaded data in a matrix form included five types of cancer, where the columns represent the samples and the rows containing the genes, i.e. features . The datasets were combined to give 2166 tumor samples obtained from all the five cancers, with 19,947 common genes. Due to the curse of high dimensionality, we performed filtration and feature selection to reduce the high number of genes in order to exclude irrelevant and noisy ones that could affect the performance of the methods. Thus, we applied normalization, transformation, and filtration steps to the data to select the informative genes that potentially could contribute positively to the classification accuracy. Table TCGAanalyze_Preprocessing function in TCGAbiolinks package26, which utilizes an array-array intensity correlation (AAIC) approach to obtain a EDASeq package to adjust the GC-content effect or other gene level effects, distributional differences between lanes, and global-scaling and full-quantile normalization28. TCGAanalyze_Filtering was used for filtering out the irrelevant genes and returned the genes with the mean intensity across the samples higher than 0.25, which was the threshold defined quantile mean. After applying this process, we found 14,899 genes to be informative meaning 5048 genes were rendered irrelevant. For further reduction and precise differential gene expression analysis, we used DESeq package in R31. DESeq analyses the gene expression based on the negative binomial distribution and a shrinkage estimator for the distribution’s variance. After using DESeq package, 12,649 genes out of the 14,899 post initial filtering were found to be differentially expressed meaning a further 2250 genes were removed.We used mentclass2pt{minim24.The RNASeq gene expression data after preprocessing had 12,649 dimensions or features, which was still huge given that the number of samples was 2166. Therefore, LASSO regression was used to decrease the number of genes or features that enabled us to effectively analyze the data. LASSO is a method that performs regularization and feature selection through a shrinkage (regularization) process. LASSO penalizes the regression coefficients with ith response. The log-likelihood of the multinomial model under LASSO model can be written in a generalized form as32In the case of the multinomial response with The outcomes in the data can be denoted in the form of a matrix Y of dimension mentclass2pt{minim0 in Eq.  we obtaiWe chose LASSO regression because it uses the sum of the absolute values of the model parameters, restricted to be less than a fixed value as the penalty. LASSO, with tenfold cross-validation returned 173 genes ; support vector machines (SVM) with radial basis function, linear, and polynomial kernels; artificial neural networks (ANN); K-nearest neighbors (kNN); and bagging trees.36, is a well-known machine learning method that has been used widely in many fields, including gene expression data analysis38. SVM aims to find an optimal hyperplane that separates the data into two different classes for the binary classification problem, determined by a subset of samples known as support vectors39. SVM can handle non-linearly separable problems by transforming the data using mapping kernel functions. These functions include radial basis, polynomial, and linear functions40. The SVM is implemented using kernlab package in R statistical software41.Support vector machines (SVM)Suppose we have The separating hyperplane can be defined byTherefore, a separating hyperplane for the two classes should followAfter the rescaling, the distance from the nearest point in each class to the hyperplane becomes For the multi-class problem there are many types of extensions that can be used such as one-vs-one, one-vs-all (one-vs-rest), decision directed acyclic graph based approach, multi-class objective function, and error-correcting output code based approach. These approaches use the same binary classification principle, where the multi-class problem is decomposed into multiple binary problems. In the one-vs-one multi-class classification problem the SVM classifier produces all possible pairs of binary classifications. Suppose we have 42. The data flows from the input layer to the output layer through the hidden layers43. The nodes between the input through the hidden layers to the output layers are connected by appropriately defined weights or weight functions. The number of input and output layers depends on the number of covariates in the dataset as well as a number of classes in the outcome variable43. The inputs are weighted by multiplying every one of them by a weight which is a measure of its contribution. Therefore, the hidden layer receives the weighted inputs and produce outputs using an activation function(s)42. ANN can be implemented using the R package nnet44.Artificial neural networks (ANN) is a computational method constructed from many layers, each layer consisting of nodes called neuronsSpecifically suppose we have gene expression data with In the second layer, the outputs of the hidden units are linearly combined to give the activationsK-nearest neighbors (kNN) is a non-parametric method used for classification and regression45. The idea behind kNN lies in finding the most nearest neighbors of the new sample, and this is based on the similarity and distance metric46. In kNN, k-neighbors determine the class of a new instance; therefore, the new sample is assigned the class that is most likely among the k-neighbors42. In general, kNN has two phases; the first is finding the nearest neighbors, and the second is assigning the class of a new sample using those neighbors by the majority vote rule. kNN is implemented using R package caret47.Suppose we have two samples A new sample is allocated the class that most of its neighbors fall, that is, model class of its neighbors.48. The method splits the data into many bootstrap samples, thereafter, train the model for each bootstrap. Then, the overall prediction obtained by averaging and voting for regression and classification, respectively.Bagging trees or bootstrap aggregation method is appealing because its ability to reduce the variance associated with a prediction and hence, improve the prediction accuracy51. Deep learning is applied successfully in many areas including medical image analysis, computer vision, drug design, and bioinformatics and yield performance that sometimes surpass expert personals' performance52. CNNs are a regularized version of fully connected networks (multilayer perceptrons), in which each neuron in one layer is connected to all the neurons in the layer that follows it. The connectivity between the neurons is inspired by the biological process and resembles the arrangement of the animal visual cortex. In contrast to other image classification and analysis algorithms, CNNs use little pre-processing by learning the filters that capture temporal and special dependencies in an image instead of hand-engineering them. A sequence of stacked layers  makes the architecture of CNNs and in each layer, a differentiable function is used to transform one volume of activations to the layer that follows it. The major building blocks in CNNs are the convolutional layers, which apply filters on an input image to create a feature map. To get a good classification performance, CNNs normally decrease the features of the image into an easier processed arrangement without dropping essential features. The pooling layers use max pooling or average pooling to reduce the dimension of the image’s features. The fully connected layer is an important component in the CNNs architecture that derives the final classification results.Convolution Neural Networks (CNNs) are deep learning architectures that have multi-layers between the input and output and are designed for image analysis and classificationf by f and stride = s are applied and the result is 3 × (m − f + 1) × (n − f + 1) hidden feature neurons if a stride of 1 is used and the pooling layer result will be 3 × (m − r + 1)/2 × (n − r + 1)/2 hidden features neurons when applied to 2 × 2 regions. The convolution operation generates the features map by multiplying the element of the input array by the element of the filter element wise and summing up the result to generate on pixel of the features map. Sliding the filter across the matrix and repeating the multiplication and summing up operations will generate the rest of features map pixels. The mathematical equation of this convolution operation is given as followsThe input to the CNNs is a tensor of order 3 that represents an image having m rows and n columns with 3 color channels (RGB). The tensor encodes the pixel intensities of the image and produces the input features that go through the convolutional, pooling, and the fully connected layers sequentially. In the convolutional layer, a filter of size 53. The one dimensional that we used in this study was constructed by Mostavi et al.53 for predicting cancer tumor based on gene expression data. The architecture of the model when using LASSO as a feature selection technique is shown in Fig. 1D-CNN is a simple CNN architecture that has only one convolutional layer. The simple design of this model leads to reduced number of parameters that can be adjusted during the training process therefore, it is highly needed in the genomic studies where it is difficult to collect large data to train a deep learning model that has very large number of parameters55. To avoid over-fitting, we can use a model with fewer parameters or obtaining more data. A model with fewer parameters can cause high bias. Since obtaining more data is not easy in the medical field, then a model with fewer parameters seems to be the alternative, but modern approaches in deep learning repeatedly show the benefits of using models with a large number of parameters57. Therefore, finding a way of adjusting the variance by minimizing noisy data can help solve the over-fitting problem. Since too much training can result in over-fitting, whereas too little training can result in under-fitting then the model can be regularized using the early stopping mechanism. We can implement the early stopping mechanism in the training procedure to make the architectures better fit the training data with each epoch and determining the number of epochs that can be run before the pre-trained model begin to overfit.We applied 1D-CNN with early stopping regularization to avoid over-fitting. The over-fitting is usually caused by training the model too much, making it pick up the noise as an essential part of the data instead of relying only on the training data. Such noise is normally unique to each training data. It can lead to high variance in the model estimates. On the other hand, too little training can result in under-fitting or high bias. Therefore, the variance and the bias have a negative relationship meaning that if the bias increases for fixed mean square error, then the variance will decrease and vice versa and that is known as the bias-variance tradeoff5. Overall, each base learner votes and the final prediction is gained by the meta-learner, which is a model that learn to correct the prediction of the base-learners. Therefore, the ensemble approach results in prediction accuracy that is better than the single learners. Generalizability of an ensemble usually reduces the variance in the prediction, and thus ensure the most stable and best possible prediction is made. The meta model takes the output of the sub-models (base-learners) as input and then learns to merge the input prediction to make the final prediction which is better than each of the base-classifiers. Figure Ensemble learning is the process of improving classifiers performance by combining the contribution of the trained sub-models to solve same classification problemWe used different performance metrics to evaluate the performance of the classification methods. These metrics are namely accuracy, kappa, specificity, sensitivity, the area under the curve (AUC), precision, F-measure, and ROC curve. The accuracy measures the percentage of correctly classified cases but is not sufficient for measuring the performance of the classifier, especially if we have unbalanced data . Sensitivity measures the percentage of the cases that are correctly classified as having cancer among those samples that are truly cancerous. Therefore, it measures the fraction of the correctly predicted cancer cases. Specificity measures the percentage of cases that do not have cancer, which are correctly identified to be so. In other words, it measures the true negative rate. Precision is the percentage of cases among those classified as positive that are truly positive, i.e., having cancer, and sometimes this measure is called the positive predicted value. F-measure is a measure that balances between precision and sensitivity.MultiROC package in R58. MultiROC calculates and visualizes ROC curve for multi-class using micro-averaging and macro-averaging approaches. Micro-averaging ROC-AUC converts the multi-class classification into binary classification by stacking all groups together. Macro-averaging ROC-AUC uses one versus the rest approach by averaging all group’s results and linear interpolation used between the points of the ROC. Confidence intervals for kappa statistics were computed using vcd package.We also compared the predictive performance of the methods using the receiver operating characteristic (ROC) curve plots. These figures were plotted using 60. SMOTE is a data augmentation method that add new data to the minority class that are synthesized from the existing data instead of duplicating the data, because the duplication will not provide any new information to the model. SMOTE works by first selecting randomly a class instance a from the minority class then it chooses randomly one of the k nearest neighbors b to create the synthetic instances as a convex combination of a and b and finally, it forms a line segment in the feature space by connecting a and b.Imbalanced class sizes may lead to poor predictive performance particularly for the classes with small samples Table . In ordek minority nearest neighbors to balance the class63. This statistical technique increases and generates the samples to reach the highest majority class and it makes the samples more general. SMOTE is implemented using caret package in R by adjusting the sampling method in the train control parameter to be ‘SMOTE’.We synthesized the minority class from existing samples by selecting randomly the closest 65. DOWN is implemented using caret package in R by adjusting the sampling method in the train control parameter to be ‘DOWN’.Under-sampling technique (DOWN) tends to produce a new balanced subset of the original dataset by randomly removing instances usually from the majority class observationsresamples method in R to analyze and visualize the estimated performance of the models. We used the summary function to compute summary statistics across each model/metric combination. Diff function in R is used to estimate the differences between the methods. The diff function performs a pairwise comparisons to compute the differences between pairs of consecutive elements using Bonferroni correction as an adjustment method. Bonferroni test is a type of multiple testing method used in statistical analysis to reduce the instance of a false positive and prevent the data from appearing incorrectly to be statistically significant67.There are many different techniques that can used for comparing the accuracies of the machine learning models. In this work, we used the We found that the performance of the machine learning methods when LASSO as feature selection technique used is by far better than when it is not used. The performance of the methods in terms of overall statistics are summarized in Table The accuracy, precision, sensitivity, and F1-Score performance measures for the overall multi-class classification problem based on the under-sampling technique (DOWN) are presented in Table The accuracy, precision, sensitivity, and F1-Score performance measures based on per-class statistics using the under-sampling technique method (DOWN) are presented in Table The results that are presented in Table Figures The multi-class classification performance of the 1D-CNN model has been evaluated for each fold, and the average classification performance of the model is calculated. The overlapped confusion matrix (CM) is shown in Figs. Although we are using RNAseq data with a high number of genes, deep learning method outperformed the machine learning methods noting that a rigorous preprocessing step including a model-based approach using LASSO regression was applied to reduce the number of genes to be less than the number of observations.The results that are presented in Table p = 0.003, ANN p =  < 0.001, and KNN p =  < 0.001. While SVML differed statistically to ANN p = 0.009, and SVMP differed statistically to ANN p =  < 0.001 and KNN p =  < 0.001. Moreover, ANN differed statistically to bagging trees p =  < 0.001, as well as KNN differed statistically to bagging trees p = 0.004.A comparison of the methods was statistically conducted using the pairwise analysis test which produced pairwise statistical significance table of scores where the lower diagonal of the table shows p-values for the null hypothesis (distributions are the same), smaller p-value is indicative of a better model. The upper diagonal of the table presents the estimated differences in mean accuracy and kappa coefficient between the distributions. From Table We applied a novel stacking ensemble deep learning model to classify five common cancers among women: breast, colon adenocarcinoma, lung adenocarcinoma, ovarian, and thyroid cancers. The performance of the current proposed model is compared with the single 1D-CNN and machine learning methods that are mostly used in cancer types classification. We showed that the best machine learning average results were obtained using 173 genes based on the under-sampling technique, while our proposed model has the highest performance based on the early stopping regularization. The improvement in accuracy was achieved by optimizing several parameters. We used LASSO as a feature selection technique with our proposed model to explore the integration of features selection method with a deep learning approach because features selection in deep learning is still unexplored area due to the black box nature of the deep learning methods. The results of the proposed model without using LASSO as a feature selection technique is comparable to the results with LASSO. This indicates that the 1D-CNN performs features selection through its layers. Bagging trees obtained excellent results, with a maximum accuracy of 99.2% among the machine learning models based on the under-sampling technique. In contrast, ANN showed the least accuracy of 80.7% for classifying the most common cancers among females. The SVM-P method showed performances that was close to the bagging trees method with an accuracy of 98.9% when we used the under-sampling technique. Overall, our results showed that SVM-R, SVM-L, SVM-P, ANN, KNN, and bagging trees were improved in performance if under-sampling is applied compared to over-sampling. We conclude that our proposed model is the best methods for the test dataset in this study. However, bagging trees is the best model among the machine learning models.19. These differences can be explained by variations in the type of cancers studied and the methods used for feature/gene selection. A study by Yang and Naiman14 introduced and validated a gene selection approach using machine learning methods but did not assess the performance of the machines. Our findings demonstrated that, our proposed model can achieve a higher performance on cancer tumor classification using gene expression data. Both deep and machine learning methods and a combination of both can assist in predicting or detecting cancer susceptibility in the early stages and therefore, aid in designing early treatment strategies, and in turn increase survival of the high-risk women.Overall, our proposed model surpassed the single 1D-CNN and the machine learning methods in the classification of common cancers among women. These findings are different from those reported in other studies68. This process enabled us to retain the most important features (genes) for classification and prediction. In order to avoid over-fitting and the bias in the skewed class distribution we used over and under-sampling imbalance handling techniques, which improve the machine learning performance. In general, our results show that under-sampling technique improved the methods performance, and this is confirmed in previous studies69.Because of the large number of genes in the gene expression data, we used LASSO regression as a rigorous feature selection method that reduced the dimensionality of the data setsp < 0.05) between the machine learning methods, which demonstrates that the performance of the machines on cancer classification is not the same. However, deep learning methods outperformed the machine learning methods in cancer classification, which is similar to a previous study23. Overall, the accuracy of our proposed model on the full features and on the features that are selected using LASSO are 99.48% and 99.45, respectively, which are 5.05% and 5.02% higher than accuracy obtained by23 which is 94.43%. We note that Tabares-Soto et al.24 used microarray gene expression data, focusing on 11 type of cancers for both males and females, compared to RNASeq data used in this study to classify five common cancers among females. This study also did not consider class imbalance handling methods as applied in the current study and had 12-times lower sample size (n = 174) than in our study (n = 2166). With larger sample size, more samples are available to train the models. These issues were, therefore, likely to affect the reliability of findings and potentially affecting the performance of the methods. Our study was limited to the gene expression profiles from RNASeq data. However, Lee and co-workers22 used several features such as mutation profiles and mutations rates. They evaluated different machine learning and feature selection methods using RNASeq data from 31 cancer types. The highest accuracy they obtained was 84%. Thereafter, they reduced the number of cancers to the six most common types and obtained an accuracy of 94%, which is low compared to our proposed deep learning model.There were statistically significant differences (19. In addition, this study is first of its kind to classify cancer tumors using RNAseq data. However, multi-class cancer classification using gene expression is not a substitute to the traditional diagnosis19, but advances in classification algorithms or methods may provide a more accurate and biologically meaningful classifications and inform future studies. Moreover, a more pressing classification problem may be that of discriminating between cancer sub-types within the same type than between cancer types. However, we postulate that the methods covered in this paper are directly applicable to this problem.Our proposed model has a very high achievement in classifying the five common cancers among women and that may potentially improve the multi-class identificationIn this work, we proposed a stacking ensemble deep learning model as a multi-class classifier to classify five most common cancers among women, that is, breast, colon adenocarcinoma, lung adenocarcinoma, ovarian, and thyroid cancer, using RNASeq gene expression datasets for each cancer tumor. Tumor classification using RNASeq data is more accurate and available compared to microarray data. We used LASSO as a feature selection method and compared the performance of our proposed method with a stand alone deep learning and machine learning methods. We conclude that our proposed model achieved the highest performance compared to the single 1D-CNN and the machine learning methods. Our proposed model is, therefore, capable of correctly classifying all the observed positive cancer cases. The proposed model can help improve the detection and diagnosis of cancer susceptibility among women in the early stages, inform decision on early intervention, and hence improve survival. Future research should consider the potential effects of using many feature types such as methylations and mutations, to be integrated with RNASeq data. Future work will also consider improvements on the stacking ensemble problem including statistical properties to improve inference.Supplementary Information 1.Supplementary Information 2.


Deconvolution analyses have been widely used to track compositional alterations of cell types in gene expression data. Although a large number of novel methods have been developed, due to a lack of understanding of the effects of modeling assumptions and tuning parameters, it is challenging for researchers to select an optimal deconvolution method suitable for the targeted biological conditions.To systematically reveal the pitfalls and challenges of deconvolution analyses, we investigate the impact of several technical and biological factors including simulation model, quantification unit, component number, weight matrix, and unknown content by constructing three benchmarking frameworks. These frameworks cover comparative analysis of 11 popular deconvolution methods under 1766 conditions.We provide new insights to researchers for future application, standardization, and development of deconvolution tools on RNA-seq data.The online version contains supplementary material available at 10.1186/s13059-021-02290-6. Deconvolution refers to a process that separates a heterogeneous mixture signal into its constituent components. In the biomedical field, researchers have used deconvolution methods to derive cell type-specific signals –3 from hBy January 2018, approximately 50 deconvolution methods had been developed  and CoboThere are three types of benchmarking frameworks for the evaluation of deconvolution methods: in vivo framework , in vitrTo provide a reliable reference for the application and development of deconvolution methods, we compared 11 deconvolution methods Fig. ” sectionWe designed three benchmarking frameworks to test the performance of deconvolution methods under multiple application scenarios. Each framework was designed to study the impact of specific technical and biological factors on deconvolution analysis Fig. . The firIn an in silico benchmarking framework, a deconvolution testing environment consists of mixture data, reference data, ground truths, and deconvolution methods for testing. Mixture data refers to heterogeneous gene expression profiles for deconvolution. Reference data refers to homogeneous cell type-specific data that used to guide the deconvolution process. Ground truth refers to the real mixing proportions of constituent cell types in the mixture data. The accuracy of deconvolution methods can be assessed by comparing estimated proportions to the ground truths. Since reference data can vary based on the required input of the tested methods, we classified 11 deconvolution methods into the following categories: marker-based, reference-based, and reference-free  where 11 indicated the number of methods and 3 indicated the number of simulation models. Based on the summarized evaluation heatmaps of correlation Fig. a and mAD1 × 3 wheIn the Sim1_simModel, we found that the noise structure is the main factor obscuring deconvolution performance assessment Fig. c–h. We sWe used the mean-variance plots to study the overall trend between variance and mean in both real and simulated data (P6 noise level) Fig. c, d. As y = x) than highly expressed genes  to the real datasets  values in the real and simulated data Fig. g, h. TheIn conclusion, the negative binomial simulation model, which successfully recapitulated the mean-variance trend, sample-sample concordance, and the density of CV, presented the noise structure that was most similar to the noise structure in the real data. The negative binomial model also kept the magnitude of noise at comparable levels to the real data and thus should be considered as the most appropriate simulation model for generating in silico mixtures.In Sim1_simModel, we observed the trivial impact of the quantification unit, but this might be due to the fact that all negative binomial simulated mixtures have the same library size the “” sectionr ≥ 0.9, mAD ≤ 0.1) regardless of the selected quantification unit  of mixture datasets. While the weight matrix is not controllable in the analysis, researchers can roughly speculate weight matrix property based on previous cellular composition-relevant studies. Therefore, understanding the impact of weight matrix property will provide researchers new insights about method selection based on the mixture conditions.To investigate the joint impact of the cellular component number and weight matrix property, we designed benchmarking framework Sim2 with six gradients of component number ranging from 5 to 10 and two types of weight matrices: “orthog” and “real” Fig. ” sectionr ≥ 0.95, mAD ≤ 0.05) in the “orthog” group   Fig. b and mADr ≥ 0.85, mAD ≤ 0.05), such as neutrophils, CD4 T, and CD8 T in the mixtures  that occupied less than 1% in the mixture, such as myeloid DC, and hematopoietic stem cells (HSC), the three methods correctly identified them as minor components and did not attribute the percentages of other cell types to these rare cell populations (mAD 0~0.01) , the “large” group refers to mixtures with high levels of tumor spike-ins 70–80%), and the “mosaic” group refers to mixtures with dynamic levels of tumor spike-ins (5–95%). Tumor spike-ins were introduced to the 12 mixture sets generated in the Sim2 framework to analyze the joint impact of the component numbers, weight matrix properties, and unknown biological contents  on the relative measurement scale       on the relative measurement scale  on the absolute measurement scale  than other tumor spike-in groups   and variance (large vs. mosaic spike-ins) of unknown content affected the deconvolution analysis. We also observed a discrepancy in performance evaluation when using different measurement scales. When the relative scale was used, CIBERSORT  and CIBEIn this study, we designed three in silico benchmarking frameworks to systematically investigate the impact of several biological and technical factors. We identified top-performing deconvolution methods for each framework and illustrated the strengths and weaknesses of the tested methods under different application scenarios. Additionally, we provided strategies for mitigating systematic biases caused by different technical and biological factors such as varied library sizes, simulation models, and cellular compositions.In the first framework (Sim1), we explored the impact of noise structure at different noise levels. We identified CAMmarker , MMAD 1, DSA 1212, CIBERBased on the results obtained in the present investigation, we offer suggestions for best practices of deconvolution analysis and evaluations. For the benchmarking data generation, we recommended that researchers (1) use the negative binomial model as the primary simulation model for in silico mixture data generation; (2) reference real biological composition data when building weight matrices,; (3) consider at least two evaluation metrics: one is used for checking linear concordance between estimation and ground truth, and the other one is used for checking estimation deviances; (4) when unknown biological content is involved, beware of the influence caused by different measurement scales (absolute vs. relative); and (5) construct multi-factor conditions on a large scale to ensure the robustness and comprehensiveness of the benchmark.For deconvolution analysis, we suggest that researchers (1) use the quantification unit  that is normalized by library sizes; (2) check for the compositional information from previous publications. When the targeted tissue type has a relatively stable composition over several samples, consider using deconvolution methods that are robust to non-orthogonal weight matrices such as CIBERSORT , CIBERSOIn addition to the suggestions listed above, previous benchmark publications also clarified the impact of signature matrices , multicoFor the future advancement of deconvolution analysis on RNA-seq data, we suggest that more efforts be put into the refinement of simulation models to generate more authentic in silico testing environments that mimic diverse application scenarios. In this study, the weight matrix property was revealed as the most important factor affecting deconvolution analysis, which was overlooked in prior research. Therefore, more studies on the cellular compositional information and its corresponding effects on deconvolution analysis are needed. All these improvements in simulation and benchmarking strategies will further enhance the efficiency of deconvolution method development.Raw SRA files were downloaded from the GEO repository, processed by SRA Toolkit 2.10.0) ; reads w0.0 ; reap, which is the percentage of samples that pass the criteria  by a step parameter  until there are at least two marker genes determined.For the marker gene selection, we selected genes that are highly expressed in the targeted cell type and lowly expressed in other cell types. The expression threshold is set at the 80th percentile for high expression (the targeted group) and 50th percentile for low expression (other groups). To successfully derive marker genes with a larger number of cellular components, we gradually relaxed the criteria by decreasing parameter padj ≤ 0.01 and ∣ log 2FoldChange ∣  ≥ 10.We performed differential expression testing on all cell type pairs using DESeq2 . We seleThe reference profiles for deconvolution analysis were constructed from purified homogeneous samples. For Sim1 and Sim2, we used a total of ten cell types listed in Table M (N by J) by multiplying component source profiles S (N by K) to the predefined weight matrix W (K by J). Here, N is the number of genes, J is the number of samples, and K is the number of cellular components. The noise term ε is used to model sample-to-sample variability where the value of ε determines the noise level. All simulated mixtures were based on purified homogeneous profiles, and only samples derived from homogeneous sources were picked into the component source profiles. We avoided using heterogeneous samples identified as “whole blood,” “whole white blood,” or “pbmc,” for the source profile generation. Those heterogeneous samples are only used in the variance analysis  to generate four types of weight matrices: orthog, real, dominant, and uniform. After the initial sampling, all weights will be rescaled so that the weights of all components sum to 1.K by J) by randomly sampling weights from a uniform distribution and then rescaled sampled weights so that the weights from all cellular components sum to 1. Among the 1000 proportion matrices , we picked the one weight matrix that has the smallest condition number.“Orthog” refers to the idealized weight matrix with a small condition number, which provides a relatively optimal mathematical condition for deconvolution analysis. We first simulated 1000 matrices (https://github.com/LiuzLab/paper_deconvBenchmark (DOI:10.5281/zenodo.4521514). We simulated 1000 weights  where the weights of each cellular component are sampled from the predefined range. In the initial weight matrix, we picked J  number of columns whose sum is closest to 1 and retrieved a K by J weight matrix. The above filtering process is added to avoid violation of the predefined range after rescaling. After weight simulation, the range of each component is checked and only weights that satisfied both sum-to-1 and predefined range is selected.“Real” refers to the weight matrix that mimics immune cell compositions in the real whole blood sample. We sampled weights from the uniform distribution with min and max values defined by previous observations of blood samples . A smallK-1 minor components. The major component was sampled from the uniform distribution within the range of 0.9–0.99. Minor components were sampled within the range “Dominant” refers to a weight matrix with one major component and “Uniform” refers to a weight matrix with all components have weights at a similar level. All cellular components are sampled within the range In this benchmarking framework, we studied the impact of unknown biological content and measurement scales :Pearson correlation coefficient (Mean absolute deviance (mAD):j is the sample index, and J is the total number of mixture samples in a dataset. xj is the estimated cellular proportion of sample j, and yj is the ground truth of sample j. When a deconvolution returns NA values, we directly assign the highest penalty for the evaluation metrics: r = −1, and mAD = 1, which is the worst possible value. NA values can arise from two sources: deconvolution algorithm and correlation coefficient calculation. For instance, we found that in the Sim1 analysis, CAMfree (reference-free deconvolution method) [where  method)  will ret method)  and Deco method)  returnedAdditional file 1: Figures S1–S18, the outline of simulation frameworks, evaluation results based on three benchmarking frameworks, variance analysis based on real and simulated data.Additional file 2: Tables S1–S4, organized description of the benchmarking framework, tested factors, tested methods, and measurement scales.Additional file 3: Data description with the GEO accession number.Additional file 4: Review history.


Epithelial-mesenchymal plasticity (EMP) underlies embryonic development, wound healing, and cancer metastasis and fibrosis. Cancer cells exhibiting EMP often have more aggressive behavior, characterized by drug resistance, and tumor-initiating and immuno-evasive traits. Thus, the EMP status of cancer cells can be a critical indicator of patient prognosis. Here, we compare three distinct transcriptomic-based metrics—each derived using a different gene list and algorithm—that quantify the EMP spectrum. Our results for over 80 cancer-related RNA-seq datasets reveal a high degree of concordance among these metrics in quantifying the extent of EMP. Moreover, each metric, despite being trained on cancer expression profiles, recapitulates the expected changes in EMP scores for non-cancer contexts such as lung fibrosis and cellular reprogramming into induced pluripotent stem cells. Thus, we offer a scoring platform to quantify the extent of EMP in vitro and in vivo for diverse biological applications including cancer. Epithelial-Mesenchymal Plasticity (EMP) is an important feature of cancer metastasis and therapy resistance, the two major clinical challenges that claim the majority of cancer-related deaths . EMP invRecent preclinical and clinical observations have suggested the high metastatic potential of hybrid E/M phenotypes, and their association with worse patient survival across cancer types ,19,20,21At the transcriptomic level, various computational methods have been proposed to quantify the EMP spectrum by calculating the extent to which a given sample has undergone EMT/MET . First, using gene expression from non-small cell lung cancer (NSCLC) cell lines and patients, a 76-gene EMT signature was identified and then used to derive a score  based on the relative enrichment of expression levels of epithelial-associated genes. The 76GS score is a weighted sum of the expression levels of these 76 genes, with the weight being the correlation coefficient of the corresponding gene with that of E-cadherin (CDH1) levels in the given dataset. Thus, the higher the 76GS score is, the more epithelial a given sample is ,31. SecoA previous study compared these three methods—each of which utilizes a distinct gene list and algorithm—and observed that these methods were largely well-correlated with one another in terms of quantifying EMP across multiple microarray datasets . This anHere, we have addressed these limitations by analyzing multiple bulk and single-cell RNA-seq datasets, as well as investigating both microarray and RNA-seq datasets for the following two non-cancer cases where EMP has been reported: (a) lung diseases—chronic obstructive pulmonary disease (COPD) and fibrosis ,36 and (https://xena.ucsc.edu/) (accessed on 18 November 2020). Statistical analysis, survival analysis and plots were all conducted in R version 4.0.3. Function ggplot was used for plots.A list of all softwares used and their URL is given in We downloaded high-throughput transcriptomics data (bulk and single cell) from GEO and EMBL-EBI databases. Microarray datasets were downloaded using GEOquery R Bioconductor package . TCGA exhttps://www.bioinformatics.babraham.ac.uk/projects/fastqc/, accessed on 18 November 2020). Bulk and single-cell RNA seq data were aligned to reference genome  using STAR- aligner  with 0 designating a fully epithelial signature, 1 maximally hybrid signature, and 2 fully mesenchymal signature.We adapted a previously developed method of quantifying EMT spectrum trained on and designed to predict microarray samples . BrieflyTo apply the microarray based MLR model to RNA-seq data, we utilized transcriptomic data available in both formats on biological replicates . Using tFrom these, unique microarray values, high and 76GSlow, MLRhigh and MLRlow, KShigh and KSlow groups based on the mean (or median) of the respective scores of the samples. Kaplan–Meier analysis was performed using R package “survival” and plotted using R package “ggfortify”. A log rank test was used to calculate the p-values. The reported hazard ratio (HR) and confidence interval (95% CI) were estimated using cox regression.Different metrics of survival data were obtained from TCGA cohort. All samples were divided into 76GSSingle-sample GSEA (ssGSEA), an extension of Gene Set Enrichment Analysis (GSEA), calculates separate enrichment scores for each pairing of a sample and gene set. Each ssGSEA enrichment score represents the degree to which the genes in a particular gene set are coordinately up- or down-regulated within a sample. We used “HALLMARK_EMT” gene set from The Molecular Signatures Database (MSigDB) database and the scores were calculated using R package “ssgsea”.We used the three different EMT scoring methods—76GS, KS, and MLR—to quantify the extent of EMP in multiple RNA-seq datasets, as was previously conducted for microarray datasets . Each me2-normalized FPKM (Fragments Per Kilobase of transcript per million Mapped reads) or TPM (Transcripts Per Million) RNA-seq data. To achieve this, the log2 RNA-seq values for three predictors (CLDN7 (claudin 7), VIM (Vimentin), CDH1 (E-cadherin)) and 20 normalizers were linearly mapped to their corresponding microarray values  with the MLR and KS scores and found a positive correlation  between the MLR and KS scores across most datasets that contain cell lines and primary tumors across cancer types  aSimilarly, these scoring metrics could recapitulate the activation of EMT in pre-malignant immortalized and Ras-transformed HMECs  as compared to primary HMECs , we observed a negative correlation between 76GS and KS scores, and between 76GS and MLR ones, with, however, a positive correlation between the KS and MLR scores . Thus, tNext, we plotted the histograms for the EMT scores of various single-cell RNA-seq datasets to decipher the heterogeneity seen along the EMP spectrum across a variety of biological contexts: 1. human embryonic stem-cell-derived progenitor cells differentiating to endoderm (GSE75748) ; 2. humaOf note, several plots for the 76GS and MLR metrics appeared saturated, which we hypothesized related to the relative sparsity of the predictor signal in the single cell datasets. For the MLR approach, we then restricted our analysis to datasets with at least 90% of all single-cell samples containing nonzero entries for each predictor, indicating the presence of measurable signal. In these cases, MLR and 76GS metrics were able to recapitulate the trends observed in KS for many such datasets .All of the three EMT metrics  have been designed and/or trained for quantifying the EMT status in cancer samples ,32,33, bAs compared to normal lung tissues, the fibrotic lung tissues from IPF patients had higher KS scores but lower 76GS scores, indicating their enhanced mesenchymal status  of datasets. Similarly, 76GS and KS scores correlated negatively in ~41% (19/46) of datasets. Further, KS and MLR scores correlated positively in ~54% (25/46) of datasets . OverallFurther, we investigated a set of datasets related to the cellular reprogramming of differentiated cell types to induced pluripotent stem cells (iPSCs), where EMT/MET are reportedly involved  Table S. Across Next, we quantified the EMT scores in patient samples using TCGA datasets of various cancer types. Here, also we found the expected trends that the 76GS scores show negative correlation with the MLR and KS scores and KS and MLR scores are positively correlated to each other A, reinfolow subgroup can be thought of as comparable to the MLRhigh and KShigh groups, given their relatively strong M signature. In bladder cancer  corresponded to worse overall survival. However, similar to PAAD, for the cases of kidney renal papillary cell carcinoma (KIRP), thymoma (THYM), and low-grade glioma (LGG), the stronger the M phenotype is (as identified by 76GSlow), the better the survival is. We also assessed the association between the EMT scores and patient survival using different survival data types , Disease-Specific Survival (DSS), Progression free interval (PFI), and Disease-free interval (DFI)) in various TCGA cancer cohorts. The samples were scored using all three methods and segregated into high and low groups based on the mean value of each EMT score. The 76GSer BLCA; B, top, wer BLCA; B, middleer BLCA; B, bottomer BLCA; . For insWhile the concept of EMP may not as be as stringently applicable to non-epithelial cancers such as LGG, these pan-cancer observations highlight the following two key aspects: (a) the categorization of patient samples with more vs. less M phenotypes is largely consistent across the three scoring metrics used, and (b) the often-assumed association of EMT with worse survival is not a universal feature; instead, it depends on the specific cancer subtype.high or 76GSlow) reflect poorer survival. This trend held for PFI as well (high or 76GSlow) (high or 76GSlow)  .Quantifying the spectrum of epithelial-hybrid-mesenchymal cell states in cancer has garnered recent interest due to a surge in the availability of in vitro and in vivo spatial and/or temporal dynamic and high-throughput data at multiple levels—transcriptomic, proteomic, epigenetic, metabolic, and morphological ,86,87,88Here, we used three different EMT transcriptomic-based scoring metrics, each of which was developed using cancer cell lines and/or tumor samples and tested on microarray data previously to quantify the extent of EMT in a continuum—76GS, KS, MLR. Using these metrics, we calculated EMT scores for over 100 RNA-seq datasets—including at bulk and single-cell levels—across multiple biological contexts  of the single-cell samples. These scores, while concentrated in the middle of the EMT interval, were able to recover features of the distributions observed via the KS method. Together, this suggests that the development of optimized signal-to-noise criteria, may improve the absolute placement of samples on the EMT MLR spectrum and is the focus of our ongoing research effort, building upon various tools used in the analysis presented here . Future Intriguingly, the EMT status of primary tumors was not found to be universally correlated with worse patient survival, but instead showed a context-dependent trend , consist