Molecular subtyping of cancer is a critical step towards more individualized therapy and provides important biological insights into cancer heterogeneity. Although gene expression signature-based classification has been widely demonstrated to be an effective approach in the last decade, the widespread implementation has long been limited by platform differences, batch effects, and the difficulty to classify individual patient samples. Here, we describe a novel supervised cancer classification framework, deep cancer subtype classification (DeepCC), based on deep learning of functional spectra quantifying activities of biological pathways. In two case studies about colorectal and breast cancer classification, DeepCC classifiers and DeepCC single sample predictors both achieved overall higher sensitivity, specificity, and accuracy compared with other widely used classification methods such as random forests (RF), support vector machine (SVM), gradient boosting machine (GBM), and multinomial logistic regression algorithms. Simulation analysis based on random subsampling of genes demonstrated the robustness of DeepCC to missing data. Moreover, deep features learned by DeepCC captured biological characteristics associated with distinct molecular subtypes, enabling more compact within-subtype distribution and between-subtype separation of patient samples, and therefore greatly reduce the number of unclassifiable samples previously. In summary, DeepCC provides a novel cancer classification framework that is platform independent, robust to missing data, and can be used for single sample prediction facilitating clinical implementation of cancer molecular subtyping. Although the prognostic value of these classifications is undisputed, they fall behind in predicting drug efficacy due to the lack of a clear molecular basis. Instead, as an example in colorectal cancer (CRC), genetic features, such as KRAS mutation and microsatellite instability (MSI) status3, have proven predictive power regarding anti-EGFR and 5-FU efficacy, respectively. However, classifications based on these molecular characteristics still leave much of additional cancer heterogeneity unaccounted for4. Therefore, in recent years, whole transcriptome-based cancer subtyping has been widely demonstrated as an efficient approach for dissecting cancer heterogeneity5. The evident benefit of this approach is the integration of genetic, epigenetic, and microenvironmental features that impact on cancer biology and clinical presentation. A widely-implemented strategy involves consensus clustering for determination of an optimal number of cancer subgroups, and classification with feature selection, i.e., selection of a list of signature genes6.Cancer subtyping is important for selection of patients that benefit most from specified therapies and design of novel targeted agents. Traditionally, cancer classification is largely based on histopathological and clinical characteristics, which makes it difficult to implement uniformly, as individual expertize of the clinicians is often a major determinant8 have been well established and widely adopted, the classification step suffers several critical limitations. First, a signature gene-based approach places sole emphasis on the role of individual genes, without effective incorporation of biological knowledge such as pathway activity, which often leads to poor interpretability11. Second, signature genes for classification are not always available due to unpaired gene annotation caused by platform differences, which hampers its portability and translational potential12. Last but not least, gene expression profiling is easily affected by factors such as technical platform variation and experimental protocols, leading to nonbiological batch effects13. Mathematical and statistical methods might be able to correct for such bias so that data from various sources are comparable. However, such correction methods are not always suitable, especially in situations when the sources of bias are unclear. For instance, the correction power of existing methods such as ComBat14 have been demonstrated to be limited to a balanced experimental design15. Most critically for clinical implementation, the batch effects also prevent the development of gene expression signature-based classifiers for single sample prediction.While several consensus clustering methods19. Deep learning-based frameworks, such as D-GEX20, DeepChrome21, and DeepSEA22 have also been developed for predicting gene expression or effects of noncoding variants based on high-dimensional genomic or epigenomic profiles. Furthermore, several supervised and unsupervised deep learning-based classification methods have been proposed for cancer detection and diagnosis, and they have been demonstrated superior performance over classical methods such as support vector machine (SVM) and random forests (RF)25. Meanwhile, pathway activities transformed from gene expression profiles have been shown to be more informative and robust for disease classification27. Motivated by these pioneering works in machine learning, we developed a novel framework Deep Cancer subtype Classification (DeepCC), which leverages both pathway activity transformation and deep learning to address the abovementioned critical limitations in cancer subtype classification.Recent advances in the machine learning community have shown a great promise to apply deep learning for cancer classification. For instance, deep convolutional neural networks have been demonstrated to improve accuracy and reproducibility of tumor classification based on histopathological or radiographic images28 for each tumor sample’s gene expression profile on thousands of gene sets obtained from public databases such as MSigDB28. For each patient sample, the vector of enrichment scores of all gene sets represents a landscape of molecular patterns associating with biological functions, and therefore it is referred to as a functional spectrum.Transformation of high-throughput gene expression data to functional spectra. We first perform gene set enrichment analysis (GSEA)29. To train a DeepCC classifier, we highly recommend employing a widely adopted molecular subtyping system, so that the deep features trained by the ANN can capture most relevant biological properties associated with each molecular subtype. In our case studies, we used the consensus molecular subtyping (CMS) system30 for CRC and intrinsic subtyping system for breast cancer (implemented by PAM5031), which are both widely adopted in respective fields. The trained DeepCC model can be subsequently used for classification of new samples.Classification based on deep learning. Taking the obtained functional spectra as input, we next train a classifier using deep learning. By using a trainable multilayer artificial neural network (ANN) to replace hand-engineered features, deep learning takes the advantage of functional spectra, which are more robust and informative. In contrast, feature selection for high-dimensional data is a challenging task for conventional machine learning algorithms, which could lead to bias especially for high-throughput gene expression profilesDeepCC is a supervised, biological knowledge-based framework for cancer classification, consisting of two major steps Fig. :Transfor32. In both case studies, we demonstrated that deep features learned by DeepCC represent characteristic biological processes of different cancer subtypes, which better dissects molecular heterogeneity than gene expression signature-based methods.Better interpretability. DeepCC transforms gene expression profiles to functional spectra, which are transcriptomic patterns that have been previously demonstrated to directly associate with biological function. Deep learning is also well known for its capability to learn high-level representations of data through multiple hidden layersPlatform independency. Different types of high-throughput gene expression data are transformed to the same form of functional spectra, and therefore are directly comparable.More robustness. Since GSEA is performed on the ranks of input data and corrected by its raw value, DeepCC is much less influenced by nonbiological factors such as batch effect and normalization methods.Single sample prediction. DeepCC can deal with single sample gene expression data regardless of the platform by adaptive rescaling to a predefined or user-defined reference (details in “Methods” section). DeepCC single sample predictor (SSP) addresses one critical limitation of previous cancer classification methods, which hampers the translation of cancer molecular subtyping into the clinic.In short, our DeepCC framework has four major advantages over previous cancer subtyping methods:30 based on gene expression data of 4151 primary tumor samples from a total of 18 data sets. The four CMSs are characterized by distinct molecular features: CMS1 (MSI immune), CMS2 , CMS3 (metabolic), and CMS4 . Also, the CMSs display vastly different clinical features including prognosis and response to therapies. Moreover, together with the CMS taxonomy we reported a 273-gene classifier based on the RF algorithm to facilitate classification of additional data sets30.To demonstrate the performance of DeepCC, we initially applied it to classify CRC. Recently, we participated in an international CRC subtyping consortium (CRCSC) and identified four consensus molecular subtypes (CMSs)33. In this case study, we collected 14 publicly available CRC data sets (n = 3578) generated from six different microarray/RNA-Seq platforms  with CMS subtype information provided by CRCSC8, and subsequently studied its performance on expression data of only a subset of genes. More specifically, we iteratively selected a random subset of genes that are most variable across TCGA samples (measured by median absolute deviations), and calculated functional spectra based on the subset of genes selected for classification. Accuracy, balanced accuracy, sensitivity, and specificity were calculated for evaluation of DeepCC’s performance. As expected, we found that DeepCC accomplished a very high accuracy  even when only 2000 genes were used for training the classifier . Notably, DeepCC SSP also showed promising performance as DeepCC .To comprehensively benchmark the classification performance, we trained a DeepCC classifier using the TCGA RNA-Seq data set with CMS subtype labels, and then applied it to classify 13 other independent data sets based on Affymetrix or Agilent microarray platforms Table , followepCC Fig. , with re30. The question remained if this represented biological and clinical reality, or was a limitation of the classification strategy. Because we detected large variation in the number of unclassifiable samples in the various data sets (range 11–48%)30, we suspected that it is predominantly the latter reason paired with data set dependent differences in data quality. Therefore, we hypothesized that DeepCC would be able to reduce the number of unclassifiable samples. To assess classification performance, we used the same criterion that was previously employed for the CMS classifier (posterior probability > 0.5). Across all data sets previously analyzed by CRCSC, DeepCC only failed to classify ~5% tumor samples, much lower than previously reported methods using the same data series  on a public data set . In the CIT data set, DeepCC can classify 531 out of 557 samples, whereas the CMS classifier based on RF failed to classify 117 out of them. Interestingly, these unclassifiable samples were mainly in the boundary regions of the CMS signature gene space  measured in the TCGA data set . This implicates that DeepCC can find more fundamental functional distinctions between cancer subtypes, which also explains its superior classification performance.An important advantage of deep learning is feature learningset Fig. . Deep feset Fig. . Especiaset Fig. . Interesset Fig. , which sset Fig. . To quan35 for intrinsic subtype classification, which is a widely used transcriptome-based classification system. In PAM50, five distinct (intrinsic) molecular subtypes are defined: Basal-like, Her2, Luminal A, Luminal B, and Normal-like. Using TCGA RNA-Seq data set (n = 517) with subtyping result predicted by PAM50, we trained a supervised DeepCC classifier to evaluate the classification performance. Similar to the case study in CRC, we first evaluated the robustness of DeepCC classifier to cross-platform missing genes in breast cancer. As expected, DeepCC accomplished a high accuracy  even when only 1000 genes were used for training . Furthermore, survival analyses on the four validation sets  separately and jointly demonstrated that DeepCC classification has higher associations with disease-free survival than PAM50  between the Luminal and HER2 subtypes predicted by DeepCC with ER/PR and HER2 receptor status, which are their corresponding characteristic markers, respectively , to train a DeepCC classifier for a particular cancer type. The trained DeepCC model can be subsequently used for classification of new samples, facilitating real clinical implementation of cancer subtyping considering its superior performance, cross-platform robustness and capability for single sample prediction, as demonstrated in our case studies.It should be noted that DeepCC is a supervised, biological knowledge-based framework specifically developed for addressing current challenges in classification but not clustering of cancer patients. In the last decades, molecular subtyping for major malignancies has been extensively studied, and many subtyping systems have been well established. However, implementing these subtyping systems is challenged by the abovementioned limitations of classical classification methods, which motivated us to develop DeepCC. In practice, it is recommended to use a widely adopted molecular subtyping systems, such as the CMS31, BluePrint38, and Oncotype DX39, which have been well established and already applied in clinical practice. However, MammaPrint, and Oncotype DX are prognostic tools for prediction of high-, (intermediate-), and low-risk of recurrence and/or metastasis, which are not strictly ‘biological’ subtypes. BluePrint can only predict Luminal, HER2, and Basal subtypes, but cannot distinguish between Luminal A and Luminal B subtypes. Instead, PAM50 classification system encompasses all the five intrinsic molecular subtypes of breast cancer, and therefore is an ideal subtyping system for training DeepCC. Similarly, a number of risk scoring assays for CRC recurrence prediction have been developed, including Oncotype DX for Colon Cancer39, ColonPRS40, ColoPrint41, GeneFx Colon42, OncoDefender-CRC43, ColoGuideEx44, ColoGuidePro45, and miRNA recurrence classifier46. All these assays have demonstrated prognostic values in independent patient series, but are not defining or predicting molecular subtypes of CRC. Therefore, in our CRC case study we selected the widely adopted CMS system developed by CRCSC to train DeepCC.There are various molecular testing assays such as MammaprintSince DeepCC employs GSEA to transform gene expression data to functional spectra, the prior knowledge of gene sets needs to be comprehensive to avoid potential bias. Of special importance are the major characteristic signaling pathways or biological processes for each cancer subtype, which should be included in the gene set database. In practice, it is highly recommended to use a database with a large scale of curated gene sets that are of high quality such as MSigDB v6.0 database employed in our case studies, which includes 17,779 gene sets encompassing all cancer hallmark signaling pathways.In conclusion, our deep learning-based framework DeepCC integrates biological knowledge, overcomes limitations of signature gene-based approach and leads to more robust performance. Through case studies on CRC and breast cancer, we also demonstrated its superior classification performance and clinical relevance. The capability of DeepCC to reliably classify single samples, using transcriptome data obtained by any platform will greatly facilitate the translation of molecular subtyping into clinical practice.http://gdac.broadinstitute.org/). For patient samples with both gene expression data based on Illumina GA and Hi-Seq platforms, we only kept the Hi-Seq version. Scaled estimates in the gene-level RSEM files were first converted to TPM (transcripts per million) by multiplying with 106 and then log2-transformed.In this study, we analyzed 14 independent CRC data sets, involving in total 3578 primary tumor samples Table . TCGA CRCRCSC (n = 2 674)8, including GSE13067 (n = 73), GSE13294 (n = 155), GSE14333 (n = 157), GSE17536 (n = 174), GSE20916 (n = 90), GSE2109 (n = 287), GSE35896 (n = 62), GSE37892 (n = 127), GSE39582 (n = 557), KFSYSCC (n = 305), and PETACC3 (n = 687). These curated gene expression data were downloaded from the official repository of CRCSC on Synapse (https://www.synapse.org/#!Synapse:syn2623706/wiki/). More details about the curation procedures can be found in Guinney et al.30, which resulted in expression levels of 5973 genes for each data set. The other two validation data sets, GSE42284 (n = 188), GSE33113 (n = 90), were downloaded from GEO directly in its processed form, using Bioconductor package ‘GEOquery’. The GSE42284 data set is based on Agilent Homo sapiens 37 K DiscoverPrint_19742 microarray platform, including 188 CRC patient samples, processed by Agilent Feature Extraction software based on MedianSignal output and normalized by Lowess normalization47. Probeset IDs were converted to gene symbols based on the corresponding gene annotations (GPL16280), and then further converted to Entrez IDs using Bioconductor package ‘org.Hs.eg.db’. The GES33113 data set is based on Affymetrix Human Genome U133 Plus 2.0 array. The GSE33113 data set includes 90 CRC patient samples and 6 normal samples, and was processed by MAS5.0 normalization and GCOS software36. We only kept colorectal samples, and then converted all probeset IDs to Entrez IDs based on the corresponding gene annotations (GPL570).For validation, 11 out of 13 data sets were curated by https://www.synapse.org/#!Synapse:syn2623706/wiki/).In addition, CMS classification labels associated with all samples were also obtained from CRCSC Synapse repository  were downloaded from Bioconductor (http://www.bioconductor.org/) in their processed form.Four breast cancer data sets , and then converted to Entrez IDs using Bioconductor package ‘org.Hs.eg.db’.For MACQ II BR cohort, the raw data were downloaded from GEO database with the accession number GSE20194 in R using ‘GEOquery’ package. The 48 were used for training and validation.In the breast cancer case study, PAM50 labels for the TCGA BRCA cohort obtained from TCGAMore details about the breast cancer data sets can be found in Table Enrichment Scores calculated by GSEA28. The following steps are used to calculate the enrichment score ES:Filter duplicate probes of gene expression to prevent overestimation.Calculate log2 fold changes D of N genes by subtracting the background signal.1,…,gN}, which is in a descending order.Rank order D to form the gene list L = {gForm a list S = {s1,…,sN} containing the contribution of each gene for the enrichment score. For a gene ghit in the gene set C, we score it by Calculate the accumulation sum from s1 in S to obtain the deviation, and the value deviated the most from zero is the enrichment score ES.A functional spectrum is a list of 28 v6.0 (downloaded on 1 Jun 2017).In this study, we used all 17,779 gene sets in MSigDBhttps://arxiv.org/abs/1512.01274) incorporating the latest optimization methods developed by deep learning community, which can run on both CPU and GPU. By default, DeepCC builds a fully connected multilayer perceptron 49 using the architecture of hidden layers: 2000, 500, 120, 30, 10 with Tanh activation function. The last layer for output is SoftMax. The whole network is initialized using Xavier50. The optimizer can be chosen from SDG51  or AdaDelta (https://arxiv.org/abs/1212.5701).The deep learning framework in DeepCC was implemented based on MXNet . For each class, the sensitivity was calculated using TP/(TP + FN).Mean of specificity (per class). For each class, the specificity was calculated using TN/(TN + FP).Mean of balanced accuracy (per class). For each class, the balanced accuracy was calculated using (sensitivity + specificity)/2.Accuracy. The overall accuracy is the total number of correct predictions divided by the total number of patient samples.Four statistical measures were used to evaluate the classification performance. For each CMS subtype, we first calculate true positive (TP), false negative (FN), false positive (FP), and true negative (TN), and then calculate the following measures:To identify highly correlated biological functions of each subtype, we first calculated Pearson correlation coefficients between deep features extracted by DeepCC and enrichment scores of gene sets in the training data set. We obtained a correlation matrix indicating the relationships between deep features and functional gene sets. Highly correlated gene sets, either positively or negatively, indicate biological functions highly associated with deep features Tables  and S7.To visualize classification results by DeepCC and signature gene-based approaches, we projected all patients into the deep feature space and the signature gene space, respectively. For DeepCC, the first two principal components of the ten deep features were visualize in a two-dimensional space, For the other classification approaches, the first two principal components of the expression levels of signature genes were visualized in a two-dimensional space.DeepCC was implemented as an R package, and the source code and instructions for running DeepCC locally are available at GitHub.Project name: DeepCChttps://cityuhk-compbio.github.io/DeepCC/Project home page: Operating system(s): Platform independentProgramming language: R and COther requirements: R version 3.3 or higher; MXNet version 0.10, or higherLicense: MIThttps://cityuhk-compbio.github.io/deepcc_online/) is also provided for transformation of gene expression profiles of tumor samples to functional spectra and prediction of cancer subtypes based on pretrained models.An online platform of DeepCC (http://www.r-project.org) and P < 0.05 was considered as significant in all cases. RF, SVM, GBM, and the multinomial logistic regression model were implemented by ‘randomForest’, ‘kernlab’, ‘gbm’, and ‘glmnet’ package respectively in R. Radial kernel and probability model was used for SVM. All other parameters were kept as default in each respective package. The linear model used in DeepCC SSP was implemented by the ‘lm’ function of R package ‘stats’. The comparison of classification results was based on McNemar’s chi-squared test in R, implemented by the ‘mcnemar.test’ function in ‘stats’ package. Analyses of disease-free survival data were performed using R package ‘survival’ and p-values were derived from log-rank tests. AWSs were used as the quantification of the relevance between features and CMS subtypes, calculated by the function ‘silhouette’ in R package ‘cluster’. Fisher’s exact tests, t-tests, Wilcoxon signed-rank tests, Pearson correlation coefficients were calculated using R package ‘stats’.All statistical analyses were performed using R (version 3.3.3; Supplementary figures.Supplementary tables.


To integrate molecular features from multiple high-throughput platforms in prediction, a regression model that penalizes features from all platforms equally is commonly used. However, data from different platforms are likely to differ in effect sizes, the proportion of predictive features, and correlations structures. Subtle but important features may be missed by shrinking all features equally.separate tuning parameter penalties for each platform that is fit using standard software. In a comprehensive simulation study, we evaluated the performance of EN logistic regression with multiple tuning penalties. We found that when the number of informative features differs among the platforms, and when there is no notable correlation between the features from different platforms, the multi-tuning parameter EN yields more predictive models. Moreover, the multi-tuning parameter EN is robust, in the sense that there is no loss of predictivity relative to a single tuning parameter EN when features across all platforms have similar effects. We also investigated the performance of multi-tuning parameter EN using real cancer datasets.We propose an Elastic net (EN) model with The proposed multi-tuning parameter EN model, fit using standard penalized regression software, can achieve better prediction in sample classification when integrating multiple genomic platforms, compared to the traditional method where a single penalty parameter is used for all features in different platforms.The online version of this article (10.1186/s12859-018-2401-1) contains supplementary material, which is available to authorized users. As multi-platform profiling of tissues enabled by advances in high-throughput ‘omic’ technologies becomes routine, efficient statistical methods to integrate multi-omic data is becoming increasingly important. Multi-omic profiling has been used to successfully investigate prognostic biomarkers and identify aberrant pathways in cancer , 2, enhaRegularized regression with a sparsity inducing penalty  with separate tuning parameters for each omic type. Through simulations with a range of scenarios differing in number of predictive features, effect sizes, and correlation structures between omic types, we show that MTP EN can yield models with better prediction performance. We apply the MTP EN to publically available prostate cancer and acute myeloid leukemia data sets from the Cancer Genome Atlas (TCGA) and Gene Expression Omnibus (GEO).n samples, and an outcome of interest . The goal is to use the training data to build a model that can be used to predict the outcome for new samples based on their corresponding multi-omic profiles. This is a standard supervised learning setting with the particularity that the features are from multiple omic types.We assume we have training data consisting of high-dimensional features from multiple omic types , measured on i, i = 1, …, n, we denote the binary outcome by yi ∈ {0, 1} and its corresponding omic profile as the partitioned vector n, and is in the hundreds or thousands in typical studies. For example, gene expression is measured for thousands of genes/features, DNA methylation for hundreds of thousands of CpGs, and genotypes for millions of SNPs. We let p1 and p2 be the number of features of the first and second omic type respectively, and p = p1 + p2 be the total number of features. In typical applications, there is also a low-dimensional vector of personal and/or clinical features , that may also be predictive and included in the feature set, but for simplicity we omit here any additional non-omic features.For simplicity, we focus our presentation on a binary outcome and omic features of two types, but the multi-tuning parameter regression framework can be applied to other outcomes  and more than two data types. For sample y =  the vector of outcomes and by X(1) and X(2) the design matrices whose rows are the ith sample vector X= [X(1)| X(2)] the matrix containing the full set of features obtained by row-wise concatenation of X(1) and X(2).We denote by y based on X in new subjects, a parsimonious prediction model based on a small subset of the full feature set is generally preferred. A model with fewer features is more interpretable and more easily translated into a custom assay deployable in a clinical setting.In addition to constructing a model that generalizes well, i.e. accurately predicts 1 norm, which gives rise to LASSO regression .The regularization parameters λ1 ≥ 0 and λ2 ≥ 0 are now specific to each omic type. Our hypothesis is that a ‘custom’ degree of regularization for each type can account for intrinsic differences between the data types and lead to a selected model with better prediction performance.In this paper, we propose using separate tuning parameters for each omic type by solving the penalized regression problem given by,Although the two-tuning parameter model (2) is non-standard, it can be fitted using elastic-net regression software provided it has an option to use a weighted elastic-net penalty of the formp-dimensional weight vector p1 entries (corresponding to the p1 features of the first omic type and the dimension of β(1)) and a p2 entries (corresponding to the p2 features of the second omic type and the dimension of β(2)). Thus, the two-penalty model can be alternatively specified using the tuning parameters λ = λ1, κ controls the shrinkage ratio between the two types. For κ = 1, the model reduces to the standard elastic net with a single tuning parameter.penalty.factor” argument. Additional file To investigate the performance of the multi-tuning parameter elastic net regression (MTP EN) for building predictive models based on multi-omic features we conduct a series of simulations under a range of scenarios. To fit the MTP EN we use the efficient and widely used elastic-net implementation in the glmnet R package , which aα to ½ and tune the overall shrinkage, λ, and shrinkage ratio parameter, κ,by k-fold cross-validation (CV), with the area under the ROC curve, AUC, as the performance metric. The training data is randomly split into k equally-sized folds, each with (approximately) the same proportion of positive (y = 1) and negative (y = 0) samples as the full training data. The MTP EN is trained on k − 1 folds for all values of  in a grid of possible tuning parameter values and the AUC is computed based on the validation held-off fold. This is repeated, using in turn each of the folds as validation held-off fold. The ‘optimal’ tuning parameters are the values κ = κmax and λ = λmax that maximize the average AUC across all folds. The optimal tuning parameter values are then applied to a fully independent test set to unbiasedly assess the model AUC. For model tuning we utilized both 5-fold and 10-fold CV, 5-fold for the analysis of real data because of a small sample size in one class and 10-fold for the simulation study.We set the elastic net penalty qj features among the pj in omic type j = 1, 2 (qj ≪ pj) to be predictive by arranging the vector of regression coefficients βj)| X(2)] by sampling from a multivariate normal distribution X~N, where Σ is a population covariance matrix with the following structure: i) dia(Σ) = 1, i.e. the X’s are already standardized and have marginal variances equal to one; ii) rj features among the qj informative ones in omic type j = 1, 2 have a common pairwise correlation ρj and correlation ρ12 with the counterpart set of features in the other platform iii) all remaining correlations are zero. The simulation scenarios are summarized in Table β0 = 0).We generated feature data X = [X| X(2)] bκ) were selected using the training data, and applied to the test data set for estimating prediction performance. To further reduce the dimensionality of the selected features beyond what is achieved by maximizing the training data prediction performance, we set λ to λse1, the largest value of λ such that the AUC is within one standard error of the maximum (achieved at λ = λmax). This strategy by Friedman et al. [max but with a more parsimonious model. Thus, the parameters κ = κmax and λ = λse1, are used to estimate AUC from the independent test data set.Model tuning parameters  and sensitivity and specificity of feature selection. The accuracy was calculated as the number of samples correctly classified in testing dataset divided by the total number of samples. The sensitivity (specificity) was calculated as the number of informative (uninformative) features correctly selected in the final model divided by the total number of informative (uninformative) features.The Acute myeloid leukemia data for 344 samples were obtained from NCBI Gene Expression Omnibus, with accession numbers for gene expression and methylation data as GSE14468 (HOVON-SAKK cohort) and GSE18700, respectively. The data was also used and described by Taskensen et al.  was usedThe prostate adenocarcinoma data was obtained from the Genomic Data Commons (GDC) Data Portal (Project ID TCGA-PRAD) and assembled by our collaborators at USC. In total 444 samples with both RNA-Seq gene expression and HumanMethylation450 (HM450) array data available were considered. The goal is to classify tumor aggressiveness, defined by both grade and stage of prostate cancer. We defined the tumor samples with Gleason score 7 and below, and T category of T1 or T2  as non-aggressive, and those with Gleason score 7 or higher and T category of T3  and T4 as aggressive. This resulted in 143 samples defined as non-aggressive and 265 defined as aggressive. Thirty-six samples (8%) were omitted for being high grade/low stage (25) or low grade/high stage (11) and unknown aggressiveness. After filtering gene features with zero variance across all samples 20,216 gene features remained. For the HM450 DNA methylation array, we excluded probes targeting SNPs, mapped to the X and Y chromosomes, in cross-reactive regions, and having missing values in at least one sample. This resulted in 371,513 remaining features. As customary, the gene expression data were log2 transformed to reduce the skewness of the gene expression distribution in its original scale.κ, and a large grid of values for the overall penalty parameter λ. The 5-fold CV was repeated 10 times and the optimal λ for each ratio value κ was selected as the one yielding the largest mean AUC across the 10 CV repeats.We split the data into training and test sets to build a prediction model and assess its performance. Training was performed using a random 80% of the samples, stratified by outcome to preserve the original proportion of positive and negative cases in the full data. The remaining 20% of samples were used to compute the test AUC. In order to obtain stable estimates, the data splitting, model training and testing was repeated 50 times with the average performance from the independent test sets reported. In the training step, MTP EN penalties were tuned by 5-fold cross-validation over a grid of values ranging from 0.1 to 1.5 for the penalty ratio parameter q1 < q2), differential penalization can yield a model with better prediction performance in terms of AUC. Specifically, the optimal parameter tuning favors a larger penalty on the omic type with fewer informative features (κ < 1) (Table q1 = q2), the optimal penalty ratio parameter κ is close to 1, indicating that the standard EN yields the best predictive performance. For Scenarios 1–3 we also evaluated the performance of MTP-EN in terms of accuracy and sensitivity and specificity of feature selection  when the effect sizes are smaller in the omic type with fewer informative features (β1 < β2), and it increases monotonically to 1  as the effect size, β1, in the first omic type increases  as the number of informative features in the second omic type, q2, increases . We compare the situations where the correlations are only between informative features within the individual platforms  for the classification of samples with data from multiple –omic platforms. The MTP EN yields more predictive models in several scenarios, including when the proportion of predictive features is larger in one omic type. In all other scenarios, the predictive performance of MTP EN matches that of the standard single penalty EN, so there is no performance downside for using MTP EN. Importantly, MTP EN can be fitted using standard EN software like the ‘glmnet’ R package.However, the MTP EN requires the tuning of one penalty parameter per omic type, and the computational effort for penalty-parameter-tuning using cross-validation grows exponentially in the number of tuning parameters when using a grid search. This should not prove a limitation for the majority of current studies that collect data on a few omic types only. Alternatively, much less costly parameter tuning using a random search of the tuning parameter space has been shown to be effective .As is the case with penalized regression approaches in general, the benefit of MTP EN decreases with the increase of the number of noise features, so it is important to consider feature pre-selection before applying MTP EN. Excluding noise features and appropriately reducing dimensionality can maximize the performance of MTP EN.). More importantly, however, we noticed that the mean AUC for MTP EN is 0.82 for the optimal weight parameter, which is higher than the AUC of 0.80 obtained from the best method in Taskesen et al. These results further support the view that when we consider the inter-correlations between different platforms when setting up the prediction model, we can better utilize the complementary information across different platforms and obtain a model with better prediction performance.The acute myeloid leukemia (AML) data set we used to illustrate the MTP EN has been previously analyzed by Taskesen et al.  using lo3ABCC5 has been reported to support osteoclast formation and promote breast cancer metastasis to bone [ITGA11 has been identified to regulate cancer stromal stiffness and promote metastasis in non-small cell lung cancer [ZNF706 has been associated with tumor progression in head and neck cancer [We also applied our method on prostate adenocarcinoma (PRAD) TCGA data, which utilized the HM450 DNA methylation array and contained many more features than the AML data set. The change of the mean testing AUCs with the change of weight parameter was very slight in the PRAD data. Still, we identified several genes that had a much larger chance to be selected in MTP EN (under the weight parameter of 1.1) than in standard EN (weight = 1). Specifically,  to bone , ITGA11 g cancer , and ZNFk cancer .Although we only investigated the simple convex EN penalty, using multiple penalties is likely to be beneficial for more sophisticated convex penalties such as group- and fused-LASSO penalties but it remains an open question whether this would extend to non-convex penalties such as SCAD, or the minimax concave penalty .We proposed a multi-tuning parameter elastic net (MTP EN) model for the classification of samples with data from multiple –omic platforms, with separate tuning parameters for each omic type that can be fitted using existing software. We found that MTP EN yields a more predictive model than ordinary EN where a single penalty parameter is used for all features in different platforms, particularly when the proportion of informative features differs between platforms and when there is no notable correlation between the informative features.Additional file 1:Self-contained R script for MTP-EN with full example. (R 5 kb)Additional file 2:The accuracy in testing dataset, sensitivity and specificity of feature selection from Standard EN and MTP-EN for different simulation settings. MTP-EN achieves better classification and sensitivity in Scenarios 1–3. (PNG 214 kb)Additional file 3:N = 200 simulation replicates. (PNG 100 kb)The optimal penalty ratio parameters versus the change of (A) number of correlated features in the second data type; (B) correlations among features in the second data type; and (C) correlations among features between different platforms. Dots represent the mean of optimal weights and caps represent the standard error of the mean; Additional file 4:Listing of features that have more chance to be selected by MTP EN in AML and PRAD datasets. (DOCX 29 kb)


Researchers also use this cell model to study obesity and insulin resistance. We surveyed the literature, the gene expression omnibus and the sequence read archive for RNA-Seq and ChIP-Seq datasets of MDI-induced 3T3-L1 differentiating cells sampled at one or more time points. The metadata of the relevant datasets were manually curated using unified language across the original studies. The raw reads were collected and pre-processed using a reproducible state-of-the-art pipeline. The final datasets are presented as reads count in genes for the RNA-Seq and reads count in peaks for the ChIP-Seq dataset. The curated datasets are available as two Bioconductor experimental data packages curatedAdipoRNA and curatedAdipoChIP. In addition, the packages document the source code of the data collection and the pre-processing pipelines. Here, we provide a descriptive analysis of the datasets with context and technical validation.The 3T3-L1 pre-adipocyte cell line is widely used to study the fat cell differentiation  The 3T3-L1 cell line is used as a cell model for studying the fat cell differentiation –4. The mThe increasing amount of available high-throughput sequencing data necessitates the development of standards for sharing and integrating data. The creators of the primary data are often required to adhere to the standards of the repository where they report and share the data. The development of across repositories metadata standards, ontologies and controlled vocabularies has been attempted to help researchers to share the data they generate and use the data generated by others . DespiteWe surveyed the literature, the gene expression omnibus (GEO) and the sequence read archive (SRA) for RNA-Seq and ChIP-Seq datasets of differentiating 3T3-L1 cells sampled at one or more time points ,9. The mIn this article, we present the methods that were used to collect and process the raw data and provide a technical validation of the final product. We start by describing the cell line model and the induction protocol for adipocyte differentiation. Then, we state the search strategy and the inclusion criteria of the studies. Next, we present the steps and the tools for obtaining and processing the raw data. Finally, we provide links to the software environment and the code for reproducing the full process. Using a subset of the data, we perform several technical validation analyses. First, we check the separation of the samples by phenotype in multidimensional scaling (MDS) and the sample replication using similarity measures. Second, we describe the expression and binding patters of adipocyte markers and enrichment of gene sets which are expected true biology for this model. Moreover, we compared several aspects of the model to human primary adipocytes. Together, the descriptive analysis provides an assessment for the validity of the model and the appropriateness of the curation process.3T3-L1 is a mouse pre-adipocyte cell line which can be induced to differentiate into mature adipocytes when it is treated with a chemical cocktail. The most commonly used variant of these chemical cocktails contains 1-Methyl-3-isobutylxanthine, Dexamethasone and Insulin (MDI). The treatment usually starts by inducing fully confluent 3T3-L1 cell culture (pre-adipocyte) which begins to differentiate shortly after by accumulating lipid into lipid droplet structures within days . In thisWe surveyed GEO and SRA repositories for high-throughput sequencing data of MDI-induced 3T3-L1 pre-adipocyte samples at different time points. The data were obtained from GEO or SRA in the form of raw reads (fastq). In total, 98 RNA-Seq and 187 ChIP-Seq samples  were included. Samples with multiple runs or paired-end runs were obtained separately and combined in later steps of the pipeline. Raw reads were downloaded from the SRA ftp server using Wget. FASTQC was used to assess the quality of the raw reads . We did left) depicts the steps of processing the RNA-Seq data.For RNA-Seq, the raw reads were aligned to UCSC mm10 mouse genome using HISAT2 . Featureright) depicts the steps of processing the ChIP-Seq data.For ChIP-Seq, the raw reads were aligned to the same mm10 genome using BOWTIE2, peaks and signal tracks were built from the aligned reads (bam) using MACS2 ,48. The The technical validation analysis presented in this manuscript was based on the reads count in genes or peaks from the RNA-Seq (n = 98) and ChIP-Seq  samples of the curated datasets. The counts were transformed using the variance stabilization transformation (VST) to adjust the distribution of counts to be comparable across samples. MDS was applied using cmdscale (base R) . The difThe signal tracks from histone modification ChIP-Seq samples (n = 9) were built from the aligned reads using MACS2 . The scodHeatmap . Three ddHeatmap . Human mdHeatmap ,64. The dHeatmap .https://hub.docker.com/r/bcmslab/adiporeg). The scripts used to collect, process and package the datasets are available on GitHub under GPL-3 licence (https://github.com/MahShaaban/curatedAdipoRNA and https://github.com/MahShaaban/curatedAdipoChIP). The code for generating the technical validation figures and the metadata tables in this manuscript is available on GitHub (https://github.com/BCMSLab/curated_adipo_describtor).We packaged the software environment where the code was executed as docker images  and PPARG targets. The counts were first transformed using VST. The stage of differentiation of the samples (0–3) was used to represent the phenotype and it showed an appropriate separation along the first two dimensions of the MDS ). The stEuclidean distances among them as a measure of dissimilarity. With the exception of a few samples, most RNA-Seq samples had low dissimilarity between replicates and close-by phenotype (time point/stage of differentiation)  ). This stiation) ). TherefCebpb and Pparg was turned on at the early (stage 1) and intermediate-late (stage 2–3) differentiation stages, respectively . The exipocytes ). The saipocytes ).10.1080adipose tissue development (GO:0060612); lipid catabolic process (GO:0016042); lipid storage (GO:0019915); glucose metabolic process (GO:0006006); and cellular response to insulin stimulus (GO:0032869) were enriched in most comparisons. The fraction of DE gene members of each term between induced (stage 1–3) and non-induced (stage 0) samples was significantly higher than what is expected by chance alone  genes and differentially bound peaks to perform gene set enrichment analysis of key biological processes-gene ontology (GO) terms – that are most expected to be actively regulated during the differentiation course. The terms ce alone ). The sace alone ).10.1080negative regulation of fat cell differentiation (GO:0045599) and positive regulation of fat cell differentiation (GO:0045600). The modifications varied across two variables; the stage of differentiation and the functional category of genes . In addition, PPARG had binding peaks in the promoter regions of each of the three lipogenic genes (right). The adipogenic factors also showed a pattern of binding consistent with the suggested auto-regulation and feedback loops among them [in vitro model for studying the adipocytes.The 3T3-L1 cell line provides a model for studying the development of the fat cells and the behaviour of the mature adipocytes . Althougic genes , right. ong them . These shttp://bioconductor.org/packages/curatedAdipoRNA/). The package provides a SummarizedExperiment R object. The object contains three tables. The first is the metadata table, which contains the manually curated sample metadata using unified language to facilitate comparing and combining the data from different studies. The two main metadata items are the time point (hours) and the stage of differentiation of each sample . In addition, the metadata contain quality assessment measurements of the samples in the form of qc_read objects. The second table is a GRanges object with essential information about the genes in which reads were counted. The third table is a count matrix of all known mouse genes. Together, the tables can be used in analyses such as differential expression, gene set enrichment and/or time-course analyses.The processed RNA-Seq dataset is available as a Bioconductor experimental data package (http://bioconductor.org/packages/curatedAdipoChIP/). The package provides a SummarizedExperiment R object. The object contains three tables. The first is the metadata table similar to the one described above and information of the ChIP antibodies. The second table is a GRanges object with essential annotations about the Peaks in which reads were counted. The third is a count matrix of the reads in peaks and their gene assignment in the mouse genome. Together, the tables can be used in differential peak binding and occupancy analyses.The processed ChIP-Seq dataset is available as a Bioconductor experimental data package (in vitro adipocyte model 3T3-L1 and curated extensive metadata on the included samples. The raw data were collected and processed using standard pipelines. The product of this study is the construction of gene expression and DNA-binding models of the differentiating 3T3-L1. The processed data were documented and made available as an open-source Bioconductor experimental data packages. The models were assessed for quality and were found to reflect the essential aspect of known adipocyte biology from the published literature and the human primary adipocytes.We surveyed public repositories for high-throughput sequencing data on the


Dozens of normalization methods for correcting experimental variation and bias in high-throughput expression data have been developed during the last two decades. Up to 23 methods among them consider the skewness of expression data between sample states, which are even more than the conventional methods, such as loess and quantile. From the perspective of reference selection, we classified the normalization methods for skewed expression data into three categories, data-driven reference, foreign reference, and entire gene set. We separately introduced and summarized these normalization methods designed for gene expression data with global shift between compared conditions, including both microarray and RNA-seq, based on the reference selection strategies. To our best knowledge, this is the most comprehensive review of available preprocessing algorithms for the unbalanced transcriptome data. The anatomy and summarization of these methods shed light on the understanding and appropriate application of preprocessing methods. The aim of normalization methods for large scale expression data, including microarray and RNA-seq, is to eliminate systematic experimental bias and technical variation while preserving biological variation. Dozens of normalization methods for correcting non-linear experimental differences between arrays have been developed during the last two decades  a majority of genes are equally expressed in each experimental unit and (ii) symmetrical distribution of genes between over-and under-expressed  data-driven procedures, (ii) external controls, and (iii) all-gene reference, are reviewed in the following sections Data-Driven Reference Normalization to All-Gene Reference Normalization, respectively.Methods in this category are generated based on the idea of rank-invariant transcripts presented by Li and Hung Wong . They arGlobal rank-invariant set normalization (GRSN) is based on the general idea of selecting rank-invariant transcripts  calculated from normalized expression value on an array, then we can match the s of the expression value for all transcripts within a particular range of A-values (intensity window) with the template. The Cross-correlation of s with t(M) can be maximized by tuning the matching parameter m. Specifically, the exact steps are described as “the optimal m is the one that can maximize the following formula,Suppose the template be the distribution of A-values. Then s and K(Ai) are calculated for each window, e.g., i in this case. The final normalization factor k(A) is calculated by fitting a spine function according to all k(Ai).”which is assigned as the normalization factor k(A), where M1 is the lower bound and M2 is the upper bound of M component. In a MA-plot, all points are first segmented into n windows according to their Non-parametric Variable Selection and Approximation (NVSA) was developed for normalization of Affymetrix microarrays with a substantially large fraction of DEGs  jointly fits kernel density in both the M and A components. loess regression is then applied to generate normalization curves locally for all data points in the MA-plot , where X and Y are the intensity on the reference array and the sample array to be normalized. A sliding window is then used to train weighted least-squared lines of log(X/Y) and log(X*Y). Finally, every transcript is normalized using the final correction factor relative to the nearest point on the fitting curve.Iterative Rank-Order Normalization (IRON) performs a pairwise normalization for each array against a common reference array iteratively  normalization for mRNA arrays models a function considering both array effect and probe effect  test. (3) An HMM with two states, variant and invariant, is applied to the M-values. Transcripts with the minimum absolute value of mean M are determined as ITS. (4) Normalize the data using a standard normalization only based on the ITS.HMM-normalization is a type of invariant method using a Hidden Markov Model (HMM)  can be adopted to both classification and regression problems  that are expected to produce consistent measurements across arrays. Better still, ITS could be achieved by using the foreign controls, if the controls are designed to be embedded on the arrays. Several types of external controls have been developed for this purpose, including housekeeping genes, spike-in controls and microarray array pool (MSP) controls  is to normalize arrays employ cyclic loess normalization relying on the external control probes  is the relative term of complete quantile normalization (CQN)  is a method of statistical analysis and is widely applied to normalize data shapes  are based on global optimization rather than local optimization. In terms of choosing the reference array, GPA normalization employs median values across all arrays as the common reference array, which is further verified to be a right choice and perform better than the individual array. Another advantage of using GPA for normalization over other methods is that it assumes nothing about data distributions, which makes it applicable for all types of data.This method stressed that normalization is not necessary for microarray studies for which traditional normalization methods may produce more false positive discoveries   between all array pairs, average expression value for each transcript over all the arrays, or the trimmed mean for each transcript among all arrays. When the ITS and common reference array are set well, non-linear regression approaches are implemented to adjust all the data points, such as loess, lowess, weighted lowess, piecewise spline, and LVS.Therefore, a series of models and approaches have been conducted for normalizing a given expression data. A tuned combination of them is expected to show a higher performance, e.g., GRiS + RMSD + LVS, which is the combination of the three steps, identifying ITS, selecting common reference, and regression analysis. Consequently, an optimal hybridization of normalization steps is expected to maximize the power of these available methods.Although more than 20 normalization methods have been developed for the skewed expression data, most of them have their own assumptions and suffer from similar problems. No real assumption-free methods exist but methods based on reasonable and adaptive assumptions are always highly required. Therefore, we stress that an in-depth understanding of data property is critical before conducing normalization. In addition, the interactions between different approaches among these steps should be noted. In other words, the result of one step may affect the other steps heavily, either the parallel or subsequent ones. Besides, it still calls for novel methods for RNA-seq data as for which limited solutions are proposed. Hopefully some existing methods originally developed for arrays can be employed after some sophisticated adaptions.The normalization algorithms we summarized are equally important for mRNAs and lncRNAs, because it is common to perform secondary analyses of lncRNAs by leveraging previously published gene expression data including both microarray and RNA-seq (Zhou et al., Normalization methods impact little to the coexpression analysis, mainly because the gene expression samples in different state or subgroups used to be stratified first and then perform the coexpression analysis. In this case, the skewness or difference between distinct biological states is not taken into account. Like the protein interaction network (Cheng et al., Normalization methods developed for the unbalanced expression data were summarized into three classes based on the expression reference, data-driven reference, extra negative controls, and all genes. To our best knowledge, this is the most comprehensive review of available preprocessing algorithms for the unbalanced transcriptome data. The anatomy and summarization of these algorithms shed light on the understanding and appropriate application of preprocessing methods.LC and XL wrote the paper. NL, SL, JW, NZ, and XZ provided the literature and algorithms. LC and K-SL contributed to the overall paper design.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.