A major challenge in clinical cancer research is the identification of accurate molecular subtype. While unsupervised clustering methods have been applied for class discovery, this clustering method remains a bottleneck in developing accurate method for molecular subtype discovery. In this analysis, we hypothesize that spectral clustering method could identify molecular subtypes in correlation with survival outcomes. We propose an accurate subtype identification method, Cancer Subtype Identification with Spectral Clustering using Nyström approximation (CSISCN), for the discovery of molecular subtypes, based on spectral clustering method. CSISCN could be used to improve gene expression-based identification of breast cancer molecular subtypes. We demonstrated that CSISCN identified the molecular subtypes with distinct clinical outcomes and was valid for the number of molecular subtypes. Furthermore, CSISCN identified molecular subtypes for improving clinical and molecular relevance which significantly outperformed consensus clustering and spectral clustering methods. To test the general applicability of the CSISCN, we further applied it on human CRC datasets and AML datasets and demonstrated superior performance as compared to consensus clustering method. In summary, CSISCN demonstrated the great potential in gene expression-based subtype identification. In particular, advances in cancer genomics studies have revealed the marked clinical and molecular heterogeneity with regard to responses from treatment and survival outcomes13. However, the heterogeneity in tumor samples poses considerable challenges for the evaluation of prognosis and selection of an appropriate treatment for each individual patient14. Thus, there is urgent need to provide the accurate subtype identification method for developing the prognostic and therapeutic strategies.Identifying the subtype of cancer is one of the leading area of study in clinical cancer research. The use of accurate subtype identification typically helps to determine the appropriate therapy and thus improves survival rate for cancer patients. To date, the rapid development of high-throughput platforms such as gene expression profiling15, discovering molecularly distinct subtypes18 and identifying oncogenic pathway signatures11 in cancer research. Specifically, consensus clustering method has been widely used for class discovery20 and the identification of consensus molecular subtypes21. While traditional clustering algorithms are mainly founded on Euclidean geometry and unable to treat nonlinear structure in data, spectral clustering could adapt to geometries in a broader range due to the identification of non-convex patterns and linearly non-separable clusters22. Importantly, spectral clustering has been widely used in machine learning and pattern recognition25. It partitions the points into distinct clusters based on the eigenstructure of the similarity matrix. Accordingly, the points have high similarity in the same cluster and low similarity in different clusters26. Despite its good performance, spectral clustering is often limited in its application for large-scale problems due to its high computational complexity27. To address this challenge, the spectral clustering using Nyström approximation is presented to reduce the computational cost of the matrix decomposition and improve the clustering accuracy29.Traditional unsupervised clustering methods have showed great potential in identifying modular networkIn this paper, we aimed to develop and evaluate spectral clustering method using Nyström approximation for identifying molecular subtypes of cancer. We investigated whether this method could identify molecular subtypes for improving clinical and molecular relevance. We proposed an accurate subtype identification method, Cancer Subtype Identification with Spectral Clustering using Nyström approximation (CSISCN), for the discovery of molecular subtypes, based on spectral clustering method. We first started with the discovery for molecular subtypes of breast cancer patients based on gene expression profiles (GEPs). Then, we demonstrated that, 1) The CSISCN identified the molecular subtypes with distinct clinical outcomes; 2) The CSISCN was valid for the number of molecular subtypes; and 3) The CSISCN identified molecular subtypes for improving clinical and molecular relevance as compared to the consensus clustering and spectral clustering methods. To test the general applicability of the CSISCN, we further applied it on human CRC datasets and AML datasets and demonstrated superior performance as compared to consensus clustering and spectral clustering methods.30, GSE2506530 and GSE653231 were downloaded from the Gene Expression Omnibus (GEO) database. Neoadjuvant study of 310 HER2-negative breast cancer cases in GSE25055 and 198 HER2-negative breast cancer cases in GSE25065 were treated with taxane-anthracycline chemotherapy pre-operatively and endocrine therapy. The clinically distinct molecular subtypes were identified in estrogen receptor positive breast carcinomas GSE6532. In this study, tumor samples from GSE25055 were used as training cohort, and those from two gene expression datasets GSE25065 and GSE6532 were used as independent validation cohorts.Breast cancer consisted of distinct biological subtypes including HER2, ER and PR for different prognostic and therapeutic implications. We have collected breast cancer gene expression datasets using the Affymetrix U133A platforms from public resources. The gene expression datasets GSE2505532. The training cohort GSE1753634 including 111 samples in CRC patients was obtained from GEO database. In our study, we analyzed independent validation cohort GSE1753734 downloaded from GEO database. Stage І and IV samples were excluded from this study. All these two CRC datasets were generated on the Affymetirx U133 plus 2.0 platform. The metastasis gene expression profiles GSE17536 (Moffitt patients) and GSE17537 (VMC patients) were developed from highly invasive mouse colon cancer cells and non-invasive colon cancer cells respectively.Mutations in specific genes APC, KRAS, PIK3CA and TP53 allowed the identification of prognostic subgroups in colorectal cancer (CRC). The TCGA (The Cancer Genome Atlas) study recently reported three transcriptomic subtypes of CRC, which were designated as “microsatellite instability/CpG islandmethylator phenotype” (MSI/CIMP), “invasive”, and “chromosomal instability” (CIN)35. For AML, two gene expression datasets including GSE1241736 (HG-U133A) and GSE1035837 (HG-U133Plus2) were downloaded from GEO database. In GSE12417, 163 samples of bone marrow or peripheral blood mononuclear cells were developed from adult patients with untreated AML. The high-throughput sequencing using genomic DNA or RNA were created from the bone marrow (tumor) and matched skin biopsy samples (germline) from over 300 patients with de novo AML in GSE10358. GSE12417 was used as training cohort and GSE10358 was used as test cohort respectively.Acute myeloid leukemia (AML) patients were classified into M0-M7 subgroups with FAB (French–American–British) criteria38 from the R Bioconductor package. Probe set identifiers (IDs) were mapped to gene symbols with the mapping from the GEO database. The probe set with the largest interquartile range (IQR) was selected owing to its high variation across samples, when multiple probe sets were mapped to the same gene. Probe sets were eliminated when they were mapped to multiple genes. The Z-score transformation was used as a normalization procedure to standardize the expression values of each gene. The datasets were performed separately to ensure their independency. The clinical characteristics of tumor samples with breast cancer, CRC and AML are listed in Table For tumor gene expression datasets, all Affymetrix based CEL files were normalized using the Robust MultiChip Analysis (RMA) algorithmii = 0.Form the similarity matrix d is the reconstitution of the similarity matrix S and constructed with Let A represent the T.Assume Calculate the diagonal matrix Define Laplacian matrix Define R is the eigenvectors.Calculate eigendecomposition of R, Calculate Define the normalized matrix j is vectors corresponding to n rows of i is the centroid of all the points uj belonging to cluster ci. We define Perform the k-means algorithm to cluster n rows of K-means iterations terminated with the relative difference between the two values of the objective function less than 0.001.Input: data points We proposed an accurate subtype identification method, Cancer Subtype Identification with Spectral Clustering using Nyström approximation (CSISCN), for the discovery of molecular subtypes, based on spectral clustering method. For spectral clustering using Nyström approximation, a Matlab implementation was used for this study. For tumor samples of training cohort, we set the parameter σ vary among the candidate set {20, 30, 40, 50} for each cancer type. Full gene symbols were used as outcome-related genes for input features. In the implementation of spectral clustering using Nyström approximation, we let the half sample size of training cohort as the number of random samples for each cancer type. The k-means algorithm was performed to identify the k clusters. The identified k clusters and real prognosis of the patients was assessed by the Kaplan-Meier survival curves and log-rank test. Each choice from the parameter σ was evaluated with log-rank p-value over 10 runs, and the parameter σ with smallest p-value was identified. The identified parameter σ was then performed to test on the independent validation dataset and the performance was evaluated with the Kaplan-Meier estimated survival curves.19. Consensus clustering has proved to be effective in solving different biological problems including gene expression-based class discovery19, identification of biologically functional modules in Protein–Protein Interaction (PPI) networks39, and cancer subtype discovery40. An R implementation of the ConsensusClusterPlus41 available in the ConsensusClusterPlus package was used for consensus clustering method. The pearson correlation coefficient distance was used with hierarchical clustering. The consensus clusters were identified as cancer subtypes from 100 resampling iterations of the hierarchical clustering, by using the full gene symbols (100%) and randomly selecting a fraction of the 80% samples. The identified cancer subtype and real prognosis of the patients was then assessed with survival analysis by the Kaplan-Meier survival curves and log-rank test. The number of consensus clusters was selected from k = 2 to k = 10 respectively.For reference, we compared performance from the CSISCN approach to that from the state of the art unsupervised clustering method consensus clustering approachThe association between the molecular subtype and real prognosis of the patients was evaluated by the Kaplan-Meier survival curves and log-rank test. Standard Kaplan–Meier survival curves were generated for each cancer subtype, and the survival difference between molecular subtypes was statistically evaluated using the log-rank test. An R implementation in the survival package was used for survival analysis. P-values of less than 0.05 were considered statistically significant.Figure We presented the CSISCN to identify the molecular subtypes from tumor GEPs. We investigated whether CSISCN could identify the molecular subtypes in breast cancer as an example. GSE25055 was used as training cohort for clustering development. GSE25065 and GSE6532 were then used as two independent validation cohorts to validate the approach. For each parameter σ, log-rank p-values were generated with repeated ten times runs in order to obtain robust performance evaluation results. In this analysis, the parameter σ = 20 was identified with smallest p-value from training cohort and then performed to test on the independent validation dataset.To identify the difference in gene expression between molecular subtypes, we performed CSISCN to stratify the cancer patients into k clusters. Figure GSE25055 was used as training cohort to develop the CSISCN for identifying molecular subtypes. As shown in Fig. Using optimized parameter σ based on training cohort, the CSISCN was developed to test on the independent dataset GSE25065. Figure To further validate the effect of the CSISCN, we developed the clustering method to test on the independent dataset GSE6532. We observed that the subtype 1 group had significantly worse distant relapse-free survival than the subtype 2 group Fig. . The disConsequently, both training results and independent test results clearly demonstrated that the CSISCN was able to identify the molecular subtypes with significant differences in prognosis.To test the general applicability of the CSISCN, we applied it to CRC gene expression datasets. A CRC gene expression dataset GSE17536 with 111 samples was used as training cohort to develop the CSISCN for identifying molecular subtypes Fig. . Using oIn addition, CSISCN was applied for AML gene expression datasets to further validate the general adaptability. Similar to the above analysis, we collected gene expression dataset GSE12417 Fig.  as trainIn order to evaluate the validity of the CSISCN for the number of molecular subtypes, we tested different numbers of molecular subtypes for each cancer type. According to the association between the molecular subtype and real prognosis in GSE25055, statistically significant differences were found in the stratified patients with log-rank p-values less than 0.05 Figs  and 5a–cAs depicted in Fig. We compared the CSISCN approach with the state of the art unsupervised method consensus clustering approach. In this analysis, we performed the comparisons with different molecular subtypes of each cancer type. Table According to log-rank p-values of breast cancer GSE25055, CSISCN achieved better performance than consensus clustering approach . Meanwhile, CSISCN achieved p-values for different molecular subtypes  which are more statistically significant than consensus clustering in AML cohort GSE12417. Compared with consensus clustering, CSISCN achieved better clustering performance for the identification of different molecular subtypes in the independent test datasets CRC cohort GSE17537 and AML cohort GSE10358 respectively. Indeed, these results reproduced the outcomes in breast cancer and further proved the progress in the CSISCN for identifying molecular subtypes.σ was set among the candidate set {20, 30, 40, 50}, evaluated with log-rank p-value over 10 runs and then identified with the smallest p-value. We tested different number of molecular subtypes for comparison. As shown in Table To further validate the effectiveness of the CSISCN, we compared it with standard spectral clustering method for further analysis. A Matlab implementation available of spectral clustering was used to identify the molecular subtypes from breast cancer GEPs. The Gaussian similarity function was used for spectral clustering to construct the similarity matrix. The parameter We compared CSISCN with spectral clustering method in terms of running time. We performed the runtime experiments on a computer with 3.2 GHz CPUs and 16 GB of memory, without exploiting multi-core parallelization. In the implementation of CSISCN, the running time was separated into three sections including the calculation of similarity matrix, eigendecomposition and k-means implementation respectively. The total runtime for different molecular subtypes with CSISCN was reported in Table The identification of molecular subtype is critical to the development of therapeutic strategy and the understanding of significant heterogeneity for cancer patients. In this analysis, our hypothesis is that spectral clustering method could identify molecular subtypes in correlation with survival outcomes. Furthermore, we developed the accurate subtype identification method for identifying molecular subtypes and thus improving clinical and molecular relevance. The CSISCN was then applied on different types of cancer to identify molecular subtypes and demonstrated superior performance as compared to consensus clustering and spectral clustering methods.42. Specially, this clustering method discards the structures which are always dominated by the arbitrariness of the sample noise and characterized by over-fitting in unsupervised learning42. The results thus demonstrated that CSISCN was able to achieve significantly better performance for three cancer types. As compared to consensus clustering, CSISCN used the pairwise similarities of samples and smaller subset of dense similarity matrix, which thus achieved significantly better performance for the identification of molecular subtypes. Indeed, spectral clustering using Nyström approximation samples columns of the affinity matrix and approximates the full matrix by using correlations between the sampled columns and the remaining columns24, which is different from general spectral clustering method. Importantly, sampling-based spectral decomposition technique, Nyström method, provides a powerful alternative for approximate spectral decomposition. They often operate on a small part of the original matrix and eliminate the need for storing the full matrix43. While the general spectral clustering method needs to construct an adjacency matrix and calculate the eigen-decomposition of the corresponding Laplacian matrix, the Nyström approximation method is typically used for efficiently computing an approximate solution of the eigen-problem. Spectral clustering is mainly based on the manifold assumption, and this assumption is not applicable to identifying a low-dimensional data manifold of high-dimensional data. Actually, the clustering performance of SC will be degraded and even become worse than K-means clustering when high-dimensional data do not display a low-dimensional manifold structure clearly44. In this analysis, spectral clustering using Nyström approximation has been applied to discover the underlying cluster structure which is a lower-dimensional representation of high-dimensional gene expression data and thus identifies the molecular subtypes of cancer. In our study, we noticed the difference between performance gain for various k clusters when CSISCN is compared with two general clustering methods. It is interesting to see that the performance gain is very large for nine and ten clusters in GSE25055 and GSE25065 respectively (Table In our analysis, we used quantile normalization across the experiments to make comparable distributions for all samples. However, strong batch effect remained after this processing step. Importantly, further application of a gene-wise z-score transformation for each dataset separately effectively reduced the batch effect. Considering unsupervised clustering method is able to summarize and explain key features corresponding to several classes to which the data belong, we apply spectral clustering using Nyström approximation for the discovery of molecular subtypes. This unsupervised clustering method is then designed to capture the underlying cluster structures for a lower-dimensional representation of the dataHowever, our findings come up with some caveats. Our analysis is restricted by the availability of genomic data for cancer patients. Moreover, we also notice some exceptional performance between CSISCN and consensus clustering in log-rank p-values Table . SpecifiWith increasing available gene expression data from different types of cancer, CSISCN could bridge unsupervised learning method and accurate subtype discovering tool for the identification of cancer molecular subtypes. In summary, CSISCN shows the great potential for the discovery of molecular subtypes for human cancers.


We present a set of statistical methods for the analysis of DNA methylation microarray data, which account for tumor purity. These methods are an extension of our previously developed method for purity estimation; our updated method is flexible, efficient, and does not require data from reference samples or matched normal controls. We also present a method for incorporating purity information for differential methylation analysis. In addition, we propose a control-free differential methylation calling method when normal controls are not available. Extensive analyses of TCGA data demonstrate that our methods provide accurate results. All methods are implemented in InfiniumPurify.The online version of this article (doi:10.1186/s13059-016-1143-5) contains supplementary material, which is available to authorized users. The role of DNA methylation in cancer has been studied extensively over several decades, in the hope of identifying diagnostic biomarkers and therapeutic targets –3. RecenOne important problem in cancer genomics or epigenomics research, especially from high-throughput technologies, is that the solid tumor tissues obtained from clinical practice are highly heterogeneous. They are always mixtures of cancer cells, adjacent normal tissues, stromal, and infiltrating immune cells. In high-throughput DNA methylation experiments, the whole tumor sample is processed to extract DNA from all cells and then the methylation levels are profiled. Thus, the measurements are in fact mixed signals from different cell types. If not correctly accounted for, such sample mixture could bias the downstream data analyses such as differential methylation and sample clustering, since it increases the within group variation and masks the true biological signal .in silico estimation of tumor purity from high-throughput data is feasible because of the marked differences in genomics and epigenomics profiles between cancer and normal cells. These differences, including differential gene expression, differential methylation, and different mutation and copy number variation patterns, can be used as predictors to estimate tumor purity. A number of statistical methods have been developed for the purpose, based on genetic variants (single nucleotide polymorphism (SNP) or copy number variation) [The sample mixture problem in cancer study was identified a while ago. Estimating the “tumor purity”, or the percentage of cancer cells in a solid tumor sample, has been an active research topic , 13. Expriation) , 16, genriation) , or DNA riation) –20. Wangriation) . Many ofriation) , 23 and In spite of the successes, there are a number of limitations in these purity estimation methods. First, many methods require data from “reference samples”  as predictors in a linear model framework to estimate tumor purity. The reference data could be difficult or expensive to obtain in practice. The reference-free method, such as RefFreeEWAS, requires data from a large number of tumor samples so that the matrix factorization can be performed stably. This poses difficulties for a smaller-scale study. Moreover, several methods require data from normal controls so that differential expression/methylation can be identified and used as predictors , 24, whiDifferential methylation (DM) analysis in cancer-normal comparison is an important task in cancer epigenomics research. The differentially methylated CpG sites (DMCs) or regions (DMRs) identified from such analysis can be further associated with somatic mutations  or gene https://zenodo.org/record/200214.With the continuous cost reduction of technology, large-scale, population level methylation studies have become increasingly prevalent for different types of cancers. The rapid accumulation of data requires a better method for analysis. In this work, we make three important contributions to the field of DNA methylation analysis in cancer. First, we extend our previously developed method for estimating tumor purity from Illumina Infinium 450 k methylation microarray data. The updated purity estimation procedure does not require data from reference samples, matched normal controls, or purity estimated from other tools. The algorithm is extremely simple, intuitive, and computationally efficient, yet it provides results highly consistent with methods based on other data types. Second, we develop a statistical method, based on a linear model, to perform DM analysis for 450 k data with the consideration of tumor purity. Parameters are estimated using a generalized least square and hypothesis testing for DMC is achieved by the Wald test. Finally, we develop a method for detecting DMCs when normal control data are absent. The method draws inferences of DMCs based on the correlation between methylation and purity levels. We show by extensive real data analyses that the proposed methods are sensitive, accurate, and computationally efficient. All proposed methods are implemented in the latest version of InfiniumPurify, which is freely available at The previously developed InfiniumPurify for purity estimation  is basedWe obtained all 450 k methylation data from TCGA  to study the effect of iDMC selection and purity estimation. We found that it is possible to use a group of “universal” normal samples to obtain iDMCs and then apply them on purity estimation for different cancers. We redesigned the purity estimation algorithm, which can be applied to data without normal controls or replicates. The essence of the newly updated method is to combine normal samples from different tissue types, construct a panel of normal methylomes, and then detect iDMCs for each cancer type using this panel for downstream purity estimation. Another important improvement of current version of InfiniumPurify is that it does not rely on ABSOLUTE to calibrate the estimation. Therefore, all purity results in this paper are from 450 k methylation array data alone. The comparison with existing methods shows that tumor purities using universal normal samples are comparable with previous version, even better for cancer types with only small number of normal samples. The algorithm of updated InfiniumPurify is illustrated in Fig. We carefully investigated the genomic locations of the iDMCs. Taking the average across all cancer types, 22% of the iDMCs are located at the transcriptional start site (TSS), 3% are at the transcriptional end site (TES), 11% at the exonic regions, 32% at the intronic regions, and 31% at the intergenic regions. Compared with all CpG sites on the 450 k array, these iDMCs are relatively depleted at gene promoter regions and enriched at intergenic regions. This indicates that the iDMCs are more likely to appear in the less important regions. Moreover, the iDMCs are rather dispersed along the genome (the top 1000 iDMCs are located in 432 genes on average). The spatial diversity of the iDMC is a desirable feature because the purity estimation will not be overly influenced by the differential methylation by a few genes. Finally, overlaps of iDMCs from different cancer types are rather low: the average pairwise overlap is only 2.8%. These results demonstrate the cancer type specificity of iDMCs, thus it is necessary to obtain a set of iDMCs for each cancer. We also associate each iDMC to a gene if it is located within 3000 bps to the gene. On average, iDMCs are located in 432 genes . To compare our estimated purities with other methods, we obtained purity estimates for all cancer samples from , based oWe further looked at the distributions of estimated purities of individuals from different cancer types, shown in Fig. We performed several analyses to investigate the effect of iDMCs selection on purity estimation. First, we investigated the effect of number of iDMCs on purity estimation. We selected different numbers of iDMCs (top N by rank-sum test statistics) and evaluated the purity estimates by their correlation with ABSOLUTE purities. An example for lung adenocarcinoma LUAD is provided in Additional file We also explored the possibility of using normal blood samples as controls in iDMC detection, since the blood samples are much easier to collect in clinical practice. We obtained DNA methylation data for whole blood of 656 human samples aged 19–101 years  and randWe also conducted a tenfold cross-validation in iDMC identification and purity estimation. In detail, all tumor samples from specific cancer types are divided into roughly ten equal groups. Each group is iteratively served as test set, where iDMCs are obtained from remaining nine groups based on the above previous procedures. Results show that the tumor purities estimated from tenfold cross-validation are almost perfectly correlated with that by the whole dataset  the high correlation among different methods and from different data types provide validation to each other; and (2) InfiniumPurify is the first tool to provide purity estimation from methylation data, which is useful and complementary to other methods.With these results, we will take the estimated purities as known constants and develop methods for differential methylation analysis with consideration of purities, detailed in later sections.Through extensive exploration of the TCGA data, we observed that at differentially methylated CpG sites (DMCs), methylation levels in tumors are highly correlated with tumor purities. To be specific, given a set of 450 k data from a number of tumor samples, each has a purity value (estimated from InfiniumPurify). For each CpG site, we computed the correlation between the beta values and the purities across samples. From there we obtained 450 k correlation values for this dataset, each for a CpG site. We also randomly shuffled the sample labels, computed the correlations, and used these numbers as controls for comparison purposes. Figure Next, we looked at the relationship of such correlations and differential methylation. We applied a Wilcoxon rank-sum test on all CpG sites to obtain test statistics. The CpG sites are then categorized into different groups by the test statistics and the distributions of correlations in each category are shown in the boxplots in Fig. We performed the above analysis for all available cancer types and observed the same phenomenon. These results indicate an important finding: DMCs tend to have beta values with greater correlation with purities. This observation is expected because only when the methylation levels are markedly different between cancer and normal samples, the mixed signal will correlate with mixing proportion. For CpG sites showing similar levels of methylation in cancer and normal samples, the mixed signal will be close to a constant regardless of the purity. This observation is an important foundation for our development of the DM calling method with consideration of purity.When normal control data are available, we developed a statistical method to call DMC with consideration of tumor purity. The performance of the method is demonstrated in both simulation studies and real data analyses.We performed extensive simulation to compare the performances of DM calling from different methods. We used the LUAD as a template to generate data so that the simulated data matched the real data characteristics. We conducted simulations for various scenarios under different sample sizes and signal-to-noise ratios. A detailed description of the simulation is provided in Additional file We further applied the proposed DM calling method to all TCGA data whenever the 450 k data were available. We compared the DMC calling results with minfi , arguablFirst, we examined the sensitivity in DM calling. Figure We compared the absolute methylation differences for InfiniumPurify exclusive, minfi exclusive, and common DMCs from BRCA data. As shown in Additional file Next, we looked at the spatial correlations of test statistics from different methods. For each cancer type, we first selected pairs of CpG sites with distances less than 50 base pairs and computed the Pearson’s correlation of their test statistics. It was known that methylation levels have strong spatial correlation , that isWe further looked at the correlations among test statistics from different types of cancers. Even though different cancer types have distinct etiologies, they also share many commonalities, such as the hyper-methylation in CpG islands and genic regions and global hypo-methylation in whole genomes especially for highly and moderately repeated DNA sequences . Hence, p values for the enrichment of DMGs in “PATHWAYS_IN_CANCER,” which contains 328 genes involved in all cancer types. The p values are much smaller from the proposed method, indicating stronger enrichment. We further examined the enrichment of DMGs in pathways related to different cancer types  with most DMCs by different methods. Then DMCs mapped to these genes are input to gometh function in missMethyl package  to test p values (p < 1e-20). These examples illustrate the importance of correcting purity in the DM calling procedure.To better understand the differences in DM calling results from the proposed and other methods, we explored the raw data of CpG sites with substantial discrepancies in test results from the InfiniumPurify and minfi. Additional file Taken together, the results presented in this section show that the proposed DM calling method is more sensitive, accurate, and provides more biologically interpretable results compared with existing methods.Taking advantage of the observation that methylation levels for DMCs tend to have higher correlation with purities, we developed a method to call DMCs without normal control. We then applied the method to all TCGA samples to call DMCs without using the data from normal samples. The DMCs called with control data are used as gold standard to benchmark these results. We generated receiver operating characteristic (ROC) curves for the results from all cancer types  and that the purities among samples need to be dispersed enough. The results could also be affected by the signal-to-noise ratios in the data (the ratios of inter-group and intra-group variations). For some cancer types, control-free DM calling could provide undesirable results. Nevertheless, the control-free DM calling results are satisfactory overall. We want to emphasize that one should profile the normal controls whenever possible if differential methylation is a major research interest. However, when normal control data are absent due to clinical or economic reasons, we provide a viable solution for DM calling.Existing methods for tumor purify estimation are mainly based on gene expression or copy number data from either SNP array or high-throughput DNA sequencing. InfiniumPurify is the first method to provide purity estimation from DNA methylation microarray data. We want to emphasize that in cancer studies, the genetic or epigenetic data  are not specifically generated to measure cancer purity: those experiments are performed to study different aspects of cancer. Thus, it is important to be able to estimate purity from all types of data. If the purity could only be estimated from copy number data, then in an EWAS study where only DNA methylation data are available, one would not be able to estimate purity. From this perspective, it is equally important and useful to have purity estimation methods from different data types. In addition, DNA methylation is deemed to be more stable than gene expression, so it is potentially more accurate in purity estimation problem. Although copy number alteration is a characteristic of cancer cells and is also less variable than gene expression, cancer cells often have aberrant overall ploidy number compared with normal cells, which greatly affects copy number-based purity estimates. For example, ABSOLUTE needs a user selected tumor ploidy in determination of the optimal likelihood. Due to these reasons, tumor purity estimated from DNA methylation data could be more stable or at least plays a complementary role to exiting tumor estimation approaches.There are some reports about the intermediate methylated (IM) sites and the methylation heterogeneity in cancer –57. We wIt is important to note that in our purity estimation procedure, combining normal samples from different tissue types might increase the variation within the normal group and it was the reason why in the earlier version of InfiniumPurify we used matched controls in each cancer type to identify iDMCs. However, through comprehensive data analysis we notice that combining normal samples in fact produces comparable results. We believe this is a better approach and will have a wider application, for example, purity estimation can be performed for cancer types not included in TCGA. For any cancer type, as long as the sample size is reasonably large (e.g. ≥20), the iDMCs can be reliably detected by comparing the cancer to the universal normal controls and the purity can be estimated. We also want to point out that our DM calling  methods require relatively larger samples size compared with minfi, limma, or related tools. Thus, we envision that InfiniumPurify will be mostly applied to population level studies. Control-free DM calling also requires that the purities among samples are dispersed enough so that the statistical test can be reliably performed. To this end, we want to emphasize that the control-free DM calling method should be used with caution and normal controls should be profiled whenever possible.Beyond differential methylation, analysis of other types of genomics data analysis such as differential expression between cancer and normal also suffer the complication of tumor purity. We believe similar principals proposed in this work can be applied to analyze the gene expression data, even though the detailed data modeling is different. This is also a research area we will explore in the near future.Tumor purity is an important factor of clinical tumor tissues reflecting both intrinsic property of a cancer type and how accurate these samples are collected. It could have a great impact on many cancer data analyses including differential expression, copy number alteration, differential methylation, genome-wide association studies, and EWAS. It is important to be able to estimate and adjust for tumor purities in these analyses. In this work, we develop a series of statistical methods for DNA methylation microarray data analysis in cancer, including purity estimation and DM calling with and without normal controls. The newly designed purity estimation procedure has greatly enhanced the application of InfiniumPurify for many cancer types with few or no normal samples. We estimate tumor purities of all tumor samples with 450 k data and show that our estimated purities are highly consistent with those from other popular tools. With consideration of purity, the DM calling results from cancer-normal comparison are shown to be more sensitive, accurate, and biologically meaningful. The control-free DM calling method provides a solution for data without normal control and provides new biological insights for more cancer types.The purity estimation algorithm by InfiniumPurify is illustrated in the purity estimation module in Fig. p values from the rank-sum test) as iDMCs and used them for purity estimation  = arcsin (2x − 1). The transformation is necessary because the transformed data follow Gaussian distribution much better compared to the raw data, especially at the boundaries (0 and 1). This allows us to use a linear model with Gaussian noise in following method. The arcsine is a “variance stabilization transformation” for random variables from a beta distribution. Such transformation possesses several advantages over more commonly used logistic (logit) transformation. First, it stabilizes variance, e.g. the variance does not depend on the mean anymore. This greatly reduces the heteroscedasticity problem in regression. Second, it is more linear than logit. The methylation level from mixed cancer-normal samples is assumed to be a weighted average of those from the pure samples and the signal mixing is at the original scale. Having a more linear transformation allows us to use a linear model for transformed data with better approximation. Previous work on differential methylation analysis from bisulfite sequencing uses the arcsine transformation and obtains good results [The proposed DMC calling method works for data from one cancer type. The input raw data are beta values for  results .i, denote the transformed beta values from normal samples as Xi, and assume Xi ∼ N. Denote the transformed beta values from “pure” cancer samples as Yi and assume Yi = Xi + δi. δi is a random variable representing the difference between cancer and normal samples. It is assumed that δi ∼ N. With Xi and Yi from a number of samples, the differential methylation detection is achieved by hypothesis testing: H0 : μi = 0. However, in practice, the data from pure cancer sample Yi is not observed. Instead, we observed the signal from mixed cancer-normal samples, denoted by Yi '. For cancer sample s with known purity λs, we have Yis ' = (1 − λs)Xis + λsYis = (1 − λs)Xis + λs(Xis + δis) = Xis + λsδis, so Yis ' ∼ N. Here, σi'2 is the variance for Yis' , and σi'2 ≠ σi2. It is worth mentioning that Xi and δi have moderate negative correlation in real data , instead of additive. Therefore, the existing model for differential methylation or differential expression with the consideration of tumor purity by using purity as an additive covariate [Xis and Yis' as input data and test μi = 0.For CpG site mi, σi. Denote ovariate  is incori, we denote all input data by a vector n0 items are numbers from normal samples and the next n1 items are from cancer samples. The input data can be represented by following the linear model: Zis = mi + asμi + ϵs, s = 1, 2, …, n0 + n1, where as = 0 when s ≤ n0 and as = λs when n0 < s ≤ n1. In this model, μi is the parameter of interest that will be tested. The residual variances are σi2 and σi'2, respectively, for normal and cancer groups. This method essentially uses tumor purity as an experimental design factor in a linear model, so that the correct inference on differential methylation can be obtained.For CpG site The parameter estimation can be performed by following the generalized least square method. For a CpG site, letZ is a vector for transformed methylation levels in n0 normal and n1 tumor samples, W is a n0 + n1 by 2 design matrix with n0 0’s and tumor purities in the second column, β is the linear model parameter to be determined, and ϵ is the error term. The linear regression model can be formulized as Z = Wβ + ϵ, and the model parameters can be solved by the following normal equation,where H = (WTW)−1WT, and var(Z) is in the form of σ2 and σ'2, the residual variances from normal and cancer groups. To estimate σ2 and σ'2, once we have So We applied a shrinkage estimator, similar to the one proposed in , on the H0 : μ = 0 isAfter getting where t distribution with n0 + n1 − 2 degrees of freedom to obtain nominal p values. Adjustment of multiple testing can be done by applying canonical procedure to compute FDRs [Finally, we assume the Wald test follow a ute FDRs .Yis ' ∼ N and wish to test the difference in average methylation levels between cancer and normal, e.g. μi = 0, without control data. With known tumor purities λs, the hypothesis testing can be performed even without control data. Through a simple linear regression using the data from tumor samples (Yis ') as response and tumor purities (λs) as independent variables, the difference in means (μi) is the slope in the regression and can be tested. We want to note that the test statistics from such regression is equivalent to the Pearson’s correlation between Yis ' and λs, but the regression procedure offers some flexibility to incorporate other criteria. We find that sometimes a CpG site has large test statistics but relatively small effect size, due to small standard error. To limit such effect, we use the posterior probability Pr(|μi| > c) to rank the CpG sites, where c is a user defined quantity to require the difference between cancer and normal to be larger than a threshold. Using the test statistics . In practice, we used c = 0.1 and found that it provides better performance than using c = 0.Following the notations in the cancer-normal DM calling method, we have


The selection of multimarker signatures is a complicated process that requires integration of data signatures with sophisticated statistical methods. We developed a user-friendly tool, called CombiROC, to help researchers accurately determine optimal markers combinations from diverse omics methods. With CombiROC data from different domains, such as proteomics and transcriptomics, can be analyzed using sensitivity/specificity filters: the number of candidate marker panels rising from combinatorial analysis is easily optimized bypassing limitations imposed by the nature of different experimental approaches. Leaving to the user full control on initial selection stringency, CombiROC computes sensitivity and specificity for all markers combinations, performances of best combinations and ROC curves for automatic comparisons, all visualized in a graphic interface. CombiROC was designed without hard-coded thresholds, allowing a custom fit to each specific data: this dramatically reduces the computational burden and lowers the false negative rates given by fixed thresholds. The application was validated with published data, confirming the marker combination already originally described or even finding new ones. CombiROC is a novel tool for the scientific community freely available at  Powerful biomarkers have become and will continue to be important tools in diagnostic, clinical and research settings. In the area of diagnostic medicine, a biomarker is often used as a tool to identify subjects with a disease, or at high risk of developing the disease. Moreover, it can be used to foresee the more likely outcome of the disease, monitor its progression and predict the response to a given therapy. Accurate markers are powerful tools to help clinicians in the choice of the most appropriate treatment, ultimately improving a personalized patient management. In the last years the identification of disease-associated signatures has been accelerated by the use of high-throughput omics techniques124567911151618In this study we describe CombiROC, an easy-to-use tool implemented as a web application to accurately determine optimal combinations of markers from diverse complex omics data. CombiROC takes advantage of the combinatorial analysis and ROC curves; although these methods are being used in medicine and other areas for many decades, they generally lack an easy-to-use interface that researchers without programming skills could use to analyze data and to make plots. CombiROC stems from our previous screening for multi-marker signatures for Autoimmune Hepatitis (AIH)http://combiroc.eu. A full online tutorial can also be found, allowing users to practice with pre-loaded real datasets.We validated the CombiROC method with independent datasets from both the screening and validation phases of a study on myotonic dystrophy type 1 patients20CombiROC, delivers a simple workflow to help researchers in selecting the optimal combinations of markers and providing an easy-to-use interface. The workflow can be split conceptually in two main sequential phases . The firData from profiling results of two classes or categories to be compared can be uploaded in text format, which may include comma, tab or semicolon separated text files ; these are popular methods in OMICs, whose use depends on the nature of data 24Then test stringency needs to be set: in this key step the test’s signal cutoff and the minimum number of positive features above the cutoff can be modified and should be dictated by the specific nature of the technological platform used for markers detection. The aim of this step is to submit to the combinatorial analysis that follows only those markers whose signal was positively detected; each user is responsible for the assessment and use of the specific detection limits since this limit depends on the detection method used to generate the data.Generation of the marker combinations based on SE and SP filters. Once the test signal and minimum features cutoffs are set, the application performs a combinatorial analysis and calculates SE and SP scoring for each marker combinations, interpreted as the probability (expressed as a percentage) that the assay data will be positive when the disease is present and as the probability that assay data will be negative when the disease is absent.Ideally a medical test should have high values of both SE and SP, so marker combinations with small values are considered to be of negligible importance from a clinical point of view. The distribution of these metrics is shown in Each marker and combination thereof are visualized by an interactive bubble plot , in whicROC curves of markers and combinations selected in the previous step are calculated with all key parameters  and graphically visualized . PredictAll four possible categories  are displayed through a violin plot which help visualize the proportion of samples falling in the four possible quadrants, especially in those of the TN and TP predicted categories, in order to evaluate the goodness of the underlying markers . The piede-novo differential analysis obtaining a ranking of all detectable microRNAs and a final signature of significantly differentially expressed microRNAs. We then asked whether the top microRNAs in the selected signature performed as biomarkers according to the CombiROC procedure when compared to non significant microRNAs, and how CombiROC would differentiate these microRNAs according to their calculated significance. Since n features  can be combined in (2n − 1) ways, just 100 features would lead to 1.26 ∗1030 combinations. Thus, in order to obtain manageable yet informative numbers, we sampled 4 different smaller sub datasets of 5 microRNAs each across the significance-ranked profiling data results. In particular, we selected i) the top 5 significant microRNAs (“top” subset), ii) the 5 microRNAs just below the significance cutoff thresholds of pValue = 0.05 (“borderline” subset), iii) 5 microRNAs in the middle of the list (“middle” subset) and finally iv) the less significant microRNAs namely the 5 microRNAs with the highest p-Values (“bottom” subset)  patients subset) . Each su subset)  and thei subset) . MoreoveWe then wanted to show CombiROC’s versatility using other datasets derived from two different studies, one focus on proteins and another on microRNAs potential biomarkers. Results from these analyses were also used to build the demo-datasets and tutorial available online to illustrate the functionality of CombiROC.As a first test, we employed CombiROC to re-analyze proteomics data published by our group. We recently discovered a 5-biomarker signatures, IL4R, AQA598, RSPO3, CHAD and UNQ9419 for AIH, by serological screening of patients with AIH and healthy donors (HD) on a protein microarray representing 1626 human proteins20We further challenged CombiROC reproducing combinatorial analysis from another work whose detailed analysis methods and original datasets at the raw, unprocessed level were publicly available and a ROC analysis was performed. To this aim, we selected microRNA transcriptomics data published by Baraniskin and colleagues, which led to the identification of a 6-microRNAs biomarker signature for the diagnosis of Primary Central Nervous System Lymphoma (PCNSL)The possibility to select combinations which are redundant in performance but different in composition, is a potential advantage in terms of flexibility of a diagnostic test that could be more robustly designed deploying parallel tests with different - but equally performing - combinations of markers, thus minimizing any detection or technical failure.shinydashboard, to create easily attractive dashboards; DT, to render data objects as HTML tables providing advanced interaction controls; gtools for creating combinations; rmarkdown for generating dynamic documents; ggplot2for creating elegant and complex plots; ineq, e1071 and stringr, for data manipulation; pROC and cvAUC for ROC analysis. During the working session, the R tasks are executed as back-end processes not requiring the R knowledge. Moreover, Highcharts JS libraryThe core of CombiROC is written in RD = {thr, minf}, according to the specific nature of the user experiment (the test), to classify samples as positive or negative cases where thr denotes the cutoff above which the features’ values are considered positive  whereas minf represents the minimum number of features that need to reach the previously cutoff within the feature panel. Successively, all possible combinations, both as single marker and combination markers, were generated; given N biomarkers, the total number of enumerable combination is:Individual biomarkers and the biomarker combinations are examined using a user-defined test stringencythr, so samples with one biomarker value exceeding the signal cutoff, thr, were classified as positive whereas when a combination of biomarkers was used, at least minf biomarkers exceeding the thr value was considered to classify the sample as positive test.where N is the total number of items  and k is the number of components in the combination. Given the collection of all the combinations of a dataset, it was mandatory to identify a combination score related to the generated combinations. At this point, each biomarker (single or combination) is assigned a binary result (a 1 or a 0) based upon whether the detected level of the biomarker in the sample exceeds the signal cutoff, SE) and Specificity (SP) filters, interpreted in terms of recognition frequency, are computed. SE is defined as the true positive rate in percentage of your sample whereas SP is defined as the true negative rate in control class in percentage. The sensitivity and specificity are reported either as a numeric value or as histogram plot. This allows to evaluate the intervals from which the best SP/SE values will be chosen, and on which markers’ “Gold Combinations” will be calculated in further steps of the analysis.For each of the combinations satisfying the hard thresholds, the Sensitivity  to 1.0 (complete separation). In a ROC curve plot, the true positive rate, “sensitivity” (y-axis), is plotted as function of the false positive rate, “1 – specificity” (x-axis). Large “y” values on the ROC curve plot correspond to higher sensitivity, while small “x” values correspond to higher specificity. The shape of the curve depicts the combinatorial variation of these two important parameters. Below the ROC curve you will be able to see the AUC (Area Under the Curve), SE, SP and optimal cutoff numerical values of the analyzed combo in tabular form. The results are reported in fractions from 0 to 1. A diagonal line of identity is reported such as the point of optimal cut-off.The “Predictions” section is visualized by violin plot and pie chart. The violin plot is a combination of a box plot and a kernel density plot showing the “probability density” of the data at different values. Prediction probabilities are plotted for both classes  according to the previously obtained optimal cutoff on the corresponding ROC curves.To give an estimate of the panel performance in an unbiased manner a cross-validation (CV) procedure was introduced; without proper validation testing, strong biases, such overfitting, undermine the robustness of a classification model.A single CV procedure allowed us to randomly generate the training and test partitions from the original dataset, in this way ROC curve of a biomarker panel based on both the training and the test set is performed. To give reliable estimate of model performance, multiple rounds of CV are performed using different test sets; so, a family of ROC curves are generated and the performance results are avaraged over the rounds. A 10-fold CV strategy was used to compare the different models generated.One limitation of CV is that it provides over-optimistic results, so in order to give a measure of the statistical significance of the AUC value and to test the model for over fitting due to chance correlation a permutation test was introduced. Thus, only the Y-block (class labels) was randomly reordered while the X-block (expression matrix) was left intact. By repeating this procedure N times, a model is fitted to the new Y-data and new estimates of AUC values were calculatedThree real experimental datasets were used to test its applicability to different “omics” fields and to validate and illustrate the functionality of CombiROC.Raw data on plasma microRNAs in myotonic dystrophy type 1 (DM1) patients were used for method’s validation. Datasets consist of 50 samples (26 controls and 24 DM1 patients) in a high through-put RTqPCR screening phase and 72 samples (36 controls and 36 DM1 patients) in a validation phase by single RTqPCR assaysmarker1 to marker5 respectively).Data on autoimmune hepatitis (AIH)miR-21, miR-19b, miR-92a, miR-15b, miR-106b, miR-204) selected by the authors as candidate markers. Data of the above mentioned microRNAs were formatted and uploaded to CombiROC application; in the “Demo data, transcriptomics” the microRNAs miR-21, miR-19b, miR-92a, miR-15b, miR-106b, miR-204 were marked with the labels from marker1 to marker6 respectively. In order to preserve their original structure, PCNLS data were taken as is and not subjected to any optional transformation step.Data on microRNA biomarkers for the diagnosis of primary central nervous system lymphoma (PCNSL)https://www.prometheuslabs.com/Products/Default.aspx; Mammaprint 70-Gene Breast Cancer Recurrence Assayhttp://www.agendia.com/healthcare-professionals/breast-cancer/mammaprint/). Nevertheless, the demand of novel markers signatures associated to life-threatening human diseases is still very high. Many research and clinic laboratories are putting their efforts to address this need by exploiting various OMICs technologies. However, the identification of multi-marker signatures is often time-consuming and labor-intensive for common research and clinical laboratories, and the use of the needed bioinformatics and statistical methods requires skilled scientists making this process of not trivial application.The investigation, discovery and optimization of biomarkers in the context of translational medicine is of fundamental importance for accurate diagnosis and monitoring of disease, predict the most likely outcome and the response to a given therapy. It is now well accepted that single markers may not be sufficiently accurate, while multi-marker combinations can achieve significant specificity and sensitivity values. Indeed, a number of assays based on multiple markers is already commercialized and used in clinic  through a simple and interactive workflow. Through its interactive interface, it allows users to make choices based on evaluation of all possible markers, double-filter scoring on sensitivity and specificity, selection of best-performing combinations, and visualization of their receiver operating characteristic (ROC) curves.n features  can be combined in (2n − 1) ways, thus 5 features lead to 31 combinations, 10 features to 1032 and 100 features would lead to 1.27 * 1030 combinations. We believe a reasonable number of initial potential biomarkers to analyze with CombiROC is usually below 10. CombiROC is fully focused on ROC-curve centered analyses bases on a 2 by 2 contingency table, thus it handles two-class problems such as the most common case in which pathological samples need to be distinguished from healthy ones. The two classes analyzed can indeed belong to two different stage/grades of a disease (as in disease subtyping) but more complicated subtyping would be a multi-class problem and it should be analytically treated differently.CombiROC is a tool devoted to the ranking and selection of potential biomarkers from signatures of any kind of feature . Prior to submission to CombiROC, any given initial screening dataset is required to undergo a data reduction process typically performed with differential expression analysis or similar methods. Given its combinatorial nature, CombiROC is not designed to directly analyze profiling datasets since this would lead to unmanageable number of combinations. In fact, any The test stringency can be manually adjusted . CombiROC then determines sensitivity and specificity of all possible marker combinations. Picking a threshold for these metrics is not straightforward as their absolute values depends on the specific nature of the experiment that generated the data and on the aim of the user. One of the most evident added value of our web application is the possibility to select the best combinations interactively, by exploring the sensitivity/specificity space with immediate visual feedback thereby evaluating any taken tradeoff. Users set their final thresholds and see identity and performance details of the optimal combinations. ROC, prediction, and performance analyses are automatically performed, allowing comparison of combinations. This guarantees an optimal classification, and is better suited to the non-normally distributed data commonly found in clinical studies, where the last increments may not be as significant as the first ones. CombiROC automatically manages all of the above using a recognition frequency analysis encompassing the limitations imposed by the nature of different biomarker panels. Graph plotting are an important support of data analysis, and many webtools are dedicated to this30An important part of our study was dedicated to the validation of CombiROC method with data from transcriptomicsOverall, we believe the CombiROC application is a valuable tool mainly due to the following reasons: 1) it is a free tool that allows to perform robust combinatorial analyses without a specific training; 2) it allows the users to interactively intervene on the choice of thresholds, thus dramatically reducing the computational burden without sacrificing the statistical soundness; 3) it makes easier to analyze panels of biomarkers lowering false negative rate given by fixed thresholds.Thanks to these advantages and ease-of use, CombiROC specifically alleviates the lack of expertise providing a fast, easy yet rigorous tool, and contributing to decrease costs to laboratory practice.In addition to these technical and practical considerations, a comment is deserved on the usefulness of CombiROC to address so far unsolved biological questions. Indeed, combinations of multiple markers may suggest the existence of functional, physical or metabolic relations among single markers composing them. Although not specifically investigated in this study, we believe that CombiROC could help provide new hints in our knowledge of human diseases.Future prospects of CombiROC foresee its applicability to integration of multi-OMICs data. Indeed, multivariate discovery approaches represent the natural evolution of this application, which enable combinatorial analysis of datasets generated by different OMICs technologies . This further development is in line with the recent directive of public funding agencies  promoting the integration of multiple OMICs to find and validate the most accurate marker combinations.How to cite this article: Mazzara, S. et al. CombiROC: an interactive web tool for selecting accurate marker combinations of omics data. Sci. Rep.7, 45477; doi: 10.1038/srep45477 (2017).Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.


This paper describes a new resource, HDBR  Expression, for studying prenatal human brain development. It is unique in the age range (4 post conception weeks [PCW] to 17PCW) and number of brains (172) studied, particularly those under 8PCW (33). The great majority of the samples are karyotyped. HDBR Expression is also unique in that both the large-scale data sets  and the corresponding RNA and DNA samples are available, the latter via the MRC-Wellcome Trust funded HDBRLarge-scale/high-throughput studies, such as next-generation sequencing, are providing raw material in a wide variety of research fields  with the majority (82%) collected between January 2010 and January 2014 (01/28/2014 last collection date). Tissues were either fixed at 4°C in 4% paraformaldehyde and embedded in paraffin wax following standard protocols  samples of embryonic-derived placenta or skin from embryos or fetuses that had been fixed and paraffin wax embedded. These tissues were used solely for DNA preparation and SNP genotyping. (b) Brain tissues where each sample was subdivided and part used for RNA preparation and part used for DNA preparation, followed by RNA-sequencing and SNP genotyping respectively. (c) Brain tissues where each sample was subdivided and part used for RNA preparation and part used for DNA preparation, followed only by RNA-sequencing. This meant that, where tissues from several different brain regions  were collected from individual human embryos or fetuses, SNP genotyping was only carried out once. However, DNA was prepared separately from each of the regions sampled for a particular individual embryo or fetus, and these are available for future studies, e.g., epigenetic analysis  telencephalon (left and right in some cases) and diencephalon or (2) cortex , basal ganglia and diencephalon; hindbrain (cerebellum and medulla). The midbrain was collected as a single sample except in a few cases where it was dissected into left and right parts. Figures We also collected 29 cerebral choroid plexus samples and 42 spinal cord samples. There are also 99 samples where the region of brain could not be determined and these are simply labeled “brain fragments.”The tissues were sent to AROS Applied BiotechnologyDNA was extracted from 435 human embryonic and fetal tissue samples on the QIAsymphony SP using manufacturer'sRNA was extracted from 705 human embryonic and fetal tissue samples. After lysis using the TissueLyser and removal of fat from the sample with chloroform, RNA  was purified on the QIAsymphony SP using protocol miRNA v05. The RNA yield was estimated using Nanodrop A260 measurement and the quality evaluated for approximately 15% of the samples using an Agilent Bioanalyzer. Seventy samples had either too little RNA or the RNA was of insufficient quality. A further 3 samples failed the quality control tests at the library preparation stage (see below), 4 samples were excluded because they did not match their corresponding DNA genotyping data meaning that RNAseq datasets were obtained from 628 tissue samples in total.SNP genotyping was carried out according to the Illumina Infinium LCG Quad Assay protocol6-fold dilutions of the libraries. The size profile of approximately 15% of the libraries was evaluated using an Agilent Bioanalyzer DNA 1000 chip. The average final library size was between 272 and 467 bp (includes 120 nucleotides of adapter sequence). The libraries were sequenced on an Illumina HiSeq2000.cDNA was generated from the RNAs using Illumina's Stranded mRNA Sample Prep Kit followed by library preparation following Illumina's guidelines for the TruSeq Stranded mRNA LT sample prep kit. Four hundred ng of total RNA was used as the input for each sample. The concentration of each library was determined using the KAPA qPCR kit (KK4835) and triplicate reactions using three independent 10Q < 30) and standard Illumina  paired-end sequencing adaptors on 3′ ends of reads were trimmed off using autoadaptRNA-seq data were processed and analyzed to identify differentially expressed genes. The quality of sequencing reads was first checked with FastQCTable The RNAseq data files are all fastq format and the raw data files for the SNP genotype have been uploaded. Both the RNAseq and SNP genotyping files are identified by the sample name which links to the sample information in the “sample attributes and variables” tab in ArrayExpress and in the Supplementary Tables on the HDBR website. The experiment number for the RNAseq data set is E-MTAB-4840 and for the SNP dataset is E-MTAB-4843. The RNAseq data set will be incorporated into the European Bioinformatics Institute (EBI) Expression AtlasPrincipal component analysis (PCA) analysis was carried out based on the normalized gene expressions from the RNA-seq datasets with the samples categorized according to gross region . The datasets from brain fragments and choroid plexus were also included. From Figure A subset of 64 RNA-seq datasets from anterior, central, posterior, and temporal cortex taken at either 9 or 12 PCW were selected for further differential expression analysis. Figure All authors contributed to revising the work, had final approval of the version to be published and agree to be accountable in relation to the accuracy and integrity of the work. SJL drafted the paper and PC, SJL, MS, and AC made substantial contributions to the conception and design of the work; YX, LH, GC, and MK made substantial contributions to the analysis and interpretation of data and SNL, DG, AT, and JC made substantial contributions to the acquisition of data.We gratefully acknowledge funding for this work from the UK Medical Research Council (grant number MC_PC_13047). PC is a Wellcome Trust Senior Fellow in Clinical Science (101876/Z/13/Z), and a UK NIHR Senior Investigator, who receives support from the Medical Research Council Mitochondrial Biology Unit (MC_UP_1501/2).The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


One approach to improving the personalized treatment of cancer is to understand the cellular signaling transduction pathways that cause cancer at the level of the individual patient. In this study, we used unsupervised deep learning to learn the hierarchical structure within cancer gene expression data. Deep learning is a group of machine learning algorithms that use multiple layers of hidden units to capture hierarchically related, alternative representations of the input data. We hypothesize that this hierarchical structure learned by deep learning will be related to the cellular signaling system.—leading to the discovery of clusters of glioblastoma multiforme with differential survival. One of these clusters contained all of the glioblastoma samples with G-CIMP, a known methylation phenotype driven by the IDH1 mutation and associated with favorable prognosis, suggesting that the hidden units in the 3rd hidden layer representations captured a methylation signal without explicitly using methylation data as input. We also found differentially expressed genes and well-known mutations  that were uniquely correlated with each of these clusters. Exploring these unique genes and mutations will allow us to further investigate the disease mechanisms underlying each of these clusters.Robust deep learning model selection identified a network architecture that is biologically plausible. Our model selection results indicated that the 1st hidden layer of our deep learning model should contain about 1300 hidden units to most effectively capture the covariance structure of the input data. This agrees with the estimated number of human transcription factors, which is approximately 1400. This result lends support to our hypothesis that the 1st hidden layer of a deep learning model trained on gene expression data may represent signals related to transcription factor activation. Using the 3rd hidden layer representation of each tumor as learned by our unsupervised deep learning model, we performed consensus clustering on all tumor samplesIn summary, we show that a deep learning model can be trained to represent biologically and clinically meaningful abstractions of cancer gene expression data. Understanding what additional relationships these hidden layer abstractions have with the cancer cellular signaling system could have a significant impact on the understanding and treatment of cancer. Understanding the cellular signal transduction pathways that drive cells to become cancerous is fundamental to developing personalized cancer therapies that decrease the morbidity and mortality of cancer. Most research studying cellular signaling pathways has concentrated on a handful of signaling proteins in a hypothesis-driven framework. This study uses a deep learning approach to study cancer signaling systems in a large-scale data-driven fashion, with an overall goal of understanding the cellular signaling pathways that drive or cause cancer. Towards this goal, we used unsupervised deep learning to find meaningful structure and relationships in cancer gene expression data.Deep learning models (DLMs) originated from artificial neural networks (ANN) and learn alternate representations of the original input data. A DLM is composed of multiple layers of latent variables  –5, whichMore specifically, we hypothesize that the activation states of signaling pathways regulating transcriptomic activities in cancer cells can be learned using DLMs Fig. . Cancer Recently, we demonstrated that DLMs could be used to learn and represent the cellular signaling system , 9. In oIn this study, we investigated the optimal architectures of DBN-based models to learn accurate representations of the signals underlying cancer transcriptomic changes across 17 cancer types. We show that a DLM can provide novel abstract representations, enabling us to reveal molecular subtypes of cancers, e.g., subtypes of glioblastoma multiforme (GBM), that exhibit significant differences in outcomes. Our analysis revealed different potential disease mechanisms (major driver genes) underlying these molecular subtypes.The data used in this study were obtained from The Cancer Genome Atlas (TCGA) Data Portal , and incWe discretized the expression value of a gene in a tumor to 1 or 0 based on whether or not the expression value significantly deviated from the expression observed in normal tissue. To achieve this, we fit the expression values of each gene in each cancer type to a Gaussian distribution based on the non-cancer control samples only from the same tissue of origin. We then set the expression status of a gene in a tumor to 1 (aberrant) if it was outside the 0.001 percentile of distribution of control samples (on either side), otherwise we set it to 0. For genes with low expression variance in normal cells, i.e., standard deviation of expression smaller than 0.2, we used 3-fold change to determine whether the genes were differentially expressed in tumor cells. Through this discretization process, we identified genes that were potentially relevant to the cancer process (aberrantly expressed in cancer only) rather than just using the whole gene expression profile of a cell, which includes both physiological and pathological signals. The gene expression changes due to copy number alteration were also masked; as such changes are not regulated by the cellular signaling system, but are due to genomic alterations. In each tumor, we identified the genes that had expression changes and the genes that had copy number alterations, i.e., GISTIC2.0  score eqFeature selection was performed to remove genes with low Bernoulli variance because of their general lack of information. We created datasets with different numbers of features by using different variance thresholds for feature selection. We identified genes that had an expression status of 1 (or 0) in 90% (Bernoulli success probability) of tumors and removed them from further analysis due to their low variance. This resulted in a dataset with 7160 features. We repeated this process using a Bernoulli success probability of 0.95 to create a dataset with 9476 features. We also removed any genes that were highly correlated with a specific cancer type or tissue type, by removing all genes with a Pearson correlation coefficient, with respect to cancer or tissue type labels, greater than 0.85.The specific deep learning strategy used for this study is called a DBN (layer-by-layer pre-training followed by “up-down” fine-tuning) , 13–15. W) of the pre-training phase, starts with the learning of the weights of each of the stacked RBMs in a layer-by-layer fashion.In the pre-training phase, the hierarchical DBN model is treated as multiple restricted Boltzmann machines (RBMs) stacked on top of each other, such that the top layer of an RBM is used as the bottom layer of another RBM above it. Learning of the parameters (often referred to as weights θ (including a weight matrix W and offset vectors (biases) b, c) that maximize the probability of the input data  [In more detail, the pre-training phase is a deep generative model consisting of a stack of RBMs. An RBM is a 2-layered undirected probabilistic network that seeks to learn a latent representation (often of lower dimension) of the input data by optimizing the instantiation of the latent variables (latent representation) and the weights of the network in order to best allow the latent representation to regenerate the input data . The objayer, v) .1\documeP of the hidden h and visible v layers is a function of the energy function E(.) [Z is a normalization factor or partition function.The joint probability ion E(.) . Z is a In the fine-tuning phase, the DBN model is “unfolded”  in order to learn to reconstruct the input data using the weights learned during the pre-training phase. The fine-tuning phase performs a global optimization of all weights using stochastic gradient descent and the backpropagation algorithm, with the goal of minimizing the difference between the distribution of the data and the distribution formed by the model’s reconstructions of the data (cross-entropy error) , 14. DeeIn more detail, a DA is a multi-layered network composed of an encoder and decoder network . The encWe implemented a DBN  using thIn order to investigate the impact of the hyperparameters  on modeling the cancer transcriptomic data, we performed a series of model selection experiments. Model selection was performed using a modified 8-fold cross-validation. In order to speed up our model selection , while still training on a large percentage of our dataset , we only performed four folds of an 8-fold split of the data. Our strategies for deep learning model selection were guided by articles from Bengio  and HintAfter model selection, we trained the deep learning model and then computed and collected the top hidden layer (the most abstract) representations for each sample. We performed consensus clustering on the top hidden layer representations  of each tumor as calculated by our trained DBN models; as well as, on the high-dimensional input data alone. Consensus clustering was performed using the ‘ConsensusClusterPlus’  package The output of the consensus clustering was a dendrogram and an associated consensus matrix heatmap (dimensions = number of samples by number of samples), representing how often samples clustered together in repeated clustering trials. The DBN models with the lowest percentage of ambiguous clustering (PAC)  values aP-values were calculated using the log-rank test.A Kaplan-Meier plot was created using the clustering assignments from the consensus of consensus clustering for GBM samples. Kaplan-Meier plots were created using the ‘survival’ package in the R statistical computing language . P-valuewww.genecards.org and the Gene Ontology Consortium (http://geneontology.org/).Correlation studies were performed to find the differentially expressed genes or mutations that correlated with each GBM cluster. The Pearson correlation between each differentially expressed gene (input features) and GBM cluster was measured using the ‘cor’ function in R. The Pearson correlation between each mutation (TCGA GBM somatic mutation data version “2015-01-27”) and GBM cluster was also measured using the ‘cor’ function in R. Functions of example genes were obtained from This study concentrated on finding the network architecture of a DBN model that was capable of learning “optimal” representations of cancer expression data, which is a model selection task. We performed a series of model selection experiments to find the best set of hyperparameters . Approximately 1500 different sets of hyperparameters, including models with up to five hidden layers, were evaluated by cross-validation, representing approximately 1.5 months of computation time on a single Tesla k40 GPU.We started model selection by performing a random search over all hyperparameters, in which hyperparameters were randomly sampled from a range of values. We performed this search subject to some constraints, such as decreasing hidden layer size  and pre-defined maximum unit thresholds for hidden layer sizes. Based on these results, we selected a partial set of hyperparameters  that appeared to perform well over a broad range of hidden layer architectures. Using this partial set of hyperparameters, we performed an extensive grid search over hidden layer architectures (from 1 to 5 hidden layers with varying number of hidden units in each layer) and evaluated the resulting reconstruction errors Fig. . For allsingle hidden layer, i.e., a single RBM that is then fine-tuned. As expected, the reconstruction error for a single hidden layer network decreases as the size of the hidden layer increases, and does not provide much insight into choosing the size of the 1st hidden layer. A single hidden layer DBN cannot learn the hierarchical relationships that we are interested in discovering. Therefore, we explored DBNs with multiple hidden layers.Figure Figure We hypothesized that, since a DBN is an encoding machine, an optimal model should be able to encode the input data and pass it through an information bottleneck—a model with a very small number of hidden units in the top layer of the network. Such a model would require each of the layers below this bottleneck to capture as much information as possible in order to efficiently pass the bottleneck . As such, one can search for an optimal architecture by starting with a very small number of hidden units in the top layer and selecting the optimal number of hidden units layer-by-layer starting from the 1st hidden layer (closest to input data).Figure In total, these results suggest that the DBN begins to capture noise in the data when the 1st hidden layer size is greater than 1300 units. Accordingly, a 1st hidden layer size around 1300 units should provide the optimal encoding of the data when the number of hidden units in the top hidden layer (the information bottleneck) is small. We hypothesized that the 1st hidden layer captures the signals encoded by TFs in human cells, and our results suggest that 1300 hidden units most effectively captures the covariance structure  within the data at the level of abstraction of the 1st hidden layer. Interestingly, our hypothesis agrees surprisingly well with the current consensus on the number of human TFs (~1400) estimated through analyzing the human genome . These rAs previously mentioned, we searched for optimal hidden layer sizes by finding ‘elbows’ in the plot of reconstruction error vs. hidden layer size, where the reconstruction error decreases less rapidly  layer and performed consensus clustering. Figure While it is tempting to use a clustering approach to find common cancer subtypes across multiple cancer types, this approach is complicated by the fact that certain pathways exhibit tissue-specific expression, and clustering will be dominated by these tissue-specific features. This will eventually lead to the clustering of all tumor samples according to tissue type, as demonstrated in the study by Hoadley et al. . Indeed,This tissue-specific clustering occurred despite our best attempts to remove all tissue-specific genes by representing a tumor using only genes with extremely high variance  to train DBNs, and then perform consensus clustering (results not shown). Using these high variance genes reduced our number of features (genes) to 2674, with each of these genes being differentially expressed in more than 2250 tumor samples. The genes in this 2674 set cannot be tissue-specific because most cancer/tissue types in our data set have only ~500 tumor samples or less . These results indicate that there are tissue-specific pathway perturbation patterns that lead to a tissue-specific covariance structure of DEGs, which were captured by the DBN, and in turn recognized by consensus clustering. These results illustrate the limitations of using gene-expression-related features  to study disease mechanisms shared by tumors across different tissue types [Although it is difficult to search for common disease mechanisms across multiple cancer types due to the aforementioned limitations, we hypothesized that, within a given tissue type, clustering using DBN-learned representations may reveal distinct disease mechanisms. Since the survival data for glioblastoma multiforme (GBM) patients from TCGA was relatively more complete than other cancer types in our data set , we studied GBM in more detail. Previously, Verhaak et al.  selectedWe then set out to investigate whether an unbiased representation learned by a DBN would reveal subtypes (clusters) of GBM. We trained six DBN models with different architectures and hyperparameters  are involved in nervous system development. BLM (blue cluster) is involved in DNA binding and regulation of cell proliferation. VAV3 (light blue cluster) is involved in GTPase activator activity. SPAG1 (green cluster) is known to be involved in epilepsy and pancreatic cancer. PLVAP (purple cluster) may function in microvascular permeability.We investigated the six GBM clusters further using correlation analysis to find DEGs and mutations that were associated with each cluster. Figure IDH1 is the mutation with the strongest correlation with the red cluster, which includes all tumors belonging to G-CIMP subtype of GBM [IDH1 lead to significant changes in DNA methylation, which underlie the G-CIMP. Similarly, NF1 mutations are strongly associated with the black cluster  and are known to be frequently mutated in the mesenchymal subtype [Figure e of GBM . This fi subtype .The above results reveal connections between genomic alterations and DEGs specifically associated with each subtype, which provide information about the disease mechanisms for each subtype. It is reasonable to assume that the genomic alterations associated with a cluster likely perturb pathways underlying the subtype, and hidden units in our DBN models could potentially represent the states of these pathways. Any aberration in these pathways causes a change in the expression of the DEGs associate with that cluster. Studying the potential causal relationships between mutation events and the changing states of hidden units may provide information about how mutations affect signaling pathways in cancer cells.IDH1. This also agrees with current knowledge. Further investigation may reveal disease mechanisms underlying the different GBM clusters. What role do these genes/mutations have in GBM? What role do they play in survival?In this study, we showed that an unsupervised DBN can be used to find meaningful low-dimensional representations of cancer gene expression data. More specifically, first, we designed a rigorous model selection scheme, which enabled us to determine the optimal number of hidden units in the 1st and 3rd hidden layers of our model. We hypothesized that the 1st hidden layer likely represented the TFs utilized by cancer cells and our results correlate with current knowledge of the number of TFs. Second, we showed that consensus hierarchical clustering of GBM tumors using the unbiased representations  hidden layer units) revealed more robust clustering results than clustering based on the raw gene expression data. Third, we showed that clinically relevant information was encoded in the representations learned by our DBN. This was demonstrated through the discovery of a subtyping of GBM with differential prognosis, which previously was not discovered by TCGA. Our methods identified a subtype of GBM enriched with the G-CIMP phenotype without using DNA methylation data, and our analysis can partially attribute this subtype to the mutation of This study represents a novel application of the deep learning algorithm developed by Hinton and Salakhutdinov  in the c