The biologic heterogeneity of soft tissue sarcomas (STS), even within histological subtypes, complicates treatment. In earlier studies, gene expression patterns that distinguish two subsets of clear cell renal carcinoma (RCC), serous ovarian carcinoma (OVCA), and aggressive fibromatosis (AF) were used to separate 73 STS into two or four groups with different probabilities of developing metastatic disease (PrMet). This study was designed to confirm our earlier observations in a larger independent data set.We utilized these gene sets, hierarchical clustering (HC), and Kaplan-Meier analysis, to examine 309 STS, using Affymetrix chip expression profiling.HC using the combined AF-, RCC-, and OVCA-gene sets identified subsets of the STS samples. Analysis revealed differences in PrMet between the clusters defined by the first branch point of the clustering dendrogram (p = 0.048), and also among the four different clusters defined by the second branch points (p < 0.0001). Analysis also revealed differences in PrMet between the leiomyosarcomas (LMS), dedifferentiated liposarcomas (LipoD), and undifferentiated pleomorphic sarcomas (UPS) (p = 0.0004). HC of both the LipoD and UPS sample sets divided the samples into two groups with different PrMet . HC of the UPS samples also showed four groups with different PrMet (p = 0.0007). HC found no subgroups of the LMS samples.These data confirm our earlier studies, and suggest that this approach may allow the identification of more than two subsets of STS, each with distinct clinical behavior, and may be useful to stratify STS in clinical trials and in patient management. Adult STS can be grouped into two broad categories. One category has simple genomic profiles and specific cytogenetic changes, such as a point mutation or translocation . The second category is comprised of tumors with more complex genomic patterns characterized by multiple gains and losses, including many leiomyosarcomas (LMS), pleomorphic liposarcomas, and undifferentiated pleomorphic sarcomas (UPS)  -5. Althosue.pdf) -10.While some differences in behavior generally correlate with histologic diagnosis and grade, significant heterogeneity of tumor biology exists even within histologic subsets. The heterogeneity of biological behavior complicates clinical care of patients with STS. One clinically important variable is whether a tumor will metastasize or not.Gene expression patterns may be useful in the subclassification of STS, both for diagnosis and for prediction of clinical behavior ,7-16. InIn earlier studies, we described gene expression profiles that identified two general subgroups in a set of clear cell renal cell carcinomas (ccRCC-gene set), a set of ovarian carcinomas (OVCA-gene set), and a set of aggressive fibromatosis samples (AF-gene set) -22. We rIn this study we confirmed the results from our earlier studies with an independent data set. We utilized our three gene sets to examine a larger group of 309 non-translocation associated STS using Affymetrix chip based expression profiling, in data sets in which all probes utilized in our earlier studies were represented. These gene sets successfully separated the STS samples into subsets with different probabilities of developing metastases.http://www.conticabse.org), and were treated in one of 11 centers. In each case, the pathology was reviewed by the pathologist subgroup and classified according to the 2002 WHO classification. For the current study, STS with no recurrent chromosomal translocation were selected. According to French law at the time of the study, experiments were performed in agreement with the Bioethics Law 2004 800 and the Ethics Charter from the National Institute of Cancer; all subjects signed a non-opposition statement for research use of their sample.Three hundred nine STS samples were obtained from patients who had surgical resection of STS as previously described . The samTotal RNA was extracted from each frozen tumor sample, and analyzed on Human Genome U133 Plus 2.0 array (Affymetrix) as previously described .Three different gene sets with limited overlap, that identified two general subgroups of a series of ccRCC samples (ccRCC-gene set), two subgroups of AF (AF-gene set), and distinguished borderline from invasive serous OVCA (OVCA-gene set) have been previously described, including 138, 161, and 173 known genes respectively ,21,22. Fhttp://rana.lbl.gov). The geometric means of the expression intensities of the relevant gene fragments were computed using Genedata Expressionist , and the ratio was reported as the fold change (up or down). Confidence intervals and p-values on the fold change were calculated using a two-sided Welch modified two-sample t-test. Differences were considered significant when p</=0.05.The ccRCC-, AF-, and OVCA- gene sets were used individually or combined, to cluster the 309 primary high-grade STS samples. For clustering, genes were median centered, normalized, and then clustered by complete hierarchical clustering using uncentered correlation with the Eisen clustering software and viewed using the TreeView software  method to calculate the metastasis-free survival probabilities, and the cumulative probabilities of metastasis  at critical time points . The p-values were calculated by using the Wilcoxon-Gehan test for comparing different groups. The Tukey-Kramer method was used to adjust for multiple pair-wise comparisons when there were more than two groups. P-values </=0.05 were considered statistically significant. Analyses were performed in SAS® 9.3 .Hierarchical clustering of the 309 STS samples was first performed with the set of 533 probes of the combined gene set  Figure  top. KapKaplan-Meier analysis was also performed using the four sample sets defined by the first and second branch points in the hierarchical clustering analysis . Hierarchical clustering of the 62 LipoD samples with the pooled probe set divided the samples into two groups (A and B) with different probabilities of metastasis  with different probabilities of metastasis . Similarly, no difference was observed when a similar analysis was performed using the 27 miscellaneous samples in the OTH subgroup (not shown). Factors possibly contributing to the inability to identify groups with different outcomes in the LMS and OTH sets could include the small number of samples in these two subgroups, and the relative mix of “low” and “high” metastatic samples in these two sample sets.We also analyzed the STS samples with the individual AF-, OVCA-, and RCC-probe sets. Analysis of the UPS subset with the RCC-gene set identified differences in time to metastasis when the UPS samples were analyzed as two groups (p = 0.0002) or four groups (p < 0.0007). Although when analyzed as four subgroups, statistically significant differences in time to metastasis were observed  stage, no significant difference in the probability of developing metastases was seen between stage III (n = 17) vs IIA (n = 2), III vs IIB (n = 26), or IIA vs IIB, though the difference between stage III and IIA trended toward significance at p = 0.15 by Tukey-Kramer, and the rank order was as expected. When the UPS samples were analyzed by stage , the three stages also did not differ in risk of metastases, though the difference between stage IIA and III and that between stage IIB and III also trended to significance, both at p = 0.07, and the rank order was again as expected. The lack of good correlation between stage and metastases seen here may reflect the limited number of cases examined in the different stages, and the fact that most cases were either IIA, IIB, or III.For the UPS samples, AJC stage was significantly correlated with both the two groups and four groups defined by clustering Figure , top.We next examined the most differentially expressed genes of the total detected in the Affymetrix U133 Plus chip set, in the first two subsets (groups A and B) of LipoD and UPS  was SCARA5, scavenger receptor class A, member 5, a gene that may act as a tumor suppressor in some models . Other gSeveral genes were upregulated in both UPS-B and LipoD-B (the poor prognosis groups) relative to UPS-A and LipoD-A, respectively  of non-translocation associated soft tissue sarcoma (STS) were separated into subsets with different probability of metastasis using gene sets derived from our earlier studies with ccRCC, AF, and OVCA ,19-22. EBoth the current and CINSARC gene sets identified subsets of the UPS samples, when analyzed as a separate group, that differed in time to metastasis. Interestingly, the current study also detected subsets of LipoD, when analyzed as a separate group, that differed in probability of metastasis, but did not detect such subsets in the LMS or OTH subgroups. These differences were apparent even after excluding grade 1 tumors from the analysis. In contrast, the CINSARC gene set identified subsets of LMS, when analyzed as a separate group, that differed in probability of metastasis, but did not detect such subsets in the LipoD subgroup . Copy nuDifferences were observed in the global gene expression patterns of the subsets that were identified in this study. For example, the subset of the LipoD samples with the lower probability of metastasis (LipoD-A) tended to over-express a number of genes expressed in adipocytes such as ADIPOQ, PLIN1, FABP4, LPL, PPARG, and THRSP. These samples might be viewed as less de-differentiated. Adiponectin, encoded by the ADIPOQ gene, is a hormone produced in adipose tissue that regulates several metabolic processes including fatty acid catabolism. It is strongly expressed in preadipocytes differentiating to adipocytes ,26. PeriOther genes that were over-expressed in the LipoD-A subset of samples included ALDH1, ADH1B, and NTRK2. Alcohol dehydrogenase 1B (ADH1B) metabolizes a variety of substrates. Aldehyde dehydrogenase 1 (ALDH1) oxidizes aldehydes to carboxylic acids and also metabolizes a variety of substrates. ALDH1 is a member of a superfamily of genes and has been reported to be a marker of normal and malignant mammary stem cells, and a predictor of poor clinical outcome in breast cancer . NTRK2 fIn contrast, the LipoD-B subset (with the higher risk of developing metastatic disease) over-expressed a number of genes involved in regulating cell growth and cell division. Among the genes over-expressed in the LipoD-B subset compared to the LipoD-A subset, RUNX2 is a transcription factor that has a Runt DNA-binding domain and is active in osteoblast differentiation. CDC20 (cell-division cycle protein 20), cyclin B1 (CCNB1), cyclin A2 (CCNA2), cyclin E2 (CCNE2), cyclin-dependent kinase-1 , cyclin-dependent kinases regulatory subunit 2 (CKS2), and cyclin-dependent kinase inhibitor 3 (CDKN3), aurora A kinase (AURKA), and aurora B kinase (AURKB) are involved in regulating cell division. TOP2A  is important in DNA replication and is felt to be a target for doxorubicin and etoposide. In some models, higher levels of TOP2A correlated with more resistance to doxorubicin . PodoplaExtracellular matrix genes were also differentially expressed in the two LipoD subsets, with FN1 and CTHRC1 over-expressed in LipoD-B. Differential expression of extracellular matrix proteins in different subsets of malignancies appears to be a common theme. The recognition of a desmoplastic response to cancer has long been recognized -36, and Genes over-expressed in UPS-A (the good prognosis group) compared to UPS-B included: SCARA5, TNXB, ALDH1A3, ADH1B, MFAP5, AKAP12, DEFB1, TGFBR3, GAS7, IL17D, IL33, and CD34, as well as the growth factors NEGR1, PDGFD, FGF18, NTRK2, and IGFBP6, a regulator of IGF1. SCARA5 was expressed 34-fold higher in the UPS-A subgroup, and may function as a tumor suppressor gene. Over-expression of SCARA5 suppressed some malignant behaviors in hepatoma cells, and SCARA5 knockdown was associated with activation of MMP9 . TenasciIn contrast, genes over-expressed in UPS-B (the bad prognosis group) compared to UPS-A (the good prognosis group) included: STEAP1, CA12, AIM2, TNC, HS3ST3A1, RUNX2, POSTN, ADAM12, PLAUR, NRP2, RGS1, extracellular matrix proteins , matrix metalloproteinases , FAP (a marker of fibroblast activation), and growth regulator proteins. Six-transmembrane epithelial antigen of the prostate (STEAP1) is an antigen detected in prostate cells, but up-regulated in multiple cancer cell lines, and is a metalloreductase. Recent studies on the expression patterns of STEAP1 and STEAP2 have suggested that they may be markers of mesenchymal stem cells . High STOf the genes upregulated in both of the poor-prognosis groups (UPS-B and LipoD-B) compared with UPS-A and LipoD-A, respectively, only RRM2 is present in the CINSARC gene set; five genes, RRM2, LEF1, KDELR3, FN1, and CTHRC1, were present in the RCC-gene set; two genes, MICAL2 and KDELR3, were present in the AF-gene set; and one gene, RRM2, was present in the OVCA-gene set. The RRM2 gene encodes ribonucleoside-diphosphate reductase subunit M2, one of two subunits for ribonucleotide reductase. Ribonucleotide reductase catalyzes the formation of deoxyribonucleotides from ribonucleotides, a rate-limiting step in DNA synthesis, and is regulated in a cell-cycle dependent manner. Over-expression of RRM2 has been reported to enhance the metastatic potential of some tumors . LEF1 isAlthough in the current study we focused on the identification of at most four subsets of each STS set with different biological behavior (manifest as metastatic propensity), in some cases we observed statistically significant differences in time to metastasis when the samples were analyzed as more than four subsets. The ability to detect multiple subgroups is strongly influenced by sample number and the distribution of samples among the various groups. Perhaps further heterogeneity that is clinically useful may be identified in larger sample sets. The ability to better predict the long-term outcome following surgery will greatly improve the treatment of patients with sarcomas, and gene expression profiles may provide a clinically meaningful approach to this problem, restricting the use of adjuvant modalities of therapy and reducing heterogeneity among groups in clinical trials of new drugs.The biologic heterogeneity of soft tissue sarcomas (STS), even within histological subtypes, complicates treatment. This study used hierarchical clustering with a gene set derived from studies of renal carcinoma, ovarian carcinoma, and aggressive fibromatosis, to examine 309 STS using Affymetrix chip expression profiling. The analysis separated STS into two groups, and also four groups, with different probabilities of developing metastatic disease (PrMet). Analysis also revealed differences in PrMet between leiomyosarcomas (LMS), dedifferentiated-liposarcomas (LipoD), and undifferentiated pleomorphic sarcomas (UPS). HC of both the LipoD and UPS samples divided the samples into two groups with different PrMet. HC of the UPS samples also showed four groups with different PrMet. These data confirm our earlier studies in with a different sample set, and suggest that this approach may allow the identification of more than two subsets of STS, each with distinct clinical behavior, and may be useful to stratify STS in clinical trials and in patient management.STS: Soft tissue sarcomas; RCC: Renal cell carcinoma; OVCA: Ovarian cancer; AF: Aggressive fibromatosis; HC: Hierarchical clustering; PrMet: Probability of developing metastatic disease; LMS: Leiomyosarcoma; LipoD: Dedifferentiated liposarcoma; UPS: Undifferentiated pleomorphic sarcoma.The authors declare that they have no competing interests.KMS participated in study design, data analysis, and helped draft the manuscript. APNS participated in study design, data analysis, and helped draft the manuscript. WX participated in data analysis, and helped draft the manuscript. XL participated in data analysis, and helped draft the manuscript. PL participated in data acquisition and analysis, and helped draft the manuscript. JMC participated in data acquisition and analysis, and helped draft the manuscript. FC participated in data acquisition and analysis, and helped draft the manuscript. All authors read and approved the final manuscript.Probability (95% CI) of no metastasis as a function of time.Click here for fileGenes over-expressed in LipoD-A vs LipoD-B.Click here for fileGenes over-expressed in UPS-A vs UPS-B.Click here for fileGenes over-expressed in both LipoD-B vs LipoD-A and UPS-B vs UPS-A.Click here for file


Pathway expression is multivariate in nature. Thus, from a statistical perspective, to detect differentially expressed pathways between two conditions, methods for inferring differences between mean vectors need to be applied. Maximum mean discrepancy (MMD) is a statistical test to determine whether two samples are from the same distribution, its implementation being greatly simplified using the kernel method.An MMD-based test successfully detected the differential expression between two conditions, specifically the expression of a set of genes involved in certain fatty acid metabolic pathways. Furthermore, we exploited the ability of the kernel method to integrate data and successfully added hepatic fatty acid levels to the test procedure.MMD is a non-parametric test that acquires several advantages when combined with the kernelization of data: 1) the number of variables can be greater than the sample size; 2) omics data can be integrated; 3) it can be applied not only to vectors, but to strings, sequences and other common structured data types arising in molecular biology. A challenging topic for the bioinformatics community is how to combine data from multiple sources to increase biological knowledge. Integrating data from various different sources is not simply a matter of summing the results of each separate source; rather, it requires the analysis at the same time of all variables from various sources –3.Nowadays, there are many methods to integrating heterogeneous data but kernel-based methods are usually the most powerful , 5. KernLet us start by recalling the main ideas of kernel-based approaches.k on k=k,i=1,…,m.Given a sample space Thus, a kernel can be interpreted as a similarity measure of the samples and allow us to identify each ϕ(x),ϕ(y)〉=〈k,k〉=k. We can turn our feature space into a Hilbert space reproducing kernel Hilbert space (RKHS), induced by the kernel function k. This remarkable property has important consequences. Indeed, “geometric” intuition can be used to build kernel-based methods, by drawing inspiration from classical statistical methods working in finite dimensional Euclidean spaces. Popular examples of kernel-based methods are kernel principal component analysis (KPCA), kernel ridge regression (KRR), and support vector machines (SVMs) [which is an element of a dot product vector space, from now on referred to as a feature space. It consists of all functions s (SVMs) .A natural question to raise is how a probability distribution X taking values in X is the unique element of the RKHS Consider a random variable In statistics, a central concern of the data integration outlined above is often referred to as the two-sample or the homogeneity problem. In this study, we explore a test statistic, known as the maximum mean discrepancy (MMD) –11, to tX= and Y= be samples composed of independent and identically distributed observations drawn from Maximum mean discrepancy statistic was designed with the aim to determine a function such that its expectation differs when observations come from two different probability distributions. The underlying premise is that if we compute this statistic on samples drawn from different distributions it will measure how these distributions are likely to differ. This consideration leads to the following statistic. Let X and Y.By choosing mentclasspt{minimaWhen m 2.2 in  ensures 2 was introduced in [p-value of the test can be addressed by sampling. From the aggregated data Z={X,Y}, we draw randomly without replacement to obtain two new m-samples {X∗,Y∗}, and compute the test statistic t times, a set of test statistics under the null hypothesis is obtained: A two-sample test based on the asymptotic distribution of an unbiased estimate of MMDduced in . The estr denotes the position of Then, we add the original statistic N pooled multivariate data in p is the number of variables of the multivariate data, and edges linking all pairs. The edge weight can be estimate by the Euclidean (or any other) distances between the nodes (pairs of multivariate data). Thus, similar nodes have similar distances. The test is based on the construction of the MST of the pooled multivariate data, then it deletes all edges for which the defining nodes originate from different multivariate samples and, finally, it counts the number of disjoint subtrees (R). For large sample sizes, the permutation distribution of the standardized number of subtrees We compare the performance of the MMD test with those of other methods, such as the Hotelling test . This issubtrees . The mulTo illustrate this procedure, we analyze data from a study in the fields of metabolomics and genomics. Specifically, the datasets are drawn from a nutrigenomic study in the mouse . Forty mx−x′||2. Basically, any value in between these two bounds will produce a good hyper-parameter estimation.For the complete analysis we used a Gaussian kernel and the hyper-parameter was determined by the sigest function of the kernlab R package . The estWe use the GSAR R package  to impleα, TP β, mHMGCoAS, Cyp4a10, Cyp4a14, ACBP,L-FABP, ACOTH and PLTP. Using a permutation procedure based on 2499 repetitions, we obtain a significant p-value . Kernel methods allow us to avoid this issue in contrast to the classical Hotelling test that does not.p-value when the pathway is represented only by the gene expressions. In contrast, when the fatty acid levels are added, the p-value is significant. The multivariate Kolmogorov-Smirnov statistic and Wald-Wolfowitz runs test have similar p-values  the number of variables can be greater than the sample size; 2) omics data can usefully be integrated; 3) it can be applied not only to vectors, but to strings, sequences and other common structured data types arising in molecular biology. Our results indicate that the kernel MMD can be used to identify differentially expressed pathways; however, further studies with several sets of pathways are needed in order to assess its overall performance. This study suggests that kernel MMD is a useful approach to the analysis of pathway differential expression, since it takes into account all the genes involved in the pathway and, moreover, offers the possibility of integrating several data types.


The assessment of non-genotoxic hepatocarcinogens (NGHCs) is currently relying on two-year rodent bioassays. Toxicogenomics biomarkers provide a potential alternative method for the prioritization of NGHCs that could be useful for risk assessment. However, previous studies using inconsistently classified chemicals as the training set and a single microarray dataset concluded no consensus biomarkers. In this study, 4 consensus biomarkers of A2m, Ca3, Cxcl1, and Cyp8b1 were identified from four large-scale microarray datasets of the one-day single maximum tolerated dose and a large set of chemicals without inconsistent classifications. Machine learning techniques were subsequently applied to develop prediction models for NGHCs. The final bagging decision tree models were constructed with an average AUC performance of 0.803 for an independent test. A set of 16 chemicals with controversial classifications were reclassified according to the consensus biomarkers. The developed prediction models and identified consensus biomarkers are expected to be potential alternative methods for prioritization of NGHCs for further experimental validation. Carcinogenicity is one of the most unwanted side effects during drug development1Toxicogenomics (TGx) as a promising tool in risk assessment147891011et al. applied statistical methods to identify 23 biomarkers for predicting NGHCs based on a training set of 23 NGHCs and 49 NHCs and 5-day gene expression data from DrugMatrixet al. utilized support vector machine algorithms to conclude 9 biomarkers based on a training set of 6 NGHCs and 54 NHCs and 28-day gene expression data from TG-GATEset al. identified 106 biomarkers from six NGHC compounds based on 28-day gene expression data from TG-GATEset al. present two novel approaches employed to capture 5 specific biomarkers both statistical and machine learning-based methods based on a training set of 2 Genotoxic hepatocarcinogens, 9 NGHCs and 11 NHCs and 14-day gene expression data from TG-GATEsThe utilization of different NGHCs as training sets, algorithms for biomarker identification and experimental parameters such as the duration of treatment and microarray platforms could result in the identification of different biomarkers. For example, Fielden This study proposed to identify and analyze consensus biomarkers based on a largest set of 50 NGHCs and 224 non-hepatocarcinogens (NHCs) without controversial classification of NGHC or NHC collected from literatures1478910121718To collect a largest set of NGHCs and NHCs for the identification of consensus biomarkers for identifying NGHCs, chemicals with classification information were collected from several studies associated with nongenotoxic hepatocarcinogenicity1478910121718ftp://anonftp.niehs.nih.gov/drugmatrix/), GSE8858 (ftp://ftp.ncbi.nlm.nih.gov/geo/series/GSE8nnn/GSE8858/) and TG-GATEs (ftp://ftp.dbcls.jp/archive/open-tggates/) as shown in Four large-scale gene expression profiles and metadata named DMA, DMC, GSE8858, and TG-GATEs were downloaded from websites of DrugMatrix (t-test (p < 0.05) and 1.5-fold change were applied to identify DEGs based on original expression values. The 1-day gene expression profiles from the treatment of chemicals using the single maximum tolerated dose or high dose were utilized for analysis due to their availability on all four datasets. Subsequently, the overlapped DEGs from four datasets were identified as consensus biomarkers.To identify consensus biomarkers from multiple microarray datasets, differentially expressed genes (DEGs) were firstly analyzed for each of the four microarray datasets. The kNN). The implementation of the machine learning algorithms was based on WEKA machine learning packageMachine learning methods have been widely used for constructing classifiers/hypothesis that can explain complex relationships in the dataThe decision tree algorithm J48, also known as C4.5 algorithm2425In addition to the single decision tree classifier, three ensemble learning algorithms of boosting, bagging and RandomForest were implemented to leverage the power of multiple classifiers that have been shown to improve the overall prediction accuracy and reduce overfitting problems27The bagging algorithm based on the combination of bootstrap sampling and aggregation methods has been shown to effectively reduce variance and avoid overfittingRandomForest3233343637kNN) classifiers. Naïve Bayes method is a simple yet powerful classification method based on the Bayes’ theoremH and an evidence E, the posterior probability can be calculated based on a prior probability and a likelihood using the following equation: In addition to the decision tree and ensemble methods, two classifiers were also evaluated for their prediction performance of the selected markers including Naïve Bayes and K-nearest neighbor  method was utilized to evaluate the performance of selected markers on the training dataset. For each iteration, a sample is selected to validate the model constructed by using the remaining samples. After all samples are evaluated, the LOOCV performance is obtained by calculating the overall AUC performance.ftp://157.98.192.110/ntp-cebs/datatype/microarray/J&J/) and log2-transformed.Johnson and Johnson (JNJ) datasetPrevious studies reported several sets of NGHC biomarkers based on only a single dataset with different chemical lists, experimental designs, and microarray platforms14789101217181781314This study aimed to identify consensus biomarkers based on multiple gene expression datasets of a relatively large chemical list without inconsistent classifications from previous studies for predicting NGHCs. A large chemical list was firstly collected from literatures consisting of 50 NGHCs and 224 NHCs . Subsequp < 0.05) and 1.5-fold changes on original expression values between NGHCs and NHCs. The numbers of DEGs are 142, 155, 231 and 1280 for DMA, DMC, GSE8858 and TG-GATEs, respectively. Finally, 4 common DEGs of A2m, Ca3, Cxcl1 and Cyp8b1 in all four microarray datasets were identified as consensus biomarkers. A Venn diagram showing the overlap between datasets was generated by jvennAmong the four microarray datasets, the common exposure styles are 1-day single dose and 3-day repeated doses using a highest tolerated dose. This study focused on the data of 1-day single dose because of its potential applications. To identify consensus biomarkers, only gene expression profiles derived from the exposure style were utilized. For each of the four microarray datasets, differentially expressed genes (DEGs) were firstly identified using t-test . Due to the platform variations are shown in To compare with previously published biomarkers, bagging decision tree algorithm were also applied to evaluate their LOOCV performance. Two biomarker sets consisting of 23et al. has proposed a resampling statistics for evaluating the robustness of biomarkersRecently, Li To further evaluate the prediction ability of the identified consensus biomarkers, a JNJ dataset consisting of 1-day single maximum dose exposures of 9 NGHCs and 45 NHCsThere are four consensus biomarkers identified in this study including Ca3, Cyp8b1, A2m, and Cxcl1. The possible roles of the biomarkers for NGHC were discussed as follows. Carbonic anhydrase III (Ca3) is a cytosolic protein found in skeletal muscle, liver, and adipose tissue of mammals1248Cyp8b1  is a key enzyme involved in bile acid biosynthetic pathway which converts chenodeoxycholic acid to cholic acid2-macroglobulin) is a member of the thiol ester protein family whose mRNA levels in rat liver were found to be linked to acute inflammation5556A2m and Cxcl1 are two novel biomarkers identified in this study that have not been directly associated with non-genotoxic hepatocarcinogenesis. A2m .Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.


Motivation: Gene-expression data obtained from high throughput technologies are subject to various sources of noise and accordingly the raw data are pre-processed before formally analyzed. Normalization of the data is a key pre-processing step, since it removes systematic variations across arrays. There are numerous normalization methods available in the literature. Based on our experience, in the context of oscillatory systems, such as cell-cycle, circadian clock, etc., the choice of the normalization method may substantially impact the determination of a gene to be rhythmic. Thus rhythmicity of a gene can purely be an artifact of how the data were normalized. Since the determination of rhythmic genes is an important component of modern toxicological and pharmacological studies, it is important to determine truly rhythmic genes that are robust to the choice of a normalization method.Results: In this paper we introduce a rhythmicity measure and a bootstrap methodology to detect rhythmic genes in an oscillatory system. Although the proposed methodology can be used for any high-throughput gene expression data, in this paper we illustrate the proposed methodology using several publicly available circadian clock microarray gene-expression datasets. We demonstrate that the choice of normalization method has very little effect on the proposed methodology. Specifically, for any pair of normalization methods considered in this paper, the resulting values of the rhythmicity measure are highly correlated. Thus it suggests that the proposed measure is robust to the choice of a normalization method. Consequently, the rhythmicity of a gene is potentially not a mere artifact of the normalization method used. Lastly, as demonstrated in the paper, the proposed bootstrap methodology can also be used for simulating data for genes participating in an oscillatory system using a reference dataset.Availability: A user friendly code implemented in R language can be downloaded from http://www.eio.uva.es/~miguel/robustdetectionprocedure.html Quantile  Loess   Loess, and Invariant Set normalization methods. For illustration purposes, in the top panel of Figure Serpina3k using Quantile (left panel) and Constant (right panel) normalization procedures. In the bottom panel of Figure Maml1 using Loess (left panel) and Invariant Set (right panel) normalization procedures. As one can see, the time-course profiles of these genes are markedly different, depending upon which normalization procedure was used. Furthermore, if rhythmicity detection algorithm ORIOS is used then Serpina3k and Maml1 are rhythmic genes if Quantile and Loess normalizations are used, respectively. But they cease to be rhythmic genes if Constant and Invariant Set normalization procedures are used. Similar conclusions are drawn if other rhythmicity detection algorithms, such as JTK and RAIN are used on these data. Such results in a genome-wide analysis can be very confusing and difficult to interpret.We begin this section by considering time-course data on two genes, namely, n and a rhythmicity detection algorithm a, the identification of rhythmic genes is based on the Benjamini-Hochberg adjusted p-values . For each gene g = 1, …, G we define the standard measure of gene rhythmicity associated to gene g, as follows:M = , whose components take values between 0 and 1. Closer 0 indicates potentially non-rhythmic gene and closer 1 indicates potentially rhythmic gene.Given a normalization method MSerpina3k = 0.996, MSerpina3k  = 0.639, MMaml1 = 0.992, and MMaml1 = 0.668. Thus implying Serpina3k is potentially rhythmic under Quantile normalization but not under Constant and similarly, Maml1 potentially rhythmic under Loess normalization but not likely under Invariant Set. This observation that normalization method n may impact the rhythmicity of a gene is not limited to the above genes but is rather a common feature of long-series time-course data as noted in Table a may also impact on determining if a gene is rhythmic or not. In modern pharmacological and toxicological studies , a modification of M which is more robust with respect to n, the normalization method, than M is. The proposed bootstrap methodology also provides us a tool to simulate time-course expression data for genes participating in oscillatory systems such as the circadian clock using a reference dataset.For the plots in Figure R denote the tri-dimensional array of raw intensities obtained from a reference high-throughput data of an oscillatory system, such as the circadian clock. Data in R are expressed at probe level, where g on probe p at time point (array) t, where g = 1, …, G, p = 1, …, P, and t = 1, …, T. Let X be the tri-dimensional array derived from R after background correction. After normalization and summarization steps, a matrix of gene-expression values is finally obtained as the output of the pre-processing * it is important to recognize that we are bootstrapping the residuals while centering the bootstrap data (log2(X)) at the true observed signal. Thus the mean signal over the bootstrap samples retains the original expression and hence there is no loss of information in the mean signal through bootstrapping. If the expression data are count data, as in the case of RNA-seq, the observed counts may be transformed using a suitable variance stabilization transformation before appealing to the above model. It is common to model RNA-seq data either using standard Poisson or Poisson with extra-variability in the Poisson parameter by using a gamma prior which leads to modeling the RNA-seq data using a negative binomial distribution. In both cases the variance stabilizing transformation is known from the literature, which are either square transformation (for Poisson) or arc sinh square transformation , see Guan y et al. , the medy et al. . This aly et al. , it allosee Guan .Copg and Bgee . Let θ = 𝔼) be the parameter of interest and bth bootstrap sample using Equation (2), b = 1, 2, …, B, let θ. Let RMSE), it reduces the effect of the normalization method used.For a rhythmicity detection algorithm http://www.ncbi.nlm.nih.gov/geo/) of Hughes et al.  ≥ 0.99 as the criterion to declare a gene to be rhythmic  Loess, 3.-Contrast, 4.-Constant, 5.-Invariant Set, 6.-Qspline, 7.-VSN). The number of rhythmic genes identified varies vastly among the normalization methods within each rhythmicity detection algorithm  was performed. MCA is an extension of correspondence analysis which allows one to analyze the pattern of relationships among several categorical variables , we plotted 3 × 8 × 2 categories denoted by a_n_r. Then, we averaged the expression values of those genes that are declared as rhythmic (or non-rhythmic) under all normalizations strategies, i.e., those with r = 1 (or r = 0) for all strategies under a given algorithm, and overlaid these averaged profiles on the plot. For algorithm a, the averaged profile of rhythmic genes is denoted by a_Av_1 and the averaged profile of non-rhythmic one is denoted by a_Av_0. Furthermore, we also overlaid on this plot six figures G1, G2, …, G6  is a pattern of all probe sets that are declared as rhythmic by all normalization methods and all three algorithms. On the other hand, G2 (Quasi Cyclic) is a pattern of all probe sets that are declared as rhythmic by all normalization methods using ORIOS but not rhythmic under all normalizations methods when using JTK or RAIN. Since genes declared as rhythmic by JTK algorithm are also declared as rhythmic by RAIN algorithm, therefore we are describing only six patterns G1, G2, …, G6 and not eight patterns as one might expect.MCA is a dimension reduction procedure that can be used to represent distances among high dimensional vectors in a low-dimensional space, such as 2-dimensional plane. Using the MCA plots, one typically tries to interpret what each axis represents and evaluates relationships among the categories of different variables based on the distance among their representations on the graph. The MCA plot based on the first two dimensions, which explain ~54% of the total variation in the data, is provided in Figure a_n_1 appear on the right hand side and almost all a_n_0 appear on the left hand side of the graph. Using G1, G2, …, G6, we see that Dim1 separates rhythmicity (Cyclic-shaped patterns) against non-rhythmicity (Flat-shaped patterns). Furthermore, it is interesting to note that rhythmic-shaped patterns  identified by ORIOS are located in the upper portion of the first quadrant of the MCA plot and the third quadrant exclusively consists of non-rhythmic patterns identified by ORIOS. Thus the first and the third quadrants of MCA plot appear to distinguish ORIOS from the others. The vertical axis (Dim2) may be interpreted as the axis drawing distinctions between ORIOS and RAIN algorithms. Lastly, it is clear from the MCA plot that ORIOS normalization methods are less separated than JTK or RAIN, i.e., rhythmic (and non-rhythmic) groups are more compact when using ORIOS, which is one more reason, in addition to the results provided in Larriba et al.  is generally robust to the normalization methods, we computed the Spearman and Pearson correlation coefficients between MRobust and MRobust, for all pairs of normalization methods ni, nj, i ≠ j and compared the correlations with those corresponding to the standard measure M. In addition to Spearman and Pearson correlation coefficient, we also computed the percent of concordance of rhythmic and non-rhythmic genes across all normalization methods using standard measure M and the proposed robust measure MRobust. Due to space reasons, in the main paper we only present the results for ORIOS, i.e., a = ORIOS, but the results corresponding to JTK and RAIN are provided in the supporting materials.To show that our proposed rhythmicity measure 6, we limited to only those probe sets that were considered to be rhythmic by the criterion Mg ≥ 0.99 for at least one normalization method n. Thus we limited to 15,369 probe sets out of 45,101. The left hand panels of Figures 6 correspond to M, whereas the right hand panels correspond to MRobust. From these figures it is clear that both correlation and the concordance increase substantially for every pair of normalization methods from the left panel to the right panel. To illustrate this fact, observe that the Spearman correlation between M and M is 0.65  and MRobust is 0.95 , Mg] (left panel) and Constant and Invariant Set normalization methods may be the least preferred normalization methods as the robust measure corresponding to these methods seem to be least correlated with others. Similar dramatic increases are also seen for JTK and RAIN as described in the figures in the online Supplementary Material  increases from 0.4 for the standard measure to 0.87 for the robust measure, while the Pearson coefficient increases from 0.21 to 0.82 and the concordance percentage goes from 62.56 to 98.67%. For the NIH3T3 cell lines, the same measures increase from 0.38 to 0.82 for the Spearman rank correlation, from 0.23 to 0.75 for the Pearson correlation and from 62.01 to 99.06% for the concordance percentage.For the dataset corresponding to the mouse pituitary data  and the NIH3T3 mouse cell line data  we obtain similar increases. For example, for the mouse pituitary, the Spearman rank correlation between Qspline and VSN nearly triples from 0.31 to 0.91 after applying our robust methodology. All statistical decision rules require a user-supplied threshold when making inferences and the proposed methodology is no exception. The threshold of 0.99 used in our criterion for rhythmicity corresponds to 1% level of significance and is largely motivated by the specificity and sensitivity findings of Larriba et al.  ≥ 0.99, the rhythmicity calls on these two genes highly depended upon the normalization method n. However, under the criterion Since the Spearman correlation coefficient is based on the ranks, we therefore make a crucial observation from Figure MRobust almost always takes positive values for all normalization methods n. However, MRobust and MRobust take negative values. Notice that something similar happens for both the mouse pituitary , the methodology described here is applicable to any modern high-throughput technology involving oscillatory systems. For example, it can potentially be used for analyzing continuous time microbiome data, such as those obtained in Caporaso et al. .CR, MF, and SP: Conceived aims, conceptual design, data analysis, interpretation of the results, wrote and approved manuscript; YL: Processed original data, generated simulations, analyzed data, interpreted results, wrote and approved manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


By the nonnegativity constraint, NMF provides a decomposition of the data matrix into two matrices that have been used for clustering analysis. However, the decomposition is not unique. This allows different clustering results to be obtained, resulting in different interpretations of the decomposition. To alleviate this problem, some existing methods directly enforce uniqueness to some extent by adding regularization terms in the NMF objective function. Alternatively, various normalization methods have been applied to the factor matrices; however, the effects of the choice of normalization have not been carefully investigated. Here we investigate the performance of NMF for the task of cancer class discovery, under a wide range of normalization choices. After extensive evaluations, we observe that the maximum norm showed the best performance, although the maximum norm has not previously been used for NMF. Matlab codes are freely available from:  Accurate clustering of tumor samples with unknown cancer types or subtypes is of great importance for advances in cancer treatment and for better understanding of biological processes and the mechanisms of cancers. Traditionally, tumors can be clustered into a group based on similar external features such as appearance and clinical evidences. Cancers from within a group discovered by such a procedure may follow significantly different clinical courses and show different responses to therapy . This haa priori selection of genes in the data or different choices of initial conditions 1: Run Basic_NMF on if Filter==0 then2: H2 by multiplying each row of H0 by the normalized value of the corresponding column of W03:   Obtain else4: W by dividing each column of W0 by the normalized value of this column5:   Obtain u of size n × 1 by calculating the difference between the maximum value and the minimum value for each row of W6:   Generate a vector I = {p: u(p)≤T quantile of u}7:   Choose index set A1 by discarding rows of A with indices in I8:   Get a matrix A1, and get 9:   Run Basic_NMF on H2 by multiplying each row of H1 by the normalized value of the corresponding column of W110:  Obtain end if11: k, and this measure can be used to select an appropriate k. Our method is a straightforward adaption of that in 2: Run Basic_NMF on H1 by multiplying each row of H by the maximum value of the corresponding column of W {because the maximum norm will be shown to be the best choice as a normalization method}3: Obtain j is placed in cluster i if the j4: Determine the cluster membership by r of two distance matrices as in [5: Calculate the consensus matrix es as in : the firIn order to evaluate the effect of different normalization methods with/without the filter in Algorithm 1, we need to compare their performance in terms of clustering accuracy on some datasets that should be selected in a fair and unbiased manner. For this purpose, we need to choose datasets carefully. Our strategy is to use some well-studied datasets as the core datasets so that it is appropriate to do extensive analyses on them as information about true number of clusters, data quality, and results obtained by others is well known. We expect that the conclusions drawn from the core datasets are more convincing than those from other datasets less studied. Nevertheless, evaluation on more datasets helps draw conclusions with more confidence, so we also choose some datasets from GEO. In addition, we select a MicroRNA expression data used in  because Datasets selected are described below. A brief description of these datasets is reported in We start the evaluation on three gene expression datasets: Leukemia dataset , CentralLeukemia datasetIt is known that Acute Myelogenous Leukemia (AML) and Acute Lymphoblastic Leukemia (ALL) are distinct, and ALL can be further divided into two subtypes: T and B . Here weCentral nervous system tumors (CNS) This dataset contains 34 samples and is composed of four types of central nervous system embryonal tumors: 10 classic medulloblastomas, 10 malignant gliomas, 10 rhabdoids and 4 normals .Medulloblastoma dataset Medulloblastoma is a childhood brain tumor and the pathogenesis of these tumors is not well understood , 5. ThisBesides the above core datasets, we extend the evaluation to four additional datasets. We selected them from GEO Datasets by searching keywords “clustering”, “Homo sapiens”, and “Cancer”. Since the number of samples are too few, we skipped the datasets GSE73099 (2 samples), GSE66733 (2 samples), GSE67939 (17 samples), GSE65201 (8 samples), GSE60271 (14 samples), GSE58478 (4 samples), GSE55807 (6 samples), GSE58326 (6 samples), GSE44800 (8 samples), and GSE32132 (8 samples). Since the research is mainly focused on supervised learning by selecting a small set of signature genes, we skipped the datasets GSE69259, GSE51827, GSE67994, GSE68545, and GSE68104. The reason for this can be found in the discussion on evaluation of unsupervised methods using datasets designed for supervised algorithms in Section Discussion. Since the data has not clear cancer types, we skipped the datasets GSE70233, GSE68606, GSE51334, and GSE60272. We skipped some more datasets because that there is no gene expression data (GSE60270), the study is complicated (GSE55689), or there is no difference between the Basic NMF and the normalized one (GSE45419).We finally chose 4 datasets from GEO: Breast carcinomas and local recurrence (GSE4913) , pancreaGSE4913 This dataset is for the analysis of primary breast carcinoma tumors from 50 patients who received breast-conserving therapy (BCT). There are two subtypes: 19 patients subsequently developed a local recurrence of the carcinoma while 31 did not.GSE46385 There are 47 samples in this dataset. There are 5 types: Patient pancreas tissue, Patient pancreatic tumor, Patient-derived cancer cell line, Patient-derived cancer cell line in mouse, and Patient-derived tumor in mouse.GSE35896 This gene expression data is from 62 colorectal cancers. There are two dominant CRC subtypes that were discovered by a pathway-based stratification of CRC tumor samples.GSE42284 This gene expression data is from 188 primary colorectal cancers with four cancers stages.Normalized read count data for 697 tumor samples were extracted from Level 3 data archives onthe TCGA Data Portal website. The most variable 25% microRNAs were used as input to an NMF algorithm. For more details, see the supplementary materials of . We usedH and W, we performed 100 independent trials and computed the mean accuracy and the standard error of the mean. Note that the Basic NMF (here it is the NMF-Brunet) does not normalize the NMF solutions and it does not use the filter in Algorithm 1.We show the results in Without using a filter, we can see that the highest clustering accuracy for each dataset is always achieved by one of the normalization methods, among which the maximum norm appears to the best (9 times among 13 settings). Moreover, the smallest standard error of the mean for all settings except 2 cases is achieved by one of normalization methods while using the maximum norm achieves it most often (7 times), indicating that using the maximum norm makes the NMF solutions less sensitive to initial conditions in the NMF iteration algorithm.We observe that using the maximum norm is the best choice. However, on dataset CNS and GSE4913, using the maximum norm is significantly worse than the Basic NMF that does not employ normalization methods. This does not mean the normalization method using the maximum norm itself goes wrong: there may be some other factors distorting the NMF solutions, based on which the normalization method seems not to be good. One such factors is the effects from irrelevant genes. This is illustrated in a simulation study in With the embedded filter, using the maximum norm is even better (11 times) than other normalization methods. It is not the best for both the Colorectal datasets GSE42284 and GSE35896. However, for GSE35896, the difference in mean accuracy between the normalization method using the maximum norm and the best is only 0.44, which is not significant ; GSE42284 seems not to be appropriate for NMF clustering analysis as all methods achieve very low accuracy. This suggests that the maximum norm is generally the best choice.Ws, based on which the filter selects different gene sets on each run, and this in turn brings a little extra variation. Nevertheless, the filter is important to enhance the performance of the normalization method using the maximum norm when there are many irrelevant genes.To see the effects of the filter on the best normalization method—using the maximum norm, we extract the last column in W returned by an NMF algorithm.With the variance-filter, using the maximum norm is still the best among other normalization methods. But the variance-filter performs slightly worse than the embedded filter, and it is not so good for CNS, one of the well-investigated core datasets as the embedded filter. One possible reason is that, unlike the embedded filter, the variance-filter is not specially designed for NMF algorithms as it does not make use of Based on the above observations, there is strong evidence that using the maximum norm together with the embedded filter is superior, and is considered as the proposed post-processing method. To compare the overall effect of both the maximum norm and the filter with the Basic NMF (NMF-Brunet), we summarize the results in The proposed post-processing method is then evaluated extensively on the core datasets. To show its stability with respect to the number of input genes, we show its performance on these three datasets using a subset of the genes see . To showk = 2, 100% correctly clustering all 38 samples into 2 clusters; and for k = 3, 100% correctly clustering all 38 samples into 3 clusters. There are two errors in the Basic NMF [k = 2 and k = 3, which proves to be better than SOM and HC in terms of accuracy or stability [k = 3; and SNMF and NMF-Reg produce zero error, and one error respectively for k = 2. Moreover, as reported in a comprehensive evaluation of matrix factorization methods, none of the six matrix factorization methods and two other methods evaluated achieves zero error [As far as we know, none of existing methods performs perfectly—for tability , 7, one ro error .k = 2 and k = 3. It is also true if we do not use the filter. This is remarkable as two Leukemia samples in this data set have previously been considered to behave as outliers [k = 2) or 3,100 (for k = 3) most highly varying genes , which is left as future work. In the following, we shall only focus on the justifications for the normalization by the maximum norm.Although the filter in Algorithm 1 is not our main focus in this paper, its justification has been discussed when it is introduced. The unsolved problem is to find a better threshold automatically adapted to the data than the current default setting  while others partly . The genes’ expression levels controlled by these activities are proportional to their memberships in these activities. Here we specify the meaning of a metagene as a fuzzy collection of genes, and the coefficient i  can thus be approximated by a linear combination of rows of H , and the combination coefficients are specified by the memberships of i-th gene in metagenes (i-th row of W), i.e., Ai,: ∼ ∑jwi,jHj,:. In other words, because of the membership wi,j of gene i in metagene j, the j-th metagene’s expression Hj,: will contribute to gene i with an amount of wi,jHj,:; and the gene i’s expression is accumulated by all such contributions.Consequently, the gene expression levels across all samples for each gene Wj—the j-th column of a solution W from an NMF algorithm—are not necessarily smaller than or equal to 1. Thus it needs to be normalized. In the proposed method, Wj is divided by its maximum value (the maximum norm of Wj) to produce j, and H′ is obtained by multiplying each row of H by the maximum value of the corresponding column of W. This normalization method has the advantage that it restricts all entries in i.e., one or more genes are always turned on for some pathways involved in a typical cancer type). Although it is not ideal that the proposed method depends on this mild assumption, other normalization methods mentioned before are less desirable: divided by the p-norm  of Wj makes the entries in While membership is never greater than one, the entries in It can be imagined that the proposed method will suffer from some problems if this assumption does not hold in some scenarios, for which other normalization methods may be appropriate. However empirical evidence in W and H will be distorted; and as a result, the post-precessing method based on a distorted output may not achieve its best performance.In the previous sections, we have shown that the proposed post-processing method works very well compared to the Basic NMF algorithm that minimizes a generalized Kullback-Leibler divergence functional corresponding to a likelihood function with a Poisson noise or a scaled Poisson noise. It is natural to ask whether the proposed method improves other Basic NMF algorithms, for example, NMF-EU  that mine.g., gender). In such cases, the clusters discovered by NMF may not be consistent with the cancer types, and the evaluation based on the cancer types will be problematic. This problem can be seen from the evaluation of the proposed method on 11 datasets which were collected for evaluating classification methods [Some expression datasets are good for the task of classification but not for clustering—if only a small fraction of genes (signature genes) in a dataset differ between different types of tumors while the majority of genes differ between levels of another categorical feature  that does not employ normalization methods from  on 9 data priori selection of genes or initial conditions, and more robust to data variations. We have also demonstrated the improved clustering performance of the proposed method on three other gene expression datasets and one additional type of data—MicroRNA expression for Breast cancer.On the same three datasets that were commonly used by , 5–7, weGiven such promising results in clustering accuracy improvement and automatic selection of number of cancer types, this method of using the maximum norm (infinity norm) to normalize NMF solutions should not be ignored as it had been for clustering analysis using NMF techniques, although it is not perfect in itself. One obvious limit of the proposed method is that it depends on the Basic NMF: it requires that the Basic NMF is derived from a correct noise model; otherwise, it may not achieve its best performance. Another imaginable limit is that our assumption about the memberships of genes in a metagene may be violated: none of the memberships is one. Moreover, the filter in Algorithm 1 needs to be investigated further: there may exist a better threshold adapted to the data than the current default setting, in order to improve the performance of the normalization method using the maximum norm.S1 FileIt contains a simulation study, evaluation of unsupervised methods using datasets designed for supervised, effects of the filter on all datasets in the main text, and Basic NMF minimizing Euclidian distance algorithms.(PDF)Click here for additional data file.


Analysis of gene sets has been widely applied in various high-throughput biological studies. One weakness in the traditional methods is that they neglect the heterogeneity of genes expressions in samples which may lead to the omission of some specific and important gene sets. It is also difficult for them to reflect the severities of disease and provide expression profiles of gene sets for individuals. We developed an application software called IGSA that leverages a powerful analytical capacity in gene sets enrichment and samples clustering. IGSA calculates gene sets expression scores for each sample and takes an accumulating clustering strategy to let the samples gather into the set according to the progress of disease from mild to severe. We focus on gastric, pancreatic and ovarian cancer data sets for the performance of IGSA. We also compared the results of IGSA in KEGG pathways enrichment with David, GSEA, SPIA, ssGSEA and analyzed the results of IGSA clustering and different similarity measurement methods. Notably, IGSA is proved to be more sensitive and specific in finding significant pathways, and can indicate related changes in pathways with the severity of disease. In addition, IGSA provides with significant gene sets profile for each sample. During the past several years, the measurement of differential gene expression has been applied to various high-throughput biological studies. To facilitate the functional analysis of long lists of genes , a numbeHowever, one weakness in these methods is that many of them neglect information about the heterogeneity of gene expression in the samples , which mHere, we present Individual Gene Sets Analysis (IGSA), a functional and efficient software tool based on Java and R, which can be used to analyze gene lists and visualize the results from various angles. IGSA takes advantage of the heterogeneity of gene expressions in samples and the homogeneity of gene expression at the functional level.Differing from other methods such as ORA, FCS and NB, whose enrichment analysis is based on the changes of gene expression between cases and controls, IGSA preserves all gene expression data in samples and involves these data in the calculation of an expression score for each gene set in each sample. This approach is similar in spirit to SS; however, methods in SS, such as PLAGE, standardize gene expression profiles (GEPs) through z-scores, which assumes that gene expression profiles are jointly normally distributed; this means that all genes act independently within each gene set. Moreover, methods such as ssGSEA and GSVA rank gene expression and expression-level statistics, respectively, resulting in the loss of detailed information about gene expression in each sample.In addition, IGSA employs the accumulating clustering strategy , lettinghttp://210.46.85.180:8080/IGSAWeb.We used IGSA to analyze several cancer-related datasets . Compared with other methods , besides the gene sets related to many cancers, this method identified many specific gene sets in each cancer and, to a certain extent, indicated changes in gene sets related to the severity of different cancer. IGSA can be accessed freely at The gastric cancer data (GSE13861) were obtained from GEO. There were a total of 84 samples, and among these, 19 were classified as normal samples.The pancreatic cancer data (GSE16515) were obtained from GEO. There were a total of 52 samples, with 16 samples consisting of both tumor and normal expression data and 20 samples consisting of only tumor data.The ovarian cancer data (TCGA_OV_exp_u133a) were accessed from TCGA. The batch 9 ovarian cancer dataset consists of 46 samples, including 8 normal samples and 37 ovarian cancer samples . The data were downloaded in September 2014. The batch 40 ovarian cancer dataset included a total of 51 ovarian cancer samples. The survival analysis was based on samples that were diagnosed after the year 2000.These data (GSE48452) were available from GEO, including 73 samples of human liver grouped into C (control = 14), H , S (steatosis = 14) and N (nash = 18) groups.There were a total of 278 pathways (version 70.0) obtained from KEGG.We downloaded the gene ontology data (data-version: releases/2015-08-20) and the annotation data (gaf-version: 2.0) from the Gene Ontology Consortium. The third-layer GO terms of BP  and MF (Molecular Function) were extracted and annotated with gene product annotation. This analysis included 1,334 BP terms and 331 MF terms.The following procedure was adopted for IGSA enrichment.All of expression data were transformed to the log scale and without any filtering for the very low expressed genes. We removed the probes which could not be mapped to any genes or mapped to multiple genes.Taking into account differences in gene expression among the samples , we calculated the proportions of gene expression in each sample and then compared the proportions with the average level.Let g_ij denote the expression value of gene i in sample j. Let N be the number of samples. Then, the normalized expression value of each gene in each sample (Expr_Gene(g_ij)) is calculated as follows:In this way, the expression values of genes higher than the average level are intended to be positive , and the values of genes lower than the average level are inclined to be negative (down-regulated).We used the average expression values of genes in both the gene set and the sample  as the gene set  expression in the sample.Let Q_i denote all genes in the gene set i , and let S_j be the all genes in sample j. Let | Q_i ∩ S_j | denote the number of genes which exist in both the sample j and the gene set i. The expression of gene set i  of sample j〖Expr_GeneSet〗_pathway) can be calculated by gene expression (denote as Expr_Gene) using the following formula:For each gene set, we counted the number of expression scores above or below the average value  in controls and cases. Then, we constructed 2x2 contingence tables and used Fisher’s exact test to obtain the significance value for each gene set. The null hypothesis is that the gene set  is independent of the cancer. We also adjusted the p value of each gene set by the FDR.In consideration of the amplification of gene expression along with the progression of cancer (more genes tend to be expressed aberrantly) , we deveThe similarity measurement in IGSA clustering focuses more on the similarity between the samples in quality  rather than in quantity. We used qualitative rather than quantitative measures to avoid the impact on the analysis that results from the over-fitting model.The similarity between two samples in IGSA clustering is calculated based on the co-expression of gene sets  and differential expression of gene sets .The similarity between two samples can be simply represented as the number of CGSs. However, the expression values of some CGSs in both samples may be simultaneously above or below the average value; this indicates that the phenomenon of co-expression among some gene sets may be a random event, which is common when the number of samples is not large enough; this phenomenon is also observed for DGSs. As a result, a weight  should be introduced into the calculating to relieve the impact of the pseudo CGSs and DGSs.Let N_cgs denote the number of CGSs and N_dgs denote the number of DGSs. Thus, the number of real CGSs in CGSs is N_cgs*θ^(N_cgs) , and the number of real CGSs in DGSs is N_dgs*〖(1‒θ)〗^(N_dgs) . The complete function of similarity measurement (〖Similarity〗_ij) for sample i and sample j is as follows:The interval of θ is  . When θ is 0.5, all gene sets are expressed at random, and when θ equals 1, all CGSs and DGSs are considered to be meaningful.Furthermore, due to the uncertainty of θ, which may be interfered by array signal quality, etc., we use the maximum likelihood estimation (MLE) method to calculate the similarity.IGSA clustering is based on the similarity of expression of significant gene sets derived from IGSA enrichment.Step 1: Create a seed set for storing seed samples that represent the start seed and samples that have been clustered and create a candidate set for storing candidate samples (disease samples) that are waiting to be clustered.Step 2: Construct a vector as a start seed through calculating the average expression value of each gene set  in normal samples, and add the vector into the seed set (the vector is a start seed). Add all disease samples into the candidate set. For example, suppose after IGSA enrichment, we obtain 11 significant gene sets. We first obtained the start seed through calculating the average expression value of these 11 gene sets in all normal samples and added it into the seed set as the start seed.Step 3: Calculate the average similarity score of each sample in the candidate set with all seeds, and then add the sample, rather than the gene-set, with the highest average similarity score into the seed set (the sample is marked as seed) and remove the sample from the candidate set. For example, suppose we have 20 cancer samples in the candidate set and 2 samples in the seed set (the one is the start seed and the other is the clustered disease sample of the highest similarity score with the start seed). For each of the 20 cancer samples, we calculate the average similarity between the cancer sample and 2 samples in the seed set. Finally, we will obtain 20 similarity values for 20 cancer samples and we will move the sample with the highest value into the seed set from the candidate set. Thus, the size of the seed set is increased to 3, and the size of candidate set is decreased to 19.Step 4: Repeat step 3 until the candidate set is null.All samples are clustered in turn according to the similarity score. To evaluate the performance of IGSA enrichment, we ran the enrichment analysis program based on the pathways involved in various cancer-related expression profiles.We first applied this method to a gastric cancer dataset, and IGSA (FDR<0.01) identified 48 significantly up-regulated pathways  and 38 significantly down-regulated pathways . Out of 48 SUPs found by IGSA, 35 pathways were reported to be related to gastric cancer . The accuracy of SDPs identification was approximately 76.32%.In addition to the common cancer-related pathways , IGSA also found some pathways that were specifically related to gastric cancer. For example, the primary pathway for bile acid biosynthesis (hsa00120), in which bile acid promotes gastric carcinogenesis without inflammatory cell infiltration ; the othOur methods were also applied to pancreatic cancer and ovarian cancer datasets to demonstrate the universality of IGSA. Significantly, at an FDR<0.01, IGSA found 37 SUPs (the accuracy was 78.38%) and 6 SDPs (the accuracy was 83.33%) in the pancreatic cancer dataset and 18 SUPs (the accuracy was 72.22%) and 28 SDPs (the accuracy was 53.57%) in the ovarian cancer dataset. We compared the IGSA method with other enrichment methods, including DAVID, GSEA, SPIA, GSVA, ssGSEA and Z-score .Most of these methods identified some pathways commonly related to cancers, including cell cycle (hsa04110) and DNA replication (hsa03030). However, IGSA also found pathways that were specifically related to the different cancer types.Three methods  performed well in gastric cancer data (accuracy > 70%)  online, In pancreatic cancer datasets, most of the tested methods identified some significant pathways, except for SPIA (found no SUPs and only one SDP), ssGSEA (the accuracy was less than 70%) and GSVA (found no SDPs). Again, IGSA identified the most paper-supported significant pathways (the proportion was approximately 73.91%).The methods performed less well in finding pathways from the ovarian cancer dataset ; however, compared with the other methods, DAVID and IGSA still performed well in identifying SUPs (accuracy > 70%). Although DAVID was 100% accurate in finding SDPs, this method found only 5 pathways, which was fewer than IGSA (16 pathways).hepatitis dataset (P<0.05), and 14 significant pathways were found . In clustering based on SUPs, the survival time was reduced when newer cancer samples were added into the seed set. In clustering based on all significant pathways (including SUPs and SDPs), the clustering was not significant. One possible reason for this result is the high false-positive rate among SDPs (the accuracy was only approximately 53.57%). We also compared the similarity measurement in IGSA clustering (SMIC) with other methods, including Euclidean distance, Pearson's correlation and Spearman's rank correlation . The resTo analyze the significant pathways found by IGSA enrichment, we first used unsupervised hierarchical clustering (UHC) to cluster the samples  based on the pathways. We evaluated the clustering of the ovarian cancer data through the corresponding survival analysis. In consideration of the low accuracy in finding SDPs (only 53.57%), we applied the clustering to the samples based on SUPs and divided the cases into three classes according to their threshold for distinguishing controls from cases. We also compared the result with clustering based on the up-regulated genes . The result based on pathways  was marginally better than the other  .Further improvement was made when we divided the cases into three classes based on the results of IGSA clustering , which were shown to be related to ovarian cancer in previously published papers. The samples were divided into two classes, and the survival time in the two classes was significantly different  gene sets and MF (Molecular Function) gene sets. Figure D in Traditional methods  focus on the analysis of gene expression. The enrichment analysis used with these methods is based on changes in gene expression between cases and controls, with only one value used to represent the expression of a gene in all samples , discarding the information from diverse individual gene expression. Particularly, in the analysis of personalized medicine, the gene sets  found using traditional methods contribute only slightly to the analysis. Other methods  calculate the gene set scores for each sample, although most of these methods lack the information of gene set  expression for various reasons .However, IGSA, as distinct from traditional methods whose enrichment strategies focus on the elimination of the impact of heterogeneity between samples at the gene expression level, takes advantage of the heterogeneity of gene expression in samples and focuses on specific and significant gene sets, as a complement of these traditional methods. In clustering, IGSA stresses the similarity between the samples in quality rather than in quantity, which averts the over-fitting caused by focusing too much on the quantity of similarity, which commonly occurs when using traditional methods. Moreover, IGSA uses an accumulating clustering strategy, adding disease samples into sets according to the extent of possible deterioration, which can reveal the severity of disease in different samples and correlate the related changes in pathways with the severity of disease.As mentioned in the results, compared with other methods , the performance of IGSA was comparatively specific, sensitive and robust. In clustering, IGSA proved to be more effective than the traditional method . The reason for this result is that IGSA concentrates on the similarity of samples at the functional level rather than at the molecular level, which is more susceptible to noise and samples. In the classification, the classes divided by IGSA are not very well separated (class 3 is not markedly separated from the other classes), which may be caused by incomplete survival information, among other reasons . In addition, using IGSA, we identified a signature (13 papers supporting SUPs and 16 paper supporting SDPs found by IGSA) for ovarian cancer, and the survival times of the classes based on the signatures were significantly distinct.IGSA also provides the expression of significant gene sets in each sample, which reveals a legible difference in different classes and is conducive to personalized medicine. It is also possible for IGSA clustering to indicate the potential progression of a disease due to the strategy used for clustering, which may be very useful for the analysis of disease progression at the functional level.To assess IGSA universality, we added other 3 non-cancer datasets from GEO for the analysis which includes diabetes (Type II) GSE9006, myocardial infarction GSE48060, and rheumatoid arthritis GSE55235. Except myocardial infarction, the performance of IGSA on the other two datasets is better. The accuracy of detecting significant pathways were as high as 100% for diabetes and 92.9% for rheumatoid arthritis, while the accuracy of detecting significant pathways for myocardial infarction was 62.5% .The resuThere are also some flaws with IGSA; for instance, this approach has some difficulty in finding pathways in datasets with a relatively low proportion of control samples. According to the results of enrichment, the proportion of controls in the samples has an impact on pathway identification. IGSA effectively identified pathways in gastric cancer and pancreatic cancer datasets (the proportion is 22.62% and 30.77%), whereas in the lower proportion samples , the performance of IGSA was decreased. In spite of this, IGSA performed well in finding SUPs compared with the other methods. It is also difficult for IGSA to analyze data with few samples because under such conditions, the strategy adopted by IGSA for enrichment will retrograde to the traditional method .In addition, relying on gene expression data alone to analyze cancer-related data is not sufficient; instead, it is more effective to integrate IGSA with other tools  to analyze cancer samples. The strategy used in IGSA is also suitable for the analysis of other data .At present, analyses based on integrating multiple genomic datasets are very common. Du et al.  analyzedS1 File(PDF)Click here for additional data file.S1 Table(ZIP)Click here for additional data file.S2 Table(ZIP)Click here for additional data file.S3 Table(ZIP)Click here for additional data file.S4 Table(ZIP)Click here for additional data file.S5 Table(ZIP)Click here for additional data file.S6 Table(ZIP)Click here for additional data file.S7 Table(ZIP)Click here for additional data file.S8 Table(ZIP)Click here for additional data file.S9 Table(ZIP)Click here for additional data file.S10 Table(ZIP)Click here for additional data file.S11 Table(ZIP)Click here for additional data file.


Long non-coding RNAs (lncRNAs) form a substantial component of the transcriptome and are involved in a wide variety of regulatory mechanisms. Compared to protein-coding genes, they are often expressed at low levels and are restricted to a narrow range of cell types or developmental stages. As a consequence, the diversity of their isoforms is still far from being recorded and catalogued in its entirety, and the debate is ongoing about what fraction of non-coding RNAs truly conveys biological function rather than being “junk”. Here, using a collection of more than 100 transcriptomes from related B cell lymphoma, we show that lncRNA loci produce a very defined set of splice variants. While some of them are so rare that they become recognizable only in the superposition of dozens or hundreds of transcriptome datasets and not infrequently include introns or exons that have not been included in available genome annotation data, there is still a very limited number of processing products for any given locus. The combined depth of our sequencing data is large enough to effectively exhaust the isoform diversity: the overwhelming majority of splice junctions that are observed at all are represented by multiple junction-spanning reads. We conclude that the human transcriptome produces virtually no background of RNAs that are processed at effectively random positions, but is—under normal circumstances—confined to a well defined set of splice variants. ANRIL [HULC [MALAT1 [TUG1 [Xist [Long non-coding RNAs (lncRNAs) are an important part of the mammalian transcriptome ,2. AlthoANRIL ,4, HULC IL [HULC , MALAT1  [MALAT1 , TUG1 [7G1 [Xist  may servG1 [Xist ,10, and G1 [Xist . Still, G1 [Xist  providesG1 [Xist ,14,15) pANRIL [There is mounting evidence, however, that lncRNA isoforms may differ drastically in their biological function ,17. Due ANRIL  exhibit ANRIL . It thusOne component towards answering this question is to ask to what extent the transcript portfolio of a particular cell type has been mapped completely. Conversely, one asks in this context to what extent reported transcripts are noise. To address these issues, we investigate here a very large set of transcriptome data from B cell lymphomas. By virtue of aggregating hundreds of independently generated transcriptome datasets, we can study in detail if the set of detectable splice junctions converges to a consensus.A comparison of published annotation data showed substantial differences in the average number of exons per transcript, with some systematic trends over time. The recent Ensembl version 83 , which sTo better understand the distribution and structure of rare isoforms in the human transcriptome, we systematically investigated the influence of the dataset size on the complexity of inferred gene structures. We focussed on RNA-Seq data from 111 lymphoma samples consisting of the different subtypes Burkitt lymphoma (BL), follicular lymphoma (FL), and diffuse large B cell lymphoma (DLBCL) that was published in the context of the ICGC MMML-Seq project . The RNAThe data show that the detected splice junctions are unlikely to be noise, since the curves saturate well for all three minimum read support thresholds instead of showing linear growth, affirming their biological reality.In Comparing our data to existing GENCODE annotations, we find that lncRNAs exhibit systematically larger numbers of exons and introns. However, the discrepancy is moderate and applicable—in particular to lncRNAs that already have a large number of exons annotated. Around 41% of genes are found to have a greater  number of introns. Fourteen percent of the genes have at least one intron more, and 19% are calculated to have at least two introns more.In We have shown that human transcriptome data harbour a large number of rare exons  that have remained unannotated. Due to their low abundance, they appear only when data from large-scale experiments are pooled. As shown in The well definedness of isoforms by itself does not imply that all isoforms carry biological functions. If the vast majority of isoforms are indeed non-functional junk, however, an explanation is needed for the precision of the processing and its restriction to very few splice sites.segemehl, version 0.1.7 [-S) enabled in addition to the default parameters. The total number of reads were around 120 million per sample. The read length was 101 nucleotides, and around 90% of the reads were mappable.The 111 RNA-seq lymphoma samples from the ICGC MMML-Seq project  were mapon 0.1.7 ,30 with segemehl identifies split-reads independent of any annotation, we consider all splice junctions located within genomic coordinates taken from GENCODE version 19 as potential introns belonging to that GENCODE gene. In order to call it an intron, we require a minimum of one, five, or ten reads representing the junction. This procedure was performed using the complete range of all dataset sizes from one sample to all 111 samples. Normalized mean expression values—quantified as RPKM (reads per kilobase and million reads)—were averaged over all 111 samples and used to define a gene’s expression level in the lymphoma data set in The read supports for all identified splice junctions  were calculated. Since To analyse the annotation landscape of lncRNAs, we used the detailed annotation data from GENCODE (versions 7 through 24) , EnsemblAn in-house pipeline was used to aggregate the multiple transcriptomes and compute the summary statistics of interest. Input datasets are provided in the standard Gene Transfer Format (GTF) files, which contain fields indexing the various features of each gene, viz. its exact location, as well as its set of constituent transcripts. Within a gene, transcripts typically overlap and share exons. We therefore first determined a collection of unique exons for each gene, which was subsequently used to determine the unique introns. Mean and median number of introns and exons per gene were computed from the number of unique exons and introns, respectively . A summary of the overlap between lincRNAs of the lymphoma dataset and GENCODE annotations can be found in


The high degree of heterogeneity observed in breast cancers makes it very difficult to classify the cancer patients into distinct clinical subgroups and consequently limits the ability to devise effective therapeutic strategies. Several classification strategies based on ER/PR/HER2 expression or the expression profiles of a panel of genes have helped, but such methods often produce misleading results due to their dynamic nature. In contrast, somatic DNA mutations are relatively stable and lead to initiation and progression of many sporadic cancers. Hence in this study, we explore the use of gene mutation profiles to classify, characterize and predict the subgroups of breast cancers.We analyzed the whole exome sequencing data from 358 ethnically similar breast cancer patients in The Cancer Genome Atlas (TCGA) project. Somatic and non-synonymous single nucleotide variants identified from each patient were assigned a quantitative score (C-score) that represents the extent of negative impact on the gene function. Using these scores with non-negative matrix factorization method, we clustered the patients into three subgroups. By comparing the clinical stage of patients, we identified an early-stage-enriched and a late-stage-enriched subgroup. Comparison of the mutation scores of early and late-stage-enriched subgroups identified 358 genes that carry significantly higher mutations rates in the late stage subgroup. Functional characterization of these genes revealed important functional gene families that carry a heavy mutational load in the late state rich subgroup of patients. Finally, using the identified subgroups, we also developed a supervised classification model to predict the stage of the patients.This study demonstrates that gene mutation profiles can be effectively used with unsupervised machine-learning methods to identify clinically distinguishable breast cancer subgroups. The classification model developed in this method could provide a reasonable prediction of the cancer patients’ stage solely based on their mutation profiles. This study represents the first use of only somatic mutation profile data to identify and predict breast cancer subgroups and this generic methodology can also be applied to other cancer datasets.The online version of this article (doi:10.1186/s12918-016-0306-z) contains supplementary material, which is available to authorized users. Breast cancer (BC) is a genetically and clinically heterogeneous disease; hence, the effectiveness of a specific treatment greatly varies among BC patients. There have been several widely accepted methods to classify breast cancers into distinct subtypes –7, such Currently, the microarray-based BC classification has been regarded as the gold standard . HoweverSeveral studies that integrated multiple types of molecular data for breast cancer clustering have been proposed. Curtis et al.  suggesteIt is often difficult to predict the impact of single nucleotide mutations in the genome at a molecular level and consequently their effect on cancer initiation and progression. In addition, somatic mutations are often sparsely distributed in different cancer samples. Therefore, previous studies used somatic mutation data as an auxiliary information in combination with other data types to classify cancer and/or used as a binary entity (the presence or absence of a mutation) . This stIn this study, we developed a novel method to classify breast cancers based on the quantification of somatic mutation profiles. We analyzed the whole exome sequencing data from 358 ethnically similar BC patients in The Cancer Genome Atlas (TCGA) project. We first scored the functional impact of each variant using Combined Annotation–Dependent Depletion (CADD) scores , and theOur initial observation on the mutation score matrix showed that, the C-scores range from 0 to 1417.14 and distribution of scores for top ten variant genes can be seen in Fig. Somatic mutation profiles of BC patients exhibit a very sparse data form, unlike other data types such as gene expression or methylation in which nearly all genes or markers are assigned a quantitative value in all the patients. Even clinically identical patients may share no more than a single mutation –18. Thern features for clustering (see methods for more details). We optimized the size of n to be 854 genes in our clustering method.In order to overcome the aforementioned challenges, generally there are two popular approaches, namely; feature extraction and feature selection. Feature extraction transforms the current existing features into a lower dimensional space and widely used example methods include principal component analysis (PCA) and linear discriminant analysis (LDA), while feature selection selects a subset of features without applying any transformation. These methods increase the sample-to-feature ratio and decrease the sparseness hence making the clustering both feasible and more effective. In this study, we used feature selection by ranking the features (genes) in decreasing order of their variance value and selected top Unsupervised clustering is the task of grouping a set of samples that have no label information, which results in grouping samples in such a way that samples in the same group are more similar in a specified measure to each other than to those in the other groups. There are several methods trying to achieve this goal such as k-means clustering, hierarchical clustering and expectation maximization (EM) algorithms. However, these methods perform poorly or cannot come to a solution when applied to sparse data, as is the case in our study. Therefore, we selected to use NMF because of its proven superior performance when tested on biological data based applications –21. NMF As a factorization method, NMF algorithm takes our mutation score matrix as the input and decomposes it to two smaller matrices (basis matrix W and coefficient matrix H). The output coefficient matrix (matrix H) is used to make sample cluster assignments. Refer to methods for more details.Using the NMF clustering algorithm on our dataset, we stably clustered the samples into three groups using the top 854 genes, which have the highest variance values of mutation scores across all the samples. The three groups Cluster 1, 2, and 3 involve 169, 121 and 68 patients, respectively. Refer to methods section for more details.In Fig. Figure We investigate the clinical significance of discovered clusters by comparing the BC stage of the patients in each cluster. For this purpose, we analyze the distribution of patients according to their disease stage provided in the TCGA data. We found that Cluster 1 was dominated by early stage patients while Cluster 3 had much higher proportion of late stage patients compared to Cluster 1 . We found that there were 358 genes, which have significantly higher mean mutation scores in the late-stage-enriched cluster (Cluster 3) than in the early-stage-enriched cluster (Cluster 1) , but none of the genes have significantly higher mean mutation scores in Cluster 1 than in Cluster 3. This interesting finding indicates that these genes may have accumulated deleterious mutations leading to the progression of breast cancer into advanced disease states. We identified that tumor suppressor genes, APC, BRCA2; and oncogene, MLL are among the 358 genes used in this comparison. Table We stratified the 358 genes into different gene families using the Gene Set Enrichment Analysis (GSEA)  tool as We carried out the network analysis of the top 25 highly mutated genes  algorithms; Random Forest (RF) , SupportWe used a 10-fold cross-validation for evaluation of classifier performances. In each loop of the 10-fold cross validation, after withdrawal of the test set, we did feature selection using the information gain feature selection method  and seleAlso we observe that SVM algorithms achieved a very close accuracy but with a loss in TPR, FPR and F measure. And KNN method yielded the worst accuracy of all the methods we used. Table Figure p-value < 10−4) (see methods for more details). This supports the robustness of our model and the predication accuracy.We also used a permutation test, by running the same class prediction procedure with RF on 10,000 randomly labeled datasets and none of the 10-fold cross-validations gave us a better accuracy, yielding a very significant p-value  from white, not Hispanic or Latino group for analysis. We obtained an average of 17,640 point variations per patient, generated by VarScan2 [We downloaded the sequence variation data in variant call format (vcf) for the TCGA breast cancer whole exome sequencing data. To eliminate the population heterogeneity effect, we selected the breast cancer patients  to keep track of all the deleteriousness scores (C-scores) of somatic mutations used for machine learning. The mutation score matrix represents a table that contains the genes in rows and the patients in columns, yielding a matrix of size 18,117 rows by 358 columns, with at least one mutation in each row. And each cell contains the sum of all C-scores of mutations found in a gene for a patient.We have obtained an average of 17,640 point variations per patient generated by VarScan2  and applm × n mutation score matrix to keep track of the sum of the variant scores in all genes, where m is the number of genes  and n is the number of samples (358 patients). The value in entry  indicates the mutation score of gene i in sample j, which is the sum of all C-scores of mutations found in the gene i for the sample j.We implemented an n features for clustering.Due to the number of features (tens of thousands genes) being much more than the number of samples (hundreds of samples), we first used feature selection to select only the informative features for clustering; thus to reduce the feature size. We ranked the features in decreasing order of their variance values (Equation 1) and selected top Equation 1: Variance formulam × n, into two smaller matrices, W ∈ ℝmxk and H ∈ ℝkxn, with positive entries, A ≈ WH using a positive integer number k < min{m, n}. Matrix W, called as a basis matrix and has size m × k, with each of the k columns defining a metagene; and entry wij represents the coefficient of gene i in metagene j. Matrix H is named as coefficient matrix and has size k × n, with each of the m columns representing the metagene expression pattern of the corresponding sample; and entry hij represents the mutation load of metagene i in sample j. There are multiple solutions to this problem and in this study we adopt a method by Brunet et al. [We used NMF method for clustering, which aims to find a small number of metagenes, each defined as a positive linear combination of all the genes so that the method can approximate the mutation load of the samples as positive linear combinations of these metagenes. Mathematically, this corresponds to factoring a given non-negative matrix A of size t et al.  that wasEquation 2: Coupled divergence equations to update the W and H matricesk) of clusters. Algorithm assigns each sample according to the highest scored metagene in patients designated column in matrix H; meaning that sample j will be assigned to the cluster i if hij is the highest entry in column j.As a result of factorization, we use coefficient matrix H to group our samples into given number  and features (genes) to use in clustering, we used consensus matrix and average silhouette width of consensus matrix.Since the NMF algorithm starts with a random initial class assignment of samples, repeated runs over the same sample set with constant input parameters may not result in the same sample assigned to the same class between the runs; however, if we observe only a little variation in these associations between runs, then we can conclude with confidence that a strong clustering was performed for this set of parameters (number of clusters and features). This idea forms the basis for our clustering performance evaluations.C of size m × m by entering cij = 1 if samples i and j are assigned to the same cluster and cij = 0 otherwise. Then the consensus matrix, C for many clustering runs. (We selected to use 100) The value in i and j assigned to the same cluster. In the case of a stable clustering then we expect to see most of the values in Consensus matrix is a concept proposed by Brunet et al.  providina (i) as the average dissimilarity/distance of sample i with all other data within its cluster, the value of a (i) will then indicate how well the sample i fits into its assigned cluster by having a smaller value showing better assignment. Then we can define b(i) by the lowest average dissimilarity of sample i to any other cluster, that i is not a member. In other words b(i) indicates the average dissimilarity of sample i to its closest neighboring cluster or its next best fit cluster. Then the silhouette score of a sample can be calculated as in Equation 3 below. The value of s(i) can range from −1 to 1, and being close to 1 means that the sample is perfectly clustered. And average of s(i) over all the samples, named as average silhouette width, shows how well the data has been clustered.In addition to the consensus matrix, we used average silhouette width of consensus matrix (silhouette(consensus)), introduced by Rousseeuw , to quanEquation 3: Equation shows how the silhouette score of sample can be computedWe used the consensus matrix’s silhouette score to determine the optimal number of genes and clusters by iteratively running the algorithm over a range of biologically reasonable parameters (from 10 to 1000 top variant genes and from 2 to 10 clusters).Lastly, among several implementations of NMF in various programming languages, we selected to use an R implementation of NMF, published by Gaujoux and Seoighe , becauseTo characterize the clusters we discovered, we correlated the samples in the clusters with their clinical features. We defined stage I and II as early stage and stage III and IV as late stage. The Fisher’s exact test was used to assess the stage tendency of clusters.We compared the mutation score of genes between clusters using the Wilcoxon rank-sum test, and adjusted the multiple testing with the false discovery rate (FDR). The FDR was estimated using the Benjamini-Hochberg procedure . We usedFor running feature selection, classification model generation using ML algorithms and performance measurements, we used the Waikato Environment for Knowledge Analysis (WEKA)  frameworFor feature selection, we used the Information gain attribute evaluator , and RanFinally, to validate the strength of the achieved prediction accuracy, we run a permutation test. For this test we generated 10,000 datasets by randomly shuffling patient labels in our dataset, while keeping the number of patients in each class constant. We run 10-fold cross-validation with RF classification algorithm together with feature selection step on these datasets, in the same way used for the real data in the study. We calculated a p-value by the number of times this validation produced a better accuracy on randomly shuffled dataset divided by 10,000.AUC, area under the curve; BC, breast cancer; CADD, combined annotation dependent depletion; dbSNP, single nucleotide polymorphism database; EM, expectation maximization; ER, estrogen receptor; FN, false negatives; FP, false positives; GSEA, gene set enrichment analysis; HER2, human epidermal growth factor receptor 2; IHC, immunohistochemical; IPA, ingenuity pathway analysis; KNN, k-nearest neighbor; LDA, linear discriminant analysis; MAF, minor allele frequency; ML, machine learning; NCBI, National Center For Biotechnology Information; NGS, next-generation sequencing; NMF, non-negative matrix factorization; PCA, principal component analysis; PPV, positive predictive value; RF, random forest; ROC, receiver operating characteristic; SVM, support vector machine; TCGA, the cancer genome atlas; TN, true negative; TNR, true negative rate; TP, positives; TPR, true positive rate; WEKA, Waikato environment for knowledge analysis