Next-generation sequencing (NGS) has advanced the application of high-throughput sequencing technologies in genetic and genomic variation analysis. Due to the large dynamic range of expression levels, RNA-seq is more prone to detect transcripts with low expression. It is clear that genes with no mapped reads are not expressed; however, there is ongoing debate about the level of abundance that constitutes biologically meaningful expression. To date, there is no consensus on the definition of low expression. Since random variation is high in regions with low expression and distributions of transcript expression are affected by numerous experimental factors, methods to differentiate low and high expressed data in a sample are critical to interpreting classes of abundance levels in RNA-seq data.A data-adaptive approach was developed to estimate the lower bound of high expression for RNA-seq data. The Kolmgorov-Smirnov statistic and multivariate adaptive regression splines were used to determine the optimal cutoff value for separating transcripts with high and low expression. Results from the proposed method were compared to results obtained by estimating the theoretical cutoff of a fitted two-component mixture distribution. The robustness of the proposed method was demonstrated by analyzing different RNA-seq datasets that varied by sequencing depth, species, scale of measurement, and empirical density shape.The analysis of real and simulated data presented here illustrates the need to employ data-adaptive methodology in lieu of arbitrary cutoffs to distinguish low expressed RNA-seq data from high expression. Our results also present the drawbacks of characterizing the data by a two-component mixture distribution when classes of gene expression are not well separated. The ability to ascertain stably expressed RNA-seq data is essential in the filtering process of data analysis, and methodologies that consider the underlying data structure demonstrate superior performance in preserving most of the interpretable and meaningful data. The proposed algorithm for classifying low and high regions of transcript abundance promises wide-range application in the continuing development of RNA-seq analysis. Transcriptome analysis is integral in understanding the role of genetic and genomic variants in disease progression, classification, and diagnosis. In recent years, next generation sequencing (NGS) technologies have catapulted transcriptome profiling to new heights by providing greater precision of cellular RNA content. As a result, RNA-seq is more sensitive to subtle changes in expression levels and typically includes many transcripts with low read count integers.Researchers have taken interest in the low regions of a sample for several reasons, all with the primary intent of quantifying the level of expression that reflects important biological meaning. One of the major problems in RNA-seq data analysis is that there is no consensus on what is considered low expression. Low counts have been referenced as less than 10 reads when summed across treatments, less thA number of factors affect the distribution of read counts for a given study. Specifically, the manufacturer, library preparation and construction, alignment algorithm, gene length, sequencing depth, and experimental design all play a role in determining the number of reads that are mapped to a gene. Since each of these factors varies from study to study, it may be misleading to ignore properties of the sample distribution by applying an arbitrary cutoff to classify the low expressed region of a sample. To address the question of what should be considered the lower bound of functional gene expression, it is useful to consider the premise that genes can be categorized into a group of high expressed (HE), meaningful genes and a group of low expressed (LE), non-informative genes. In previous studies, the empirical distribution of a sample was used to identify classes of mRNA abundance levels. This con or n + 1 components, where 0 < n < 9. Mclust[Hebenstreit et al. studied . Mclust, an R pa. Mclust. However. Mclust-22. SecoHebenstreit et al. were ablWhen the LE and HE regions of RNA-seq data are well separated ; 0.20 and 0.16 (L5 SNL – 2 months); 0.23 and 0.24 (control – 2 weeks); 0.25 and 0.24 (L5 SNL – 2 weeks). Overall, DAFS showed small variability in estimating quantile cutoffs for technical replicated sequencing data. These results further demonstrate the impracticality of an arbitrary cutoff even within the same study. As evidenced by the data, it is reasonable to suspect that the separation of LE and HE genes is more homogeneous within replicates of the same experimental treatment. However, we may presume less agreement across biological replicates, treatments, and experiments.Additionally, we analyzed two different RNA-seq datasets with technical replicates to assess the consistency of DAFS. Overall, DAFS demonstrated great consistency in estimating cutoffs from technical replicates. For SEQC, a r et al., mRNA-seTo explore further the spectrum of possible expression profiles, we measured the performance of DAFS on sequencing reads from cDNA fragments of cultured human B-cells (GSE12526) in order0.85), 0.90 (Sen0.90), and 0.95 (Sen0.95) are reported in Table qc (estimated by the 2.5th and 97.5th quantiles of cutoff estimates) , and negative predictive value (NPV) were evaluated for DAFS and Mclust Table . The meaFor log2 transformed count data, which presented a clear bimodal distribution, the estimated cutoffs from DAFS and Mclust were similar. Average sensitivity for both methods exceeded 95%. On average, the theoretical cutoff estimated by Mclust was closer to the 95% sensitivity estimate. Although Mclust shows an improvement in specificity and DAFS does a better job of optimizing the sensitivity and NPV, the differences in performance measures were small. Analysis of the 95% confidence intervals for cutoff values demonstrated that both DAFS and Mclust were not significantly different from estimates obtained from achieving 95% sensitivity.In evaluating cutoffs for log2 transformed RPKM data, DAFS returned an average sensitivity estimate that exceeded 98%. However, Mclust estimates fell between values obtained for achieving 90% and 95% sensitivity. Analysis of the 95% confidence intervals for cutoff values presented similar findings. The results indicated that DAFS captured a higher proportion of true high expression. As such, DAFS demonstrated superior performance in sensitivity and NPV, whereas PPV was comparable between Mclust and DAFS. The low specificity of DAFS is explained by the proportion of low expression. Since the low expressed region represented roughly 10% of the total sample size, a small number of misclassified observations will decrease specificity rapidly.Based on the two different scenarios, the results suggest that DAFS is comparable to Mclust at maximizing sensitivity when the mixture components are well separated; however, DAFS is superior at providing a cutoff that maximizes sensitivity when the LE and HE components are not distinguishable. The simulation results clearly demonstrate how DAFS adapts to different data patterns. The pattern of log2 RPKM data is relatively closer to one normal distribution than the log2 raw counts. Therefore, DAFS adjusts and retains as many genes as possible when analyzing log2 RPKM data. Consequently, relative to the results obtained from simulating log2 raw counts, sensitivity and PPV are increased but specificity and NPV are decreased.H0 : F(x) = F0(x), for all x versus the alternative H1 : F(x) ≠ F0(x), for some x. Thus, the reference distribution F0(x), e.g. normal, lognormal, Student’s t, etc., is completely specifiable by the researcher. To our advantage, DAFS performed consistently well when the HE component departed from normality (as evidenced in log2 RPKM and FPKM data).This study was motivated by the need to provide a data-adaptive algorithm for separating RNA-seq data with low and high expression, particularly when the distributions of expression abundance are not distinctly separated. In order to compute the optimal cutoff between low and high expression, our method relied heavily on the assumption that high-expressed data are normally distributed. An advap by step sizes of 0.01 or 0.005 provided a good balance between parsimonious and over-saturated input. Nevertheless, the increment selection is a variable in the proposed methodology that must be investigated by the researcher.Careful consideration should be taken with regard to the number of observations used to model the distribution of K-S statistics. The benefit of adding more data to characterize the underlying density should be balanced with the disadvantage of modeling added noise. If the predictor space is segmented too finely, then it is possible for multiple percentiles to map to the same K-S statistic. Multiple many-to-one mappings would make it difficult for the MARS algorithm to differentiate true variation from random noise. In the present study, incrementing k > 2 component Gaussian mixture. In our own analysis, many of the identified components were not separated enough to be heterogeneous populations. We employed a number of methods/packages to merge Gaussian mixture components  with noad hoc techniques or methodology that may be prone to model-data misfit will facilitate interpreting the quality of sequencing reads and lead to improved power for differential detection of high expressed, reliable data.In our web search of the literature, the question of a cutoff for low expression in RNA-seq was frequently asked. Some questions were motivated by a desire to quantify what is considered expressed in RNA-seq. Others were motivated by a need to classify the level of measurement that could be trusted in assessing the significance of differential expression in low-expressed regions, particularly since research shows that the precision of RNA-seq data analysis improves as genes are more highly expressed. Whether transcripts with low expression are simply flagged or removed prior to testing via independent filtering, the work presented here provides a data-driven methodology for separating RNA-seq expression into meaningful components. Providing an accurate separation of RNA-seq data that is not based on In this study, we presented a method for classifying transcripts with low and high expression that promises wide-range application. The robustness of DAFS was demonstrated by applying DAFS to a number of RNA-seq data samples  that varied by sequencing depth, species, normalization, and density shape.Several datasets were used to the test the performance of our methodology.Part III of the Microarray Quality Control was an FDA-led, collaborate work to evaluate the technical performance of sequencing quality control (SEQC). In SEQC, reference RNA samples included the universal human reference RNA (UHHR) from Agilent/Stratagene (sample A) and the human brain total RNA (HBRR) from Life Technologies Corp. (sample B). External spike-ins were added to both samples for purposes of testing validation and assessing accuracy. In order to minimize sources of technical variance in our study, we restricted our analysis to gene-level Illumina RNA-seq data generated from one library preparation, processed on one lane, and obtained from one sequencing site. In total, our analysis included eight replicates of sample A and eight replicates of sample B.To demonstrate the ability of DAFS to handle various empirical data structures, four additional datasets were downloaded from the ReCount webpage. RNA-seqThe algorithm for carrying out DAFS on a single sample is comprised of several steps. Since zero reads present no information, the first step consists of removing the zero counts from analysis. After removing the zero counts and transforming the data to log2 scale, the Kolmogorov-Smirnov statistic is applied to segmented quantiles of the empirical distribution. Finally, the multivariate adaptive regression splines function is fit to the distribution of Kolmogorov-Smirnov statistics and spline knots are used to determine the optimal cutoff for high expression.The Kolmogorov-Smirnov (K-S) statistic is a quantitative measure of the maximum distance between the empirical distribution function of a sample and the cumulative distribution function of a reference distribution. In the present study, we rely on the K-S statistic to determine a cutoff for data attributed to the HE region by assessing agreement between the empirical distribution of assumed HE genes and the reference distribution. RNA-seq data after logarithmic transformation is approximately normally distributed. Thus, from a practical standpoint, we assumed a normal reference distribution for HE genes and used log2-transformed data to compute the optimal cutoff.X1, …, Xn are i.i.d. random variables from an unknown mixture distribution function. Let Yp = {Xi|Xi > Xp) was considered. For each sample, p0 is the proportion of data aggregated at the minimum expression level e.g. the percentage of 0 values when analyzing log2-transformed raw RNA-seq. The decision to exclude minimum expression levels was motivated by a desire to eliminate a proportion of the noise produced by extreme low counts. The stopping value is set at 0.50 to allow for at least half the data to be used in analysis. This seems fitting to adequately describe the normal mixture component of HE genes. For each ith observation of {p}, a corresponding K-S statistic is computed, denoted by Di,, i = 1, 2, …, length of {p}.where p0 to p = 0.90 is presented. Global analysis of the profile of K-S statistics demonstrates that the minimum K-S distance, i.e. the minimum distance between the observed distribution and theoretical normal distribution, occurs where the large normal distribution is clearly distinguishable. Empirical densities of other RNA-seq datasets (not shown) showed similar K-S profiles across percentiles of the data.A profile of the Kolmogorov-Smirnov distances for a sample taken from Nagalakshmi et al. is preseD}, across percentiles of the sample is used to classify the differentiation between data with low and high expression. Ideally, the data should be separated where {D} achieves a local minimum. To estimate the slope change points in Kolmogorov distances, multivariate adaptive regression splines (MARS)[f : p → D.The distribution of K-S statistics, {s (MARS) was usedD} and {p} be as previously defined, then D ⊂ Rp can be described by a general regression model,MARS is an adaptive nonparametric regression algorithm that uses piece-wise linear basis functions to model non-linear relationships. The predictor space is partitioned by knots into subregions to fit splines that distinguish segments of differing slopes. Let {ε denotes residual error. The MARS algorithm approximates f using recursive partitioning[f into linear combinations of basis functions and is denoted by a model of the form:where itioning,36 to exBm and αm are the mth spline function and its regression coefficient, respectively. Points that signify a break in the sample space and describe distinct linear splines are of key importance. These knots pinpoint a change in the model function, which indicate critical points of marked changed in K-S distances.where M is the number of spline basis functions and αm are stable through the use of linear basis functions of the form:The MARS algorithm uses a forward and backward stepwise selection algorithm to automatically determine the basis functions and set of knots or breakpoints to partition the predictor space. For a continuous variable, MARS defines the sub-regions under which spline coefficients andt is a spline knot selected from observed values of x. Friedman[where Friedman controlsFriedman:N is the number of observations and C(M) is the cost complexity function of M basis functions[where unctions. This crqc, is determined by the quantile p such that f has a local minimum. In the presence of multiple local minima, qc = min{qc} i.e. the left-most point indicating a decreasing-to-increasing change in slope. The MARS algorithm was performed using the ‘earth’ package in R.The final MARS model includes the minimum number of necessary knots to capture the true model. Thus, the selection of knots is used to identify critical points along the range of K-S statistics. The optimal quantile cutoff, R script to generate quantile cutoffs for a sample is provided in Additional filex1, …, xn be independent, identically distributed random observations with densityLet πk > 0 ∀ k = 1, 2 is the mixing weight of the kth Gaussian distribution,where kth Gaussian mixture with mean μk and varianceis the density of the is given by:σ = σ1 = σ2 and by:under an equal variance mixture model with v = σ2/σ1 when the variance terms are unequal. xevar or xuvar were used to determine the theoretical cutoff between expression abundance classes when two-component mixture modeling was used to fit the distribution of LE and HE regions.where Two simulation studies based on distributions of log2 transformed raw counts and log2 transformed RPKM data were proposed to demonstrate the ability of DAFS to differentiate LE/HE regions. For the first scenario, Mclust was used to fit the distribution of log2 raw counts of sample A from the SEQC study with no restrictions placed on the number of mixture components. The proportion, mean, and variance parameter estimates of the fitted 5 component mixture model wereg et al. and made0.85), 0.90 (Sen0.90), and 0.95 (Sen0.95). The total number of genes was fixed at 5,000 and the simulation was repeated 500 times.In the simulation of log2 raw counts (scenario 1), the first two components were treated as LE. In scenario 2, only the first component of log2 RPKM data was treated as LE. To evaluate DAFS’s performance with competing methodology, results of DAFS were compared to cutoffs determined by fixing values of sensitivity and by cutoffs determined by the point of intersection between the two theoretical distributions of a fitted two-component Mclust model. Several methods for selecting an optimal cutoff have been proposed based on specific underlying objectives e.g. cost-benefit analysis or diagnostic test accuracy measures . In this study, since the objective is to retain as many of the HE genes as possible, the decision criterion is based on setting sensitivity. Empirical cutoffs were computed for sensitivity values set to 0.85 (SenR script to generate quantile cutoffs for a sample is provided in Additional fileDAFS: Data-Adaptive Flag Method for RNA-Sequencing Data; NGS: Next-generation sequencing; RNA-seq: whole transcriptome sequencing; HE: High expressed; LE: Low expressed; RPKM: Log2-transformed reads per kilobase per million; FPKM: Fragments per kilobase per exon model per million mapped reads; EM: Expectation-maximization; SNL: Spinal nerve ligation; PPV: Positive predictive value; NPV: Negative predictive value; K-S: Kolmogorov-Smirnov; MARS: Multivariate adaptive regression splines; GCV: Generalized cross-validation.The authors declare that they have no competing interests.NIG performed the real data analyses and wrote the first draft of the manuscript. CWC conceived the study and performed the simulation study. Both authors participated in discussions related to analysis and interpretation and have read and approved the final manuscript.Supplementary materials.Click here for file


It has been shown that a random-effects framework can be used to test the association between a gene’s expression level and the number of DNA copies of a set of genes. This gene-set modelling framework was later applied to find associations between mRNA expression and microRNA expression, by defining the gene sets using target prediction information.Here, we extend the model introduced by Menezes et al. 2009 to consider the effect of not just copy number, but also of other molecular profiles such as methylation changes and loss-of-heterozigosity (LOH), on gene expression levels. We will consider again sets of measurements, to improve robustness of results and increase the power to find associations. Our approach can be used genome-wide to find associations and yields a test to help separate true associations from noise.We apply our method to colon and to breast cancer samples, for which genome-wide copy number, methylation and gene expression profiles are available. Our findings include interesting gene expression-regulating mechanisms, which may involve only one of copy number or methylation, or both for the same samples. We even are able to find effects due to different molecular mechanisms in different samples.Our method can equally well be applied to cases where other types of molecular  data are collected, such as LOH, SNP genotype and microRNA expression data. Computationally efficient, it represents a flexible and powerful tool to study associations between high-dimensional datasets. The method is freely available via the SIM BioConductor package.The online version of this article (doi:10.1186/s12859-016-0926-8) contains supplementary material, which is available to authorized users. The wealth of omics data being currently produced raises the need for efficient and flexible models to analyse these data. One typical objective is to understand which molecular changes affect gene expression levels or, if available, protein expression levels. Molecular profiles reliably measured currently include DNA methylation and copy number, sequence information including SNP and loss-of-heterozigosity (LOH) information, as well as microRNA expression levels. All these are known to be involved in gene expression regulation.Many methods have so far been proposed for finding associations between two omics data sets (for an overview see ). These Methods have also been proposed for the joint analysis of microRNA expression and mRNA expression profiles. In this context, many microRNAs can target the same mRNA, and one microRNA can target multiple mRNAs. So it makes sense to consider target prediction information while looking for associations between mRNA and microRNA expression levels. van Iterson et al.  use the With multiple types of molecular profiles measured, it makes sense to consider methods for analysing all of them together. Only a few of the integrated analysis methods so far proposed have been extended to handling more than two data sets. Methods proposed by Waaijenborg et al.  and by WHere we extend the integrated analysis method proposed by Menezes et al.  to handlcovariate sets instead of gene sets, essentially because sets may contain any set of variables, and need not refer to a gene. In the “This paper is organized as follows. In the “Yni the expression for gene i, and by Xnj the genomic copy number for gene j, where Yi},{Xj} are available per sample, although it is not necessary that both sets of probes correspond to the same loci. Then we write, for any given i, Menezes et al.  have proh(.) is a given inverse link function and Yi for a single gene indexed by i, but we omit the index i for clarity.where mentclass2pt{minimY and {Xj} is statistically significant, for each gene expression probe Y. This is done by making use of the global test =tr(RV) and, similarly, E[Q(S)]=tr(SV). A detailed derivation is given in Section 2 of the Additional file Y and X,Z separately. It then follows that a test statistic to test the hypothesis H0: σ=τ=0 can be obtained by using U(θ0)tU(θ0), leading to the expression where Zt. From  we know X. For simplicity, we may ignore the centering and the squaring, then write the unscaled test statistic for two covariate sets as Note that this is not the score test statistic, although it is derived from the score vector. This is similar to the suggestion in Goeman et al.  of usingXXt+ZZt is the matrix in the quadratic form that would have been obtained if our model had a single set of covariates given by the merged set {X,Z}, and with effect modelled by a single vector of random effects. In such a case, the design matrix would have been obtained by binding the columns of X and Z together and, thus, the unscaled score test statistic would be given as above.It is interesting to note that, from , \documσ=η∗τ, where η is a given constant, and deriving the test in terms of τ only. The choice of η is free but must be made a priori, each choice resulting in a test with different power properties. It is intuitive to see that this is possible, and that the derivation used here corresponds to choosing η=1, by the equivalence to a test for association with a single set of covariates.The same test statistic  could haX and Z is equivalent to the one obtained for a single covariate set formed by {X,Z}.Thus, a test statistic for two covariate sets can be obtained as the sum of the individual (centered) test statistics per covariate set, as in . SimilarQ(X),Q(Z) may be used, especially when covariates take values in different ranges, are of very different sizes and/or display different variances. In such cases, centering and scaling can ensure the separate sets are given the correct weights in the combined test statistic Q. This can be done by using the scaled individual test statistics as in  where W is a matrix formed by the columns of X,Z bound together. As such, the test statistic proposed considers not only correlation between the covariates and the dependent variable Y, but also pairwise correlations between covariate pairs, as shown by Goeman et al. }1/2. The amount of correlation between Xk and Zk can be changed by simply changing Var=0.83 if Var(Wk)=1 and Cor=0.56 if Var(Wk)=5, for all k, whilst Cor=0 for j≠k, although observed values may vary due to randomness.Here we assume that Zk≡−Xk+Wk, so that Xk,Zk have a negative correlation. For this case, we will take Var(Wk)=1, so that Cor=−0.83. Note that the {Xj} are uncorrelated, as are {Zk}, in all three cases.We will also simulate data with the same assumptions as above, but now take X and Z are positive and strongly correlated . The biggest difference in power is observed in the situation where X affects the response, followed by the split-samples case. This is surprising, but not entirely unexpected. Indeed, in this case Q(Z) brings information about Q(X) and thus, when their sum is used for testing, the power to find effects increases the most.ROC curves produced for each different type of effect/region suggest that most power is typically yielded when the covariate sets GGT7 and GDAP1L1, both on 20q. Expression of both genes was found to be associated with CN as well as ME in our analyses .In practice, correlation structures can be considerably more complex than assumed in the previous subsection, so we here consider two empirical correlation matrices, derived from the TCGA colon cancer data studied in the subsection “N=100). However, when effects are either additive or multiplicative, then power to find effects is greater if the covariate sets are positively correlated, as is the case with GDAP1L1. Conclusions remain the same for N=50,200 (data not shown).The ROC curves suggest that power achieved is comparable in the two cases, for situations where only one of the covariate sets is correlated with the response, or where the covariate sets are correlated to different subsets of observations for the response Fig. , N=100. We conclude that the test statistic  yields aQ given by (Q(X),Q(Z) can help elucidate the relative effects of molecular mechanisms under study on the response. To illustrate this, we will consider molecular profiling  data for colon (N=125) and breast (N=173) cancer samples, extracted from The Cancer Genome Atlas (TCGA). For details about the data, see Section 2.1 of Additional file In our simulation studies we have shown that the joint test statistic given by  can testcolon N=15 and breYi represents the ith gene expression probe, {Xj} represents the set of copy number probes that are within 1 Mb in either direction of the transcription start site of the gene, and {Zk} represents the set of methylation probes that are within 50 Kb of the gene’s transciption start site in either direction. These window sizes are arbitrary and reflect current knowledge on the distance of cis–regulatory effects of copy number  and andYi re number , which we will refer to as the joint test, as well as for the individual test statistics Q(X),Q(Z), which will be referred to as the CN test and the ME test, for association between DNA copy number or methylation with the gene expression. In all comparisons, we use p-values not corrected for multiple testing, as the comparisons between different test results can be more reliably done in this way . Associations for which p≤0.001 were selected. In all cases, empirical p-values distributions were verified to be enriched with small p-values, so that in none of the cases does the set of mRNA probes selected have small p-values entirely due to chance. Indeed, if we consider the number of tests done per chromosome arm , the ratio of new discoveries with the joint test, points out clearly that large fractions of joint test new discoveries arise, when compared with the ME test Fig. . This waThe overlap ratios show that, as expected, a relatively large proportion of mRNA probes selected by joint tests was also selected by the CN test  selected joint tests overlapping with selected CN tests for colon cancer. For breast cancer, selected joint tests corresponded to selected CN and ME tests in precisely 50 % of the cases each, with no overlap between selected CN and ME tests , suggesting that methylation effects may drive gene expression regulation on 19q, for both cancers. Indeed, methylation effects dominated the joint test statistic value, although copy number effects were still found for genes not found to be associated with methylation change  with these two tests. In contrast, in the colon cancer data only one probe would be selected with a less stringent cut-off, by the methylation and the joint tests (p ≤ 0.005). Indeed, methylation and gene expression were associated in both data sets, although the correlation was more widely and evenly spread across both methylation as mRNA-expression probes , compared with the colon cancer data  that was selected by the joint test, whilst that was not the case with either of the individual tests  on the diploid samples.GDAP1L1, is interesting because copy number also separated samples into diploid and copy gain, but now copy gain samples are under-expressed compared with diploid ones .After observing that methylation plays an important role in gene-expression regulation in breast cancer, we also looked for genes that had, for the ME test, r cancer . Of the The above mechanisms illustrate effect types that can be found by our approach, squeezing relevant information out from thousands of mRNA probes, copy number and methylation measurements and yielding a list of mRNAs for further investigation. Obviously, correlation does not mean causation: further research would be needed to verify which of the associations found indeed is functional. In particular, statistical significance need not lead to biological relevance: as with any statistical method, there are effects found of too small a magnitude to be biologically relevant.We have proposed a test to find associations between a dependent variable and two or more sets of covariates. It can be used for example to test for associations between a gene’s expression levels with copy number and/or methylation in a region around it. These help us better understand molecular mechanisms of gene expression regulation, individually and jointly, as we showed with the colon and breast cancer TCGA data.p-values. Indeed, multiple testing correction such as FDR may for example yield the same corrected p-value for many hypotheses, and it often does, even if uncorrected p-values are distinct. In practice, obviously multiple testing correction must be used if the test is applied many times.In our illustrations, we have not corrected for multiple testing, even though many hypotheses were tested simultaneously. This was done for the sake of clarity: our illustrations involved comparing results from different data sets, and such comparisons are better made on the basis of uncorrected Other methods have been proposed to look for associations between one molecular profile, and two or more other profiles. One framework that has been used by various authors for the two-dataset case (one independent) and, by some, has been extended to the three-dataset case, is that of penalized canonical correlations , 6, 15. cis, in and around the gene are of interest. We could have also considered trans effects by simply expanding the set. To study gene expression-regulating mechanisms by microRNA expression, one could form a set of microRNA (families) that may target that gene. To study eQTLs affecting a gene’s expression, natural sets are all SNPs in haploblocks mapping to a 2 Mb region centered around the start of the gene [In contrast, our approach is meant to be used when researchers have given sets of interest, which is is often the case in practice. Here we studied gene expression-regulating mechanisms by copy number and methylation, and used sets of measurements is the gene . Our appBayesian approaches have also been proposed to analyse two datasets (one independent) together –19, but Our approach is computationally simple, so it can be scaled up to the whole genome and be used with many covariate sets. The simplicity results from the focus on testing, rather than on model fitting, as do Bayesian approaches, or on finding canonical vectors. This allows for quick, simple and objective evaluation of results, and easy prioritization of found associations.Vaske et al.  proposeda priori of interest. The lack of imposed structure makes it more flexible. Indeed, our approach could be used on the matrix of inferred pathway activities generated by the approach of Vaske et al. [Our method is less structured, and this enables us to find a wide variety of types of association between covariates and the response which, as we saw in the TCGA data example, may not be linked to any known pathway. So it is preferred in cases where no particular effect is e et al. , insteade et al. .In the TCGA data examples, we observed similarities and differences between how copy number and methylation regulate gene expression in these two cancer types. For some chromosome arms, such as 1q and 17q, proportions of discoveries are similar between the two cancers. This needs not mean that the same gene expression-regulating mechanisms are involved, but that genomic changes of the same type are involved at the same ratio. Also, copy number and methylation do not necessarily explain gene expression variation of the same genes, and the joint test finds additional associations compared to the individual tests. For other arms, such as 13q and 20q, there are clear differences between the genomic changes found to affect gene expression in colon and breast cancer.Some genes were selected to illustrate patterns found by the method, which showed some expression-regulating mechanisms driven by a combination of copy number and methylation changes. This is just the tip of the iceberg, with many other potentially interesting effects contained in the results. Researchers interested in further exploring our results can use the whole-genome results table provided , but that of course is very much dependent on the signal-to-noise ratio in the data sets.Our approach can also be used in a generalized-linear model context, where the dependent variables are related to the sets of covariates via a link function, such as the logarithm or the logit. This extension is straightforward, as we use the framework of the global test , 9. ThisAnother application of interest is to study the impact of changes in multiple molecular profiles on patients’ survival outcome, a variable formed by time from diagnosis to event and event information. This can be done by means of a Cox proportional-hazards model as an extension of the global test for this setting proposed earlier .We propose a method to find complex associations between multiple covariate sets, for example molecular profiles, and a response, which can be a single clinical variable or part of a molecular profile. Computationally efficient and flexible, our method can help unravel complex gene expression-regulating mechanisms.


A major challenges in the analysis of large and complex biomedical data is to develop an approach for 1) identifying distinct subgroups in the sampled populations, 2) characterizing their relationships among subgroups, and 3) developing a prediction model to classify subgroup memberships of new samples by finding a set of predictors. Each subgroup can represent different pathogen serotypes of microorganisms, different tumor subtypes in cancer patients, or different genetic makeups of patients related to treatment response.This paper proposes a composite model for subgroup identification and prediction using biclusters. A biclustering technique is first used to identify a set of biclusters from the sampled data. For each bicluster, a subgroup-specific binary classifier is built to determine if a particular sample is either inside or outside the bicluster. A composite model, which consists of all binary classifiers, is constructed to classify samples into several disjoint subgroups. The proposed composite model neither depends on any specific biclustering algorithm or patterns of biclusters, nor on any classification algorithms.The composite model was shown to have an overall accuracy of 97.4% for a synthetic dataset consisting of four subgroups. The model was applied to two datasets where the sample’s subgroup memberships were known. The procedure showed 83.7% accuracy in discriminating lung cancer adenocarcinoma and squamous carcinoma subtypes, and was able to identify 5 serotypes and several subtypes with about 94% accuracy in a pathogen dataset.The composite model presents a novel approach to developing a biclustering-based classification model from unlabeled sampled data. The proposed approach combines unsupervised biclustering and supervised classification techniques to classify samples into disjoint subgroups based on their associated attributes, such as genotypic factors, phenotypic outcomes, efficacy/safety measures, or responses to treatments. The procedure is useful for identification of unknown species or new biomarkers for targeted therapy. Recent advances in biotechnology have generated great interest in the development of statistical methods and data mining techniques to analyze massive amounts of biological and medical data for understanding biological processes, discovering new species, or identifying new biomarkers for safety assessment, disease diagnostics and prognostics, and prediction of treatment response, etc. For example, metagenomics utilizes DNA sequence data to detect and identify representative species in environmental and clinically relevant samples and to discover genes or organisms with novel or useful functional properties In clinical treatment, patients are heterogeneous due to differences in genetic pre-dispositions, lifestyle, and disease characteristics. Personalized medicine utilizes genomic predictors of target patient population for assignment of more effective therapies to ensure safety and avoid adverse events or unnecessary treatment Salmonella isolates in outbreak investigations. However, standard methods for serotype identification of strains are tedious and time-consuming Salmonella isolates could be deduced and predicted based on PFGE fingerprints. Thus, PFGE fingerprint profiling using data mining algorithms can potentially provide a possible alternative method for fast screening and identifying Salmonella serotypes.In food safety surveillance, serotyping of pathogen strains is usually the first important step for identification and characterization of In the aforementioned applications, the primary goal is to develop a class prediction model that can accurately identify population subgroups (cancer or strain subtypes) for new samples. There are three main aims: 1) classifying samples into distinct subgroups from large and complex unlabeled multivariate data, 2) characterizing the relationships among the subgroups identified, and 3) developing a prediction model to classify subgroup memberships of new samples by finding a set of predictor variables.Classification is the standard approach to developing a model for class prediction of new samples. Classification is a supervised analysis, in which each sample has a predefined class label. A classification model builds a mathematical function for predicting class memberships of new unlabeled samples by learning the relationships between the class memberships of samples and their attributes from the sampled data Cluster analysis is the standard data mining technique for identification of structures and patterns in the data by partitioning samples into disjoint subgroups and finding their relationships. There are hierarchical and non-hierarchical clustering algorithms. The hierarchical algorithm clusters the objects into a tree-like dendrogram Clustering techniques provide a global analysis of samples by partitioning samples with similar attributes in the same cluster. Each sample is assigned to one and only one cluster, based on all attributes. In many applications, such as gene expression experiments, functionally related genes may exhibit a similar pattern only in a subset of patients with certain medical conditions, not in all patients; also, some genes may involve more than one function or no function at all, and associate with more than one condition or no condition. A primary goal in these applications is to identify those subsets of co-expressed/co-regulated genes with associated subsets of samples with similar conditions. Cluster analysis cannot effectively identify the substructures between a subset of genes and a subset of samples. Biclustering analysis provides an approach to identify substructures in the sampled data. Biclustering techniques identify biclusters by simultaneously clustering both samples and attributes Both cluster and bicluster analyses are unsupervised analyses, in which samples do not have a predefined class label. These two methods are effective techniques for subgroup identification and characterization, but, are inefficient for subgroup prediction. Several supervised biclustering procedures have been proposed for classification of labelled sample datasets In this paper, we propose a composite modeling approach for subgroup identification and prediction via a bicluster analysis. The proposed approach combines an unsupervised biclustering technique to identify potential sample subgroups in the first step, and a supervised classification technique to predict sample subgroup memberships in the second step. The proposed composite model neither depends on any specific biclustering algorithm or patterns of biclusters, nor on any classification algorithms. Any biclustering methods can be used in the first step of bicluster identification. This paper uses a SVD-based biclustering algorithm to identify constant biclusters Consider a two-way data matrix with rows representing the measured attributes and columns representing samples. Many singular value decomposition (SVD) approaches for bicluster analysis of microarray data have been proposed and demonstrated to be effective 1, C2,…,Ck}. Each bicluster Ci consists of a subset of samples Si that have similar attributes Gi . Thus, each Si represents a subgroup in the sampled population. A subgroup-specific binary classifier mi can be built to determine whether or not a sample s with the attribute g is in the associated subgroup Si, that is, mi(gi|G) = I{s ∈ iS}, where I is an indicator function  with the corresponding subgroups . For example, the outcome  of the composite model implies that the sample is in S1,  implies that the sample is in S3 and S4, and  implies that the sample is not in any of the five subgroups. For k binary classifiers, there are 2k possible patterns of predictive outcomes. Each pattern represents a subgroup. However, when k is modest or large, many patterns would contain very few samples or no samples at all. When the number of patterns is large, a minimum of n* = 5–10 samples may be set as the criterion to form a (major) subgroup for further analysis. The patterns that contain less than n* samples are referred to as minority subgroups.For a given sample Binary classifiers can be developed using any classification algorithms. This paper uses the three well-known algorithms: support vector machine (SVM) In the development of a classification model, the most important consideration is to unbiasedly evaluate its “performance”. The common measures of performance are sensitivity (the proportion of correct positive classifications out of the number of true positives), specificity (the proportion of correct negative classifications out of the number of true negatives), and accuracy . Procedures with both high sensitivity and high specificity will have high accuracy. To obtain unbiased estimates, the current sampled data are divided into a training set and a separate test set A simulation experiment was conducted to illustrate the proposed approach using asynthetic dataset of size 300 (rows)×100 (columns). The dataset consisted of two main bicluster regions with the size of 50×50 having 10 overlapping columns. The first main bicluster consisted of rows 1–50 and columns 1–50, and the second bicluster consisted of rows 51–100 and columns 41–90. The remaining columns 91–100 were in neither biclusters. The bicluster  data were generated from the normal distribution N  and background data were generated from the normal random variable N . For masking purpose, random signals were also generated in the first 100 attributes for the last 10 samples. This dataset can be summarized as four biclusters as follows. The columns represent 100 samples consisting of 4 subgroups: S1 , S2 , S3 , and S4 ; the first 100 rows represent attributes: G1 (rows 1–50), G2 (rows 1–100), G3 (rows 51–100), and G4 (rows 1–100) .1, C2, C3, and C4 were 100×16, 50×51, 50×58, and 100×15, respectively. Three classification algorithms were then used to develop four binary classifiers m1, m2, m3, and m4. There were 16 possible patterns.Applying the SVD-based biclustering method 1S, 3S, and 2S, respectively. The sensitivity and specificity are shown in the last two rows. The overall accuracy is 0.90. All the subgroups 1–SS3 were identified correctly. A test dataset consisting of 1,000 samples were generated for performance evaluation. The four subgroups were generated according to the probabilities 0.4, 0.1, 0.4, and 0.1 in contrast to the training set where the numbers of four subgroups were fixed at 40, 10, 40, and 10. The sensitivity and specificity for the 1,000 simulated samples were calculated for each of the four subgroups. The procedure was repeated 1,000 times. The averaged sensitivity and specificity over the 1,000 repetitions were shown in 1S and 3S samples. 2S has the attributes across two subgroups 1S and 3S. 4S was designed to have indefinite attributes and difficult to be identified. The pattern corresponding to 2S is  using SVM and DLDA, and the pattern is  using RF. SVM appears to perform slightly better than RF and DLDA. For 4S, as expected, the sensitivity is low in all three algorithms.A public lung cancer microarray dataset was used to evaluate the performance of the proposed procedure and compare with k-means cluster analysis. The dataset was from a study (GSE3141) of using gene expression signatures to identify patterns of oncogenic pathway deregulation in lung cancer subtypes 1, m2, m3} was built based on the three biclusters. The leave-one-out cross (LOU) validation was used to classify each sample into one of the possible 8 subgroups.The proposed approach was performed on the data matrix of 100 genes by 111 samples. The bicluster analysis identified 32 biclusters. A cutoff of at least 10 samples was used to eliminate small biclusters, such as sizes of 2×2 or 2×3, resulting in 3 clusters . The siz1 generated from the bicluster C1 appears to be associated with the SQ subtype. Note that the classifier m3 by itself or in combination with m1 and m2 assigned none samples in a subgroup. That is, all samples, including 10 samples from C3 were not in C3, as predicted by m3. Based on the majority rule, SVM, RF, and DLDA correctly identified 41, 40, and 42 out of the 52 AD subtypes, respectively. All three algorithms identified 52 out of the 58 SQ subtypes. The performance between the composite models and 2-mean are generally similar. The 52 SQ subtypes identified by the 2-means and by the three composite models are identical. The 42 AD subtypes identified by 2-means contained those 41, 40, and all 42 ADs identified by the SVM, RF, and DLDA composite models, respectively. Using n* = 5, SVM identified two subgroups, including 39 AD and 52 SQ subtypes; RF identified two subgroups of 38 AD and 50 SQ subtypes; DLDA identified four subgroups with 42 AD and 52 SQ subtypes.In the 4-means analysis, Groups 0 and 1 were from the split of Group 0 in the 2-means analysis, and Groups 2 and 3 were from the split of Group 1. However, the results of the 3-means analysis were peculiar. For example, there were 32, 14, and 7 adenocarcinomas for Groups 0, 1, and 2, respectively. Comparing with the 4-means analysis, the 32 consisted 21, 9, and 2 from Groups 0, 1, and 2, respectively; similarly, the 14 consisted of 12 and 3 from Groups 0 and 2, respectively.The dataset of van’t Veer et al. 1 and m2, were developed. The leave-on-out cross validation analysis divided the 97 patient into 4 subgroups. 1 classifier identified low risk group patients and m2 identified high risk group patients. Two biclusters with dimensions of 45×27 and 13×18 were identified from the 6391 genes and 97 patients . Two patSalmonella isolate dataset consisted of 45,924 PFGE isolates covering 32 mostly encountered serotypes published by Zou et al. The 1–m10) from the training dataset. The SVM and RF composite models then were applied to each training sample; 16 patterns were identified. The SVM model identified 16 patterns. Based on n* = 5 as a cutoff, 13 subgroups were identified (9 and m10 by itself or in combination with other classifiers assigned no samples into a subgroup. The interpretation (and presentation) of the performance was based on the known serotypes; in the analysis, the serotype was determined by majority rule. The sensitivities were from 86% to 99%, specificities were high at least 99%, except the Typhimurium with 93.2%. The overall accuracy was 94.2% and specificity was 98.2%.The data were first randomly divided into a training and a test dataset for each serotype. The bicluster analysis identified 10 biclusters and built 10 binary classifiers  was 94.2% and specificity was 98.2%.The RF model identified 15 patterns, one less than the SVM model . The difThe SVM and RF composite models were applied to the test dataset, which included 1,000 additional samples (named “Decoy”) from the serotypes other than the five training serotypes. The analysis of the test dataset classification described below is for the SVM composite model, the results for the RF composite model are given in 1–m10. Based on the n* = 5 as a cutoff, 14 subgroups were identified , ; Hadar: m6, ; Oranienburg: m4, m5, , , ; Thompson: m3; Typhimurium: m2.The sensitivities between the training and test datasets were similar for the data of the five training serotypes. The overall specificity was lower since there were 1,000 additional “Decoy” isolates . Salmonella surveillance and outbreak investigation Salmonella isolate is essential. The same serotype may have different subtypes, such as Salmonella Newport, and Dublin etc. These subtypes are closely related with their gene composition and variations. Current routine serotyping methods cannot provide sufficient information for subtype classification. The serotype subtype classification is important for the studies of genetic diversity and evolution. The composite model not only contributes to the PFGE-based characterization and surveillance of Salmonella isolates in outbreak investigations, also provides a better understanding of Salmonella genetic diversity and epidemiology.Currently, PFGE is routinely used molecular subtyping method by CDC (Centers for Disease Control and Prevention) and state health labs in the US for Cluster analysis has been the primary data mining technique for dividing samples into disjoint subgroups where the samples in a cluster contain all attributes that characterize the cluster. Bicluster analysis techniques are being developed to identify which subsets of attributes are associated with which subsets of samples In the analysis of the lung cancer and PFGE datasets, Biclustering algorithms have been extended to supervised biclustering classification for labelled sampled data 4, C5, and C6, to identify (discriminate between) these subtypes. For example, the two biclusters C5 and C6 in combination identified seven Oranienburg isolates. In addition, the bicluster C9 and C10 were not shown in any of the 14 patterns.In the proposed approach, a binary classifier is developed to predict whether or not a sample is in the associated bicluster. For the samples that are assigned into two or more biclusters, the composite model will separate those samples into a new subgroup. Some classifiers, either by itself or in combination with other classifiers, may assign only a small number samples, or none, into a subgroup. The PFGE analysis appeared to support some comments of Odibat and Reddy k disjoint subgroups. Those small biclusters are too small to be considered as representative subgroups for further partition. The composite model assigns each sample to one and only one subgroup, including those samples in the small biclusters. In the lung cancer example, the composite model was composed of three binary classifiers from three “large” biclusters of at least ten samples, out of the 32 biclusters identified. These three binary classifiers could generate up to 8 subgroups. However, only three subgroup patterns were identified. The smallest subgroup  contained only two samples  can be used to develop a composite model. However, classifier developed by a small bicluster is likely to predict that the samples are outside the bicluster. This problem is known as class-imbalanced classification In this paper, a minimum of five samples is recommended, n* = 5. In the lung cancer example, three biclusters with sample sizes of 40, 22, and 10 were used to generate subgroups. Cluster C1–C4 were identified. The sizes of C1–C4 were 100×16, 50×51, 50×58, and 100×15, respectively. Samples 1–50 and samples 41–90 were in biclusters C3 and C2, respectively; and samples 41–50 appeared in all four biclusters C1–C4. The notion of the composite modeling approach via biclusters for class prediction is intuitive and straightforward. For a given bicluster, a sample is either inside or outside the bicluster. There are k predicted outcomes for each sample. Each predicted pattern represents a subgroup. In the simulation experiment, four biclusters CThe three classification algorithms, SVM, RF, and DLDA, are considered for the development of a composite prediction model. The SVM and RF have been the most popular and successful classification algorithms and applied to numerous areas of applications. These two algorithms can be applied to high dimensional data without feature selection. DLDA is a variant of the Fisher’s linear discriminant analysis. DLDA has been shown to be robust against imbalanced class size data Personalized medicine is the goal of much current research. A general aim is to identify a set of molecular biomarkers that can match disease of an individual patient with an optimal therapy. Several procedures have been proposed utilizing the classification and regression tress Figure S1The prediction model divided the 97 patients into four subgroups using RF. The logrank test for differences among the four subgroups , , , and  was 0.717.(TIF)Click here for additional data file.Figure S2The prediction model divided the 97 patients into four subgroups using DLDA. The logrank test for differences among the four subgroups , , , and  was 0.186.(TIF)Click here for additional data file.Table S1Upper panel. Frequency distributions of classification patterns identified by the RF composite model  for the synthetic training dataset consisting of 4 subgroups, S1, S2, S3, and S4. Lower panel. Performance of the RF composite prediction model for the test dataset of 1,000 simulated samples. Table values are the averages over 1,000 repetitions.(DOC)Click here for additional data file.Table S2Upper panel. Frequency distributions of classification patterns identified by the DLDA composite model  for the synthetic training dataset consisting of 4 subgroups, S1, S2, S3, and S4. Lower panel. Performance of the DLDA composite prediction model for the test dataset of 1,000 simulated samples. Table values are the averages over 1,000 repetitions.(DOC)Click here for additional data file.Table S31–m10) for the Salmonella PFGE training dataset consisting of five serotypes. Sixteen classification patterns are identified; 13 of the 16 have frequencies of at least 5 (last column). The last two rows show the sensitivity and specificity of the model performance.Frequency distributions of classification patterns identified by the RF composite model (m(DOC)Click here for additional data file.Table S4Frequency distributions of subgroup patterns identified by the RF composite model for the Salmonella PFGE test dataset, which consisted of 5,055 isolates from five training serotypes and 1,000 additional “Decoy” isolates. The serotypes I4,(DOC)Click here for additional data file.


K-nearest neighbor, C4.5, naive Bayes, and CART.The classification of different cancer types owns great significance in the medical field. However, the great majority of existing cancer classification methods are clinical-based and have relatively weak diagnostic ability. With the rapid development of gene expression technology, it is able to classify different kinds of cancers using DNA microarray. Our main idea is to confront the problem of cancer classification using gene expression data from a graph-based view. Based on a new node influence model we proposed, this paper presents a novel high accuracy method for cancer classification, which is composed of four parts: the first is to calculate the similarity matrix of all samples, the second is to compute the node influence of training samples, the third is to obtain the similarity between every test sample and each class using weighted sum of node influence and similarity matrix, and the last is to classify each test sample based on its similarity between every class. The data sets used in our experiments are breast cancer, central nervous system, colon tumor, prostate cancer, acute lymphoblastic leukemia, and lung cancer. experimental results showed that our node influence based method (NIM) is more efficient and robust than the support vector machine,  Cancer research is one of the major research areas in the medical field. In cancer, cells divide and grow uncontrollably, forming malignant tumors and invading adjacent parts of the body. The cancer may also spread to more distant parts of the body through the lymphatic system or bloodstream. Many things are deemed to increase the risk of cancer, including tobacco use, dietary factors, certain infections, exposure to radiation, lack of physical activity, obesity, and environmental pollutants. The famous Apple founder Steve Jobs also died of pancreatic cancer. Any method which benefits cancer treatment should receive sufficient attention.The biggest challenge facing cancer treatment process is a means of developing individualized treatment programs for specific tumor types. Traditional diagnosis of cancer depends on the type of tissue-derived tumor cells, cell morphology, and protein markers, and biological behavior does not adequately reflect the real situation of the tumor; it is sometimes difficult to make a correct diagnosis of forecasts.In order to gain a better insight into the problem of cancer classification, systematic approaches based on global gene expression analysis have been proposed , 2. The 1) High Dimension. Each species genome is composed of a nucleotide sequence encoding a protein and nonprotein coding; the former is the traditional sense of the gene, which is a potential gene. Usually, the number of genes is the total of both. The number of genes in the human genome is approximately 30000. The dimension of the data is so high, brought great difficulties to the analysis of the experimental results. For example, used in our experiments the maximum dimension of a data set is up to 24481 in breast cancer  range in each dimension. Only in this way can we make meaningful comparisons in later steps. ConsiderStep 2.  Select the appropriate distance metric function based on the actual problem background. If there is no prior knowledge, we recommend using the Euclidean distance. ConsiderStep 3.  Set the only parameter δ; calculate the similarity between every two samples to construct the similarity matrix. ConsiderStep 4.  The training set and test set are treated as a non-negative weighted undirected network. That is, each sample in the training set or test set is treated as a node in a graph. The similarity obtained in Step 3 for every two samples is treated as the weight of the edge connecting the two corresponding nodes. Then we obtain the adjacency matrix for the whole cancer samples. ConsiderStep 5.  Calculate the node influence centrality of each training sample node, and treat it as the weight. Considerxa is an arbitrary element in set {Xtrain, Xtest}. ConsiderStep 6.  Calculate the similarity between every test sample and each class. ConsiderStep 7.  Classify each test sample to the class with highest similarity. ConsiderSimilarity Matrix is used twice in seven main steps of NIM1. The first is located in Step 4, in order to obtain the adjacency matrix. The second is in Step 6, in order to calculate the similarity between every test sample and each class. We believe in two steps used in different similarity matrix, resulting in node influence based method 2 (NIM2). Only two main steps of NIM2 are different from NIM1, as shown below.Step 3. Set the parameter δ1; calculate the similarity between every two samples to construct the similarity matrix. ConsiderStep 6. Set the parameter δ2; calculate the similarity between every two samples and then obtain the similarity between every test sample and each class. ConsiderWe use 6 data sets to validate NIM1 and NIM2. Below are six publicly available gene expression data from DNA microarray that are widely used by researchers for cancer classification experiments. All the data sets are used to predict various kinds of cancers by measuring gene sequences and are outlined in The first data set is breast cancer . The traThe second data set is central nervous system . SurvivoThe third data set is colon tumor . It contThe fourth data set is prostate cancer . The traThe fifth data set is acute lymphoblastic leukemia . The datThe sixth data set is lung cancer . It is aIf the dataset has not been divided into training set and testing set, we adopt leave-one-out cross validation (LOOCV) to validate NIM1 and NIM2. LOOCV involves using a single observation from the original sample as the validation data and the remaining observations as the training data. This is repeated such that each observation in the sample is used once as the validation data. This is the same as a K-fold cross-validation with K being equal to the number of observations in the original sampling.k-nearest neighbor (KNN) [Most proposed cancer classification methods are from the statistical and machine learning area, ranging from the old nearest neighbor analysis to the new support vector machines. There is no single classifier that is superior over the rest. Some of the methods only work well on binary-class problems and are not extensible to multiclass problems, while others are more general and flexible. The methods we choose for comparing are all top 10 algorithms in data mining, mentioned in . They aror (KNN) , C4.5 [1or (KNN) , naive Bor (KNN) , and CARor (KNN) . And we or (KNN)  for the Due to high dimension, small sample size, and nonbalanced distribution, traditional classification algorithms do not obtain high accuracy in these data sets. From δ1 = δ2. So the results of NIM2 are at least as good as NIM1. From NIM2 is an improved version of NIM1 and has one more parameter. NIM1 can be viewed as a special case of NIM2 when δ, and NIM2 has only two parameters δ1 and δ2.The traditional classification methods usually tend to have many parameters need to be set before application. And the parameters are closely related to the performance. However, there is little information on how to set parameters, usually based on experience. So we try to propose an algorithm with as few parameters as possible. NIM1 has only one parameter δ in NIM1 is shown in δ1 and δ2 in NIM2 is shown in δ in the 3 data sets. Figures δ1, δ2 in the 3 data sets. From the experimental results shown in Figures δ1 and δ2 play an important role in the performance of NIM2.The parameter setting for m indicate the number of genes measured. Every cancer sample can be viewed as a point in m-dimensional space. And the set of cancer samples can be viewed as a graph (or network) in m-dimensional space.Graph is a powerful representation formalism that has been widely employed in machine learning and data mining. In order to gain deep insight into the cancer classification problem, we analyze the problem from graph-based view. Let In the method NIM1, after selecting the appropriate distance metric, the graph (or network) of all samples is created by computing the similarity matrix. Then the node influence of training samples is calculated. Treat node influence as weight; the similarity between every test sample and each class is obtained. At last, every test sample is classified according to its similarity between each class.δ1 = δ2. Both NIM1 and NIM2 can deal with binary and multiclass cancer classification. NIM2 is more time consuming than NIM1 but owns a higher accuracy.Furthermore, we also propose NIM2, which is an improved version of NIM1. NIM1 can be viewed as a special case of NIM2 when Due to high dimension, small sample size, and nonbalanced distribution, SVM, KNN, C4.5, Naive Bayes, and CART do not obtain high accuracy in these cancer gene data sets. From the experimental results in the 6 cancer gene data sets, it can be seen that NIM1 and NIM2 are more efficient than these traditional algorithms. At the end, we also discuss the parameters in both NIM1 and NIM2. The parameters play an important role in the performance of NIM1 and NIM2.


We propose an extension to quantile normalization that removes unwanted technical variation using control probes. We adapt our algorithm, functional normalization, to the Illumina 450k methylation array and address the open problem of normalizing methylation data with global epigenetic changes, such as human cancers. Using data sets from The Cancer Genome Atlas and a large case–control study, we show that our algorithm outperforms all existing normalization methods with respect to replication of results between experiments, and yields robust results even in the presence of batch effects. Functional normalization can be applied to any microarray platform, provided suitable control probes are available.The online version of this article (doi:10.1186/s13059-014-0503-2) contains supplementary material, which is available to authorized users. In humans, DNA methylation is an important epigenetic mark occurring at CpG dinucleotides, which is implicated in gene silencing. In 2011, Illumina released the HumanMethylation450 bead array  and largStudies of DNA methylation in cancer pose a challenging problem for array normalization. It is widely accepted that most cancers show massive changes in their methylome compared to normal samples from the same tissue of origin, making the marginal distribution of methylation across the genome different between cancer and normal samples -8; see ASeveral methods have been proposed for normalization of the 450k array, including quantile normalization ,12, subset al.  to evaluate them pointwise, like Functional normalization extends the idea of quantile normalization by adjusting for known covariates measuring unwanted variation. In this section, we present a general model that is not specific to methylation data. The adaptation of this general model to the 450k data is discussed in the next section. The general model is as follows. Consider which has the functional form α is the mean of the quantile functions across all samples, βj are the coefficient functions associated with the covariates and εi are the error functions, which are assumed to be independent and centered around 0.The parameter function In this model, the term represents variation in the quantile functions explained by the covariates. By specifying known covariates that measure unwanted variation and that are not associated with a biological signal, functional normalization removes unwanted variation by regressing out the latter term. An example of a known covariate could be processing batch. In a good experimental design, processing batch will not be associated with a biological signal.j=1,…,m, we form the functional normalized quantiles by In particular, assuming we have obtained estimates Yi into the functional normalized quantity We then transform This ensures that the marginal distribution of j=1,…,m. Our model Yi are high dimensional and continuous, the jumps of the empirical quantile functions are very small. This allows us to circumvent the smoothing approach used in traditional function-on-scalar regression. We use a dense grid of H equidistant points between 0 and 1, and we assume that H is much smaller than the dimension of Yi. On this grid, model qemp(r) have very small jumps, the parameter estimates of these linear models vary little between two neighboring grid points. This allows us to use H standard linear model fits to compute estimates j=1,…,m, with h being on the dense grid {h∈d/H:d=0,1,…,H}. We next form estimates j=1,…,m, for any r∈ by linear interpolation. This is much faster than the penalized function-on-scalar regression available through the refund package . These data were available to researchers under the following constraints: (1) the use of the data is limited to research on cancer, (2) the researchers have local Institutional Review Board approval and (3) the researchers have the approval of either Colon Cancer Family Registries  or MountData were available in the form of IDAT files from the various experiments (see above). We used minfi  and illuP value greater than 0.01, with the intention of excluding a sample if the percentage was higher than 10%. We used the minfi [P value. We also used additional quality control measures [We performed the following quality control on all data sets. As recommended in Touleimat and Tost , for eache minfi  implemenmeasures  and we imeasures ; all arrWe performed the following filtering of loci, after normalization. We removed 17,302 loci that contain a SNP with an annotated minor allele frequency greater than or equal to 1% in the CpG site itself or in the single-base extension site. We used the UCSC Common SNPs table based on dbSNP 137; this table is included in the minfi package. We removed 29,233 loci that have been shown to cross-hybridize to multiple genomic locations . The totet al. [M and U separately [We have compared functional normalization to the most popular normalization methods used for the 450k array. This includes the following between-array normalization methods: (1) quantile: stratified quantile normalization as proposed by Touleimat et al.  and implet al. , (2) dasparately  and (3) parately , implemeparately  and (5) parately . FinallyIn its current implementation, noob yielded missing values for at most a couple of thousand loci (less than 1%) per array. This is based on excluding loci below an array-specific detection limit. We have discarded those loci from our performance measures, but only for the noob performance measures. In its current implementation, BMIQ produced missing values for all type II probes in five samples for the TCGA-AML data set. We have excluded these samples for our performance measures, but only for our BMIQ performance measures.For clarity, in the figures we focus on the top-performing methods which are raw data, and quantile and noob normalization. The assessments of the other methods, dasen, BMIQ and SWAN, are available in Additional file M values obtained from the raw data. Surrogate variables were estimated using the iteratively re-weighted SVA algorithm [We used the reference implementation of SVA in the sva package . We appllgorithm , and werThe RUV-2 method was originally developed for gene expression microarrays . The metk of factors to remove .To determine the number nd Speed  for detand Speed . First, We used the reference implementation of ComBat in the sva package . BecauseF-statistics from a linear model of the beta values from the array. The linear model was applied on a probe-by-probe basis. In most cases, the model included case/control status as a factor. In the 27K data, we adjusted for batch by including a plate indicator (given by TCGA) in the model.To identify DMPs, we used in silico confounding of treatment (case/control status) by batch effects as follows. In a first step, we selected a set of samples to be the discovery cohort, by choosing samples where the treatment variable is not visibly confounded by plate effects. Then the validation step is achieved by selecting samples demonstrating strong potential for treatment confounding by batch, for example by choosing samples from different plates (see descriptions of the data). The extent to which it is possible to introduce such a confounding depends on the data set. In contrast to earlier work [in silico confounding between batch and treatment has been previously explored in the context of genomic prediction [To measure the consistency of each normalization method at finding true DMPs, we compared results obtained on a discovery–validation split of a large data set. Comparing results between two different subsets of a large data set is an established idea and has been applied to the context of 450k normalization ,46. We eediction .We quantified the agreement between validation and discovery in two ways: by an ROC curve and a concordance curve. For the ROC curve, we used the discovery cohort as the gold standard. Because the validation cohort is affected by a batch effect, a normalization method that is robust to batch effects will show better performance on the ROC curve. Making this ROC curve required us to choose a set of DMPs for the discovery cohort. The advantage of the ROC curve is that the plot displays immediately interpretable quantities, such as specificity and sensitivity.k DMPs from the discovery and the validation sets, and displayed the percentage of the overlap for each k. These curves do not require us to select a set of DMPs for the discovery cohort. Note that these curves have been previously used in the context of multiple-laboratory comparison of microarray data [For the concordance curve, we compared the top ray data .On the 450k array, 11,232 and 416 probes map to the X and Y chromosomes, respectively. Because some genes have been shown to escape X inactivation , we onlyFor the ROC curves, we defined the true positives to be the 1,678 probes undergoing X-inactivation and the probes mapping to the Y chromosome (416 probes); by removing the probes that have been shown to cross-hybridize , we weren=10,20,30,50 and 80, we randomly chose half of the samples from the EBV-transformed samples, and the other half from the lymphocyte samples. For instance, for n=10 samples, we randomly picked five samples from each of the treatment groups. We repeated this subsampling B=100 times, which generated 100 discovery–validation ROC curves for each n. For a fixed n, we considered the mean of the B=100 ROC curves as well as the 0.025 and 0.975 quantiles to mimic a 95% confidence interval.To assess the performance of functional normalization for different small sample sizes, we devised the following simulation scheme for the Ontario-EBV data set. First, we kept the discovery data set intact to ensure a reasonable gold standard in the discovery–validation ROC curves; we only simulated different sample sizes for the validation subset. For sample sizes A machine-readable document detailing our analyses is available at GitHub .Additional file 1Supplementary information. Supplementary Figures S1–S9 and supplementary material with a description of how control probes are treated.


Meta-analysis of gene expression has enabled numerous insights into biological systems, but current methods have several limitations. We developed a method to perform a meta-analysis using the elastic net, a powerful and versatile approach for classification and regression. To demonstrate the utility of our method, we conducted a meta-analysis of lung cancer gene expression based on publicly available data. Using 629 samples from five data sets, we trained a multinomial classifier to distinguish between four lung cancer subtypes. Our meta-analysis-derived classifier included 58 genes and achieved 91% accuracy on leave-one-study-out cross-validation and on three independent data sets. Our method makes meta-analysis of gene expression more systematic and expands the range of questions that a meta-analysis can be used to address. As the amount of publicly available gene expression data continues to grow, our method will be an effective tool to help distill these data into knowledge. The amount and continued growth of publicly available gene expression data are staggering. NCBI GEO and ArrayExpress currently have available more than 1.6 million samples spread across more than 54 000 studies . To takeP-value of differential expression. As a result, current methods struggle to select genes that each contribute non-redundant information and to systematically determine  how many genes to include in the set. Such methods are also difficult to generalize when comparing more than two conditions. Finally, meta-analyses whose goal is diagnosis or prognosis often need to account for additional variables, such as histological findings or patient characteristics. Unfortunately, incorporating covariates into a meta-analysis of gene expression is a problem that currently has no general solution.Despite those successes, current methods for meta-analysis of gene expression have several limitations. Often the goal of a meta-analysis is to obtain a small set of genes whose expression correlates with the variable of interest, such as healthy versus disease. Most current methods select genes based on univariate summary statistics, such as The elastic net , a generIn this work, we describe a methodological framework for using the elastic net to perform a meta-analysis of gene expression. To show how our approach addresses the limitations of previous methods, we perform a meta-analysis of lung cancer gene expression based on publicly available data. Our meta-analysis results in a robust and accurate multinomial classifier that distinguishes between four lung cancer subtypes using a small set of genes. Our method also enables us to rigorously demonstrate the value of a meta-analysis, in that training a classifier on multiple studies improves prediction compared to training a classifier on only one study.α = 0.9 for the elastic net penalty, where α = 0 corresponds to ridge  and α = 1 corresponds to lasso . Lower values of α led to a classifier with more genes, but with identical performance. We always set glmnet's intercept option to true, although setting it to false did not appreciably affect the results. When calculating accuracy, the predicted class for each sample was taken to be the class with the highest probability.We curated data from eight publicly available microarray studies of lung cancer (Supplementary Table S1). For each study, we used the information regarding each sample's cancer subtype, as well as any information regarding patient sex, age and smoking status, as provided. For GSE11969, smoking history was converted from Brinkman index (number of cigarettes per day multiplied by number of years of smoking) to either ‘current’ (if Brinkman index was greater than zero) or ‘never’ otherwise. For visualizing the samples using t-SNE, we used samples corresponding to cancer subtypes that were present in at least two studies. For our meta-analysis, we used only the samples in each data set that were histologically defined as adenocarcinoma (AD), squamous cell carcinoma (SQ), small cell lung carcinoma (SCLC), or carcinoid (CAR). The discovery data sets were selected to include a variety of microarray platforms and to have a sufficient number of samples of SCLC and CAR. Because the Bhattacharjee data set is very heavily biased toward AD, but also has samples from the other subtypes, we included only 60 of the AD samples. Altogether, the merged discovery data comprised 639 samples and 7200 genes that were present in all five discovery data sets. We used Each data set should first be curated to include the samples of interest and any information for each sample that the classifier will use, such as cancer subtype. Expression values in each data set are normalized and log-transformed . Raw Affymetrix data are normalized using RMA  and mappIn our framework for meta-analysis, first the data sets are merged, then the analysis is done . One chastandardize option is then set to false. If the distribution of classes (e.g. types of cancer) in the training set is representative of the expected distribution in the testing set, i.e. they are an accurate prior, then glmnet's intercept option can be set to true. If desired, additional variables can be added alongside the genes. Categorical variables can be incorporated as dummy variables. Continuous variables should be scaled to have the same mean and standard deviation as the gene expression data. By default, when training the classifier, the samples are weighted such that each study is weighted equally, although this can be adjusted.When we refer to the ‘elastic net,’ we mean the method of using the elastic net penalty to fit a generalized linear model (GLM), as implemented in the R package glmnet . Before β0 and β are the coefficients of the GLM, N is the number of observations (i.e. samples), iw is the weight of observation i, l is the negative log-likelihood contribution for observation i , λ is the regularization parameter (which controls the amount of shrinkage), α is the elastic net penalty , ∥β∥2 is the L2-norm of β and ∥β∥1 is the L1-norm of β. The weights are scaled such thatThe objective function of the elastic net takes the form of ‘loss + penalty’:M batches (i.e. studies), then to achieve equal weighting of each batch, we setin is the number of observations in the batch to which observation i belongs.If there are After merging the discovery data sets, cross-validation is performed to determine how the performance of the classifier depends on the regularization parameter of the elastic net. By default, our method uses leave-one-study-out cross-validation, although random n-fold cross-validation is also possible. Using leave-one-study-out cross-validation requires that each class has samples from at least two studies.Our procedure for validation is designed so that each validation data set is tested independently of the others. For each validation data set, all discovery data sets and the current validation data set are merged in the manner described above. Note that the set of genes in this merged data set may be slightly smaller than the set of genes in the merged discovery data set that was used for cross-validation. In particular, a gene in the classifier from cross-validation might not have been measured in the current validation data set. In addition, the inclusion of the validation data set may slightly alter the cross-study-normalized expression values. Therefore, after merging the data sets, a new classifier is trained on the samples from the discovery data sets using the regularization parameter obtained from previous leave-one-study-out cross-validation. Again, samples from the discovery data sets are weighted such that each study is weighted equally. The classifier is then tested on the samples from the current validation data set. The process is repeated for each validation data set.Our approach can be divided into three stages Figure . In the To demonstrate the utility of our method, we conducted a meta-analysis of lung cancer gene expression using publicly available data. Our goal was to use multiple studies to build a robust multinomial classifier containing a small set of genes that could distinguish between several lung cancer subtypes. Such a classifier and corresponding gene set could aid development of better diagnostic tools and could inform our understanding of the biology of the respective subtypes, but could not be built using existing methods for meta-analysis.We first curated eight publicly available data sets containing samples of various lung cancer subtypes (Supplementary Table S1 and –29). We We next merged only the discovery data sets and used the elastic net to perform leave-one-study-out cross-validation across a range of values of the regularization parameter lambda Figure . ImportaAt the value of the regularization parameter that gave the lowest multinomial deviance, the overall accuracy (fraction of correctly classified samples) of the multinomial classifier on cross-validation was 91.2% . Prediction accuracies for the four cancer subtypes ranged from 74% (SCLC) to 97% (AD). Using that value of the regularization parameter, we trained a classifier on all samples from the discovery data sets. The resulting classifier contained 58 genes Figure , the genTo further test our method, we validated the classifier on three independent data sets Figure . Our appWe compared the performance of the classifier trained on all the discovery data sets against that of classifiers trained on only GSE30219 or only the Bhattacharjee data set . For each of the two classifiers, the value of the regularization parameter was determined by 5-fold cross-validation on the samples of the respective data set. For the classifier trained on GSE30219, the overall accuracy on the validation data sets was 87.8%, only marginally worse than the performance of the classifier trained on all the discovery data sets (Supplementary Table S4). For the classifier trained on the Bhattacharjee data set, however, the overall accuracy on the validation data sets dropped to 76.1% (Supplementary Table S5). These findings exemplify the power of a meta-analysis to overcome the biases of individual studies.Although our approach to build a predictive model based on multiple data sets uses the elastic net, it can also be adapted to use other machine learning techniques. Within the context of our lung cancer meta-analysis, we compared the elastic net to the method of nearest shrunken centroids called PAM . SimilarWhen the goal of a meta-analysis of gene expression is to build a predictive model for diagnosis or prognosis, it is important to account for other possibly predictive variables in addition to gene expression. Because our method merges multiple data sets into a single matrix, additional variables, such as patient characteristics, can simply be appended as new features in the matrix. The elastic net, which can handle both continuous and categorical features, can then build a predictive model based on gene expression and the additional variables.To demonstrate the feasibility of including variables using our method, we performed a second meta-analysis of the four lung cancer subtypes. Samples from five data sets included patient sex, age and smoking status . Using those data sets, we performed leave-one-study-out cross-validation with and without those three variables. We expected that at least patient sex would be included in the classifier and would improve prediction, because men and women have different distributions of lung cancer subtypes. In particular, men have a lower relative frequency of AD and a higher relative frequency of SQ Figure ). SurpriWe hypothesized that any information contained in the patient variables might already be present in the gene expression. We noticed that one of the genes in the classifier Figure . In factOur methodological framework represents a significant advance in meta-analysis of gene expression. Rather than analyzing each data set separately and then combining summary statistics, our method performs cross-study normalization to merge the raw data and then analyzes the merged data using the elastic net. By expanding the reach of the elastic net to analyze multiple studies, our method offers several advantages compared to previous methods, including those that directly merge the raw data ,36. MostThe prediction accuracy of our multinomial classifier varied considerably across the four subtypes. Several factors could explain this behavior, in particular the relatively high misclassification rate for SCLC. The first is the small sample size for SCLC. Other potential factors include imperfect correction of batch effects and an inconsistent definition of SCLC across studies. In addition, our unsupervised analysis suggests that although most samples of SCLC cluster together, a number of AD and SQ samples have a pattern of gene expression that is similar to SCLC Figure . This maEven if prediction is not the primary goal of a meta-analysis, our approach is useful in generating a prioritized list of genes for further investigation. Many of the 58 genes in our meta-analysis-derived classifier of lung cancer subtypes are known to be relevant to lung cancer. For example, expression of MUC1 is associated with patient outcome  and MUC1Batch effects are always a concern in biological data. Our method corrects for batch effects and performs cross-study normalization using ComBat. Although ComBat has worked well in our hands, one should be especially cautious when merging data sets that are extremely unbalanced, e.g. when one data set has samples of one disease subtype and a second data set has samples of a different subtype. In that case, neither ComBat nor any other algorithm can reliably distinguish batch effects from real differential expression. Whereas our method uses ComBat because the batch information is known, a method called frozen surrogate variable analysis (fSVA) is designed for prediction when the batch information is unknown . In the One caveat with our approach is that the microarray probes in each data set are mapped to Entrez Gene IDs and their expression is condensed to one value per Entrez Gene ID, before the data are analyzed. This step is necessary, in order to merge the data sets. The cost of this step, however, is that if one probe for a gene is differentially expressed but other probes for the gene are not, the signal from the one probe could be drowned out by the noise of the others.In the current implementation of our approach, the step of merging data sets involves excluding any gene not measured on each data set. Using only the intersection of genes may at first seem to be a severe restriction, especially as the number of data sets increases. The size of the intersection, however, is determined not by the total number of data sets, but by the number of unique microarray platforms. Given that the vast majority of publicly available gene expression data are based on a small number of platforms and that any given phenotype is typically associated with the expression of many genes, we believe that the current implementation will work well for most meta-analyses. In this respect, our deliberate choice to use the Bhattacharjee data set for discovery represents a near worst-case scenario. Excluding the Bhattacharjee data set, which was collected on the Affymetrix HGU95Av2 GeneChip, would raise the number of Entrez Gene IDs in the merged discovery data from 7200 to 13 609 . In the future, our method could be altered to impute the expression of genes that are present in some data sets but absent from others. Such imputation would be unlikely to significantly improve prediction, but would allow those genes to be included in the predictive model.Our approach both addresses the limitations of previous methods and expands the range of questions that can be addressed using meta-analysis of gene expression. As the amount of publicly available gene expression data continues to grow, our method will be an effective tool to help distill these data into knowledge.https://zenodo.org/record/16006. All computation is done in R.Software and instructions for performing a meta-analysis, as well as all data necessary to reproduce our meta-analysis of lung cancer, are available at Supplementary Data are available at NAR Online.