When we were asked for help with high-level microarray data analysis (on Affymetrix HGU-133A microarray), we faced the problem of selecting an appropriate method. We wanted to select a method that would yield "the best result"  as possible, without false positives and false negatives). However, life scientists could not help us – they use their "favorite" method without special argumentation. We also did not find any norm or recommendation. Therefore, we decided to examine it for our own purpose. We considered whether the results obtained using different methods of high-level microarray data analyses – Significant Analysis of Microarrays, Rank Products, Bland-Altman, Mann-Whitney test, T test and the Linear Models for Microarray Data – would be in agreement. Initially, we conducted a comparative analysis of the results on eight real data sets from microarray experiments (from the Array Express database). The results were surprising. On the same array set, the set of DEGs by different methods were significantly different. We also applied the methods to artificial data sets and determined some measures that allow the preparation of the overall scoring of tested methods for future recommendation.We found a very low level concordance of results from tested methods on real array sets. The number of common DEGs  ranged from 6 to 433 . Results on artificial data sets were better than those on the real data. However, they were not fully satisfying. We scored tested methods on accuracy, recall, precision, f-measure and Matthews correlation coefficient. Based on the overall scoring, the best methods were SAM and LIMMA. We also found TT to be acceptable. The worst scoring was MW. Based on our study, we recommend: 1. Carefully taking into account the need for study when choosing a method, 2. Making high-level analysis with more than one method and then only taking the genes that are common to all methods (which seems to be reasonable) and 3. Being very careful (while summarizing facts) about sets of differentially expressed genes: different methods discover different sets of DEGs. Microarrays are used to detect gene expression levels. Using this technology, we can simultaneously detect the expression levels of several thousand genes with one experiment . MicroarWhen we were asked for help with processing microarray data, we faced the problem of selecting an appropriate method. We were interested in finding a method that would yield "the best result". We found publications that provided comparisons of methods , 3, 4, 5We decided to examine six commonly accepted and widely used methods for detecting DEGs ,11. The Experiments were conducted using real data from eight microarray experiments . We found that the first results were surprisingly divergent. Thus, we decided to test the methods on artificially prepared data sets  with known outstanding values  to be detected.To obtain information about the types of microarrays and the principles of their operation, we referred to various sources , 13. FigAside from the usual steps that are common in most experiments—conception work, laboratory work (wet-lab) and closing work (blocks (1), (2), and (3) , S1 Fig Low-level data analysis (3a), where the intensity of fluorescence (raw data) is translated into numbers that reflect the fluorescence level for each probesetID for each microarray reading.High-level data analysis , where we exclude probesets without expression changes and select the highest level of data analysis with probesets that undergo expression changes.Highest-level data analysis (4), where annotation, pathway analysis, interpretation, reporting, and visualization take place.In this study, we were only interested in high-level analysis methods, with a special interest in DEGs detection. Because tested method starts with a data table of microarray reading , all factors concerning any biological or molecular mechanisms and tissue-specific questions were out of scope in this work. We started our examinations when we saw a normalized table with fluorescence levels for probesets and our goal was to determine which probesets represent genes with transcriptional activity chance . In addition, co-expression and pathway analysis was out of scope in our study; this can be done later, based on the results from the high-level analysis. As can be seen , all levIn short, high-level analysis can be classified as: (3 b) simple methods, which are mainly based on statistics, and (3 c) complex methods, which are based on artificial intelligence and discrete mathematics , 15, 16.In the comparisons presented in this paper, we only focused on simple methods. Below, we provide a short characterization of each of the examined methods.Significant Analysis of Microarrays (SAM) , 18SAM is a statistical method used to determine statistical significance in gene expressions between groups. In terms of mode of action, SAM reassembles a T test. However, SAM uses non-parametric statistics, due to the fact that microarray data are not normally distributed.Rank Product (RP) , 20, 14RP is a statistical method for detecting gene expression changes. It belongs to non-parametric statistical tests and is based on ranks of fold changes.Bland-Altman (BA) , 21BA analysis is a statistical method that allows the comparison of two groups of results. In addition to using BA on data from microarray experiments, it is also very popular in medical data analysis of medical data.Mann-Whitney (MW) MW is a non-parametric test used to test the conformity between two populations. The null hypothesis is that the two populations are identical. It is one of the most popular tests used to check the conformity between groups. One of its usages is to detect gene expression changes in microarray data.Test T (TT) TT is a statistical test that determines whether two sets differ from one another in a statistically significant way. This test is based on the average and variance of the population. It is one of the simplest and most frequently used statistical tests.Linear Models for Microarray Data (LIMMA) LIMMA is available as a BioConductor package for analyzing gene expression in microarray data. It uses linear models to analyze microarray data.We also examined the frequency of using certain methods in scientific papers by searching Google Scholar and PubMed .We searched the name of the method, along with (AND operator) two phrases. In the first search, we used “differentially expressed genes” and in the second, we used "gene expression".We compared the results of a high-level analysis for eight microarray (Affymetrix HG-U133A) experiments from ArrayExpress , 26. AllThe first necessary step was to conduct a low-level data analysis.In order to perform a low-level analysis, we used standardized Robust MultiChip Average (RMA) , 23 methFor the purpose of conformity verification, we carried out a high-level analysis of array sets. The parameters for each method are presented in These parameters remained intact throughout all of the testing procedures (both for Arraysets and Datasets).The number of DEGs detected by different methods is presented in From For each Arrayset, we decided to present the results as Venn diagrams. Because the visualization of common parts of more than five sets in the form of Venn diagrams is not trivial, we used the approach presented by . We presOn the basis of the presented Venn diagrams, a very low concordance of results between tested methods can be noticed. A large part of DEGs identified by one of the methods was not considered as DEG by other methods. For each Arrayset, the number of DEGs detected by each method can be read in From the analysis, it is clear the diversified concordance between DEGs detected by certain methods on Arraysets.σ = 1.3. Next, in the case of Dataset1, 73 elements were changed with values from mean = 15, σ = 1.3 (as up regulated aDEGs) to mean = 5, σ = 1.3 (as down regulated aDEGs). The range of certain values of exemplary artificial array readings are shown on the boxplot chart in st quartile, median value, 3rd quartile, and max value. The values which "expression" does not change are marked as “other”, up regulated aDEGs are marked as “up regulated”, and down regulated aDEGs are marked as “down regulated”. It can be seen that all ranges are in this case completely separate.Such a low agreement level surprised us. As such, we decided to test the methods on artificially generated data to determine whether the methods themselves  work properly. We prepared special artificial datasets, with a priori known outstanding in value elements—aDEG (to mimic DEGs). We prepared two datasets with 2,000 elements, generated as normal distribution with mean = 10 and σ = 1.3 to mean = 6.5, σ = 1.3 respectively . Range of certain values for Dataset2 (“other” together with up and down regulated aDEGs) is shown on Dataset2 had slightly outstanding values—ranging from mean = 13.5, Datasets are uploaded as Supplemental Materials; for more detailed information, refer to adeg_info.txt and ReadMe.txt file in Supplemental Materials. The results of aDEGs detection in both Datasets are presented in the next section.In the case of the Arraysets, it was difficult to assess which method gave the best results, because we did not know which (and how many) values should be detected as DEGs. Thus, we needed a procedure that would allow scoring the methods used for DEGs detection. The problem of detecting DEGs can be regarded as a problem of classifying (grouping) data into three groups: DEGs without change, DEGs up regulated and DEGs down regulated.TP), true negatives (TN), false positives (FP), and false negatives (FN). Therefore, we assessed the quality of methods using measures that were used in computer science during the classification algorithms evaluation. These measures were: accuracy (acc), recall (rec), precision (prec), f-measure (fm), and Matthews correlation coefficient (MCC). MCC interpretation is similar to ROC/AUC, but it is presented in the form of one number. Furthermore, it is regarded to be more stable when class (groups) sizes may be different. The first used measure was accuracy. Accuracy is defined as:In the case of the artificially prepared Datasets, we knew what number of aDEGs to detect. Thus, we could make an algorithm assessment. We determined all true positives . An accuracy value equal to 1 means that the tested algorithm only found the values that should be detected. An accuracy of 0 means that the algorithm has not found any of the values that it should.The second measure used was recall, also known as true positive rate or sensitivity. Recall is defined as:Recall describes how many values are correctly detected by the tested algorithms, in proportion to all of the values that should be detected. Algorithms give the best results when the recall is equal to 1 and the worst results when the recall is 0.The next measure used was precision. Precision is described as:Precision describes how many values are correctly detected by the tested algorithms. As in recall, the best results are given when the precision is equal to 1 and the worst results are given when the precision is 0.The next metrics was the f-measure, which is described as:fm = 1 the best, 0 the worst).F-measure is a kind of compromise between recall and precision. When we only use recall or precision, we are not able to decide which algorithm gives better results, detects less "unwanted" values or misses a small amount of that which should be detected. F-measure is usually used for choosing an algorithm with optimal ratio of precision and recall . This measure gives information similar to correlation, which inform us to what extent the values that are detected by the tested algorithm are similar to those we expected.Dataset1 and Dataset2 were examined with the same methods, i.e., SAM, RP, BA, MW, TT, and LIMMA (and the same parameters see ).For Dataset1, almost all of the methods detected at least 73 modified values. Furthermore, some recognized additional values. Only the BA method and LIMMA detected less than we expected (72 of 73). The distribution of values in Dataset1 is presented on the boxplot .A summary of values detected by all tested methods, as well as the measures for method evaluation, is presented in SAM and LIMMA had the best values in the case of four  measures. SAM had the highest values for acc, rec, fm, and MCC. LIMMA had the highest values for acc, prec, fm, and MCC. Moreover, SAM and LIMMA reached the same values for acc, fm, and MCC. However, they differed in rec and prec. SAM had the maximum value of rec parameter , which means that the algorithm had no false negatives. LIMMA had prec that equaled 1, meaning that it had no false positives. The worst results were obtained in the case of the MW method. Here, in four  measures, it had the worst values. Its measures had low values, so even maximum values for rec  would not have justified its usage. BA and TT reached relatively high overall scoring for all measures and the RP results were at an acceptable level.It appears that the Dataset1 tested methods worked correctly and were able to detect almost all of the values that should be recognized as aDEGs.Based on the experiment of Dataset1, we can conclude that the best algorithms for use are SAMM and LIMMA (ex-equo), followed by BA and TT, and, eventually, as a last option, RP and BA. MW had the worst scoring and, therefore, we do not recommend it.A similar evaluation procedure was carried out on Dataset2. Also, like Dataset1, the distribution of values in Dataset2 is presented on boxplot.The results for Dataset2 differ from those for Dataset1—they were not so sharp. Similarly to the Dataset1 result, the worst overall result had MW. RP and BA reached better but rather intermediate values in overall range. Additionally, BA had the lowest rec measure for all of the methods. On the other hand, RP had the maximum possible rec measure value. In the case of Dataset2, LIMMA reached the best scores for all of the methods. SAM only had slightly worse scoring. Again, they can be regarded as comparable. The third best-scored method was TT. It had worse results than LIMMA and SAM, but better than BA and RP. It is clear that almost all of the methods detected more aDEGs than was expected—the methods gave worse results on a less diversified Dataset.Based on the results obtained for Dataset2, the best results had LIMMA, followed by SAM and then TT. RP and BA were also acceptable. Similarly to the experiment with Dataset1, MW gave the worst results.A final summary of the scoring methods is shown in Based on The low level of concordant results for the Arraysets was surprising. When conducted in the Datasets, our analysis showed that, in most cases, the methods themselves (as well as their implementation) work properly . All of the evaluation measures used for scoring methods were better when the outstanding values were well “separated” . Therefore, one possible conclusion is that microarray experiments data were slightly diversified (similar to Dataset2).In our opinion, such results show the need to recommend how studies based on microarray experiments should be carried out:The list of DEGs should be obligatory, published with precise specification concerning the high-level analysis (and software used).When selecting an algorithm for high-level analysis, it is important to take into account the measures for each method and choose a variant method based on special needs .In general, when the list of DEGs is only obtained with the use of one high-level analysis, it should not be regarded as reliable and definitive. One could argue that an official recommendation about high-level analysis should also be carried out. A possible approach is to use a few methods and acknowledge DEGs as only those genes that are within an intersection of sets of DEGs obtained by different methods. Based on the overall method scoring presented , we recoThe presented results should also be taken into account by authors of reviews (or those who search for DEGs under certain condition in different papers), while compiling results from different publications that describe a set of detected DEGs. It is very important to pay special attention to the methods of high-level analysis. This is because a resultant set of DEGs can vary, depending on the method used.S1 Data(ZIP)Click here for additional data file.S1 Fig(TIF)Click here for additional data file.S2 Fig(TIF)Click here for additional data file.S3 Fig(TIF)Click here for additional data file.S4 Fig(TIF)Click here for additional data file.S5 Fig(TIFF)Click here for additional data file.S6 Fig(TIF)Click here for additional data file.S7 Fig(TIF)Click here for additional data file.


We assessed the utility of the approach using hypothetical data, as well as determining similarities and differences in the set of processes in two instances: (1) denervated skin vs. denervated muscle, and (2) colon from Crohn's disease vs. colon from ulcerative colitis (UC). The hypothetical data showed that in many cases comparing annotations gave superior results to comparing only at the gene level. Improved analytical results depended as well on the number of genes included in the annotation term, the amount of noise in relation to the number of genes expressing in unenriched annotation categories, and the specific method in which samples are combined. In the skin vs. muscle denervation comparison, the tissues demonstrated markedly different responses. The Crohn's vs. UC comparison showed gross similarities in inflammatory response in the two diseases, with particular processes specific to each disease.Assessment of high-throughput—omics data initially focuses on relative or raw levels of a particular feature, such as an expression value for a transcript, protein, or metabolite. At a second level, analyses of annotations including known or predicted functions and associations of each individual feature, attempt to distill biological context. Most currently available comparative- and meta-analyses methods are dependent on the availability of identical features across data sets, and concentrate on determining features that are differentially expressed across experiments, some of which may be considered “biomarkers.” The heterogeneity of measurement platforms and inherent variability of biological systems confounds the search for robust biomarkers indicative of a particular condition. In many instances, however, multiple data sets show involvement of common biological processes or signaling pathways, even though individual features are not commonly measured or differentially expressed between them. We developed a methodology,  With the plethora of publicly-available high-throughput molecular biology data sets including DNA microarray studies  in each data set would be determined and then compared. Most meta- and comparative-analysis methods require either that the features  across experiments are identical , or can be mapped directly to the same biological entity . In both cases, only those features commonly found across experiments will be used for subsequent analysis. Even if the same underlying biological process or pathway is impacted in multiple experiments, the above limitations may make the elucidation of common and different biological processes across data sets challenging.p-value or log-fold change. The goal is then to find over-represented feature annotations  in the set compared to all of the measured features . In contrast, RBTF methods do not set a threshold but rather the features are “ranked” according to an appropriate statistic, and feature annotations are tested to determine if they are significantly enriched at either extreme of the ranked list.Methods commonly used to summarize and infer relationships from high-throughput experiments include the set-based threshold-based (SBTB) and rank-based threshold-free (RBTF) methods. SBTB methods define a “set” of features using a “threshold.” A typical set might be differentially expressed genes with a particular ermFinder   resulting from multiple high-throughput experiments and their annotations (such as GO terms), p-values for all the annotations in each feature list are calculated. Significant annotations for each list are filtered using a pre-specified p-value and a minimum number of annotated features, resulting in a series of annotation lists , each corresponding to a feature list. From the set of enriched annotation lists, a “list membership” can be determined using any or all of the 2n-1 non-empty subsets of annotation lists. For example, if four feature lists  are supplied, four corresponding enriched annotation lists are determined . In this example, there are 24−1 = 15 possible non-empty subsets of annotation lists for determining “list-membership” including: {A1}, {A2}, {A3}, {A4}, {A1,A2}, {A1,A3}, {A1,A4}, {A2,A3}, {A2,A4}, {A3,A4}, {A1,A2,A3}, {A1,A2,A4}, {A1,A3,A4}, {A2,A3,A4}, and {A1,A2,A3,A4}. The final lists of enriched annotations, along with the statistics from each feature-list are supplied in a table for examination.Given feature lists (such as genes)  is stored as meta-data for each node. In contrast to a long table of annotations, the graph structure facilitates determination and visualization of clusters of highly related and possibly redundant annotations with various list-memberships. The annotation graphs can then be combined with a graph visualization framework (such as Cytoscape), nodes can be colored by their “list-membership,” or represented by a pie-chart, wherein each slice represents a separate sample list, with significance in the list denoted by assigned color, and non-significance by de-saturated color.As a means of working with both the large number of possibly redundant annotations and highly similar annotations that may have different list-memberships, the annotations are also reported as a graph relation similar to that implemented in EAlternatively, for GO term annotations, the annotation list-membership may also be determined based on the induced ancestor graph generated from the significantly enriched GO terms from each feature-list. This allows exploration of the results in the GO-directed acyclic graph, while maintaining information about the location of the terms in the GO hierarchy.categoryCompare has been implemented using the R statistical language , allowing users to easily integrate it into high-throughput data analysis workflows, and facilitating the use of the many different gene annotation packages available in Bioconductor. The package supports the direct calculation of enriched GO and KEGG annotations on gene lists. Interactive visualization of the annotation graphs is facilitated using the RCytoscape , animals were euthanized with an overdose of pentobarbital and the rostral-most skin samples corresponding to the acutely mapped zones  were removed and then trimmed to approximately 100 mg before RNA extraction using 1.5 ml Trizol reagent  by homogenizing using a Polytron-style rotor-stator and then centrifuging at 12,000 g for 10 min at 4°C. RNA was then extracted from the supernatant as per the manufacturer's protocol. RNA pellets were dissolved in 50 ul nuclease-free water before purity assessment using UV spectrometry .RNA quality was assessed by formamide-gel electrophoresis. Biotin-labeled fragmented cRNA probes were then synthesized and hybridized to Affymetrix® rat gene expression Rat Genome 230 2.0 GeneChip™ microarrays as per manufacturers protocols. This data is available from GEO as series GSE54356.The mouse muscle denervation normalized microarray data was obtained directly from the NCBI GEO databank (GEO GSE4411). See Wang et al.  for a deThe Crohn's and ulcerative colitis (CUC) data was obtained directly from the NCBI GEO databank (GEO GSE36807). See Montero-Melendez et al.  for a den = 6) and arrays from rats 7 days (n = 5) and 14 days (n = 5) after denervation. These times correspond roughly to the initiation and maintenance phases of the axonal collateral sprouting (CS) process, respectively  compared to naïve calculated using LIMMA  for further details.The skin denervation data set consisted of arrays from naïve (innervated) rats , Crohn's (n = 13), and ulcerative colitis (n = 15) individuals  for further details.The Crohn's and ulcerative colitis (UC) dataset consists of intestinal biopsy samples from healthy  defining the gene sets. A p-value cutoff of 0.01 was used to determine significant GO::BP terms in both the “Up” and “Down” results from romer for each experiment. Major GO term groups were determined by visualizing and inspecting the GO terms and their connections in Cytoscape. Inspection of each group by the authors determines an overall naming for the group of terms. For visualization in Cytoscape, edges with an overlap score less than 0.8 were removed from the annotation graph. The reported “name” for each group of enriched annotations was reported by the authors' in an attempt to summarize commonalities among the GO terms in a specific group. The full list of GO terms for each gene list is available in the supplemental materials, as well as the GO terms in each group.Enriched annotations were calculated using LIMMA's ccPaper R package (https://github.com/rmflight/ccPaper). A summary of both the set-based threshold-based and rank-based threshold-free hypothetical datasets follows.A full description of the hypothetical data generation and analysis is available in the vignette “hypothetical example” of the Homo sapiens and the annotated genes were considered. One-hundred GO::BP terms from the resulting set were selected to define the biological processes present in hypothetical samples; 50 with 10–100 genes annotated, 30 with 250–500 genes annotated, and 20 with 500–1500 genes annotated  category for p-values were calculated for the 100 GO::BP terms for each sample independently as well as a combined sample consisting of the intersection of the sample lists. For each GO::BP term, the p-value was transformed by taking −1*log of the p-value, and the difference between the minimum transformed p-value from each sample and the combined list was reported.Hypergeometric p-value and difference on gene samples was estimated by repeating the calculation using 100 different gene samples. Dependence on the GO::BP term set used was estimated using 100 different GO::BP term samples.The dependence of the The effect of adding noise genes (genes not annotated to the 100 GO::BP terms selected) was assessed by adding variable numbers of noise genes , and in each case varying the fraction of noise genes that were shared between two samples with the same number of noise genes .ioconductor package, and a paired case-control lung cancer dataset from GEO (LUNG – GSE18842)  data set Li,  availablP-values were on the interval , whereas t-statistics were on the interval . Different proportions of genes in the top 50% of entries were defined using a uniform distribution on the range of . For each term, a random set of indices in the rank list were determined, and then for each sample a random set (defined by the proportion in the top 50% for that term) of those indices were assigned values from the upper 50% of ranked statistics, and the rest from the bottom 50%. This results in an independent set of statistics for each sample for each term. LIMMA's geneSetTest was used to calculate probabilities of the chance of the gene set being at the “lower” extreme of the ranked list by chance by comparing the average value of statistic of the set against 10,000 random samples the same size from the same ranked list of statistics. Prior to use with geneSetTest, a linear transformation was performed so that p-values were in the range of  to enable testing in geneSetTest of annotations being at the “end” of the ranked list. A combined sample was generated by combining the gene-level statistics by one of: averaging t-statistics; Fisher's method for p-values; the maximum p-value between samples.The directed acyclic graph relationship of GO:BP terms makes simulating RBTF samples using GO::BP as the gene sets challenging. Therefore, a hypothetical set of independent terms was created. Independent terms were identified by the number of genes annotated to the term, defined using a uniform distribution over a specified range, using three groups of ranges with similar limits as in the SBTB case, namely 20–250 , 250–500  and 500–1000 . Each term used the same uniform distribution of 10,000 values prior to assignment of gene indices. p-values were calculated as the difference of the minimum of the −1*log(P) for sample and −1*log(P) for combined.Differences between sample-based and combined annotation term Traditional methods for comparing high-throughput data sets depend on having shared features between the data sets for comparison. A simple first level analysis is to compare the expression levels of shared features , and “list-membership” assigned based on which feature list produced which annotation .Figure categoryCompare differ from the “traditional analysis,” hypothetical datasets for both SBTB and RBTF analysis situations were generated .In an effort to determine how results using p-value calculated from the combination of the samples  compares to the minimum transformed p-value for that GO term from the two samples. In many cases, the sample wise p-values are much better than those from the combined sample; however, this appears to be dependent both on how many genes are annotated to the term and the proportion of genes annotated to the term in the differentially expressed set.For the SBTB analysis, two hypothetical samples of 1000 genes were sampled from 100 GO terms to represent differentially expressed genes (the set) from two different experiments where the same fundamental biological processes were responsible. Figure The reproducibility of these results are shown in Figures p-value and combined p-value as a function of the number of noise genes added, colored by the fraction of shared noise genes. As can be observed, in general the sample wise p-values do better when there are more noise genes shared between the samples; however, the difference can be large depending on the number of genes annotated to the GO term .The effect of noise genes (genes not annotated to any of the GO terms under consideration) was assessed by varying both the total number of noise genes (10–1000 incremented by 10) and the fraction of noise genes that are shared between the two samples (0–1 incremented by 0.01). Figure p-values, and taking the maximum p-value across samples. Figure p-values between the minimum transformed p-value from the samples and the transformed p-value from the combined sample where the sample was combined using an average of t-statistics. As the proportion of genes in the top 50% of ranks varies, the combined sample gives at least as good or better p-values, until a threshold proportion is reached. Combining samples using Fisher's method generated similar results (results not shown). Combining samples using the maximum p-value across samples demonstrated completely opposite behavior, with the combined sample yielding worse p-values . Generating combined samples was achieved by averaging t-statistics, using Fisher's method to combine s Figure .categoryCompare approach was used to compare gene expression following denervation of skin and muscle. The denervated skin data set consisted of DNA microarray measurements from rats, comparing the transcript expression in the naïve skin, and skin at 7 and 14 days following denervation. The muscle data set consisted of DNA microarray samples of control muscle and muscle 3 days post-denervation from mice  those annotations with a reported Analysis of the 79 significantly enriched annotation groups shows that the majority of these (59) respond to a single condition, including 10 in Skin.T7.Down ; fifteen in Skin.T7.Up; eight in Skin.T14.Down; nineteen in Skin.T14.Up ; five in Muscle.Down (including response to VEGF); and three in Muscle.Up. There are two annotations down-regulated in all three data sets (ATP biosynthesis and muscle adaptation and contraction); one annotation down-regulated in skin and up-regulated in muscle (mitotic cell cycle checkpoint); one down-regulated in Skin.T7 and upregulated in Skin.T14 and Muscle (pH and lysosome regulation); four down-regulated at Skin.T7 which are up-regulated at Skin.T14 ; one down-regulated in Skin.T7 and Muscle (response to glucose synthesis); one up-regulated in Skin.T7 and down-regulated in Skin.T14 (microtubule organization); two up-regulated in both skin datasets ; and three down-regulated in Skin.T14 and Muscle . A complete list of the membership for each of the significantly enriched annotations is provided in Table categoryCompare was employed to compare the gene expression from Crohn's disease and ulcerative colitis (UC) intestinal samples when each was compared to normal samples. GO::BP annotation enrichment was used. Table p-value cutoff of 0.01.As listed in Table For both the CROHNS vs. UC and Skin vs. Muscle datasets, the enriched annotations are reported as “themes” of groups of highly similar annotations, where similarity is defined by the shared number of genes between annotations (see Methods). Figure categoryCompare algorithm per se, returning the annotations as a graph where each node is an annotation and edges are weighted by their similarity (see Methods) enhances the biological interpretation by providing the ability to determine sets of highly related annotations that is not easily accomplished in list form. For example, of the nine annotations in the middle group of Figure p-value; CROHNS.Up p-value; and then by their list-memberships; they are scattered throughout the table, being located in rows 40, 123, 136, 149, 155, 167, 191, 206, and 215. Deriving the theme of “nucleoside and nucleotide metabolism” from the items in the list would be more difficult. It should be noted that all of the tabular data is available in Cytoscape when the user selects a node, and multiple nodes may be selected simultaneously, thus providing all the tabular output for a particular group of nodes.Although not part of the categoryCompare to determine the similarities and differences between these two conditions at a higher annotation-based level, which is highlighted in the visualization of the GO results .Overall the annotation groups which emerged for denervated skin and denervated muscle are largely different. This is likely due in part to the intrinsic difference in tissues, one being a barrier tissue with rapid cellular turnover that is highly sensitive to innervation status , should result in the lowest fraction of genes being highly ranked in the combined sample.In contrast, with an RBTF type analyses such as GSEA, the degree of improvement using categoryCompare has advantages in particular situations, but is still limited, especially with respect to a direct meta-analysis. It is likely that these same limitations are present in a comparative-analysis, and where features are present in both data sets a feature level comparison should also be performed in conjunction with an annotation level comparison.Both of these results suggest that Analysis of both the data sets comparing denervation of skin with muscle and Crohn's with ulcerative colitis highlighted several biological processes shared across conditions as well as several that appear to be specific to each condition. These annotations provide a starting point for hypothesis generation and testing either by subsequent high-throughput experiments  or directed experiments using potential targets derived from this analysis.categoryCompare is able to illuminate processes highly relevant to the fundamental underlying biology that may be missed using a more traditional feature level analysis. This ability is dependent on the method of annotation enrichment employed (SBTF vs. RBTF), and the degree of noise. Although this analysis considered only data with identical classes of features, the annotation-level approach could enable meta-analyses of experiments with different types of features but shared annotations, such as genes and metabolites.The hypothetical data analysis demonstrates that categoryCompare provides an easily extensible, general framework and interface for performing high-throughput data meta-analysis at the annotation level, in a commonly used programming environment with large amounts of available feature annotation data. Future work includes providing easy access to other types of feature annotations, calculating annotation enrichment for user provided annotations, and visual exploration of the feature—annotation relationships.categoryCompare package is available from http://bioconductor.org/packages/release/bioc/html/categoryCompare.html, while a development version is hosted on GitHub at https://github.com/rmflight/categoryCompare. It should be noted that the ccPaper branch of the version hosted on GitHub was used for all of this work. Many features from that branch will be available in the development version of the software package.The current version of the ccPaper R package available from https://github.com/rmflight/ccPaper. The original raw data used (from denervated skin) is available from GEO as series GSE54356.The data supporting the results reported are available in the ioconductor package. Fahim Mohammad tested the package on various platforms, and edited the package vignette. Jeffrey C. Petruska, Lawrence D. F. Moon, and Benjamin J. Harrison posed the original question motivating the work, provided the skin denervation data set and biological interpretation. Eric C. Rouchka provided materiel, direction, and overall supervision of the project. Mary B. Bunge provided funding and consultation for the denervated skin microarray experimental design. All authors contributed to the writing of the manuscript.Robert M. Flight conceived of the methodology, wrote, and maintains the BFunding was provided by the CDRF International Consortium on Spinal Cord Injury Research ; (Both are now in acknowledgements rather than authors.) Kentucky Spinal Cord and Head Injury Research Trust (Grant 09-12A to Jeffrey C. Petruska); Paralyzed Veterans of America (Fellowship to Benjamin J. Harrison); National Institutes of Health (NIH) grants P20RR016481 (ECR), 3P20RR016481- 09S1 , P20GM103436 (Eric C. Rouchka), R21NS080091 (Jeffrey C. Petruska), and R21NS 071299 (Jeffrey C. Petruska); and Department of Energy contract DE-EM0000197 (Eric C. Rouchka). The article contents are solely the responsibility of the authors and do not represent the official views of the funding organizations.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Escherichia coli that was further enriched with meta-information through an iterative learning procedure. We then constructed an ensemble method to predict environmental and cellular state, including strain, growth phase, medium, oxygen level, antibiotic and carbon source presence. Results show that gene expression is an excellent predictor of environmental structure, with multi-class ensemble models achieving balanced accuracy between 70.0% (±3.5%) to 98.3% (±2.3%) for the various characteristics. Interestingly, this performance can be significantly boosted when environmental and strain characteristics are simultaneously considered, as a composite classifier that captures the inter-dependencies of three characteristics  achieved 10.6% (±1.0%) higher performance than any individual models. Contrary to expectations, only 59% of the top informative genes were also identified as differentially expressed under the respective conditions. Functional analysis of the respective genetic signatures implicates a wide spectrum of Gene Ontology terms and KEGG pathways with condition-specific information content, including iron transport, transferases, and enterobactin synthesis. Further experimental phenotypic-to-genotypic mapping that we conducted for knock-out mutants argues for the information content of top-ranked genes. This work demonstrates the degree at which genome-scale transcriptional information can be predictive of latent, heterogeneous and seemingly disparate phenotypic and environmental characteristics, with far-reaching applications.A tantalizing question in cellular physiology is whether the cellular state and environmental conditions can be inferred by the expression signature of an organism. To investigate this relationship, we created an extensive normalized gene expression compendium for the bacterium  Escherichia coli that we have further enriched via an iterative learning approach. We then apply an ensemble of various machine learning algorithms to infer environmental and cellular information such as strain, growth phase, medium, oxygen level, antibiotic and carbon source. Functional analysis of the most informative genes provides mechanistic insights and palpable hypotheses regarding their role in each environmental or genetic context. Our work argues that genome-scale gene expression can be a multi-purpose marker for identifying latent, heterogeneous cellular and environmental states and that optimal classification can be achieved with a feature set of a couple hundred genes that might not necessarily have the most pronounced differential expression in the respective conditions.The transcriptional profile of an organism contains clues about the environmental context in which it has evolved and currently lives, its behavior and cellular state. It is yet unclear, however, how much information can be efficiently extracted and how it can be used to classify new samples with respect to their environmental and genetic characteristics. Here, we have constructed an extensive transcriptome compendium of  Genome-scale transcriptional profiling has become a standard and relatively inexpensive way to identify the overall cellular state and condition-specific cellular responses to external stimuli. For instance, different sets of genes are known to be active in each growth phase and medium , while sE. coli, the most well-studied model microbe, we are still limited to a few thousands microarray or RNA-Seq experiments that cover more than 30 strains, a dozen different media and a multitude of other genetic , or environmental  perturbations. Although this collection has already increased by an order of magnitude from the roughly two hundred genome-wide transcriptional profiles that we had eight years ago, it is still an inadequate sampling of the relevant experimental space. In addition, since these experiments have been performed in different technological platforms  and technologies (e.g. microarrays vs. RNA-Seq), in different labs and under different environmental conditions, appropriate normalization schemes are both of paramount importance and with an added complexity. As such, efficient training of machine learning methods is hindered due to data complexity, compatibility and the curse of dimensionality that plagues datasets with thousands of features (genes) but only a few samples (conditions).Despite its obvious correlation with various physiological and cellular states, we lack a clear understanding of the information content related to the manifold phenotypes that can be extracted from the genome-scale transcription profiles. Until now, a significant obstacle was the absence of sufficient transcriptional data to support the training of multi-feature and multi-label classifiers. Indeed, after aggregating all high-throughput transcriptional data that is currently available for Saccharomyces cerevisiae study that involved tens of data samples was able to predict growth rates [The application of high-dimensional prediction algorithms has been widespread in biology ranging from gene function prediction –12, diseth rates , while ath rates . More reth rates .E. coli , consists of publicly available data that were curated from online public databases such as GEO [EcoGEC compendium  can be achieved with less than 400 genes, which is close to its maximum performance (71.55%) that is achieved when considering all 4166 genes. To investigate the relationship of data size with classification performance, we systematically reduced the dataset, keeping a balanced class/label distribution. Our results argue that although there is an expected reduction in classification performance, as the dataset is progressively reduced by up to 75%, the method is quite robust with an average reduction of 6% classification performance per quartile reduction in data size , with the balanced accuracy of all classifiers ranging between 69.95% (±3.52%) to 98.27% (±2.32%) . The simultaneous prediction of all seven characteristics of a sample using seven individual classifiers yields an accuracy of 84.21% (±1.39%) , we had to reduce the amount of classes to 13 due to insufficient data  . For pre(±2.32%) . Interes(±2.32%) . The ite(±1.39%) . To creadata see . InteresP < 3.1 × 10−4). Similarly, for phase and strain classifier, organic acid biosynthesis (P < 2.7 × 10−4) and nitrogen biosynthesis (P < 1.2 × 10−3) are up-regulated, respectively. Genes that are related to carbohydrate metabolism (P < 6.1 × 10−7) are noticeably most informative to classify different carbon sources as well as strains. Some functional characteristics were statistically significant across multiple classifiers, including cell wall/peptidoglycan (P < 2.7 × 10−7) and ATP-binding (P < 1.5 × 10−8), hydrolases (P < 9.1 × 10−6), membrane (P < 4.1 × 10−6), ribosome (P < 2.2 × 10−7) and transport (P < 4.2 × 10−6) (P < 4.3 × 10−7) which is in agreement with previous studies that report the prevalence of phase-dependent transcriptional regulation in a variety of biosynthetic processes [Next, we investigated which genes have the highest information content and the respective pathways they belong to. The decrease of mutual information in ranked genes follows an inverse logarithmic relationship . For eac × 10−6) . The glo × 10−6) . Moreoverocesses –39. Fig.gltK, gltJ, gltL, dppF, glnD). Different media contain different amounts of amino acids and nutrients required for bacterial growth so the activation of their biosynthesis is expected to be an informative feature about the media where bacteria are growing. Another 3 genes are involved in the enterobactin synthesis , a siderophore that has been very recently revealed to be related to the growth of E. coli in M9 [The analysis of the most informative genes for the media classifier reveals 14 genes encoding for membrane transporters and 7 involved in nitrogen metabolism , S1 Textli in M9 .RpoS and, as expected, it is present in the set of genes informative for growth phase, along with several genes belonging to its regulon like dnaK, clpx, hemL, dps, rpsK, hfq, rplA, crr, rpsE and gapA [hpf, crr and sspA [rpsL, rpsQ, rpsE, rplA, rplT, rpmJ, rrsG) are also implicated to be phase-dependent, which is in agreement with previous reports [Over the course of the growth curve, the metabolic pathways change in order to optimize the use of the available nutrients and to ensure survival under stress conditions. The major transcriptional regulator for the entry into stationary phase is and gapA . In thisand sspA –44. In a reports .In the case of the strain classifier, the analysis displays a wide variety of genes involved in different pathways and cellular processes. Different strains have evolved differentially from their common ancestor and, hence, have developed different regulatory pathways for various processes including carbon assimilation, degradation, and membrane formation. All informative genes for the medium classifier  are inclsucA, acnB, nuoJ, cyoE) and another 4 in fermentation . For carbon source prediction, we can find 15 proteins associated with membrane formation, with 6 of them described transporters . In addition, 5 differentially expressed genes involved in carbohydrate metabolism also stand out .Environmental perturbations, such as carbon source and oxygen abundance, give rise to informative genes that are specific to those cellular processes (ompF) that is known to bind ampicillin [Regarding antibiotics, we have tested Norfloxacin, which functions by inhibiting DNA gyrase. Unexpectedly, in its informative gene list we cannot find any gene related to DNA repair or SOS response , possiblpicillin .cyoD, nuoK, sucD, sucC and cyoB) and anaerobic respiration , although these genes would not be selected if we ranked based on differential expression. Similarly, in carbon source classification this set includes 11 genes involved in carbohydrate catabolic processes  and 6 in cellular polysaccharide metabolic processes . The 24 transporters used for the medium classification, the 5 genes involved in translation for phase classification and 72 membrane proteins that are contained in the antibiotic feature set are indeed expected to be informative in the respective classification task, despite not being in the top differentially expressed genes.Interestingly, a substantial subset of the informative genes that were selected as features were not differentially expressed in the respective samples . A closeppiD) and another gene that is highly ranked for classification between aerobic and anaerobic respiration (ldcC). The MI of each gene is only high in the classifier of interest and not in the rest  while growth with the ldcC mutant remains similar to the WT demonstrating the involvement of ppiD in the use of different carbon sources . ppiD has been described as a membrane-anchored chaperone [t – test, P < 0.006 and P < 0.008, respectively).The results obtained in this study can be used to decipher novel, condition-specific gene functions. To assess whether biological function can be predicted by targeted experimentation of classifier-specific informative features, we selected one gene with high MI for carbon source classification (the rest . We then results  show thahaperone  but its ppiD and ΔldcC) were grown in M9 with glucose in aerobic and anaerobic conditions, in order to assess the influence of the ldcC mutation in these conditions. Here, the ppiD mutant serves as the negative control and the ldcC mutation is indeed informative of the aerobic conditions, although the difference is not as pronounced as in the case of carbon source classification . A closer look at the MI values show that the informative genes for aerobic respiration are two orders of magnitude lower than those for medium, which suggests that information content is dispersed among a number of genes.We performed a similar experiment where the three strains . Interestingly, the performance of the composite multi-class classifier performs significantly better for the overall classification of these characteristics, than an aggregate of individual classifiers for phenotypes, demonstrating large interdependencies across different conditions.Multiple characteristics of an organism are interrelated, implying its heterologous transcriptional landscapes in different combinations of phenotypic conditions. These complex dependencies in phenome are not readily analyzable even in the compilation of thousands of publicly available transcriptome profiles as the experimental conditions in published data are often disproportionate, typically skewed in favorable settings (e.g. MG1655 strain over LB medium), which produces small sample sets or even empty sets in combinatorial conditions. Indeed, the results on composite classification argues that with the current ppiD in the utilization of different carbon sources. Further analysis involves the use of over/under-expressed copies and protein-protein assays to discover quantitative associations and interaction partners. By analyzing the expression levels of the genes in the phase classifier that are not predictable using RT-PCR and transcriptional fusions we can find out novel regulation when growth phase changes from exponential to stationary. Another potential application is in the case of the antibiotics Ampicillin and Norfloxacin where this analysis can be used to identify implicated pathways in lethal and non-lethal concentrations.By looking at the top informative genes in two classifiers, we demonstrated the involvement of the In recent years, the capacity of microorganisms to sense and act upon environmental stimuli  has sparE. coli transcriptional profiles from 17 different GEO entries [E. coli Microarray Compendium (EcoMAC) that consists of 2198 microarrays of 4189 genes for which raw files were downloaded and normalized by RMA (robust multichip average) method [We downloaded 83 RNA-Seq  entries  that cor entries . For oth) method . The intAlthough integrative analysis of multiple microarray gene expression (MAGE) datasets allows to distill the maximum relevant biological information from genomic datasets, the unwanted variation, so-called batch-effects arising from data merged from difference sources has been a major challenge to impede such effort . To adju2Fold Change (FC) of gene expression with respect to the WT expression for each gene. WT samples were identified from experiments that didn’t undergo genetic and environmental perturbations from the three platforms . log2Fold Change (FC) was separately measured for each platform by comparing the mean of WT data. Using transformed data, we estimated a normal distribution N for each gene and finally converted each log2FC gene value into one of the 3 categorical values by measuring deviation from the mean . The platform-specific categorization of gene expression effectively removes platform biases  in order to deal with biases arising from combining different platforms and improve the classification accuracy . We firsm biases .ab initio were repeatedly inferred. Inference is based on consensus-based approach of four machine learning methods described above. Re-labeled phase information accompanying with annotated data is used for training the consensus model in next iteration. This procedure is halted once the similarity of phase labels between consecutive iterations is convergent when the similarity of phase labels between consecutive iterations converges . Although the use of inferred labels through iterative learning demonstrates an increased performance, compared with the prediction using known labels only  of inference of unannotated data, we performed the simulation study for each classifier by randomly masking 30% of total labels of each class. First, the accuracy of inferred annotation after iterative learning is measured by comparing with real labels before and after iterative learning . Then weA label is assigned for each of the seven classification characteristics . We have identified 4 classes for medium , 3 classes for phase  having both annotated and predicted data, and 4 classes for strain . A class of “others” was added that corresponds to conditions that are unclear or scare in quantity. Classification of the strain, medium, and growth conditions can be integrated also as a multi-class problem. We synthesized a new predictor variable called composite by combining values of 3 characteristics. From the 48 possible classes , only 13 combinations have enough data (more than 5 samples) for training, hence we have encompass all other labels with insufficient data under the label “others”, resulting in a total of 13 classes .We use Naïve Bayes  to the label where MI is measured byx is the gene selected and y is the predictor variable. This process is iteratively repeated by increasing k with an interval of 10 and the exception of start (10) and end points . Basically, the selection procedure of k features are performed in training data only and k showing the highest performance is selected for testing. All the analyses in this study other than the cross-validation of model used the features selected from the complete data.Mutual information is a stochastic measure of dependence  and it hP < 3.7 10−4) for each classifier are displayed in E. coli is reconstructed from [The most informative genes are selected by measuring the mutual information (in bits) for each of the characteristic variables and then selecting the top 10% genes based on their information content. These top informative genes are then used for finding shared genes across different classifiers  and for ted from  with patted from . Pathwayted from .In addition to DAVID, we have performed a GSEA analysis  where eappiD and ΔldcC were performed in M9 complemented with 0.4% of glucose, glycerol and sodium lactate. For growth curves, the starter cultures of all strains were grown and therefore adapted (B7–9 generations) to M9 glucose for 12 hours at 37C. Cultures were started at OD600 of 0.004. OD600 was measured every 10 minutes on a Tecan Plate Reader. Two independent replicate growth tests were performed for each strain. For the anaerobic and aerobic growth curves bacteria were grown in M9 supplemented with glucose at 37C without shaking. The anaerobic growth was made in an anaerobic chamber where media was inserted 2 days prior to the experiment to extract all the oxygen present in the media. Samples were taken at 2, 8 and 24 hours through a spectrophotometer .For consensus-based prediction using four different classifiers, we used the Statistics Toolbox in MATLAB. For the multi-class SVM, one-versus-rest (OVR) approach was used in which for each class, a binary classifier is built for the class label and the rest. Each binary SVM was built using Gaussian Radial Basis Function (RBF) kernel and the default sigma factor of 1 was used. For soft margin, C parameter showing best performance was selected in the range of 0.5 to 4 in the training phase. For KNN, K was set to one in S1 FigMissing phase information was imputed by an iterative learning approach in which the prediction model for growth phase is trained iteratively until convergence. In each iteration, the phase of all samples that were originally unannotated is predicted, based on an ensample of 4 machine learning methods  that produce a consensus outcome, as described in the Methods section of the manuscript. We used the fraction of correctly re-annotated data over all unannotated data to measure the similarity of the two vectors. If the confidence level of the prediction does not reach a threshold of 0.75 (i.e. 3 out of the 4 methods agree in the putative annotation), then the sample remains tagged as un-annotated. Otherwise, the imputed phase information is used for training the model during the next iteration. This procedure repeats until the similarity of phase labels between consecutive iterations converges . For our dataset, this led to annotation of more than 90% of the un-annotated data samples.(EPS)Click here for additional data file.S2 Fig(A) Prediction of unknown phase information by using the iterative approach is validated by using testing data that consist of de-labeled samples constructed from each of the 3 phase types . Validation of iterative approach to impute missing data is performed by comparing the actual and predicted labels produced by iterative learning. The de-labeled, i.e. artificially un-annotated, data were 2%, 5%, 10% and 20% of the total dataset. For each of the 3 phase classes, the predicted classes for each actual class type is shown. (B) Unannotated portion of phase data in the EcoGEC is inferred by using iterative learning. After four iterations, the similarity of predicted labels between consecutive iterations converge (691 out of the 764 samples). The 72 leftover samples are discarded as unidentified and/or noisy data points. (C) Simulation of iterative learning for all classifiers by randomly masking 30% of all class labels in the original dataset. We set the threshold of confidence of consensus-based prediction to 1 for selecting data that needs to go to next iteration over the iterative learning. In other words, the samples that reach perfect consensus in assigning labels from 4 different methods are finalized for annotation and used for training over the iterative learning. The purpose of the more stringent threshold was to observe the benefit of iterative process in learning. The percentages in the legend indicate the total increase of re-labeled classes after the first iteration.(EPS)Click here for additional data file.S3 FigppiD and ΔldcC for the three carbon source classes in our dataset, glucose, glycerol and sodium lactate, (B) growth curves of WT, ΔppiD and ΔldcC in aerobic and un-anaerobic conditions.(A) Growth curves of WT, Δ(TIFF)Click here for additional data file.S4 FigEach growth curve was made in duplicate and the average was plotted.(TIFF)Click here for additional data file.S1 TableThe relationship between performance of classifiers and the data size is investigated. The dataset with balanced class distribution is prepared from the original compendium and it is reduced by 25% until only 25% remains. Each dataset is separately trained and tested.(XLSX)Click here for additional data file.S2 TableThe intersection of the feature gene set when mutual information (MI) and differential expression (DEG) are used for ranking. Differential expression ranking was determined by ANOVA. In the parenthesis, we report the classification performance when the class labels are uniformly distributed (maximum entropy). The “null” and “dataset” baselines correspond to the base prediction accuracy in the case where the classes are uniformly distributed for each classification task, or the most representative class based on the data (highest prior) for each classification case is selected, respectively.(XLSX)Click here for additional data file.S3 TableWe assessed the iterative learning (IL) method for each class by randomly masking 30% of the class labels (testing dataset). Accuracy refers to the percentage of the testing dataset that was correctly re-annotated by IL. Classification performance is measured with and without IL being applied to the final dataset. The “null” and “dataset” baselines correspond to the base prediction accuracy in the case where the classes are uniformly distributed for each classification task, or the most representative class based on the data (highest prior) for each classification case is selected, respectively.(XLSX)Click here for additional data file.S4 TableTo evaluate the efficacy of iterative learning to correctly annotate missing phase information, we constructed a testing set where data for all 3 different phase categories  were de-labeled. We evaluated how many samples in this simulated, artificially un-annotated, dataset were re-labeled to their original phase labels after iterative learning. The iterative learning procedure was performed with testing sets that included 2%, 5%, 10% and 20% of un-annotated samples, as a percentage of all samples available. The tables below (1A-D) show that the contingency tables of simulated inference for different settings.(XLSX)Click here for additional data file.S5 Table(XLSX)Click here for additional data file.S6 Table(XLSX)Click here for additional data file.S7 Table(XLSX)Click here for additional data file.S8 Table(XLSX)Click here for additional data file.S9 Table(XLSX)Click here for additional data file.S10 Table(GO). For DAVID, we use all selected features for each classifier to know associated GO. For GSEA, we use MI of all genes to know enriched GO. The resulting lists are compared and the proportion of DAVID results in GSEA enriched terms are reported.(XLSX)Click here for additional data file.S11 TableThe prediction performance with and without batch control is compared for each classifier. For carbon source, phase, and composite classifier, the profiles having early-exponential phase or acetate are studied in a single project. For testing without batch control, typical 5-fold cross-validation was used without considering the batch information. For batch controlled experiments, a leave-one-batch-out cross validation was conducted to verify model performance while removing batch effects.(XLSX)Click here for additional data file.S12 Table(XLSX)Click here for additional data file.S13 Table(XLSX)Click here for additional data file.S14 Table(XLSX)Click here for additional data file.S1 TextThe citations on the functional studies of the ranked list of genes in (DOCX)Click here for additional data file.


High dimensionality and small sample sizes, and their inherent risk of overfitting, pose great challenges for constructing efficient classifiers in microarray data classification. Therefore a feature selection technique should be conducted prior to data classification to enhance prediction performance. In general, filter methods can be considered as principal or auxiliary selection mechanism because of their simplicity, scalability, and low computational complexity. However, a series of trivial examples show that filter methods result in less accurate performance because they ignore the dependencies of features. Although few publications have devoted their attention to reveal the relationship of features by multivariate-based methods, these methods describe relationships among features only by linear methods. While simple linear combination relationship restrict the improvement in performance. In this paper, we used kernel method to discover inherent nonlinear correlations among features as well as between feature and target. Moreover, the number of orthogonal components was determined by kernel Fishers linear discriminant analysis (FLDA) in a self-adaptive manner rather than by manual parameter settings. In order to reveal the effectiveness of our method we performed several experiments and compared the results between our method and other competitive multivariate-based features selectors. In our comparison, we used two classifiers (support vector machine,  Microarray gene expression based cancer classification is one of the most important tasks in bioinformatics. A typical classification task is to separate healthy patients from cancer patients, based on their gene expression “profile”. However, because cancers are usually marked by changing in the expression levels of certain genes Feature selection techniques for microarray data can be broadly grouped into three categories that are wrapper (classifier-dependent) methods Partial least squares(denoted as PLS), which shares the characteristics of other regression and feature transformation techniques, has proven to be useful in situations when the number of observed variables(Reproducing Kernel Hilbert SpacekernelPLS(publicly available at https://github.com/sqsun/kernelPLS). Determining the number of components is a thorny problem in PLS method. In order to obtain a reasonable number of components, we make use of the relationship between PLS and linear discriminant analysis to determine the number of components in kernel space based on kernel linear discriminant analysis. We find that the two classifiers combined with our feature selection method obtained promising classification accuracy on eleven microarray gene expression datasets.Motivated by mentioned above, in this paper, we develop a feature selection method based on the partial least squares(abbreviated PLS) The rest of this paper is organized as follows. In section 2 we proposed a filter method based on PLS and kernel method. Then we proceed in section 3 to determine the optimal parameters for our method. In section 4 we compared our approach with several competitive filters. The conclusion can be found in section 5.In the following, let components, leading to a parsimonious model which shared characteristics with other regression and feature transformation techniques PLS is one of the widespread use of a class of multivariate statistical analysis technique introduced by loadings of When the first two components kernel trick”.The kernel version of PLS uses a nonlinear transformation In order to state dot product operation in the algorithm, we can restrict Gram matrixLet https://github.com/sqsun/kernelPLS).The first component for kernel PLS can be determined as eigenvector of the following square kernel matrix for In original space, let It is worth noting that the above equation can also be used in kernel space. The reason is holding of equation Two issues are still unresolved before applying kernel PLS for feature selection. The number of components and the number of features are unknow.In order to determine the number of components To fully circumvent these difficulties, In kernel space, the FLDA turns out to be an optimization problem It denotes the contribution of the To assess the performance of our method, we have conducted several experiments on a number of publicly available datasets. Summary of all data sets we used in our experiments can be found in ).AMLALL(A). There are two parts containing the initial (train), 38 bone marrow samples from two classes: 27 cases of acute lymhoblastic leukemia(ALL) and 11 cases of acute myeloid leukemia(AML); independent (test), 34 samples from two classes: 20 cases of ALL and 14 cases of AML. Each case is described by expression levels of 7129 probes from 6817 human genes. Source: http://www-genome.wi.mit.edu/cgi-bin/cancer/datasets.cgi;).Breast(B). The dataset used the raw intensity Affym-etrix CEL files and normalized the data by RMA procedures. A final expression matrix comprising 22283 features and 209 samples, 71 of which are from patients, the rest 138 samples are normal samples. Source: http://math.bu.edu/people/sray/software/prediction;Lung(L)([29]). This dataset contains 86 samples: 24 are tumor samples and 62 are normal controls, 7129 genes with highest intensity across the samples are considered. Source: http://math.bu.edu/people/sray/software/prediction/;Prostate(P) ([30]). This dataset contains 52 prostate tumor samples and 50 normal samples with 12600 genes. An independent set of testing samples is generated from the training data, 25 tumor and 9 normal samples are extracted according to Singh's publication. Source (training): http://www.broadinstitute.org/cgi-bin/cancer/datasets.cgi;DLBCL(D)([31]). The goal of this dataset is to distinguish diffuse large B-cell lymphoma (DLBCL) from follicular lymphoma (FL) morphology. This dataset contains 58 DLBCL samples and 19 FL samples. The expression profile contains 7129 genes. Source: http://www-genome.wi.mit.edu/mpr/prostate;Medulloblastoma(M)([32]). Patients outcome prediction for central nervous system embryonal tumor. Survivors are patients who are alive after treatment whiles the failures are those who succumbed to their disease. The dataset contains 60 patient samples, 21 are survivors and 39 are failures. There are 7129 genes in the dataset. Source: http://www-genome.wi.mit.edu/mpr/CNS;Stjude(S)([13]). The dataset has been divided into six diagnostic groups, BCR-ABL (9 samples), E2A-PBX1 (18 samples), Hyperdiploidhttp://www.stjuderesearch.org/data/ALL1;Lymphoma(Ly)([33]). The dataset consists of measurements of 4026 genes from 62 patients. The patients are classified into three classes: lymphoma and leukemia , follicular lymphoma  and chronic lymphocytic leukemia . We estimated the missing values of “NA” symbol in original ratio data by KNN-imputed method ([34]). The dataset contains 83 samples and 2,308 gene expression values. It can be divided into four classes, the Ewing family of tumors (EWS), Burkitt lymphoma(BL), neuroblastoma (NB) and rhabdomyosarcoma (RMS). Among the 83 samples, 29, 11, 18, and 25 samples belong to classes EWS, BL, NB and RMS, respectively. Source: http://www.biomedcentral.com/content/supplementary/1471-2105-7-228-S4.tgz.MLL(ML)([35]). The dataset contains 72 samples in three classes, acute lymphoblastic leukemia (ALL), acute myeloid leukemia (AML), and mixed-lineage leukemia gene (MLL), which have 24, 28, 20 samples, respectively. In our experiment, we preprocessed this dataset according to reference http://www.biomedcentral.com/content/supplementary/1471-2105-7-228-S4.tgz.Lung(Lu)([37]). The total of this dataset contains 203 samples with 12600 genes in five classes, adenocarcinomas (139), squamous cell lung carcinomas (21), pulmonary carcinoids (20), small-cell lung carcinomas(6) and normal lung (17). We preprocessed the dataset according to reference http://www.biomedcentral.com/content/supplementary/1471-2105-7-228-S4.tgz.In our first experiment, we used two datasets, namely the Leukemia data (two-class) of For the Leukemia data, we collected several most important genes in  that werFor Leukemia data, the top-ranked 40 features obtained by our procedure are shown in For the Lymphoma data of particle swarm optimization algorithm to pick the pair . The parameter Without loss of generality, we used two datasets, Breast(two-class) and Lymphoma(three-class) to show the performance of our method. Similarly in It is worth noting that our method outperforms others on three hard-classify datasets, Wang's Breast cancer, Gordon's Lung adenocarcinoma and Pomeroy's Medulloblastoma. We also make a comparison with the results of other feature selectors in published papers. Fox example, the reference To estimate the performance of our method we did not limit our evaluation to only two-class datasets we also used 5 multi-class datasets in our experiments. In this article, we proposed an effective multivariate-based feature filter method for cancer classification, namely, kernelPLS-based filter method. We showed that gene-gene interactions cannot be ignored in feature selection techniques to improve classification performance. In other words the nonlinear relationship of gene-gene interactions is a vital concept that can be taken into account to enhance accuracy. To capture these nonlinear relations of interaction between genes we used kernel method because kernel method can be used to reveal the intrinsic relationships that are hidden in the raw data. In order to capture the reasonable number of components, we make use of the relationship between PLS and linear discriminant analysis to determine the number of components in kernel space based on kernel linear discriminant analysis. To verify the importance of gene-gene interactions we compared our feature selector with other multivariate-based feature selection methods by using two classifiers SVM and KNN. Experimental results, expressed as both accuracy(Acc) and area under the ROC curve(AUC), showed that our method leads to promising improvement in ACC and AUC. We can conclude that the gene-gene interactions whats more, nonlinear relationships of gene-gene interactions are core interactions that can improve classification accuracy, efficiently. We can summarize the characteristics of proposed approach as follows: (1)Fast and efficient. The time complexity of deflation procedure used after the extraction of each component scale is In our method, the choice of kernel functions can affect the results. When high dimensionality exist(such as microarray datasets), the performance of linear kernel is better than Gauss kernel for our method. What's more, in case of linear kernel there is no noticeable effect on the results while adjusting its parameters.