With the explosion in data generated using microarray technology by different investigators working on similar experiments, it is of interest to combine results across multiple studies.In this article, we describe a general probabilistic framework for combining high-throughput genomic data from several related microarray experiments using mixture models. A key feature of the model is the use of latent variables that represent quantities that can be combined across diverse platforms. We consider two methods for estimation of an index termed the probability of expression (POE). The first, reported in previous work by the authors, involves Markov Chain Monte Carlo (MCMC) techniques. The second method is a faster algorithm based on the expectation-maximization (EM) algorithm. The methods are illustrated with application to a meta-analysis of datasets for metastatic cancer..The statistical methods described in the paper are available as an R package, metaArray 1.8.1, which is at Bioconductor, whose URL is  With the increasing availability of published microarray data sets, there is a tremendous need for developing approaches to validate and integrate results across multiple studies. One major issue to deal with in the meta-analysis of DNA microarrays is the lack of a single standard experimental platform for data generation. The dominant technologies so far have been two-color microarrays and oligonucleotide  arrays. Because these technologies measure fundamentally differing genetic materials designed to represent identical targets, many properties of expression measurements may vary across platforms including scale of measurements, sensitivity in detecting fold changes, control of cross-hybridization, and so forth. The heterogeneity in array design poses a great challenge for cross-platform comparisons and integration of results across independent microarray studies. The general area of combining data across multiple studies is referred to as meta-analysis ,2.et al. )et al.  as a poset al. 13]. The et al. plotted against that in POE EM. The ES signature shared 15 UGIDs in common with the POE EM signature and 18 genes with the POE MCMC signature only, which suggests that the two signatures will have different characteristics.First, since the method of  pools thMeanwhile, the Conlon signature had an overlap of two genes with the ES signature and one gene with the POE EM and MCMC signatures. The poor overlap of Conlon signature with others is consistent with the high Bayesian FDR estimated above.To assess the classification performance, we performed hierarchical clustering of tissue samples from the individual studies using the ES signature. Figures We note that clustering with all signatures give fairly accurate results in all three studies. In the ES signature, only a few metastatic samples are grouped together with two other primary tumors for the Chen liver study Figure . Two metThe POE EM and MCMC signatures give comparably good clusterings of the two types of tumors across all studies. In Figures et al.  and 0 otherwise.where Note that the supports of the two uniform distributions are disjoint. As a result, the probabilities of differential expression are mutually exclusive with the following forms:i in sample j in an individual study. On first glance, our formula differs from that in [We then construct the following expression measurement:  that in  in that To sample from the posterior distributions of the parameters, a Gibbs sampling algorithm (with Metropolis-Hastings step for mixture proportion parameters) was then implemented where the gene-specific parameters were repeatedly sampled from the corresponding full conditional distributions [See Additional File eij, which can then be combined across multiple studies. Second, the transformed values carry meaningful interpretations as signed probabilities of differential expression of a gene in a particular sample. Third, the underlying normal and uniform mixture distributions give equal density in the tails and is effective in reducing the influence of extreme expression values. Finally, the Bayesian hierarchical modeling approach borrows strength across genes, resulting in shrinkage-type estimators for the gene-specific parameters. Consequently, the high-dimensional gene expression data are denoised. However, the algorithm for inferring the posterior distribution of the latent variables is fairly computationally intensive. In the next section, we discuss an alternative mixture model specification that leads to a more computationally efficient algorithm.There are several advantages to the mixture model-based transformation. First, the method estimates the posterior distribution for the latent variables Maximum likelihood estimation (MLE) using the EM algorithm leads to greater increases in computational speed for mixture models. Such an approach might be useful since what we are interested in eventually is estimates of POE that we can integrate across studies. There is a difficulty in implementing an EM algorithm for the three-component mixture model we considered in the previous section. Recall the restriction that the uniform components must have the same heights. Since the MLE of end points of uniform distributions are the most outlying observations, we have found in some examples that the EM algorithm with these MLEs provides parameter estimates that are unstable.eij from the previous section  and collapse them into two possible values, eij = 1 and eij = 0. Note that for the ith gene and jth sample, eij = 1 corresponds to differential expression in either direction, while eij = 0 represents nondifferential expression. We now consider the following two-component mixture model for the ith gene:As an alternative modelling approach, suppose we take the three values for the latent variables μi is the mean of the normal distribution, πi is the mixing proportion and ai, bi are the two end points of Uniform distribution respectively. Conceptually, there are only two populations of genes in model (4). There is a constitutively expressed population common to both tumor and normal samples  as well as a differentially expressed part (the uniform component). Such a model has been proposed in the situation for K = 1 in [where K = 1 in . Their ii (gene), the likelihood to be maximized for each gene is the following:For a fixed E step, we calculateIt is simple to find parameters that maximize the mixture model likelihood using the EM algorithm . Maximizt indexes the iteration of the EM algorithm, and eij is the unobserved indicator for the gene i in sample j to have originated from the uniform component of the mixture distribution. In the M step, we calculatewhere ai and bi are estimated by the minimum and maximum of the xij for a fixed j. We obtain estimates of eij, eij, they were set at zero for all points except those that were three standard deviations from the mean expression.As mentioned in , ai and N genes, while the EM algorithm is calculated for one gene at a time. Letting S and I denote the periodic skip in Gibbs sampler and the number of iteration respectively, the computation speed is O(NSI) for the MCMC algorithm, while it is O(N) for the EM algorithm. We compare the performances of the two methods on a real dataset in the Results section.Such an approach is much less computationally challenging compared to the Markov chain Monte Carlo described in the previous section. In particular, the MCMC method requires iterating over all a priori information on the eij. In the example considered here, the goal of meta-analysis is to compare localized tumors from metastatic tumors. By using phenotypic data, we are able to explicitly normalize the POE values in metastatic tumors to those in localized tumors. This involves making the likelihood a classification likelihood in which we know the identities of the samples. Letting Until now the discussion of transforming data into POE values has implicitly assumed that no phenotype information was used in the estimation algorithm. Nonetheless, there is often phenotypic data available in experiments, which provides The derivation of the E- and M steps remains the same except that at the M step,A| denotes the cardinality of the set A.where |P. We i. We ieijeij themselves. First, we can define sample subtypes based on the eij; this was also noted in [eij pertains to the false discovery rate [P(eij = 0|X) represents the Bayesian false discovery rate [While we describe potential meta-analytic approaches in the next section, we mention two interpretations of the noted in . Recentlnoted in  proposednoted in . A seconery rate . The falery rate .Xk be the study-wise transformed expression data for the study k. After fitting one of the two types of models described previously, the data we have are N genes that are profiled in each of the studies of interest, data integration is subsequently based on the rescaled values Let Treating the analysis  and is nResults section, the group label is the tumor type .One goal in genomic data analyses is determining which genes are differentially expressed. There are many available methods for doing this, such as SAM , analysiAnother important goal of genomic data analysis is to determine if there exists a gene expression profile that can distinguish the two groups. This is referred to as classification and is widely performed in gene expression studies. We deal with the two-group classification problem here. We use the compound covariate predictor  here; ot(a) Fit a univariate logistic regression model using group label as the response and gene expression as the predictor; obtain the estimated regression coefficient.(b) Define a risk index as the weighted average of gene expression, where the weight is the estimated regression coefficient from step (a).j by i in sample j, j = 1, ..., i in the combined dataset without the sample j. The risk index for sample j is a weighted linear combination of the expression profiles of the top p genes, where the ranking of the genes is based on their corresponding significance in the univariate logistic model fit. As a result, large positive values of RI indicate high risk of failure, whereas large negative values of RI indicate low risk of failure. Classification of sample j to the risk groups is then based on the jth leave-one-out risk index. The classifier is X*) = I{RIj > c}, with c being the empirical quantiles of the RI's. The number of genes p in a classifier is also treated as a parameter and optimized to minimize the prediction error rates. Note that to maintain unbiasedness of the procedure, we re-fit the univariate logistic regressions for each dataset with a withheld sample and re-rank genes based on the corresponding results. One other issue involves the use of POE values on training versus testing sets. As described in the previous section, we can obtain POE values either using or not using the class labels. In training sets, we typically use the class labels. We cannot use them in the test set as this will lead to overfitting. However, it is still possible to get POE values without using the phenotypic information for the test set.We will assess the performance of the genes found using classification accuracy. If we want to assess the performance of the classifier, we must deal with the issue of training and testing the model using the same data. An honest estimate of the prediction error rate is obtained using leave-one-out cross-validation. Define the risk index for sample The author(s) declares that there are no competing interests.DG and HC conceived the method and prepared the manuscript. HC implemented the software and performed the analyses. RS and AC contributed to the discussion. All authors have read and approved the final manuscript.Dendrogram of samples from liver data using the Conlon signature. This is a heatmap representing the hierarchical clustering results of the data in  using thClick here for fileDendrogram of samples from lung data using the Conlon signature. This is a heatmap representing the hierarchical clustering results of the data in  using thClick here for fileDendrogram of samples from prostate data using the Conlon signature. This is a heatmap representing the hierarchical clustering results of the data in  using thClick here for fileDerivation of conditional distributions for the MCMC-based POE algorithm. This file contains the details of the full conditional distributions from which samples of posterior distribution are drawn.Click here for file


Analysis of gene expression data provides an objective and efficient technique for sub-classification of leukemia. The purpose of the present study was to design a committee neural networks based classification systems to subcategorize leukemia gene expression data. In the study, a binary classification system was considered to differentiate acute lymphoblastic leukemia from acute myeloid leukemia. A ternary classification system which classifies leukemia expression data into three subclasses including B-cell acute lymphoblastic leukemia, T-cell acute lymphoblastic leukemia and acute myeloid leukemia was also developed. In each classification system gene expression profiles of leukemia patients were first subjected to a sequence of simple preprocessing steps. This resulted in filtering out approximately 95 percent of the non-informative genes. The remaining 5 percent of the informative genes were used to train a set of artificial neural networks with different parameters and architectures. The networks that gave the best results during initial testing were recruited into a committee. The committee decision was by majority voting. The committee neural network system was later evaluated using data not used in training. The binary classification system classified microarray gene expression profiles into two categories with 100 percent accuracy and the ternary system correctly predicted the three subclasses of leukemia in over 97 percent of the cases. Leukemia affects more than 44000 individuals each year in the United States alone, and it has one of the top mortality rates among different types of cancer.Traditionally, leukemia cells have been categorized considering their morphological appearance. Highly skilled resources are needed to identify the differences between tumor cells. The process can be tedious, time intensive and highly expensive. It is interesting to note that even with the adequate availability of the necessary resources the technique is not foolproof. Cells can appear similar morphologically but respond very differently to cytotoxic drugs and therapy.Recently, several studies have reported utilization of machine learning techniques for differential classification of leukemia. The first study on leukemia subtype classification using microarray data was performed by Golub et al.This study used the leukemia gene expression data collected by Golub et al.absent” calls across samples were eliminated from consideration. Genes with less than 2.5 fold change across samples were eliminated from consideration. Genes with expression values of less than 20 and greater than 16000 were thresholded to 20 and 16000 respectively. The remaining gene expressions were normalized so that the expression levels are in the range of −1 to 1. For the binary system, non-parametric t-test was used to select informative genes. The genes were resorted according to increasing p-values. Highly informative genes can be obtained at the top of the list. In case of the ternary system, an analysis of variance (ANOVA) was carried out across three classes for each gene set.The training dataset consisted of gene expression profiles for 37 patients. Each profile comprised of 7129 gene expression values. The preprocessing was carried out on the same lines of Dudoit et al.It had been reported that neural network classifiers based on about top 100 genes outperformed classifiers with significantly less or more genes.The trained networks were subjected to initial validation. The initial validation set comprised of eight samples encompassing all subclasses of leukemia. Networks that performed best in evaluating the initial validation dataset were considered in the recruitment of a neural network committee. The best networks were recruited into a committee based classification system. The majority opinion of the individual members of the committee formed the overall decision.The performance of the recruited committee was evaluated using a fresh dataset of 27 samples. These consisted of 17 B-cell ALL, 2 T-cell ALL and 8 AML samples. The samples of the final evaluation dataset were different from those present in the training dataset and the initial validation dataset. The data preprocessing steps led to filtration of 2334 genes which were deemed non-informative and eliminated from consideration. After the genes were ordered according to increasing p-values, the genes at the top of the list yielded a high level of differential information. The top 250 genes from this list were considered for preparation of parameter sets for the neural networks. For the binary classification system, the preprocessed data was used to train thirty six neural networks. The trained networks were initially tested against an initial validation set comprising of eight samples. The worst case performance was from a network which correctly predicted only four of the eight samples of the initial validation set. The best case performance was achieved by one network which rightly classified all 8 samples. Five top performing networks were recruited into a committee. The committee gave 100 percent classification accuracy for the initial validation set as shown in Similarly, in case of the three class problem, the preprocessed data was divided into ten groups of 25 genes each. Each of these groups was used to train around 10 to 12 neural networks. A total of 115 networks were trained in this manner. Each of the trained networks was validated against the initial validation datasets. The worst case performance was from 5 networks which correctly predicted only four of the eight samples of the initial validation set. The best case performance was achieved by 20 networks which rightly classified all 8 samples. Networks which accurately predicted at least 7 out of the 8 test cases were considered. A committee of 11 networks was formed and evaluated against the final testing dataset of 27 samples. The recruited committee gave 100 percent classification accuracy for the initial validation set . The recThe present study represents an application of committee neural networks for leukemia classification using microarray gene expression data. The classification systems required a very simple preprocessing procedure and gave an accuracy of 100 percent for a two class classification problem  and 97.1The present study represents a step forward in sub-classification of Leukemia. Several researchers have worked on the Leukemia dataset as a two class problem and obtained accuracies in the range of 80 to 100 percent in their efforts. Golub et alThe present study developed a technique of using different data parameters to train different networks. The committee consisted of member networks trained with different data. The size of the informative gene sets has varied from less than ten to greater than hundred. The drawback of using small gene sets in designing classification systems is that the decision is based on a very small informative set thus raising questions on the reliability of the system. Since the expression of a gene is dependent on a lot of conditions some known while others unknown, it is never advisable to make a decision based on a very small gene expression set. Utilization of large gene expression sets, suffer from other problems. Although, it ensures that enough information is available to base a decision upon, the drawback of using large datasets is the increased complexity of the machine learning algorithms and, the high probability of numerical instability in the system. The present study utilized the positives of both approaches. By incorporating several processing elements with small input parameter sets (25 inputs per processing element) each, we kept the mathematical complexity as low as possible. At the same time, by using 250 informative genes, our classification system based its judgment on a sufficiently large informative set. It was observed that the 250th most informative gene had a p-value between classes less than 0.001 thus giving an idea about the level of information that the genes imparted to the system.The preprocessed set comprised of the top 250 highly informative genes. It is known that genes interact with each other through biological networks such as regulatory networks and signaling networks. To identify the interconnection and reduce the redundant genes in the classification models are yet to be done. In this study, we relied on the neural networks with varying architectures to interpret the information content of each gene. More than 100 networks were trained using the sample sets of 25 genes each. The architectures of the individual networks varied in the number of hidden layers, the hidden layer neurons, the transfer functions and the learning functions. Although the training experiments were scheduled to terminate either upon reaching 1000 epochs or an error goal of 1e–17, it was observed that network convergence was obtained well before the limiting conditions were reached. The networks gave a consistent performance in terms of training time and accuracy for one to two hidden layers comprising of 14 to 25 hidden neurons. The performance became inconsistent when these parameters were modified. Intermediate validation was carried out on a set of eight samples.For the two class classification problem, the committee neural networks provided 100 percent accuracy for both initial and final validation data sets. In case of the three class problem, the initial validation gave 100 percent accuracy. However, the classifier misclassified a T-cell ALL sample as an AML when tested against the fresh validation set. One reason for this misclassification was the limited number of samples in the training set. Although the neural network converged to zero, only five samples of T-cell ALLs were present in the training set. Availability of more samples to represent the class of T-cell ALLs may have improved the prediction accuracy.Reddy et alThe gene expression profiles, each more than 7000 genes, were successfully processed to identify a subset of 250 genes that could distinguish the classes from each other. A committee neural network system was designed to classify leukemia into ALL and AML using the microarray gene expression data. The classification accuracy was 100 percent. A committee neural network system for three class classification was also developed to classify leukemia into T-cell ALL, B-cell ALL and AML using the expression data. The committee members were trained with different input parameters. The committee, through majority voting, correctly classified a total of 34 of the 35 validation data sets, yielding an accuracy of 97.14 percent for the three class classification problem.


Toxicogenomics can measure the expression of thousands of genes to identify changes associated with drug induced toxicities. It is expected that toxicogenomics can be an alternative or complementary approach in preclinical drug safety evaluation to identify or predict drug induced toxicities. One of the major concerns in applying toxicogenomics to diagnose or predict drug induced organ toxicity, is how generalizable the statistical classification model is when derived from small datasets? Here we presented that a diagnosis of kidney proximal tubule toxicity, measured by pathology, can successfully be achieved even with a study design of limited number of training studies or samples. We selected a total of ten kidney toxicants, designed the in life study with multiple dose and multiple time points to cover samples at doses and time points with or without concurrent toxicity. We employed SVM (Support Vector Machine) as the classification algorithm for the toxicogenomic diagnosis of kidney proximal tubule toxicity. Instead of applying cross validation methods, we used an independent testing set by dividing the studies or samples into independent training and testing sets to evaluate the diagnostic performance. We achieved a Sn (sensitivity) = 88% and a Sp (specificity) = 91%. The diagnosis performance underscores the potential application of toxicogenomics in a preclinical lead optimization process of drugs entering into development. Drug discovery and development is an expensive and time consuming process. It is estimated that about one third of drug candidates are terminated due to lack of clinical safety or toxicity concerns -3. IdentThe kidney is a major organ for filtration, secretion, re-absorption and ultimately excretion of drugs or drug metabolites. As a consequence of its primary function, the kidney is especially vulnerable to toxic insults by various drugs or xenobiotics, and thus nephrotoxicity is one of the major concerns in preclinical safety evaluation. Despite the morphological complexity of the kidney, the renal tubular epithelial cells stand out as one of the most sensitive components in the kidney and are thus highly susceptible to damage. Drug induced tubular damage has been well documented and studied extensively . MoleculThe success of statistically modeling microarray gene expression data to diagnose or predict renal toxicity is often constrained by a limited number of samples in combination with a large number of features (genes) to be monitored. This common conundrum is typically the case with most "omics" studies and has been referred to as "the curses of sample sparsity and feature dimensionality"  which ofThe support vector machine (SVM) algorithm  is one oTwo groups of independent in-life studies were carried out by Merck and Charles River Laboratories . For histopathological evaluation, a kidney from each animal was examined in situ, dissected free, and fixed in 10% neutral buffered formalin. Histopathology was performed on kidney from all animals . Fixed tissues were trimmed, embedded, and sectioned. Slides were stained with hematoxylin and eosin. To collect kidney samples for RNA profiling, the second kidney from each animal was dissected free and placed in tubes containing RNAlater™ (Qiagen). The tubes were allowed to remain at room temperature for a minimum of 30 minutes (maximum of 2 hours) before freezing at -20°C.Similarly for Merck conducted studies, rats were anesthetized under isoflurane, bled via the vena cava, exsanguinated and necropsied. Kidneys were subject to preparation for histopathology examination and toxicogenomics study as described above.RNA extraction, expression profiling, data processing and quality control were performed as previously described . Briefly, RNA was extracted from tissues using a combination of TRIzol RNA extraction  with RNeasy RNA extraction kit . Expression profiling was carried out using Rosetta custom arrays consisting of 22.5 K 60 mer oligonucleotides (plus control sequences) representing rat genes, Rat 2.25 K chip. The arrays were synthesized using The Agilient inkjet printing method. Two Cy3- and Cy5- two color reverse hybridization was applied. Individual samples were hybridized against a pool of RNA from time matched (concurrent) control animals. The ratio of individual animal expression to control pool was used for all data analysis. All hybridizations were performed in duplicate, with fluor reversal (Cy3 or Cy5) in the second hybridization. The resultant fluo-reversed pairs were combined to give a single ratio measurement for each gene of each sample.SVM light is from Thorsten Joachims . BrieflySpotfire™ is a type of data analysis and visualization software licensed from business intelligence company TIBCO . SpotfirThe paradigm used for the study design or data analysis assumes that any test compound can be toxic at a given dose and time. A drug is safe as long as margins which fall short of its toxicity can be established. Therefore, preclinical drug safety assessment is more concerned with diagnosed drug toxicity relative its effective dose and duration of dosing. A study design of multiple doses with different time points covering toxic dose/time and non toxic dose/time enables the differentiation of gene expression changes associated with toxicity from those due to pharmacology and can potentially define safety margins. If the study design is expanded to incorporate several structurally different compounds which induce the same toxicity by pathology, the different pharmacological effects reflected in the gene expression can be further diminished or neutralized in the analysis (by cancelling each other out), while at the same time gene expression changes associated with the defined common toxicity are qualified for the purpose of diagnosis of such toxicity. As an alternative to the conventional analysis which would require involving large numbers of studies or compounds in the training set, the above reasoning underscores the importance of this type of focused expression profiling design for diagnosis drug induced toxicity.As described in materials and methods, a total of nine kidney proximal tubule toxicants and one glomerular toxicant were selected for the study. All of them are known for inducing kidney toxicities, mainly proximal tubule toxicities identified as necrosis/degeneration by pathology. Puromycin and Tobramycin are known to cause a combination of glomerular and proximal tubule toxicity . The toxA standard approach to the pathological evaluation was employed. Significant histopathological finding for PT toxicity were summarized animal by animal. Merck studies are shown in supplement data, Charles River studies are shown in Table Supervised learning algorithm, SVM, was employed as the binary classification algorithm for its reported better performance as discussed in introduction. Histopathology observations were used as the anchor for the SVM training and testing for the kidney proximal tubule toxicity classification. Histopathology scores of 1 or higher for tubular toxicity were viewed as toxicity class and labeled as +1, while vehicle controls or treated samples with histopathology score of 0 were assigned in negative class and labeled as -1. Such class labels were used to supervise the SVM learning during SVM training and to measure the SVM model performance while in testing. SVM class designations for samples from Charles River Laboratories are listed in table In brief, there were a total of 250 Merck samples. Forty-one of these samples have no histopathology data available (not performed), leaving 209 samples with histopathology, to be included in the data analysis. 96 of the 209 Merck samples had tubular toxicity (pathology grade 1 to 5). In the studies conducted by Charles River Laboratories, a pathology evaluation was carried out on 128 samples, and 45 out of 128 had tubular toxicity indicated by pathology including 4 vehicle controls. The tubular pathology in the vehicle controls was considered non drug treatment related and was potentially misclassifications. These four vehicle samples could be excluded from analysis. Microarray gene expression data for all the samples was generated using the Agilient rat 22.5 K chip. The log ratio of the gene expression for the ratio of treated versus control pool was used as the feature values in SVM training and testing described below.Since the number of Merck samples is more than that of Charles River samples, Merck samples were used for training the SVM algorithm. Charles River studies were used as the independent testing set. When training SVM with Merck samples, linear kernel function (linear SVM) was selected and leave one out cross validation (LOOCV) was performed. For performance testing with the independent Charles River study samples, a positive SVM score indicates predicted positive or toxic and negative SVM score indicates predicted non toxic  for each gene and each sample with kidney proximal tubule toxicity. The averaged product value obtained over all toxic samples in the training sample set is used to rank all the genes. We took the top ranked 100 positive weighted and top ranked 25 negative weighted genes to generate a heatmap using spotfire™ (Methods). Figure 9 to eachThe positive prediction and negative prediction values by this SVM classification model are also listed in Table By reversing the data sets and use the Charles River study for training, the Merck samples for testing, we can achieve an accuracy of 81%, which is similar to but less than the 87% accuracy achieved above. This is most likely due to the fact that there are almost twice as many Merck samples as Charles River samples. Knowing the acceptable performance of the linear SVM model by independent testing, as a common practice in data mining to ensure optimal training, we can actually train the linear SVM model with both Merck and Charles River samples for future diagnosis of kidney proximal tubule toxicity.Using toxicogenomics data to diagnose histopathology of kidney proximal tubule toxicity has usually achieved accuracy in the eighties. Thukral SK et al.  attempteOn the other hand, for diagnosis of drug induced concurrent toxicity, in this report, the SVM model built with thousands of genes as features gave highly desirable performance, without requesting the understanding of how genes in the SVM model contribute to the classification. If interpretability of the diagnosis is of concern, feature selection algorithms could be applied to identify the more important genes or features for the classification or diagnosis of the toxicity of interest.An additional exercise (not shown) using random half of the genes on the microarray to do the classification, similar performance could be achieved. Thus reducing the number of genes in the model does not really affect the classification performance as much. This implies that there is rich and maybe redundant classification information in the gene expression profiles. Such rich toxicogenomics diagnosis information, in turn, confirms that the study design of small number of compounds representing different pharmacology is a working design for diagnosis of concurrent toxicity (identified by histopathology).Our effort here was primarily to apply SVM and microarray gene expression profiles in diagnosis of concurrent kidney proximal tubule pathology. It could be potentially applied to diagnosis of other well defined drug induced toxicity. With the cost of profiling experiments going down, such toxicogenomics approach could be applied early in lead optimization or it would even be integrated into preclinical drug safety assessment processes, so to reduce cycle time and improve attrition rates in drug development.The author(s) declare that they have no competing interests.Merck study samples. The data provided represent SVM analysis results on Merck study.Click here for file


Standard genetic mapping techniques scan chromosomal segments for location of genetic linkage and association signals. The majority of these methods consider only correlations at single markers and/or phenotypes with explicit detailing of the genetic structure. These methods tend to be limited by their inability to consider the effect of large numbers of model variables jointly. In contrast, we propose a Bayesian analysis of variance (ANOVA) method to categorize individuals based on similarity of multidimensional profiles and attempt to analyze all variables simultaneously. Using Problem 1 of the Genetic Analysis Workshop 15 data set, we demonstrate the method's utility for joint analysis of gene expression levels and single-nucleotide polymorphism genotypes. We show that the method extracts similar information to that of previous genetic mapping analyses, and suggest extensions of the method for mining unique information not previously found. The extent to which the natural variation in gene expression is heritable has recently been the subject of some intriguing studies -5 the moThe Problem 1 data set for the Genetic Analysis Workshop 15 (GAW15) consists of 3554 lymphoblastoid gene expression values observed in a sample of 194 healthy individuals from 14 three-generation CEPH (Centre d'Etude du Polymorphisme Humain) Utah families. Additionally, the sample contains genotypes for 2882 autosomal and X-linked SNPs for each of these individuals. The challenge thereby presented is to measure and characterize the heritability of expression phenotypes.In this paper we present our analysis of the GAW15 Problem 1 data set based on a variable-selection method tied to a Bayesian ANOVA model that has been shown to be particularly effective in separating signal from noise within expression data.Yi, j be the gene expression value for the jth gene on the ith individual, where j = {1, 2,..., M} and i = {1, 2,..., n}. Further, let Gi denote the group membership of the ith individual, where Gi ={1, 2,..., g}, let I(·) denote an indicator function, and let nk = #{i: Gi = k} denote the number of individuals belonging to group k, such that the total sample size is Let θj is the baseline effect for the jth gene, βk, j is the group-differential effect (comparing the jth gene in group k against a baseline group) and εi, j is the error term. The errors are assumed to be independently distributed with mean zero such that E = 0 and Var = σj2. This means that every gene can have its own variance component. Aside from this, there are no distributional assumptions about the data.where Yi, j+ denote data produced by this transformation, and select a baseline group g. Then the data are further centered and scaled byThis multigroup model can be restated as a linear model as described in Ishwaran and Rao  and the N = (n - ng)M is the total sample size and where spike and slab hierarchical model [Following some additional pre-processing steps  to general model  is givenj is the diagonal matrix with diagonal entries obtained from γj = t, and γ = t is a M(g - 1) -dimensional hypervariance vector. Each design matrix Xj is chosen for orthogonality such that XjtXj = NI. The prior distribution for the coefficient variances, π (dγ), is a continuous bimodal prior in which one spike component specifies small values, thus favoring small values of βj, and a right continuous tail specifies preferences for large non-zero values of βj.where Γβ values is the optimal summary measure to use. Hard thresholding is done using a data-adaptive rule based on looking for coalescence of the posterior variance at a value of 1.0 with corresponding large posterior mean values , and coclumpy dependence across genes [The model is robust to non-normality of the response variable, robust to ss genes  and alloss genes ).expression profile is a multi-gene pattern of expression values by which a given individual may be assigned membership to one of a set of mutually exclusive groups or categories. Our strategy for determining expression profiles is to ask whether gene expression measurements for each individual can be classified as up, down or equi-regulated with respect to a "median" family expression profile. Interestingly, this can be handled as a special case of the multi-group model in what is termed a no-baseline analysis, and by accruing this information across genes we obtain a novel profile from which to assign individuals to expression profile clusters across families. This process allows us to define expression profiles using all of the genes present rather than trying to do things on a gene-by-gene basis. This also means that a parameter vector of length equal to the number of genes on the chip must be estimated for each individual. In order to conduct such an analysis, we extended the data set with pseudo samples obtained as follows: for each possible pairing of individuals within each pedigree, we calculate the pair-wise difference vector and add it to the existing data set. The effect of taking all possible pair-wise differences between family members is to increase the sample size while at the same time centering the expression profiles on a per-family basis. Each vector difference is added as a pseudo sample to the existing set of data, and for a pedigree of K individuals, the total number of such additions is (K)(K-1). Note also that with the generation of the pseudo samples and by use of the no-baseline variation of the spike and slab BAM model, we are implicitly allowing each individual to act as a "group".For our purposes, a gene The generation of pseudo samples and the inherent familial structure of the pedigrees means that the expression values will be correlated. But as previously indicated, the spike and slab BAM model is robust to these correlations and gives correct inferences.gap statistic [p-values were determined and then adjusted for multiplicity by using the Benjamini-Hochberg version of false detection rate (FDR) control [After estimating Zcut vectors from the BAM no baseline model for each individual, we clustered those vectors across individuals using agglomerative hierarchical clustering, wherein the optimal number of clusters is determined by Tibshirani's tatistic . In orde control .Using the GAW15 data set, we sought to relate gene expression to SNP genotypes collected from pedigrees of healthy individuals. In essence, we were trying to determine the genetic determinants of the natural variation of gene expression in this data set. Given that we do not have explicit groupings of individuals into different phenotypic groups  our approach was to use the clustered expression profiles described above and relate these latent clusters to the SNP genotype information using the spike and slab multi-group methodology described above. Similar arguments about correlation between SNP measurements across individuals still hold but the correlations across individuals within the latent expression profile clusters are expected to be less strong than the expression correlations within same pedigree. We can apply the spike and slab BAM model to SNP data because the model does not make distributional assumptions about the data generating mechanism, and employs a limiting distribution result for its thresholding rule.We began by generating the additional pseudo samples as described above, and were thus able to increase the analyzable sample size from the original 194 records to 2496. On average, each individual from the original data set was mapped to a unique set of 182 expression profile records. The no-baseline analysis on these data revealed that, on average, 30% of each individual's genes make a significant contribution to profile differentiation. At the same time, none of the genes examined could be immediately eliminated from further consideration. In other words, there were no genes with Zcut values equal to zero for all individuals in the sample. The sample proportion of non-zero Zcut values, on a per-gene basis, ranged from 18% to 88%, indicating that some genes are likely to appear in all expression profiles, while others are more discriminatory.C1, C2, C3, C4, and C5.We performed an agglomerative hierarchical cluster analysis on the Zcut values to determine how individuals cluster with respect to their expression profiles. The optimal number of clusters for the data set was estimated to be five based on the gap statistic measure Fig. , and theC1, C2, and C4. The middle generation of parents (Gen 2) clusters as a mix of C1, C2, and C3, and the "sibship" generation (Gen 3) clusters almost exclusively together as C2. Because we are interested in finding evidence of heritable effects within the expression data, we sought to identify clusters that appear to represent mutually exclusive generations, which in this case would be C2 and C4.The clusters themselves appear to suggest a fairly strong age effect, as shown in Figure heat map designed to reveal specific points of aggregation within the data  and C4 (shown in the rightmost block) represent distinctive groups on the basis of their gene expression levels.We also summarized the cluster analysis into a ata Fig. , and ourF statistic and its p-value, and then applied a p-value correction for multiple testing to control the false discovery rate [α = 0.01  as the reference group.Having characterized the data in terms of expression-driven clusters, we then turned to the question of how those clusters relate to the SNP genotypes. We recoded the SNP markers from the given coding: {0/0, 0/1, 1/1} to an additive coding: {0, 1, 2} such that the homozygote for the minor variant at a given genetic locus  is coded as 2. We then applied the multi-group ANOVA model described above to the SNP data, but instead of a no-baseline model, we compared the different clusters against each other, using the sibship cluster , and of this reduced set, 118 had nonzero Zcut scores for exactly one cluster. And of these, only 90 SNPs were applicable to the C2 vs. C4 comparison.Of the 2882 SNP loci analyzed under the model, 1480 had non-zero Zcut scores for at least one of the four possible clusters , both of which were found to significantly differentiate C4 from C2. Additionally, two of our most significant genes, DDX17 (chromosome 22), and DSCR2 (chromosome 21) have shown evidence of cis-acting and both cis- and trans-acting linkage, respectively [While none of our significant SNPs correspond to reported associations with expression phenotypes, we did find two significant SNPs (rs945942 and rs945945) within a previously reported gene  have beC2 and C4. Explicit modeling of the SNP/expression relationship that best explains this profile will enable us to model networks of genes that may be co-regulated. In most cases this type of clustering of phenotypic profiles (using expression or other phenotype data or some mixture thereof) can assist with incorporation of protective as well as susceptibility alleles inherent in the profiles.We applied a whole-genome approach to determine the best set of SNPs that can explain cluster membership, and found that 90 SNPs best explained the difference between groups In conclusion, our method can be generalized to address problems other than expression profile modeling. The key aspect we wish to emphasize is its applicability to variable selection when the number of variables is large compared to the number of observations. The ability to accurately partition a data set, based on the variance of SNP genotype or expression levels, into representative clusters of individuals is particularly appealing and likely to have many applications beyond those we have discussed here.The author(s) declare that they have no competing interests.


Cy3 and once with Cy5. While this strategy ensures an automatic correction of the gene-specific labelling bias, it also induces dependencies between log-ratio measurements that must be taken into account in the statistical analysis.In individually dye-balanced microarray designs, each biological sample is hybridized on two different slides, once with We present two original statistical procedures for the statistical analysis of individually balanced designs. These procedures are compared with the usual ML and REML mixed model procedures proposed in most statistical toolboxes, on both simulated and real data.The UP procedure we propose as an alternative to usual mixed model procedures is more efficient and significantly faster to compute. This result provides some useful guidelines for the analysis of complex designs. DNA microarray technology is a high throughput technique by which the expression of the whole genome is studied in a single experiment. Experiments must be well organized and design issues are crucial, see ,2. In duglobally balanced but not individually balanced.1. Balanced reverse dyes for which each biological sample is hybridized only one time and therefore present with only one dye, on only one array Table . These dIndividually-balanced design for which each biological sample is divided into two parts, one hybridized with Cy3 on one array and the other with Cy5 on another array. Each biological sample is hybridized exactly two times  can be performed. On the contrary, if we consider the second class of designs, the experimental units are not independent, a feature that must (or must not) be accounted for. The literature about the statistical study of such designs is limited: some papers proposed some theoretical contributions for their analysis [A major point to notice is that the statistical analysis may be very different for the three classes of design. The analysis of the first and third classes is straightforward and well described in articles : analysis ,13, but A and B are compared in a two-color cDNA microarray experiment, with n biological samples for each condition. The design is the following: each RNA sample  is divided into two parts, one labelled with Cy5 and the second labelled with Cy3. Then 2n microarrays are hybridized with respectively A1Cy5 and B1Cy3, B1Cy5 and A2Cy3, A2Cy5 and B2Cy3, and so on till BnCy5 and A1Cy3,  on two different arrays. We propose a simple, efficient and robust method for the statistical analysis of this experiment.We consider here the simplest individually-balanced dye-switch design: two conditions Xi is the expression measure on the log-scale, for a given gene, corresponding to condition A on array i. Let j(i) denote the sample number corresponding to condition A and array i.After the normalization step, Yi is the expression measure for the condition B sample on the same array, and j'(i) the sample number corresponding to condition B and array i. In the following the gene index is not present in order to simplify the mathematical expressions, but it is important to note that all the terms in the following models are gene-specific. Here we use an analysis of variance (ANOVA) model for the expression measure as introduced by [Similarly, duced by .Xi and Yi is the following:The model for whereμA and μB are the population mean expression measures for condition A and B.• δl(i) is a two-level fixed effect corresponding to the dye effect. δl(i) = δ1 (resp. δ2) for all the samples labelled with Cy5 (resp. Cy3). This term accounts for the gene-specific dye bias.• Bj(i) represents an independent gaussian random term with mean 0 and standard deviation σB, corresponding to the random effect of sample j(i). This variable is specific to the biological sample and is called biological error, related to the variability of the biological material inside each population A and B.• Mi represents an independent gaussian random term with mean 0 and standard deviation σM. Mi is the effect of the spot associated to the gene under concern in microarray i and has the same value for the two samples which are hybridized on array i. This error term takes into account the spatial heterogeneity in each array that affects both Cy3 and Cy5 measurements.• Ti represents an independent gaussian random term with mean 0 and standard deviation σT, corresponding to the technical variability, including the steps of labelling, hybridization and measure of intensity of fluorescence. This variable has a specific value for each combination gene×dye×sample, even if the samples are hybridized on the same array and at the same spot, so that Ti and Ti and Mi are the two components of the so-called technical error.• Di = Xi - Yi, i = 1,...,2n. Using equation (1) we obtain:Let which may be writtenwhereμ = μA - μB is the true differential expression between conditions A and B for the gene under concern,• BDi = Bj(i) -Bj'(i) is a random variable with mean 0 and standard deviation σB,• TDi = Ti - σT,• δ = δ1 - δ2 is the difference between the Cy3 and Cy5 dye effects. This term accounts for the gene-specific dye bias.• Di follows a Gaussian distribution with mean E(Di) = μ + (-1)i+1δ and variance Di, Dj) are equal to zero except the following ones:Each variable n + 1 = 1.with the convention that 2In this study, we present and compare different strategies for the statistical analysis of individually-balanced designs. The article is organized as follows. In the Results section, five statistical procedures to analyze individually balanced designs Table  are compIn this section, we investigate the efficiency of five test procedures for the differential analysis of datasets corresponding to the design of Table • Naive Method NM: for each gene, the naive test statisticis computed.• Unbiased Paired Method (UP): for each gene, the unbiased paired statisticC must be positive. In practice, the estimated value may be negative. In such a case, C is truncated at 0.is computed. Notice that from the Methods section, the theoretical value of • Unbiased Unpaired Method (UU): for each gene, the unbiased unpaired statisticCXY must be positive. If not, CXY is truncated at 0. Furthermore, the unbiased variance estimator is CXY is non-negative, the variance estimator may have a negative value. In such a case, the variance can be fixed at a given threshold (0.001 in the following).is computed. As for the previous method, the value of • Mixed Model with ML estimation (ML): for each gene, model (1) is adjusted with the Maximum Likelihood algorithm.• Mixed Model with REML estimation (REML): for each gene, model (1) is adjusted with the Restricted Maximum Likelihood algorithm.It is important to consider both the ML and REML algorithms for the mixed model since each algorithm has its own advantages. While ML is known to provide biased estimates of the variance components, computations are faster and the algorithm does converge. REML gives unbiased estimates of the parameters, but may not converge if the number of observations is small. Both ML and REML computations were performed using the R package Maanova .μ = μA - μB . The parameter σT was fixed at 1. For each combination of the parameters, 10,000 genes were simulated.To study the behavior of the 5 procedures, we performed a simulation study using model (1). We considered 3 different values for μ = 0. Table n = 5, the discrepancy between the theoretical and the actual level is even worse for REML than for the other methods.We first consider the case From these first observations we conclude that we can discard procedures NM and ML, since in differential analysis an effective control of the Type I error rate is necessary.n - 2 degrees of freedom), that could be more erroneous in the case of the UU statistic.We now compare the performance of the 3 remaining procedures to detect differentially expressed genes. Table n larger than 20, we observe that the absolute values of the approximate REML T-test range from 0 to 32, except for some genes where the absolute value is larger than 400. These outliers come from an erroneous estimation of the variance of the mean difference, that is evaluated to be  0. This does not happen with (UP) since the estimated variance is max, i.e. the variance is overestimated to avoid outliers. Notice that despite this overestimation in many cases the power of UP is larger than the power of REML.An interesting point is that UP results are more stable than the REML results. If we consider sample sizes We now consider the important question of computational time for the 2 competitive procedures UP and REML. Since microarray experiments can involve hundreds of thousands of genes, it becomes critical to use efficient algorithms for the statistical analysis of the data. Table Furthermore, REML can result in inconsistent estimates of the variance, as shown in the previous sections, or may not converge. Table The impact of pregnancy on the cattle endometrium transcriptom is investigated in . In MammThe Venn diagram of Figure The high number of DE genes specifically found with REML is odd. Figures p treatments, where p is 3 or higher. Figure An important application of the methodology proposed in the previous section is the analysis of loop design experiments. Loop and interwoven loop designs were initially proposed in  to compaN1 → S1 → N3 → S3 → N5 → S5 → N2 → S2 → N4 → S4 → N1• S1 → G1 → S3 → G3 → S5 → G5 → S2 → G2 → S4 → G4 → S1• N1 → G2 → N3 → G4 → N5 → G1 → N2 → G3 → N4 → G5 → N1• Each of these comparisons corresponds to the design of Table The Teleost fish experiment aims to compare 3 populations of fish . Five individuals were examined in each population to determine the variation in gene expression between populations. Each individual is used to probe four cDNA microarrays, according to the design of Figure In , the sigA, D, G and V stand for Array, Dye, Gene and Variety, respectively. Then the 4 measurements corresponding to a given individual are averaged, and an F statistic is computed per gene to check whether the variety effect is significant or not.where This strategy roughly amounts to the UU test procedure of section when the number of treatments is higher than 2. The main difference is that in model (4), the model does not include the array random effect which takes into account the dependency between two measures on the same array. According to the results of section, this implies that the variance estimator is biased, leading to a loss of power.2 for 2 of the 3 tests to be significant under H0 at level 0.02). This is a good compromise between the 0.01 threshold adopted in the original articles with no correction for multiple testing, and the 0.5 × 10-4 (0.05/907) threshold given by a 5% level per test combined with a Bonferronni multiple testing correction. While the drawback of our strategy is to replace one test by three, the advantage is that the variance estimate is unbiased.As an alternative, we perform the statistical analysis using the UP procedure. Each pairwise comparison between 2 varieties is made, and a gene is declared differentially expressed if at least 2 of the 3 tests are significant. Each test is performed at the level 0.02, meaning that for a given gene, the nominal level is roughly 0.001  method. At a first glance, the (UP) procedure seems less powerful than the original method since only 9 genes are found here compared with the 15 genes of the original article. But due to the threshold adopted by the authors in , the expThese 3 classes roughly correspond to the three populations of interest, up to 2 misclassified observations. In the original article, the partition in 3 classes gaveWith only 9 genes (rather than 15), the classification obtained with (UP) is improved compared with the classification of the original method.Random terms taking into account the array and the sample effects must be included in the statistical model at the gene level for dye-switch experiments. We showed on simulations that the naive paired T-test, which does not take into account the biological sample effect, leads to more false positives than expected, especially when the biological sample effect is high. It may be safely used only when the biological variance is lower than the technical variance. The REML estimate for mixed model provides an approximatively correct false positive rate, at the price of high computational complexity, lack of convergence for low or medium sample sizes and sometimes spurious results. To the contrary, the UP method we propose is easy to implement and not computationally intensive. The method is protected against spurious results, leading to a more robust and powerful analysis than REML when the biological variability is high and the number of samples low, an usual situation in microarray experiments.For small sample size experiments, it is advised to use regularized T-test, see -19. ReguIn this paper the proposed estimate of the variance of the differential expression is assessed for the comparison between two conditions in a dye-switch design. The same methodology could be extended to more complex designs involving more than two conditions and duplicate hybridizations of the same biological sample on different arrays.μ is According to expression (2), an unbiased estimator of μ, we need to estimate To perform a statistical test on parameter The naive estimate of T-tests. But in a dye-switch experiment the variables Di - Di and μ + (-1)i+1 δ and μ, respectively. Hence we consider the alternative estimatorwhich is used to perform paired whereThe expectation of this alternative estimator isS2 is a downward biased estimator of V we can deduce the following unbiased estimate of H0 = {μ1 = μ2} isFinally, the "unbiased paired t-statistic" for testing the null hypothesis n - 2 df under H0.which is approximately distributed as a Student distribution with 2Let j is the biological sample index , i(j) and i'(j) are the arrays on which sample j has been hybridized. where The usual unpaired estimate of ThereforeThis method overestimates the true varianceThe overestimation is more dramatic as with the convention that CXY has the same mean. Therefore an unbiased estimate of It is easy to see that the second sum in TMH and JJD conceived the method and prepared the manuscript. TMH and JA implemented part of the software and performed the statistical analysis. NM made the Embriogenomics experiment under the direction of OS. All authors contributed to the discussion and approved the final manuscript.


Analysis of gene expression data for tumor classification is an important application of bioinformatics methods. But it is hard to analyse gene expression data from DNA microarray experiments by commonly used classifiers, because there are only a few observations but with thousands of measured genes in the data set. Dimension reduction is often used to handle such a high dimensional problem, but it is obscured by the existence of amounts of redundant features in the microarray data set.Dimension reduction is performed by combing feature extraction with redundant gene elimination for tumor classification. A novel metric of redundancy based on DIScriminative Contribution (DISC) is proposed which estimates the feature similarity by explicitly building a linear classifier on each gene. Compared with the standard linear correlation metric, DISC takes the label information into account and directly estimates the redundancy of the discriminative ability of two given features. Based on the DISC metric, a novel algorithm named REDISC (Redundancy Elimination based on Discriminative Contribution) is proposed, which eliminates redundant genes before feature extraction and promotes performance of dimension reduction. Experimental results on two microarray data sets show that the REDISC algorithm is effective and reliable to improve generalization performance of dimension reduction and hence the used classifier.Dimension reduction by performing redundant gene elimination before feature extraction is better than that with only feature extraction for tumor classification, and redundant gene elimination in a supervised way is superior to the commonly used unsupervised method like linear correlation coefficients. Here, p is always much larger than n, which hurts generalization performance of most classification methods. To overcome this problem, we either select a small subset of interesting genes  or construct K new components summarizing the original data as well as possible, with K <p (feature extraction).DNA microarray experiments are used to collect information from tissue and cell samples regarding gene expression differences for tumor diagnosis ,2. The oGene selection has been studied extensively in the last few years. The most commonly used procedures of gene selection are based on a score which is calculated for all genes individually and genes with the best scores are selected. Gene selection procedures output a list of relevant genes which can be experimentally analyzed by biologists. The method is often denoted as univariate gene selection, whose advantages are its simplicity and interpretability. However, interactions and correlations between genes are omitted during gene selection, although they are of great interest in system biology. Furthermore, gene selection often fails to pick relevant genes, because the score they assign to correlated genes is too similar, and none of the genes is strongly preferred over another.Feature extraction is an alternative to gene selection to overcome curse of dimensionality. Unlike gene selection, feature extraction projects the whole data into a low dimensional space and constructs new dimensions (components) by analyzing the statistical relationship hidden in the data set. Although feature extraction is often criticized for the lack of interpretability, the new components often give good information or hints about the data's intrinsic structure. Researchers have developed different feature extraction methods in applications of bioinformatics and computational biology -5, whichConsidering of the fact that gene selection and feature extraction algorithms have complementary advantages and disadvantages. Feature extraction algorithms thrive on correlation among features but fail to remove irrelevant and redundant features from a set of complex features. Feature selection algorithms fail when all the features are correlated but do well with informative features. It would be an interesting work to combine gene selection and feature extraction into a general model. In practical, the simplest way is to apply a preliminarily gene selection procedure before feature extraction.As to analysis of microarray data whose speciality is the huge amount of genes with few examples, it is believed that there exist many redundant genes among the full gene set . Preservp genes in n microarray samples are collected in an n × p data matrix X = (xij), 1 ≤ i ≤ n, 1 ≤ j ≤ p; of which an entry xij is the expression level of the jth gene in the ith microarray sample. As we only consider binary classification problems, the labels of the n microarray samples are collected in the vector y. When the ith sample belongs to class one, the element yi is 1; otherwise it is -1. The matrix SX denotes the p × p covariance matrix of the gene expressions.Some notions used in this work are clarified here. Expression levels of XT represents the transpose of X, X-1 represents the inverse matrix of X. The matrices X and y used in the following are assumed to be centered to zero mean by each column.Besides, || • || denotes the length of a vector. C = 1.According to the framework proposed in this paper, dimension reduction is performed by combining redundant gene elimination with feature extraction, then the classifier is used to perform classification on the extracted feature subsets. The novel proposed algorithm REDISC (Redundancy Elimination based on DIScriminative Contribution) is compared with the commonly used algorithm RELIC (Redundancy Elimination based on LInear Correlation) to perform redundant gene elimination on two microarray data sets, i.e. Colon and Leukemia, where the threshold of δ in REDISC and RELIC is varied from 0.1 to 0.9. Feature extraction is performed by principle component analysis (PCA) and partial least squares (PLS). The classifier is a linear support vector machine (SVM) with Statistical results of the number of remained genes after performing REDISC and RELIC are showed in Figure Comparative results of BACC obtained by SVM on the new feature sets by using PCA or PLS after performing REDISC and RELIC are illustrated in Figure The results in Figures δ, REDISC obtains more compact subsets than RELIC does.1. Both REDISC and RELIC dramatically reduce the number of genes from the original data. With the same value of δ = 0:1, RELIC always obtains better results than REDISC, but when δ increases, the results of REDISC are better than those of RELIC. On average, REDISC obtains better results than RELIC does.2. With 3. When the results of REDISC and RELIC reach their highest point, REDISC uses less features than RELIC.4. The effect of REDISC is positive for both PCA and PLS, while RELIC loses in some case, i.e. BACC of PCA on the Leukemia data set.5. REDISC and RELIC with different threshhold values produces different results, no one is optimal for all the data sets.The experimental results prove our assumption that redundant features hurt performance of feature extraction and classification, other considerations on the above results are listed as below:1. The results confirm that there exist many redundant genes in the microarray data and it is necessary to perform redundant gene elimination. Usually, there are four types of features in one data set, I is strong relevant features, II is weak relevant but non redundant features, III is weak relevant and redundant features and IV is irrelevant features. I and II are the essential features in the data sets, and III and IV should be removed . The pre2. REDISC obtains better results with less features than RELIC, which shows that REDISC has the higher ability to select relevant features and eliminate the redundant features than RELIC. Proper redundant feature elimination help improve performance of feature extraction and classification. Simply reducing redundant genes by linear correlation is not always positive, because without considering the label information in the data set, linear correlation does not give properly redundancy estimation. REDISC takes label information into account for redundant gene elimination, which may be viewed as a supervised way. Since the final step is classification, so a supervised redundant gene elimination is better than an unsupervised one like RELIC.3. It shows the performances of dimension reduction is improved when redundant genes are properly eliminated. The improvement for PLS is much more dramatic than that of PCA. A possible reason is redundant genes obstruct the performance of supervised methods more obviously, since supervised methods often build more precisely model than unsupervised ones.Dimension Reduction is widely used in bioinformatics and related fields to overcome the curse of dimensionality. But the existence of amounts of redundant genes in the microarray data often obscure the application of dimension reduction. Preliminarily redundant gene elimination before feature extraction for dimension reduction is an interesting issue, which was often neglected.In this paper, a novel metric of redundancy based on Discriminative Contribution (DISC) is proposed, which directly estimates the similarity between two features by explicitly building linear classifiers on each genes. The REDISC algorithm (Redundancy Elimination based on Discriminative Contribution) is also proposed. REDISC is compared with a commonly used algorithm RELIC (Redundancy Elimination based on Linear Correlation) on two real microarray data sets. Experimental results demonstrate the necessariness of preliminarily redundant gene elimination before feature extraction for tumor classification and the superiority of REDISC to RELIC, a commonly used method. This work is an attempt to propose a general framework performing dimension reduction for tumor classification by combing redundant gene elimination and feature extraction. More investigation need to be done on the efficiency of fusion of feature selection with feature extraction in the future.In this paper, we propose a novel framework for dimension reduction by combining redundant feature elimination with feature extraction to improve performance of classification. The framework is illustrated as in Figure Redundant gene elimination is the critical part in the framework, we propose a novel algorithm based on discriminative ability to improve performance of commonly used linear correlation, which is described in detail in the following subsections. Feature extraction is performed by using two methods, one is supervised, i.e. partial least squares, another is unsupervised, i.e. principle component analysis, which are briefly introduced in the following subsection. As for the classifier, support vector machine is used.As redundant features have no contribution for classification, we consider eliminating them preliminarily before feature extraction, which has the following benefits:1. Eliminating redundant features improves classification accuracy. In general, original microarray data sets have many irrelevant and redundant genes, which hurts performance of feature extraction. In practical, biologists often expect noises are reduced, at least in some extent, during the stage of feature extraction. But, if some redundant genes are reduced beforehand, performance of feature extraction may be improved.2. Preliminarily feature selection facilitates the application of feature extraction. Compared with modeling on the original data directly, the computational and RAM consumptions of feature extraction on preliminarily gene selected data are much less. Especially for the RAM consumption, most feature extraction methods are often not practical for high dimensional data, since the requirement of loading all data into RAM at one time. However, any additional gene selection procedure may bring some extra computation, so the computational complexity of preliminarily feature selection must not be too high.3. Preliminarily feature selection improves the interpretability of the components. The meanings of the components are always difficult to be interpreted in feature extraction. Biologists often analyze the relation between extracted components and original features by the coefficients, but it is obscured by the large amount of genes. Reducing a number of original features is obviously helpful when the components are needed to be related with original genes manually.χ2 statistic, odds ratio etc.  to a new set of K predictor variables, T = , which are linear combinations of the original variables. In mathematical terms, PCA sequentially maximizes the variance of a linear combination of the original predictor variables,Principle component analysis (PCA) is a well-known method of feature extraction . The basi <j. The orthogonal constraint ensures that the linear combinations are uncorrelated, i.e. Cov = 0, i ≠ j. These linear combinationssubject to the constraint are known as the principal components (PCs).K is determined by the number of nonzero eigenvalues, which is the rank of SX, and K ≤ min. But in practical, the maximum value of K is not necessary. Some tail components, which have tiny eigenvalues and represent few variances of original data, are often needed to be reduced. The threshold of K often determined by cross-validation or the proportion of explained variances [p and the number of samples n, is in the order of min. In other words, the cost is O(pn2 + n3) when p > n.The maximum number of components ariances . The comPartial Least Squares (PLS) was firstly developed as an algorithm performing matrix decompositions, and then was introduced as a multivariate regression tool in the context of chemometrics ,13. In ry, the task of regression, and reconstruct original matrix X, the task of data modeling, at the same time.The underlying assumption of PLS is that the observed data is generated by a system or process which is driven by a small number of latent (not directly observed or measured) features. Therefore, PLS aims at finding uncorrelated linear transformations (latent components) of the original predictor features which have high covariance with the response features. Based on these latent components, PLS predicts response features y and the original predictor variables X,The objective of constructing components in PLS is to maximize the covariance between the response variable i <j. The central task of PLS is to obtain the vectors of optimal weights wi  to form a small number of components, while PCA is an "unsupervised" method that utilizes the X data only.subject to the constraint Like PCA, PLS reduces the complexity of microarray data analysis by constructing a small number of gene components, which can be used to replace the large number of original gene expression measures. Moreover, obtained by maximizing the covariance between the components and the response variable, the PLS components are generally more predictive of the response variable than the principal components.O(npK), i.e. the number of calculations required by PLS is a linear function in terms of n or p. Thus it is much faster than the method of PCA for K is always less than n.PLS computes efficiently with a cost only at Feature extraction methods extract components to represent original data, which are linear or non-linear transformations of original genes. Although the new subspace is effective for data analysis, no original gene is excluded during the process, which often obstructs the explanations of PCs. In order to solve this problem, eliminating redundant genes before dimension reduction is an alternative way.S = {} ⊆ {ℝn × {-1, 1}}ℓ, SVM discriminant hyperplane can be written asSupport vector machines (SVM) proposed by Vapnik and his co-workers in 1990s, have been developed quickly during the last decade , and sucw is a weight vector, b is a bias. According to the generalization bound in statistical learning theory [where g theory , we needξi is introduced when the problem is infeasible. The constant C > 0 is a penalty parameter, a larger C corresponds to assigning a larger penalty to errors.in which, slack variable Two microarray data sets used in our study are listed in Table Colon used Affymetrix oligonucleotide arrays to monitor expressions of over 6,500 human genes with samples of 40 tumor and 22 normal colon tissues. Expression of the 2,000 genes with the highest minimal intensity across the 62 tissues were used in the analysis [Leukemia The acute leukemia data set was published by [ished by , which cWe use the stratified 10-fold cross-validation procedure, where each data set is firstly merged and then split into ten subsets of equal size. Each subset is used as a test set once, and the corresponding left subsets are combined together and used as the training set. Within each cross-validation fold, the gene expression data is standardized. The expressions of the training set are transformed to zero mean and unit standard deviation across samples, and the test set are transformed according to the means and standard deviations of the corresponding training set. We use 10 fold cross validation because the 10 × 10 cross-validation measurement is more reliable than the randomized re-sampling test strategy and the leave-one-out cross-validation due to the correlations between the test and training sets, some detail discussions can be found at .C = 1 is used as the classifier, which is trained on the training set to predict the label of test samples. Figure The linear Support Vector Machine (SVM) with In order to precisely characterize the performance of different learning methods, we define several performance measures below (see ). Here TSensitivity is defined as Specificity is defined as BACC  is defined as Precision is defined as PPV  is defined as NPV  is defined as Correction is defined as The authors declare that they have no competing interests.Guo-Zheng Li and Xue-Qiang Zeng proposed the idea, designed the experiments and wrote the paper; Xue-Qiang Zeng performed experiments; Geng-Feng Wu helped in writing the paper; Mary Qu Yang helped design the experiments; Jack Y. Yang conceived and guided the project.