Clustering methods are widely used on gene expression data to categorize genes with similar expression profiles. Finding an appropriate (dis)similarity measure is critical to the analysis. In our study, we developed a new measure for clustering the genes when the key factor is the shape of the profile, and when the expression magnitude should also be accounted for in determining the gene relationship. This is achieved by modeling the shape and magnitude parameters separately in a gene expression profile, and then using the estimated shape and magnitude parameters to define a measure in a new feature space.K-means clustering procedure to perform analyses. Applying these algorithms to a simulation dataset, a developing mouse retina SAGE dataset, a small yeast sporulation cDNA dataset, and a maize root affymetrix microarray dataset, we found from the results that the algorithm associated with the first feature space, named TransChisq, showed clear advantages over other methods.We explored several different transformation schemes to construct the feature spaces that include a space whose features are determined by the mutual differences of the original expression components, a space derived from a parametric covariance matrix, and the principal component space in traditional PCA analysis. The former two are the newly proposed and the latter is explored for comparison purposes. The new measures we defined in these feature spaces were employed in a TransChisq is very promising in capturing meaningful gene expression clusters. This study also demonstrates the importance of data transformations in defining an efficient distance measure. Our method should provide new insights in analyzing gene expression data. The clustering algorithms are available upon request.The proposed  K-means clustering [With the explosion of various 'omic' data, a general question facing the biologists and statisticians is how to summarize and organize the observed data into meaningful structures. Clustering is one of the methods that have been widely explored for this purpose -3. In paustering , self-orustering , and varustering -9.Recent research in clustering analysis has been focused largely on two areas: estimating the number of clusters in data -12 and tPearson correlation coefficient and Euclidean distance, which however, in some situations, could be unsuitable to explore the true gene relationship. The Pearson correlation coefficient is overly sensitive to the shape of an expression curve, and the Euclidean distance mainly considers the magnitude of the changes of the gene expression. For other model-based methods [The most commonly used distances or similarity measures for analyzing gene expression data are the  methods -9,15, thSpearman Rank Correlation, CLARITY was defined for detecting the local similarity or time-shifted patterns in expression profiles [Y =  with all components generated from the same Poisson distribution of mean 100. Clearly, the differences among the components in Y are due to the distribution variance and ranking in this case is meaningless. Briefly, Spearman Rank Correlation cannot distinguish the real differences from random errors in some situations and thus may provide a wrong estimate of the pattern.In recent literature, several advanced measures with emphasis on the expression profile shape have been developed in different contexts -18. In pBy separately modeling the shape and the magnitude parameters in a gene expression profile, we developed a new measure for clustering the genes when the profile shape is a key factor, and when the expression magnitude should also be accounted for in determining the gene relationship. The approach is to use the estimated shape and magnitude parameters to define a Chi-square-statistic based distance measure in a new feature space. An appropriate feature space helps summarize the data more effectively, hence improving the identification of gene relationships. We explored different transformation schemes to construct the feature spaces, which include a space with features determined by the mutual differences of the original expression components, a space derived from a parametric covariance matrix, and the principal component space in PCA analysis . The forK-means clustering procedure to perform clustering analyses. We designated the algorithm using the measure from the first transformed space as TransChisq, and the one associated with the principal component space as PCAChisq. The space derived from a parametric covariance matrix is not included in comparison for computational reasons (see Methods). For evaluation purposes we also implemented a set of widely used measures into the K-means clustering procedure, including Pearson correlation coefficient (PearsonC), Euclidian distance (Eucli), Spearman Rank Correlation (SRC), and a chi-square based measure for Poisson distributed data (PoissonC) [TransChisq outperforms other methods. Using the gap statistic [TransChisq was also found to be advantageous in estimating the number of clusters. The underlying probability model of our method was adopted from the model of PoissonC, a method for analyzing the expression patterns in Serial Analysis of Gene Expression (SAGE) data [The new measures associated with different feature spaces were employed in a oissonC) . All theoissonC) , a smalloissonC) , and a moissonC) . The restatistic ,25, TranGE) data . The MATTransChisq, PCAChisq and other methods to a simulation dataset, a yeast sporulation microarray dataset, and a mouse retinal SAGE dataset.First, we will illustrate the property of the proposed new transformations by applying them to a maize gene expression dataset. Then we will present the applications of The maize dataset, consisting of nine Affymetrix microarrays, was generated to investigate the gene transcription activity in three maize root tissues with three replicates for each tissue: the proximal meristem (PM), the quiescent center (QC) and the root cap (RC) . 2092 siTransChisq to the data. Figures Firstly, we applied the transformation employed in TransChisq, is that the information carried by each component is explicit, and hence the region in the new space corresponding to each class can be clearly determined.We then applied the transformation suggested by a parametric covariance matrix to the same data (see Methods). Figures For comparison, we performed a traditional PCA analysis to the same data. Figures This example demonstrates the advantage of the proposed new data transformations over the traditional PCA in keeping class information intact.TransChisq to a simulation dataset to evaluate its performance. For comparison purposes, other modified K-means algorithms, i.e. PCAChisq, PoissonC, PearsonC, and Eucli were also applied to the same dataset.We applied μ) and variance (σ2) of the Normal distributions are constrained by σ2 = 3μ and described in Table The simulation dataset consists of 46 vectors of dimension 5 and the components are independently generated from different Normal distributions. The mean  to a set of mouse retinal SAGE libraries. The raw mouse retinal data consists of 10 SAGE libraries (38818 unique tags with tag counts ≥ 2) from developing retina taken at 2-day intervals. The samples range from embryonic, to postnatal, to adult [For further validation we applied to adult . Among tTransChisq, PCAChisq, PoissonC, PearsonC, Eucli and SRC were applied to group these 153 SAGE tags into five clusters. Here we assumed that the number of the clusters, K, is known. A study to evaluate the performance of different measures in determining K when it is unknown can be found in a later section of this paper. The clustering results showed that TransChisq and PCAChisq outperform others  to optimize the clustering results. The best result can be reached when the first 5 PCs were used: 3 out of the 39 genes were incorrectly grouped. This optimal result is the same as the one from TransChisq. However, in practice, it is not feasible to exhaust all possible combinations of PCs to search for the optimal clustering result.For K-means clustering analysis is how to estimate K, the number of clusters. In the recent literature the Gap statistic was found useful [R2)' with that expected under an appropriate reference null distribution. A high R2 value represents high variability between clusters and high coherence within clusters. Below we sketch how to calculate the Gap statistic: Let Dk be the R2 measure for the clustering output when the number of clusters is k. To derive the reference expected value of Dk, the elements within each row of original data are permuted to produce the new matrices with random profile patterns. Assume B such matrices are obtained. Then for each matrix, a new R2 is calculated based on the original clustering output and the pre-selected similarity measure. The average of these R2's, denoted by Dk. With Dkand An unsolved issue in d useful ,26. The k)= Dk - Gap. This study shows that when the number of clusters, K, is unknown, the Gap Statistics can be used to estimate K, and TransChisq is favorable over others on estimating the true number of clusters in both experimental datasets.For comparison, we used different measures including TransChisq, to group genes with similar expression shapes. The expression magnitude was considered when measuring the shape similarity. Results from applications to a variety of datasets demonstrated TransChisq's clear advantages over other methods. Furthermore, with the gap statistics, TransChisq was also found to be effective in estimating the number of clusters. Regarding the computational efficiency, TransChisq, PCAChisq and PoissonC have similar costs but usually run a few times (2 to 5 times) slower than the PearsonC and Eucli.In this study, we proposed a method, K-means clustering procedure to reveal the important gene expression patterns. In addition to K-means, our new measure can also be implemented in other clustering methods, e.g., hierarchical clustering [We have embedded different measures in the ustering , to perfTransChisq would be effective when it is necessary to consider the magnitude information in measuring the shape similarity. In clustering analyses of SAGE and microarray data, very often the magnitude information should be taken into account, whereas the shape could be a more critical factor to determine the gene relationship.We view different measures as complementary rather than competing in that each has its advantages. In general, Although the proposed method is very promising, it does require further study on possible data transformation schemes when the original data show a more complex structure, or when the clustering purpose is different. We suggest our method could provide new insights to the applications of different data transformations in clustering analysis of gene expression data.The underlying probability model of our new measures was adopted from the work of Cai et al. , where tPoissonC and PoissonL were developed under this context [SAGE is one of the effective techniques for comprehensive gene expression profiling. The result of a SAGE experiment, called a SAGE library, is a list of counts of sequenced tags isolated from mRNAs that are randomly sampled from a cell or tissue. As discussed in Man et al. , the sam context . The metYi(t) be the count of tag i in library t, and Yi = (Yi(1),..., Yi(T)) be the vector of counts of tag i over a total of T libraries. Yi(t) is assumed to be Poisson distributed with mean γit. To model the magnitude and shape of the expression profile separately, Cai et al. [γit = λi(t)θi, where θi is the expected sum of counts of tag i over all libraries, and λi (t) is the contribution of tag i in library t to the sum θi expressed in percentage. The sum of λi(t) over all libraries equals to 1. So λi(t)θi redistributes the tag counts according to the expression shape parameter (λi(t)'s) but keeps the sum of counts over libraries constant. The genes with similar λi(t)'s over t are considered to be in the same cluster.Let i et al.  further m with the common shape parameter λ = (λ(1),..., λ(T)), the joint likelihood function for Y1, Y2,...,Ym isFor a cluster consisting of tags 1,2,..., λ and θ1,..., θm areThe maximum likelihood estimates of f. The larger the log-likelihood is, the more likely the observed counts are generated from the expected Poisson distributions. So for a cluster consisting of tags 1,2,..., m, a likelihood based measure is defined asFormula (2) forms the basis of the following two measures for evaluating how well a particular tag fits in a cluster. One natural measure is to use the log-likelihood function: log The other measure is based on the Chi-square statistic, a well known statistic for evaluating the deviation of the observations from the expected values. It is defined asL or D, the more likely the tags belong to the same cluster. We should also note that the statistics in measure (3) and measure (4) consider both the shape and magnitude information when measuring the cluster dispersion, i.e., the cluster is specified by the shape parameter λ, but the relationship of a tag to a certain cluster is determined by the deviation of observed counts (i i) from the expected values (i λ). Here i is the estimated profile shape of tag i (i = (i (1),...,i (T)) and i and λ directly.Using Chi-square statistic as a similarity measure, the penalty for the deviation from large expected count is smaller than that for small expected count. It is consistent with the above likelihood-based measure in that the variance of a Poisson variable equals to its mean. In general, the smaller the value of K-means clustering algorithm to perform clustering analysis. K-means clustering procedure [K clusters so as to minimize a measure of dispersion within the clusters. The algorithm is outlined below:Cai et al.  have emprocedure  generateK sets. Estimate initial parameters 1. All SAGE tags are assigned randomly to i to the cluster with minimum deviation from the expected model. The deviation is measured by either 2. In the (b+1)th iteration, assign each tag 3. Set new cluster centers 4. Repeat step 2 till convergence.c(i) denote the index of the cluster that tag i is assigned to. The above algorithm aims to minimize the within-cluster dispersion ∑iLi,c(i) or ∑iDi,c(i). The algorithm using measure L is called PoissonL, and the algorithm using measure D is called PoissonC. PoissonL and PoissonC perform similarly in applications. But PoissonC is more practical in terms of running time. So we use PoissonC for comparison in this paper.Let PoissonC is designed to group the objects by their departure from the expected Poisson distributions. The success of PoissonC has been shown in applications [PoissonC may be necessary. For instance, if the shape difference should be more emphasized in determining the relationship, the direction of departure of observed from expected may/should also be considered. As an example, we consider an expression vector Y =  and its relationship with two clusters with shape specified by λ1 =  and λ2 =  respectively. The expectation of Y in cluster 1 is Y would be expected to be closer to the first cluster because of the large value observed on the middle component in both Y and PoissonC, however, determines that Y has the same distance to Y and Y and PoissonC ignores the direction of departure. To address this omission we propose to emphasize the profile shape through suitable data transformations, and to define a distance measure in the transformed space. The construction of a proper feature space under a certain clustering purpose is essential to define an effective distance or similarity measure.ications ,21. HoweYi = (Yi(1),..., Yi(T)) the transformed vector Zi is of dimension T(T-1)/2 with components in the form of Yi(t1)-Yi(t2) for t1 = 1,..., T-1 and t2 = (t1 + 1),..., T.A simple yet natural data transformation to emphasize the expression shape is to consider the mutual differences of the original vector components. Given a gene with expression profile E(Yi(t1)-Yi(t2)) = (λi(t1)-λi(t2))θi and Var(Yi(t1)-Yi(t2)) = (λi(t1)+λi(t2))θi. For a cluster consisting of tags l, 2,..., m, we can define the following statistic to measure the cluster dispersion:According to the Poisson model in the previous section, t) and i can be estimated by formula (2). We call the modified K-means algorithm with this measure TransChisq. Applying it to the toy example in the previous section, TransChisq determines that Y is closer to where To better understand the effects of the proposed data transformation, we performed a simple simulation study and presented the results in Additional file Now we consider a data transformation determined by a parametric covariance matrix:R = cov(X) = (γij)i,j = 1,..., T, with γij = α > 0 if i = j and γij = β if i ≠ j,X is the data matrix with n observations on the rows and T variables on the columns, and R is the covariance matrix of the T variables. The matrix R in this form implies that the variables have identical variances and covariances with each other. These properties are biologically reasonable in that normalized arrays have identical distributions, hence equal variances. Also all pairs of variables would exhibit equal covariance (or un-correlated when β = 0) if each component had been equally important (or independent) to determine a class.where R. One set of column orthonormal eigenvectors, denoted by e1,e2,...,eT, is presented in Additional file Yi = (Yi(1),..., Yi(T)), a transformation based on R isA data transformation can be defined through the eigenspace of Zi =  = Yi (e1 e2...eT).e1 = [1/T, e2 = [1/T and e3 = [1/T, for a profile Y = , the component associated with e1 is Ye1 = (Y1 + Y2+...+YT)/e2 is Ye2 = (Y1-Y2)/Y1 and Y2; the component associated with e3 is Ye3 = (Y1+Y2-2Y3)/Y1, Y2 and Y3.A convenient property of this transformation is that each component has a clear meaning: with E(Zit) = E(Yi)et = (λi(1)θi,..., λi(T)θi)et, Var(Zit) = (λi(1)θi,..., λi(T)θi)Cov = 0 when t ≠ k. Then for a cluster consisting of tags 1, 2,..., m, we can measure the cluster dispersion by:According to the Poisson model, Strans in formula (5). As we discussed above, the component associated with e2 is (Y1-Y2)/Strans is equivalent to the space determined by e2 and all its row-switching transformations. We can also define a measure similarly through e3 or other eigenvectors. Strans seems to have the potential of losing the information carried by e3 and other eigenvectors. However, applications of TransChisq to a variety of datasets suggested that this potential information loss is minor and can be ignored in most cases in practice. In fact, the row-switching transformations of e2 make up most of the information included in e3 and other eigenvectors.We should note the connection between this measure and the Strans_N comes from the fact that it is defined based on only one set of eigenvectors. The orthonormal eigenspace of a covariance matrix is not unique  and different eigenspaces may result in different values of Strans_N . Although one can consider all possible eigenspaces to overcome the limitation of Strans_N, it is not computationally feasible.A potential shortcoming of Strans_N to several different datasets, we observed that i) using the eigenvectors e1,e2,...,eT in Additional file Strans_N performs very similarly to Strans and ii) when a different set of eigenvectors used, the clustering results can be different, though the difference is not obvious. These results are not presented in this paper.Applying For comparison purposes, we applied PCA to transform the data . PCA is e1 e2...eT in measure (6) by the eigenvectors from the sample covariance matrix, we defined a new measure and implemented it in the PCAChisq. The Results section gives examples showing the positive and negative effects of the PCA transformation. In general, PCAChisq is difficult to use. Firstly, it is unclear what types of variances the principal components are capturing . Next, it is unclear how many principal components should be used. The optimal number of PCs is unavailable before we compare the results to the ground truth. To be brief, PCAChisq is only efficient when the principal components happen to match the key features that determine a cluster.In some theoretical  and empin genes in T experiments, the expression of gene i in experiment t, Xi(t), is assumed to be normally distributed with mean μi(t) = λi(t)θi and variance t) = kλi(t)θi, where k is an unknown constant. The derivation of the maximum likelihood estimates (MLEs) of λi(t) and θi under the normal model is rather involved. So we borrowed the estimators in formula (2). It can be shown that i in formula (2) is unbiased and t in formula (2) is consistent under the restricted normal model [see Additional file i and t available under the normal model, TransChisq, PCAChisq and PoissonC can be applied.We explored the potential application of the proposed measures to a clustering analysis of microarray data. We proposed the following restricted normal model for this purpose. The parameter notations in the Poisson model were adopted. Given a microarray dataset of expressions of TransChisq in analyzing microarray data (see the Results section). We should also note that TransChisq would deliver less promising results if the assumption on the relationship between the variance and the mean is seriously violated.For both oligonucleotide and cDNA microarray data, it is widely observed that there is strong dependence of the variance on the mean: variance increases with mean ,37. So iKK participated in the design of the study, performed the analysis and drafted the Results section of the manuscript. SZ, KJ and LJF provided the Maize root microarray data, which helped in motivating this research. SZ, KJ and LJF were responsible for the biological explanations on the results related to maize data. LC provided the developing mouse retina SAGE data and was responsible to the biological explanations on the clustering results related to SAGE data. IBL helped in formulating PCA related studies. HH conceived of this study, proposed the method, coordinated the collaborations and wrote the paper. All authors read and approved the final manuscript.One set of orthonormal eigenvectors. This PDF file contains one set of orthonormal eigenvectors referred in the Method section.Click here for fileProof of the properties of the estimators under the restricted normal model. This PDF file shows that the i in formula (2) is an unbiased estimator of θi and t) in formula (2) is a consistent estimator of λ(t) under the proposed restricted normal model.Click here for fileThe performance of new measures in a hierarchical clustering algorithm. This PDF file presents the application results of the hierarchical clustering algorithms with different measures implemented.Click here for fileTransChisq data transformation in measuring pattern similarityThe effects of the . This PDF file presents a simple simulation study for the effects of the data transformation in TransChisq with a comparison to PoissonC.Click here for fileThe guideline on the various parameters in the simulation dataset in Table Click here for file


Signal transduction pathways convey information from the outside of the cell to transcription factors, which in turn regulate gene expression. Our objective is to analyze tumor gene expression data from microarrays in the context of such pathways.p-value. This approach defines a projection of high-dimensional gene expression data onto low-dimensional pathway activity scores. For each dataset and many pathways we find a much larger number of significant samples than expected by chance. Finally, we find that several sample-wise pathway activities are significantly associated with clinical classifications of the samples.We use pathways compiled from the TRANSPATH/TRANSFAC databases and the literature, and three publicly available cancer microarray data sets. Variation in pathway activity, across the samples, is gauged by the degree of correlation between downstream targets of a pathway. Two correlation scores are applied; one considers all pairs of downstream targets, and the other considers only pairs without common transcription factors. Several pathways are found to be differentially active in the data sets using these scores. Moreover, we devise a score for pathway activity in individual samples, based on the average expression value of the downstream targets. Statistical significance is assigned to the scores using permutation of genes as null model. Hence, for individual samples, the status of a pathway is given as a sign, + or -, and a This study shows that it is feasible to infer signal transduction pathway activity, in individual samples, from gene expression data. Furthermore, these pathway activities are biologically relevant in the three cancer data sets. By usines [e.g. ). What wSignaling pathway activity scoring is a more direct measurement of biological processes than ontology mapping, which aims at finding over-representation of genes in various groups of contextual annotation. A cellular signaling pathway see Fig.  is compoTo characterize pathway activity, it would be desirable to have both proteomic and gene expression data. Gene expression data alone is not sufficient for assessing protein concentrations  and postCellular signaling pathways are subject to intense research, and current knowledge is compiled into databases such as STKE , TRANSPAi.e., which pathways have significantly co-expressed downstream targets. This is done both with and without accounting for the fact that downstream targets of a single transcription factor are correlated irrespective of pathway behavior. The second point of view is to assess pathway activity of individual samples, relative to the other samples in the same experiment, yielding an active or inactive status for each pathway in each sample. Finally, we relate the sample-wise pathway activity to clinical classifications of samples by way of contingency tables. Several pathways are found to be highly predictive of the clinical classifications.For the three data sets, we assess pathway activity from two related, but different, points of view. The first is to examine which pathways behave in a coherent way across the entire data set, In this section we assess variation in transcription factor and pathway activity across the samples. We then proceed to probe pathway activity in individual samples. Finally, we study the association between pathway activity and the clinical classifications of the samples.Group Correlation Score defined in Methods. The p-values were calculated using random reshuffling of the genes. Table et al., 8 out of 42 transcription factors have a p-value below 0.1, and for the Golub et al. data set, the corresponding numbers are 6 out of 39. Among these three data sets, the p-values are noticeably better for data sets with more samples and genes. With this in mind we conclude that the downstream targets of transcription factors are co-expressed in the these data sets, albeit not to a high degree. The full lists for all 3 data sets can be found in As a prelude to the study of pathways, we quantified the degree of correlation among downstream targets of single transcription factors. For this purpose we used the Group Correlation Score as above, applied to the downstream targets of entire pathways rather than those of individual transcription factors. Table p-value below 0.05, which is significantly more than expected by chance. However, many of the downstream targets have common transcription factors, which might be the major cause of the co-expression. To eliminate such a contribution we used theExclusive Group Correlation Score, which considers only pairs of downstream targets lacking common transcription factors. The p-values for the Exclusive Group Correlation Score are also shown in Table p-values are larger, they are still significant; out of 20 pathways with more than one transcription factor, 12 have a Exclusive Group Correlation Score p-value below 0.05, which is still more than expected by chance. We conclude that the co-expression of downstream targets in a pathway can only in part be explained by the genes having common transcription factors. This co-expression at the pathway level justifies the view of pathways as functional units. Similar tables for the two other data sets are shown in p-values than expected by chance, albeit not as convincingly as the van 't Veer data set.Here we use the same Group Sample Score, which for each pathway designates every sample in a data set as either active or inactive, with an associated p-value.After having established that downstream target genes are co-expressed in some pathways, we proceeded to study the status of pathway activity in individual samples. To this end we employed the p-values and pathway activity status, for six samples in the van 't Veer data set. For example, in the first sample the RANK pathway is designated as inactive with a p-value of 0.004, whereas the inactiveness of the ER-induced pathway cannot be considered significant. The full tables for all samples and pathways in all 3 data sets are provided in Table p-value, defined in Methods, which gives the probability of observing at least this total number of significant samples for a pathway. The family-wise p-value assumes that the samples are independent, which is only approximately true since the mean expression value of a gene across all samples is zero. Corresponding tables for the two other data sets can be found in p-values are numerically different.Table α respectively. These two examples are illustrative; there is a substantial fraction of genes that are clearly upregulated in the active samples and downregulated in the inactive samples, as seen by the red upper right corner and green upper left corner. There are also many genes in the pathway that do not seem to be regulated, and even a few which act oppositely. The latter ones are possibly genes, that are repressed by the pathway. Similar figures for the entire genome (not shown) do not show large red or green upper corners, confirming the statistical analysis.In Figures χ2 p-values were calculated.We analyzed the association between sample-wise pathway activity and clinical classifications using contingency tables. For every pathway and data set, we divided the samples into three groups: Samples where the pathway was active at a 5% significance level, samples where it was inactive at a 5% significance level, and insignificant samples referred to as undecided. For each data set, contingency tables of pathway activity versus clinical classifications were created, and et al., the only available clinical classification is tumor type, i.e., ALL or AML. Table χ2 p-value below 0.01.In the data set of Golub et al., we investigated six clinical classifications: metastasis status (0), estrogen receptor status (20), progesterone receptor status (12), lymph node status (12), BRCA mutations (15) and histological grade (8). The numbers in parentheses refer to the number of significant contingency tables at the 0.01 level. The total number of pathways was again 29. For metastasis status, only 97 out of the 117 samples were labeled in the original data set, and this may contribute to the low degree of association between this clinical classification and pathway activity. However, similar results were obtained for the data set of Sotiriou et al., which indicates that it may be difficult to obtain any association between the pathways analyzed in this work and breast cancer metastasis status. Table et al., but with fewer significant associations.For the breast cancer data set of van 't Veer p-value cutoff makes the association to clinical classifications more specific but less sensitive. The complete set of contingency tables for all three data sets can be found in A general tendency of the contingency table analysis is illustrated in Table We have shown that downstream target genes of signal transduction pathways behave coherently in gene expression tumor data sets. First, we confirmed that downstream targets of transcription factors are correlated across samples. We then demonstrated that the same holds true for downstream targets of an entire pathway, even after discounting the correlations due to genes having a common transcription factor. The correlations for entire pathways were found to be more significant than those for individual transcription factors.Group Sample Score. With this score we classified the samples into those where the pathway was significantly active, significantly inactive or undecided, respectively. As seen in Table The presence of significant correlations confirms the expectation that gene expression is controlled by the activity of pathways. However, these correlations do not tell us in which samples a pathway is active or inactive. To reveal this, we devised the In many cases, the active/inactive pathway status was highly correlated with independent clinical classifications. This confirms the relevance of pathways for understanding of the underlying biology. Furthermore, the activity status of one or more pathways may be used to subdivide the samples into groups with distinct biological characteristics. Such a subdivision is feasible if, for instance, tumors of a certain clinical diagnosis are an agglomerate of several subtypes.Group Sample Score is natural if a pathway either induces all its downstream targets, or represses them. However, in most pathways some downstream genes are induced, while others are repressed. To account for a mixture of induction and repression, one should include a sign, or more generally a weight, to each term in the sum. Such a weight might even depend on the type of tissue and the environment. Since this information was not readily available for the studied pathways, all genes were weighted equally. Nevertheless, we obtained significant results, indicative of a dominant trend among the downstream genes. For the estrogen receptor (ER) pathway, we did have information about the sign, but instead of introducing a more general score for this pathway alone, we split the ER pathway into two parts, with induced and repressed genes, respectively. In the breast cancer data set of van 't Veer et al. [The r et al. , there wIt should be stressed that correlations, and the pathway activity status observed in a sample, are only defined relative to the other samples in the same data set. If a pathway were active in all samples, it would not show up in our significance test. The status of a pathway, as we define it, is given by the downstream genes, and the connection to ligands, receptors and other pathway components cannot be inferred from this analysis.et al. is NF-κB. This transcription factor is also the most one in the leukemia data set of Golub et al., whereas NF-κB1 is the most significant one in the data set of Sotirou et al Recently, NF-κB has been shown to be involved in the transformation from benign to malignant cells in inflammation-associated cancers. Pikarsky et al. [α (TNF-α) is shown to play an important role as an activator of NF-κB. Greten et al. [Table y et al.  demonstrn et al.  find simOur current knowledge of pathways, and of downstream targets of transcription factors, is far from complete. However, we find that the results presented herein constitute a proof of concept for analyzing microarray gene expression in the context of signal transduction pathways., using UniGene Hs build 171. For the analysis of the data sets, gene identifiers were converted into UniGene IDs and expression values of clones belonging to the same UniGene cluster were averaged.Transcription factors for 23 pathways were extracted from TRANSPATH . The dowThe following three publicly available data sets were analyzed:et al. [1. The breast cancer data set of van 't Veer et al. , consistet al. [2. The breast cancer data set of Sotiriou et al. , consistet al. [3. The leukemia data set of Golub et al. , derivedGroup Correlation Score and Exclusive Group Correlation Score the affymetrix average difference values were logarithm transformed, since the Pearson correlation is very sensitive to single outlier samples. For the Group Sample Score the original differences were kept. We denote the expression value for gene g in sample s by xgs, with missing values allowed. We normalized the expression values in two steps. First, for each sample, the mean of all genes was subtracted, in order to ensure that no samples are up- or down-regulated on average. The transformed expression values satisfy:The data sets in ,16 are gSecond, for each gene, the mean of all samples was subtracted from the expression values of that gene, yielding:The second normalization implies that the expression value of a gene is measured relative to the same gene in other samples.g and h:To determine if a group of downstream target genes is significantly co-expressed, a total score of the group is needed. Two scores were used here, both based on the Pearson correlation of a pair of genes  is the mean of expression values for gene g.where the sums exclude missing values and Group Correlation Score is defined as the sum of squares of Pearson correlations among all pairs of genes in a group of genes:The Group Correlation Score for the downstream target genes of a single transcription factor, as well as for those of an entire pathway.where the sum runs over all genes in the group. The square ensures that both correlations and anti-correlations contribute to the score. We use the Exclusive Group Correlation Score, on the other hand, is only applicable for the downstream targets of a pathway. It is defined asThe g and h that do not share any transcription factor.where the sum runs exclusively over pairs of genes p-value of a score is defined as the fraction of random cases, drawn under the null hypothesis, which achieve a higher score than the score tested. For both scores, GCS and EGCS, our null hypothesis is reshuffling of the genes on the microarray. This null hypothesis keeps the structure and overlap of all pathways fixed, but changes the identity of the genes.The s, and pathway, PW, the Group Sample Score is defined as follows:For each sample, where the sum runs over all downstream target genes of the pathway.p-values for the score being higher (p+) and lower (p-) than random, respectively, and the final p-value is given by two times the smaller of these two p-values:The null hypothesis is again reshuffling of the genes from the microarray. We are interested in pathways both with high and low scores. Hence, we consider the p = 2·min.p+ <p-, and inactive (-) otherwise.The pathway is said to be active (+) if N independent hypotheses are tested simultaneously, the probability to obtain K or more p-values below q is given by a binomial distribution:If family-wise p-value.We refer to this probability as the TB, MK, and CT implemented the algorithms employed. All authors contributed conceptually to the methods presented herein, as well as to the preparation of the manuscript.All additional files are in tab-delimited format. Tablel_Golub.csv Tablel_Sotirou.csv Table2_Golub.csv Table2_Sotirou.csv Table2_Veer.csv Table3_Golub.csv Table3_Sotirou.csv Table3_Veer.csv Table4_Golub.csv Table4_Sotirou.csv Contingency_Tables_Golub.csv Contingency_Tables_Sotirou.csv Contingency_Tables_Veer.csvClick here for file


Many of the most popular pre-processing methods for Affymetrix expression arrays, such as RMA, gcRMA, and PLIER, simultaneously analyze data across a set of predetermined arrays to improve precision of the final measures of expression. One problem associated with these algorithms is that expression measurements for a particular sample are highly dependent on the set of samples used for normalization and results obtained by normalization with a different set may not be comparable. A related problem is that an organization producing and/or storing large amounts of data in a sequential fashion will need to either re-run the pre-processing algorithm every time an array is added or store them in batches that are pre-processed together. Furthermore, pre-processing of large numbers of arrays requires loading all the feature-level data into memory which is a difficult task even with modern computers. We utilize a scheme that produces all the information necessary for pre-processing using a very large training set that can be used for summarization of samples outside of the training set. All subsequent pre-processing tasks can be done on an individual array basis. We demonstrate the utility of this approach by defining a new version of the Robust Multi-chip Averaging (RMA) algorithm which we refer to as refRMA.® arrays. We show that the refRMA workflow, when used in conjunction with a large, biologically diverse training set, results in the same general characteristics as that of RMA in its classic form when comparing overall data structure, sample-to-sample correlation, and variation. Further, we demonstrate that the refRMA workflow and reference set can be robustly applied to naïve organ types and to benchmark data where its performance indicates respectable results.We assess performance based on multiple sets of samples processed over HG U133A Affymetrix GeneChipOur results indicate that a biologically diverse reference database can be used to train a model for estimating probe set intensities of exclusive test sets, while retaining the overall characteristics of the base algorithm. Although the results we present are specific for RMA, similar versions of other multi-array normalization and summarization schemes can be developed. Many of the commonly used algorithms utilize models where parameters are estimated using data from multiple arrays. These approaches are typically used in the normalization and summarization steps. Examples of multi-array procedures are RMA, gcRMA, MBEI, and, most recently, PLIER [Pre-processing of Affymetrix GeneChipy, PLIER -4. Each y, PLIER -8. From A problem associated with these algorithms that has not received much attention is the limitation they impose on data archiving. When data from a new study becomes available, all arrays are pre-processed together to obtain expression measures. Because different data is used to define the normalization routine and estimate probe-effects, data from different studies might not be comparable because of this pre-processing bias. A solution is to group together the feature-level data for both experiments and re-run the pre-processing algorithm. However, this approach will be logistically impossible in the case of building large probe set-level reference databases. These databases can be used to quickly combine samples that may have been generated from a variety of different experiments into a user-defined data set based on some common attribute such as clinical or pathological status, treatment level, or technology-derived attribute. The lack of comparability between sets of samples normalized based on different schemes does not allow for archiving and continual updating of probe set-level data and ultimately prevents the analyst from combining disparately normalized samples into cohesive sets.There is a need to develop summarization schemes that offer both the statistical advantages of multi-array algorithms such as RMA, gcRMA, MBEI, and PLIER and the flexibility of a workflow that is not specific to a single set of samples. We utilize the RMA model as an example methodology to demonstrate the feasibility of such an approach,We have utilized an alternate RMA workflow that includes two distinct steps: (a) training of an RMA model based on a large number of biologically distinct Affymetrix GeneChip samples from Gene Logic's BioExpress reference database and (b) application of the resulting RMA model parameters to multiple test sets. We have named this alternate workflow, "Reference RMA" (refRMA), to emphasize the reliance of this methodology on a standardized training set of samples exclusive of a test set. Results in this paper are shown for the HG U133A Affymetrix GeneChip array for a specific implementation of refRMA trained using 1,614 .cel files representing 144 different organ types from four different pathology states. We show that the application of such a biologically diverse model to test data results in similar probe set-level data when compared to classic RMA outputs as measured by overall data structure, correlation, differential expression, and other metrics. Furthermore, a model trained on a large training set of biologically distinct samples seems to be robust for tissues that were not used in its training. Finally, we show that the model can be applied to data sets external to the site of microarray processing specific for the training set such that the model can be used universally.Two goals for building a common reference refRMA model are to incorporate as much biological variability as possible into the training set and to appropriately balance the principal sources of variability such that there is not a large degree of overrepresentation of any one of the types. The Gene Logic BioExpress reference database is comprised of thousands of samples from 144 tissues and cell types, which are represented in up to four pathological categories  as determined by a board certified pathologist's review of each sample. Individual brain regions were treated as separate organs due to their highly heterogeneous gene expression profiles.Selection of the samples was based on a two-step filtering system. The first step involved an initial balancing protocol that attempted to correct for unintended effects on the normalization due to organs that are disproportionately represented within the database. For instance, samples from organs such as liver, lung, and kidney have high numbers of samples, whereas samples from organs of more limited research may only have a few samples. A balanced pool of samples was created by capping the maximum samples per organ and pathological category at 20 samples and randomly selecting samples for those organs with sample numbers above the cap. This first filter resulted in a balanced pool of over 6,000 samples. The next filter randomly selected 1,500 non-blood samples from the balanced pool in order to accommodate computational memory limitations.Whole blood was considered a special case due to its unique expression profiles as compared with other tissue types, its relative importance in clinical genomics, and the usage of two types of blood processing protocols that have or have not undergone globin depletion procedures. The final number of samples used for the training set was 1,614 after the addition of 114 whole blood samples available at the time of model training. A summarized list of the individual organ or cell types, their pathological status as determined by a board certified pathologist, and the number of samples within each category is provided as a supplemental file; see The Full refRMA model trained as described above was used to summarize HG U133A arrays with spiked-in probe sets as part of the commonly used affycomp comparative benchmark system . This syTable The overall interpretation from the affycomp results is that refRMA outperforms MAS5 summarization, but does not perform as well as Classic RMA where the training set and test set are identical. This is to be expected under normal circumstances given that Classic RMA is highly specific to its training set. The spiked-in nature of the experiment introduces an additional probe-level effect into the system and most likely contributes to the size of the disparity between Classic and refRMA, as Classic RMA will be trained on this effect and refRMA will not. These results indicate a limitation to the applicability of refRMA to data that has been heavily manipulated. The manipulation may be due to exogenous addition of probe, such as in the case of a spike-in data set, or may be due to other sample processing factors such as alternate amplification and labelling processes. In addition, the data suggests that a refRMA model trained on a large, biologically diverse set of samples may not summarize highly tissue-specific probe sets as well as Classic RMA, although refRMA may outperform single chip-based summarization methods such as MAS5.To test the performance of the Full refRMA model on unmodified biological samples, a variety of normal control samples from multiple organs were extracted from the Gene Expression Omnibus (GEO) database based on the descriptive fields within each sample's annotation . Random As expected, the correlation to Classic RMA increases as the refRMA training set size increases until correlations approaching 1 are achieved. The Full refRMA model also approaches 100% correlation. The plot also indicates differences in correlation as a function of independence of training and test set arrays. As the GEO-based refRMA models randomly select from the same population as that of the 50 test set arrays, there are likely to be many common biological and technical factors such as microarray processing site, replicate arrays from the same treatment or disease group, etc. The Full refRMA represents a more independent training set. The slight drop in correlation suggests that this effect is observable, but not of substantial consequence.The results generally indicate that the Full refRMA model can be applied to data external to Gene Logic's processes with good confidence.® database that were not used to build the Full refRMA model. For each probe set, the mean intensity was calculated by averaging individual expression values across the 15 samples in each set. MA plots of the mean probe set intensities are shown in Figure Data structure was assessed after application of separate normalization schemes using 2 exclusive sets of 15 liver normal samples from Gene Logic's BioExpressMA plots of the mean probe set intensities were also constructed for comparisons of a single set of 15 normal liver samples summarized using either Classic RMA, Full refRMA, or a similar version of the Full refRMA where all liver samples have been replaced with samples from other organs. The results are shown in Figure Histograms for coefficients of correlations and variation are shown in Figure The histograms indicate almost identical distributions for both probe set-based variation and sample-based correlation for both the Classic RMA and Full refRMA workflows with a slight unfavorable shift for Full refRMA compared to Classic RMA. This small shift can be viewed as the necessary "cost" associated with the use of a static normalization scheme for true biological samples. Two small "bumps" in the distribution of Classic RMA correlations are evident. The bump centered at 0.83 is explained by a single sample's correlation to the rest of the data set. We were not able to explain the smaller bump centered close to 1. Interestingly, both are resolved by the refRMA model, which indicates potential differences between the two models. Unfortunately, it is not clear as to whether this is a case of Full refRMA more successfully dealing with technical artifacts on the chip or if it is a case of Full refRMA diluting true biological effects.Fold change and p-value conservation is important to establish with any summarization algorithm. Although it cannot be expected that different normalization and summarization techniques will select exactly the same probe sets, our goal is to show that a large number of probe sets are selected with both the Classic RMA and Full refRMA normalization schemes.Consistency of regulation events was compared for the case where the same model was used to summarize different test sets of 15 normal vs. 15 malignant liver samples using 20 bootstrapping iterations with sample replacement and for the case where Classic RMA and Full refRMA were used to summarize the same test set of 15 normal vs. 15 malignant liver samples .Results are shown in Figure One of the limitations with the Classic RMA workflow and other types of multiple array-dependent normalization and summarization schemes is that the resulting models are usually applied to the same samples that were used in its training. This tight dependency between the training and test sets results in a situation where the probe set data cannot be archived in a continuously updated database in the absence of a static normalization scheme.® reference database of human samples to build a static normalization scheme that can be applied to incoming human data from the HG U133A array on a continual basis. We have shown that a Full refRMA model, trained on a widely varying number of biologically distinct samples, results in similar probe set intensities compared to the classic RMA workflow as far as general data structure, similarity metrics, and number of regulation events. We have also shown that the Full refRMA model is applicable to data external to the processes underlying the BioExpress® database.To counter this limitation, we have used the extensive content from Gene Logic's BioExpressOur conclusion from the above observations is that the Full refRMA model can generally be used to summarize an archival database on an ongoing basis for incoming samples. By applying the model to all samples, we can create a probe set-level expression database that is both consistent across organ and pathology category and provides the inherent benefits of RMA summarization. This same observation may be transitive across other multi-array summarization algorithms such as gcRMA, MBEI, and PLIER or variants thereof. The appropriate reference objects would need to be trained and provided for each of these according to their basic input requirements.ad hoc manner . Analysts can use the normalization schemes on single sample set activities. A number of logistical advantages are inherent within such a workflow.Reference database normalization schemes such as the Full refRMA model described can also be applied to samples in an First, a pre-computed reference model can be used to alleviate memory-intensive calculations for studies of large sample sizes. This issue has been noted previously and has been addressed by re-sampling and partitioning methods . Models Second, any one of a number of the models could eventually be used as a standardization mechanism across the industry if a large number of users find such models applicable and valid for their data sets. Obviously, the degree of validation necessary for such an "industry-universal" model is extensive and is outside the scope of this publication. However, given the recent emphasis on microarray analysis standards, the development of a static normalization scheme on a widely varying training set for such algorithms as RMA is a useful starting tool.Despite attempts to maximize the applicability of any reference model for the multi-array algorithms, there are potential technical and biological variables which may degrade their performance. Microarray data generation protocols are comprised of many hands-on technical processes and reagents. The combination of all possible variants of each step and types of reagents prohibits claiming that any single model is robust for all variables. Sources of technical variability include different labeling technologies, PMT settings, human operators, RNA quality, reagent lots/manufacturer, and innate day-to-day variables. Despite this multi-step process, technical variability has been reported to be among the smallest variability factors within organ systems in human and mouse, although these studies have not attempted to quantify individual technical factors and have not assessed cross-laboratory factors ,15. In oIn addition to normal technical variability associated with microarray data generation, there are more extreme deviations in sample preparation associated with small- or micro-sample amplification, laser-capture microdissection techniques, and multi-gene knockout experiments that may perturb the probe-level signal distributions beyond what is reasonable for a conventional training set. The ability of the Full refRMA model to properly summarize these types of purposeful deviations has not been investigated and cannot be claimed. This concern is also generally applicable to the other multiple array-dependent algorithms.Biological variables such as organ, pathology state, gender, age, race, and others are well represented within the training set of 1,614 unique samples and the Full refRMA model should be relatively robust for these factors. We have demonstrated that samples from an organ naïve to a large-sample refRMA training set are normalized almost identically to a large-sample refRMA training set where the organ is well-represented.Finally, it should be noted that most applications of the RMA workflow involve a relatively small number of experiments (n < 50) obtained directly from .cel file data. We believe that the Classic RMA workflow should be utilized for these situations whenever possible due to the optimal applicability of a model to its component samples. The refRMA workflow is valuable for situations where summarized expression data is archived within large, continually updated enterprise database systems or for alleviating memory constraints for experiments of large size, but in no way substitutes for the Classic RMA workflow when study designs permit the use of the latter.® arrays such that it can be applied to a wide range of sample organ types and pathology states.In conclusion, a primary logistical issue associated with multiple array-dependent normalization and summarization algorithms is their reliance on a limited set of arrays to produce a model that is specific to that set. A reference model based on a training set comprised of a large, biologically diverse training set has been developed for the HG U133A GeneChipT(PMij) = ei + aj + εij, i = 1,..., I, j = 1,..., J, where T represents the transformation that background corrects, normalizes, and logs the PM intensities, ei represents the log2 scale expression value found on arrays i = 1,..., I, aj represents the log scale affinity effects for probes j = 1,..., J, and εij represents error..." The original implementation of RMA uses multiple array information in two ways. The first is through quantile normalization, which uses all available arrays to form an average empirical distribution. Normalization is achieved by forcing all arrays to have this distribution [The refRMA algorithm is a version of the log scale linear additive RMA procedure as described in Irizarry, et al.  and is eribution . The secaj, with vector length equivalent to the number of probes on the array, and 2) a "normalization" vector, also of length equivalent to the number of probes on the array, is compiled based on the intensity, ij)T as input and outputs two data objects which will be persistently archived: 1) a "probe effect" vector is compiled based on the individual log scale probe affinity effects, pport it .The inputs for the application portion of the workflow are the two archived vectors and a single sample run on the same array type. The normalization and probe effect vectors are applied to the test sample, as defined by the model, to calculate probe level intensity values. Note that the application of these two static vectors to the test sample constitutes a one-way function in which the test sample is normalized and summarized by both vectors, but does not modify either one. BioConductor code for the Classic RMA implementation can also be utilized for this portion of the workflow with a modification to utilize the archived vector objects from the training portion instead of the default intermediate objects. Note that there is no longer a group of samples to perform the full median polish summarization. Therefore, a median is taken across the resident probes of each probe set to establish probe set level summaries. The output is a vector of probe set expression values for the single sample. The application process can be repeated over each sample in a multiple array test set such that a single matrix of probe set expression values is derived. An important advantage of this procedure is that it does not require multiple arrays.Note that, although this alternate workflow is defined for the RMA algorithm, a similar workflow can be defined for other multiple array-dependent algorithms.The HGU133A Full refRMA model will be provided for public use as a part of the Bioconductor project . In the SK trained the Full refRMA model and conducted all computational and statistical comparisons involving liver samples. RAI performed the computational and statistical comparison of refRMA models to publicly available data sets. XL performed the affycomp assessment. MT coded the refRMA workflow into R and prepared the code for use by the general public. MWP compiled and drafted the full manuscript and developed the training and testing strategies supporting the manuscript. All authors read and approved the final manuscript.The numbers of samples used to train the Full refRMA model based on organ and pathology category.Click here for file


The most fundamental task using gene expression data in clinical oncology is to classify tissue samples according to their gene expression levels. Compared with traditional pattern classifications, gene expression-based data classification is typically characterized by high dimensionality and small sample size, which make the task quite challenging.In this paper, we present a modified K-nearest-neighbor (KNN) scheme, which is based on learning an adaptive distance metric in the data space, for cancer classification using microarray data. The distance metric, derived from the procedure of a data-dependent kernel optimization, can substantially increase the class separability of the data and, consequently, lead to a significant improvement in the performance of the KNN classifier. Intensive experiments show that the performance of the proposed kernel-based KNN scheme is competitive to those of some sophisticated classifiers such as support vector machines (SVMs) and the uncorrelated linear discriminant analysis (ULDA) in classifying the gene expression data.A novel distance metric is developed and incorporated into the KNN scheme for cancer classification. This metric can substantially increase the class separability of the data in the feature space and, hence, lead to a significant improvement in the performance of the KNN classifier. DNA microarray technology is designed to measure the expression levels of tens of thousands of genes simultaneously. As an important application of this novel technology, the gene expression data are used to determine and predict the state of tissue samples, which has shown to be very helpful in clinical oncology. The most fundamental task using gene expression data in clinical oncology is to classify tissue samples according to their gene expression levels. In combination with pattern classification techniques, gene expression data can provide more reliable means to diagnose and predict various types of cancers than the traditional clinical methods.Compared with traditional pattern classifications, gene expression-based data classification is typically characterized by high dimensionality and small sample size, which make the task quite challenging. In the literature, a number of methods have been applied or developed to classify microarray data -6. TheseThe KNN method is a simple, yet useful approach to data classification. The error rate of the KNN has been proven to be asymptotically at most twice that of the Bayessian error rate . Howeveret al. in m × m = QK0Q, where Q is a diagonal matrix whose diagonal elements are q(x1), q(x2),...,q(xm). Let us denote the vector (q(x1), q(x2),..., q(xm))T and T by q and α respectively, we have q = K1α, where K1 is an m × (l + 1) matrixLet the kernel matrices corresponding to α, aiming to increase the class separability of the data in the feature space. A Fisher scalar measuring the class separability of the training data in the feature space is adopted as a criterion for our kernel optimizationWe optimized the data-dependent kernel in Eq.(l). This requires optimizing the combination coefficient vector Sb represents the "between-class scatter matrix", and Sw "within-class scatter matrix".where m1 data belong to one class, and the remaining m2 data belong to the other class (m1 + m2 = m). Then the basic kernel matrix k0 can be partitioned asSuppose that the training data are grouped according to their class labels, i.e., the first m1 × m1, m1 × m2, m2 × m1, and m2 × m2. A close relation between the class separability measure J and the kernel matrices can be established 2 + 2q(x)q(y))k0 = 1, just like the Gaussian function.where we assume that the basic kernel function satisfy: Since the kernel optimization scheme increases the class separability of the data in the feature space, the performances of kernel machines should be improved. However, for the classification of gene expression data, due to the small size of training samples, the kernel optimization, which performs on training data, may cause overfitting, which means the algorithm may work very well on the training data, but worse on the test data. To handle this problem, we adopted a disturbed resampling strategy to increase the sample size of the training data.xi, ζi}  are the training data, we construct a new set of training data {yi,ξi}, whereSuppose that {xr is a sample randomly selected form {xi} with replacement and ε denotes a normal random disturb, that is, ε ~N), we can effectively reduce the possible overfitting and computational instability, which are mainly caused by the lack of enough training samples for the gene expression data.KNN: K-nearest-NeighborSVM: support vector machineDLDA: diagonal linear discriminant analysisULDA: uncorrelated linear discriminant analysisKerNN: kernel optimization-based KNNALL: acute lymphoblastic leukemiaAML: acute myeloid leukemiaMLL: mixed lineage leukemiaCNS: embryonal tumor of central nervous systemHX and XWC conceived the study. HX designed and implemented the algorithms, and drafted the manuscript. XWC coordinated the study, participated in the algorithm design, and helped draft the manuscript.The core source codes of our algorithms are available at


Even though real-time PCR has been broadly applied in biomedical sciences, data processing procedures for the analysis of quantitative real-time PCR are still lacking; specifically in the realm of appropriate statistical treatment. Confidence interval and statistical significance considerations are not explicit in many of the current data analysis approaches. Based on the standard curve method and other useful data analysis methods, we present and compare four statistical approaches and models for the analysis of real-time PCR data.t-test and non-parametric analogous Wilcoxon test. SAS programs were developed for all four models and data output for analysis of a sample set are presented. In addition, a data quality control model was developed and implemented using SAS.In the first approach, a multiple regression analysis model was developed to derive ΔΔCt from estimation of interaction of gene and treatment effects. In the second approach, an ANCOVA  model was proposed, and the ΔΔCt can be derived from analysis of effects of variables. The other two models involve calculation ΔCt followed by a two group Practical statistical solutions with SAS programs were developed for real-time PCR data and a sample dataset was analyzed with the SAS programs. The analysis using the various models and programs yielded similar results. Data quality control and analysis procedures presented here provide statistical elements for the estimation of the relative expression of genes using real-time PCR. Real-time PCR is one of the most sensitive and reliably quantitative methods for gene expression analysis. It has been broadly applied to microarray verification, pathogen quantification, cancer quantification, transgenic copy number determination and drug therapy studies -4. A PCRBoth genomic DNA and reverse transcribed cDNA can be used as templates for real-time PCR. The dynamics of PCR are typically observed through DNA binding dyes like SYBR green or DNA hybridization probes such as molecular beacons (Strategene) or Taqman probes (Applied Biosystems) . The basReal-time PCR data are quantified absolutely and relatively. Absolute quantification employs an internal or external calibration curve to derive the input template copy number. Absolute quantification is important in case that the exact transcript copy number needs to be determined, however, relative quantification is sufficient for most physiological and pathological studies. Relative quantification relies on the comparison between expression of a target gene versus a reference gene and the expression of same gene in target sample versus reference samples .target) to the power of target ΔCt (ΔCttarget) and reference gene efficiency (Ereference) to the power of reference ΔCt (ΔCtreference). The ΔΔCt model can be derived from the efficiency-calibrated model, if both target and reference genes reach their highest PCR amplification efficiency. In this circumstance, both target efficiency (Etarget) and control efficiency (Econtrol) equals 2, indicating amplicon doubling during each cycle, then there would be the same expression ratio derived from 2-ΔΔCt [Since relative quantification is the goal for most for real-time PCR experiments, several data analysis procedures have been developed. Two mathematical models are very widely applied: the efficiency calibrated model ,8 and thm 2-ΔΔCt ,9.Whereas ΔCttarget = Ctcontrol - Cttreatment and ΔCtreference = Ctcontrol - CttreatmentRatio = 2Ct -ΔΔ    Equation 2Whereas ΔΔCt = ΔCtreference - ΔCttarget® program [P values and confidence intervals [Even though both the efficiency-calibrated and ΔΔCt models are widely applied in gene expression studies, not many papers have thorough discussions of the statistical considerations in the analysis of the effect of each experimental factor as well as significance testing. One of the few studies that employed substantial statistical analysis used the REST program . The sof program . Howeverntervals -13. We bA priori, we consider the confidence interval and P value of ΔΔCt data to be very important because these directly influence the interpretation of ratio. Without a proper statistical modeling and analysis, the interpretation of real-time PCR data may lead the researcher to false positive conclusions, which is especially potentially troublesome in clinical applications. We hereby developed four statistical methodologies for processing real-time PCR data using a modified ΔΔCt method. The statistical methodologies can be adapted to other mathematical models with modifications. SAS programs implementing the methodologies and data control are presented with real-time PCR practitioners in mind for turnkey data analysis. Standard deviations, confidence levels and P values are presented directly from the SAS output. We also included analysis of the sample data set and SAS programs for the analysis in the online supplementary materials.From the two mathematical models for relative quantification of real-time PCR data, we observe disparities between data quality standards. For efficiency-calibrated method, the author who described this procedure  assumed Xlcon represents the logarithm transformed concentration, β0 represents the intercept of the regression line, and βcon represents the slope of the regression line [Data quality could be examined through a correlation model. Even though examining the correlation between Ct number and concentration can provide an effective quality control, a better approach might be to examine the correlation between Ct and the logarithm (base 2) transformed concentration of template, which should yield a significant simple linear relationship for each gene and sample combination. For example, for a target gene in the control sample, the Ct number should correlate with the logarithm transformed concentration following the simple linear regression model in equation 3. In the equation, ion line . The accCt = β0 + βconXlcon + ε     Equation 3The input data is grouped as shown in Table Several effects need to be taken in to consideration in the ΔΔCt method, namely, the effect of treatment, gene, concentration, and replicates. If we consider these effects as quantitative variables and have the Ct number relating to these multiple effects and their interactions, we can develop a multiple regression model as follows in Equation 4.Ct = β0 + βconXicon + βtreatXitreat + βgeneXigene + βcontreatXiconXitreat + βcongeneXiconXigene + βgenetreatXigeneXitreat + βcongenetreatXiconXitreatXigene + ε     Equation 4β0 is the intercept, βxs are the regression coefficients for the corresponding X (independent) terms, and ε is the error term [genetreat. The four groups in Table genetreat and estimate the ΔΔCt from it. As shown in the ΔΔCt formula in Equation 2, if a ΔΔCt is equal to 0, the ratio will be 1, which indicates no change in gene expression between control and treatment.In this model, Ct is the true dependent, the ror term . The modSAS procedure PROC GLM was used for ΔΔCt estimation in Program2_MR.sas in P = 0.05. If the ΔΔCt is not significantly different from 0, then we conclude the treatment does not have a significant effect on target gene expression; otherwise, the inverse is concluded. If the effect is significant, we are interested in the standard deviation of ΔΔCt value, from which we can derive the ratio of gene expression as discussed later. The SAS output provides the point estimation (-0.6848) and standard error (-0.1185) for the ΔΔCt. PROC GLM or PROC MIXED are interchangeable in this application. If the experiments involve multiple biological replicates, replicate effect can also be considered through modifying the SAS program. Then the estimation will be the combined effect of gene, treatment and replicate.The SAS output gives a very comprehensive analysis of the data. We are interested in two aspects of the analysis. First, we want to test whether the ΔΔCt value is significantly different from 0 at Another way to approach the real-time PCR data analysis is by using an analysis of covariance (ANCOVA). A simplified model can be derived from transforming the data into a grouped data as shown in Table Ct = β0 + βconXicon + βgroupXigroup + βgroupconXigroupXicon + ε.     Equation 5We are interested in two questions here. First, are the covariance adjusted averages among the four groups equal? Second, what is the Ct difference of target gene value between treatment and control sample after corrected by reference gene? In this case, the null hypothesis will be (μ2-μ1)-(μ4-μ3) = 0, and the test will yield a parameter estimation of ΔΔCt as shown in the Program3_ANCOVA.sas .P value is presented in SASOutputs.doc , as well as its standard error (0.1185) and 95% confidence interval . The SAS output with both confidence level and puts.doc .t-test, which will yield the estimation of ΔΔCt.More simplified alternatives can be used to analyze real-time data with biological replicates for each experiment. The primary assumption with this approach is that the additive effect of concentration, gene, and replicate can be adjusted by subtracting Ct number of target gene from that of reference gene, which will provide ΔCt as shown in Table t-test, a Wilcoxon two group test can also be used to analyze the two pools of ΔCt values. Two of the assumptions for t-test are that the both groups of ΔCt will have Gaussian distributions and they will have equal variances. However, these assumptions are not valid in many real-time PCR experiments using realistically small sample sizes. Therefore a distribution-free Wilcoxon test will be a more robust and appropriate alternative in this case [As a non-parametric alternative to the his case .t-test and Wilcoxon two group test as shown in the attached program Program4_TW.sas , a ΔCt value of 3 can only be transformed into 5.27 times differences in ratio instead of 8 times. This problem gets amplified when the ΔΔCt or ΔCt values are larger and the amplification efficiency is lower, which could lead to severely skewed interpretations.Many of the current real-time PCR experiments do not include a standard curve design, nor do they use a method to estimate the amplification efficiency. We argue here that real-time PCR data without proper quality controls are not reliable, since the efficiency of real-time PCR could have significant impact on the ratio estimation and dynamic range. For example, if a PCR has a percentage amplification efficiency (PE) of 0.8  template amount should yield a slope not significantly different from -1, which indicates a nearly 2 amplification efficiency. Even though both efficiency-calibrated model and modified ΔΔCt model tolerates the amplification efficiency lower than 2, it is most reliable to have all the reaction with amplification efficiency approximating 2 through optimizing primer choices, amplicon lengths and experimental conditions. From our experience, maintaining all the amplification efficiency near 2 is the best way to reach equal amplification efficiency among the samples and thus to ensure high quality data. It is also observed that a near 2 amplification efficiency can help to expand the dynamic range of ratio estimation.P-value is an important parameter for significance level, and confidence intervals help to establish the reliable range for ΔΔCt estimation. Most of current real-time PCR publications do not present P-values and confidence intervals [P-values is important when the researchers claim differential expression between the samples or treatments exists. In the program we present, all the P-values are derived from testing the null hypothesis that ΔΔCt are equal to 0. Therefore, a small P-value indicates that the ΔΔCt is significantly different from 0, which demonstrates a significant effect. The interpretation of a P-value will depend on the experimental objectives. For example, at P = 0.05 in a treatment versus control experiment, we can claim that the treatment has a significant effect; and in a tissue comparison experiment, we can claim that the gene expression is significantly different among the tissues.The ntervals -13. We b-ΔΔCt and the confidence interval for ratio should be . Since Ct is the observed value from experimental procedures, it should be the subject of statistical analysis. The practice of performing statistical analysis at ratio directly is not appropriate. The presentation of data needs to refer to the ΔΔCt and subsequently the ratio and confidence intervals derived from 2-ΔΔCt.Some publications present a standard deviation of the ratio as a meaningful metric. However, we argue here that the standard deviation of ratio should be derived from the standard deviation of ΔΔCt; and the confidence interval of the ratio should be derived from the confidence interval of ΔΔCt. In other words, the point estimation of ratio should be 2As stated before, the PCR amplification efficiency can be optimized to be approximately 2 with proper amplification primers, RNA quality, and cDNA synthesis protocol. Recent advancements in real-time PCR primer design have allowed easier experimental optimization ,17. HoweXlcon represents 10 based logarithm transformed concentration, the amplification efficiency (E) is 10-(1/slope) or Xlcon represents the 2 based logarithm transformed concentration, the amplification efficiency (E) therefore is 2-(1/slope) or con).Several approaches have been developed to calculate the amplification efficiency in the low quality data. One of such approach is so called 'dynamic data analysis', in which the fluorescence history of a PCR reaction is employed to calculate the amplification efficiency ,20. The In the first scenario discussed above, all PCR amplification have the same efficiency, but the efficiency is not equal to 1. Then the ratio of gene expression can be represented in the following equation.PE = -(1/βcon), and ΔΔCtadjust = PE*ΔΔCtwhereas con is the pooled slope of the plot with Ct against logarithm 2 based concentration. The βcon can be calculated with a correlation function in SAS as shown in Program5_LowQualityData.sas in 0 is the pooled slope of the plot of Ct against log2 (concentration) for each gene.In the Equation 6, βPEtarget = -(1/βconTarget), PEcontrol = -(1/βconControl), and ΔΔCtadjust = PEtarget*ΔCttarget-PEcontrol*ΔCtcontrolwhereas βconTarget and βconControl are the pooled slope for the plot of Ct against logarithm 2 based concentration for target gene and reference gene respectively. The slopes can be calculated by the Program5_LowQualityData.sas  with logarithm transformed concentration, which in turn indicates the unequal slope among different groups of PCR. If all PCR amplification efficiency are equal, then the pooled amplification efficiency can be calculated and integrate into the SAS program for ΔΔCtadjust calculation. In this set of data, the Type III sums of squares has a p value smaller than 0.05, and the amplification efficiency are not equal for all PCRs. Tests of equal slopes are then performed for each gene to decide whether PCR amplification efficiency is the same for each gene. For either gene, the amplification efficiency is not significantly different with an α of 0.05. All of the Type III sums of squares outputs can be found in SASOutput.doc  by observing the Type III sums of squares for lcon and class interaction. A low tput.doc .con) for each gene to derive the percentage amplification efficiency (PE = -(1/βcon)) for each gene. The pooled slopes are derived based on the correlation between Ct and logarithm 2 based concentrations. The βcons for the two genes are -1.0813 and -1.0137 respectively as shown in SASOutput.doc  or PE can be calculated for each gene as 0.925 and 0.987 respectively. The ΔΔCtadjust can then be computed with PEs substituting the 1 for each gene in the 'estimate' and 'contrast' statement. The SAS program is as follows in The next step is to calculate the pooled slope .The SAS output for the analysis is in SASOutput.doc . The ΔΔCOverall, in the less optimized PCR reactions, statistical analysis is not only complicated but also compromised for precision and efficiency. Therefore caution should be exercised when performing statistical analysis with the low quality real-time PCR data, which may easily introduce error due to the efficiency adjustment ,18.In this report, we presented four models of statistical analysis of real-time PCR data and one procedure for data quality control. SAS programs were developed for all the applications and a sample set of data was analyzed. The analyses with different models and programs yielded the same estimation of ΔΔCt and similar confidence intervals. The data quality control and analysis procedures will help to establish robust systems to study the relative gene expression with real-time PCR.Arabidopsis thaliana (Col1) plants were grown in the growth chamber at 23°C with 14 hours of light for four weeks. Total RNA was isolated with RNeasy Plant Mini Kit  from methyl-jasmonate treated Arabidopsis, alamethecin treated Arabidopsis and control plants, and DNA contamination was removed with an on-column DNase  treatment. One microgram of total RNA was synthesized into first strand cDNA in a 20 μL reaction using iScript cDNA synthesis kit (BioRad Laboratories). cDNA was then diluted into 10 ng/μL, 2 ng/μL, 0.4 ng/μL and 0.08 ng/μL concentration series. Three replicates of real-time PCR experiments were performed for each concentration using an ABI 7000 Sequence Detection System from Applied Biosystems (Applied Biosystems). Ubiquitin was used as the reference gene, and the primer sequences for Arabidopsis ubiquitin gene were CACACTCCACTTGGTCTTGCG (F) and TGGTCTTTCCGGTGAGAGTCTTCA (R). The primers for target gene (MT_7) were designed by Primer Express software (Applied Biosystems) and the sequences were CCGCGGTACAAACCTTAATT (F) and TGGAACTCGATTCCCTCAAT (R). MT-7 gene is the Arabidopsis thaliana gene At3g44860 encoding a protein with high catalytic specificity for farnesoic acid [The sample data set Table  used foroic acid . Primer A main limitation of efficiency calibrated method and ΔΔCt method is that only one set of cDNA samples are employed to determine the amplification efficiency. It was assumed that the same amplification efficiency could be applied to other cDNA samples as long as the primers and amplification conditions are the same. However, amplification efficiency not only depends on the primer characteristics, but also varies among different cDNA samples. Using a standard curve for only one set of tested samples to derive the amplification efficiency might overlook the error introduced by sample differences. In our experimental design, we have performed standard curve experiments with four concentrations of three replicates for all samples and genes involved. The ΔΔCt will derive from the standard curves only, and the data quality is examined for each gene and sample combination. The analysis of two samples is presented in the paper as an example. A minimal of PCRs of two replicates in three concentrations will be required for each sample. Even though more effort is required, the data is more reliable out of stringent data quality control and data analysis based on statistical models.® Excel to open the exported Ct file from an ABI 7000 sequence analysis system and then to transform data into a tab delimited text file for SAS processing. The sample data set is shown in Table The output dataset included Ct number, gene name, sample name, concentration and replicate. We used MicrosoftAll programs were developed with SAS 9.1 (SAS Institute).JSY carried out the real-time PCR experiments, developed the statistical model and SAS programs for analysis, and drafted the article. AR provided assistance in SAS programming and data modeling. FC provided assistance in real-time PCR experiments. CNS provided oversight of the work, conceptualized non-parametric elements, and finalized the draft.The SAS program implements the data quality control processes.Click here for fileThe data provides the input data for Program1_QC.sas and Program3_ANCOVA.sas. The data has grouped the Ct values according to the different combination of sample and gene.Click here for fileThe abbreviated SAS output for all the analyses.Click here for fileThe SAS program implements the multiple regression model and derives ΔΔCt.Click here for fileThe input data for Program2_MR.sas, which has contains sample, gene, concentration and Ct number.Click here for fileThe SAS program implements the ANCOVA model and derives ΔΔCt.Click here for fileThe SAS program performs both student t test and Wilcoxon two group tests on the ΔCt to derive ΔΔCt.Click here for fileThe SAS program is a macro that derives confidence interval for Wilcoxon two group tests.Click here for fileThe input data for Program4_TW.sas. The data contains only sample name and ΔCt.Click here for fileThe SAS program performs test for equal slope, grouped slope and adjusted ΔΔCt.Click here for fileThe input data for Program5_LowQualityData.sas.Click here for file