Ratio-based analysis is the current standard for the analysis of dual-color microarray data. Indeed, this method provides a powerful means to account for potential technical variations such as differences in background signal, spot size and spot concentration. However, current high density dual-color array platforms are of very high quality, and inter-array variance has become much less pronounced. We therefore raised the question whether it is feasible to use an intensity-based analysis rather than ratio-based analysis of dual-color microarray datasets. Furthermore, we compared performance of both ratio- and intensity-based analyses in terms of reproducibility and sensitivity for differential gene expression.By analyzing three distinct and technically replicated datasets with either ratio- or intensity-based models, we determined that, when applied to the same dataset, intensity-based analysis of dual-color gene expression experiments yields 1) more reproducible results, and 2) is more sensitive in the detection of differentially expressed genes. These effects were most pronounced in experiments with large biological variation and complex hybridization designs. Furthermore, a power analysis revealed that for direct two-group comparisons above a certain sample size, ratio-based models have higher power, although the difference with intensity-based models is very small.Intensity-based analysis of dual-color datasets results in more reproducible results and increased sensitivity in the detection of differential gene expression than the analysis of the same dataset with ratio-based analysis. Complex dual-color setups such as interwoven loop designs benefit most from ignoring the array factor. The applicability of our approach to array platforms other than dual-color needs to be further investigated. During the last decade, microarray technology has evolved into an indispensable tool for high-throughput gene expression studies. For example, microarrays are now routinely applied to identify differentially expressed genes between paired sample series, classify tumors in prognostic groups, and identify transcriptional alterations during development -3. Two min vitro characterization of putatively identified genes. Due to their dimensionality , microarray experiments suffer from high rates of false positive and negative findings [The outcome of comparative microarray experiments is a ranked list of significant genes, possibly involved in the process under investigation. The resulting gene list then serves as a starting point for further investigations, such as constructing new hypotheses, or the findings . Thus, tAnalysis of variance (ANOVA) is a widely used tool to analyze and rank genes in both one- and dual-color comparative gene expression experiments . The ANOWe have performed a set of experiments to determine whether the intensity-based analysis of dual-color arrays is more reproducible than the conventional ratio-based analysis. Two independent datasets were used: a human keratinocyte cell line dataset, and a dataset based on human brain tissue. By selecting these datasets, we were able to study the performance of the ratio- and intensity-based models in two distinct situations: no biological variation (cell line dataset) versus substantial biological variation (brain dataset).For both the intensity-based and ratio-based analysis, we estimated the reproducibility and sensitivity in the detection of differential gene expression by analyzing technical replicates. Technical replicates, consisting of two non-overlapping sets of microarrays, were used rather than biological replicates, as our focus is on inclusion or exclusion of the array factor, which is a technical factor. Furthermore, we used a model selection algorithm to determine whether either intensity or ratio based analysis was most suitable for each dataset.Our results indicate that intensity-based analysis outperforms the standard ratio-based analysis of the same dataset. Intensity-based results are more reproducible, and increase the sensitivity of detecting regulated genes. Our results also indicate that differences between ratio- and intensity-based results become smaller in large datasets with simple designs, suggesting that more complex designs such as factorial and loop designs benefit most from our approach.To determine the feasibility of intensity-based analysis of dual-color arrays, we performed a microarray experiment in which the effects of 4 different treatments  were investigated using two keratinocyte-derived cell lines by measuring transcript levels on Agilent 4 × 44K Whole Human Genome arrays. The entire experiment was technically replicated . We selUltimately, the reliable detection of differential gene expression between treatments is the main interest of the cell line experiment. We therefore, for the cell line dataset, compared the ranking of genes by p-values generated with two different ANOVA models: one including the array effect , and one without the array effect . We observed a substantial overlap of 64% (replicate dataset C1) and 66% (replicate dataset C2) between the ratio- and intensity-based lists of the 1,000 most significant genes, suggesting that the intensity-based model yields similar results as the ratio-based model Figure . FurtherThe detection of regulated genes should be reproducible; between two technically replicated experiments, one expects to find highly similar sets of differentially expressed genes. To assess the reproducibility of the ratio and intensity models, we separately calculated p-values for both technical replicate datasets C1 and C2 of the cell line experiment. We observed a strong correlation (r = 0.75) between p-values generated by the ratio model for the two datasets Figure . HoweverThe results from the p-value rank-based reproducibility experiments suggest that it is preferable to exclude the array factor in the linear model. To provide further evidence that the intensity model is indeed superior to the ratio model, we applied the Bayesian information criterion (BIC) for model selection on the ratio- and intensity-based ANOVA models . This teThe cell line experiments demonstrated that intensity-based analysis of dual-color data provide more reproducible results, and is more sensitive in the detection of differentially expressed genes. It is however unknown how these results translate to other types of experiments, consisting of different sample types and experimental setups. We therefore analyzed a separate dataset, consisting of 49 human prefrontal cortex samples, divided over 7 equally sized groups. Samples were obtained from different human subjects, thus the biological variance in this dataset is expected to be large . Consequently, samples were not pooled, but hybridized individually. As each biological sample was hybridized two times  was estimated using ANOVA models with and without the array factor Figure . In all In both the cell line and human brain datasets, the factors treatment and array are partially confounded (since one cannot assign every treatment to each array). In the MAQC dataset, which consists of 10 technical replications of two commercially available RNA samples, these factors are not confounded. A BIC analysis revealed that the model without array effect is preferable for 71% of the genes in the MAQC dataset. Thus, even though dropping the array effect is beneficial for roughly 2 out of every 3 genes, this percentage is lower than in the other two data sets. This is however not unexpected: BIC includes a penalty which is proportional to the sample size (number of arrays) for the model with array effect while constant for the other model, and the sample size of the MAQC dataset is smaller than those of the other two data sets. The simple repeated measurements design of the MAQC data set allowed us to study the trade-off between less degrees of freedom and variance reduction caused by inclusion of the array effect from a power perspective. Figure Our results demonstrate that the analysis of dual-color microarray gene expression experiments using intensity-based linear models outperforms the standard ratio-based analysis. Both reproducibility and sensitivity were enhanced in detecting differential gene expression in two independent datasets.By analyzing technically replicated experiments we determined the effect of both models on the reproducibility of gene rankings. Our studies show that for both the cell line and brain datasets the intensity-based analysis provides more reproducible gene rankings than the ratio-based analysis of the same dataset. For the cell line dataset, 78% of the 1,000 most significant genes is reproduced between the two duplicate datasets C1 and C2 when using the intensity analysis, whereas only 73% of the genes is reproduced with the ratio analysis . For the relatively simple MAQC data set BIC selects the model with array effect for 29% of the genes, much more frequently than for both the brain and cell line data sets. The beneficial effect of dropping the array effect from the model seems more pronounced in experiments that employ direct designs to address complex comparisons, such as time series and multifactorial experiments.th most significant gene as a cutoff, almost all of the 1000 genes  are also significant in the intensity-based analysis using the same cutoff. Interestingly, this analysis also reveals that 3335 genes, not selected by the ratio model, are reproducibly more significant than the 1000th gene in the ratio results. This provides additional evidence for the enhanced sensitivity of the intensity model over the ratio model. Due to the poor reproducibility of the ratio-based results in the brain dataset, such calculations were not meaningful for that dataset.Adding to the enhanced reproducibility, intensity-based analysis is more sensitive in the detection of differential gene expression, as derived from more significant p-values. It is important to note that, by selecting the ratio-based p-value of the 1000The observation that ratio-derived p-values can be improved by intensity-based models can be attributed to the inclusion of the array effect in the ratio-based linear model. Pairing of data is a powerful concept for removing subject specific bias. In particular, when the quality of the spot printing procedure is not constant (often the case with in-house spotted arrays), it is essential to account for an array effect in the ANOVA model . But thein silico reconstructed ratios  between p-values generated by the ratio model on the replicate datasets B1 and B2, whereas the intensity-based p-values reproduced quite well r = 0.46). These findings can be attributed to the following. First of all, the overall p-values (both intensity- and ratio-based) are less significant in the human brain experiment than in the cell line experiment, due to the large biological variation between individuals. Second, due to the relatively low level of biological replication, few degrees of freedom are left for estimating the biological effect. Third, the brain experiment was not designed with splitting the data into two technical replicates in mind. While the two data sets are biologically identical, the samples are paired differently on the arrays between the two replicate datasets  derived from a single primary keratinocyte culture. The two cell lines were subjected to four different treatments . After RNA isolation and labeling , equal amounts (1 μg) of Cy3-CTP and Cy5-CTP labeled samples were hybridized to Agilent 4 × 44K Whole Human Genome arrays , according to the manufacturer's instructions. The hybridization set-up on the 4 × 44K array was chosen in such a way that for each cell line, both Cy3- and Cy5-labeled samples for all treatments were hybridized on a single slide (containing 4 arrays). The entire experiment was technically replicated. The hybridization setup can be found in Additional file lgorithm  in LIMMAMicroarray hybridization data were extracted from the Gene Expression Omnibus . This dataset consists of 10 technical replications of a hybridization of Stratagene Universal Human Reference RNA  and Ambion Human Brain Reference RNA  as described in [Fresh-frozen human brain tissue samples were obtained from the Netherlands Brain Bank, Amsterdam (NBB). Written informed consent for a brain autopsy and the use of the material and clinical information for research purposes was obtained by the NBB from the donor or next of kin. Gray matter was isolated from the prefrontal cortex of 49 individuals  with increasing levels of AD-related neuropathology, as defined by the Braak staging for neurofibrillary tangles . For eacod function of the SMIDA package (version 0.1) in R. The resulting hybridization setup can be found in Additional files http://www.vumc.nl/braindataset and as supplementary information to this manuscript  with the Agilent Low RNA Input Fluorescent Linear Amplification Kit (Agilent Technologies). The most efficient hybridization scheme was calculated with the Clustering of the intensity channels was performed using complete linkage hierarchical clustering. Only probes for which the average log2-transformed intensity  was above A = 7 were included. As this procedure removes data from all arrays for a particular probe, the sample sizes are the same for each probe in the final dataset. For the cell-line data, p-values for differential expression between treatments were generated as follows. First, the entire data set was split into two biologically identical parts by simply distinguishing the technical replicates A and B  effect, αk(j) is the array effect, and εijk is the error component. Dye effects have not been incorporated, because the design was balanced for dyes and the data were normalized to remove dye-specific bias. Model 2 lacks the factor αk(j) and hence represents the intensity-based model. The treatment effect is the factor of biological interest, to which an F-test was applied to compute p-values. This resulted in four lists of p-values: ratio-based and intensity-based p-values for technical replicate C1, and ratio-based and intensity-based p-values for technical replicate C2.Here, ηj was dropped from the model, and the treatment effect τi now represented the Braak stage. The F-test was performed on the Braak stage factor. Again, two ANOVA models were used: the ratio model which included the array effect αk(j), and the intensity model without array effect. Consequently, four lists of p-values were generated: ratio-based and intensity-based p-values for dataset B1, and ratio-based and intensity-based p-values for dataset B2.A similar approach was taken for the human brain data. Each patient was hybridized twice. The resulting set of arrays was split in such a manner that each patient was represented exactly once in both data sets  might distort the comparison between the models somewhat. Also, since both splits contain an equal numbers of samples, sample size 'bias' is absent.in silico reconstructed ratios of the two intensity measurements of the same sample pair, as measured on separate arrays, and against different samples. For example, in dataset C1, the directly measured ratios between samples T1 and T2 on array 1, were compared with the reconstructed ratios between T1 on array 4, and T2 on array 2 , the proportion of overlap was defined as the fraction of genes, occurring in both sets. Third, Bayesian information criterion (BIC) model selection was used to score the ratio- and intensity-based linear models for each array feature. Information criterion methods, which aim to determine which set of model parameters the data support best, penalize models with more unknown parameters in order to select a model with a lower generalization error and hence more reproducible results [nlme package in R.Both for the cell line datasets C1 and C2, and the human brain datasets B1 and B2, reproducibility between the replicated datasets was determined as follows. First, the correlation between sets of p-values was calculated using Spearman's  results . The preThe authors declare that they have no competing interests.KB generated the human brain dataset, conceived of the study, performed statistical analyses, and drafted the manuscript. RHB and SJS generated the human keratinocyte cell line dataset. BY and JV participated in the design of the study and helped to draft the manuscript. MAW conceived of the study, performed statistical analyses and helped to draft the manuscript. All authors read and approved the final manuscript.Hybdridization setup cell line experiment. DN = P53DN mutant, SH= shRNA, MI = MIR372, E6 = HPV16 E6. The array numbers are given in the top left corner of each array.Click here for fileCorrelation real and in silico reconstructed ratios. Correlation between ratios, as measured on the array, and the in silico reconstructed ratios of the same sample pair, as measured on different arrays and against different co-hybridized samples. Only genes with an average log2-transformed expression > 7 were included.Click here for fileCorrelation real versus virtual hybridizations. Correlation between M-values derived from real hybridizations versus M-values derived from virtual hybridizations for the MAQC dataset. Main title: hybridization number. X-axis: directly measured ratio between sample A and B. Y-axis: mean M-value between 9 virtual comparisons sample A versus sample B, where sample A and B where measured on different arrays. Correlation: Pearson's correlation coefficient.Click here for fileHybridization setup human brain experiment. The column dataset denotes in which replicate dataset (B1 or B2) the intensity measurement was used.Click here for fileReproducibility of between-group treatment effects based on ratio and intensity models. Reproducibility of ANOVA-derived treatment effects between group 2 and group 4, and group 1 and group 5, in replicate brain datasets B1 and B2. Reproducibility is enhanced when using intensity-based models instead of ratio-based models.Click here for fileHybdridization setup brain experiment. BR = Braak stage.Click here for fileHuman keratinocyte cell line dataset. Normalized expression values and raw hybridization data for all hybridizations have been deposited into the Gene Expression Omnibus (GEO) database. Reviewer link GEO dataset: http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?token=djkxjoomeyecqja&acc=GSE12553.Click here for file


The use of gene expression profiling for the classification of human cancer tumors has been widely investigated. Previous studies were successful in distinguishing several tumor types in binary problems. As there are over a hundred types of cancers, and potentially even more subtypes, it is essential to develop multi-category methodologies for molecular classification for any meaningful practical application.A jackknife-based supervised learning method called paired-samples test algorithm (PST), coupled with a binary classification model based on linear regression, was proposed and applied to two well known and challenging datasets consisting of 14 (GCM dataset) and 9 (NC160 dataset) tumor types. The results showed that the proposed method improved the prediction accuracy of the test samples for the GCM dataset, especially when t-statistic was used in the primary feature selection. For the NCI60 dataset, the application of PST improved prediction accuracy when the numbers of used genes were relatively small (100 or 200). These improvements made the binary classification method more robust to the gene selection mechanism and the size of genes to be used. The overall prediction accuracies were competitive in comparison to the most accurate results obtained by several previous studies on the same datasets and with other methods. Furthermore, the relative confidence R(T) provided a unique insight into the sources of the uncertainty shown in the statistical classification and the potential variants within the same tumor type.We proposed a novel bagging method for the classification and uncertainty assessment of multi-category tumor samples using gene expression information. The strengths were demonstrated in the application to two bench datasets. The use of gene expression profiling for the classification of human cancers has been widely investigated. Previous works were successful in predicting tumor types in the context of binary problems. Many algorithms for feature extraction and sample classification have been proposed -6. More Multi-category prediction can be achieved using binary classification algorithms via the one-versus-one (OVO) and/or one-versus-rest (OVR) partition of the training data set. However, in a cancer type prediction, multi-category problems proved to be more challenging than simple binary problems, and the reported results were less than satisfactory ,10. On oIn disease diagnostic, a measurement of confidence or uncertainty reported with the type determination is always desirable . HoweverThe jackknife is a well known, non-parametric method often used for estimating the sampling distribution of a statistic. Given a sample dataset and a desired statistic , the jackknife works by computing the desired statistic with an element (or a group of elements) removed from the equation. The process is repeated for each element in the dataset. The application in cancer classification with gene expression profiling has been reported in the context of binary problems . In thatIn this study, a new learning method called paired-samples test algorithm (PST), which is based on the jackknife method, was used to classify multiple tumor types using gene expression data. The proposed method is designed for solving multi-category problems under an OVR scheme with a very limited training data set, and it is similar to the bootstrap aggregating (bagging) procedure, which proved to be helpful in improving weak classifiers ,14. In oDetermination of the optimum number of genes (features) to be used by the classification algorithm is usually a difficult task that depends on several factors, including the classification algorithms and the complexity of the data set. For the used binary regression algorithm, previous studies have shown that a feature set of one to two hundred top genes is adequate for a simple two category problem ,7. In thThe prediction accuracy of the 54 validation samples, using different gene selection procedures, is summarized in Figure It should be noted that while the largest improvements were seemingly coming from the weaker gene selection mechanisms, the application of PST made the binary regression algorithm more robust in relation to the gene selection methods and the size of the gene set to be used.et al. (2001) obtained their best result with 42 tumors correctly predicted among the 54 test samples, corresponding to an accuracy of 78% [et al. (2003) achieved a prediction accuracy of around 80% [et al (2004) proposed a Maximal Margin Linear Programming (MAMA) procedure that combines linear programming (LP) and SVM [These prediction results are, in general, better than those obtained by several previous studies using the same data set Table . Using ay of 78% . Using aound 80% . Based o and SVM . Using M and SVM . The outet al (2001) and Bagirov et al (2003) [It is possible that the superiority of the proposed method over SVM and other learning algorithms could be related to the difference in gene selection methods used in this study and by Ramaswamy l (2003) ,15. Howel (2003) ,18 demonet al (2001) [et al (2003) [As indicated in Table l (2001) , three wl (2003) , and thret al (2001) [To further validate the results behind the use of the independent 54 test samples, a four-fold cross validation was conducted for the 144 training samples. The results of this validation are presented in Figure l (2001) .The NP-AF(T) (>.75) and nearly 3/4 of the tumors had their prediction confidence < 0.5. For the classification algorithm used in this study, F(T) was defined as the aggregate probability that the test sample T belongs to the assigned tumor type. In this context, considering F(T) alone makes the current prediction results seem unexpected. However, when taking R(T) values into account, confidence measurement, or F(T), appears to be in better agreement with the results of this study. Of those samples with lower prediction confidence, the majority had their R(T) between -0.15 and 0.15, suggesting that their lower prediction confidences were mainly due to the potential weakness of the classifiers and/or some moderate heterogeneity. In addition, the profile of the four metastatic prostate (PR) samples was interesting. Although they were predicted with 100% accuracy, their relative confidences were low. This suggests the metastatic tumors can be distinguished from the primary tumors of the same type by using the proposed relative confidence criterion.Uncertainty of the 45 correctly classified test samples from the best result is graphically presented in Figure There were no independent test  samples in the NCI60 dataset. Consequently, ten-fold cross validation was conducted as in Statnikov et al 2005) . The res . The reIn cancer type predictions, multi-category problems have proven to be more challenging than binary cases, not only in the classification accuracy but also in the assessment of uncertainty. In this paper, a jackknife-like classification method, called paired-samples test algorithm (PST), was proposed and applied to two bench datasets of multiple tumor types ,20. The R(T) of the prediction can be accessed by measuring the difference between the prediction probability of the test sample and the corresponding value of the withheld training sample. R(T) provided insight into the sources of the uncertainty in the statistical classification by revealing the loss in confidence due to the utilization of weak classifiers or heterogeneity in a given tumor type. It is possible to combine the measurement F(T) and R(T) to make a better score for type determination. Our continuous work will consider this possibility in regards to penalizing a negative R(T) value.The core idea of the proposed method is to repeatedly test a certain known tumor type with a blind test sample while withholding an associated training sample; in this way, not only can the prediction be made but also the relative confidence When the distribution of the data is complex and/or the training set is small compared to the feature dimension, the combined decision of an ensemble of multiple classifiers can be used to improve the performance of a single classification rule . The bagA to J, and each type has 8 samples. In establishing an OVR classifier for separating type A from others, the training data will be divided into two groups, one containing the A samples (8 samples) and another containing the remaining 72 samples (B to J). Although the training set is not small in size, it is extremely asymmetric. Theoretically, in a bootstrap replicate of the same size, the probability of a sample being included is A samples in some replicates may be very small, leading to an inaccurate classifier. Furthermore, a valid bagging technique requires a great deal of replicates. Consequently, combining a bagging technique with computationally intensive classification algorithms and gene extraction methods may become impractical due to high computational cost. In order to overcome these shortcomings, we propose a paired-samples testing algorithm, a parsimonious jackknife-like method.Assume the original training set includes 10 tumor types labeled with letters from luded is . Thus, tThe algorithm is based on the concept of jackknifing. For every tumor type, multiple classifiers are established. The size of the classifier ensemble is equal to the number of the training samples of the tumor type in the original data. Each classifier is trained on a data set with one sample of the tumor type withheld. Using the same classifier, the withheld sample is "predicted" together with the test samples . The probabilities that the two samples belong to the same tumor type are computed and a relative value is calculated as the difference of the two probabilities. By aggregating the results from the multiple classifiers, the probability that the test sample belongs to the tumor type and its relative confidence are obtained. Similarly, the same quantities (probability and relative confidence) are calculated for the remaining other tumor types in the test sample. Finally, the maximum confidence rule is used to determine the tumor type for the test sample and to report the confidence and relative confidence for the prediction. Figure A, B and C with na, nb and nc training samples, respectively. T is the test sample for which the tumor type is to be determined, and CLA, CLB and CLC represent the three classifier ensembles for tumor type A, B and C, respectively. p(T → A|CLAi- ) represents the probability that test sample T belongs to tumor class A under classifier CLA-i,, where the subscript i indicates that sample Ai was removed from the training set. All other probabilities are defined in the same way. T belongs to tumor type A, B and C, respectively, and P(T) = λ indicates that sample T has been classified as tumor type λ , and F(T) and R(T) are the prediction confidence and relative confidence, respectively.The mathematical process is illustrated in Figure F(T) to be used for class determination, the relative confidence R(T) was calculated in order to provide insight into the sources of uncertainty in the statistical classification. It was defined as the aggregated difference between the estimates (from the series of sub-classifiers based on the training sets) of the probability that the blind test sample T belongs to the assigned tumor type, and the correspondent values for the training samples that actually belong to the class which are paired with T. As showed in Figure R(T) measures the similarity of a particular test sample that has been assigned to a specific tumor type class relative to samples known to be of the same tumor type. If R(T) does not deviate much from zero, the sample could be considered as an "average" tumor of the assigned type. In this case, even a low F(T) does not represent a severe problem because the uncertainty is mainly due to potential limitations of the used classifier. However, when R(T) deviates substantially from zero in the negative direction, a small to medium magnitude for F(T) could indicate that the test sample is likely to be a member of an unknown subtype or variant that is absent or less represented in the training set. A large positive R(T) will very likely indicate a high heterogeneity between the training samples in the tumor type in which the test sample was classified as indicated in Figure Uncertainty is usually evaluated through its reverse relationship with some confidence measurements ,7,11. HoCLAi - and p(T → A|CLAi- ) as indicated in Figure A binary classification algorithm was nested in PST and was performed to establish the series sub-classifiers and calculate the classification probabilities, such as Y = , and the gene expression matrix, X = , was modeled via a continuous and unobserved latent random variable (liability) λ =  such thatIn this algorithm, the relationship between the binary response, andYi was structured via a probit model [where β was a vector containing model parameters. The link function of the linear predictor it model ,22,23. Twhere Φ (·) was the standard normal cumulative distribution function.(SVD) was performed on the gene expression matrix X [γ equal to the number of samples in the training set. The parameter vector, γ was estimated using Gibbs sampling, and β in equations (2) and (3) was obtained as a by linear transformation of the estimates of the reduced model [Due to the fact that the number of genes was larger than the number of samples, a dimension reduction technique called singular value decomposition matrix X . The resmatrix X .The redued model ,7. The ted model ,7. We aled model , however2 transformed gene expression data.Feature selection steps were required for cancer classification with gene expression profiling. In this study, four gene ranking methods were combined with PST. All the calculations were based on logFC = |Mo - Mr|. In OVR setup, Mo represents the mean of the training samples in a single tumor type to be separated from the others, and Mr represents the mean of the training samples in all other cancer types.It is calculated with the formula It is defined asSp is the pooled standard deviation, No and Nr are the numbers of the training samples in the two groups, respectively, and Mo and Mr are the same as defined above. Gene ranking is based on the absolute values.where Sp by adding a positive quantity, a, to the denominator of the t-statistic, represented by the formula:It penalizes genes with small Sp for all genes in the array [In this study, δ was set to a value equal to the ninetieth percentile of the distribution of the standard deviations of he array .The statistic in this test was defined asRi is the sum of ranks in group i, ni is the number of observations in the ith group, and n is the sample size. There are e distinct values, with v1 equal to the smallest, v2 equal to the next smallest and so on. In OVR setup, the test reduces to the two-sided Mann-Whitney's Test. For gene ranking, only the statistic W is required.where The proposed method was applied to two well-known and challenging datasets that have been analyzed previously by several groups.It consisted of 144 and 54 training and testing samples, respectively, representing 14 tumor types . These tThe 60 cell lines were derived from tumors: 8 breast, 5 central nervous system, 7 colon, 6 leukemia, 8 melanoma, 9 non-small-cell-lung-carcinoma (NSCLC), 6 ovarian, 2 prostate, and 8 renal . Because2 transformation was applied. Further, genes with the highest transformed intensity smaller than two times the minimum expression across all samples of each dataset were deleted.For both datasets, the expression intensities for each gene were calculated using Affymetrix GENECHIP analysis software. In the current study, some preprocessing was conducted on the data provided in literature ,20. It cWZ and RR developed the algorithm. WZ carried out statistical analysis and drafted the manuscript. KR, YW and KB participated in writing. All authors read and approved the final manuscript.


Clinical progression of colorectal cancers (CRC) may occur in parallel with distinctive signaling alterations. We designed multidirectional analyses integrating microarray-based data with biostatistics and bioinformatics to elucidate the signaling and metabolic alterations underlying CRC development in the adenoma-carcinoma sequence.Studies were performed on normal mucosa, adenoma, and carcinoma samples obtained during surgery or colonoscopy. Collections of cryostat sections prepared from the tissue samples were evaluated by a pathologist to control the relative cell type content. The measurements were done using Affymetrix GeneChip HG-U133plus2, and probe set data was generated using two normalization algorithms: MAS5.0 and GCRMA with least-variant set (LVS). The data was evaluated using pair-wise comparisons and data decomposition into singular value decomposition (SVD) modes. The method selected for the functional analysis used the Kolmogorov-Smirnov test. Expressional profiles obtained in 105 samples of whole tissue sections were used to establish oncogenic signaling alterations in progression of CRC, while those representing 40 microdissected specimens were used to select differences in KEGG pathways between epithelium and mucosa. Based on a consensus of the results obtained by two normalization algorithms, and two probe set sorting criteria, we identified 14 and 17 KEGG signaling and metabolic pathways that are significantly altered between normal and tumor samples and between benign and malignant tumors, respectively. Several of them were also selected from the raw microarray data of 2 recently published studies (GSE4183 and GSE8671).Although the proposed strategy is computationally complex and labor–intensive, it may reduce the number of false results. Colorectal cancer (CRC) arises as a multi-step process of successive cellular clone selection. As a result of the growth advantage of dysplastic cells over their normal neighbors, the morphological counterpart of molecular alterations leads to progressive cytological and architectural derangement recognizable as the adenoma-carcinoma sequence An individual cancer phenotype is the result of cell-specific, developmental stage-specific, and metabolism-related changes in gene expression selectively occurring at a time and modified by epigenetic interactions To answer this question, microarray data originating from two experimental procedures were analyzed by multiple methods employed for identification of consensus differences in pathways underlying CRC development through the adenoma-carcinoma sequence. Samples of normal mucosa, adenomas, and carcinomas obtained during surgery or colonoscopy were processed to select differences between: (i) epithelium and mucosa in normal tissue away from and directly adjacent to carcinoma, in adenoma and carcinoma (using microdissected samples) and (ii) normal and neoplastic tissues and adenomas and carcinomas (using whole tissue sections). Finally, we addressed the potential and challenges of translating microarray-based gene expression profiles into the functional aspects of carcinogenesis.A total of 170 GeneChips were hybridized in this study. Twenty-five of 130 arrays representing whole tissue section samples were rejected from the data analyses. Twenty-four of these arrays were rejected on the basis of poor GeneChip quality according to the parameters established by Affymetrix and due to their internal inconsistency with others as established by the principal component analysis (PCA) (not shown). One microarray was rejected because of mechanical damage. A parameter summary of 145 arrays of suitable quality, 40 and 105 represented microdissected and whole tissue section samples, respectively, is provided in From 54,675 probe sets of the Affymetrix HGU133plus2 microarray, 31,962 and 25,410 probe sets for whole tissue sections and 29,242 and 24,002 probe sets for microdissected samples passed the filtering procedure according to MAS5.0 and GCRMA+ LVS algorithms, respectively. As shown in Probe set selection from microarray data representing tissue samples may be significantly affected by differences in the cell type content of the normal and dysplastic mucosa . Bearingvs. MUC dissected from distant full-thickness normal colon (NC), normal colon mucosa dissected from tumor (NT), adenoma (AD), and carcinoma (CA). Pair-wise comparisons were performed separately on microarray data sets normalized by MAS5.0 and GCRMA+LVS algorithms. Probe sets sorted according to significance of differentiation of CEC and MUC in any pair set were used in Kolmogorov-Smirnov (K-S) test to evaluate KEGG pathways alterations. The results of KEGG annotations for each comparison are summarized in Data sets from these microdissected samples were analyzed in the following pairs: CEC To establish oncogenic signaling distinguishing between normal colon and neoplasms, data sets representing whole tissue sections of normal colon, adenomas, and adenocarcinomas and normalized by both MAS5.0 and GCRMA+LVS algorithms were decomposed into major statistically independent variability modes (supergenes) using singular value decomposition (SVD). We assumed that SVD may establish some co-variations in gene expression that could enable better definition of expression-based functional alterations p-value was less than 0.01 in at least one data set normalized by either the MAS5.0 or GCRMA+LVS algorithm, or if it was less than 0.05 in both data sets. The results of selected KEGG annotations are shown in The probe sets selected by the first mode of SVD and those differentially expressed between normal and neoplastic tissues were used for the calculation of their attribution to pre-defined KEGG signaling pathways using the KS test, as summarized in Next, to estimate molecular alterations underlying tumor progression to malignancy, data sets representing whole tissue sections from colonic neoplasms were decomposed into SVD modes  and analPoor reproduction of microarrays by quantitative RT-PCR was observed in this (not shown) and previously in independent studies We conducted normalization, summation and filtration of the raw datasets provided by four recently published microarray-based studies Next, to test the potential of the proposed analytical procedure in identification of consensus signaling alterations, we employed the microarray data provided by Galamb et al. (GSE4183) KEGG signaling pathways related to cell cycle, DNA replication, purine metabolism, pyrimidine metabolism, RNA polymerase, proteasome, aminoacyl-tRNA biosynthesis, mismatch repair and nucleotide excision repair were selected in both our and two recently published microarray-based studies.Axon guidance, discriminating between adenomas and adenocarcinomas  throughout CRC progression that were found in all three sample sets  are presented in Gene expression signature that can be employed for diagnostic purposes should consist of probe sets with signal progressively increased or decreased over the normal tissue-adenoma-carcinoma sequence. To select CRC progression markers, the procedure was consistently applied to analyze data sets obtained by two tissue handling procedures. Probe sets with gradually increasing or decreasing expression  and metabolites representing short-term storage, selectively provided at the time and grouped into signaling and metabolic pathways, cancer complexity at the gene level is likely reduced to a limited number of altered pathways. Thus, the clinical progression of colon tumor phenotypes may occur in parallel to distinctive signaling alterations Several microarray-based studies in CRC have been performed We aimed to identify the essential oncogenic signaling in CRC. Because there is no method that allows the direct estimation of changes in cellular signaling on the genomic scale, we applied an integrative genomics approach that may connect gene expression profiles with molecular pathway alterations. The assessments were directed to minimize the number of prior assumptions and arbitrary choices made during data processing, particularly in the two steps known to have the most severe impact on the results of the analysis: probe-level processing and selection of differentially expressed genes.Several probe-level processing methods have been proposed, including MAS5.0 The reliable evaluation of the performance of low-level processing methods is not an easy task because it requires prior knowledge of the data properties, especially which genes are truly differentially expressed. Consequently, most of the published comparisons of low-level processing algorithms rely on an RNA spike-in or dilution datasets To avoid the arbitrary choices in expression summary generation and data normalization, all presented analyses have been carried out using two algorithms: MAS5.0 and GCRMA combined with LVS normalization As expected, the use of two different algorithms resulted in data sets with considerably different distributions of normalized relative expression values. Therefore, to maximize reliability, both data sets were used in further analyses. In addition, we filtered out low-expression probe sets that, due to unfavorable signal-to-noise ratio, were likely to be a source of false positives.Another factor strongly affecting the final results of microarray data analysis is the method used to define differentially expressed genes. A multitude of strategies have been proposed for this task, ranging from simple methods using only the fold change criterion to more sophisticated approaches using permutation-based statistical testing In order to overcome the mentioned limitations of typical feature selection procedures, we used an approach that combines supervised statistical testing with an unsupervised method, which does not require prior knowledge of sample attribution.p-values of a statistical test in a pair-wise comparison (supervised method) or the contribution to a selected SVD component (unsupervised method). Such methodology requires only one arbitrary cut-off: the adjusted p-value designating the significance of a given gene set.The method selected for the functional analysis used the K-S test. In contrast to common over-representation methods, the test does not require arbitrary cut-off criterion for studied probe sets. The significance of gene sets is determined based on their position on a sorted list of all genes present in the assay. Here, sorting was based on either the vs. ∼30,000). Therefore, the chance of a false positive result is significantly less when the KEGG database is used and less strict correction for multiple hypothesis testing is required.The gene set dictionary selected for analysis was KEGG. Each set consists of the genes involved in a physiological process. Most of the KEGG subsets contain genes that interact with each other; thus, the structure of the database, in contrast to Gene Ontology (GO), is “flat”. No relationship between the specified pathways simplifies the correction for multiple hypothesis testing. The number of categories in the KEGG database is a factor of 100 less than GO  pathways, aminoacids and lipids metabolism may be considered the predominant alterations, appearing on different levels of molecular interaction and reaction networks of oncogenic signaling in the colon.Using microdissected tissues, a considerable set of pathways have been selected which differentiate between epithelia and mucosa, regardless of the disease stage. Of these pathways, a wide array of biological processes, including antigen processing and presentation, immune response, and adaptive inflammatory host defenses, cell migration, cell-cell and cell-matrix adhesion, clearly differentiated functions attributed to cells forming the epithelial layers from those of the resident immune cells infiltrating the lamina propria. Several KEGG pathways distinguishing between microdissected colonic epithelial cells and mucosa were also selected from comparisons between whole tissue sections of adenomas and adenocarcinomas .Results of NC - AD/CA comparisons in our dataset are, in general, in good concordance with such comparisons performed on datasets provided by Galamb et al. (GSE4183) There are over 80 KEGG pathways found significant when all comparisons from publicly available data are taken into account. This, and the fact that there are no "standard" protocols for functional analysis of microarray experiments could lead to tuning the analysis procedure to fit the expected results. We regard that such pitfall could be avoided by selecting the intersection of results acquired with multiple protocols as proposed in this study.To date, a variety of methods have been used to dissect the tissue of interest, as well as RNA extraction and amplification protocols and algorithms for data normalization and significant features selection. Thus, any variation in the procedure introduces difficulties in the direct comparison of the results. Consistent with this idea, probe sets with the same signal level alterations, regardless of the sample handling protocol applied, are of particular interest. Genes found to be differentially expressed between normal colon, adenoma, and carcinoma can be used as markers of the progression process. If the differences are progressive, the interpretation of the results is straightforward; the higher the difference detected, the more advanced the carcinogenesis.We found 17 probe sets with both attributes: significant (FDR<0.01) progressive signal level changes in the NC → AD → CA sequence for microdissection and whole tissue section collected samples, and for MAS5.0 and GCRMA+LVS normalization  and 7. EIn summary, microarray-based gene expression profiles were applied to describe gene regulatory networks appearing on different levels of molecular interaction and reaction networks forming oncogenic signaling in the colon. Although some of the KEGG pathways selected in our analyses may result from differences in the proportion of epithelial and stromal cells excised from adenomas and carcinomas, these studies highlighted significant differences in the molecular makeup of adenomas and adenocarcinomas related to oncogenic signaling. However, though the changes in patterns of individual probe sets annotated to defined KEGG signaling pathways intuitively fit predictions, the microarray data could be translated into the functional aspects of carcinogenesis only as indirect annotations.The lack of independent methods of verification of functional annotations to expression profiles makes the final conclusions from microarray readouts prone to the subjective selection of biostatistical tools. We proposed to tackle this problem by using a wide range of computational methods. Although this strategy is computationally complex and labor–intensive, it reduces the fraction of false results. Herein, we provided an example of such a multidirectional algorithm directed for maximizing the reliability of microarray data results. On the other hand, with such strict conditions, only highly reliable biological processes are selected. Although many pathways may be missed this way, the low concordance of published results justifies the applied conditions.The main disadvantage of the proposed functional analysis relates to the assumption that the number of affected genes within a single pathway determines the degree of its alteration. This assumption may be correct for most metabolic pathways that rely on the mass-action law Patients were prospectively selected for the study between January and December 2006 at the Department of Gastroenterology and Hepatology and the Department of Colorectal Cancer, Cancer Center-Institute, Warsaw. The study protocol was approved by the Cancer Center Bioethics Committee, and all patients signed informed consent before inclusion.Sporadic colonic carcinomas were obtained by surgical resection through laparotomy, and one to four tumor fragments, depending on the tumor's size, and two fragments of paired full-thickness normal colon were cut. Colonic adenomas were obtained during colonoscopic polypectomy, and one to three adenoma fragments, depending on the polyp's size (0.8–3.0 cm), were cut from the tip of each polyp immediately after removal. The amount of removed polyp did not interfere with the histological diagnosis. All carcinoma and polyp tissue specimens were collected by pathologists using the same procedure. In addition, three biopsies were taken from the normal colonic mucosa of 7 healthy subjects who underwent screening colonoscopy using large biopsy forceps. All tissue specimens were snap frozen in liquid nitrogen within 10–30 min of harvesting and stored at −72°C until use.The clinical characteristics of patients and histopathology of analyzed tissue samples are presented in Frozen tissue specimens were cut as a series of 6-µm thick cryosections and mounted on a polyethylene naphthalate (PEN) membrane slide and dehydrated for 2 min in 70% ethanol and 5 min in 100% ethanol. Subsequently, tissue sections were stained in 5% (w/v) alcoholic solution of cresyl violet, rinsed in 100% ethanol, and the slides air-dried for 15 min. The interested areas were independently isolated from the slides using the PALM laser microdissection and pressure catapulting (LMPC) system . Microdissected samples from different parts of the tumor and normal colon were pooled in separate microtubes, immediately lysed with 100 ml RTL buffer  containing 1% β-mercaptoethanol, and stored at −72°C.2 of total captured area (range 2.1–3.5 mm2), whereas captured mucosa that represented a normal or neoplastic epithelial layer with an absent lamina propria of the muscularis mucosae yielded an average 13 mm2 of total area (range 10–15 mm2). The relative cell type content within normal and dysplastic mucosa estimated per 1 mm2 is presented in To obtain suitable reproducibility and reliability of the estimations, tissue samples were dissected by LCM of five replicates for each type of epithelial cell and mucosa from one colon tumor and the paired full-thickness normal colon. Independent microdissections of epithelial cell layers yielded an average of 2.8 mmSeveral series of cryostat sections were prepared from different parts of each specimen using a Microm HM 505E . Upper and lower sections from each cryosection collection were evaluated by the pathologist to control the relative cell type content. RNA was isolated from those cryostat sections representing a given tissue specimen which contained the highest percentage of epithelial cells.Histological evaluation of the examined tissues revealed a median relative content of 60% (range 18–98%) normal mucosa in surgically obtained normal colon specimens and 90% (28–99%) and 55% (15–98%) dysplastic mucosa in specimens representing benign and malignant colon tumors, respectively . EndoscoTotal RNA was isolated from whole tissue sections and microdissected tissue samples using the RNeasy Plus Mini Kit and QIAshredder columns and the RNeasy Plus Micro Kit , respectively. RNA samples were checked for quality on the Agilent 2100 Bioanalyzer. Each sample used for further microarray analysis presented distinct peaks corresponding to intact 28S and 18S ribosomal RNA.Five micrograms of total RNA isolated from each whole tissue section sample and 10–50 ng of total RNA isolated from microdissected samples were used as starting material for the synthesis of biotin-labeled cRNA with one and two rounds of amplification using One-cycle and Two-cycle Target Labeling and Control Reagents (Affymetrix), respectively, the latter with the MEGAscript High Yield Transcription Kit . The biotin-labeled cRNA was purified using RNeasy spin columns, fragmented, and hybridized on Affymetrix oligonucleotide microarrays (GeneChip HG-U133plus2).To measure gene expression, probe set data (cell intensity files) were generated using two standard normalization algorithms: Affymetrix Microarray Suite v.5 (MAS5.0) and GCRMA with least-variant set (LVS) probe sets. The calculations were performed using R/BioConductor (version 2.8.1) packages affy (version 1.20.2), gcrma (version 2.14.1), and FLUSH.LVS.bundle . For data filtration, we selected the probe sets with signal intensity above the threshold limit in at least 5% of samples. The threshold was established at the 98th percentile of the expression levels from Y-chromosome–linked probe set signals detectable in female samples. In addition, the probe sets with signal FC higher than 1.5 (in relation to median) in less than 6 samples were removed from whole tissue sections dataset.p-values were adjusted for multiple hypothesis testing using the Benjamini-Hochberg procedure p-values of the t-test or the contribution of a selected SVD mode were assigned to Kyoto Encyclopedia of Genes and Genomes (KEGG) Pathways.To establish differences in gene expression between tissue groups, both gene-by-gene statistical testing using the permutation test with t statistics and singular value decomposition (SVD) of the data matrix were employed. Because experimental variability is common among microarray data sets, the p-values were corrected for testing multiple hypotheses. We considered the alterations in KEGG pathways as most significant if the adjusted KS test p-values derived from both orderings (by the pair wise test statistic and SVD) were less than 0.01 in at least one data set normalized by either the MAS5.0 or GCRMA+LVS algorithm, or if the adjusted p-value was less than 0.05 in both data sets.The one-sided Kolmogorov-Smirnoff test (KS test) was used to calculate whether probe sets attributed to a particular signaling pathway located closer to the top of the lists than expected by chance. Next, the resulting Statistical and functional analyses were performed using proprietary software working in MATLAB  and R/Bioconductor environments.The same workflow was applied for external datasets: GSE4183 and GSE8671. The threshold for data filtration was established on the same level as for our “whole tissue section” dataset. Probe sets with signal level higher that threshold in less than 5 and 6 samples  were filtered out.Supplementary data and all MIAME compliment microarray data are available at Gene Expression Omnibus (GSE20916).http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?token=bhervmwuqemiyra&acc=GSE20916Figure S1Distribution of probe set signals.  Histograms of signals extracted from microdissected samples using MAS5.0 and GCRMA+LVS, respectively.  Histograms of signals extracted from macrodissected samples using MAS5.0 and GCRMA+LVS, respectively.(0.18 MB PDF)Click here for additional data file.Figure S2Diagrams of the first SVD modes representing macrodissected adenomas (red) and carcinomas (blue); data normalized by MAS5.0 (left panels) and GCRMA+LVS (right).(0.05 MB PDF)Click here for additional data file.Figure S3Venn diagram presenting numbers of probe sets differentiating normal colon (NC) mucosa and adenoma (AD) (left panel) or normal colon and colorectal cancer (CRC) (right panel) in a given studies. Data was normalized with GCRMA+LVS. Difference was considered significant if FDR in permutation test was less than 0.05.(0.06 MB PDF)Click here for additional data file.Table S1Summary of the quality parameters of individual arrays. Analyzed parameters were established by Affymetrix for GeneChip hybridization with cRNA synthesized by one- or two-cycle amplification procedures.(0.28 MB DOC)Click here for additional data file.Table S2Significant KEGG terms selected by K-S test according to lists of probe sets sorted by p-value in pair-wise comparisons of pure colonic crypt epithelial cells (CEC) and mucosa (MUC) dissected from normal colon (NC), normal mucosa adjusted to neoplastic tissue (NT), adenoma (AD) and carcinoma (CA).(0.06 MB DOC)Click here for additional data file.Table S3Lists of the probe sets sorted either by contribution to a selected SVD component or p-value in pair-wise comparisons for the whole tissue section samples. Provided as zipped excel file.(3.46 MB ZIP)Click here for additional data file.Table S4The summary of the significance of differential representation of KEGG terms according to lists sorted by contribution to a selected SVD component or p-value in pair-wise comparisons of microarray data of whole tissue section samples.(0.07 MB DOC)Click here for additional data file.Table S5Probe sets sorted according to the p-value in pair-wise comparisons between benign and malignant whole tissue sections of colonic neoplasms.(3.59 MB XLS)Click here for additional data file.Table S6Summary of the significance of the differential representation of KEGG pathways selected by the K-S test from probe sets lists sorted either by contribution to a selected SVD component or p-value in pair-wise comparison of whole tissue sections of adenoma (AD) and carcinoma (CA) samples.(0.07 MB DOC)Click here for additional data file.Table S7Significant KEGG pathways for comparing normal colon samples with colonic neoplasms and adenomas with adenocarcinomas in GSE4183 and GSE8671 datasets.(0.03 MB XLS)Click here for additional data file.Table S8Patient clinical characteristics and histopathology of analyzed tissue samples.(0.18 MB DOC)Click here for additional data file.


Several preprocessing algorithms for Affymetrix gene expression microarrays have been developed, and their performance on spike-in data sets has been evaluated previously. However, a comprehensive comparison of preprocessing algorithms on samples taken under research conditions has not been performed.We used TaqMan RT-PCR arrays as a reference to evaluate the accuracy of expression values from Affymetrix microarrays in two experimental data sets: one comprising 84 genes in 36 colon biopsies, and the other comprising 75 genes in 29 cancer cell lines. We evaluated consistency using the Pearson correlation between measurements obtained on the two platforms. Also, we introduce the log-ratio discrepancy as a more relevant measure of discordance between gene expression platforms. Of nine preprocessing algorithms tested, PLIER+16 produced expression values that were most consistent with RT-PCR measurements, although the difference in performance between most of the algorithms was not statistically significant.Our results support the choice of PLIER+16 for the preprocessing of clinical Affymetrix microarray data. However, other algorithms performed similarly and are probably also good choices. Preprocessing of raw probe intensities is an essential procedure in the analysis of gene expression microarray data. Generally, background correction and normalization are used to reduce the impact of variations in experimental conditions. Arrays featuring multiple probes per gene target (such as those from Affymetrix) require an additional probe-level summarization step to integrate intensities of the multiple probes into a single expression estimate. These steps involving background correction, normalization, and summarization are often combined into a single all-in-one preprocessing algorithm that takes raw probe intensities as input and produces gene expression estimates as output. Several such algorithms have been proposed, each based on a different model or set of underlying assumptions. Because the choice of algorithm can affect the conclusions drawn from the data, it is important to compare the accuracy of these algorithms.evaluative data sets). For example, the Latin square spike-in data sets from Affymetrix are derived from composite RNA samples, in which a small number of exogenous RNA species are added at various concentrations to a fixed background. Here, the known concentrations of the exogenous genes can be used as a reference to evaluate the accuracy of the microarray expression values The performance of preprocessing algorithms has been characterized in several ways, often with the help of data sets created expressly for this purpose  Therefore, it is also important to evaluate the performance of preprocessing algorithms on microarray data created to address biological or clinical research questions  and on 96-well RT-PCR arrays (TaqMan). The first data set comprises 36 colon biopsies To assess whether these two data sets were representative of typical clinical data sets, we utilized three “bias metrics” as indicators of technical variation Several preprocessing algorithms for Affymetrix-type oligonucleotide arrays have been proposed, but we evaluated only algorithms that were published or referenced in peer-reviewed journals and that were implemented in the R statistical environment. We identified the following nine algorithms meeting these criteria: DFW To a first approximation, the measured probe intensity is proportional to its target transcript concentration, but is also affected by other factors such as probe sequence. Therefore, the absolute expression of a single gene in a single sample is not directly comparable between different gene expression platforms. However, the relative expression of a gene between two samples should be directly comparable, as long as the normalization factor is comparable. Microarray data is often normalized globally, meaning that the distribution of expression levels of all genes is assumed to be constant in all samples. On the other hand, RT-PCR data is often normalized using a small number  of housekeeping genes that are assumed to be constant. For our study, to make the normalization of the microarray and RT-PCR data as similar as possible, we normalized each sample according to the median expression value of the common genes. This additional normalization step increased concordance in both data sets (data not shown).2-transformed expression values as measured by microarray and those measured by RT-PCR. We then compared the distributions of PCCs obtained using the various preprocessing algorithms  between the loggorithms . Using tgorithms .However, the use of PCC as measurement of concordance can be problematic for two important reasons. First, a gene that is essentially unchanging often has low PCC due to random noise, even if both platforms agree that the gene is unchanging. Second, a gene can have high PCC even if the two platforms are highly discordant in magnitude; for example, when one platform experiences compression due to chemical saturation at higher intensities.g we define the log-ratio discrepancy (LRD) as:N is the number of samples, and xgs and ygs are the expression values of gene g in sample s, as measured on each platform. In simpler terms, the LRD of a gene is the absolute difference in log-ratio as measured between platforms, averaged over all possible pairs of samples. Therefore, the LRD has a minimum possible value of zero (perfect agreement) and a maximum value limited only by the dynamic range of the two platforms. An R script implementing the LRD is available in We sought to define a second metric for cross-platform comparison that more intuitively indicates the agreement or disagreement between sets of measurements. In practice, we are most interested in the extent to which a gene measured on the two platforms has the same log ratio for a given pair of samples. Thus, for each gene We calculated the LRD between the microarray and RT-PCR expression values for each gene and compared the LRD distribution between the nine preprocessing algorithms . Using tSeveral preprocessing algorithms have been proposed for Affymetrix arrays, but so far it is unknown whether one particular algorithm provides more accurate results. Several important studies attempting to answer this question using artificially produced RNA samples have been a key source of guidance for investigators RT-PCR is often used to “confirm” microarray results because of its relatively accurate measurements over a wide dynamic range. However, the interpretation of these results can be problematic because RT-PCR expression values can vary based on choice of normalization controls Although the accuracy of preprocessing algorithms for Affymetrix microarrays could be compared in numerous ways, we have used two performance metrics. First, we used the PCC because it is intuitive and because it gives a useful measure of concordance under many conditions. Additionally, we introduced the log-ratio discrepancy, which we believe is a more useful assessment of expression value accuracy in the context of our biomarker research due to its ability to take into consideration the impact of genes that do not change their expression across the various samples in the cohort. Researchers using microarrays for other types of experiments may find other performance metrics to be more relevant to their research, and our results should be considered in this context. For example, the LRD penalizes compression-type artifacts; whereas PCC does not. Such artifacts may be acceptable for simple analysis such as searching for differentially expressed genes between two relatively homogeneous groups. However, such artifacts are not acceptable when the response must be linear, e.g. in principal components analysis. An open question is whether the optimal choice of preprocessing algorithm is dependent on the type of data, or on the biological question being asked. For example, some analysis, such as regulatory network reconstruction, may be highly sensitive to random correlations, in which case concordance with RT-PCR measurements may not be the only consideration for selecting an algorithm Naturally, our choice of data sets from colon biopsies and cancer cell lines reflects our research interests. These data sets, like many recent large-scale data sets, were generated using the HG-U133A or HG-U133 Plus 2.0 microarrays, which use a 3′-biased amplification protocol and a transcript-based probe set design. In contrast, the newer generation of microarrays from Affymetrix utilizes a whole-transcript amplification protocol and an exon-based probe design, which may offer a more specific portrait of expressed sequences et al. described a series of tests to evaluate preprocessing algorithms using the spike-in data set from Affymetrix and the dilution data set from Genelogic. They generally observed better performance from RMA and MAS5 than from dChip (MBEI), which is consistent with our findings. Choe et al. generated a spike-in data set with a defined background, in contrast to the Affymetrix spike-in data sets, which use HeLa RNA as background et al. compared Affymetrix arrays with RT-PCR measurements of 48 genes in various human samples using PCC as the performance metric et al. compared microarrays to RT-PCR using mouse heart tissue and Pearson correlation of fold change as a performance metric, and found that MAS5, dChip-with-mismatch, and GCRMA outperformed dChip-without-mismatch , RMA, and VSN et al. used variability in redundant measurements to estimate noise, and found that RMA outperformed dChip and MAS5 without decreasing the number of differentially expressed genes Several other studies have compared preprocessing algorithms using different data sets and/or different metrics. Cope It is our impression that the most commonly used algorithms are MAS5, MBEI, and RMA Overall, if we had to choose a single preprocessing algorithm, we would choose PLIER+16. However, several other algorithms performed in a comparable manner. Ultimately, it is likely that any of the top algorithms would be suitable for most purposes.36 colon biopsies were taken during routine endoscopical intervention before treatment at the 2nd Department of Internal Medicine at the Semmelweis University Budapest and have been described previously At the beginning of sample collection the written informed consent of patients was obtained for the use of their samples for diagnostic and scientific purposes. At the beginning of data processing all samples received a unique three digit number, and during further analyses this number was used without referring to the patient's name. The ethical committee of the Semmelweis University approved the study; the study conformed to the principles outlined in the Declaration of Helsinki.Samples were profiled using HG-U133A or HG-U133 Plus 2.0 microarrays , and RT-PCR measurements were made on TaqMan 96-well arrays  as described We identified the targets (gene symbol and Refseq accession numbers) of the RT-PCR probes using online annotation provided by the manufacturer . We identified the targets of the microarray probe sets using annotation files downloaded from the manufacturer's website . To find comparable sets of measurements made on both platforms, we identified RT-PCR probes and microarray probe sets that queried not only the same gene, but also the same set of transcript isoforms as determined by Refseq accession numbers. For several genes, multiple microarray probe sets matched an RT-PCR probe; in this case we chose the probe set with the most individual probes matching the queried gene, and if this criterion was not sufficient to choose a single probe set, we arbitrarily chose the first probe set from those matching the criteria. By this procedure we identified 84 genes in the colon cancer data set and 75 genes in the cell line data set measured by both platforms . Only thwww.R-project.org) with Bioconductor packages (www.bioconductor.org). All calculations and comparisons were performed using log2-transformed expression values; all preprocessing algorithms except MAS5 and MBEI return such values by default. RT-PCR log2 expression values are given as the negative of the CT (threshold cycle). All results can be reproduced using the scripts provided in All analysis was performed using the R statistical environment Click here for additional data file.Dataset S2A tab-delimited text file containing raw RT-PCR CT values for the colon data set.(0.03 MB TXT)Click here for additional data file.Table S1An Excel spreadsheet listing the comparable RT-PCR probes and microarray probe sets used in this study.(0.03 MB XLS)Click here for additional data file.Text S1A PDF document providing R code to reproduce the analysis described in the manuscript.(0.32 MB PDF)Click here for additional data file.


Estrogen receptor positive (ER+) breast cancers (BC) are heterogeneous with regard to their clinical behavior and response to therapies. The ER is currently the best predictor of response to the anti-estrogen agent tamoxifen, yet up to 30–40% of ER+BC will relapse despite tamoxifen treatment. New prognostic biomarkers and further biological understanding of tamoxifen resistance are required. We used gene expression profiling to develop an outcome-based predictor using a training set of 255 ER+ BC samples from women treated with adjuvant tamoxifen monotherapy. We used clusters of highly correlated genes to develop our predictor to facilitate both signature stability and biological interpretation. Independent validation was performed using 362 tamoxifen-treated ER+ BC samples obtained from multiple institutions and treated with tamoxifen only in the adjuvant and metastatic settings.We developed a gene classifier consisting of 181 genes belonging to 13 biological clusters. In the independent set of adjuvantly-treated samples, it was able to define two distinct prognostic groups . Six of the 13 gene clusters represented pathways involved in cell cycle and proliferation. In 112 metastatic breast cancer patients treated with tamoxifen, one of the classifier components suggesting a cellular inflammatory mechanism was significantly predictive of response.We have developed a gene classifier that can predict clinical outcome in tamoxifen-treated ER+ BC patients. Whilst our study emphasizes the important role of proliferation genes in prognosis, our approach proposes other genes and pathways that may elucidate further mechanisms that influence clinical outcome and prediction of response to tamoxifen. Breast cancers are biologically heterogeneous with regards to their clinical behavior and response to therapies. However, treatment-decision making for women diagnosed with breast cancer is still reliant on classical histopathological appearance and immunohistochemical markers that give little insight into tumor biology and potential response to treatment. There are a few biomarkers routinely used that can predict response to commonly prescribed therapies. The presence of estrogen receptors is the best indicator of response to anti-estrogen agents such as tamoxifen. However, 30–40% of women with estrogen receptor-positive breast cancer (ER+BC) will develop distant metastases and die despite tamoxifen treatment. The underlying biological mechanisms of resistance to tamoxifen are incompletely understood.Gene expression profiling of tumors appears to be a promising new strategy for predicting clinical outcome in breast cancer patients. Recent studies have proposed that the heterogeneity of clinical response can be correlated with different molecular "portraits" ,2. Gene We have recently shown that in ER+ BC, its proliferative status is the most important predictor of prognosis in these women : highly The dataset used for training the classifier consisted of 255 early-stage  BC samples, diagnosed between 1980 and 1995, all of whom had received tamoxifen only as their adjuvant treatment (hereby referred to as the "tamoxifen-treated dataset"). The demographics can be found in [Additional file Figure pclust).We used a clustering method in order to identify clusters of highly correlated genes, prior to feature selection and model building as we hypothesized that this would to reduce the number of variables, increase signature stability, allow platform independency and to preserve biological interpretation . Prelimii asAlthough the preliminary clustering significantly reduced the dimensionality of the data, the number of features remained too large to efficiently build the classifier. The selection of the most relevant pclusts was performed using a ranking based on the likelihood ratio statistic of univariate Cox model. The Cox model specifies the hazard of a patient λ0(t) is the baseline hazard assumed to be equal for all patients , and pclustj is the jth pclust of patient i. The likelihood ratio statistic is twice the difference in the log partial likelihood between the null model  and the model with estimate of β. The only parameter of this feature selection is the signature size, i.e. the number k of pclusts that will be used to build the classification model.where k, let P be the list of all pclusts. Let freq(p) be the number of sampling steps in which the probe cluster p ∈ P has been selected out of a total of m sampling steps. The set P is sorted by frequency into the set p(1), p(2),..., pn) (such that freq(pi) ≥ freq(pj) if i <j where i, j ∈ {1, 2,..., n}. The stability statistic for a signature size k is defined asSignature size was set in order to maximize a stability criterion using a multiple 10-fold cross-validation algorithm (M10FOLDCV) on the tamoxifen-treated dataset after randomization of the order of patients in the data set. For each signature size, a criterion designed to estimate the stability of a signature was computed, as recently introduced by Davis and colleagues . For a gStab statistic is equal to 1 if the same signature is always selected over M10FOLDCV given a signature size and Stab statistic converges to 1 as the signature size converges to the total number of variables. Therefore, k was chosen as a trade-off between signature size and stability, i.e. a signature size exhibiting maximal possible stability and being smaller than the total number of variables.The β pclustj, also referred as risk score in the literature. We used the sum rule as this method outperforms more complex combination schemes [As multivariate survival models using microarray data are prone to overfitting, we built the model by combining the univariate Cox models computed during feature selection . Each un schemes . We set To avoid over-optimistic estimation of prediction accuracy, a leave-one-out cross-validation (LOOCV) and M10FLODCV procedures were used. As LOOCV does not depend on the order of patients in the dataset, these results will be discussed in details.Four independent validation sets were used to assess the performance of the classifier. These demographics are shown in [Additional file This external validation set was kindly provided by the Guy's Hospital, London, United Kingdom, consisting of 77 patients diagnosed with early stage breast cancer and treated with adjuvant tamoxifen monotherapy. Samples were hybridized using Affymetrix U133PLUS2 Genechips™ according to standard Affymetrix protocols. Gene expression values from the CEL were normalized by use of the standard quantile normalization method in RMA  and are This dataset consisted of 60 patients diagnosed at the Massachusetts General Hospital, Boston, United States of America, and who were treated with adjuvant tamoxifen monotherapy. The samples were hybridized on the Agilent microarray platform and have been previously described . The rawThis external validation set was kindly provided by the Department of Experimental Oncology, Istituto Nazionale per lo Studio e la Cura dei Tumori, Milan, Italy, consisting of 113 patients who had received adjuvant tamoxifen monotherapy. Samples were hybridized on their local cDNA microarray platform. Part of this dataset has previously been published . We wereThis dataset consisted of 112 patients diagnosed at the Erasmus MC, Rotterdam, Netherlands and who were treated with tamoxifen in the metastatic setting as first line hormonal therapy. Tumour response previously described  includedMapping across microarray platforms was done using the "Cleanex" database  to retriAlthough the risk score can be used as a continuous variable, we divided the dataset into two prognostic groups to generate a high or low risk status as this allowed us to estimate hazard ratios and produce Kaplan Meier curves. For the purposes of this study, a binary classification was generated using a 70:30 cutoff that is, 70% of samples would be considered low risk  and 30% high risk of relapse on adjuvant tamoxifen monotherapy. This cutoff was an arbitrary figure chosen by the authors to balance the cost of tamoxifen vs. other more expensive endocrine agents against relapse risk. The results shown in this manuscript were from analyses using the 70:30 cutoff for tamoxifen-treated and the external validation GUYT2 datasets, though similar results were obtained with a 50:50 cutoff. However, the samples in the Ma and Reid datasets were chosen to be balanced for recurrences within 5 years and non-recurrences after 6 years . Therefore, a 50:50 cutoff was used to take into account for the balanced number of events. Performance of the classifier computed by LOOCV was assessed using Kaplan-Meier survival curves and log rank p-values. The overall performance of the classifier in the three adjuvant data sets was estimated using classical meta-analysis methods .Hazard ratios (HRs) for the risk groups defined by the classifier were calculated using a Cox's regression stratified by clinical center to account for possible heterogeneity in patient selection or other potential confounders among the various centers. For each independent validation data set, the HR  was displayed on a forest plot and tested for heterogeneity using a chi-square test . HRs werTo establish if the model predicted response to treatment, a univariate logistic regression model was used with the risk score as explanatory variable. Significance was determined by the Wald test and a false discovery rate (FDR) < 0.05.Statistical analysis was performed using SPSS statistical software package version 13.0 and the R software package version 2.3 .The Spearman's correlation between the risk scores produced by the predictor and that produced by GGI, previously described in Sotiriou et al, 2006 , was calAnalysis of gene interactions for each cluster of the final classifier was performed using Ingenuity Pathways Analysis (IPA) tools version 3.0 . AffymetGene expression profiles of the 255 patients in the training set were used to derive the predictor. A signature of 13 clusters was assessed to be highly stable [Additional file A risk score derived from 13 clusters was developed to predict the patient's risk of developing distant metastases as high or low risk. The final model consisted of 13 clusters and 239 associated probe sets  consisting of 250 samples taken from women at diagnosis and who had received the same adjuvant systemic therapy. These three datasets had been hybridized using different microarray platforms. The number of probe sets that could be mapped per cluster is shown in Table pclusts 148,120,375,201). However, overall, we found that our classifier had no discrimination ability in this group of patients. Interestingly one cluster centroid, cluster 375, was significantly associated with response (FDR = 0.008), suggesting that this cluster of 3 genes [see Additional file In order to delineate whether our classifier was predicting response to tamoxifen and/or the intrinsic aggressiveness of a breast tumor (prognostic), we applied our classifier to a data set of women who had received tamoxifen in the advanced setting where response to the treatment was clearly defined . Twenty-The GGI is an algorithm which can quantify the expression of proliferation genes in a breast tumor . Given tThe biological functions of each of the 13 clusters were analyzed in the context of a curated list of published molecular interactions by IPA. Table Developing gene signatures that are stable, are effective at distinguishing prognostic groups and provide important biological information from whole genome microarray data remains a significant challenge. We propose a method which has similarities to a technique proposed by Bair and colleagues ,22, in cWith this method, we were able to develop and validate a gene classifier that could predict which patients with ER+ BC were at high risk of relapse despite tamoxifen treatment. Importantly, we were able to validate the classifier on independent samples utilizing raw data from different microarray platforms using a meta-analytical approach. Demonstration of prognostic ability is important if we are to assemble gene lists from microarray data for biological hypotheses generation and potential laboratory experimental validation, which was one of the most important aims of this study. Validation of gene classifiers with independent samples from which they were developed from is a major challenge for microarray studies, especially those with clinical implications, and combining multiple datasets can be difficult due to different patient populations, sample preparation and microarray platforms. Our study uses one of the largest training and validation sets reported in the literature on tamoxifen (only) treated patients.Whilst, in the future we may have a microarray-based diagnostic test incorporating all 181 genes in the 13 clusters, at present the routine use of this technology is not logistically feasible. However, the advantage of our approach is that as each cluster consists of a group of genes that are highly correlated and hence effectively act as one covariate. Thus, a diagnostic test of just 13 genes (one per cluster) could be developed for clinical use if desired, even though for biological study the researcher would be more interested in all the genes per cluster. To demonstrate this, we took a series of 13 individual probe sets (one per cluster) and correlated their performance with the full classifier on the training set of 255 patients. The median correlation was 0.94 (range: 0.88–0.97). The top 26 ranked 13-gene classifiers (with a correlation ranging from 0.95–0.97) and their corresponding probe sets are listed in [Additional file Due to the pressing clinical need, several other investigators have also developed gene predictors that can predict outcome in ER+ BC treated with adjuvant tamoxifen monotherapy ,13,23,24Using a discovery-based whole genome approach, we have developed and validated a gene classifier that can distinguish patients at high risk of distant metastasis despite adjuvant tamoxifen monotherapy. In the future, these poor prognosis patients could be selected for prescription of other treatment modalities, such as chemotherapy and/or biological agents. In this study we propose an approach which has the advantage of facilitating both signature stability and biological interpretation. These are critical issues in the challenging task of building gene predictors for breast cancer patients as we endeavor to delineate meaningful biological and clinically useful information from the microarray-produced data.SL, BH–K was responsible for the design and execution of the study, collation of study materials, the microarray analysis of study samples, the collection, assembly and verification of the data, data and statistical analysis and interpretation and final manuscript writing; CD assisted with the collation of study materials, the microarray analysis of study samples, data analysis and interpretation and final manuscript writing; PW assisted with the microarray analysis of study samples, data analysis and interpretation; FL provided technical support with the microarray analysis; AMT, PE, CG, KR, supplied the tissue samples from Guy's Hospital; JR, MGD, MAP performed the bioinformatics analysis using the Reid dataset in Milan; EMJJB, MPHMJ, JAF provided the microarray raw data for the Jansen dataset, and assisted with the data analysis and interpretation; MJP provided the study funding; MD, GB, and CS supervised the study. CS conceived the idea for the study. All authors read and approved the final manuscript.Training dataset characteristics. Summary of patient and tumor characteristics of the 255 patients used in the development of the classifier.Click here for fileProbe clusters. (a) Full annotated gene list of all clusters  used for classifier development after preliminary clustering and filtering on the separate dataset of 137 samples. (b) Full annotated gene list of probe clusters (pclust) used in final classifier and global model.Click here for fileIndependent validation datasets characteristics. Summary of patient and tumor characteristics of the four independent validation sets.Click here for fileSignature size stability. Evolution of Stab criterion with respect to the signature size using multiple 10-fold cross-validation. The vertical dashed line represents the stability of the ranking using 13 pclusts.Click here for fileSignature size performance. Evolution of the log2 hazard ratio with respect to the signature size using multiple 10-fold cross-validation. The vertical dashed line represents the performance of the classifier using 13 pclusts. The horizontal red dashed line represents the limit of statistical significance.Click here for fileExternal validation of the classifier . (a) Kaplan Meier curves for Ma et al. The risk group was defined by the classifier using a 50:50 cutoff. The two survival curves were not significantly different according to the logrank test . (b) Kaplan Meier curves for Reid et al. The risk group was defined by the classifier using a 50:50 cutoff. The two survival curves were significantly different according to the logrank test . (c) Kaplan Meier curves for Jansen et al. The risk group was defined by the classifier using a 50:50 cutoff. The two survival curves were significantly different according to the logrank test .Click here for fileSimplified classifiers. Top correlated "simplified" classifiers of 13 genes (one probe set per cluster) and histogram displaying correlation between "simple" classifiers and final model.Click here for file


A method is presented to correct for technical bias in clinical microarray data, which increases concordance with known biological relationships. The performance of gene expression microarrays has been well characterized using controlled reference samples, but the performance on clinical samples remains less clear. We identified sources of technical bias affecting many genes in concert, thus causing spurious correlations in clinical data sets and false associations between genes and clinical variables. We developed a method to correct for technical bias in clinical microarray data, which increased concordance with known biological relationships in multiple data sets. The introduction of massively parallel measurement techniques such as gene expression microarrays has led to a much disputed paradigm shift in experimental design . When a Controlled reference data sets featuring spike-ins or mixtures have demonstrated the possibility of high accuracy in measurements and have been valuable for comparison of normalization algorithms and analytical techniques -4. Howevet al. [In most clinical microarray data sets, very little is known about the true expression levels, and without such reference points it is difficult to evaluate the effectiveness of data transformations or statistical methods. However, one possible criterion is that multiple samples taken from the same tumor should be more similar to each other than they are to samples taken from other tumors. In a breast cancer data set previously published by Signoretti et al. , 98 surget al. . After net al. , we perfIn an effort to increase the number of replicate pairs clustering together, we tried various thresholds for filtering out low-variance genes. This common practice is intended to reduce noise by removing genes that are essentially unchanged between samples . As expeAlthough it is remotely possible that the dendrogram resulting from the original, unfiltered data Figure  indicateTherefore, we searched for evidence of systematic bias in the Signoretti data set. We anticipated that any systematic bias would affect a subset of genes having some common characteristic, rather than a random subset of genes. Such bias would then tend to increase the apparent correlation between pairs of genes within this subset. By visualizing the correlation between pairs of probe sets, as a function of intensity, we observed intensity-dependent correlation bias Figure . RemarkaBecause PSQN only partially resolved the misclustering of replicates in the Signoretti data, we sought to understand the underlying causes of the bias. Two sources of variation, those arising during hybridization and those intrinsic to the starting RNA, seem especially likely to influence many probe sets in a coordinated manner. During hybridization, a subset of probes is susceptible to nonlinear effects caused by saturation at high intensities or by the noise floor at low intensities . For theA second potential source of bias is the variable quality and quantity of starting RNA, which may be especially problematic in data sets featuring samples from surgical specimens . For an In our current analysis we hypothesized that a set of four metrics could characterize the relative amount of bias affecting each sample in a data set. To capture the saturation and noise floor of the raw probe intensities, we used the median and interquartile range (IQR) of the perfect-match probes on the array ('PM median' and 'PM IQR'). The effect of RNA degradation was estimated by the average decrease in expression between 5' probes and 3' probes ('degradation'). Finally, the diversity of starting mRNA was characterized by the IQR of the RMA-summarized expression values ('RMA IQR').In the Signoretti data set, each of the four bias metrics is correlated with the expression values of more probe sets than expected by chance Figure . When apSince the bias metrics did not correspond to any available clinical covariates , we removed the component of the expression matrix explained by the bias vectors . The resulting bias-corrected data had substantially less intensity-dependent correlation bias Figure , and when range 99-289), a common platform (Affymetrix HG-U133A), and ER status annotation [To investigate the effects of bias correction on single-gene analysis, we used the estrogen receptor (ER) status of breast tumors as a reference point . Since tnotation ,12,19. BAs an alternative to our bias correction method, we considered batch adjustment, in which the expression matrix is adjusted, one probe set at a time, to remove any difference in means between the batches. From the scan dates embedded in the CEL files, we inferred that the Signoretti breast cancer data were scanned in three batches. We applied batch adjustment to this data set and found that the number of correctly paired replicates increased from four to five (data not shown). Thus, on this data set, batch adjustment is less effective than our bias correction method. The other data sets used in this study were each scanned in a relatively large number of batches , so batch adjustment was not attempted.Various analytical approaches are susceptible to errors caused by technical bias. For simpler analyses of individual genes , technical bias may manifest as an unmodeled factor that complicates significance testing . HoweverMore complex analyses have inferred functional relationships between genes based on their coordinated expression . Such apWe have found that technical bias can substantially affect clinical microarray data, and our results indicate that it should be corrected, or at least considered as a potential confounding effect, in any analysis. We have presented a simple approach for correction of biased data, but it may not be optimal for all data sets. Our choice of bias metrics was based on consideration of the processes involved in the generation of microarray data, but other variables, such as those used to justify the exclusion of outlying arrays, may also quantify technical bias . As moreAll analysis was performed using the R statistical environment , with usA reference distribution was calculated by averaging the sorted expression values across all samples. Then, for each sample, the original expression values were replaced by those of the reference distribution in the same order, with ties broken at random. This results in all samples in a data set having exactly the same distribution of expression values.n rows (probe sets) of the expression matrix were sorted according to mean expression value. The rows were then split into 50 bins of approximately equal size. Thus, the first bin contains the n/50 probe sets having the lowest mean signal, and the last bin contains the n/50 probe sets having the highest mean signal. For each possible pair of bins, all (n/50) × (n/50) pairwise Pearson correlation coefficients (between rows) were calculated, and the median correlation coefficient was recorded. These median correlation coefficients for each pair of bins were plotted as a color-coded 50 × 50 matrix. Note that for the diagonals of the color-coded matrix, which corresponds to the correlation between a bin and itself, only correlations between non-identical probe sets were counted.First, the 2-transformed perfect-match probe intensities; PM IQR is the interquartile range of the log2-transformed perfect-match probe intensities; RMA IQR is the interquartile range of all summarized RMA values ; and 'degradation' is the slope of the least-squares regression of the log2-transformed perfect-match probe intensities versus their relative position (within the probe set) along the targeted transcript - this was calculated with the AffyRNAdeg function in the Bioconductor package affy [The four bias metrics defined here were calculated for each microarray sample: PM Median isthe median of the logage affy .To calculate the random distributions in Figure Eij be the RMA-normalized expression value of probe set i in sample j. The four bias metrics Bmj (m ∈ 1..4) were calculated for each sample j independently. The effect of technical bias on the expression levels was estimated by least-squares regression with the bias metrics as independent variables. Specifically, for each probe set i independently, the parameters αi and βim were chosen to minimize the sum square of the residuals εij.Let The bias-corrected expression values For each data set, we calculated the Pearson correlation coefficient of each probe set with the binary ER status. The CC is the Pearson correlation coefficient of the resulting vector with the corresponding vector from a different data set. Thus, the CC indicates how well two data sets agree on the extent to which each probe set is correlated with the ER status.CC, correlation of correlations; ER, estrogen receptor; GCRMA, GeneChip robust multi-array average; IQR, interquartile range; MAS5, Microarray Suite 5; MBEI, model-based expression index; PSQN, post-summary quantile normalization; RMA, robust multi-array average.ACE and ZS conceived and designed the study. ACE performed all analysis. ZS provided guidance. Both authors wrote, read, and approved the final manuscript.The following additional data are available with the online version of this paper. Additional data file Eight arbitrarily chosen cancer data sets -12 were Click here for file