There are many potential sources of variability in a microarray experiment. Variation can arise from many aspects of the collection and processing of samples for gene expression analysis. Oligonucleotide-based arrays are thought to minimize one source of variability as identical oligonucleotides are expected to recognize the same transcripts during hybridization.We demonstrate that although the probes on the U133A GeneChip arrays are identical in sequence to probes designed for the U133 Plus 2.0 arrays the values obtained from an experimental hybridization can be quite different. Nearly half of the probesets in common between the two array types can produce slightly different values from the same sample. Nearly 70% of the individual probes in these probesets produced array specific differences.The context of the probe may also contribute some bias to the final measured value of gene expression. At a minimum, this should add an extra level of caution when considering the direct comparison of experiments performed in two microarray formats. More importantly, this suggests that it may not be possible to know which value is the most accurate representation of a biological sample when comparing two formats. Microarrays are rapidly becoming a standard tool for molecular biologists. The technology can reveal a multitude of information from even the simplest biological experiment. The technique is primarily used by scientists to generate ideas in experiments colloquially referred to as "fishing expeditions". However, the result of even simple experiments is hundreds to thousands of gene expression changes. Because of this the cost of validation is prohibitive. The microarray technology itself is expensive and this often prevents replication. Furthermore, the preliminary experiments are usually composed of the minimum number of samples absolutely necessary to address the question. This minimalistic approach is also used for other common microarray experiments. Rather than replicate an experiment or increase the size of a study, it is increasingly common for researchers to turn to other published microarray experiments to attempt to verify or expand their findings. Early attempts at cross-platform verification of microarray experiments utilized data generated in other laboratories from independent experiments -6. TheseHowever, recent studies have shed a more positive light on cross-platform comparisons. Several studies have now shown that the probes used to detect transcripts are the root cause of differences between platforms -10. The Oligonucleotide based arrays appear to have higher resolution and lower variability than cDNA based arrays ,20,21. TThe Stratagene Universal Human Reference RNA standard is frequently used in microarray experiments to provide a standard reference for comparison purposes. We previously processed this RNA standard several times for microarray experiments using the U133A and U133B arrays from Affymetrix. When the newer U133 Plus 2.0 (U133 Plus) arrays were introduced we also processed this Universal Reference RNA on the new array. The U133 Plus arrays contain 54,675 probesets. This includes 22,277 probesets with identical probes to those used on the U133A arrays, 22,645 probesets from the U133B arrays, and an additional 9927 newly designed probesets. After processing the first few Reference RNA samples on the U133 Plus arrays we evaluated the similarity of these measurements to those acquired with the corresponding probesets on the U133A arrays. We directly compared the values obtained from 12 U133 Plus arrays to those obtained from 12 U133A arrays. We extracted the data from the probesets found on both array types and performed invariant set normalization to place these sets of data on the same scale . FollowiWe decided to investigate this phenomenon further to be certain that the differences were not due to unknown bias introduced during processing of the samples. To minimize the possible sources of bias, we processed 5 aliquots of the Reference RNA in parallel using the same reagents. Following the production of labeled RNA we prepared enough hybridization mixture from each preparation to hybridize both a U133A array and a U133 Plus array at the same time. We further processed the samples in parallel to minimize staining and scanning issues. Following this carefully controlled experiment we examined the U133A probeset values relative to the same probesets on the U133 Plus arrays. This time the t-test indicated that more than 5000 probesets were yielding different values on the two array types . Nearly 10% of the evaluated probesets were strongly separated into distinct groups . There were 1568 probesets in which every recorded measurement from the A chip was larger than every recorded measurement from the Plus chip. There were 869 probesets in which every recorded measurement from the Plus chip was larger than every recorded measurement from the A chip. By chance alone we would only expect 43 such occurrences.et al. . This is less than would be expected by chance if all the probes were producing random measurements. This is not surprising. RNA in the hybridized samples should bind to the appropriate probes providing directed signal. This reduces the number of probes producing random signal and therefore reduces the number of probesets producing divergent measurements when two groups of 5 are compared. This phenomenon occurs at all p-values  within the same array type. A different 5 array by 5 array comparison yielded a different group of 500 probesets. Less than 2% of the probesets were in common between any two gene lists produced by such 5 by 5 comparisons. This illustrates that the apparent differences were nothing more than random variation that happened to fall into distinct distributions when the groups were formed. In contrast, the probesets found to be different when comparisons were made across the array types were more consistent. Approximately 5000 differences were observed at a p-value < 0.05 when 5 U133 Plus arrays were compared to a group of 5 U133A arrays. There was 60 to 85% overlap between any two lists generated by comparisons across array types. This suggests that the same probesets were being identified no matter which set of samples were chosen for comparison. This is strong evidence that the differences are real and not chance events.We ended up with a total of 34 replicate measures of the Universal Reference RNA, including 17 for each chip type. A comparison of the entire group of U133A arrays to the entire group of 17 U133 Plus arrays yielded more than 10,000 probesets with differential expression values specific to the array type. Therefore, as more data is added to the analysis the number of differential probesets increases rather than decreases, further suggesting that this is a function of the array and not due to chance. There were 101 probesets for which all 17 measurements for one array type were higher than all 17 measurements for the other array type. By chance this would be expected to occur only once in ~400,000 experiments of this type. We show 4 instances of this occurrence in Figure We have demonstrated that the measurements produced by the U133A arrays are different than the measurements from the U133 Plus arrays. At the same time the measurements are so similar that the correlation between array types is around 96%. In order to develop a frame of reference for the differences reported here we re-examined the latin-square data produced by Affymetrix scientists. This data set was produced from 42 hybridizations in which known quantities of RNA were added to a hybridization mixture containing labeled cRNA from HeLa cells. If we remove from consideration those probesets affected by the supplemented RNA, the rest of the data consists of 42 replicate measurements of the exact same hybridization mix. This is analogous to the experiment we performed by hybridizing the same cocktail to both a U133A chip and a U133 Plus chip. As a measure of variation we looked for probeset measurements that deviated more than 2-fold from the mean value of all the measurements. In both the latin square experiment and our experiments the probesets should have yielded the same value on all arrays as the same sample was measured. It is well known that the highest variation occurs for probesets with the lowest measurements. Therefore, we sliced the full data into three groups representing the highest expression values, the middle range, and the lowest measurements from the chips. Table The measurement differences between the A chip and the Plus chip are probeset specific but not transcript specific. This is illustrated by the results presented in table The transcript for KIAA0676 is detected by 4 different probesets. Three of these probesets behave very similarly on the two array types. The fourth probeset behaves in the opposite manner table . The indWe looked at the measurements recorded for the individual probes that comprise these probesets. We found that 10 of the 11 probes produced higher values on the U133A arrays than on the U133 Plus arrays for probesets 206431_x_at, 215994_x_at, and 212054_x_at. Conversely, the probeset 212052_s_at, which behaved in the opposite manner, had more individual probes yielding a higher value on the U133 Plus arrays. Therefore bias at the probe level appears to generate the bias observed for the full probeset. We looked at all the individual perfect match probes for the probesets that produced different measurements on the two array types. When the probeset produced a larger value in the U133A arrays an average of 7.6 individual probes (out of 11) were higher in the A arrays than in the U133 Plus arrays. Conversely, when the Plus chip produced a higher value, an average of 7.7 probes from these probesets were higher in the U133 Plus environment.We suspected that this bias might be due to a little bleed over from the surrounding probes on the array. We examined a number of individual probes. In many cases brighter probes surrounded the probes producing the higher values; however this was not universally true. More consistently, it appeared that the probes yielding higher values were just a little brighter on the arrays with the higher measurements. This suggested a possible manufacturing difference as the underlying cause rather than bleed over from neighboring probesets.One of the most important considerations in performing a microarray experiment is that the data obtained at the end is an accurate representation of the RNA present in the biomaterial used to start the process. Bias can lead to erroneous conclusions if the bias happens to track with the experimental condition evaluated or obscures the differences because the bias is larger than the biological effect. Bias can come from many sources -27. WhenThe impact of probe level variation depends on the nature of the question addressed in a microarray experiment. For a simple experiment performed in either platform the observed performance of a gene should be substantially reproduced by the alternative platform. However, if clinical samples are being accumulated for a disease state, the differences between platforms might prevent the pooling of samples processed in both platforms. This depends on the differences one finds in the experimental system. Small differences might be obscured while large differences would still be observable. We observed that many of the probesets with target genes present in the Universal Reference RNA showed differences greater than 2-fold. This bias could compromise experiments where the number of samples evaluated might otherwise allow one to detect differences less than 2-fold.Our data shows that microarray data can be very consistent at the same time as it shows a bias. Overall, any two arrays produced with the Universal Reference RNA yielded fairly consistent microarray results. The final measurements form a tight cluster of values for most probesets and the overall correlation was 0.96 for any two arrays. However, a significant number of probesets produced two distinctly different or partially overlapping distributions when the array formats were viewed separately figure  and 3. TThe trend in microarray research is towards generating larger sets of data for analyzing complex biological problems and diseases. Institutions are pooling resources to attain the large datasets required for analysis. Several groups have already evaluated the comparability of data produced at distant sites -30. TherSince their introduction, microarrays have been forecast for clinical use. Many people believe that patterns of gene expression will be used to identify disease states. Diagnosis, prognosis, and treatment options are all believed to lie in patterns of gene expression. While some people begin the search for these patterns, the microarray community has also been working towards improving the technology to produce more reliable results. Some sources of bias can be minimized or corrected. Others will simply be accepted as part of the process. Classifiers developed to interpret microarray data must allow for any variation that occurs in the measurement of individual probes. It remains to be determined whether the variation caused by the type of array will interfere with the performance or development of classifiers. For the present time it seems wise to use a single array type for the evaluation of microarray data. If the classification of samples relies on small gene expression differences, the classification may be microarray format specific. However, large differences will probably still be observed despite the small differences reported here between the U133A arrays and the U133 Plus arrays.Microarray data is often useful beyond the intentions of the original experiment. The microarray community is continuously developing standards for microarray processing and data management that allows scientists to utilize microarray data held in repositories. Much like sequence data, this stored gene expression information can be used in a multitude of creative ways. The comparison of microarray data between two formats, or even between two laboratories using the same format, requires knowledge about the sources of error that can arise in a microarray experiment. We have demonstrated that the same sequence can provide slightly different measurements of gene expression in different array formats. This implies that the comparison of microarray data between formats may require an additional, array specific, correction factor for each probe. The larger implication is that it may not be possible to establish equivalent correlations between the measured value in an array and the absolute value in a biological sample for every gene on an array. If sequence, as well as sequence context, introduces subtle adjustments to the final measured value of a transcript, then it may not be possible to know which measurement is the most accurate measurement of transcript abundance. Therefore attempts to perform cross-platform verification either have to use gene specific correction factors or be satisfied with similarity rather than exact replication.et al. [et al. [® scanner 3000 at a 2.5 μm resolution and the U133 Plus chips were scanned at 1.56 μm resolution. All data was initially generated from the scanned image files using the MAS 5.0 software embedded in the GeneChip Operating Software from Affymetrix. The data was initially normalized globally to an average intensity of 500. The signal values from each array were then exported to a text file. The probesets in the U133 Plus arrays but not found on the U133A arrays were removed from these files and then all the arrays used in this study were renormalized in a single group using the iterative process described by Li and Wong as well as by Wang et al. [The source of RNA was the Universal Human Reference RNA from Stratagene  for all samples. This RNA was processed using the established Affymetrix protocols for the generation of biotin-labeled cRNA, hybidization, staining, and scanning. A more complete description of this process is available in the papers by Warrington et al.  or Dobbi [et al. . Hybridig et al. ,23. ThisAll data used in this manuscript are publicly available for download. The initial samples placed on U133A arrays are available at the Gene Expression Database Portal (, experimThe Latin Square data set used for comparison was generated by scientists at Affymetrix and is available from the company's website .For the probe level analysis the values in each CEL file were loaded into a database. The probe values corresponding to those probesets found on the U133A arrays were extracted from each U133 Plus array and used to generate an artificial U133A CEL file. All the CEL files were then normalized to the same average value and individual probe values were compared across arrays. This technique also allowed comparisons using batch methods such as RMA or MBEI and perform other normalization techniques on the probe level data. The artificial U133A CEL files and corresponding U133A CEL files were loaded into a local implementation of the RMA program available through Bioconductor to calculate the expression values.LZ and SY produced the microarray data and some of the analysis presented in this manuscript. SE directed this project and wrote the majority of the manuscript.


Very few analytical approaches have been reported to resolve the variability in microarray measurements stemming from sample heterogeneity. For example, tissue samples used in cancer studies are usually contaminated with the surrounding or infiltrating cell types. This heterogeneity in the sample preparation hinders further statistical analysis, significantly so if different samples contain different proportions of these cell types. Thus, sample heterogeneity can result in the identification of differentially expressed genes that may be unrelated to the biological question being studied. Similarly, irrelevant gene combinations can be discovered in the case of gene expression based classification.in silico. The computational method provides estimates of the expression values of the pure (non-heterogeneous) cell samples. The inversion of the sample heterogeneity can be facilitated by providing accurate estimates of the mixing percentages of different cell types in each measurement. For those cases where no such information is available, we develop an optimization-based method for joint estimation of the mixing percentages and the expression values of the pure cell samples. We also consider the problem of selecting the correct number of cell types.We propose a computational framework for removing the effects of sample heterogeneity by "microdissecting" microarray data The efficiency of the proposed methods is illustrated by applying them to a carefully controlled cDNA microarray data obtained from heterogeneous samples. The results demonstrate that the methods are capable of reconstructing both the sample and cell type specific expression values from heterogeneous mixtures and that the mixing percentages of different cell types can also be estimated. Furthermore, a general purpose model selection method can be used to select the correct number of cell types. Recent developments in high-throughput genomic technologies have revolutionized the approaches aimed at understanding biological systems and emphasized the need for computational and systems biology research. Microarray analysis, for instance, can provide massive amounts of information about a biological sample by simultaneously measuring thousands of transcript levels. Application of such methodologies has already yielded important molecular insight into cellular phenotypes under various experimental conditions  and provin silico. Similar analytical approaches have been previously proposed by Lu et al. [et al. [et al. [et al. focused on estimating the fraction of cells in different phases of the cell cycle whereas Stuart et al. considered the problem of estimating the cell type specific expression patterns over all samples. Here we focus on estimating both the sample and cell type specific expression values using carefully controlled microarray experiments. The inversion of the 'cell mixing effect' can be made appreciably easier by providing estimates of the mixing percentages of different cell types in each measurement, which can be measured by an experienced pathologist. The entire process does not hinge upon such measurements, however, as the mixing percentages can be estimated within the modeling framework. Venet et al. [A recent paper by Ghosh  introducu et al. , Stuart  [et al.  and Vene [et al. . Lu et at et al.  introduct et al.  and as wThe microarray data to which we apply our computational methods consists of five different heterogeneous mixtures of lymph node and colon cancer samples which are hereafter abbreviated as normal and RKO, respectively. For more details, see Materials and methods Section. Each heterogeneous mixture consists of different fractions of different cell samples, see Table The first goal is to invert the mixing effect caused by sample heterogeneity. We apply the linear model developed in Materials and methods Section to the heterogeneous microarray data. The obtained results are presented below. and , i = 1, ..., K, denote the measured mixture and reference samples, respectively, and let  and  denote the estimated RKO and normal expression profiles. Let , where . Instead of finding the eigenvalues of the original sample covariance matrix DTD, we compute them for the matrix DDT. The eigenvalues of DTD and DDT are the same and the eigenvectors of DTD can be obtained from the eigenvectors of DDT by multiplying them by DT. Results of the inversion of the sample heterogeneity are shown in Figures Because of the inherent variability of individual gene expression values, the performance of the inversion method cannot be estimated based on results for individual genes.  are initialized using the measured values shown in Table In practice, the true mixing percentages are not known but must be measured by some means. Therefore, they are also likely to contain some error. So, in addition to inverting the mixing effect, it is also useful to simultaneously estimate the most likely value of the mixing percentages. This problem can be formulated as type of optimization problem, the details of which are shown in the Materials and methods Section. The proposed optimization scheme was applied to the heterogeneous microarray data. Since the heterogeneous samples #1 and #5 correspond to the cases where only colon cancer cells and lymph node cells are used, respectively, we may assume that α2 ≈ 0.9296) and the heterogeneous sample #4 is fairly far away from the heterogeneous sample #5 (α4 ≈ 0.3796). Note that estimation of the mixing parameters may also compensate for some other errors/biases in the data than just the mixing percentages.The found optimal values of the mixing percentages are shown in Table The obtained expression estimates for the pure colon cancer and lymph node samples, when all five heterogeneous samples are used in estimation, are shown in Figure #1 corresponds to a measurement of the pure colon cancer cells. Similarly, the estimated values from the lymph node sample are close to those from mixture #5 as well as to all of the reference samples. In Figures Again, the estimated expression values for the pure colon cancer cells (RKO) are close to those from mixture #1, as it should be, since mixture Above we were only interested in estimating the expression values of the pure cell types. Often it is also useful to assess the confidence intervals of the obtained expression estimates. For that purpose, we consider two methods: one based on Gaussian approximation and the other using bootstrap. x-axis according to the corresponding mixing fractions. The vertical lines at x = 0 and x = 1 expand over the maximum of the two confidence intervals. In most of the cases the two confidence intervals are in good agreement. The confidence intervals can be tightened by measuring more heterogeneous mixtures.For illustration purposes, Figure The proposed inversion methods for the sample heterogeneity were also tested on standard non-replicated microarray data by treating the replicated measurements for each gene as individual "genes." The obtained results were qualitatively similar with the ones shown above and only slightly more variable. In a similar fashion, we examined the effect of low quality replicates on the heterogeneity inversion. Slightly less variable results were obtained with a method  that detIt is known that the heterogeneous mixtures used in our experiments consist of only two cell types. However, in general case, heterogeneous mixtures may contain an unknown number of cell types. In those cases, it is useful to assess the validity of the model  as well. As introduced in Materials and methods Section, the linear mixing model can be extended to incorporate more than just two cell types. We use a general purpose cross-validation for model selection. In particular, we apply the so-called leave-one-out cross-validation (LOOCV) and test the one, two, and three cell type models. For the three cell type model, the number of samples does not permit us to optimize the mixing percentages for each cross-validation training data set separately. Therefore, within the cross-validation loop, we use fixed mixing percentages and only estimate the expression values. For the two and three cell type models we use the estimated mixing percentages shown in Tables This paper presents an inversion method for the effects of sample heterogeneity. The proposed method is successfully applied to a carefully controlled microarray data consisting of five different heterogeneous mixtures of lymph node and colon cancer samples. The results demonstrate that both the sample and cell type specific expression values can be reconstructed from heterogeneous mixtures. In some situations, such as cancer metastases in the lymph node, lymphocytes constitute a major cell type beside tumors. Hence, with careful sample preparation, the two cell type model can directly be applied to such cases. For unknown heterogeneous mixtures obtained from more complex cancer samples, the analysis may be a bit more difficult. For example, contaminating cells may include several cell types, such as fibroblasts, endothelial cells, macrophages and lymphocytes. As the proposed method can be applied to any cell types and to any number of cell types, the method works in principle in more complex cases as well. Requirement for the number of measurements necessary for reliable inversion, however, increases together with the number of cell types present in the sample.We have emphasized that proper inversion of the mixing effect results in more accurate expression values of the pure cell types. While this is true, it must be noted that clinically relevant information may also be incorporated into other populations than the pure (cancer) cells. For example, the degree of lymphocyte infiltration may be clinically important and could be used to complement microarray analysis. However, for comparative microarray analysis, it is important to make comparisons between homogeneous samples so as to minimize the confounding influence of different proportions of contaminating cell types.ith gene in the colon cancer sample, , is regarded as being differentially over-expressed  if the ratio of  to the expression value of the same gene in the lymph node sample, , is at least 2 . Of course, in practice, more sophisticated methods for detecting differential expression, including correction for multiple testing, should be used. However, for illustrative purposes, this example will suffice. Since only the heterogeneous samples are available, without any inversion of the mixing effect, one must compare the mixture measurement  and . Figure In order to illustrate the above 'in silico microdissection' in practice, consider the following  experimental setting. Given the three middle mixture measurements #2, #3, and #4), a goal is to identify a set of genes which are differentially expressed between the colon cancer and the lymph node samples. In a simple approach, often used in practice, the most heterogeneous sample would be discarded since it is measured to contain about 56%  of colon cancer cells , thus giving no direct discriminative information about the underlying two samples. For illustration purposes, let us measure the expression difference of a given gene between these two samples using the fold-change, i.e., the expression value of the , and #4,As is evident from the example above, heterogeneity in the biological sample preparation can hinder further statistical analysis steps. Not only can the heterogeneity blur the identification of differentially expressed genes, it can also cause contrary effects. Presence of a considerable percentage of additional cell types can result in the identification of differentially expressed genes that may be unrelated to the biological question being studied. Similarly, irrelevant gene combinations can be discovered in the case of gene expression based classification. For an illustration, see  where thet al. showed that a certain amount of "drop-outs" can be recovered from the multiplexed samples, thus providing more error-resilient measurements. Following the methods developed in [Although the microarray technology has been improved during the recent years, the measurements are still moderately noisy. The easiest and the most widely used approach for improving the measurement quality is to capture replicated measurements. This may become costly because each additional measurement requires an extra spot on the array, or an extra array. An alternative approach based on so-called composite microarrays was introduced in , where sloped in ,18, instloped in ,20, wherIn this paper, we proposed a computational framework for removing the effects of sample heterogeneity. In addition to providing estimates of the expression values of the pure (non-heterogeneous) cell samples, the proposed computational methods can also be used to estimate the mixing percentages of different cell types. Furthermore, we also proposed a way of applying general-purpose model selection method for the selection of the correct number of cell types. Application of the proposed methods to a carefully controlled cDNA microarray data obtained from heterogeneous samples shows that the computational methods can invert the effect of sample heterogeneity and, at the same time, estimate the mixing percentages of the different cell types. Furthermore, a general purpose model selection method can be used to select the correct number of cell types.μg aliquots of total RNA from normal lymph node and RKO colon cancer cell line were reverse transcribed using Superscript II RT  in conjunction with oligodT-T7 primers according to the manufacturer's suggested protocol. The second strand was synthesized using 10U E. coli DNA ligase (vendor), 40U E. coli DNA polymerase I (vendor), and 2U E. coli Rnase H (vendor). This reaction was stopped with EDTA and then cleaned with Qiagen's PCR Purification kit . The double stranded cDNA was then amplified by an in vitro transcription reaction  and cleaned with Qiagen's Rneasy kit . Each amplified cRNA sample was then quantitated using a Beckman DU640 spectrophotometer . Five μg amplified cRNA from Stratagene's normal lymph node was labeled with Cy5 for each microarray hybridization. Mixtures of appropriate volumes of cRNA from normal lymph node and RKO were labeled with Cy3 in a reverse transcription reaction using Superscript II RT  takes place on the molecular level, the phenomenon must be modeled using the absolute expression values. Therefore, after the correction of the dye-bias, the data were transformed back to the original domain using the inverse of the log2-transformation. Correspondingly, single-channel data were used for further analysis. In order to mitigate the between array variability, the data were further standardized for each array and the two channels separately.The microarray data consists of five different heterogeneous mixtures of lymph node and colon cancer samples which are hereafter abbreviated as normal and RKO, respectively.  with sm and  denote the expression level of the ith gene in the colon cancer (RKO) and in the lymph node  samples, respectively. Assuming only two different cell types are mixed, the sample heterogeneity is modeled by a simple linear modelThe two samples, RKO colon cancer cells and normal lymphocytes, are mixed at the extracted RNA level. Therefore, without any further verification, the model can be assumed to be linear. Lymphocytes were used because tumor tissues often contain infiltrating lymphocytes, especially in tumor metastases in the lymph nodes. Let  denotes the expression value of the ith gene in the kth heterogeneous sample, and 0 ≤ αk ≤ 1 denotes the fraction of the colon cancer cells in the kth mixture. It is worth noting that we use the same mathematical model for the sample heterogeneity as in [) and normal  is "fixed" and does not change between heterogeneous measurements. In other words, the measurements  come from the same heterogeneous sample with different mixing fractions. In order to allow variation in the expression values between different samples/treatments/time points, the same model can be applied separately to each set of measurements from the other samples/treatments/time points. The same model can also be extended to more than two cell types .where ty as in -11. Alsoyi, include one or more sources of noise. By making some distributional assumptions, one could use standard model-based estimation methods. However, in order to avoid making additional modeling assumptions, we prefer to use a general purpose least squares method to estimate the gene expression levels corresponding to the pure samples.The first objective is to invert the mixing effect shown in Equation (1), that is, to obtain estimates for the expression values of the pure colon cancer cells and the pure lymphocytes. In practice, however, the measured expression values, n and assume that one has measured the expression values for K different heterogeneous mixtures. Thus, one has measurements , 1 ≤ i ≤ n, 1 ≤ k ≤ K. Let us also assume for now that the mixing percentages are known or have been measured. For the ith gene the sample heterogeneity can be expressed as Let the number of genes be n genes, the above model can be rewritten asWhen including all 0 denotes the K-by-2 zero matrix. Let the block matrix in Equation (2) above be denoted as Ã Assuming the column rank of A is full, the well-known least squares solution is given bywhere . Due to the structure of the matrix Ã, the least squares solution can be obtained gene-wise as .where Ã ensures that the inversion can also be performed for each gene separately. Consequently, it is not necessary for the homoscedasticity to hold globally. Indeed, all we need to assume is that the noise variance is approximately constant for each gene separately.The Gauss-Markov theorem says that the standard least squares solution is indeed the best linear unbiased estimate if the noise in the measurements is additive and i.i.d. with constant variance. However, a common observation is that the homoscedasticity does not always hold for microarray data, but instead, the noise variance depends on the underlying signal intensity ,24. SuchAlso note that, in this two cell type model, no prior knowledge about the expression values of either of the two cell types is needed since the method estimates the expression values for both of the two cell types. The same is also true for more general models including more cell types, assuming the model is sufficiently over-determined .In practice, the mixing percentages must be measured by some means. Therefore, they are also likely to contain some error. So, overall, one would like to estimate not only the expression values for the pure cell types but also the most likely value of the mixing percentages.K denotes the number of different heterogeneous mixtures measured. Therefore, the regression matrix Ã in Equation (2) has only K free parameters. Since the number of expression values to be estimated is 2n, the total number of free parameters in Equation (2) is 2n + K. The number of equations in Equation (2) is Kn. Hence, the model is over-determined if Kn > 2n + K, which, for a fixed n ≥ 3, holds if K > 2.  As above, no assumptions on the noise distributions are being made and we use the least squares method. This results in the following optimization problemAssuming the model in Equation (2) is sufficiently over-determined, the mixing parameters can be adjusted computationally, too. Let us again consider the case where only two different cell types are mixed. Note that A similar optimization problem was also introduced in .Ã and x, the objective function is not linear in the parameters anymore and, therefore, cannot be solved as in Equation (3). In general, any iterative optimization method can be used to get a solution. Iterative methods usually become inefficient/unstable as the number of parameters to be optimized increases. In this case, the number of free parameters in Ã and x is 2n + K. Therefore, we use a two-step approach in the optimization. In the first step, given proper initial value for Ã, the least squares solution for x is found using Equation (3). In the second step, the mixing percentages are optimized in the least squares sense  using the previously found value for x. These two steps are then repeated, essentially resulting in a type of expectation-maximization (EM) approach. A similar iterative procedure was also proposed in [Ã, given the value of x, the optimization problem is again linear in its parameters. Assuming the constraints are not violated, the standard equation (similar to the one in Equation (3)) can be applied. If that is not the case, then any general-purpose constrained optimization method may be applied. Let  (resp. Âj) denote the value of x (resp. Ã) after the jth iteration. Details of the algorithm are shown in Figure Because the objective function in Equation (4) above is minimized over both posed in , except σ2 be constant, then the variance of the estimated expression values would be . Due to the special structure of the matrix Ã , the variance of the estimated expression values for the ith gene can be expressed asIt is useful to assess the confidence intervals of the obtained expression estimates. As explained above, the Gauss-Markov theorem is applied gene-wise that greatly alleviates the issue of heteroscedasticity. Should the noise variance  is the noise variance for the ith gene. A straightforward way of obtaining an estimate of the variance is to compute the sample noise variance  for each gene and then apply Equation (5) to get . That would result in somewhat sensitive variance estimates since there are only K = 5 error residuals associated with each gene. A better alternative is to pool genes which have approximately the same average expression value  and then compute the sample noise variance from the error residuals of the pooled genes. Although we do not assume Gaussian noise distribution, we can resort to the Gaussian approximation when computing the confidence intervals. For example, using the Gaussian approximation, the 1 - 2α confidence interval for estimated expression value of the ith gene in the colon cancer cells is , where Φ-1(·) is the inverse of the standard normal cumulative distribution function and  denotes the  element of the estimated variance matrix  (similarly for the lymph node sample: ). Alternatively, the confidence intervals can be obtained using the non-parametric bootstrap framework [α and 1 - α percentiles of the bootstrap distribution of the expression estimates.where ramework . Here we, where  denotes the expression value of the ith gene in the jth cell type, and 0 ≤  ≤ 1 denotes the fraction of the jth cell type in the kth mixture. The mixing percentages must also satisfy  for all k. The significance of different 'regression coefficients'  could be tested using standard regression-based statistical tests. Since those tests apply only to Gaussian noise we recommend using a general purpose cross-validation for model selection . Here wVD and WZ conducted the experiments and HL, IS, OY-H and WZ developed the computational methods. HL, IS and WZ prepared the manuscript.


Although microscopic diagnosis has been playing the decisive role in cancer diagnostics, there have been cases in which it does not satisfy the clinical need. Differential diagnosis of malignant and benign thyroid tissues is one such case, and supplementary diagnosis such as that by gene expression profile is expected.With four thyroid tissue types, i.e., papillary carcinoma, follicular carcinoma, follicular adenoma, and normal thyroid, we performed gene expression profiling with adaptor-tagged competitive PCR, a high-throughput RT-PCR technique. For differential diagnosis, we applied a novel multi-class predictor, introducing probabilistic outputs. Multi-class predictors were constructed using various combinations of binary classifiers. The learning set included 119 samples, and the predictors were evaluated by strict leave-one-out cross validation. Trials included classical combinations, i.e., one-to-one, one-to-the-rest, but the predictor using more combination exhibited the better prediction accuracy. This characteristic was consistent with other gene expression data sets. The performance of the selected predictor was then tested with an independent set consisting of 49 samples. The resulting test prediction accuracy was 85.7%.Molecular diagnosis of thyroid tissues is feasible by gene expression profiling, and the current level is promising towards the automatic diagnostic tool to complement the present medical procedures. A multi-class predictor with an exhaustive combination of binary classifiers could achieve a higher prediction accuracy than those with classical combinations and other predictors such as multi-class SVM. The probabilistic outputs of the predictor offer more detailed information for each sample, which enables visualization of each sample in low-dimensional classification spaces. These new concepts should help to improve the multi-class classification including that of cancer tissues. Histopathological analysis is the traditional mainstay of cancer diagnostics, and is relied on heavily to discriminate between malignant and benign tissues. However, in spite of its long and successful history as a routine medical technique, it is plagued by a number of well-known problems. For example, histological diagnosis often depends upon judgment calls by individual pathologists, leading to variations in diagnosis ,2. In adIn most examples of cancer classification, binary classifiers have been used, with multi-class predictors only rarely applied. However, because more than three tissue types are often differentiated during pathological diagnosis, a stable or optimized multi-class predictor is needed.In this study, we developed a new multi-class predictor based on a probabilistic model. We examined various combinations of binary classifiers to perform optimized multi-class prediction, and applied our system to gene expression data from four tissue types of human thyroid origin. Thyroid cancer is a relatively common cancer, accounting for roughly 1% of total cancer incidence. There are two main types of thyroid cancer, papillary carcinoma (PC) and follicular carcinoma (FC). In addition to these malignant types, a benign tumor, follicular adenoma (FA), is also prevalent. The intriguing problem in thyroid cancer therapy is the preoperative differential diagnosis of follicular carcinoma vs. adenoma. The main diagnostic procedure is fine needle aspiration, but because the tissue structure is disrupted during the sampling process, the differential diagnosis is extremely difficult . PapillaWe performed gene expression profiling of the four thyroid tissues by adaptor-tagged competitive polymerase chain reaction (ATAC-PCR) ,6, a hig-7 for the other four comparisons.We first examined whether there were significant differences in gene expression between the four thyroid tissue types by performing spectrum analysis based on correlation ratio (CR) . CorrelaWe then compared the CR spectra of all pairwise comparisons. The CR of each pair was calculated; genes were sorted by CR value order, and plotted Figure . Figure The CR spectra directly represent global differences in gene expression. We also evaluated these results by conventional measures, using p-values and q-values  of the sBefore performing multi-class prediction, we examined the performance of binary classifiers in the differential diagnosis of each pair of the four thyroid tissues. We constructed each diagnostic system based on the weighted-voting algorithm , which hp ≤ 0.0001. In this criterion, all of the corresponding q-values were less than 0.05, maintaining the least inclusion of false positive genes. It should be noted that application of the following strategy is not limited to weighted-voting. We examined the following four types of construction methods using different numbers of binary classifiers.In this study, we constructed a multi-class prediction system based on a combination of binary classifiers. Each unit binary classifier was constructed by the weighted-voting algorithm, where the diagnostic genes were selected by the criterion 1R: constructed using classifiers discriminating between one class and all of the other classes. 1R is conventionally referred to as "one-to-the-rest"  [and 1 - ql|m]  and . Our preliminary results using a method similar to that previously used by Hastie et al. in the pairwise case  [(x) is determined from a learning data set Xl|m] [= {(xn) ∈ X|tn)  may be a real value of 0 ≤ ql|m] [and bl|m] [are determined by maximum-likelihood estimation so that r(xn) for all learning data xn) (∈ X[l|m]  in a set B, and calculation of ql|m][(x) for l and 1 - ql|m][(x) for all targets [l|m] where subset l or m contains class c, and where probability scores are shared equally to member classes of the subsets. Therefore, this is called shared summation (SHS) of probabilistic output. Subsequently, we can obtain a Bayes-optimal class prediction as Here SIS), we determined class probability pc(x) for each class c ∈ C as a summation of ql|m][(x) for all targets [l|m] such that l or m is {c} itself,In simple summation (Z is a normalization constant. Because of the condition l = {c} or m = {c}, classifiers whose output has more than one label were not included in the above summation. As a result, AA and 1A are equivalent in the SIS case. Although SIS is a straightforward extension of conventional voting criterion to handle probabilistic guesses, it lacks a consistent model of decision process, such as that presents in the background of SHS.where 1A/AA-SIS/SHS using the weighted-voting algorithm is available from NY.Matlab programs to perform NY and SO implemented their algorithm, and NY carried out application experiments to gene expression data sets. SO and SI developed the mathematical formulation of the multi-class classifiers proposed in this paper. KK initiated and supervised the whole project. KT and KI-K did experimental parts of the thyroid cancer study. YT and SN collected thyroid tumor and normal tissues, and are responsible for clinical aspects of the study. All authors read and approved the final manuscript.contains all gene expression data.Click here for filecontains explanation of the data file 1.Click here for file


We provide a re-analysis of the Golden Spike dataset, a first generation "spike-in" control microarray dataset. The original analysis of the Golden Spike dataset was presented in a manuscript by Choe et al. and raised questions concerning the performance of several statistical methods for the control of the false discovery rate . These original findings are now in question as it has been reported that the p-values associated with the tests of differential expression for null probesets  are not uniformly distributed. Two recent publications have speculated as to the reasons the null distributions are non-uniform. A publication by Dabney and Storey concludes that the non-uniform distributions of null p-values are the direct consequence of an experimental design which requires technical replicates to approximate biological replicates. Irizarry et al. identify four characteristics of the feature level data . Irizarry et al. argue that the four observed characteristics imply that the assumptions common to most pre-processing algorithms are not satisfied and hence the expression measure methodologies considered by Choe et al. are likely to be flawed.We replicate and extend the analyses of Dabney and Storey and present our results in the context of a two stage analysis. We provide evidence that the Stage I pre-processing algorithms considered in Dabney and Storey fail to provide expression values that are adequately centered or scaled. Furthermore, we demonstrate that the distributions of the p-values, test statistics, and probabilities associated with the relative locations and variabilities of the Stage II expression values vary with signal intensity. We provide diagnostic plots and a simple logistic regression based test statistic to detect these intensity related defects in the processed data.We agree with Dabney and Storey that the null p-values considered in Choe et al. are indeed non-uniform. We also agree with the conclusion that, given current pre-processing technologies, the Golden Spike dataset should not serve as a reference dataset to evaluate false discovery rate controlling methodologies. However, we disagree with the assessment that the non-uniform p-values are merely the byproduct of testing for differential expression under the incorrect assumption that chip data are approximate to biological replicates. Whereas Dabney and Storey attribute the non-uniform p-values to violations of the Stage II model assumptions, we provide evidence that the non-uniformity can be attributed to the failure of the Stage I analyses to correct for systematic biases in the raw data matrix. Although we do not speculate as to the root cause of these systematic biases, the observations made in Irizarry et al. appear to be consistent with our findings. Whereas Irizarry et al. describe the effect of the experimental design on the feature level data, we consider the effect on the underlying multivariate distribution of putative null p-values. We demonstrate that the putative null distributions corresponding to the pre-processing algorithms considered in Choe et al. are all intensity dependent. This dependence serves to invalidate statistical inference based upon standard two sample test statistics. We identify a flaw in the characterization of the appropriate "null" probesets described in Choe et al. and we provide a corrected analysis which reduces (but does not eliminate) the intensity dependent effects. Normalization of microarray data is essential for removing systematic variation and biases that are present due to the nature of the assay. In experiments where the goal is to determine differential expression scientists have developed a variety of tests and algorithms to identify differentially expressed genes. One such experiment was the "Golden Spike" experiments by . In the A key component to the Golden Spike dataset is knowledge of the null p-values for tests of differential expression, that is, information of the genes that are present in a 1:1 ratio on the S chips and the C chips provides knowledge of which tests for differential expression are truly null. Figure In microarray experiments, the end-stage analysis usually consists of simple two-sample test statistics such as the t-statistic or the Wilcoxon Rank Sum test statistic to test for differential expression. However, it is important to note that these statistics generally operate upon data matrices which have been subjected to potentially significant amounts of pre-processing. With this technology, there are several steps required in order to process the data in order to achieve a single value representing the intensity for a given probe. It is worthwhile to consider the Affymetrix data acquisition in two stages. A Stage I analysis includes image processing where each spot is deemed to consist of a collection of pixels. From the collection of pixels at a spot an overall signal value is determined by taking a summary measure (often a median) of the pixel set at each hybridization location on the chip. In the Affymetrix data design there are 11 probe pairs spotted for each gene or SNP. Each probe pair contains two 25-mer DNA oligonucleotide probes; the perfect match (PM) probe matches perfectly to the target RNA, and the mismatch (MM) probe which is identical to its PM partner probe except for a single homomeric mismatch at the central base-pair position. The MM probe serves to estimate the nonspecific signal. In this stage, the PM and MM signals are combined into one score representing the expression signal for a specific probe. The major software packages for Stage I analysis include Bioconductor's "Affy" package, dChip and MAS 5.0 executables ,4.p by m where the p rows refer to the different probes, and the m columns refer to the different chips. The general procedure in normalizing this data is to use loess smoothers on the data set. One of the motivations for the Golden Spike experiment was to examine the numerous and varied normalization methods that currently exist for this data. Most of the normalization methods consider the data as a function of the matrix column. The goal of any of these normalization schemes is to reduce the systematic variation that exists in each chip. By considering each column of the data matrix as a separate chip, in each column we can scale and center the values, via loess smoothers so that each column has roughly the same "center" and "scale." This general approach  does noussed in  is to trX matrix of dimension p by m for each experiment where the p rows correspond to each (smoothed) probe value and the m columns correspond to the sample. In the Golden Spike datasets, numerous options in the Stage I analysis were examined, resulting in 152 different X matrices with each matrix corresponding to a different set of parameters in a Stage I analysis. From these 152 datasets, 10 "best" datasets were chosen that represented the best combination of processing in terms of detecting approximately 95 percent of true differentially expressed genes (DEGs) with changes greater than twofold, but less than 30 percent with changes below 1.7 fold before exceeding a 10 percent false-discovery rate. At this point, each data matrix X represents the input for Stage II analysis. The goal of Stage II analysis is to answer the researcher's questions of the experiment. Usually in the microarray setting this consists of a ranked list of genes determined to be differentially expressed between two groups such as treatment versus control. The methods of Stage II generally take in to account facets of the experimental design and allow the user to control for things like the false discovery rate (FDR) within a given two sample test environment. Often the validity of Stage II analyses depends upon the assumption that the Stage I analysis has provided a Stage II data matrix such that the two sample test statistic null p-values are uniformly distributed.Ultimately, the Stage I analysis results in an Dabney and Storey  provide We have replicated and extended the analyses of Dabney and Storey and we agree with the assessment that the null p-values are indeed non-uniform. We also agree with the conclusion that, given current pre-processing  technologies, the Golden Spike datasets should not serve as reference datasets to evaluate FDR controlling methodologies. However, we disagree with the assessment that the non-uniform p-values are merely the byproduct of testing for differential expression under the assumption that chip data are approximate to biological replicates when, in fact, they are not. Whereas Dabney and Storey attribute the non-uniform p-values to violations of the Stage II model assumptions, we provide evidence that the non-uniformity can be attributed to the failure of the Stage I analyses to correct for systematic biases in the raw data matrix.A recent article by Irizarry et al.  identifiAlthough the scope of this manuscript is in large part restricted to the re-analysis of the Golden Spike dataset, we also apply several of the same diagnostic plots to another Affymetrix spike-in experiment. The results suggest that some of the intensity dependent effects may exist in other settings. We relegate the extensive application and the continued development of such diagnostics to future research.Our analysis of the Golden Spike Dataset reveals that the null two sample t-test p-value distributions are non-uniform across the 152 combinations of Stage I analyses. The fact that all distributions were non-uniform implies that this problem can not be attributed to the procedure utilized to identify the ten best datasets. The two sample test was conducted using the equal variance t-test so that the analysis would be consistent with the one presented in [Dabney and Storey attributed the non uniform distribution of p-values to the fact that the Golden Spike experimental design requires technical replicates to masquerade as biological replicates. In their response to , the auti genes, i = 1, 2,...m, j treatments, j = C,S and k technical replicates, k = 1, 2, 3 and the Stage II expression data matrix X:Dabney and Storey proposed the following model for whereφijk~N  that a t-test statistic calculated with respect to random variables governed by model (1) constitutes an observation from a distribution which is heavier in the tails than a t4 distribution provided 2(1 - ρ)> 0. This follows from the fact that square of the denominator of the test statistic is an unbiased estimator of Wi. Hence, evaluating t-test statistics such as (4) against a t4 distribution will provide p-values which are negatively biased. This is the crux of the Dabney and Storey critique of the Golden Spike experimental design. Unfortunately the experimental design does not provide enough data to fit model (1) and directly estimate the relative magnitudes of where th order polynomial for rankit intensity, τ = 0.5 (solid) and τ = 0.25, 0.75 (dashed). Solid and dashed grey lines indicate the theoretical medians and quartiles, respectively. Inspection of Figure th order polynomial for rankit intensity. The curves were fit using the quantile regression and are coded as in Figures t4 distribution. This observation is consistent with the previous observation that the p-values associated with the "less than" alternative contain an excessive number of large p-values. For dataset 10e, the test statistics corresponding the null genes with overall signal intensities falling in the lower 15–20% appear to be negatively biased while test statistics corresponding the null genes with overall signal intensities falling in the upper 15% appear to be positively biased. The results in Figures Most importantly, the model (1) does not adequately explain the most intriguing aspect of the observed p-value distributions for all Stage II datasets; that the distributions are not invariant with respect to the overall signal intensity. Figures Each of the ten best Choe et al. Stage II data matrices were obtained using Stage I steps that included correcting the observed intensity with a loess curve that was fit to values from an invariant set of genes. This invariant set included present null  as well as empty null  probesets. The inclusion of the empty null probesets appears to have had a deleterious effect on the distributions of the null p-values. We have calculated a new set of ten best datasets in which the invariant set contains only the present null probesets. These calculations were performed at our request by the authors of  using anAlthough removal of the empty nulls from the invariant set provides data that is better centered than the original ten best, the results depicted in Figures th order polynomial for rankit intensity. To evaluate relative scaling, we propose modeling the probability that, for a randomly sampled probeset, the control samples will have a median absolute deviation (MAD) greater than the matched spike-in samples using the logit of a function of a 4th order polynomial for rankit intensity. Given that there were only three replicates in the Golden Spike dataset we used the average of the two absolute deviations from the median value in place of the more common formulation of the MAD (which would have provided only the minimum of the two non-zero absolute deviations). The curves presented in Figure A distribution free analysis of the ten best datasets  reveals that removal of the empty nulls from the invariant set provides for Stage II datasets which are adequately centered but inadequately scaled. We (loosely) refer to the analysis as distribution free because it does not include distributional assumptions associated with a test statistic. The adequacy of the centering and scaling of the data is, of course, relative. The re-loessed data appears to be adequately centered in that the probability that randomly selecting a null probeset such that the average expression value for the control samples is larger than that for the spike-in samples is approximately one half regardless of the overall signal intensity. The re-loessed data appears to be inadequately scaled in that the probability that randomly selecting a null probeset such that the variation in the expression values for the control samples is larger than that for the spike-in samples is less than one half. Further this variation is dependent on the overall signal intensity. Figure th order polynomial for rankit intensity) and were tested against a null model that the appropriate probability was constant with respect to rankit intensity. The deviances and asymptotic p-values (in bold) are reported. Given the possibility for cross hybridization of probesets, the assumption that the observed expression values are independent is dubious, although less tenuous than in a non-controlled experiment. The p-values, albeit approximate, indicate that the relationship between relative centering and intensity is highly significant in the original datasets and insignificant at a marginal level of 0.05 for all re-loessed datasets save 10e. The tabulated results indicate that the relationship between relative variability and intensity is highly significant for all datasets. However, the deviance values are significantly improved for several re-loessed datasets, most notably 10a-d.Table th order polynomial for rankit intensity. Inspection of Figures A set of diagnostic plots were created to assess whether the differences in relative centering and variability could be attributed to a one or two rogue samples. Figure Diagnostic plots were created for the Affymetrix "SpikeInSubset" data contained in the Bioconductor  SpikeInSFigure This analysis does not constitute a thorough investigation of the suitability of the SpikeInSubset data for validation of FDR estimation techniques. Unlike the Golden Spike data set, only a few naive "out of the box" algorithms were applied to the raw data . Figures The Golden Spike dataset was generated to address a dearth of controlled spike-in array datasets. The original analysis of the data was presented in  and concWe suspect that the underlying null distributions are adversely effected by failures of the normalization algorithms to properly account for the abnormal feature level characteristics identified by . While IOur analysis of the Golden Spike data also reveals that the invariant set of genes used for the pre-processing steps in Choe et al. should not have included the empty null probesets. We demonstrate that removing the empty probesets from the invariant set can provide Stage II data which appears to be adequately re-centered, a result which is dependent upon artificial knowledge of the true invariant set. Unfortunately, even under these ideal conditions, issues pertaining to higher moments  remain and these issues appear to invalidate the null distributions of commonly used two sample test statistics.Our analysis constitutes proof of principle that the distributions of the p-values, tests statistics, and probabilities associated with the relative locations and variabilities of the expression values can vary with signal intensity. This implies that Stage I algorithms do not always adequately adjust for intensity dependent effects. If the variation of the expression values for the null genes is a consequence of the unbalanced design of the experiment, then it is reasonable to speculate the existence of biological conditions which could engender similar imbalances. For example, such imbalances could occur when comparing different tissue types, in cases of immune challenge or in certain developmental time course studies. Although it remains an open research question whether our findings apply to other datasets we note the assessment by Irizarry et al. that experiments for which normalization assumptions do not hold are becoming more common.th order polynomial logistic regression model. The order and nature of the model were chosen for computational convenience. A higher order model or a spline based approach could conceivably provide an improved diagnostic. However, the properties of the diagnostic are dependent on the unknown multivariate distribution of the feature level values for invariant genes. In order to compare statistical tests for intensity dependence of the p-values we would need to characterize the multivariate distribution of the invariant probes. This is very difficult due to the complicated correlation structure among the probes ; a correlation structure that may vary from experiment to experiment. Thus the task of optimizing the diagnostics is fraught with challenges and has been relegated to future research.If the diagnostics which we have introduced prove useful for other datasets, then questions of optimality must be considered. For example, one of the diagnostic tests was based upon a 4The Golden Spike dataset was generated according to the experimental design described in  and clarThe Affymetrix "SpikeInSubset" data is contained in the Bioconductor  SpikeInSDG developed the analyses plan, conceived of the diagnostics and authored the required R code. JM tested the R code and contributed refinements to the analysis plan and the diagnostics. DG and JM contributed equally to the development of the manuscript.


With microarray technology, variability in experimental environments such as RNA sources, microarray production, or the use of different platforms, can cause bias. Such systematic differences present a substantial obstacle to the analysis of microarray data, resulting in inconsistent and unreliable information. Therefore, one of the most pressing challenges in the field of microarray technology is how to integrate results from different microarray experiments or combine data sets prior to the specific analysis.Two microarray data sets based on a 17k cDNA microarray system were used, consisting of 82 normal colon mucosa and 72 colorectal cancer tissues. Each data set was prepared from either total RNA or amplified mRNA, and the difference of RNA source between these two data sets was detected by ANOVA  model. A simple integration method was introduced which was based on the distributions of gene expression ratios among different microarray data sets. The method transformed gene expression ratios into the form of a reference data set on a gene by gene basis. Hierarchical clustering analysis, density and box plots, and mixture scores with correlation coefficients revealed that the two data sets were well intermingled, indicating that the proposed method minimized the experimental bias. In addition, any RNA source effect was not detected by the proposed transformation method. In the mixed data set, two previously identified subgroups of normal and tumor were well separated, and the efficiency of integration was more prominent in tumor groups than normal groups. The transformation method was slightly more effective when a data set with strong homogeneity in the same experimental group was used as a reference data set.Proposed method is simple but useful to combine several data sets from different experimental conditions. With this method, biologically useful information can be detectable by applying various analytic methods to the combined data set with increased sample size. DNA microarrays are a useful tool for the study of complex systems and have applications in a wide variety of biological sciences. Despite their usefulness, however, systematic biases caused by different handling procedures present a challenge. Microarray experiments are often performed over many months, and samples are often collected and processed at different institutions. Further, the samples may be assayed using different microarray print batches or platforms, or using different array hybridization protocols. When two microarray data sets are directly compared, systematic biases arising from variability in experimental conditions can be erroneously detected as differences in gene expression patterns. Such systemic biases present a substantial obstacle in the analysis of microarray data. However, due to the limited numbers of available microarray experiments, the motivation to use an entire data set regardless of platforms or experimental procedure is increasing. Therefore, it is necessary to investigate new methods that can effectively combine microarray data sets which were derived from different experimental environments, while simultaneously minimizing systematic bias.A commonly utilized method to integrate microarray data sets is to focus on the differential expression, i.e. comparing significantly expressed genes selected separately from each data set -7. AnothRecently, the practice of integrating data sets prior to selecting significant genes was introduced and standardization has been used for this as the simplest method . Singulamixture score.A method that transforms the distributions of gene expressions of two data sets similarly was proposed . HoweverSeventy-eight experiments  from data set A and 76 experiments  from data set B were used in this study. The whole data set included missing entries in the range of 448 to 1298 genes for each experiment. A total of 12293 genes without missing entries were used for further analysis.We confirmed RNA source effect in more than 5000 genes by ANOVA using adjusted p-value of 0.05 by Bonferroni correction (data not shown). The numbers of genes which were significantly differentially expressed between two experimental groups were 2325 and 1654 in data set A and data set B, respectively. This showed that there was some difference in sensitivity between two data sets. Even in same experimental group, there were many genes that expressed differently between two data sets, 4668 genes in normal group and 3364 in tumor group. The significant difference between different experimental groups, normal and tumor groups, was not detected by RNA source effect in these genes. Figure As shown in Figure In analyzing the five randomly selected genes with density plots, we found significant differences in locations of the first four genes, and in dispersions of gene203 and gene5112 between the two data sets  clustered the samples into two distinct groups according to the data sources rather than different experimental groups Figure . Howevermixture score. Euclidean distance and Pearson correlation coefficient were used as similarity measures to calculate the mixture score, and k was considered from 5 to 35 as a number of nearest neighbors (NNs). From the mixture scores before transformation, we found that the normal groups of data sets A and data set B were hardly intermixed  Take random subsamples from data set A and data set B  without replacement. This process was repeated 10 times with same sample size for reducing sampling bias.(2) Find the 500 significant genes list from the subsample of A.(3) Find the 500 significant genes list from (subsample A)(subsample B) ', which is a combined data set with subsample of A and transformed data set of subsample of B. (4) Compare (2) and (3) with 500 significant genes list selected from data set A.Figure The gene sets with a similar OOB (out-of-bag) error rate were compared (information of each gene set not shown), and none of the gene sets had 0% OOB error regardless of the transformation methods. Only two significant genes of data set AB' had 0.65% OOB error. Even if the number of significant genes was increased to 500, the OOB error rate did not decrease. The prediction accuracies of top 500 significant genes were compared, and the prediction accuracies were 100% for all of transformation methods (data not shown). In this case, integrated data sets by each transformation method were used as training data sets to create classifiers, and separated data sets were used for testing these classifiers.An inescapable problem with combining several microarray data sets is the variation of expressions between data sets. In cases where the microarray analyses are from different experimental conditions, integration without transformation may skew the expression ratios of the same genes from different data sets. When the experimental bias exceeds biological variation, the use of microarray data sets without adjustments for this bias may make biological variation unidentifiable, meaning that reliable results cannot be obtained. In addition, due to the limited numbers of available microarray experiments, the motivation to use the whole data set, regardless of platforms or experimental procedure, is increasing.We attempted to minimize experimental bias by transforming the expression ratios of the data sets such that they have similar expression patterns in the corresponding experimental groups of different data sets. Compared with previous studies, the proposed transformation method is a relatively simple algorithm -13, furtOur method transforms expression ratios by three approaches. In A'B, data set B was used as a reference data set, data set A in AB'. A'B' is a combined data set after transforming both of two data sets using pooled standard deviation. In selecting the reference data set, we did not consider biological meaning and we rather compared the effects of transformation with diverse references. When a method used a data set with strong homogeneity as a reference data set, its performance was slightly better than other transformation methods as shown in Figure k NNs of a experiment in one data set includes experiments of the other data set  because the expression values of a specific experimental group are transformed into the form of the corresponding experimental group of a reference data set. Thus, we suggest that the proposed integration method is useful when each data set includes phenotypically or biologically homogenous experimental groups.In conclusion, our method is simple and useful to combine several datasets experimented under different experimental conditions and available to any data set including more than two groups. With this method, biologically useful information can be detectable by applying various analytic methods to combined data set with increased sample size.A total of 154 colorectal tissue samples  were obtained from colorectal cancer patients who had undergone surgery at the Severance Hospital, Yonsei University College of Medicine, Seoul, Korea. Informed consent was obtained from patients prior to using their surgical specimens and clinicopathologic data for research purposes. Fresh tissues obtained from patients were snap-frozen and stored at -80°C.Total RNA was extracted from the tissues using Trizol  and then purified using an RNeasy kit . The purified RNA samples were divided into two groups for gene expression profiling using total RNA and amplified mRNA. Gene expression profiling using total RNA samples consisted of 20 paired normal and tumor colon tissue samples, 23 tumor samples, and 15 normal colon tissues. This data set is used by data set A in this study. Of the remaining 34 paired samples, 5 tumor and 3 normal colon tissues were used for gene expression profiling with amplified mRNA, which was obtained using the linear T7 mRNA amplification method with the Megascript T7 kit . This data set is used by data set B in this study. Each sample of total RNA (50 ug) and amplified mRNA (2 ug) was directly labeled with Cy5-dUTP and transcribed to cDNA. The microarray experiment was performed using a reference design with the Cy-3 dUTP labeled Yonsei reference RNA . We usedThese two microarray data sets have only difference on RNA source. Previous studies have concluded that it is vital to use equally treated samples for any particular study, and all other samples should be amplified when one sample requires amplification. In addition, the sensitivity to detect differential gene expression from microarray data set using amplified RNA was also different compared to using total RNA ,21. TherExpression intensities were normalized such that they would have similar distributions across a series of arrays. In this study, the MAD (median-absolute-deviation) scale estimator was used as a robust estimate of scale, and both A-values, as well as the M-values, were normalized. Within-slide and between slide normalization were used to transform expression values to make intensities consistent within each array and transform expression values to achieve consistency between arrays, respectively. It was necessary to apply between-slide normalization to the expression data because there were different dispersions between arrays after within-slide normalization. The normalization process was executed using the 'limma' library of the R package .The gene expression intensities of each data set were transformed based on the reference data set by the following three different methods to have similar expression patterns in corresponding experimental group.1) A'B: The gene expression ratios of data set A were transformed into the form of data set B which is considered as a reference data set. The transformed expression ratios of normal and tumor groups in data set A were calculated for each gene as follows:AN', AT': transformed expression ratios of normal and tumor groups in data set A.where AN, AT: normal and tumor groups in data set A.sd(AN), sd(AT), sd(BN), sd(BT): standard deviation of expression ratios of tumor and normal groups in data set A and B.2) AB': The gene expression ratios of data set B were transformed into the form of data set A which is considered as a reference data set. The transformed expression ratios of normal and tumor groups in data set B were calculated for each gene as follows:BN', BT': transformed expression ratios of normal and tumor groups in data set B.where BN, BT: normal and tumor groups in data set B.3) A'B': The gene expression ratios of data set A and data set B were transformed using the pooled standard deviation and mean expression values of the two data sets. The transformed expression ratios of the normal and tumor groups in data set A and data set B were calculated for each gene as follows:where nAN, nBN, nAT, nBT: number of experiments of AN, BN, AT and BT.mixture score, defined to evaluate the efficiency of the integration method proposed in this study. The principle of this metric is to measure how many k-nearest neighbors (kNNs) of data set B in combined data set belong to data set A. The metric was calculated as follows, where k is the number of nearest neighbors (NNs).We evaluated our proposed integration method by several plots and x is any experiment belonging to kNNs(data set B) and data set A.where The mixture score ranges from 0 to 1. A value close to 0.5 is indicative of two different data sets that are perfectly intermixed. Conversely, values close to either 0 or 1 indicate a poor level of intermixing between the two different data sets.We evaluated the classification accuracies of selected gene sets from integrated data sets by different transformation methods and used Random Forest algorithm (RF)  for thisn datasets of bootstrap samples {B1, B2,..., Bn} by allowing repetition of the same sample.(1) Generate Bk to construct a Tree classifier Tk to predict those samples that are not in Bk, called out-of-bag (OOB) samples. These predictions are called out-of-bag estimators.(2) Use each sample (3) Final prediction is the average of out-of-bag estimators over all bootstrap samples and we get average of them which is overall classification error (OOB error).ANOVA  model was used to evaluate the RNA source effect of data sets derived from different experimental conditions. ANOVA model used in this work is as following.gijk is kth expression ratio of a gene in ith treatment and jth RNA source. Ti, Rj and (TR)ij represent treatment effect, RNA source effect and interaction effect, respectively.where SVD: Singular Value Decompositin, SVM: Support Vector Machine, KNN: K Nearest Neighbors, NNs: Nearest Neighbors, ANOVA: Analysis Of Variance, OOB error; Out Of Bag error, HCA: Hierarchical Cluster Analysis.KYK participated in the design of algorithms, performed statistical analysis and drafted the manuscript. DHK performed the microarray experiments with amplified RNA. HJJ carried out the microarray with total RNA. HCJ participated in getting the consent form the patients and obtained the clinical data. HCC participated in the study design and data interpretation. SYR conceived of the study, participated in its design and coordination, and finalized manuscript.