Spotted cDNA microarrays generally employ co-hybridization of fluorescently-labeled RNA targets to produce gene expression ratios for subsequent analysis. Direct comparison of two RNA samples in the same microarray provides the highest level of accuracy; however, due to the number of combinatorial pair-wise comparisons, the direct method is impractical for studies including large number of individual samples . For such studies, indirect comparisons using a common reference standard have been the preferred method. Here we evaluated the precision and accuracy of reconstructed ratios from three indirect methods relative to ratios obtained from direct hybridizations, herein considered as the gold-standard.We performed hybridizations using a fixed amount of Cy3-labeled reference oligonucleotide (RefOligo) against distinct Cy5-labeled targets from prostate, breast and kidney tumor samples. Reconstructed ratios between all tissue pairs were derived from ratios between each tissue sample and RefOligo. Reconstructed ratios were compared to (i) ratios obtained in parallel from direct pair-wise hybridizations of tissue samples, and to (ii) reconstructed ratios derived from hybridization of each tissue against a reference RNA pool (RefPool). To evaluate the effect of the external references, reconstructed ratios were also calculated directly from intensity values of single-channel (One-Color) measurements derived from tissue sample data collected in the RefOligo experiments. We show that the average coefficient of variation of ratios between intra- and inter-slide replicates derived from RefOligo, RefPool and One-Color were similar and 2 to 4-fold higher than ratios obtained in direct hybridizations. Correlation coefficients calculated for all three tissue comparisons were also similar. In addition, the performance of all indirect methods in terms of their robustness to identify genes deemed as differentially expressed based on direct hybridizations, as well as false-positive and false-negative rates, were found to be comparable.RefOligo produces ratios as precise and accurate as ratios reconstructed from a RNA pool, thus representing a reliable alternative in reference-based hybridization experiments. In addition, One-Color measurements alone can reconstruct expression ratios without loss in precision or accuracy. We conclude that both methods are adequate options in large-scale projects where the amount of a common reference RNA pool is usually restrictive. Gene expression studies using either oligonucleotides or spotted cDNA microarray platforms are based mainly on data generated in single or dual-channel analysis. While industrially manufactured oligonucleotide arrays (e.g. Agilent Whole Genome 44 k Oligoarray and CodeLink Whole Genome Bioarray) have optimized protocols that perform well on both single and dual-channel microarray experimental designs, custom made spotted cDNA or oligonucleotide microarrays have been mainly employed in two-channel designs . Single-While direct hybridization of two experimental RNA samples in the same slide is highly desirable, indirect comparison through the co-hybridization of a test sample together with a common reference standard is the most used experimental design ,4. RelatIn this study we designed a set of experiments to evaluate the precision and accuracy of gene expression ratios derived from two-color microarray hybridizations using each of three tumor tissue RNA samples and two different external references: (i) a pooled tumor RNA sample (RefPool) that was labeled in parallel with the test tissue sample; and (ii) a 27-mer reference oligonucleotide (RefOligo) complementary to every feature of the array which was labeled by chemically coupling of a fluorescent nucleotide. The RefOligo method was originally proposed by Dudley et al. to control intensity ratios of gene expression studies in yeast , a systeRNA isolated from three types of human tumor tissues  was used to generate fluorescent targets for microarray hybridizations. Finding differentially expressed genes among different cancer tissues and cell lines and identifying gene expression signatures for each of them is a frequent task in the study of human gene expression using microarrays ,18,19. IA detailed description of data normalization, filtering and averaging of replicates is described in the Methods section. Intensity ratios for Kidney vs. Breast comparison are depicted in Figure To estimate the precision of RefOligo and RefPool in reconstructing ratios obtained in direct hybridizations we calculated the coefficient of variation (CV) for intra- slide replicates Figure . As all The accuracy of indirect methods was estimated by correlation analysis between direct ratios and reconstructed ratios. Representative scatter plots between direct and reconstructed ratios are shown in Figure The ability of each indirect method to identify the same set of genes found in the direct comparisons was evaluated. Genes differentially expressed in the direct hybridizations between each pair of tumor tissues were selected using 95 % credibility intervals for differential expression , and werData presented here show that RefOligo is a reliable alternative to a RNA pool in reference-based hybridization experiments when studying organisms with a complex transcriptome such as humans. The implication of our results is that the unlimited availability of an inexpensive (~US$ 0.30 per hybridization), chemically synthesized reference oligonucleotide makes its use very convenient in large-scale projects where the availability of an RNA pool is usually restrictive. The RefOligo method enables an efficient and flexible experimental design because one may relate expression measurements to a common reference  or, alternatively, use intensity values only (single-channel) . As the Another important conclusion of our study is that use of One-Color ratios does not compromise precision more than other currently used methods based on indirect measurements. This observation is in line with recent evidence showing that gene classifiers based on intensity measurements may outperform ratio-based classifiers . Intra- Prostate, kidney and breast tumor samples were obtained from freshly-frozen tissue collections maintained by Instituto Nacional de Câncer, Rio de Janeiro  and Unidade de Genética e Patologia Moleculares, Hospital do Divino Espírito Santo, Portugal (breast adenocarcinomas). All samples were collected between 1999 and 2002 with informed consent from patients submitted to surgery, and were snap-frozen in liquid nitrogen within 10 min from resection. All samples were examined by a pathologist at each Institution, and, in the case of prostate, hematoxylin/eosin stained micro-sections obtained from each side of the frozen blocks were used to grossly delimit the spatial distribution of the tumor mass. If necessary, tissue blocks were further dissected to warrant that at least 70% of the section used for RNA extraction was composed of malignant cells. Macro-dissected tumor samples were returned to liquid nitrogen until use.Total RNA was isolated with TRIzol (Invitrogene) using the protocol recommended by the manufacturer. For each tissue, aliquots of RNA isolated from 5 patient samples were pooled, in order to minimize the effect of biological diversity within each set of tissue samples, as well as to circumvent the problem of limited amounts of RNA available from each individual tumor sample. A reference RNA pool (RefPool) was assembled by combining equal amounts of RNA from each one of the three tumor tissues. The amount and quality of each RNA sample was verified on an Agilent 2100 Bioanalyser.RNA samples were labeled with either Cy3- or Cy5-labeled nucleotides , using 10 μg of DNAse-treated RNA and a mixture of oligo-dT and 9-mer random oligonucleotides.For the indirect hybridizations, Cy5-labeled cDNA targets derived from RNA of each tissue were co-hybridized to either a fixed amount (100 pmol) of a 27-mer 5'-end Cy3-labeled reference oligonucleotide , or to Cy3-labeled cDNA targets from the reference RNA pool (RefPool) labeled as described above with CyScribe first-strand cDNA labeling kit . The 27-mer RefOligo has no sequence similarity to any human expressed sequence available in GenBank. For RefOligo and RefPool, two replicate hybridizations were performed for each tissue. For direct hybridizations, fluorescently-labeled targets derived from each test tissue sample were combined pair-wise. A replicate hybridization with dye-reversal was performed for each pair of tissues.2O) and manually hybridized for 16 h at 42°C to microarray slides containing 4,608 different cDNA probes, each spotted in duplicate on either half of the slide. A detailed description of the spotted cDNA microarray platform used is presented elsewhere /2 + log2 [test sample 2(cy5)]/2A = logAs there were 2 technical replicates for each indirect hybridization, and each cDNA probe was deposited in duplicate in each slide, 8 possible reconstructed expression ratio values could be generated for a given cDNA probe: K1L/B1L, K1R/B1R, K1L/B2L, K1R/B2R, K2L/B1L, K2R/B1R, K2L/B2L and K2R/B2R, where K and B denote Kidney and Breast for example, 1 and 2 denote different hybridizations and R and L denote the right and left spot sets from each slide, respectively. Final ratio values was obtained by taking the median value of all 8 reconstructed ratios for each cDNA probe, for RefOligo, RefPool and One-Color comparisons.2 ratios from dye-swap replicate experiments as described in [For direct hybridizations, LOWESS normalization was performed by combining logribed in . For dirdir = 0.5 * log2(cy5/cy3 * cy3'/cy5') =M2(sample 1 (cy5)/sample2(cy3) * sample 1(cy3')/sample 2(cy5'))= 0.5 * logand2(Samaple 1(cy3) * sample 1(cy5) * sample 2(cy3) * sample 2(cy5))A = 0.25 * logRaw and processed data files from direct and indirect hybridizations are available at author's website .Precision of intra-slide replicates from direct or reconstructed ratios was estimated by calculating, for each tissue comparison, the average coefficient of variance (CV) of the two replicated spots representing the same cDNA that are present in the microarrays. Final average intra-slide CV +/- SD from RefOligo or RefPool was calculated from the average intra-slide CV of each method measured in each tissue comparison. Average inter-slide CV was estimated by calculating the average CV of reconstructed ratios measured across the same spots in different slides.2-ratios reconstructed from RefOligo or RefPool along the range of expression intensities, MA-plots were generated for each hybridization method (for each tissue comparison) using M and A values calculated from all replicates as defined above. For this analysis, the 2 SD intensity filter was not applied to access the variance on the entire range of intensities. Next, a sliding window of 1.5, moving 0.4 at each step, was used to calculate the variance of reconstructed ratios M along the range of expression intensities A (X axis). These analyses were performed using scripts written in R [To evaluate the variance of expression logten in R . The R sten in R .To evaluate the accuracy of expression ratios reconstructed from RefOligo, RefPool and One-Color measurements, a set of genes differentially expressed between each pair of tissues based on the direct hybridizations was identified, and was defined as the gold-standard set. Genes that were differentially expressed in the pair-wise tissue comparisons were selected with the statistical approach described in . In shorEMR and BRP conceived the initial idea, the experimental design, supervised the work, performed statistical analysis and wrote the manuscript. RZNV contributed with the experimental design and the development of R scripts for data processing and for the identification of differentially expressed genes using the HTself method. CE helped with nucleic acid isolation and performed all labeling and hybridization experiments. LMV articulated the collection of tumor samples along with detailed ethical guidelines. SVA contributed to the experimental design and to the writing of the manuscript and provided the facilities (wet lab and informatics) for execution of the experimental work. All authors read and approved the final manuscript.MA-scatterplots from direct and reconstructed ratios.Click here for fileDetermination of intensity-dependent credibility intervals for differential gene expression in direct and indirect comparisons.Click here for fileCorrelation between direct and reconstructed ratios.Click here for file


There are currently many different methods for processing and summarizing probe-level data from Affymetrix oligonucleotide arrays. It is of great interest to validate these methods and identify those that are most effective. There is no single best way to do this validation, and a variety of approaches is needed. Moreover, gene expression data are collected to answer a variety of scientific questions, and the same method may not be best for all questions. Only a handful of validation studies have been done so far, most of which rely on spike-in datasets and focus on the question of detecting differential expression. Here we seek methods that excel at estimating relative expression. We evaluate methods by identifying those that give the strongest linear association between expression measurements by array and the "gold-standard" assay.Quantitative reverse-transcription polymerase chain reaction (qRT-PCR) is generally considered the "gold-standard" assay for measuring gene expression by biologists and is often used to confirm findings from microarray data. Here we use qRT-PCR measurements to validate methods for the components of processing oligo array data: background adjustment, normalization, mismatch adjustment, and probeset summary. An advantage of our approach over spike-in studies is that methods are validated on a real dataset that was collected to address a scientific question.We initially identify three of six popular methods that consistently produced the best agreement between oligo array and RT-PCR data for medium- and high-intensity genes. The three methods are generally known as MAS5, gcRMA, and the dChip mismatch mode. For medium- and high-intensity genes, we identified use of data from mismatch probes (as in MAS5 and dChip mismatch) and a sequence-based method of background adjustment (as in gcRMA) as the most important factors in methods' performances. However, we found poor reliability for methods using mismatch probes for low-intensity genes, which is in agreement with previous studies.We advocate use of sequence-based background adjustment in lieu of mismatch adjustment to achieve the best results across the intensity spectrum. No method of normalization or probeset summary showed any consistent advantages. Nguyen et al [Affymetrix GeneChipen et al  give an ® oligonucleotide arrays. Methods employ different procedures for adjusting for background fluorescence, normalizing the data, incorporating the information from "mismatch" probes, and summarizing probesets . In particular, the value and proper use of data from MM probes have been subjects of some controversy [Quantitative researchers have proposed a variety of methods for handling probe-level data from Affymetrixtroversy . It is itroversy . Note thtroversy .across genes. Instead, Bolstad et al [Previously, spike-in studies have been used to study the variance and bias of different estimates of relative expression derived from oligo array data. These studies are useful and important, but are not the end of the story. First, spike-in datasets are inherently artificial, and may not realistically represent the operating characteristics of a methodology on real data . For exaad et al  largely Choe et al  conducteAmong evaluations that do not rely on spike-in datasets, Ploner et al  favored In contrast to the studies that use spike-in datasets, our study is based on a real dataset that was collected to answer biological questions. The studies by Choe et al  and Shed® (Murine Genome Array U74Av2) were used to assay RNAs from these tissue samples. Quantitative RT-PCR measurements for 47 genes were taken on these same 24 RNAs. As the "gold-standard" method of measuring gene expression, we treat the qRT-PCR measurements as "truth" for the purposes of this study. In our "overview" investigation, array data were processed in six different ways to arrive at estimates of gene expression among the 24 mice. The six methodologies are MAS5, gcRMA, RMA, VSN, and two versions of dChip  is the variance of X, and Var(Y|X = x) is the variance of Y in groups that have the same value of X. In our application, X is the measurements from qRT-PCR and Y is the measurements from array. X is fixed, and so also Var(X), but Y depends on what method is applied to the array data. Since Var(Y|X = x) appears in the denominator, a method's performance is improved if it minimizes Var(Y|X = x). Therefore, this metric tends to favor methods with smaller variability. Similarly, the larger the slope between Y and X, the larger r is and the more favorable a method's performance. In this sense, by using Pearson's correlation, we simultaneously take into account both the variance and bias of the measurements produced by arrays. That is, we seek methods that achieve the right balance between variance and bias to yield the strongest association between array measurements and qRT-PCR. However, it is also of interest to specifically examine variance and bias, and we will come back to this.We note that the 47 genes assayed with qRT-PCR were selected based on primer availability, initial evidence for differential expression, signal intensity, and biological interest. The 47 genes do not comprise a random sample. In particular, the genes for which we have qRT-PCR data do not include low-intensity genes . Genes were initially selected primarily for their biological interest, but then some of these candidates were excluded. Two elements of the selection process are notable. First, genes were selected if they appeared to be promising candidates for differential expression based on processing the data with gcRMA. This introduces a possible bias for gcRMA into our results. Second, genes with average signal intensity less than 2 were excluded. These factors resulted in the selection of primarily high- and medium-intensity genes. Previous work [We argue that correlation is a reasonable measure of agreement in this study because it accounts for both the bias and variance of measurements, favoring methods that find the right balance between the two. However, Figure Our results, narrowly interpreted, favor MAS5, gcRMA, and the dChip mismatch model. However, our assessment of variability, together with previous studies that demonstrate the unreliability of using MM data for low-intensity genes , leads tWe have treated the qRT-PCR measurements as "truth" because they are the gold standard laboratory measurement of gene expression. Yet qRT-PCR measurements are also subject to error. However, our sensitivity analysis should partly address this concern.We reiterate that we have compared the performance of array methodologies for estimating relative gene expression levels for a chosen list of genes. We have not compared methods on their abilities to identify differentially expressed genes, which is an important goal that is related but not identical. Still, it is useful to compare our findings with other validation studies, including those that used other criteria to evaluate methods. Of the three recent studies ,7,8, ourUsing qRT-PCR data as an independent measurement tool, we compared the performance of six methodologies for the quantification of gene expression from Affymetrix oligonucleotide arrays. Three methods – MAS5, gcRMA, and the dChip mismatch model – performed better than VSN, dChip without mismatch, and RMA. The factor driving these results was whether a method used mismatch data or, alternatively, a sequence-based background adjustment. Other differences among methods, such as the normalization scheme, made little difference in overall performance. Further analysis of variability lead us to favor the sequence-based background adjustment over procedures using mismatch probes. In summary, for estimating relative expression using oligonucleotide array data, we advocate (1) foregoing methods that use mismatch subtraction and (2) using the sequence-based background adjustment method in gcRMA.in vitro transcription using biotinylated ribonucleotides . Samples were hybridized to mgu74av2 arrays at the University of Washington Center for Expression Arrays according to recommended procedures .Total RNA was extracted from flash-frozen heart tissue using Trizol  extraction followed by cleanup with the RNeasy kit . Samples were prepared for Affymetrix arrays using 7 μg total RNA and following the manufacturer's instructions for One Cycle Eukaryotic Target Preparation  including first and second cDNA generations from oligo-dT and linear Samples were prepared for qRT-PCR using the cDNA archive protocol , which uses random hexamers for first strand cDNA synthesis. cDNA samples representing 50 ng total RNA and 2× Universal PCR master mix (Applied Biosystems) were loaded into each port of the 384 well Applied Biosystems Low Density Arrays, which were custom designed for this experiment, and run in an Applied Biosystems 7900 real time PCR instrument according to manufacturer's instructions. Technical and biological replicates were balanced across ports and cards for each of the groups to minimize any effect of loading port position or variability between cards. Each cDNA sample was run in duplicate on 2 different cards.qRT-PCR data were quantified using SDS 2.1 (Applied Biosystems). For the Low Density Arrays, baseline and threshold were identified automatically by the software and adjusted manually where necessary and the settings were applied to all arrays. Gene expression values were normalized to the 18s endogenous control and corrected for measured efficiency as calculated by a standard curve run on one of the Low Density Arrays .Genes were selected for qRT-PCR based on an analysis of the entire Affymetrix data set  processed using gcRMA. Chosen genes either exhibited a large average fold-change or highly statistically significant evidence of differential expression as determined by the LIMMA package of Bioconductor . Therefo2 >2) and magnitude of change (log2(ratio) >0.5) for each contrast. Some genes were chosen based an analysis of additional array data from pools of the RNA samples, choosing the 200 genes with the largest changes for the contrasts listed, and then filtering based on magnitude of signal and magnitude of change as above. 110 genes were selected by these methods. This number was then reduced based on availability of assays on the ABI low density arrays and availability of annotation to yield the 47 genes studied here. The 18s endogenous control is included on the Applied Biosystems Low Density Arrays by default and was used for normalization of the qRT-PCR data as described above.In detail, we generated gene lists that contained the top 200 candidates for each of the 6 contrasts we study. These gene lists were filtered based on magnitude of signal  and by qRT-PCR . For a given contrast, the data plotted on the horizontal axis are the same. See Figure We considered several metrics for quantifying agreement within a scatterplot. While it would be ideal if array measurements and qRT-PCR measurements agreed exactly, it is satisfactory for there to be a linear relationship. Therefore, we considered the most important characteristic of these plots to be the overall linear trend. This led us to use Pearson Correlation as a measure of agreement. AdditionalFigures.doc gives results for four other measures of agreement: mean squared error, median absolute error, Canberra distance, and Spearman correlation. However, findings using these four measures were inconclusive.Our measure of agreement, Pearson Correlation, is not robust as it can be disproportionately influenced by individual data points. For example, a gene with a large change for a certain contrast could heavily drive the results for that contrast, but this would be misleading. Therefore, we performed a sensitivity analysis to ensure that our findings were not driven by one or two genes. We systematically removed the data for (1) all single genes and (2) all pairs of genes from the processed datasets, and then re-computed the correlation of the scatterplot with the removed gene(s). We called a gene (or gene-pair) "influential" if the ranking of the six methods changed upon removal of the gene (or gene-pair). For the leave-two-out sensitivity analysis, we only considered gene pairs that were not influential singleton genes.This sensitivity analysis also takes the place of computing confidence intervals for the correlations we compute. Because the set of genes included in this study was not a random sample, such confidence intervals would not be valid.LXQ analyzed the qRT-PCR data and performed the overview analysis including the sensitivity analysis. RPB participated in all stages of design and analysis, provided expertise on Bioconductor, and reviewed all code. NJL and FNH conducted all of the RNA assays. NJL additionally provided biological expertise to the project. DEM conducted the exploratory aspect of the study and the follow-up analysis. KFK directed the project and wrote the manuscript. All authors read and approved the final manuscript.This is an MS Word document showing the full set of results for the initial analysis. The file includes figures corresponding to Figures Click here for fileThis excel file gives the results of the follow-up analysis. For each combination of sub-methods, the Pearson correlation is given for each contrast.Click here for fileThis MS Word document gives examples from our exploratory analysis seeking associations between probe-level data and agreement between array and qRT-PCR data.Click here for file


Numerous nonparametric approaches have been proposed in literature to detect differential gene expression in the setting of two user-defined groups. However, there is a lack of nonparametric procedures to analyze microarray data with multiple factors attributing to the gene expression. Furthermore, incorporating interaction effects in the analysis of microarray data has long been of great interest to biological scientists, little of which has been investigated in the nonparametric framework.In this paper, we propose a set of nonparametric tests to detect treatment effects, clinical covariate effects, and interaction effects for multifactorial microarray data. When the distribution of expression data is skewed or heavy-tailed, the rank tests are substantially more powerful than the competing parametric F tests. On the other hand, in the case of light or medium-tailed distributions, the rank tests appear to be marginally less powerful than the parametric competitors.The proposed rank tests enable us to detect differential gene expression and establish interaction effects for microarray data with various non-normally distributed expression measurements across genome. In the presence of outliers, they are advantageous alternative approaches to the existing parametric F tests due to the robustness feature. High density oligonucleotide microarray, spotted cDNA array, or other array technologies have presented not only daunting amount of expression data for biologists to explore the inherent biological mechanisms, but also challenging statistical analysis problems. A replicated microarray experiment involves multiple arrays to compare gene expression profile under different conditions. However, normality assumption justifying parametric testing is often untenable in microarray studies ,2. For iFurthermore data contamination can arise in microarray setting due to different reasons. For instance, an image contamination can occur if a long scratch is present on the array image or a corner of the array is misaligned in the image processing stage. Sample contamination can occur if the mRNA sample is contaminated with other sources of RNA present in the laboratory environment. Such outliers are dramatically different from the majority of the observations and can greatly undermine the sensitivity of parametric approach. A method of assessing goodness of fit to a linear model has been used to automatically detect outliers that possess too large deviation from the overall pattern . Alternat or t-like statistics and differ primarily in the estimation of variance ,..., ε[20] be the ordered noise. We gradually introduced heavy-tailedness into the data set by pulling the left and right end points of the ordered list of noise further away from the center. Each time a new data set was generated with  = εi] [* d, for i = 1, 2, 3, 18,19, 20, and d was chosen from the varying range of 1.1, 1.3,..., 3. The corresponding W and the p-values of the FT, PFT and MRT tests were recorded for the data set. The result was summarized based on 1000 replications. In Figure W. Our simulation demonstrates that as heavy-tailedness is introduced into the data set, W level decreases correspondingly. When W value is above 0.92, the two parametric methods outperform the nonparametric method. When W value is below 0.92, the nonparametric method is superior to the two parametric competitors. Thus for the specific design setting that we simulated, we choose a threshold value of τ = 0.92. If W <τ and W > W*, we would recommend the use of the nonparametric method. Among many sources of the normality violation discussed above, if τ = 0.92 was used as the cutoff, we found about 10% of genes in the data set of Sandberg et al. [For data that are really generated from normal distribution, the g et al.  whose exJ levels and another covariate with K levels can be combined as a single factor of JK levels, so that the treatment effects can still be tested using the above two-way layout. When the data contains continuous covariates in certain applications, one can simply apply the proposed rank test method on the basis of residuals, the differences between the observations and the least squares fitted values calculated by using all the continuous covariates. Furthermore, it is possible to extend our methods to accommodate the dependence or heteroscedasticity which might occur in the microarray data sets. If the variances vary across different covariate groups, j = 1,..., J, the MRT statistic can still be employed to test for treatment effects using the standardized overall rank Zijn [With regard to future extensions of the proposed methods, the tests discussed above can be applied to a high-way layout by collapsing these covariates into one. For example, a covariate with ank Zijn . To dealank Zijn .In this article, we have focused on the interaction effects between multiple attributing factors to the gene expression. Currently there has been an increasing interest in studying interactions between genes as opposed to clinical factors. To address this problem, we could select a number of genes and treat their expressions measurements as explanatory variables. The biological phenotype of interest can be chosen as the response variable. Then a linear model can be fitted linking the gene expressions and biological phenotype. The aforementioned interaction tests can be applied to this setting to investigate the possible interactions among the genes.We have presented a set of nonparametric tests to detect treatment effects, clinical covariate effects, and interaction effects for multifactorial microarray data. These methods can be extended to accommodate high-way layouts, continuous covariates, dependent observations and heteroscedasticity which might occur in the microarray data sets. The proposed nonparametric procedures will prove to be of wide use in microarray data analysis as they can accommodate various noise distributions across genome.H01 is formulated to test for treatment effects in two-way layout. Correspondingly, we have proposed a modified rank transform (MRT) test. This test standardizes the rank scores before plugging them into the analysis of variance formula. For simplicity in notation, we suppress the index k, as all the observations in the model are from a specific gene k. Let Rijn denote the rank of Xijn among all of the observations and define . Let  denote the sample variance of ranks within the jth column. Define the standardized rank score Zijn = Rijn/sj. Denote the marginal and overall averages of the standardized rank scores by  and . The proposed modified rank transform statistic takes the following form:The first hypothesis p-values of the observed statistic. In implementation, we randomly relabel I treatment groups within each of J covariate levels. Namely, the set of observations Xj11,..., XjN1,..., XIj1,..., XIjN are shuffled within column j for 1 ≤ j ≤ J. For illustration purpose, consider a microarray data set with the covariate consisting of six different tissue regions and the treatment consisting of two distinct mouse strains. The six covariate levels correspond to the six tissue regions. To generate a permuted data set, for the 2N measurements obtained from the same tissue region, we randomly assign N of them to the first mouse strain and assign the remaining observations to the second mouse strain. Repeat this procedure until we have permuted for all the tissue regions to generate a new permuted data set. Then we calculate the proportion of the resulting statistic (3) being equal to or larger than the observed statistic over 10,000 permutations to obtain the permutation p-value.It has been shown that the standardization procedure is essential for the validity of the MRT method as the nonlinear rank transformation introduces the heteroscedasticity into the ranked data . To asseH02 is formulated to test for interaction effects in two-way layout. To address this testing problem, the ART test is proposed to perform the analysis of variance test on the ranked residuals,  of the aligned observations .The second hypothesis  and  are the Hodges-Lehmann estimates of the two main effects given byHere p-values under the null H20. In implementation, we randomly relabel both indices i and j within all the aligned observations and obtain the empirical p-value over 10,000 permutations.Again, with low replicates, we propose to use the permutation method to compute H03 is formulated to test for main effects in the absence of interaction effects in a two-way layout. We propose to employ the rank transform statistic suggested in [The third hypothesis ested in  which isχ2 distribution with I – 1 degrees of freedom. Likewise, we can test if there is a difference of gene expression among the clinical covariate levels for gene k, using a test statistic similar to (4), with only indexes I and J being swapped.The resulting RT statistic asymptotically follows a XG and PS developed the methods and wrote the manuscript.


Previously, a total of five breast cancer subtypes have been identified based on variation in gene expression patterns. These expression profiles were also shown to be associated with different prognostic value. In this study tumour samples from 27 breast cancer patients, previously subtyped by expression analysis using DNA microarrays, and four controls from normal breast tissue were included. A new MetriGenix 4D™ array proposed for diagnostic use was evaluated.We applied MetriGenix custom 4D™ arrays for the detection of previously defined molecular subtypes of breast cancer. MetriGenix 4D™ arrays have special features including probe immobilization in microchannels with chemiluminescence detection that enable shorter hybridization time.The MetriGenix 4D™ array platform was evaluated with respect to both the accuracy in classifying the samples as well as the performance of the system itself. In a cross validation analysis using "Nearest Shrunken Centroid classifier" and the PAM software, 77% of the samples were classified correctly according to earlier classification results.The system shows potential for fast screening; however, improvements are needed. An array platform  was introduced in which molecular interactions occur within three-dimensional volumes of ordered microchannels rather than at a two-dimensional surface . The micfied RNA ,3. The pMolecular signatures based on gene expression patterns of breast carcinomas, specifying different subtypes of tumours, have been identified by the use of DNA microarrays -7. SpeciGene expression patterns from 269 genes selected to optimally define the different subtypes of breast cancer were analyzed in breast tumours and normal tissue from 31 individuals . The aim of this pilot study was to investigate whether this type of arrays would have the potential as a diagnostic tool for molecular classification in a clinical setting.Here we show that the MetriGenix 4D™ arrays with a limited number of carefully selected probes, perform similarly to other microarray platforms, although technical difficulties were experienced with the prototype system.In total, 27 breast tumour samples and 4 normal tissue samples were analyzed in this study. Among the tumour samples, 20 biopsy tissues from early breast carcinomas were included of which ten have been previously sub-classified as luminal subtypes, the other half as non-luminal subtypes using traditional two-dimensional microarray platforms such as Stanford cDNA arrays, Agilent Human Whole Genome Arrays and Applied Biosystems Human Genome Survey Microarrays . Further®  as described by the manufacturer. The RNA quality was evaluated by microcapillary electrophoresis using an Agilent 2100 Bioanalyzer  and concentration measured by using NanoDrop .Total RNA was extracted from fresh frozen tissue samples by using TRIzolWe selected 269 genes that best represented the classification scheme in breast cancer to be synthesized and immobilized on the MetriGenix-Chip . Double-stranded cDNA was purified in Phase-Lock Gel Tubes  and in vitro transcribed by Ambion's MEGAscript ™ T7 High Yield Transcription kit  followed by cleanup with RNeasy® RNA isolation kit columns (Qiagen). Amplified cRNA was evaluated on the Agilent 2100 Bioanalyzer (Agilent Technologies). The cRNA was biotin-labelled using MetriGenix Bio ULS  (0.5 Units/μl) in a one-step chemical coupling reaction at 85°C for 30 min .Total RNA was amplified using a two-step cRNA synthesis scheme typical for microarray experiments. First strand cDNA was synthesized by annealing T7-(T) 24- primer (100 pmol/μl) with 5 μg total RNA in a final volume of 12 μl at 70°C for 10 min, followed by addition of first-strand master mix  to a final volume of 20 μl. The reaction was incubated at 42°C for one hour. Second strand synthesis followed immediately by adding 5× Second Strand Buffer, 10 mM/each dNTPs mix, 10 U/μl Prior to hybridization the biotin-labelled cRNA were mixed with spike-in controls , Sample Dilution Buffer 2 (MetriGenix) and herring sperm DNA (Invitrogen) and denatured for 5 min at 90°C. The sample was then injected into the sample compartment of the 4D array, along with blocking and staining reagents into their respective compartments.Custom 4D arrays to monitor the genes of interest were supplied by MetriGenix . For each gene, a 50- to 60-mer probe was designed based on publicly available sequences and to have GC content in the range of 45 to 55 percent and a melting temperature between 64 and 68°C. For product quality control (QC) the following steps were performed: First, hybridization was performed with just the complements to the control probes to confirm that there was no cross-contamination of probes on the chip. Second, a test cRNA was hybridized to the chip in the absence of the control targets; since the controls were bacterial and the test cRNA mammalian, no hybridization was observed in the control probes (otherwise the chips failed QC). Third, control targets were added to every cRNA that was hybridized to the chip and the intensity of the spots was used qualitatively to confirm the hybridization results. Probes were synthesized with a 5' amino modification and printed on MetriGenix arrays using a Gene Machine Omnigrid arrayer. The arrays are housed in a 4D cartridge that includes reagent reservoirs and interfaces with the MGX2000 and MGX1200CL array processing stations.4D array hybridizations were performed on the MGX™ 2000 hybridization station, which controlled all subsequent steps . After four hours of hybridization , the chip was placed in the MGX 1200 CL detection unit for chemiluminescence (CL) detection with exposure times usually ranging from 2 to 5 s. Subsequent image analysis was performed with the MetriSoft software (MetriGenix) that generated an excel file containing the experiment data for subsequent analysis.The Metrisoft software operated on two different concepts, the noise floor and a stringent threshold value, to filter spots in the individual chip analysis. The noise floor was a value calculated by the software in each individual chip analysis that related to the amount of noise in the chip and which was subsequently used to determine the threshold value. The stringent threshold value was calculated as 3 times the noise floor, an empirical estimate of an 'absent' spot based on the image noise. Any signal below this value was not considered significant and assigned to the threshold value. For intra-chip normalization, the signal intensity of each individual spot was divided by the threshold to produce the normalized values within each chip. Data from 3 successful hybridized controls  were averaged for each gene to obtain a mean expression value. Next, to create log-transformed (base 2) pseudo ratios the value of each sample was divided by the mean of the three controls for every gene.Principal Component Analysis (PCA), hierarchical clustering and ANOVA were performed by using Avadis Prophetic software . Data were mean-centered, clustered using Euclidean distance measures and visualized using a heat map in which numeric values are represented in colour intensities . For ANOVA, samples were assigned class designations and gene expression data were analyzed assuming equal variance. Data were ranked based on p-values and F-statistics. In addition, a set of genes that best discriminated the two identified main subtypes of breast cancer were determined using "Nearest Shrunken Centroid classifier" and the PAM software . For thiThe aim of this study was to demonstrate whether these novel arrays would have the potential to be used in molecular classification in a diagnostic setting in a future implementation. The limited number of MetriGenix chips available in this study allowed us to successfully hybridize and analyze altogether 25 of 31 samples . Therefore, in the further evaluation with respect to the accuracy in classifying the samples we concentrated on the luminal  vs. non-luminal  groups instead of all five subtypes.ESR1, GATA3, XBP1, MUC1 while the basal-like tumours showed no expression of these genes. Further, the luminal cluster was divided into four sub-groups corresponding to the luminal A (dark blue), luminal B (light blue), the latter with less expression of luminal epithelial specific genes expressed, the previously identified ERBB2+ - (purple) and the normal-like group (green). Although there were few samples analyzed in this study, those representing each subtype grouped together.To evaluate the data generated by the MetriGenix analyses, an ANOVA analysis was performed excluding the control samples to identify the genes that best separated the samples into two groups; luminal and non-luminal see . Genes wTo test whether the clustering pattern could be visualized by using data from all the 226 "intrinsic" only genes printed on the 4D array, a hierarchical clustering was performed see . As prevTo further evaluate the dataset, prediction of tumour subtypes was performed using the PAM software and the results are shown in figure Principal component analysis (PCA), based on all genes on the 4D array, confirmed the groupings of samples seen in the hierarchical clustering analysis figure . Again, The sample preparation procedure including two-step cRNA synthesis and hybridization/detection was accomplished in three days. An advantage of the system was the short hybridization time of four hours; however, the multiple-day sample preparation protocol makes the overall analysis not much shorter than other microarray platforms. The system throughput was limited to a single chip per hybridization unit and the handling and purification was difficult due to small exterior buffer reservoirs and small buffer tubes, leading to contamination concerns. Moreover, it was difficult to trace errors, both on the hardware system and the Metrisoft software, as the system lacked an error message function. For more extensive applications, the image analysis software will require significant refinement. For example, artefacts such as pinspots and high background on the sample images occurred that led to the rejection of several arrays from further analysis. In addition, when analysing the images in the Metrisoft software, we consistently observed that the spot-finder did not find all of the spots on the image and manually re-reading of the image was necessary.We conclude that by selecting only the data from the well performed arrays (25/31) and by using key signature genes for breast-cancer subtypes immobilized on the MetriGenix 4D™ array, we were able to classify samples into the same subtypes as they were previously classified by using other microarray techniques, with a relatively high probability (23% misclassification). With higher throughput and improved performance this microarray platform has the potential to be a valuable tool for rapid routine gene expression profiling.DB and AS are previous employees of MetriGenix. MetriGenix is a private company that manufactured the arrays employed in this study, which was performed on a collaborative basis. All other authors declare that they have no competing interests.AM was involved in laboratory experiments, analyzing the data and drafted the manuscript. HJ was involved in laboratory experiments. DB carried out the statistical computation of the data. AS were involved in data analysis. PEL and BN were involved in providing breast cancer samples. TS and ALBD initiated and designed the study and were involved in writing the manuscript.The pre-publication history for this paper can be accessed here:Genes-on-chip. A listing of the genes immobilized on the 4D array along with sequence information of each probe.Click here for fileANOVA-analysis output. ANOVA analysis of all the genes immobilized on the MetriGenix breast cancer chip.Click here for fileHierarchical clustering of "intrinsic" only genes. Hierarchical clustering diagram of the 226 "intrinsic" only gene subset immobilized on the MetriGenix chip. Coloured branches represent the different subtypes as previously determined using a different DNA microarray platform [platform ; Dark blClick here for filePAM-analysis output. PAM output of the most differentially expressed genes between the two groups: luminal and non-luminal.Click here for file


Molecular profiling generates abundance measurements for thousands of gene transcripts in biological samples such as normal and tumor tissues (data points). Given such two-class high-dimensional data, many methods have been proposed for classifying data points into one of the two classes. However, finding very small sets of features able to correctly classify the data is problematic as the fundamental mathematical proposition is hard. Existing methods can find "small" feature sets, but give no hint how close this is to the true minimum size. Without fundamental mathematical advances, finding true minimum-size sets will remain elusive, and more importantly for the microarray community there will be no methods for finding them.We use the brute force approach of exhaustive search through all genes, gene pairs (and for some data sets gene triples). Each unique gene combination is analyzed with a few-parameter linear-hyperplane classification method looking for those combinations that form training error-free classifiers. All 10 published data sets studied are found to contain predictive small feature sets. Four contain thousands of gene pairs and 6 have single genes that perfectly discriminate.et al. The key component of these is the "placental gene of unknown function", PLAC8. Our HMM modeling indicates PLAC8 might have a domain like part of lP59's crystal structure . The previously identified HCC biomarker gene, glypican 3 (GPC3), is part of an accurate gene triple involving MT1E and ARHE. We also find small gene sets that distinguish leukemia subtypes in the large pediatric acute lymphoblastic leukemia cancer set of Yeoh et al.This technique discovered small sets of genes (3 or less) in published data that form accurate classifiers, yet were not reported in the prior publications. This could be a common characteristic of microarray data, thus making looking for them worth the computational cost. Such small gene sets could indicate biomarkers and portend simple medical diagnostic tests. We recommend checking for small gene sets routinely. We find 4 gene pairs and many gene triples in the large hepatocellular carcinoma  data set of Chen  Transcriptional profiling studies can produce data in the form of abundance measurements for genes in samples assigned to one of two classes. A recent exemplar employed cDNA microarrays to assay 6605 clones from normal liver and liver cancer (hepatocellular carcinoma) tissues . Given si.e., one which makes zero (or few) errors in assigning the class of an unseen data point. Frequently, application of these approaches to a data set results in the definition of one discriminatory subset with tens to hundreds of features and requiring similar numbers of free parameters. This work focuses on subsets smaller than those produced by existing algorithms: all subsets of one-, two-, (and sometimes three-) features that can be separated by a linear surface without error. A multiplicity of error-free linear classifiers constructed from few features could facilitate the creation of cost-effective clinical tests and guide further basic research.Existing classification and feature selection techniques can be employed to ascertain the cardinality of a feature subset yielding a classifier that generalizes well, m-feature classifier is defined as a decision surface for m-dimensional data points where the m features are a subset of P a priori features, m ≪ P. The potential number of these classifiers is equivalent to choosing m items out P, i.e., . This number increases when different types of decision boundaries are permissible for each value of m. The scope of the problem can be reduced and simplified if only m-feature linear classifiers (m-LCs) are considered. This restriction of neglecting non-linear decision surfaces is reasonable because hyperplanes can be calculated efficiently, and Support Vector Machines with linear kernels are sufficient for classification problems associated with profiling data . The high dimensionality of transcriptional profiles and the logistical issues associated with exhaustive enumeration of all 1-, 2- and 3-LCs have lead to the prevailing assumption that such searches are both too expensive and unlikely to be informative. Thus, while some recent studies have made use of Kim's method [Here, an  example -6). Rece example  and Kim  example  demonstrdata set . This sos method -12, mostm + 1, hence is very small relative to the size of the data, greatly reducing the problem of over-fitting to the training data (see Random Data section). It seems plausible that the existence of single genes and gene pairs with the ability to form perfect linear classifiers may be a widespread phenomenon. To demonstrate the biological utility of the strategy, the gene pairs and triples discovered in the aforementioned LiverCancer data set were examined and found to yield new and unanticipated scientific insights. Overall, the results indicate the importance of ascertaining, as a matter of routine, the presence (or absence) of small distinguishing feature subsets.Here, a relatively inexpensive method for calculating maximal margin hyperplanes, LIKNON ,13, is uAll results are available through the web site .n2 pairs (a triangular matrix). So a 2000 gene, 30 sample set would need about  seconds.The 10 published cancer data sets examined here are listed in Table As expected, small data sets are found to have many thousands of gene pairs while large data sets have few. The gene sets discovered in the two large data sets (LiverCancer and YeohALL) are likely biologically relevant and are discussed later. The thousands of pairs found on small data sets are mostly due to the small sample size of the data, and would likely not maintain their perfect classification upon addition of further patient samples. Generalization performance was estimated with a LOO methodology (see Methods) and is often reasonably good.However, even gene sets found in small data sets can be interesting depending on the end use. For use in a medical diagnostic it is desirable that the gene set be highly accurate on large number of patient samples so the test result error rate is low. If used to guide basic research, even a poor error rate could indicate a productive research direction.The results are easily visualized for single and pairs of genes by a scatter plot. For each sample the expression values for the gene(s) set the x-y position and the point is labeled with the sample class as shown in the example plot Figure Many of the genes sets found accurately classify the data with large separation between the classes. It is exciting to consider the possibilities for medical diagnostics if some small gene set is found to accurately and reproducibly indicate a disease state. While general machine learning principles suggest that having more features (genes in this case) is desirable in order to make more noise resistant classifiers, this is data dependent and gene pairs with a training error rate of 1/90 as found in the LiverCancer data could be perfectly acceptable. Small gene sets, even if not accurate enough for medical purposes, can indicate fruitful new research directions. Like other classification methods that produce large numbers of genes, considering the corpus of all genes found can provide insight to the underlying biology.Experimental design and construction of a data set profoundly influences the presence of small sized classifiers. For example, the BRCA sets are labeled according to their known BRCA1/BRCA2 mutation status. If some of the measured genes reflect this mutation status we would apriori expect to find some (possibly many) small feature sets, and not finding any could indicate errors in the class labels (a sample is mislabeled).In data where the class sizes are small, most of the small gene sets will prove sensitive to noise. Thus they are not likely to perform well as predictors on new samples, or in different experiment setups. Classifiers made from large data sets are more likely to be reproducible and perform well in other situations. Classifiers with large margin and zero LOO error are more likely to indicate real biological effects that would hold true on new patient samples.The GIST and BreastBRCA experiments are examples of both of the above conditions, and both lead to very large numbers of pairs. Both have small class sizes and have pre-disposed differences between the classes. The GIST experiment compares cancers from different tissue types which means there will be a very strong signal from just the tissue differences rather than just the cancers alone. The BreastBRCA experiment has the pre-disposition of being split along BRCA status lines. In both cases the number of patient samples and class sizes is small and 100,000+ pairs are found. We suspect there are some biologically real pairs hidden in the large background noise due to small sample sizes. Random data tests (see Methods) indicate that 30+ samples with more even class sizes are needed in order to reduce the random chance noise to a very small level Table .Gene sets that perform well across independent experimental data sets also likely indicate real biological effects. However, cross-experiment array comparisons are difficult and would be much easier and more broad if experimenters used more common clones and references.Is the cost of such computation worth it? Certainly it is for single genes and pairs. Modern computers are powerful enough to solve these size problems in a few minutes. Triples needs tens to hundreds of hours on a single computer, which is still tractable. Quadruples are beyond single computer tractability. However, this type of algorithm is trivially parallelized over a standard network of computers leading to linear speed up. Each computer would be instructed to examine a given part of the search space and thereafter be independent from all the rest. A super computer or dedicated computer cluster is not required. It seems possible that the occurrence of such small sized classifiers is a common characteristic of microarray data, thus making the effort of searching for small gene sets worth the computational cost. The technique is not restricted to RNA/DNA transcript microarray data as used here. It can be applied to protein microarray, mass-spectra, or any data with similar characteristics.We can't discuss all the gene sets found in all data sets: there are too many. Here we discuss results from the 2 large data sets that we think produce highly biologically relevant results. The supplement see  containslog2 values for each gene) reduced the number of genes examined from 6605 down to 1956, which yielded 43 gene triples  and 9496 gene triples (with the 2 "outlier" genes relabeled). Only 291 of these triples were saved for further analysis. All 4 gene pairs, 35 of the 43 and 229 of the 291 triples include the recently annotated gene PLAC8 (IMAGE:491644) a "placenta specific gene of unknown function".The large liver cancer data set contains 2 classes  with 181 patient samples and measurements for 6605 genes. We examined this data set in more detail than the others as it is large and any results found are likely to be biologically relevant. The original data  (this isThe 4 genes in pairs with PLAC8 are• IMAGE:669379 GLCCI1 glucocorticoid induced transcript 1• IMAGE:590591 ADCY6 adenylate cyclase 6• IMAGE:260259 Transcribed sequence with moderate similarity to protein sp:P39188 (H.sapiens)• IMAGE:756490 BCAT2 branched chain aminotransferase 2, mitochondrialFigure • IMAGE:1472735 MT1E metallothionein 1E  Hs74170• IMAGE:784593 ARHE ras homolog gene family, member E Hs6838• IMAGE:878564 GPC3 glypican 3 Hs119651and is shown in Figure • IMAGE:78353 RNAHP RNA helicase-related protein Hs8765• IMAGE:491644 PLAC8• IMAGE:667883 PHLDA1 pleckstrin homology-like domain, family A, member 1 Hs82101In all, the 291 triples make use of 168 of the genes. The fact that triples exist when using the original labelings argues that the "outlier" examples are correctly labeled as tumor, albeit maybe a different type of tumor (or in a different development stage). That these small gene sets exist in such a large data set argues that they are biologically relevant. These 4 gene pairs only make 2 errors out of the 181 patient samples (the two "outlier" samples are the errors), which is an error rate of 2/181 = l/90. The triples found with the original labelings make no errors. Based on such data, one can imagine a simple few-gene diagnostic test based on these pairs and triples.It is perhaps not surprising that a placental gene is associated with liver cancer. Both are blood organs, and cancers often recapitulate early development stages, of which the fast growing placenta might be an example. In addition, mitochondria related genes have been associated with cancer progression.during the experiments would allow such samples to be identified in a timely manner for further investigation.For this data, normal LIKNON was a useful aid in identifying outlying data examples. The two outlying tumor samples in this data set could represent a rare tumor type or development state. Using such aids bacillus stearothermophilus. The PFAM model locates to the C terminus of the 1P59 crystal structure. The sequence of 1P59 is some 85 amino acids longer than PLAC8, and the alignment hit aligns the last 110 amino acids or so. Visualizing the 3D structure of 1P59 with the alignment hit in PLAC8 colored silver in the RASMOL tool . For each of the 6 classes we compare each class against all others combined, thus asking the question can each class be distinguished from all the others. There are 248 patient samples and the 6 classes are T, E2A, BCR, TEL, MLL and Hyperdiploid > 50. Both the T vs the rest and E2A vs the rest splittings have single genes that perfectly separate T or E2A from the others. Five out of the 6 splittings have gene pairs, only Hyperdiploid vs the rest does not have any small gene sets. We applied a variation filter to reduce the original 12625 Affymetrix probes down to 4196 genes (the filter level used requires a variation of at least 10000 within each gene). We only discuss the top results for the T and E2A splittings here, see the supplement see  for moreThe only single gene separating T vs the rest is the same one identified by Yeoh, "38319_at CD3D antigen, delta polypeptide (TiT3 complex) Hs95327". This gene has a high value in T and a low value otherwise Figure . In T, HThe 4 single probe sets for E2A are• "33355_at PBX1 pre-B-cell leukemia transcription factor 1 Hs408222"• "1287_at ADPRT ADP-ribosyltransferase  polymerase) Hs177766"• "430_at NP nucleoside phosphorylase Hs75514"• "32063_at PBX1 pre-B-cell leukemia transcription factor 1 Hs408222" All of these are high in E2A and lower in the rest. Probe 33355_at for PBX1 is shown in Figure There are 386 pairs, the best pair is "35125_at RPS6 ribosomal protein S6 Hs408073" and "35974_at LRMP lymphoid-restricted membrane protein Hs124922". Figure Small sets of genes  able to accurately classify two-class microarray data occur in many real-world data sets. These small sets could portend simple medical diagnostics and point to important research targets. The many small sized gene sets found here were not noted previously in the literature, and seemingly went unnoticed. Many members of the pairs discovered here have known associations with cancer and indicate the possibility of simple, accurate medical diagnostic tests based on such results.Exhaustively examining all pairs in thousand gene size datasets is easily tractable on modern computer hardware. All triples is harder, needing a few days, but this is only computer time, and powerful computers are cheap and plentiful. Given that the compute time is small enough and the results possibly important, we conclude that examining microarray data for single genes, gene pairs and maybe gene triples should be done routinely. When performed along with acquiring the biological data, the results can be used as a quality check on the experimental process.et al.The gene of unknown function, PLAC8 appears to have a role in Liver cancer, and based on our HMM modeling might have a domain similar to part of the crystal structure of 1P59. We find that there are 4 genes that when paired with PLAC8 form a classifier with the low error rate of 1/90. In addition there are many gene triples, often including PLAC8, that form zero error classifiers and might be good biomarkers. We find the previously identified HCC biomarker gene, glypican 3 (GPC3), is part of an accurate gene triple involving MT1E and ARHE. We also find small gene sets able to accurately distinguish leukemia subtypes in the large pediatric acute lymphoblastic leukemia cancer set of Yeoh log transformed; Affymetrix data were used as is. Only the LiverCancer data set was subjected to advanced re-normalization procedures. The one data set with a few missing values (LungStanford) had them set to the appropriate class average value. If there were too many genes , a variation filter was used to remove genes that didn't vary enough across all samples. The filterings (if used) were applied once, before LIKNON analysis, solely to reduce the number of genes and hence the runtimes. They were not used to adjust for "good" results. When three or more categories of samples had been defined by the original authors, two-class data sets were produced by partitioning these categories. Table m-LCs.The existing transcriptional profiling data sets investigated here are summarized in Table N data points, . Each data point is a P-dimensional vector of features, xn ∈ ℝP, assigned to one of two classes, yn ∈ {+1, -1}. Assume that the classes are linearly separable. A classifier for such data is a hyperplane parameterized by a weight vector, w ∈ ℝp, and an offset from the origin, b ∈ ℝ. A hyperplane, , can be used to predict the class of a data point x ∈ ℝP by computing sign(wTx + b). If this value is positive, x is identified with the +1 class, otherwise it belongs to the -1 class. Data points that define the hyperplane, positive half-space (+1 class), and negative half-space (-1 class) are the sets {x|wTx = b}, {x|wTx >b}, and {x|wTx <b} respectively. For two-class profiling data, an error free m-LC is a maximal margin hyperplane based on m of these P features, which assigns the class of every data point correctly. The number of free parameters in such models is m + 1, so in this setting is quite low (2–4), which is much smaller than the number of samples . By optimizing of the choice of m this might be over fitting the data, but in the Random Data section we show that we are finding many more features than chance alone would account for.Consider a two-class data set composed of P features, the potential number of m-LCs is equivalent to the combinatorial problem of choosing m items out of P, i.e., . In this work all single genes (m = 1) and pairs (m = 2) and in some cases triples (m = 3) were evaluated using a linear sparse hyperplane method that has been described elsewhere (LIKNON) [Given data points specified by (LIKNON) . LIKNON enes m =  and pairThis work accepts only perfect (no training error) linear hyperplane classifiers from the above method. This criteria for accepting only perfect classification is very stringent. We first thought that this would lead to few result sets being found, and that allowing non-perfect classification would need be done to find more classifiers. But this loosening turned out not to be necessary.These models have only 2–4 free parameters. Thus the problem of over-fitting the model to the data during training is not a large issue and we don't perform any stringent generalization tests such as multi-round cross validation. Others interested in evaluating particular gene sets should perform such tests.m-LC's where m is small, LOO error is not a good performance measure. In general, only somewhat isolated points lying close to the classifier decision boundary will be found in error during LOO testing. Thus LOO error rates tend to be very low, 0 – 3 out of the total number of points. The small magnitude of this error count allows one to be misled thinking the classifier is "good because the LOO error is low". The LOO error is included in the results available from the web site.Generalization performance of classifiers was evaluated using a Leave-One-Out procedure. In LOO testing, one data point is removed and a classifier re-learned using the rest of the data. The resultant classifier is tested on the held out data point and if it is in error, a LOO error is counted. However for these In the end, classifiers with a larger separation (margin) between the classes are able to tolerate more noise without errors. Thus larger margin classifiers are more desirable for use on future data points.n2/2 , and all triples about n3/6. A data set with P = 3000 features necessitates the evaluation of 4.5 × 106 pairs and 4.5 × 109 triples. An approximate average time is 1 second/106 pairs/sample. Thus, evaluating all pairs for these data sizes requires only a few minutes whereas all triples needs many hours.The program is an adaptation of the lp_solve version of LIKNON  implemenN items with a in one class, being linearly separable in 1 dimension is  and in 2 dimensions is approximately . Experiments were performed where a data matrix containing random numbers from a uniform distribution was analyzed by LIKNON. The results are shown in Table How many such pairs occur by chance? Theory suggests  that theThe web site  containsLG carried out all work outlined in this article. LG wrote and approved the manuscript.The expanded discussion section. A discussion of the top results for all data sets.Click here for file


The Stanford Tissue Microarray Database (TMAD) is a repository of data serving a consortium of pathologists and biomedical researchers. The tissue samples in TMAD are annotated with multiple free-text fields, specifying the pathological diagnoses for each sample. These text annotations are not structured according to any ontology, making future integration of this resource with other biological and clinical data difficult.We developed methods to map these annotations to the NCI thesaurus. Using the NCI-T we can effectively represent annotations for about 86% of the samples. We demonstrate how this mapping enables ontology driven integration and querying of tissue microarray data. We have deployed the mapping and ontology driven querying tools at the TMAD site for general use.We have demonstrated that we can effectively map the diagnosis-related terms describing a sample in TMAD to the NCI-T. The NCI thesaurus terms have a wide coverage and provide terms for about 86% of the samples. In our opinion the NCI thesaurus can facilitate integration of this resource with other biological data. Tissue Microarrays allow for the immunohistochemical analysis of large numbers of tissue samples and are used for confirmation of microarray gene-expression results as well as for predictive pathology . A singlSuperficially the datasets generated from TMA and gene expression arrays appear similar in that both are matrix type data and each entry in the matrix provides information about the expression of a biological entity (gene or protein) in a particular sample. However, gene expression arrays query a large number of genes in one sample or patient, whereas Tissue microarrays query a large number of samples/patients for one protein. The key query dimension in TMA data is a tissue sample, rather than a gene. As a result, queries such as 'find all genes that have a function X' get morphed to a query such as 'find all tissue samples that have a particular diagnosis'. Currently, because of the lack of a commonly used ontology to describe the diagnosis or disease state for a given TMA sample in TMAD – analogous to the Gene Ontology for the function of gene products – it is not possible to perform such as query. The lack of an ontology for diagnoses and disease states also limits future integration of this resource with other public genomic scale data, such as gene expression arrays ,3.For the purpose of this project, the key challenge was to create consistent ontology/terminology labels for each sample/record in the TMAD that would allow the identification of all samples that are of the same type at a given level of granularity. . One mechanism of achieving this objective is to map the text-annotations describing the diagnosis of a particular sample to ontology terms that allow us to formulate refined or coarse search criteria .In the current work, we map the text annotations for records in the TMAD to terms from the NCI thesaurus, present the results on the quality of the mapping effort, describe the implementation of the mapping tools on the TMAD website and explain how the mapping enables better querying of the data in TMAD. To the best of our knowledge this is the first work that describes the use of the NCI thesaurus for this purpose.The Tissue Microarray Database (TMAD) contains data from immunohistochemical and in situ hybridization analysis of a large number of tissue samples that were studied with tissue microarrays. The TMAD provides tools for quick upload, storage and retrieval of the TMA images and the analysis of immunohistochemical staining results . Each reMost of the samples in TMAD are cancerous tissue samples and therefore we needed ontologies that provide a broad coverage of the various cancers. The NCI Thesaurus was thus a natural candidate to consider for annotating the samples in TMAD. In previous work, we also included the SNOMED-CT for consideration. However. We downloaded version Thesaurus-05.09 g in the tab delimited text format. This text file contains columns for an id, name, parents, synonyms and definition for each NCI thesaurus term. The parents and synonyms columns contain the immediate parents of the term and its synonyms separated by '|' respectively.The NCI Thesaurus is an ontology providing broad coverage of the cancer domain, including cancer-related diseases, findings and abnormalities . In certHowever, this format is not optimal for searching parent-child terms rapidly as well as for the purpose of matching against individual synonyms. Therefore we reorganized the thesaurus into three separate tables which we named nci_term, nci_children and nci_synonyms. The nci_term table contains the Thesaurus-05.09 g as is. The nci_children table contains one row for each direct parent-child relationship and the nci_synonyms table contains one row each for each synonym of a term. This reorganization allows us to rapidly search the synonyms of a term as well as identify the parents and child terms of a given term. The tab-delimited files corresponding to these tables as well as the scripts to perform this reorganization are available on request.In order to map existing annotations of samples in the TMA database to ontology terms during the test phase , we creaEach record in the TMA database has an organ, a diagnosis and four sub diagnosis terms associated with it. For example, a record might contain the entries breast, carcinoma, ductal, <null>, in situ, <null> for the organ (O), diagnosis (d0), sub diagnosis 1 (d1), sub diagnosis 2 (d2), sub diagnosis 3 (d3) and sub diagnosis 4 (d4) fields respectively. We refer to the diagnosis related entries  as a term-set.We generated all possible permutations of the non-null entries in every term-set. In theory, we would generate over a million permutations for the 10734 records contained in the TMAD at the time of this writing. However, these 10734 records correspond to 1045 unique term-sets, many of which contain fewer than five entries, and in practice we ended up with about twenty thousand permutations. We used the pair of a particular term-set permutation and the associated organ in the search for an ontology term to associate with that TMA recordWhen searching the NCI thesaurus, instead of searching for an ontology term for each permutation-organ pair, we first filtered out the non-informative permutations using heuristics that identify such permutations. For example, there are a number of uninformative records where d0 = normal and d1 = 19w which we identify using regular expressions. We also "tweak" a permutation to convert it to the most useful form. For example, if the first word in a permutation is carcinoma and the second word is adeno then flip the order and merge the two words to make the first word to be adenocarcinoma. In another example, if the first word in a permutation is carcinoma and the second word is squamous then flip the order but keep the words separate.We use such a processed permutation and search for an exact match with a term or a synonym in the NCI thesaurus. If there is no match, then we drop the right most term (similar to right truncation in the UMLSKSS search) and repeat the search. When, during this process, the permutation contains two words or fewer, we add ' of <organ>' to the search string where <organ> is obtained from TMAD.We also use simple heuristics to weed out bad matches. For example, if a matched term contains uterine but the organ associated with the record is not ovary, uterus, or fallopian tube, then we discard the match. If the matched term contains mouse, then we discard the match.After initial success reported at AMIA Annual Symposium in 2006, we examined the unmatched records and their corresponding term-sets in collaboration with Inigo Espinosa (pathologist) and Kelly Montgomery (research staff member) from TMAD to refine our heuristics. For example, if the permutation contains 'sertoli leydig' convert it to Sertoli-Leydig. We also added exceptions to our heuristics such as: if the permutation contains two words or fewer and one of the words is 'cholangiocarcinoma' do not add the 'of <organ>' to the search. Such consultations, though time-consuming, turned out to be crucial in achieving high match accuracy.The setup, mapping to NCI terms and browsing are all carried out using modular perl scripts that are used as shown in Figure We mapped the term-sets corresponding to the 10734 records from the TMAD, which specified a diagnosis, to ontology terms in the NCI thesaurus. Out of the 1045 distinct term-sets (corresponding to the 10734 samples) we were able to match 902 term-sets to the NCI thesaurus. As mentioned before, in our initial work we had also mapped the term-sets to SNOMED-CT terms. TMAD would need a license to use the SNOMED-CT in their regular setup because they have non-US based users; therefore, while deploying the final system on TMAD we restrict ourselves to the NCI thesaurus only. In total we were able to map 9283 (86%) of records in TMAD to one or more terms from NCI-T.The NCI thesaurus has a wide coverage in providing matching terms for associating with diagnosis term-sets from the TMAD. Term-sets for a total of 9283 records could be mapped to one or more terms from the NCI thesaurus. The type (and the granularity level) of the ontology terms that matched a given term-set varied over a wide range. For example, there are records where the term-set contained just four characters, such as MMMT (of ovary) and matched a very specific ontology term such as 'Malignant_Mixed_Mesodermal_Mullerian_Tumor'. At the same time, there are records where the term-set was highly descriptive such as carcinoma adeno intraductal (of prostate) which matched an ontology term such as 'Prostate_Ductal_Adenocarcinoma' and records where the term-set was highly descriptive such as carcinoma transitional cell in situ (of bladder) which matched two ontology terms – 'Stage_0_Transitional_Cell_Carcinoma' and 'Bladder_Carcinoma' because no single term existed that would capture all the information.sarcoma clear cell (of soft tissue) – which matches 'Sarcoma of the soft tissue and bone' – and mucinous neoplasm papillary intraductal (of pancreas) – which matches 'Mucinous Neoplasm' – to be appropriate hits in this evaluation. We repeated this procedure 3 times. The results of this exercise are presented in Table In our work, we mapped 9283 records from TMAD to one or more ontology terms; it is extremely time consuming for a domain expert to evaluate every matched term manually. Therefore, we devised a sampling strategy where from the records that had a match, we randomly selected 50 rows comprising a distinct term-set, the associated organ, and the matched ontology term to determine the percentage of matches that were appropriate (true positive) or inappropriate  on manual inspection. We do not consider high-level matches such as For 1451 rows, corresponding to 143 term-sets (13.3%), we did not have a match with any NCI term. Potentially all of these are false negatives putting a lower bound on the recall to be 86.7%. We adopted a similar sampling strategy to select 3 sets of 50 records where no match was found. A domain expert then examined the NCI thesaurus attempting to find a match for those records. The results are presented in Table carcinoma conventional granular pauci cellular area (of kidney) where a match exists in the NCI thesaurus as conventional renal cell carcinoma or clear cell renal cell carcinoma This fraction of the unmatched records are false negatives, giving us an estimated false negative rate of about 4.52 % and an estimated recall of 95%.We were manually able to find a match for 34% of those records, meaning that for these records, a true match did exist in the NCI thesaurus that our methods were unable to find. Examples of such record is proximal area history of lymphoma (of Colon), mesial schelorsis epilepsy (of brain) and no cancer in prostate. It is not surprising that these term-sets did not match anything in the NCI thesaurus because these are not cancer related terms. In some cases the term-sets were contradictory such as the diagnosis being leiomyosarcoma but the organ specified as skeletal muscle, instead of smooth muscle. There are several records that have words like 10w as the diagnosis, these are skipped during matching and considered as true negatives.For the remaining records, no good match exists and they are true negatives. Among these true negatives, many term-sets did not describe any cancer and consisted of words such as We note that Metamap is considered the gold standard in mapping medical text to UMLS concepts -10. HoweOnce the mapping is accomplished, the simple assignment of the ontology terms to tissue samples is still not immediately useful to the end-user unless these ontology terms are used to drive specialized query-interfaces. Plain keyword-based querying of ontology terms is not very useful. Therefore, we are developing a querying and browsing interface , and can click on the retroperitoneal neoplasm term to find that these are adrenal gland neoplasms and that four of them are from the medulla, and that eight are from the cortex of which four are malignant  to pose requests such as: 1) Show all retroperitoneal tumors 2) How many skin neoplasms does TMAD have? 3) What subtypes of malignant abdominal neoplasms does TMAD have? These queries could not be posed with the standard keyword based search interface.Currently there is a lot of effort directed towards classification of tumors using gene expression data. There aintegrate the results from these two complementary data types. Recent reviews have suggested that it is essential to address this issue and synchronize the analysis, interpretation and data standards for these data [DNA microarrays and tissue microarrays query two orthogonal dimensions of the data. The former query the expression of many genes for one patient in one experiment whereas the latter query many patients for changes in expression of one protein in a single experiment. The results from such studies have to be integrated with each other and with traditional histopathological diagnosis methods in order to form testable hypotheses about the underlying molecular mechanisms which determine prognosis of a particular cancer. However, little attention is being paid to the problem of developing mechanisms to ese data samples in tissue and gene expression databases to perform integrative analyses  and/or the Gene Expression Omnibus (GEO). If the microarray samples are similarly annotated with the diseases and/or tissue they are derived from, that will enable integrated analysis of protein and mRNA expression datasets. In follow up efforts to the current work, we have applied our methods to text annotations of GEO and are able to identify disease related experiments with high precision and recall as well as identify candidate datasets for integrated analysis of protein and mRNA expression . We noteThis also presents an exciting research opportunity for specific cancer types, such as lung cancer, where integration of these data is needed for better tumor profiling . Such inWe have demonstrated that we can effectively map the diagnosis-related terms describing a sample in TMAD to the NCI-T. The NCI thesaurus terms have a wide coverage and provide terms for about 86% of the samples. We deployed a set of mapping scripts at TMAD that will update the ontology annotations as new records are entered, ensuring that the structured annotations using NCIT terms are always up to date. We have developed and deployed a graphical interface to browse the sample collection in TMAD using the NCI thesaurus terms and their hierarchical organization. We described how such a mapping allows a rich querying facility and offers the ability to identify "similar" or "related" tissue microarray samples, even though they may be described by different terms. For example, the four neoplasms of the adrenal medulla and eight neoplasms of the adrenal cortex  are all related to each other by the fact that they are all retroperitoneal neoplasms. Finally, we note that such ontology based annotation can enable better data integration across diverse repositories similar to how GO has enabled such integration across model organism databases.Source code for both, the mapping scripts and the browsing interface is available as additional files [see Additional files Operating system(s): Windows XP, Linux;Programming language: PERL;Other requirements: PERL 5.6 or higher, Graphviz 1.2 or higher;License: GNU GPL;Any restrictions for non-academics: none.NS conceived of the project, developed the mapping method and wrote the code. DL, IE, KM participated in the evaluation of the method. IE, KM helped refine the match heuristics. All authors have read and approve of the manuscript.Mapping Code. This zipped file contains the Perl scripts to perform the mapping of the TMA annotations as described in the methods section.Click here for fileBrowsing Code. This zipped file contains the Perl scripts to create the browsing interface shown in Figure Click here for file