Microarray technologies have both fascinated and frustrated the transplant community since their introduction roughly a decade ago. Fascination arose from the possibility offered by the technology to gain a profound insight into the cellular response to immunogenic injury and the potential that this genomic signature would be indicative of the biological mechanism by which that stress was induced. Frustrations have arisen primarily from technical factors such as data variance, the requirement for the application of advanced statistical and mathematical analyses, and difficulties associated with actually recognizing signature gene-expression patterns and discerning mechanisms. To aid the understanding of this powerful tool, its versatility, and how it is dramatically changing the molecular approach to biomedical and clinical research, this teaching review describes the technology and its applications, as well as the limitations and evolution of microarrays, in the field of organ transplantation. Finally, it calls upon the attention of the transplant community to integrate into multidisciplinary teams, to take advantage of this technology and its expanding applications in unraveling the complex injury circuits that currently limit transplant survival. Microarrays, the basicsEvolution of microarray technology over timeThe importance of microarrays in human biologyMicroarray-based insights for the transplant physicianCurrent unmet biological questions in transplantation and how can we use microarrays to address themMethods of applying microarrays to clinical practiceLimitations of using microarrays in clinical practiceThe black-box of microarray data analysisThe purpose of this article was to review:The completion of the human genome project has revolutionized translational medicine. High-throughput technologies, mapped to known target sequences, many with known functions, now permit investigators to interrogate the genome, transcriptome, proteome and metabolome systematically, and to assess genomic mutations, polymorphisms, epigenetic alterations, and micro-RNAs. These technologies herald the potential for us to translate their results into novel sensitive and specific diagnostic tests and less toxic therapeutics, with the anticipation of moving away from protocol-based approaches to personalized medicine. In this new era, investigators could potentially assess the molecular and pathophysiological characteristics of individual patients and their transplanted organs, tailor therapeutic regimens, and administer them based on these profiles. One of the key steps in this process will be the identification and validation of biomarkers. To date, microarray technologies are, perhaps, the most successful and mature methodology for high-throughput and large-scale genomic analyses.Care must be taken to choose a platform that will allow data to be analyzed and disseminated in the manners desired. Experimental design and analysis are generally more straightforward with one-color microarrays.Microarray technology is based on the principle of complementary, single-stranded, nucleic acid sequences forming double-stranded hybrids; thus, in essence, it is a high-throughput Southern blot, where thousands of single-stranded sequences that are complementary to target sequences are synthesized, or spotted on to a small glass or membrane support. The gene probes on the array are either small (20–60 bp) single-stranded oligonucleotides, synthesized in situ , or cloned complementary DNA (cDNA) amplified by polymerase chain reaction (PCR) and obtained by reverse transcription of messenger RNAs (mRNAs). Illumina uses designed oligonucleotide probes attached to beads that are deposited randomly on a substrate. The selection of microarray platforms will have important effects in the later stages, determining the complexity and flexibility of the data to be analyzed. In organ transplantation, a prospective study of the gene-expression profile of graft injury usually involves sample collection from tissue biopsy, blood and biofluids, such as urine, bile, or broncho-alveolar lavage, taken before, during, and after injury. A schematic presentation of microarray study is shown in Fig. RNA is next labeled with a detectable marker (fluorescent dye) and hybridized to an array containing individual gene-specific probes, in either a dual-color   or singlEarly microarray technology allowing the hybridization of high-density cDNA with samples on nylon membranes provided a technological step in the right direction, but this technology is limited in scope by the nature of the matrix supporting the clones. Two later innovations made possible the new microarray technologies. One was the use of solid supports, such as glass, which are much more amenable to miniaturization and fluorescence-based detection . The secThe overall differences in cDNA (e.g. Lymphochip) vs oligonucleotide-based arrays   lie in tMicroarray technologies were initially designed to measure the transcriptional levels of RNA transcripts derived from thousands of genes within a genome in a single experiment. This technology has made it possible for one to relate physiological cell states to gene-expression patterns for studying tumors, disease progression, cellular response to stimuli, drug target identification and transplant injury mechanisms. For example, subsets of genes with increased and decreased activities  have been identified for acute lymphoblast leukemia , breast More recent uses of DNA microarrays in biomedical research are not limited to gene-expression. DNA microarrays are being used to detect single nucleotide polymorphisms (SNPs) of the human genome (Hap Map project) , aberratGene-expression profiles for prognostic classifiers are usually built by the correlation of gene-expression patterns, generated from specimens, with clinical outcome (e.g. acute rejection vs stable without rejection). Gene-expression predictive classifiers of response to treatment are generated by the correlation of gene-expression data, derived from samples taken before treatment, with clinical and pathological response to treatment. Although the identification of the most relevant information from microarray experiments is still under active research, well-established methods are available for a broad spectrum of experimental set-ups. The analysis of gene-expression data at the pathway and functional level, along with a systems biology approach, will provide deeper insights into the biological effects of complex disease states, such as in the organ transplant milieu, and will improve risk assessment of the same.It is challenging to dissect any allograft injury mechanism with single-gene studies because of the complexity of the mechanisms for renal allograft rejection with different immunosuppressive protocols and the spectrum of the response with immunological injury. Previously researchers have reported that expression of the cytotoxic molecules granzyme B and perforin has been associated with rejection and has been detected in blood , urine , and bioThe advent of microarray technology has enabled researchers to detect the expression of thousands of genes simultaneously, rather than measuring the expression of one gene at a time, and has unlocked information about disease heterogeneity that could not have been predicted by standard clinical or pathologic criteria. Pioneering studies of gene-expression profiles in breast cancer have identified the molecular classification of breast cancer into clinically relevant sub-types. This has provided new tools with which one can predict cancer recurrence and response to different treatments, and new insights into various oncogenic pathways and the process of tumor progression . SubsequThese findings have brought a global paradigm shift from traditional hypothesis-driven experiments toward large-scale hypothesis generation and testing through clinical trials. In the past few years, there has been an increasing number of publications on solid organ transplantation, with particular emphasis on the heart and kidney. Supporting the results of earlier studies, microarrays have also corroborated the finding of known pathways in rejection injury, such as evidence of the dysregulation of the complement system , 39, intThe interrogation of minimally invasive or non-invasive biomarkers of graft injury has been more challenging than the direct interrogation of the transcriptional changes in the injured graft. Until recently, most gene-expression profile studies were performed on biopsy specimens. Transcriptional changes in peripheral blood during graft rejection demonstrate significantly disparate gene-expression changes from those of the inflamed graft, suggesting that the local response to inflammation and injury in the rejecting organ is highly localized. Additionally, the intensity (fold-change) and quantity (number of significant genes in rejection) of the rejection response in peripheral blood is much smaller than the corresponding response in the organ, even when biopsy and blood samples from the same patient are examined simultaneously  Fig. 2)2). A recConventional wisdom holds that long-term allograft survival requires life-long immunosuppression. A tremendous advance in peripheral blood monitoring for immunosuppression customization may lie in the data emerging from studies on a highly selected group of organ transplant patients with spontaneous graft acceptance or prope tolerance in liver  and kidnWhilst great advances have been made in the discovery of putative biomarkers in transplantation, disappointingly few have been translated into clinically applicable assays; much of this is due to a lack of well-designed clinical validation studies. The most important challenge is having well-designed validation and varying endpoint definitions. To adapt molecular endpoints from single-gene studies as representative of a particular mechanism of toxicity/injury often assumes that a postulated mechanism must be known beforehand, and this may result in “over-fitting” of the data, making the inference not entirely accurate.Chronic graft injury, potentially an indolent immune response resulting in slow deterioration of organ function, characterized pathologically in the kidney by tubular atrophy, interstitial fibrosis, and fibrous intimal thickening of the arteries, has a relatively transcriptionally homogeneous response of tissue fibrosis, when investigated at the time of the established injury . AlthougThe limitations of microarray studies and the importance of well-designed validation strategies have been demonstrated by microarray applications in cancer. In an attempt to predict prognoses of cancer patients on the basis of previously published DNA microarray studies, re-analysis of data from the seven largest published studies showed that the list of genes identified as predictors of prognosis was highly unstable; the selection of training sets strongly affected molecular signatures , and bioAcute rejection (AR) depends on an orchestrated immune response to histocompatibility antigens expressed by the grafted tissue. The redundancy of AR mechanisms and the problems with peripheral blood transcriptional analysis (outlined above) has made it difficult for reliable biomarkers to be developed for prediction of allograft rejection and its outcome, irrespective of immunosuppression usage, concomitant infection, recipient age or organ type. The delineation of AR from antibody-mediated rejection  is still debated. Though effector mechanisms primarily responsible for the rejection process classically involve activation of effector T cells and memory T cells, alternative mechanisms of acute rejection also recruit (to varying degrees) B-cells, natural killer cells, eosinophils and neutrophils, antibody-mediated rejection is currently thought to play a role in approximately 33% of AR episodes . Thus, tFinding ways to use kidneys more effectively for transplantation has the potential to extend the donor pool using dead donors who meet the Standard Criteria for Donation (SCD) as well as those from Expanded Criteria Donors (ECDs), created by the United Network for Organ Sharing in 2002. Higher-risk donor organs, once considered unsuitable, could also be transplanted safely. With regard to higher-risk donor organs, the question remains: Are all kidneys from older donors equivalent with regards to biological and cellular health? An array-based analysis of kidneys from a wide spectrum of ages (8 months to 80 years)  suggestehttps://doi.org/cran.r-project.org), the Gene Expression Profile Analysis Suite (GEPAS) [fold-change in fluorescence intensity (expressed as the logarithm (base 2 or log2) of the sample divided by the reference), is often used descriptively, it fails to ascertain the true significance of small but significant changes in gene-expression levels. Univariate analysis measures of significance are preferred, e.g. for data sets with normalized distribution, the t-test and analysis of variance (ANOVA) test can be used, and for data without normalized distribution, the Wilcoxon or Mann–Whitney tests are often used. Another commonly used measure of significance testing is significance analysis of microarrays (SAM) (https://doi.org/www-stat.stanford.edu/~tibs/SAM/). It should be pointed out that the methods given in this review are a selection from many others possible.Determining biomarkers for clinical phenotypes of diseasehttps://doi.org/www-stat.stanford.edu/~tibs/PAM/) is a powerful tool that can be adapted for this use. The selection of a unique list of genes by this approach does not, in and of itself, offer sufficient knowledge for one to understand the biology of a given system, suggesting the necessity to incorporate biological knowledge into array analysis. Recent approaches to microarray analysis address the limitations of conventional bioinformatics approaches by enriching the analysis with knowledge of biological processes [https://doi.org/www.ingenuity.com/; Pathway Studio: https://doi.org/www.ariadnegenomics.com/) or to use hypergeometric enrichment analysis from published data sets of biologically relevant experiments [The identification of gene-expression “signatures” associated with diseases categories is called biomarker detection or supervised classification. As the biomarker panel needs to be predictive of disease class or clinical outcome, learning and validation sets of samples are required, making the sample size relatively large for this type of analysis. Prediction analysis of microarrays  [Other applications of microarrays in clinical practiceAltered methylation patterns in genomic DNA can also be identified by microarrays, by the use of methylation-sensitive restriction enzymes to generate fragments enriched with either unmethylated or methylated CpG sites. Epigenetic phenomena, such as cytosine methylation, histone acetylation and phosphorylation, control the activation and deactivation of genes, such that genes methylated in their promoters can become inactive and can predispose individuals to cancers [Chromatin immune-precipitation (ChIP-on-chip) assays [Pathogen specific microarrays have been generated [Commercially available microarrays can detect single nucleotide polymorphisms (SNPs), which are an important tool for identifying genetic loci linked to complex disorders . Unfortu cancers . Chromat) assays , 73 can ) assays ). Pathogenerated , 28 and Identifying whether genes have increased or decreased in expression between two or more groups of samples is the most common and basic type of analysis and provides a simple characterization of the specific molecular differences that are associated with a specific biological phenotype. Using power analysis, it is possible to estimate the number of samples required to identify a high percentage of truly differentially regulated genes between sample groups. Unsupervised analysis  is a use (GEPAS) , The Ins (GEPAS) , 67, Gen (GEPAS)  and Gene (GEPAS) . TypicalQuality controlThe high-throughput nature of this technology, combined with the expected large numbers of data, result in a high risk for error. With the increasing use of genomic studies in transplantation, there is a need to control for various confounder effects that obscure biomarker discovery in graft rejection. In view of many concerns raised, the US Food and Drug Administration (FDA) lunched the Microarray Quality Control (MAQC) project. An excellent correlation of gene expression of human reference RNA (Stratagene) and human brain reference RNA (Ambion), across seven different array platforms, across five different laboratories, using three different amplification protocols , was shoHigh costAt present, because of the sophistication of microarrays, this is a costly technology available only in selected laboratories. Microarray technologies, however, are rapidly improving, and the costs of the technique continue to fall, thus paving the way for wider access and more generalized usage.Sampling variabilityParticularly for renal transplant biopsies, differing amounts of cortex vs medulla represented in a sample can affect the pattern of gene expression of a sample. Therefore, one can cross-reference a publicly available gene list specific for different compartments of kidney to minimize false clustering of samples . There iDifficulty in detecting some disease processes in transplantation by microarraysExisting collagen, readily visible to the pathologist, is not necessarily associated with mRNA changes if the process of active fibrogenesis is complete. Small cell populations that make a major contribution to disease might give only a weak signal in transcriptome studies of whole biopsies or unseparated blood. Antibodies produced in lymphoid tissues could damage the kidney without any mRNA being detectable in the kidney. Microarray analysis cannot offer insights into these critical cellular and molecular processes in the tissues.Discrepancy in array studiesWeak overlap exists between gene lists from individual studies of similar phenotypes in transplantation. The disparity among microarray data can be attributed to several factors: differences in microarray platforms with differing gene sets; weak statistical power and small sample sizes; biological variance because of variability in patient characteristics; experimental variance including lack of uniform protocols for study design, sample collection, RNA processing, and sample labeling and hybridization; different tools for data processing and statistical analysis Table ; variablConfounders exist in microarray experimentsPrevious studies have demonstrated that the abundance of globin genes in whole blood may mask the underlying biological differences in whole-blood samples. In a comparison of gene-expression profiles of peripheral blood, using different protocols of sample preparation, amplification and hybridization on the Affymetrix platform, we demonstrated that the globin reduction method is not sufficient to unmask clinically relevant, rejection-specific, transcriptome profiles in whole blood. Additional mathematical application for globin gene depletion improves the efficacy of globin reduction but cannot remove the confounding influence of globin gene hybridization . Other pGene-expression profiling studies of renal transplantation are highly complex, and the successful execution of such studies requires close collaboration between physicians, molecular biologists and bioinformatictists. The most important challenge is the lack of a true gold standard for the classification of disease in organ transplantation. The current histologic classification of deteriorating organ transplants has many limitations, including arbitrary cut-off points and poor inter- and intra-observer reproducibility. The classical approach to microarray analysis, which starts with the identification of genes that differentiate between two sample groups, depends on the assumption that distinct disease entities exist and that we know, with certainty, what these classes are. While the cancer literature may be able to rely on disease classification based on outcome data, the situation in organ transplantation is much more complicated, with many overlapping disease processes occurring simultaneously. If microarray analysis in clinical transplantation starts with a classification based on a flawed clinical gold standard, the results of the microarray study will not be any better than histologic examination and may even be misleading. Currently, there is no simple solution to this problem.All published microarray studies should be made publicly available through the internet, on proprietary websites and in public microarray database repositories, and should generally follow the minimum information about a microarray experiment (MIAME) compliance format  or microhttps://doi.org/www.gepas.org)The Gene Expression Profile Analysis Suite The Institute for Genomic Research ,Significance analysis of microarrays ,Prediction analysis of microarrays ,Expression Profiler: next generation ,Bioconductor (https://doi.org/bioinformatics.upmc.edu/GE2/GEDA.html),Cancer gene expression data analyzer ,Analysis of microarray data ,GenePattern (https://doi.org/genome.tugraz.at/Software/Genesis/Description.html)Genesis . This could potentially be caused by an annotation problem, with at least 5% of misannotation in each generation of the platform , e.g. muAgilent cut off, absolute value of log2 red channel/green channel > 0.5 for at least one array; cDNA mean of channel 1 intensity/media background intensity > 1.5, and/or normalized mean of channel 2 intensity/media background intensity > 1.5; Affymetrix using a perfect-match-only model, the value for each probe or probe sets are extracted after background subtraction.There are also different levels of detection (background over noise) for probes or probe sets, with specific criteria for each platform, which can have an impact on downstream analyses. Some of the different criteria used for a probe detection for different array platforms are discussed: https://doi.org/ailun.stanford.edu/) was developed by a Stanford University group that re-annotates all gene-expressions/proteomics data from the Gene Expression Omnibus , which is a public repository for gene-expression and other high-throughput experimental data covering numerous platforms and species. The AILUN server builds a universal identifier ID table by relating all probe IDs to Entrez Gene IDs on a monthly basis, and it is the first tool available that allows researchers to compare microarray data across different platforms and map genes across species [The exiting tools for converting the probe ID between microarray platforms are very limited. The difficulty in reusing data lies with the mapping of probes to established gene identifiers. Therefore, microarray results need to be re-evaluated periodically with the latest probe annotations. Most recently, a tool (Array Information Library Universal Navigator, AILUN,  species . It also species . Some st species . When mi species .High-throughput DNA microarray technology has been increasingly applied in kidney transplantation to classify molecular sub-types, to predict outcome and the response to treatment, and to identify novel therapeutic targets. Although results hold promise, this technology will not have a full impact on routine clinical practice until there is further standardization of techniques and optimal clinical trial designs to set up higher volume validation studies for the generated biomarkers. Owing to substantial disease heterogeneity and the number of genes being analyzed, collaborative, multi-institutional studies are required to accrue enough patients for sufficient statistical power. Customized arrays or multiplex PCR for informative biomarkers can then be applied to the clinics for event prediction, treatment stratification, immunosuppression customization and improved graft and patient survival.Our scientific environment is ripe for research-based implementations of integrative tools that support knowledge-based data mining. This integration can provide the cornerstone of research in the coming years. Developments in this area will require close interdisciplinary collaboration and will lead not only to the integration of data and knowledge, but also to computer-supported experiments and to knowledge generation platforms, thereby closing the loop of data gathering, hypothesis generation and hypothesis testing.


Much of the public access cancer microarray data is asymmetric, belonging to datasets containing no samples from normal tissue. Asymmetric data cannot be used in standard meta-analysis approaches (such as the inverse variance method) to obtain large sample sizes for statistical power enrichment. Noting that plenty of normal tissue microarray samples exist in studies not involving cancer, we investigated the viability and accuracy of an integrated microarray analysis approach based on significance analysis of microarrays (merged SAM) using a collection of data from separate diseased and normal samples.We focused on five solid cancer types , where available microarray data allowed us to compare meta-analysis and integrated approaches. Our results from the merged SAM significantly overlapped gene lists from the validated inverse-variance method. Both meta-analysis and merged SAM approaches successfully captured the aberrances in the cell cycle that commonly occur in the different cancer types. However, the integrated SAM analysis replicated the known cancer literature (excluding microarray studies) with much more accuracy than the meta-analysis.The merged SAM test is a powerful, robust approach for combining data from similar platforms and for analyzing asymmetric datasets, including those with only normal or only cancer samples that cannot be utilized by meta-analysis methods. The integrated SAM approach can also be used in comparing global gene expression between various subtypes of cancer arising from the same tissue. Microarray studies typically provide intensity levels for thousands of genes. However, not only are the individual datasets usually small in size, but the inferences made from individual studies are often inconsistent with similar studies . As thouThe meta-analysis procedures cited above combine results from different studies. Each dataset is analyzed separately. Genes are associated with an effect size or a p-value. These are then combined across all analyses and a top-ranked gene list is generated based on the aggregated effect size or p-value . While sTo test the performances of the meta-analysis and the merged SAM approach, we compiled microarray data from 31 laboratories, resulting in a database containing 339 healthy tissue samples and 1,429 cancer samples from 5 different tissue types using comparable Affymetrix platforms. The tumor tissue types considered in this study -colon, kidney, liver, lung, and pancreas - had multiple microarray datasets containing both normal and disease samples. The meta-analysis approach has already been employed by a few cancer microarray studies either focusing on a single tissue type ,22-24 orThree different groups of microarray datasets were used to evaluate (a) the intersection of significant gene lists predicted by meta-analysis and merged SAM methods and (b) compare these predictions with research literature excluding microarray studies. Group 1 is composed of Affymetrix microarray datasets containing both cancer and normal samples for five different cancer tissues  were chosen. The expression values for the three arrays were obtained based on classical RMA and refRMA normalization techniques. Quantile-quantile plots were produced to compare the distributions of the different datasets in a pair-wise manner Figure . The poiAs in previous microarray studies of cancer ,26-31, tTo identify significantly altered genes across the five considered tissue types, the datasets from all tissues were pooled together. Again, SAM2 included additional datasets with cancer or normal samples only. Similarly, the significance of the overlap between the results increased as more top-ranked genes were considered, with p-values equal to 6.82E-97 and 2.80E-103 for the intersection at the top 400 genes level in IV1/SAM1 and IV1/SAM2, respectively . Complement and coagulation cascades were enriched in four out of the five tissues under study. These results show that both methods of integration are capable of reproducing a significant portion of the research literature on cellular pathways activated in cancer.Next, we tested the SAM1, SAM2, IV1, and IV2 gene lists for PubMed hits associated with cancer. We conducted an automated PubMed abstract search for the genes in the aforementioned lists. All available abstracts in Pubmed were used excluding those that belonged to microarray-based research. Also excluded were abstracts that did not contain the word "cancer". A gene had to have at least one such PubMed abstract match to be considered as a literature search hit. The number of successful hits produced from the merged SAM methods and the IV tests intersected the research literature with significantly higher coverage than would be expected for randomly generated gene lists Figure . The p-vPubMed hits on gene lists presented by meta-analysis and merged SAM approaches fell inside and outside the intersections. Consider for example the case of colon cancer in IV1 and SAM1 gene lists. There were 93 hits on IV1 ∩ SAM1 (p = 1.19E-07), 103 hits on IV1 - IV1 ∩ SAM1 (p = 5.09E-02); and 205 hits on SAM1 - IV1 ∩ SAM1 (p = 2.32E-23).As an additional control, the next top 400 genes (ranks 401-800) in each list, if available, were subjected to a similar PubMed abstract search. The p-values representing the results revealed decreased literature coverage of these genes compared to the first top 400 genes in all cases except for SAM2 results in lung tissue. In this test, majority of the IV results (except for lung and pancreas) dropped below the 0.0001 p-value threshold marked by a horizontal line in the figure.Our results show that in the merged SAM approach, the more symmetric the datasets are in terms of containing both disease and control samples, the better is the match between gene lists produced by microarray analysis and the PubMed literature. Both SAM1 and SAM2 (containing asymmetric data) produced more significant p-values per tissue than the average p-value obtained from the SAM tests performed on the individual datasets for a given tissue (data not shown). The addition of single sample-type datasets  in SAM2 resulted in fewer literature-associated gene lists than the SAM1 approach; however, the results improved when considering the top 400 genes as opposed to the top 300.Next we considered an extreme case of SAM2, where the dataset was composed of disease samples from some studies and control samples from other studies. In this case, there would be no symmetric core in the dataset under consideration. To develop such purely asymmetric datasets, we deleted either the disease or the control samples of any symmetric study included in SAM2, considering all possible permutations for the datasets from three tissues: colon, liver and lung. The resulting gene lists were annotated with PubMed hits. We calculated an average number of PubMed hits over all possible combinations and corresponding p-values. The results  with very low p-values for random occurrence. However, the merged SAM approach replicated the existing literature much more accurately than the meta-analysis approach in five of the six cases under study. Addition of cDNA arrays into meta-analysis resulted in reduced overlap with the cancer literature. Meanwhile, the inclusion of asymmetric datasets also produced slightly less statistically significant results in merged SAM analyses, nevertheless, the approach still generated results that were at least as significant as the meta-analyses, again surpassing meta-analysis in five out of the six cases. Despite the addition of hundreds of samples from asymmetric sets, the merged SAM continued to perform well, matching literature as well as results of symmetric microarray data. We also showed that the match between microarray lists and the literature became less pronounced as lesser-ranked significant genes (401 - 800) were used in the comparison. The gene lists obtained in all the tests were further validated by associating them with functional annotation through KEGG pathways. While individually each tissue possessed a unique list of pathways and processes with which it was associated, overall, cell division appeared to be the common driving factor to all tissues, as would be expected.We used automated text searches as an instrument for validation of the prediction value of the two different approaches to integrate microarray data associated with cancer. Typical validation used in microarray analysis for illustrating relevance of gene list to disease state under consideration is usually via partitioning the dataset into learning, testing/validation subsets in a supervised learning approach -41. HoweOne reason for asymmetry in the current public access microarray data is that the goals of global gene expression quantification in cancer research shifted towards identifying significant genes associated with cancer subtypes -47. The PubMed hits on gene lists produced by meta-analysis and merged SAM approaches fall on the intersections of such lists as well as outside the intersections, suggesting the use of both approaches whenever data is available. The top ranked 400 genes in both cases are highly statistically enriched with PubMed hits and for which the intersection between the two approaches had typically the lowest p-value. When considering the role of well studied genes such as hub genes or genes in public access cellular pathways, it is straightforward to project both gene lists onto known pathways to generate new hypotheses for experimental verification. The merged SAM technique provides a unique opportunity to obtain a candidate list for genes associated with a perturbed state in cases where the public microarray data is largely asymmetric.Typical meta-analysis approaches allow for the use of various platforms at the expense of utilizing large amounts of data that exist in datasets containing either normal or cancer tissues only. Our merged SAM approaches have been shown to reproduce much of the known cancer literature while effectively being applied to asymmetrical microarray datasets. In our merged data approach, SAM analysis could be replaced by other widely used statistical methods, thus increasing the extent of the methodology. Such methods may include both parametric approaches such as PAGE  and T-prA total of 31 Affymetrix microarray datasets containing 1,768 unique samples from human cancer 1,429) and corresponding healthy control tissues (339) were collected from the Gene Expression Omnibus   in order1(i) and x2(i) are the average levels of gene expression for gene (i) in states 1 and 2, respectively, and s(i) is the gene-specific pooled standard deviation which is equal to:where x1 and s2 are the sample standard deviations of groups 1 and 2, while n1 and n1 are the number of samples in group 1 and 2, respectively. The false discovery rate (FDR) was set at 0.001%. Moreover, the samr package [where s package  in R was package  on each While IV analyzes each dataset separately before combining the results, SAM can be applied to previously merged data. This merger was achieved by using the refRMA algorithm , designei is the log2 scale expression values of array i, aj is the log scale affinity effect of probe j and εij is the error. A more detailed description can be found in [where T is the transformation for the background correction, normalization and log transformation of the perfect match intensities, efound in .The genes common to all three platforms are then chosen allowing for the integration of data from all three platforms together, limiting results to the 9,409 genes. To verify the application of refRMA to the added data compared to the original training set, one sample was randomly chosen from each of the three colon Affymetrix datasets that contained both healthy and cancer samples. Quantile-Quantile plots (Q-Q plots) were then generated for these arrays based on their individually normalized values (RMA) and collectively normalized values (refRMA). In each case, all gene expression values from one array are plotted against all gene expression values from the second array in order to assess the similarity in their distributions . A mergeAs noted above, the IV test is limited to datasets that contain both cancer and normal tissues. The merged SAM method, however, allows for the inclusion of datasets containing solely normal or solely cancer samples. Thus, to test the effect of adding such datasets, microarray samples from all datasets of the same tissue were combined together and another series of SAM analyses were applied using the same test parameters as above. For the purpose of this paper, the first set of SAM tests, based on the data from the 19 datasets containing both normal and cancer tissues, is referred to as SAM1 Figure . The secTo identify consistent changes that are associated with multiple cancer tissue types, an IV1 test was conducted on all 19 Affymetrix datasets containing both cancer and normal samples together, regardless of tissue type. Similarly, a SAM test was performed on the same samples (SAM1) and another SAM test was applied to all 1,768 available Affymetrix samples from the 5 tissues considered (SAM2; Figure An additional 5 datasets using cDNA microarray platforms were obtained from GEO (Additional File To determine the extent to which each method replicated the known cancer literature an automated text search was performed. A search of the gene symbol and the term "cancer NOT microarray" was conducted in PubMed abstracts for all genes available from the different platforms, limiting results to non-microarray literature. All gene lists obtained through IV and SAM analyses were then annotated with these results, identifying those genes that were cited in relation to cancer at least once from those that had no cancer association. A hundred random gene lists from the same platform of equal size to the lists under consideration were obtained and used as a control. The number of cancer-related genes in each of these random iteration was determined, and the mean and standard deviation were calculated from these values to obtain the parameters of a normal distribution. The expected value and the standard deviation were then used to compute the p-values for the significant association of each of our cancer gene lists with the known non-microarray literature.IV: Inverse Variance; KEGG: Kyoto Encyclopedia of Genes and Genomes; SAM: significance analysis of microarrays.The authors declare that they have no competing interests.ND and AT conceptualized the research. ND implemented the algorithms and discussed results with AT and prepared the first draft of the manuscript. ND and AT both have read and approved the final manuscript.The refRMA model is available by request from  and a MATop 800 Ranked Genes: Annotation of the top 800 genes for each tissue according to IV1 and SAM1analyses. Fold changes shown are based on overall values across all platforms and samples. Genes not shared by all three platforms are marked as unique.Click here for fileMicroarray Samples Used in the Study: This file contains five worksheets that list all normal and cancer tissue microarray samples used including accession numbers of datasets, sample labels, sample tissue annotation and platform, in addition to malignancy description of cancer samples and available clinical annotation.Click here for file


Widespread use of microarrays has generated large amounts of data, the interrogation of the public microarray repositories, identifying similarities between microarray experiments is now one of the major challenges. Approaches using defined group of genes, such as pathways and cellular networks , have been proposed to improve the interpretation of microarray experiments. We propose a novel method to compare microarray experiments at the pathway level, this method consists of two steps: first, generate pathway signatures, a set of descriptors recapitulating the biologically meaningful pathways related to some clinical/biological variable of interest, second, use these signatures to interrogate microarray databases. We demonstrate that our approach provides more reliable results than with gene-based approaches. While gene-based approaches tend to suffer from bias generated by the analytical procedures employed, our pathway based method successfully groups together similar samples, independently of the experimental design. The results presented are potentially of great interest to improve the ability to query and compare experiments in public repositories of microarray data. As a matter of fact, this method can be used to retrieve data from public microarray databases and perform comparisons at the pathway level. Since their first inception a decade ago, microarray studies have become widely used in the research community, thanks to their ability to assess the expression of thousands of genes in a single laboratory event. The belief that such wealth of genomic information the community could not afford to lose has led to the development of microarray standards Identifying biologically meaningful information in relatively noisy data represents a significant tasks and so far breakthrough have been few and far between. Comparisons made on the level of gene lists obtained by different statistical methods or from different datasets hardly converge In the past years collections of all the differentially expressed genes in a given condition that exclusively characterize that condition, have been proposed as “gene signatures” for a condition The number of inconsistencies and discrepancies when microarray data sets are compared are often reduced when approaches that take into account biologically related sets of genes, rather than single entities, are used Recently we developed a bioinformatic environment called Eu.Gene We developed a method allowing to assess similarity of samples in microarray databases. To this aim we used EuGene to generate “Pathway Signatures”, recapitulating the biologically meaningful pathways related to some clinical/biological variable of interest, and used them in an analysis workflow to compare different microarray experiments.Saccharomyces cerevisiae (S. cerevisiae) using the dataset from the Rosetta compendium of deletion mutants Homo sapiens dendritic cells (DCs) to different stimuli.We decided to use as reference experiments two distinct data sets: firstly, a data set of gene expression experiments on a model organism, ssn6 and tup1 mutants would be similar, as acting as part of the same protein complex. Regarding the dendritic cells data set, we performed a wet lab experiments stimulating DCs with cells of the yeast S. cerevisiae in exponential growth phase  and we predicted that this experiment would show similarity with the samples from GEO data set GSE6965 (gene expression profile of monocyte-derived DCs stimulated for 6 h with germinating germ tubes of Aspergillus fumigatus).Following up the data set collection, data were pre-processed and subject to pathway analysis see . Then, wWe initially compared different statistical approaches for pathway signature generation. To address the issue of the most appropriate statistic, we evaluated both Fisher's Exact Test (FET) and Gene Set Enrichment Analysis (GSEA), two well-established methods for pathway analysis. As GSEA required samples to be divided into two distinct classes, it was not performed on the yeast data. On the other hand both GSEA and FET were used with the DC samples. FET produced results in the form of signed p-values, while GSEA produced signed enrichment scores (ES).10 ratios and thus, after a conversion to log2, was used directly for the Fisher's Exact Test.Yeast data was in the form of logIn order to use dendritic cell data for the Fisher's Exact Test, we converted the Affymetrix absolute expression values into ratios see . When deIn order to selecting the most representative FET threshold, FET thresholds were selected from the 2σ interval of the binomial distribution of the expression values.When using FET, we addressed the question of how to use the collection of p-values for a defined set of pathways to generate a pathway signature with the aim of comparing different microarray experiments. Firstly, we corrected the p-values for multiple testing see  then ourOn the other hand, GSEA enrichment scores (ES) were used directly, without any modification.We then evaluated the performance of these metrics (sBEF or enrichment score), as expressed in the ability of assessing similarity according to the expectation that biologically similar experiments should show identical or similar pathway profiles .To find reliable ways to group samples together, we investigated a number of unsupervised hierarchical clustering techniques that would be used to group together the metrics we had selected. For this purpose, Unweighted Pair Group Method with Arithmetic mean (UPGMA) clustering, standard hierarchical clustering, and hierarchical clustering with support trees were used , S4, S5.Additionally, as sBEFs were categorical values, we also evaluated the similarity between samples by using a metric suited for this type of data (the Jaccard index).ssn6 and tup1 deletion mutants clustered together when using Pearson's correlation as clustering metric  and we predicted that this experiment would show similarity with the samples from GEO data set GSE6965. The Pathway Signatures of our ad-hoc sample produced stimulating DCs with cells of the yeast umigatus , S7. Theumigatus . Since wumigatus  and S9. On the other hand, when we clustered enrichment scores from GSEA on the dendritic cell data, results were not in agreement with the prediction : our expIn this work we propose a novel method to compare microarray experiments at the pathway level, our work addressed three major tasks: (i) defined a metric measuring the probability of a set of pathways to be related to some clinical/biological variable of interest and the relative importance of that set in the context of the biological problem, (Pathway Signature) (ii) proposed a method using these pathway signatures to assess relative similarity of the experiments ; (iii) proved the validity of this method applying it to two different well defined biological problems on two independent collections of microarray experiments.S. cerevisiae, the most extensively studied model organism, with a well characterized genome where all the genes are represented on the array, widely used to test bioinformatics methods. We selected the Rosetta compendium of deletion mutants Homo sapiens because the majority of the experiments stored in GEO and ArrayExpress are from human samples. We chose transcriptomic datasets from DCs as they have some clear advantages respect to other fields. Firstly, the cell type is well-defined, which enables the study of the alterations in gene expression following stimulation. Secondly, there is the possibility to perform prediction and hypothesis-driven functional genomics studies aiming at the reconstruction of the networks of molecular interactions characterizing specific DC differentiation programs.As a proof of concept we selected datasets representing well known stimuli in well known biological systems. One of the datasets was from The aim of our work was to propose a procedure to assess similarity between microarray experiments at the pathway level generating “Pathway signatures” (PS) for a set of experiments and use these signatures to interrogate microarray databases.The need for better methods to identify similarities in microarray data sets arises from the fact that although the analysis techniques have constantly improved over the past years, one of the biggest hurdles remains the comparability among distinct data sets produced by different researchers and laboratories, resulting in lists of genes which do not overlap, or overlap in a very limited fashion. A possible reason relies on the different assumptions on the data used by different statistical methods Also, as the number of publicly available data sets increases , it is important to have reliable method to compare microarray data from different sources.The improvements observed when comparing different experiments at the pathway level The selection of the statistical method used to measure enrichment is central to our approach.Different metrics have been proposed to integrate the probability of alteration of a sector of the cellular network (pathway) and the relative importance of that pathway in the context of the biological problem, such as the probability vector ssn6 and tup, genes that form a co repressor complex which is responsible for repressing a large number of S. cerevisiae genes, including glucose-responsive genes, DNA damage genes and oxygen utilization genes ssn6 and tup1 act together, deletion of one or the other gene is expected to yield a similar behavior. On the yeast dataset the exact topology of the cluster was dependent on the metric used. This is quite understandable: sets of PEFs can be noisy, as there are all the possible ranges of p-values, and identifying which pathways were “significant” and which “not significant” was not always straightforward. This can influence the clustering, and so certain metrics prove to be more useful than others, for example Pearson's correlation was the metric which most correctly grouped ssn6 and tup1 in our yeast data. This result could be biologically relevant, as although ssn6 and tup1 are part of a complex, tup1 has also a function alone, and as a matter of fact, the pattern of significantly altered pathway (shown by their sBEFs) in the two deletion mutants exhibits differences. This biological difference could result in changes in the direction in which some pathways are affected by one deletion or by the other. Alternatively this result could reflect greater sensitivity to technical “noise” in the data of some of the metrics used.FET p-values had to be transformed to improve interpretation: p-values express a probability, and the smaller they are, the more significant the result is, while from a conceptual point of view it is better to express pathway enrichment as a number that it is either categorical or the more significant the greater it is. That is the reason we first used the logarithm of reciprocal of the p-value (the Pathway Enrichment Factor), to express the measure in a scale that would avoid interpretation problems, the PEF was finally obtained multiplying the value for a sign representing the “direction” of a pathway, with the same approach described in S. cerevisiae follow the same downstream signalling as the ones in response to the fungus A. fumigatusBiclustering of PEFs on the dendritic cell data sets gave concordant and biologically relevant results, the responses elicited by stimulation with On the other hand, sBEFs produced consistent results from a clustering point of view, both for the yeast and the DCs datasets, independently from the clustering method, thus making very easy the identification of similar datasets.We selected sBEFs to produce the PS as they can be easily used as a “barcode” that can uniquely attached that sample, facilitating “querying” a database of PSs.The observation that sBEFs, PEFs and PSs, identify similarity between samples that are biologically meaningful, indicates that the categorical transformation of the p-values does not affect the observed result, concluding that the observed similarity has a biological meaning rather than resulting from a manipulation of the data.Iris data set When testing different methods to compare experiments at the pathway level, we initially chose clustering because of its flexibility and robustness in representing biological data. The type of clustering used was biclustering with support trees over 100 or 1000 iterations Yet since sBEFs are categorical rather than continuous, we investigated the use of an alternative metric (the Jaccard index), since biclustering may not be completely suited for this type of data. Our results showed an excellent agreement with the clustering for DC data, but some inconsistencies with the yeast data, in agreement with the clustering of the PEFs for this dataset. This could reflect some biological instances, or alternatively could be related to noise deriving from technological issues, as the yeast dataset is a none year old two color array. Another possibility is that measures like the Jaccard index may not be the most robust to capture the inherent complexity of “omics” data. In any case, our future work will be aimed at improving the grouping metrics for sBEFs.Results with GSEA on the dendritic cell data did not satisfy our prediction, due to the different numbers of experiments present in the different classes and the difficulty to classify one data set in a class or another. In fact, the grouping observed with FET scores on DC data was absent when using GSEA data. There are two possible reasons to this inconsistency. First of all, GSEA compares two groups (treated versus untreated) as opposed to single-sample analysis performed with FET. Thus, the effect of inter-donor variability  is noticeable and is not corrected by the pathway approach. The second reason is the heterogeneity of the samples themselves, that are subject to different treatments. As a result, GSEA rounds everything to the lowest common denominator, presenting an “average” profile where individual differences are smoothed out. Also, the number of control and treated samples in each data set is different , so there is an imbalance among the various GSEA analyses.We can conclude that the ability to investigate at the single sample makes the Fisher's Exact Test conceptually more appropriate to search for similarities among experiments in a microarray databases, that contain a number of hybridizations that should be interrogated without necessarily specifying the membership to a data set or another. Lastly, GSEA requires dividing the samples in two distinct phenotypic classes in order to operate, and division in classes is not necessarily straightforward. As a matter of fact, such a type of analysis is not suitable for two-color microarrays (which present data as a ratio between treatment and control), and therefore we were not able to use it with our yeast data set.Overall our results show that using pathway signatures in conjunction with hierarchical clustering with support trees is a powerful and useful technique to compare experiments produced by different people and laboratories with greater power than with the traditional analysis techniques. The results are also easier to interpret and discover biologically meaningful implications, making this approach an ideal candidate to analyze data from different sources. PS generated using sBEFs can be useful as “barcodes” to classify experiments in microarray databases and clustering of sBEFs can be a useful way to query experiments in databases according to their similarity at a pathway level. Thus, we propose to store PSs as an additional experimental annotation in the microarray databases and implement methods using pathway signatures to query experiments in public databases and concurrent analyses of subsets of experiments.Despite the effectiveness of the method, there are still some drawbacks that will need to be addressed in the future. First of all, Fisher's Exact Test result depend on the lower and upper cut-offs for expression, and at the current time they are defined by the user. We were well aware of the limitations of this method but also aware of its robustness if the appropriate threshold is used, in agreement with the findings from Bussemaker and Boorsma, that proved that the Fisher's Exact Test outperforms other metrics when appropriately selecting the threshold 10 ratios) were transformed into log2 ratios.A selection of 37 experiments on yeast deletion mutants were obtained from the Rosetta compendium http://www.r-project.org) which made use of the Bioconductor package GEOquery (http://www.bioconductor.org). Raw data (CEL files) were then downloaded and extracted using custom Bash shell scripts on the Linux operating system.106 microarray samples, corresponding to 6 data sets, were retrieved from the Gene Expression Omnibus (GEO) database . During 6 DCs were either not stimulated or stimulated with yeast grown in exponential phase. Cells were harvested after 4 hr of stimulation.Peripheral blood mononucleated cells (PBMCs) were isolated from buffy coats by density gradient centrifugation using Biocoll (Biochrom). CD14+ monocytes were isolated from PBMCs by positive selection using MACS anti-CD14 microbeads and a Midi-MACS® magnetic cell sorting device . Cells were cultured in RPMI 1640 medium (Gibco BRL) supplemented with 2 mM L-glutamine (Sigma), 1% (v/v) non-essential amino acids, 100 mM sodium pyruvate, 50 U/ml of penicillin and 50 mg/ml of streptomycin (Gibco BRL) containing 10% (v/v) FCS (Hyclone). Differentiation of monocytes into dendritic cells was promoted by addition of granulocyte-macrophage colony-stimulating factor and recombinant IL-4  for 5 days. 3*10RNA extraction was done with TRIzol reagent (Invitrogen). Sample pre-processing and biotin labeling were performed using the Affymetrix GeneChip® cDNA Synthesis Kit and IVT Labeling Kit (Affymetrix) according to the manufacturer's protocols. Microarray were then hybridized on Affymetrix GeneChip® HG-U133A 2.0 microarrays, and scanned according to the manufacturer's instructions on a GeneChip® Scanner 3000 (Affymetrix).Extraction, hybridization and scanning were performed by the Genopolis consortium . Microarray data have been submitted to GEO (accession number GSE13901).http://rmaexpress.bmbolstad.com), on the Linux operating system.CEL files were pre-processed and normalized with the Robust Multi-array Average (RMA) To identify biologically meaningful changes in our data sets, we assembled two pathway sets, depending on the species. All pathways were defined as gene lists (containing genes associated with the pathway). The sets were prepared as follows:Saccharomyces cerevisiae data: All the pathways present in Eu.Gene (n = 852)Dendritic cell data: Pathways containing less than three elements were excluded. Out of the remaining pathways, we built a selection curated by experts from the DC-THERA European network of excellence, where pathways were classified by their level of immunological relevance. Out of the 1038 human pathways available , 80 satisfying the experts' criteria were selected to be part of the set.Eu.Gene 2 ratios using the following procedure. For each data set, treated and untreated sample data were separated, and the expression level of each gene in each treated sample was divided by the expression level of the same gene in the corresponding paired untreated sample, and the resulting ratio was log2 transformed, obtaining a list of log2 ratios for each gene in each treated sample. The transformed expression data, corresponding to 66 ratios, were then used for Fisher's Exact Test.Prior to analysis, the Affymetrix data was converted into log2 ratios, was instead used directly, without any other modifications.Yeast data, once converted to loget al.The upper and lower cut-offs for FET were determined from the 2σ interval of the binomial distribution of expression values Resulting p-values from the FET analysis were then transformed prior to clustering (see below).For each data set, samples were divided into treated (“Type-A”) and untreated (“Type-B”) groups and their expression measures  were used directly for the analysis. GSEA was run on each data set separately, excluding pathways which contained less than 12 genes or more than 400 genes, to improve specificity and then significance of the obtained Enrichment Score was assessed by performing 1000 random permutations on the data. Enrichment scores from the analysis were then used for clustering.The FET p-value tables from Eu.Gene were then transformed in order to be used for hierarchical clustering. Two approaches were used. Firstly, p-values were transformed into Pathway Enrichment Factors (PEF) as follows:With the second approach, p-values were converted into signed Binary Enrichment Factors (sBEFs)Hierarchical clustering was performed with the TIGR Multiexperiment Viewer (TMeV), version 4.1.1 Unweighted Pair Group Method with Arithmetic mean (UPGMA) clustering was performed using Eu.Gene's internal “Path-Blast” function, creating similarity trees between different experiments in each data set with a p-value threshold of 0.05.http://code.google.com/p/scipy-cluster/).From the sBEF matrix, the Jaccard similarity index was calculated for every possible pairing between the various sample columns. The computation was performed with the Python programming language, using the scipy-hcluster package  up- (red) or down- (green) regulation. The colors of the dendrogram indicate the percentages of the tree support (significance), from 50% (pink) to 100% (black).(0.94 MB PNG)Click here for additional data file.Figure S2Clustering of signed Binary Enrichment Factors using Pearson's Correlation using support trees on yeast data. Colored spots indicate significant (p< = 0.05) up- (red) or down- (green) regulation. The colors of the dendrogram indicate the percentages of the tree support (significance), from 50% (pink) to 100% (black).(0.92 MB PNG)Click here for additional data file.Figure S3Clustering of signed Binary Enrichment Factors using Manhattan Distance using support trees on yeast data. Colored spots indicate significant (p< = 0.05) up- (red) or down- (green) regulation. The colors of the dendrogram indicate the percentages of the tree support (significance), from 50% (pink) to 100% (black).(0.94 MB PNG)Click here for additional data file.Figure S4Standard hierarchical clustering of signed Binary Enrichment Factors on yeast data. Colored spots indicate significant (p< = 0.05) up- (red) or down- (green) regulation.(0.92 MB PNG)Click here for additional data file.Figure S5UPGMA clustering of Fisher's Exact Test analysis results on yeast data.(0.16 MB JPG)Click here for additional data file.Figure S6Clustering of signed Binary Enrichment Factors using Pearson's Correlation using support trees on dendritic cell data. Colored spots indicate significant (p< = 0.05) up- (red) or down- (green) regulation. The colors of the dendrogram indicate the percentages of the tree support (significance), from 50% (pink) to 100% (black).(0.22 MB PNG)Click here for additional data file.Figure S7Clustering of signed Binary Enrichment Factors using Manhattan Distance using support trees on dendritic cell data. Colored spots indicate significant (p< = 0.05) up- (red) or down- (green) regulation. The colors of the dendrogram indicate the percentages of the tree support (significance), from 50% (pink) to 100% (black).(0.22 MB PNG)Click here for additional data file.Figure S8Standard hierarchical clustering of signed Binary Enrichment Factors on dendritic cell data. Colored spots indicate significant (p< = 0.05) up- (red) or down- (green) regulation.(0.20 MB PNG)Click here for additional data file.Figure S9UPGMA clustering of Fisher's Exact Test analysis results on dendritic cell data.(0.63 MB JPG)Click here for additional data file.


Using genomic DNA as common reference in microarray experiments has recently been tested by different laboratories. Conflicting results have been reported with regard to the reliability of microarray results using this method. To explain it, we hypothesize that data processing is a critical element that impacts the data quality.Shewanella oneidensis. Pair-wise comparison of three experimental conditions was obtained either with two labeled cDNA samples co-hybridized to the same array, or by employing Shewanella genomic DNA as a standard reference. Various data processing techniques were exploited to reduce the amount of inconsistency between both methods and the results were assessed. We discovered that data quality was significantly improved by imposing the constraint of minimal number of replicates, logarithmic transformation and random error analyses.Microarray experiments were performed in a γ-proteobacterium These findings demonstrate that data processing significantly influences data quality, which provides an explanation for the conflicting evaluation in the literature. This work could serve as a guideline for microarray data analysis using genomic DNA as a standard reference. This polynomially increasing number could become unmanageable for individual laboratory when n is a big number. In addition, it is nearly impossible to compare data across experiments since the cDNA reference sample composition is subjected to differences of experimental design and hence not universal. It has been desired for a long time to develop novel strategies to integrate data across multiple, initially unrelated studies between laboratories or over a long period of time to promote data sharing and integration. Lastly, this approach provides no information on the absolute mRNA abundance, which is often useful since it has been established in that global transcriptional levels in microorganisms have strong correlation with global protein levels and gene essentiality [DNA microarray technology has been quickly adapted by mainstream laboratories to explore gene expression profiles of part or whole-genome for an organism ,2. A numntiality -6.n microarrays are needed to calculate the ratios of any possible pairs of n samples. Apparently, this strategy greatly reduces the costs and time incurred in type 1 experiments. In addition, the absolute mRNA abundance of each gene could be deduced from γ1 and γ2, when the copy number of each gene is known for the common reference.A conceptually sound solution to the problems in type 1 experiments is to use "reference design", which requires co-hybridization of a common reference with all of the samples of the microarrays. Typically, the ratio (γ1) from cDNA: common reference is compared to another ratio (γ2) from cDNA: common reference. The computed "ratio of ratios" (γ1/γ2) is considered to be equivalent to direct cDNA: cDNA comparisons. In contrast to the type 1 approach, this "reference design" approach is called type 2 approach , in whicAn ideal reference should fulfill the criteria of universality, reproducibility and uniformity, meaning that it should be universal across diverse microarrays, reproducible over a long time frame and in different laboratories, and represents each gene at a uniform level. One kind of such references is common RNA pools assembled from a number of different cell lines, tissues and conditions. Commercial universal RNA references are now available for mouse and human samples (Stratagene). However, the RNA references fall well short of the aforementioned criteria. Although RNA pools are more comprehensive than a single source of RNA sample, it still partially represents the whole genome; there is inherent biological variability among different RNA samples; and RNA could be degraded over time. Therefore, data quality across multiple studies is inevitably compromised. To address these issues, genomic DNA has been proposed to replace universal RNA reference . It is eNevertheless, adopting genomic DNA as reference also creates new challenges. It is conceivable that though this strategy enables the integration of disparate studies, it brings in new variations. For example, spots with low signal intensity from labeled genomic DNA are prone to high standard errors for measurements, and spots with high intensity considerably interfere with the hybridization of cDNA samples to the probes, leading to low fidelity in the ratio of cDNA to genomic DNA. For quality control purpose, it is critical to identify these variances and remove ambiguous values by data analyses. However, to our best knowledge, so far this problem has not been unequivocally tackled and there is no consensus among the scientific community for the data analyses of microarray using genomic DNA reference. For instance, some researchers conducted array-to-array comparison with little data processing except for background subtraction and removal of poor or negative spots ,16, whilShewanella oneidensis, which was capable of respiring with oxygen, fumarate, trimethylamine-N-oxide (TMAO), manganese (IV) oxides and ferric oxides as terminal electron acceptors [S. oneidensis were generated under three growth conditions – aerobic growth or anaerobic growth with fumarate or ferric citrate as electron acceptor. Variations among gene expression profiles were compared and we concluded that data processing techniques, including setting minimal number of replicates, logarithmic (log) transformation and random error analyses, appeared to be valuable to improve data quality.In this study, we address this need by conducting a comparative study of type 1 and 2 experiments in a γ-proteobacterium cceptors -19. GeneShewanella oneidensis strain DSP10 under aerobic condition (O2), or under anaerobic conditions with fumarate (Fum) or ferric citrate (Fe) as electron acceptors. cDNA was subsequently transcribed and labeled with Cy5 or Cy3, and any pair of two conditions was co-hybridized on microarray slides, yielding three direct ratios, namely Fum/O2, Fe/O2 and Fe/Fum. Meanwhile, RNA from each condition was reversely transcribed and labeled by Cy5 and co-hybridized with Cy3-labeled Shewanella genomic DNA. To obtain expression ratios of Fum/O2, Fe/O2 and Fe/Fum, the ratios of cDNA: gDNA were calculated, and then the inferred (indirect) ratios were obtained by calculating the "ratio of ratios" as (cDNA1/gDNA) over (cDNA2/gDNA). Results from type 1 experiments were compared to those obtained in type 2 experiments. Two previous studies employed few data processing techniques except for basal ones such as background subtraction and removal of poor or negative spots [As indicated in the introduction, type 2 experiments using genomic DNA reference could add an additional layer of variance and hence impair data quality as compared to type 1 experiments. To evaluate it, we performed both type 1 and 2 microarray experiments. RNA was extracted from mid-logarithmically grown ve spots ,16. Ther2, 17 values for Fe/O2 and 8 values for Fe/Fum) fell into two opposite categories (induction vs. repression) , a powerful statistical approach to determine differences, was applied to selected genes. Table S. oneidensis [c-type cytochromes OmcA and OmcB exist as a complex on outer membrane and function to reduce extracellular Fe(III) and U(VI) as terminal electron acceptors. Their expression is induced for several folds under anaerobic conditions  of both approaches were increased under all pairs of conditions when the minimal number of slides was set higher, demonstrating that it significantly improves data quality at global level  of the replicates and their mean, logarithmic transformation is often conducted to remove a large portion of the relationship between the SD and mean. This approach is called proportional model. If there is no positive relationship between SD and mean, no log transformation should be applied and the data are analyzed in the raw form. This is called additive model. It is interesting to test whether applying logarithmic transformation has an impact in microarray analyses. Figure l Figure . To explNo measurement is entirely accurate. It is hence important to estimate the amount of measured value that could randomly deviate from the true value. This technique is called random error analyses or uncertainty analyses. A series of repeated measurements are usually used to make a reasonable estimate. The repeated measurements include biological and technical replicates. It is necessary to take random error into account to determine the significance of the results. One method, called small sample method, estimates random error on the replicates of individual genes, regardless of all of the other genes in the array. In contrast, random error could also be estimated on the entire array because this might be more accurate than estimation of small number of replicates for individual genes. A common error approach makes the assumption that the SD of replicates is unrelated to mean signal intensity. Alternatively, a curve fit approach could recognize the relationship between SD and mean by a regression line (curve fit).r values  were removed from each of Fum/O2, Fe/O2 and Fe/Fum comparisons. If minimal number of replicates is not used, outlier removal has a slight impact on data quality (data not shown).Outliers are data points that are not faithfully reproducible among replicates, yet have a disproportionately large effect on the average values. Thus removal of outlier from source file is likely to improve data quality. Hence we compared results with and without outlier removal. Two criteria, namely We also tested the effect of flooring. Low signals in direct comparison of RNA samples often produce spurious expression ratios, thus signals below a certain threshold level are often set to the threshold level ,24. To tz-test or t-test) and biologically meaningful. Thus two-fold is used here as a general guideline to simplify our study.It is often desirable to compare results from any two experimental conditions in microarray studies. Type 2 microarray experiments using a common reference such as genomic DNA allows for inter-conditional comparisons. However, the reliability of the comparison is often questionable since microarray is notorious for its considerable fluctuation of signals. In this report, we test a variety of data processing techniques in order to improve data quality in type 2 experiments. Two criteria are used to evaluate these techniques: by (1) correlation coefficient to the corresponding type 1 experiments; and (2) classifying differential expression values of genes into "up", "down" and "constant" categories, and then focus on genes in opposite categories. The first criterion evaluates the impact of techniques at the whole-genome level, while the latter addresses the most inconsistent data. Two-fold was used as threshold to classify the categories, which was reported to be a solid benchmark for induction or repression of gene expression . HoweverOne important prerequisite in our study is that the results from type 1 experiments are more reliable than those from type 2 experiments. One way to comprehend it is to analogize with triangle inequality relation for metric spaces: errors of an indirect path should be no less than errors of a direct path. To date, the reliability of data in type 1 experiments has been extensively studied. It has been estimated that over 90% of the results could be verified by other techniques such as quantitative reverse transcription PCR or northern blot . TherefoAmong data processing techniques, the minimal number of replicates seems to be a critical one to filter out inconsistency between the results of type 1 and 2 experiments, as demonstrated in Figure A large number of data processing strategies are available for microarray data analyses. In this short report, we could only examine several of them. Moreover, caution should be taken to extend conclusions from our study to other microarray experiments. It is likely that some of our conclusions would not hold for specific microarray datasets. Nevertheless, since we show here that data analysis process has a significant impact on the reliability of the results in type 2 experiments, it is thus advisable for researchers to evaluate their data processing techniques carefully when genomic DNA or another common reference is used in microarray experiments.Shewanella oneidensis whole-genome microarray was constructed as described previously [Shewanella oneidensis. It is thus of interest to catalog DSP10's gene expression.eviously . Strain 2O, 3.33% SSC (Ambion), 0.33% sodium dodecyl sulfate (Ambion), and 0.8 μg/μL bovine serum albuminin (New England Biolabs). Slides were hybridized at 50°C over night with Cy5- and Cy3-labeled probes in the above solution, minus BSA and with the addition of 0.8 μg/μL herring sperm DNA (Invitrogen) to prevent random binding. Pre-hybridization and hybridization were carried out in hybridization chambers (Corning). Slides were then washed on a shaker at room temperature in the following order: 7 minute in 1 × SSC, 0.2% SDS; 7 minute in 0.1 × SSC, 0.2% SDS; and 40 second in 0.1 × SSC. For type 2 experiments, 100 ng S. oneidensis genomic DNA (gDNA) was amplified by incubated at 37°C for 3 hours using Klenow fragment of DNA polymerase (Invitrogen) and random primers followed by transferring on ice to stop the labeling. Cy3 dUTP was incorporated in the product (Amersham Biosciences) and then Cy3-labeled genomic DNA was co-hybridized with Cy5-labeled cDNA on pre-hybridized microarray slides as described above. A total of twelve replicates were prepared for both approaches. A program ImaGene version 5.5 (Biodiscovery) was used to grid and quantify microarray images. Background signals around each spot were calculated and subtracted from the signal intensity of each spots. Spots of Signal/background ratios < 3 were regarded as negative spots. All negative, poor and empty spots were flagged and discarded.DSP10 was grown aerobically in 100 ml Luria-Bertani medium  to mid-logarithmic phase at 30°C. Alternatively, DSP10 was grown anaerobically to mid-logarithmic phase in 200 ml LB liquid supplemented with 20 mM lactate, and with either 10 mM fumarate or 10 mM ferric citrate as electron acceptor. Mid-logarithmic phase was determined by measuring the turbidity at 600 nm in a spectrophotometer for aerobic or anaerobic 10 mM fumarate cultures, or by epifluorescence microscopy using acridine orange staining  for anaeS. oneidensis was carried out as previously described [p < 0.05. The significance of differential expression was determined by two-way t-test.Data analysis of type 1 experiments in escribed . Quantif2/T1 = (T2/R2)/(T1/R1), where T and R represented the mean value of cDNA and genomic DNA reference signals from all of the twelve replicates, respectively. To evaluate selected data processing techniques, data were processed in the same order as type 1 experiments, with change in only one parameter for each time. Certain parameters were specified as: floor of 20 and normalization by mean, data with less than the minimal number of replicates of 4 were removed, and then followed by execution of proportion model and small sample model. Then outlier was removed by p < 0.05 and finally, expression ratios were obtained by calculating the division of two ratios.Local background subtraction and flagged spots removal were implemented in the same way as type 1 experiments. If no other data processing technique was used, inferred ratio was calculated by Tr) was computed between results acquired from type 1 and 2 approaches. To obtain the number of genes in opposite categories, two-fold change was used as criterion. We consequently categorized the differential expression values into three classes: "up" for expression ratios of more than 2, "down" for expression ratios of less than 0.5, and "no change" for all other ratios. A gene was considered to be in opposite categories if its expression ratio was classified as "up" in type 1 experiment and "down" in type 2 experiment, or vice versus. For one-way ANOVA, the logarithmic transformation was applied to the ratio values to normalize the expression variation among genes and equalize the data scale intervals for ANOVA test. The significance level of p value < 0.05 was used as criterion to reject or accept the null hypothesis "the two reference methods are not significantly different from each other".Pearson correlation coefficient (The authors declare that they have no competing interests.YY contributed to the experimental design, conduction and analysis of the microarray experiments and manuscript writing. MZ and LW contributed to the evaluation of microarray data processing techniques. JZ contributed to the experimental design and manuscript revision. All authors read and approved the final manuscript.


Molecular signatures are sets of genes, proteins, genetic variants or other variables that can be used as markers for a particular phenotype. Reliable signature discovery methods could yield valuable insight into cell biology and mechanisms of human disease. However, it is currently not clear how to control error rates such as the false discovery rate (FDR) in signature discovery. Moreover, signatures for cancer gene expression have been shown to be unstable, that is, difficult to replicate in independent studies, casting doubts on their reliability.We demonstrate that with modern prediction methods, signatures that yield accurate predictions may still have a high FDR. Further, we show that even signatures with low FDR may fail to replicate in independent studies due to limited statistical power. Thus, neither stability nor predictive accuracy are relevant when FDR control is the primary goal. We therefore develop a general statistical hypothesis testing framework that for the first time provides FDR control for signature discovery. Our method is demonstrated to be correct in simulation studies. When applied to five cancer data sets, the method was able to discover molecular signatures with 5% FDR in three cases, while two data sets yielded no significant findings.Our approach enables reliable discovery of molecular signatures from genome-wide data with current sample sizes. The statistical framework developed herein is potentially applicable to a wide range of prediction problems in bioinformatics. More complicated parametric forms such as polynomials in xi could be used in a similar way, although the number of weights would increase accordingly.where wi is unknown, we used a bootstrap technique to test whether wi] = 0. By sampling with replacement from the given data set and re-training the classifier on each sample, we obtain B vectors i, the corresponding wi. This interval is inverted to obtain a bootstrap p-values pi for each variable i  confidence interval does not cover zero). Importantly, this procedure preserves the full dependency structure of the data distribution. Finally, FDR control was performed using the Benjamini-Hochberg procedure  = 0 is technically correct only in the limit of large samples where wi] → p-values, satisfying the inequality P(p ≤ α) ≤ α + l), where l is the sample size [l) was negligible in our simulations, this should be verified in each particular case before applying bootstrap testing. A possible future improvement could be to estimate this term from simulations and correct the bootstrap p-values accordingly, thereby "calibrating" the method.Some technical issues remain to be considered. First, testing the null hypothesis ple size . While tAs we have shown, neither predictive accuracy nor stability constitute valid measures of FDR for molecular signatures. Indeed, highly accurate predictions may be obtained despite an FDR as high as 50% Figure , while iWe have developed and validated a statistical hypothesis testing framework that for the first time provides false discovery rates control for signature discovery. In application to cancer gene expression, we have showed that reliable signature discovery is feasible with currently available sample sizes. Many important problems in bioinformatics are prediction problems and may benefit from reliable signature discovery. We therefore hope that our method will be of general interest.S, S' derived from independent, replicate experimental data sets,Signature stability is defined as the normalized expected overlap between two signatures where f (x | y) = N  with equal class frequencies, covariance matrix Σ independent of the class (phenotype) variable y and varying degrees of class separation to achieve different effect sizes. Results were averaged over 100 randomly selected Gaussian distributions. for each parameter setting tested. We measure the effect size of the resulting prediction problem by the expected SVM accuracy. Here the accuracy was computed exactly for each SVM from the data density: for any given μ and Σ, a separating hyperplane with normal vector w has classification accuracySimulations were performed with data drawn from two-class multivariate Gaussian distributions where f (x | y) = N , the optimal separating hyperplane has normal vector w* = Σ-1 μ, and so the optimal set S* can be determined as the nonzero components To evaluate signature error rates, we used the fact that for B = 50 repetitions, fitting a Gaussian distribution N  to the observed p-values. In preliminary studies, the difference between this method and a nonparametric bootstrap with B = 1000 was negligible, while the parametric version is computationally more efficient since a much smaller B can be used. The SVM [C-parameter and the KFD regularization parameter were set to 1. Recursive Feature Elimination (RFE) was performed as previously described [For hypothesis testing, we used a parametric bootstrap with  The SVM , KFD [19 The SVM  methods escribed , using tescribed  as accuret al. [et al. [Microarray data sets -5 were pet al.  and Alon [et al.  which we [et al. -29 were Since many data sets were had low minor class frequencies are Table , perform+ and Acc- are the accuracy measures for each class. Except for the independent test sets, these were measured by cross-validation, where in each round a randomized set consisting of 2/3 of the samples was used for training, and the remaining 1/3 was used for testing. Splits were balanced so that class frequencies were equal between training/test data. Mean and standard deviation of the balanced test error over 50 cross-validation repetitions are reported.where AccRN, JB and JT designed research; RN performed research; RN and JT wrote the paper.Proofs. This document provides proofs of uniqueness and optimality of the optimal signature S*.Click here for fileKFD and WV methods, and convergence with increasing sample size. This figure shows the results corresponding to Figure Click here for fileGene signature for the Alon data set. Excel file detailing the gene signature discovered by the bootstrap method using the SVM classifier. The corresponding signature from Recursive Features elimination is also provided for reference.Click here for fileGene signature for the Golub data set. Excel file detailing the gene signature discovered by the bootstrap method using the SVM classifier. The corresponding signature from Recursive Features elimination is also provided for reference.Click here for fileGene signature for the Singh data set. Excel file detailing the gene signature discovered by the bootstrap method using the SVM classifier. The corresponding signature from Recursive Features elimination is also provided for reference.Click here for file