Human tissue displays a remarkable diversity in structure and function. To understand how such diversity emerges from the same DNA, systematic measurements of gene expression across different tissues in the human body are essential. Several recent studies addressed this formidable task using microarray technologies. These large tissue expression data sets have provided us an important basis for biomedical research. However, it is well known that microarray data can be compromised by high noise level and various experimental artefacts. Critical comparison of different data sets can help to reveal such errors and to avoid pitfalls in their application.We present here the first comparison and integration of four freely available tissue expression data sets generated using three different microarray platforms and containing a total of 377 microarray hybridizations. When assessing the tissue expression of genes, we found that the results considerably depend on the chosen data set. Nevertheless, the comparison also revealed statistically significant similarity of gene expression profiles across different platforms. This enabled us to construct consolidated lists of platform-independent tissue-specific genes using a set of complementary measures. Follow-up analyses showed that results based on consolidated data tend to be more reliable.Our study strongly indicates that the consolidation of the four different tissue expression data sets can increase data quality and can lead to biologically more meaningful results. The provided compendium of platform-independent gene lists should facilitate the identification of novel tissue-specific marker genes. Tissue in the human body shows a fascinating variety of structures and functions ranging from simple protection by the epidermis to complex information processing in the cortex. All these different types of tissues have in common that they are highly adapted to their specific task. How such variety can emerge from the same genetic code is not only scientifically an intriguing question, but also of fundamental medical relevance. To address this question, a necessary step is the generation of comprehensive catalogs of gene expression in different tissues.Microarrays have become highly suitable tools for such an ambitious endeavour as they allow the genome-wide measurement of transcription abundance. Most early gene expression studies of human tissues were disease-orientated, i.e. the expression profiles of pathogenic tissues were compared to corresponding normal tissues. For instance, microarrays have extensively been used to detect previously unrecognized cancer subtypes ,2 and toHowever, general concerns about the reliability of microarray data have been raised by several studies showing poor congruence of different microarray platforms -12. MoreThis motivated us to critically compare and to consolidate four major microarray data sets of human tissue expression due to their outstanding importance for functional genomics and large-scale systems biology. Notably, although these data sets have rapidly become popular resources in biomedical research -19, thisThe outline of the study was the following: After compilation of the tissue expression data sets, we first examined their overall similarity. For the assessment of tissue specific expression, we generated multiple gene lists utilizing complementary measures to detect distinct features of tissue expression. These lists were subsequently used for a critical comparison of results derived from the different experiments. Finally, we assembled lists of platform-independent tissue-specific genes. The assessment of the consolidated data indicated that they can provide a more reliable basis for the study of tissue expression.Methods section. The four data sets were produced using three different microarray platforms: Agilent spotted oligonucleotide microarrays , AffymeRosetta1 ), and cDRosetta1 ). Table Methods section and the Additional file Supplementary Materials).For comparative analysis, the microarray probes were mapped to their corresponding Entrez Gene identifiers to obtain a common index system. Expression values of replicated probes were averaged. To facilitate the comparison, the samples were assigned to 19 main tissue classes based on their physiology and histology. Subsequently, the expression values were averaged over samples belonging to the same tissue class. Further details of the applied data pre-processing and the assignments of samples to tissue classes can be found in the First, we examined whether the four different data sets show similar gene expression profiles. This comparison was based on 6685 non-redundant genes that were common to all data sets. To assess their global similarity, we calculated the correlations for all pairs of genes within a data set and correlated them between different data sets. This so-called correlation of correlation measure was previously proposed to examine the overall similarity of expression patterns in two different microarray data sets . Large c-16).These results are supported by the analysis of the differential gene expression for specific tissue types. Figure A main purpose of the four tissue expression data sets has been the identification of tissue-specific and housekeeping genes. Does the observed divergence influence the detection of such genes? To address this question we used two different measures to capture distinct features of tissue-specific expression.Methods section. Since the data distributions differ considerably between the compared microarray experiments, the permutation procedure was applied to each microarray set separately. Figure First, we scrutinized the expression data with respect to tissue-specific over- or under-expression. For this task, we used the Preferential Expression Measure (PEM) originally proposed by Huminiecki and co-workers . The PEMunique tissue type as they can serve as suitable candidates for marker genes. Thus, we introduced an alternative measure which addresses this aspect: First, we determined the tissue for which the gene is maximally over-expressed. Subsequently, the difference between the largest and second largest value for expression was calculated. This difference was termed MAX. A gene with a large MAX value displays high gene expression in one tissue but not in the other tissues, i.e. it is uniquely over-expressed in a single tissue. The corresponding MAX scores for remaining tissues  were set to zero. We determined the significance of the MAX scores using the same permutation-based procedure as described for the PEM scores. Figure Since PEM is defined with respect to the average expression, a gene can obtain high PEM scores for multiple tissues. However, researchers are frequently interested in genes that are over-expressed in a uniquely over-expressed in a given tissue if it is maximal expressed in this tissue and FDR < 0.25 holds for the corresponding MAX score. Note that we have chosen a rather high FDR, but the main conclusions of our comparison also hold for more stringent values  remains the same as for the full data set with the original FDR threshold. As before, we selected randomly brain samples, constructed reduced data sets and calculated the significance of genes. For calibration, the number of selected brain samples was set equal to the number of samples found in the other tissue classes. As result, we observed that the FDR threshold had generally to be increased for tissue classes including fewer samples , as most other previous studies employing these resources have also used the readily available pre-processed data and our analyses should be applicable for the interpretation of these studies. However, it is well known that normalization and pre-processing procedure can have considerable influence on the results of microarray data analyses. Also, it has been shown that re-annotation of Affymetrix GeneChips can improve the quality of derived transcriptional data. Thus, we examined whether alternative data pre-processing procedures may improve the concordance of the microarray data sets. For the Geneatlas dataset, we used an alternative annotation and normalization scheme as described in the The conducted comparison revealed a substantial influence of the microarray platform on the composition of the constructed gene lists. As such platform-dependent bias can compromise the data quality; we reasoned that a consolidated list of common tissue-specific genes might yield more reliable results. This list could be simply generated by direct intersection of lists derived from the different experiments. However, Figures Genes obtaining large MAX scores in all four data sets should be strong candidates for reliably being tissue-specifically over-expressed. We assessed this hypothesis by examining in detail brain- and liver-specifically over-expressed genes. For integrative scoring, we first ranked the genes in the four data sets according to their MAX values and calculated their mean ranks, which was subsequently used for generated consolidated lists. The follow-up analysis of the resulting gene lists demonstrated that this simple scheme is remarkably efficient for a reliable identification of tissue-specific genes.th in the consolidated list, is a crucial protein for the myelination of axons in the central nervous systems. In contrast, it was ranked relatively low (as 100th) in the Stanford dataset. Similarly, enolase 2 is a neuron-specific enzyme in the glycolytic pathway and is used as marker [th, whereas it was ranked considerably lower in two datasets . This suggests that bona fide markers can be detected by consolidation despite having a less prominent position in single data sets.For brain tissue, the 20 top-ranking genes were individually inspected and the vast majority proved to be previously known as brain-specifically or preferentially in brain expressed genes Table . Most ofs marker . In the For liver tissue, the top-ranking genes are listed in Table The inspection of the derived consolidated lists for brain and liver indicated that multiple confirmations led to accurate identification of tissue-specific genes. Since the comparison of tissue-specific genes pointed to a considerable platform dependency, another positive aspect of the proposed integration method would be a reduced dependency, or respectively, an increased reliability of the detected gene lists. Clearly, the derived consolidated gene lists show reduced platform dependence. But is this simply the consequence that the MAX scores from all compared platforms were merged, or can the proposed integration method still reduce the platform dependency of gene lists when compared to data which were not included in the integration? To assess whether the latter is the case, we conducted a cross-validation. We examined whether the consolidated MAX scores for brain tissue derived from three integrated data sets show a higher correlation with the MAX scores of the fourth data set than with the MAX scores of the three individual data sets. This procedure was performed for all four microarray experiments.Strikingly, the consolidated gene lists showed considerable larger correlation with the independent gene list than expected based on the observed correlation between the single data sets Figure . For insBesides enhancing the data reliability, do the consolidated gene lists also improve the functional characterization of tissue types? To address this question, each tissue type was represented by their corresponding specifically expressed genes. Subsequently, these lists were associated to biological processes described in Gene ontology (GO). Using Fisher's exact test, the significance of enrichment of the over-expressed genes in a biological process was derived for each tissue type. To facilitate its evaluation, we restricted the GO analysis to 18 so-called informative GO categories. For functional characterization, tissues types and GO categories were hierarchically clustered based on the significance of enrichment as similarity measure. This functional characterization allowed us to compare the reliability of the consolidated gene list with the gene lists obtained from single data sets. In Figure Assessment of the results showed that we obtained biologically more reasonable results for consolidated lists. For example, pancreas tissue was strongly associated with the GO category "proteolysis", which corresponds to its exocrine function. Similarly, liver tissue was strongly associated with "metabolism", which is known to be the most prominent function of the liver Figure . In contMicroarrays have become widely used tools to measure gene expression for the prediction of disease outcome, the profiling of new disease subtypes and the identification of disease-specific markers -4. To acRemarkably, we found that the detected tissue-specific expression is strongly influenced by the choice of microarray platform. The observed divergence could have multiple reasons. Foremost, the set of tissue samples was not the same for the compared experiments. Thus, differences would be caused by biological variation. Furthermore, experimental factors such as difference in probe sequences, RNA handling, hybridization protocols, scanning, and image analysis are likely to contribute to the observed divergence. In our study, we applied some alternative pre-processing schemes to improve the concurrency of the data sets. Although these schemes did not yield a larger overlap of tissue-specific genes, we anticipate that improved pre-processing procedures can support the consolidation of the data generated by different microarray data sets. Especially, a more accurate annotation of probes (i.e. their mapping of corresponding genes) will help to improve the comparability of microarray platforms ,27. ThusDespite the differences, however, all four data sets show a larger similarity than expected by chance. This similarity provided us the basis for the consolidation of the different expression experiments. In fact, dissimilarity of expression patterns can be used to eliminate spurious expression patterns. It has recently been demonstrated that the expression of genes displaying a similar pattern for different platforms tends to have an increased reproducibility with respect to external validation by qRT-PCR . Thus, cTherefore, we propose the integration of multiple tissue expression data sets. We generated lists of tissue-specific genes for each single data set and integrated them to a single consolidated list. Naturally, data integration would not be beneficial if any of the data sets to be integrated is unreliable. However, the re-analyses as well as the original studies strongly demonstrated that each of the data sets shows convincible overall reproducibility. Nevertheless, we observed biologically more accurate results for the integrated list in contrast to lists derived by one experiment only in an exemplary GO analysis. Additionally, we constructed a compendium including tissue expression data for 18909 non-redundant genes and scores for tissue specificity to allow researchers their own analysis. The consolidation is based on simple, but yet effective scoring methods. We would like to note that the rather stringent requirements can be relaxed using the compendium. For example, we would find a considerably larger overlap of tissue-specific genes if we only require that they are shared between two or three data sets. Therefore, researchers using the compendium are encouraged to adjust the thresholds to their own requirements. To facilitate such further analysis our compendium provides the complete data sets with the specificity scores. The compendium will be particularly beneficial for analyses involving a large number of genes. A possible application might be the construction of tissue-specific networks which have attracted increased attention in systems biology ,30. UsinIn conclusion: Although various studies have compared disease-related microarray experiments, this analysis constitutes the first systematic comparison of large-scale microarray data sets of normal human tissues. We detected a prominent platform bias, which however can be overcome by data integration. Thus, an important contribution of this study is in the compilation of platform-independent tissue-specific lists which will be provided freely. We anticipate that they will be of great asset for biomedical scientists. Finally, we hope that our study provides a solid basis for the future wide-spread use of microarray tissue expression data in genomic research.The data sets Rosetta1  and Roseet al. [Shyamsundar et al.  measuredet al. ,33. Optiet al. [Su et al.  created http://www.xlab.unimo.it/GA_CDF/) and the GC-RMA normalization method for the calculation [Expression summaries had been derived using the Affymetrix Microarray Suite 5 (MAS5) algorithm with global median scaling. The MAS5-processed data were downloaded from the Geneatlas webpage. To improve the concordance with the other data sets, we also calculated expression levels using an alternative annotation and normalization scheme. Starting from the CEL files (downloaded from NCBI GEO repository - Series GSE1133), we used an customized CDF annotation file for the U133A GeneChip .The obtained results were used for hierarchical clustering to evaluate the similarity of expression patterns in the compared data sets. As distance measure, we chose 1-ti can be defined as:We used the previously introduced Preferential Expression Measure (PEM) to identify genes that are over- or under-expressed in a specific tissue . PEM desS is the average expression of a gene in the specific tissue class ti and A is the arithmetic mean expression of the gene in all tissues . To derive the significance of observed PEM scores, a permutation-based procedure was employed. Background distributions of PEM scores were generated based on repeated (N = 100) random association of the same number of arrays to a tissue class as observed for the original data set. Notably, this procedure conserves the number of arrays included in a specific tissue class. Subsequently, PEM scores were calculated separately for each tissue class and each data set. As measure for statistical significance, we used the false discovery rate (FDR). It is defined here as the expected proportion of false positives among all genes detected as tissue-specifically expressed. We can derive an empirical false discovery rate for a chosen PEM score s:where PEMo and PEMb are the scores calculated for the original and randomized data sets, N is the number of randomization (N = 100) and δ(x) = 1 for x ≥ 0, respectively δ(x) = 0 for x < 0. Thus, the significance of the observed PEM scores can be derived by comparison with the generated background distributions [where ibutions . For ourThe uniqueness of tissue-specific over-expression was evaluated by comparing the fold changes of the two tissues with the highest gene expression for each gene. We called this measure MAX, which can be defined as:fc(t) comprises the fold changes in all tissues t compared to the mean expression and fc(t') comprises the fold changes in all tissues except the one with the highest fold change of this gene, respectively. Note that this MAX score is assigned to a gene only for the tissue class with the largest expression; all other MAX scores of the gene are set to zero for the remaining tissue classes. A gene with a large MAX value shows high gene expression in one tissue but not in the other tissues, i.e. it is uniquely over-expressed. To assess the significance, the same permutation-based procedure as applied to PEM scores was employed. In our study, genes with a FDR < 0.25 were denoted as uniquely tissue-specifically over-expressed. The derived MAX and PEM scores as well as their corresponding FDR for different tissues and data sets can be found in Additional file where N which were significantly over-expressed in brain in a chosen data set based on a FDR threshold of α . Second, a sub-set of s brain samples were randomly selected and the remaining non-selected brain samples were subsequently excluded from the data set. PEM scores for brain-specific over-expression and corresponding FDR were derived as previously described for this reduced data set. The FDR threshold for significant brain-specific over-expression was adjusted to a level α' so that the number of significant genes was the same as detected number N for the full set using a FDR threshold of α. The procedure was repeatedly performed (n = 103) generating a distribution of adjusted thresholds. As a conserved estimate for adjustment of the FDR threshold, the 5th percentile was chosen. The size s of the sub-set was set to values that reflected the number of observed samples in different tissue classes . For those genes we calculated the Spearman rank correlation between each of the four data sets. For cross-validation we consolidated three of the four data sets for each of the four possible combinations. For assessment of reliability we calculated the Spearman rank correlation of the consolidated gene lists and the gene list set not included in the consolidation and compared this value to the mean of the correlation values for the gene lists set based on individual data sets.informative GO categories: Each GO category should contain more than 300 genes and each of their child categories should contain less than 300 genes [We utilized gene annotation by Gene Ontology to assign tissue types to biological processes. Here, we assessed whether over-expressed genes in a given tissue type tend to be associated with specific biological processes. First, we calculated the enrichment of tissue-specific genes based with a PEM > 0.8 in biological process categories. The obtained scores for enrichment can subsequently be used to assign weighted profiles of biological processes to the given tissue type. A well-known difficulty, however, is the selection of the suitable set of GO categories, as many of categories are highly specific. Here, we aimed to select only those categories that include enough genes for adequate statistical validation but not too many genes to ensure the functional homogeneity of member genes. Following a previously proposed scheme, we chose therefore so-called 00 genes . The sig00 genes . A smallJR assembled the data sets, performed the comparative analysis and prepared the original draft of the manuscript. MF conceived of the study and guided the gene expression analysis and the interpretation of the results. Both authors read and approved the final manuscript.Supplementary materials including tables and figures (pdf).Click here for fileExpression values of Rosetta1 data set after merging to the 19 main tissue classes (tab-delimited table).Click here for fileExpression values of Rosetta2 data set after merging to the 19 main tissue classes (tab-delimited table).Click here for fileExpression values of Geneatlas data set after merging to the 19 main tissue classes (tab-delimited table).Click here for fileExpression values of Stanford data set after merging to the 19 main tissue classes (tab-delimited table).Click here for fileMAX scores derived for the different expression data sets and tissue classes and the corresponding FDR for the 6685 non-redundant genes included in the comparison (tab-delimited table).Click here for filePEM scores derived for the different expression data sets and tissue classes and the corresponding FDR for the 6685 non-redundant genes included in the comparison (tab-delimited table).Click here for fileMAX scores and corresponding FDR derived for the Geneatlas data set after application of alternative annotation and normalization (tab-delimited table).Click here for file


Publicly available datasets of microarray gene expression signals represent an unprecedented opportunity for extracting genomic relevant information and validating biological hypotheses. However, the exploitation of this exceptionally rich mine of information is still hampered by the lack of appropriate computational tools, able to overcome the critical issues raised by meta-analysis.This work presents A-MADMAN, an open source web application which allows the retrieval, annotation, organization and meta-analysis of gene expression datasets obtained from Gene Expression Omnibus. A-MADMAN addresses and resolves several open issues in the meta-analysis of gene expression data..A-MADMAN allows i) the batch retrieval from Gene Expression Omnibus and the local organization of raw data files and of any related meta-information, ii) the re-annotation of samples to fix incomplete, or otherwise inadequate, metadata and to create user-defined batches of data, iii) the integrative analysis of data obtained from different Affymetrix platforms through custom chip definition files and meta-normalization. Software and documentation are available on-line at  Public databases of microarray gene expression data have been quickly growing as the use of high-throughput techniques has become routine in genome-wide studies. Major repositories of microarray data, e.g. Gene Expression Omnibus , ArrayExad-hoc normalization step of the raw data files.In recent years, different strategies to combine results from independent but related studies have been proposed. The choice of the most effective meta-analysis technique depends on the type of response  and on the objective of the study. Meta-analysis strategies can be divided into two broad classes: data integration and data combination. Statistical techniques as vote counting , p-valueDespites numerous efforts, mining and analyzing publicly available microarray data still represents a bioinformatics challenge and the lack of appropriate tools able to overcome critical issues, as annotation, cross-platform comparison and handling of metadata, is still hampering the potentialities of large-scale meta-analyses. Performing a meta-analysis of independent microarray studies requires to carefully handle the heterogeneity of array designs, which complicates cross-platform integration, and of sample descriptions, which impact the correct characterization of specimens. At least for the case of Affymetrix arrays, cross-platform comparison has partially been solved by the adoption of custom Chip Definition Files (custom-CDF) which, relating probe sequences to annotated entities as genes or transcripts, allow matching expression profiles across subsequent generations of microarrays -11. InstGEOImporter) for importing single GEO Data Sets or Series [Different computational tools have been proposed for retrieval and meta-analysis of microarray expression data -21. In pr Series ; WGAS isr Series ; GEOquerr Series ; EzArrayr Series ; Microarr Series ; EMAAS aThe purpose of this paper is to present A-MADMAN, an open source web application for the meta-analysis of Affymetrix data contained in Gene Expression Omnibus (GEO). A-MADMAN allows retrieving, annotating, organizing and analyzing gene expression datasets. In particular, A-MADMAN addressees several of previously stated open issues in the meta-analysis of gene expression data allowing the integrative analysis of data obtained from different Affymetrix platforms through custom chip definition files and meta-normalization and the sharing of analysis flows and results.ExpressionSet) which contains the integrated expression levels and all available metadata and sample characteristics. The gene expression data are obtained through a meta-analysis approach which includes signal generation, probe re-annotation into gene-centered identifiers, merging of expression levels from different experiments and a normalization step. A-MADMAN generates an ExpressionSet object in which the meta-expression levels from multiple experiments are completed by GEO-derived and user-defined metadata. The final ExpressionSet contains all the necessary information to perform, directly in R, any higher level analysis  of all downloaded and integrated data.A-MADMAN is a web application that allows retrieving gene expression datasets from GEO, annotating and locally organizing the downloaded samples, and generating an R object  to which one or many samples refer to;Tag: a free-text, descriptive label assigned to any sample and used for sample selection and grouping. The tags menu allows managing and creating tags to annotate/describe samples, through a labeling system whose complexity depends on the project, on the biological question and on the type of analysis.Basket: a named collection of samples obtained making a logical query on tags.Series and Samples, automatically downloaded from GEO, are imported into A-MADMAN and organized in a file system hierarchy. Sample characterization can be further enriched by user-defined annotations and tags. The tag based annotation system is central to A-MADMAN meta-analysis philosophy and can be used to:Data and metadata of selected i) exploit GEO metadata information, although incomplete, inaccurate and not conform to a controlled vocabulary. The A-MADMAN tag system allows the user to define his own vocabulary and consistency is enforced by the software itself (i.e. new tags must be defined and described before usage);ii) test new hypotheses not envisioned by the original authors enriching the sample annotations with information derived from bibliographic references or from other sources.Samples can be grouped in user-defined data baskets on the basis of logical queries performed on tag information ' allows populating a Basket with only those samples derived from young, male patients affected by any muscular dystrophy different from the Becker's or the Erb's ones. Figures  and 3. ABaskets can contain Samples from different platforms. Gene expression signals generation, platforms integration and normalization are then performed on Baskets according to the basic analysis workflow shown in Figure Create a custom Tags system and use it for tagging series/samples. The tag annotation system is based on the conceptual model of bottom-up classification, recently exploited by many social network web sites to organize and search data not amenable to enter a predefined and unambiguous taxonomy. This model enables the user to create a project-tailored, controlled, but unconstrained vocabulary to describe biological or technical characteristics of samples, which may be significant and informative according to the specific scopes of a meta-analysis.The labeling system allows creating data structures which are tailored to the subsequent meta-analysis. For instance, samples can be tagged to identify differentially expressed genes from two-class or multi-class comparisons or may be labeled to generate a large data matrix, accounting for different biological contests, when the goal is to detect co-expressed genes. Practically, the user can Sample to an Individual in two ways, i.e. through an automatic assignment  or using a knowledge-based assignment. In the latter case, the user-friendly interface allows matching samples and individuals according to the information derived from GEO or from original references or project websites.One of the most critical problems in meta-analyses is originated by the inexplicit relationship among individuals/patients and samples. The granularity of Affymetrix-based gene expression data in GEO is represented by .CEL files but, depending on the platform, the RNA of a single individual may have been hybridized to more than one array, i.e. may be associated to more than a single .CEL file. A typical example of this one-to-many association between individuals and .CEL files is represented by the HG-U95 and HG-U133 sets with up to 5 or 2 possible raw data files for each biological sample, respectively, and a given .CEL file is not directly associated to a patient/sample ID. Said that, A-MADMAN interface considerably facilitate the matching of individuals to .CEL files and metadata, which can be inferred by combining metadata with information available in other formats and repositories, e.g. in a publication, in a supplementary information file, in a study-dedicated web site. A-MADMAN can relate each Extract samples and put them in a basket. The baskets represent the starting point of the subsequent analysis, i.e. the objects from which expression profiles are generated and information for the supervised comparisons are retrieved. Once selected one or more available baskets, all analyses will be conducted automatically, following a workflow selected by the user.Once samples are tagged and assigned to individuals, the user may set up a query, based on a Boolean combination of annotation tags, to select and Analyses of A-MADMAN allows viewing analyses status, debugging information, logs and results, while the Administration action allows the administration of users, groups and projects, according to the allotted permissions. Analyses are conducted by the R backend powered by packages of the Bioconductor project [The action  project , downloaUser can create custom workflows, editing a workflow template, and share them within a Group. This feature makes easily customizable and expandable the analysis flow. The basic analysis workflow comprises the following steps:A-MADMAN contains a standard analysis workflow which may be applied to one or more baskets, but any 1. creation of a work directory named after the current analysis and copy of the selected raw data files to the work directory;2. signal reconstruction using RMA algorithm ;ExpressionSet containing expression profiles and metadata;3. generation of an 4. integration of different chip platforms trough matching of platform-dependent probesets to integrated probesets.ExpressionSet with quantile distribution transformation [5. normalization of the final ormation .cdf_flavour, i.e. the type of CDF used in the signal reconstruction (the default is GeneAnnot-based CDFs), the signal_reconstruction, i.e. the algorithm and parameters for signal reconstruction (the default is RMA with standard parameters), and the additionalcode block containing code to be run after the platform integration phase . Blocks are written mixing R code [The standard workflow is implemented as a template with three blocks of code that can be modified or replaced by the user, through the web interface, to define custom workflows. Specifically, the user can customize the g R code  and someg R code  and mustA-MADMAN supports different Affymetrix platforms and the integration of data obtained with subsequent generations of chips is achieved through the annotations of several, gene-centered CDF packages [see Additional file ExpressionSet object, i.e., into the Bioconductor class that contains and describes microarray expression level assays.If the data baskets under consideration are composed by samples obtained from more than one platform, RMA will be carried out separately for each platform on all those samples which are homogeneous with respect to the selected CDF Figure . Gene exThe annotations of the selected CDFs are used to generate a non-redundant list of gene-centered identifiers, each derived from one or more platform-specific probesets/metaprobesets. This list constitutes the backbone grid of the integrated data matrix whose values will be composed from gene expression signals derived from the various samples. Since each platform represented in the baskets will contribute expression data according to a list of platform-CDF dependent probesets (or metaprobesets), the values of the integrated data matrix will be obtained averaging the expression levels of the various platform-specific probesets/metaprobesets Figure . It's woAccording to the adopted CDF and to the combination of platforms in the basket, a fraction of the gene identifiers in the integrated data matrix may include some <NA> values. Since expression vectors including some <NA> values may represent a problem for the subsequent analyses, the user may choose to fill empty cells using a specific function, e.g. SAM imputes missing values via a K-Nearest Neighbor algorithm. In the case that a given gene is not represented in any platform of the baskets, i.e. if the data matrix has an entire row of <NA>, the record will be pruned so that the final integrated expression matrix contains only non-empty expression vectors.Finally, the integrated data matrix is normalized to remove experiment-dependent biases. This normalization procedure is a crucial step to limit misleading outcomes due to the direct integration of different datasets. Among the various normalization techniques proposed for microarray meta-analysis, the quantile distribution transformation has been demonstrated to be highly efficient for normalizing data across experiments . Data trExpressionSet object, while GEO-derived and user-defined metadata compose the phenoData slot. As such, the ExpressionSet contains all the necessary information to perform any higher level analysis  directly in R.After normalization, the integrated data matrix represents the observed expression levels of the final As a simple case study, we collected, annotated and analyzed with A-MADMAN a muscle-related gene expression dataset with the following main steps:1. Data belonging to the GEO series GSE3307 (239 CEL files and related metadata) were downloaded and imported in a Project called "muscle-survey". The series belongs to the platforms GPL96 and GLP97 (Affymetrix HG-U133A and HG-U133B respectively), since most muscle biological samples were hybridized to both chips of the set.2. Metadata and information coming from the original paper  were usea) Build up a tag system, defining samples characterization ;b) Apply the tags to the samples, obtaining an annotated database.3. Samples were manually assigned to the corresponding individual IDs: Some individuals are represented two times in the samples , since the same biological sample was hybridized to both platforms.4. The query-based system was used to extract from the database three baskets:a) A basket of 34 normal muscle samples for a total of 18 individuals;b) Two baskets of Duchenne and Becker muscular dystrophy samples containing 20 and 9 samples respectively for a total of 15 individuals.5. The default workflow  was applied separately to the basket of normal samples and to the pair of Duchenne and Becker baskets, thus obtaining two ExpressionSets representing 11,174 GeneAnnot genes:a) An ExpressionSet, collecting gene expression levels in normal muscle tissue, which can be exported and used for a classic co-expression analysis.b) An ExpressionSet ready for the comparison of Duchenne and Becker expression profiles, for instance by SAM, to find genes differentially expressed in the two forms of the muscular dystrophy.ad-hoc computational methods to retrieve raw data files and any related meta-information, to fix incomplete, or otherwise inadequate, sample annotations, to integrate multiple data sets obtained using different technologies and to remove center- and platform-specific biases from the final integrated signals.In recent years, high-throughput technologies represented a breakthrough for the analysis of gene expression levels and hundreds of microarray-based studies generated an overwhelming mass of potentially valuable data. Most of these data are organized in major repositories, are publicly available, and their meta-analysis represents an enormous opportunity for biologists and geneticists. However, the meta-analysis of data, obtained using a plethora of array platforms and experimental protocols in different laboratories, represents also a complex task for bioinformatics. Indeed, the integrated analysis of microarray data generated by different research groups on different array platforms requires Despite major efforts to provide guidelines, formats, and standards for the annotation of microarray experiments, the identification, collection, and analysis of publicly available data sets still remains a cumbersome and error-prone process, further complicated by the heterogeneity of array designs and of sample characterization. The latter are among the most critical issues hampering meta-analysis approaches, since heterogeneous array designs complicate cross-platform integration and incomplete, or often inadequate, characterizations of specimens limit the robustness of statistical analyses. For the case of Affymetrix assays, the cross-platform comparison can partly be solved re-defining the array geometry through custom definition files and re-annotating the probesets in terms of unique entities . An expression profile associated to a unique gene identifier can be more easily matched across subsequent generations of microarrays -11 than Several software and web tools have been proposed for retrieval, organization and meta-analysis of microarray expression data from public repositories -21. HoweA-MADMAN is a novel tool for meta-analysis of Affymetrix data contained in Gene Expression Omnibus, which allows retrieving, annotating, organizing and analyzing gene expression datasets. In particular, A-MADMAN addressees i) the batch retrieval from Gene Expression Omnibus and the local organization of raw data files and of any related meta-information, ii) the re-annotation of samples, iii) the creation of user-defined batches of data through queries performed on sample labels, and iii) the integrative analysis of data obtained from different platforms through custom chip definition files and meta-normalization.GEOImporter module of GenePattern [GEOImporter module of GenePattern allows retrieving only a single series at once and does not permit to perform sub-selections of samples. MaRe, EMAAS and WGAS can retrieve multiple series/datasets. However, MaRe is limited to the simple search and retrieval of data , while the web centralization of WGAS poses serious operative limitations with very large datasets. A-MADMAN, once downloaded raw files and meta-information, locally organizes and displays the data through a user-friendly interface. Moreover, once data and metadata are retrieved, organized and annotated by A-MADMAN, they constitute a stable dataset which, being available through a web application, can be used to define and analyze different subsets of samples for answering different biological questions. The possibility of re-annotating samples, through user-defined assignments, is a peculiar feature of A-MADMAN and an improvement over all other available tools for meta-analysis. Indeed, a flexible assignment of .CEL files to specimens and a proper tracking of this association are necessary steps to efficiently exploit the mass of available data. However, handling multiple sample labeling and creating sub-classes of data may be so laborious and error-prone to discourage any meta-analysis diverse from the simple reproduction of the original experimental design. In A-MADMAN, samples are annotated using not only the original metadata but also a proprietary, controlled, but unconstrained vocabulary of descriptive terms. This project-tailored vocabulary allows adding biological, clinical, technical descriptions to the original sample annotations, thus widening the scopes of a meta-analysis. The annotation of samples through tags allows the definition and the extraction of any subset of data and, in principle, the meta-analysis of any user-defined experimental design. Specifically, one or multiple baskets of samples can be defined and extracted from the whole database simply using Boolean queries on the available tags and then used to construct any custom data matrix.The batch retrieval of gene expression data from both GEO and Array Express can be carried out using WGAS , MaRe 2 and EMAAePattern  are limiExpressionFileCreator module of GenePattern, although allowing the use of custom CDF, cannot consider more than one platform at the time. Similarly, EzArray and EMAAS permit the analysis of only one type of Affymetrix platform in a given project. CrossChip [mask file, which is later applied to the original .CEL files to generate new .CEL files. The new .CEL can be then be processed using any signal quantification algorithm. The major limitation of CrossChip is that only two platforms at the time can be compared. Differently, A-MADMAN can handle and integrate data generated by any type of Affymetrix arrays. Specifically, considering a set of samples obtained from more than one platform, expression signals are quantified separately for each platform on all those samples which are platform-homogeneous  are then integrated into a unique data matrix using a non-redundant list of gene-centered identifiers each derived from one or more platform-specific probesets/meta-probesets, using standard or custom CDFs. Each platform represented in the baskets contributes expression data according to a list of platform-CDF dependent probesets (or meta-probesets) and the values of the integrated data matrix are obtained averaging the expression levels of the various platform-specific probesets/metaprobesets  perform the batch retrieval and local organization of raw data files and of any related meta-information, ii) re-annotate samples using meta-information, iii) create user-defined batches of specimens, and iv) integrate data obtained from different platforms through custom chip definition files and meta-normalization.Project name: A-MADMAN• Project home page: • Operating systems: POSIX compliant and Win32 (developed on GNU/Linux)• Programming languages: Python, R• Other requirements: GNU R >= 2.8• License: GNU GPL v3 or any later version• GEO: Gene Expression Omnibus; CDF: Chip Definition File; HTTP: Hypertext Transfer Protocol; SOFT: Simple Omnibus Format in Text; SAM: Significance Analysis of Microarrays; NA: Not Available; miniml: MIAME Notation in Markup Language; MIAME: Minimum Information About a Microarray Experiment; RMA: Robust Multi-chip Analysis.AB planned, wrote and tested most of the software. AC contributed user interface code and participated to code review and quality control. FF participated to code review and algorithm planning, specifically for platforms integration. DR developed the inter-datasets normalization routine. SBo, CR, and SBi provided guidance during all phases of planning and implementing the software. AB, Sbo and Sbi wrote and revised the manuscript. All authors read and approved the final manuscript.A-MADMAN 1.4 source code. Version 1.4 of A-MADMAN source code.Click here for fileSupplementary Figure 1. The role of raw data-to-individuals assignment in the process of platforms integration.Click here for file


The concomitant effects of varying data signal-to-noise ratio and replication number on the performance of statistical tests also remain largely unexplored. In this study, we compared the effects of different underlying statistical error structures on the GLR test’s power in identifying differentially expressed genes in microarray data. We evaluated such variants of the GLR test as well as the one sample t-test based on simulated data by means of receiver operating characteristic (ROC) curves. Further, we used bootstrapping of ROC curves to assess statistical significance of differences between the areas under the curves. Our results showed that i) the GLR tests outperformed the t-test for detecting differential gene expression, ii) the identity of the underlying error structure was important in determining the GLR tests’ performance, and iii) signal-to-noise ratio was a more important contributor than sample replication in identifying statistically significant differential gene expression.Currently, statistical techniques for analysis of microarray-generated data sets have deficiencies due to limited understanding of errors inherent in the data. A generalized likelihood ratio (GLR) test based on an error model has been recently proposed to identify differentially expressed genes from microarray experiments. However, the use of different error structures under the GLR test has not been evaluated, nor has this method been compared to commonly used statistical tests such as the parametric  The development of microarray technology has been phenomenal in the past decade, with transcriptional profiling now a standard tool in many genomics research laboratories . The rapt-test. The performance of the t-test depends on the sample size, and whether the expression intensities can be assumed as normally distributed  were exposed to 21 days of hypobaric hypoxia (0.5 atm). Four control mice were housed at sea level for the duration of the experiment (normoxia). On day 21, all mice were sacrificed, whole lungs removed, total RNA isolated and hybridized to cDNA microarrays consisting of 5313 murine genes and expressed sequence tags. The RNA from the four control mice was pooled and used as reference for all microarray experiments. Four microarray replications with dye swapping were performed for three of the hypoxic mice; the fourth hypoxic mouse was studied in triplicate for a total of 15 labeling experiments. By performing replications for each animal and using multiple animals, our experiment design captured both the biological and technical noise in differential gene expression during hypoxic exposure.The first step in our data simulation strategy was to construct a realistic statistical model based on the experimental microarray data, and then to use this model to generate artificial gene expression values with varying statistical characteristics. Note that our statistical model described data from a cDNA microarray experiment in which control and treatment samples are hybridized on the same array. Based on exploratory data analysis of our 15 replicate microarray experiments, we observed that the variability of each gene’s intensity under hypoxic or control conditions in the log scale was approximately Gaussian. In addition, we also noted that two intensity measurements (under hypoxia and normoxia) of the same spot were highly correlated. These observations were in agreement with published findings (diffgenes) and 1000 genes as unchanged (nullgenes) during hypoxia relative to the unperturbed normoxic condition. Each diffgene had a unique expected value under hypoxic and normoxic conditions, in addition to a noise-induced error per replication. By definition, each nullgene had the same expected mean intensity during hypoxia and normoxia but different measurable values from one replication to another due to experimental variability. We used the following model for simulating the observed paired intensities, , of background-subtracted and normalized diffgenes and nullgenes during replicate experiments:We simulated 2000 genes in this study: 1000 genes were defined as differentially expressed  is the expected “true” intensity under hypoxia and μndiffgene (or μnnullgene) is the expected “true” intensity during normoxia . Recall that indiffgeneμ ≠ ihdiffgeneμ and jnnullgeneμ = μjhnullgene. For each gene, the random error terms during hypoxia and normoxia, ɛh and ɛn, were chosen from an independent bivariate normal distribution with mean zero, variance under hypoxia (ωh) and normoxia (ωn), and correlation .for μhdiffgene and μndiffgene (1a). In addition, we used the variance of each of these 1000 genes over 15 replicate experiments as the variance of the error component of each assigned diffgene (the same procedure applied for each nullgene). For the nullgenes, we chose a different set of 1000 genes from the original data set, and assigned, by definition, μhnullgene = μnnullgene (1b). The correlation term for the bivariate normal distribution for each diffgene and nullgene was obtained directly from the expression measurements in these experiments. Note that there was a range of differential gene expression values among the diffgenes since each gene had a different μ (during hypoxia and normoxia) and each replicate had a different ɛ. Similarly, although each nullgene was defined to have equal mean intensity during hypoxia and normoxia, each replication resulted in a different expression value because of the error term ɛ. Gene expression values were now derived by randomly drawing error terms from the bivariate Gaussian distribution unique to each of the 2000 genes. Simulated gene expression replicates based on this approach should capture both the biological variability and the technical “noise” of the original microarray experiments. Furthermore, since these simulated log-transformed intensities were normally distributed, parametric approaches such as the parametric t-test and GLR test could be applied to identify differentially expressed genes.In order to create realistic data simulations, we chose the parameters for the above model from our set of 15 replicate microarray experiments. We randomly selected 1000 genes from this data set and assigned their mean expression values during hypoxia and normoxia as our 1000 simulated μhdiffgene and μndiffgene for the 1000 diffgenes by two-fold to obtain ‘medium’ (s = 2) and three-fold to obtain ‘high’ (s = 3) signal-to-noise ratios. These data sets were aimed at modeling data generated during more profound perturbations such as ionizing radiation or yeast sporulation studies. Similarly, replications with r = 2, 3 or 4 and 6 or 8 were data sets containing ‘low’, ‘medium’ or ‘high’ number of replications per gene, respectively. In total, 15 sets of simulated data at each signal-to-noise ratio  and replication number  were generated. These data sets therefore differed in terms of quality, with r2s1  having the lowest and r8s3  having the highest quality respectively. This pool of 15 data sets constituted this study’s original simulated sample. Because we knew a priori which genes were differentially expressed and which were not, altering signal-to-noise ratios and replications per gene allowed us to compare the robustness and performance of various statistical tests in detecting differentially expressed genes.We denoted data sets with s = 1, 2 or 3 as data containing ‘low’, ‘medium’ or ‘high’ signal-to-noise ratios respectively. We generated the ‘low’ signal-to-noise ratio (s = 1) data based on differential gene expression values in our experimental model of hypoxic pulmonary hypertension. As discussed above, we believe this perturbation resulted in only a modest level of differential gene expression. Therefore, we increased the difference between We implemented the statistical data model proposed by Ideker and co-authors . In addiμix, μiy) is the pair of mean intensities for gene i, multiplicative error ɛH is drawn from a bivariate normal distribution with mean 0, variances ωxH and ωyH, and correlation 0 ) (ρ denotes correlation. The correlation term models the observation that the log-transformed expression intensities (under control and treatment conditions) per spot consistently correlate over repeated measurements. ɛH and ɛA are assumed to be independent of each other. Additionally, f(μ) allows one to incorporate into the statistical error structure the common observation that the standard deviation is dependent on the magnitude of intensity (mean) (where (f (μix) and f (μiy) equal to 1. Under this model, the measured expression intensity is dependent on a linear combination of normal error components. Hence, GLR1 is:For the simplest case, we set f (μix) and f (μiy) to μix and μiy respectively (4), so that the variance could be approximately proportional to the mean intensity (f (μix) = 1/μix and f (μiy) =1/μiy. In GLR3, we explicitly modeled the observation that correlation between expression levels under control and treatment conditions decreases at lower intensities by weighting the multiplicative error term using the reciprocals of the mean intensity, as shown in (5).Based on (2), GLR2 was constructed by equating Describing raw intensity data, the error structure proposed by http://db.systemsbiology.net/software/VERAandSAM. ) ((6) was implemented in a public-domain computer program, Vera and Sam (ndSAM. ) . Hencefot-test was applied to each gene separately without the requirement of a constant variance or a constant coefficient of variation across genes, we expected the t-test to be well suited in detecting differential gene expression.Although GLR1—3 may appear similar to the data simulation model in (1), there were fundamental differences between them. Crucially, we used a different Gaussian distribution for each of the 2000 genes in our data simulation while the GLR tests were applied to each gene to determine whether, under a single statistical data model encompassing all 2000 genes, the expression levels under control and experimental conditions were different. For example, although the GLR1 structure (3) may appear very similar to the data simulation model (1), a different Gaussian distribution was sampled for each of the simulated genes, whereas the GLR test was performed for each gene to assess whether applying one distribution to the entire set of 2000 genes can discriminate expression levels between normoxic and hypoxic states. Furthermore, since the parametric β =  as well as a mean pair per gene μ = . The probability density function for gene i is P = P  The GLR method is essentially composed of two consecutive steps: parameter estimation followed by likelihood ratio test. In the Vera (Variability and ERror Assessment) program, β and μ are first estimated from the expression levels of all genes via MLE. MLE is the procedure of finding the value of one or more parameters for a given statistic that makes the known likelihood distribution a maximum . All parameter values converged after 100–200 iterations.The MLE parameter values that maximize μix ≠ (μiy) between the two cell populations, a likelihood ratio test is subsequently performed, which assumes a univariate normal distribution for the expression data of a gene not differentially expressed (null hypothesis) and a mixture of two univariate normal distributions (with different means) for the expression data of a gene differentially expressed under control and treatment conditions . To this end, the GLR test statistic λi is computed using SAM (Significance of Array Measurement)  :μx = μy = μ is imposed (null hypothesis), whereas in the denominator, the optimization is unconstrained, i.e. μx ≠ μy . In the case where μxi = μyi, λi follows a χ2 distribution with one degree of freedom. To select differentially expressed genes, a critical cutoff value λc must be determined beforehand based on the results of a set of control to control microarray experiments. However, since our data simulation provides us a priori information on which genes are differentially expressed and which ones are unchanged we can circumvent the control to control experiments and use ROC curves to compare the performance of variants of the GLR test and the t-test.Two maximizations are performed in (8): in the numerator, the constraint t-test statistic and its variants are commonly used to detect differential gene expression because genes deemed as significantly expressed have maximal difference in mean expression values between two groups and minimal variation of expression within each group. In this study, we compared the log ratios of spot intensities between control and treatment groups using the parametric one-sample t-test (The e t-test .diffgene or nullgene) was known and served as the reference or ‘gold standard’. The area under the ROC curve (AROC) was used as a unidimensional summary of the ability of the statistical test to discriminate between diffgene or nullgene, ranging from 0.5 for a test with no diagnostic capability to 1 for a test with perfect separation of the two groups. The ROC curve was derived by: 1) ranking values of the test metric (i.e. t-statistic or λ) of each gene; and 2) identifying the correctly and incorrectly assigned diffgenes and nullgenes by varying a threshold; and 3) computing the full range of true-positive fraction or sensitivity values, and the false-positive fraction or 1-specificity values. ROC curves thus provide a more complete measure of a test’s discriminatory ability than the choice of an arbitrary and isolated  point. Once the ROC curve was obtained, the area underneath it was computed to estimate AROC.ROC curve analysis was used for performance evaluation. Expression status of each gene  = 0.95. To compare the performance between two statistical methods, we computed a z-statistic using the bootstrap to generate the variance of the AROC estimates  without strict distributional assumptions . In shorstimates :A1 and A2 refer to the AROC’s from the two statistical tests to be compared, SE is the standard error, which was estimated using the bootstrap, and Cov represents the covariance of A1 and A2. Cov was derived by taking the product of the correlation between the bootstrapped samples  and standard deviations. Under the null hypothesis of no difference between two statistical tests, the z-statistic is approximately normally distributed  were above those for r8s1, suggesting that stronger signals could lead to a higher test power than larger sample repeats. Individual AROC values ranged from 0.549 to 0.93, and one-sided test at an α level of 5% confirmed that the tests’ performance was always superior to random classification.Representative ROC curves generated from our analysis are shown in z-statistic. For two statistical tests to be significantly different at the P < 0.05 level, it is expected that the null hypothesis would be rejected in 950 comparisons. These tables and figure reveal that, overall, the best discriminator was obtained by the GLR test employing GLR3 as its underlying statistical model, which attained significance with respect to all other tests in virtually every condition studied. There were significant differences between the GLR test employing the VS model and every other statistical test, indicating that the former was the least discriminatory. Of note, the t-test underperformed relative to the GLR tests when data quality was poor , but improved when the number of gene replicates and signal-to-noise were ‘high’ .The mean (±S.D.) AROC’s calculated on the 1000 bootstrap replications for each of the five statistical tests and for each data condition are presented in In t-test improved by 34.4% when analyzing data sets with a signal-to-noise ratio of 1 compared to 3 (mean AROC of 0.614 and 0.821 respectively). In contrast, after increasing the number of replications by 4 fold (from 2 to 8 repeats per gene), we observed a lower increase of 11.9% (67% and 76% accuracy for 2 and 8 replicates respectively) in the average discriminating power  and therefore dramatically simplifies the complexity of the simulated gene expression data. Yet, our results indicate that the discriminatory ability of the GLR test was still equivalent or more powerful than the t-test for data sets containing ‘low’ and ‘medium’ gene replication number. It follows that GLR test’s assumption of a single statistical data model to quantitate the error distribution and structure of pooled gene expression data does not necessarily compromise its performance at identifying differentially expressed genes.We compared the power of the  studies . Furthert-test’s performance on data sets containing ‘high’ gene replications and signal-to-noise ratios, we showed that the GLR test (employing GLR3) could still attain a higher power than the t-test. A fundamental problem with employing tests dependent on Gaussian likelihoods in microarray studies  is that replications are often few due to costs or availability of biological material. Though such shortcomings can be addressed by increasing sample size, the potential gain in the statistical power of the GLR and t-test may be markedly offset by increases in cost and effort (While there was an improvement in the d effort . This paf(μ) = 1 in GLR1 whereas f(μ) = μ and 1/μ in GLR2 and GLR3, respectively). Our results showing that the GLR3 was the most powerful statistical test. We believe that GLR3 performed better because its model structure accounts for two common traits associated with microarray expression data: (i) log-transformed microarray expression data are normally distributed ; (ii) correlation between log(xi) and log(yi) increases at higher spot intensities ; and (iii) for each gene i, log(xij) and log(yij) are correlated over repeated measurements j. This suggests the importance of adjusting for a decrease in correlation between expression values of control and treatment genes at low intensities.Overall, our results suggest that GLR3 was the best performing error structure compared with the GLR1, GLR2 and VS models. The GLR test using VS as its underlying error model consistently generated the lowest AROC’s for all bootstrap re-samples. We attribute this to the fact that the VS model’s error structure considers raw, non-log-transformed gene expression levels. We contend that such an assumption can potentially limit this test since logarithm transformation can improve the normality of expression intensities . Among tA novel finding of this work was to show the greater influence of signal-to-noise ratio compared to gene replication on test performance. This implies that experiments that maximize the signal-to-noise ratio can facilitate to a larger extent the identification of significant differences in gene expression than those with more sample repeats. To the best of our knowledge, this study represents the first attempt to directly compare the influence of replication and signal-to-noise ratio on the performance of statistical analysis of microarray data, although several reports have commented on the significance and implications of each factor separately . The greIt has been previously reported that performing microarray studies without sufficient replicates can lead to poor sensitivity and reliability . HoweverAn issue that bears consideration is that, in a practical setting, it may be difficult to assess the signal-to-noise ratio of actual microarray data. Signal-to-noise ratio depends on many factors, including quality of tissues, sample size relative to the number of variables to be classified, experimental variation, and inherently variable original signal intensities . In thist-test. Additionally, the four statistical data models that were implemented under the GLR test do not represent the full spectrum of possible error structures. We also did not use partial AROC as a performance indicator to differentiate between statistical tests’ power at low false positive rate (or high specificity range). Finally, the problem of multiple comparisons was not addressed in this work because we were primarily interested in comparing the inherent statistical power of the t-test and GLR methods.There are several limitations in this study. First of all, we were constrained by how realistically our data simulation models the entire variability of microarray experiments. Furthermore, our experiments and data simulations were performed using cDNA microarrays and therefore might not be applicable to single dye oligonucleotide arrays such as the Affymetrix platform. We restricted our analyses to the GLR method and the parametric t-test statistical approaches were compared, our findings have three important implications for analysis of microarray data:In summary, although the present study depended on a specific set of simulated data derived from cDNA experiments and only the GLR test and the t-test for detecting differential gene expression. This underlines the importance of incorporating and modeling the error structure of microarray data during the development of future statistical tests.The GLR method is more powerful than the parametric Within the GLR approach, an error structure that contains a multiplicative error term weighted by the reciprocal of mean expression intensity outperforms other models (GLR3).Designing experiments that maximize signal-to-noise ratio, instead of just raising the number of replicates per gene, can better identify differentially expressed genes in microarray data.


Several high-throughput technologies can measure in parallel the abundance of many mRNA transcripts within a sample. These include the widely-used microarray as well as the more recently developed methods based on sequence tag abundances such as the Massively Parallel Signature Sequencing (MPSS) technology. A comparison of microarray and MPSS technologies can help to establish the metrics for data comparisons across these technology platforms and determine some of the factors affecting the measurement of mRNA abundances using different platforms.Arabidopsis thaliana tissues and floral mutants. We calculated correlations and used clustering methodology to compare the normalized expression data and expression ratios across samples and technologies. Abundance expression measurements were more similar between different samples measured by the same technology than between the same sample measured by different technologies. However, when expression ratios were employed, samples measured by different technologies were found to cluster together more frequently than with abundance expression levels.We compared transcript abundance (gene expression) measurement data obtained using Affymetrix and Agilent microarrays with MPSS data. All three technologies were used to analyze the same set of mRNA samples; these samples were extracted from various wild type Furthermore, the two microarray technologies were more consistent with each other than with MPSS. We also investigated probe-position effects on Affymetrix data and tag-position effects in MPSS. We found a similar impact on Affymetrix and MPSS measurements, which suggests that these effects were more likely a characteristic of the RNA sample rather than technology-specific biases.Comparisons of transcript expression ratios showed greater consistency across platforms than measurements of transcript abundance. In addition, for measurements based on abundances, technology differences can mask the impact of biological differences between samples and tissues. Arabidopsis thaliana [The availability of whole genome sequences and the development of high-throughput technologies have enabled global or genome-wide measurements of gene expression. The complete genome sequence of thaliana  is availthaliana ,3 and tathaliana -7, that thaliana . The AffHybridization signal intensities can be used to generate relative or abundance measurements that correspond to the amount of target mRNA that has hybridized to a specific probe, but relative measurements are determined by a comparison of two samples. Tag-based technologies measure the expression level of a gene by counting the abundance of a specific transcript in a sample. This count provides an abundance measure of each gene's expression level within the sample. Until recently, only two tag-based technologies have been widely used: SAGE  ,5 and MPPrevious studies have compared different gene expression measurement technologies to determine how well they correlate with each other and/or to use one technology as a benchmark for another -24. SomeArabidopsis thaliana. Here, we extend this preliminary work by carrying out a more detailed data analysis and by including data obtained with the Affymetrix platform. We compare transcript abundance measurements obtained with two types of microarrays (Affymetrix and Agilent technologies) and with MPSS. These data were generated from the same set of mRNA samples extracted from a variety of Arabidopsis thaliana tissues, mutants and treatments. We investigate the consistency of transcript abundance measurements across the three technology platforms using comparison of abundance expression levels and expression level ratios. We discuss factors which may affect the measured abundance and contribute to variation in measurements across different technology platforms. This study uses a broad range of samples in order to assess these different platforms.In a preliminary study, Coughlan et al.  comparedWe believe that there should be some biologically meaningful relationships among different samples from different technology platforms. For example, samples from same tissues with different condition/treatment are expected to have higher correlation than samples from different tissues, and the same samples measured by different technology platforms would be expected to have higher correlation than others. To test our hypothesis, we make comparisons of abundance expression level and comparisons of expression level ratios.Dpn II site detectable by MPSS . The MPSS data showed a wider dynamic range than Affymetrix and Agilent data  values , which we used for abundance expression levels comparison. We calculated the Pearson's correlation and Spearman's rank correlation of logged NEU normalized expression data for 1,648 genes in MPSS, Affymetrix and Agilent  and "present" calls determined by the Affymetrix software), and we performed hierarchical clustering using these data Figure . The NEUWhen comparing the same RNA samples across technologies, the highest correlations were observed for MPSS versus Affymetrix microarrays (0.66 to 0.76) and for Affymetrix microarrays versus Agilent microarrays (0.65 to 0.74) while the lowest correlations were exhibited for Agilent microarrays versus MPSS 0.53 to 0.56)  tail and anchored by a four base restriction enzyme recognition site such as GATC, the primary tag was defined as the 3'-most tag in an annotated gene, which was determined based on an analysis of s Figure  and 6D. s Figure  and 6B.Next, we investigated the relationship between the probe position and perfect match (PM) probe values in the Affymetrix microarrays. The probe positions were determined relative to the annotated 5' and 3' ends of each gene. We found that the measured transcript abundances correlated with the position from the 3' end such that probes close to the 3' end tended to have lower PM values Figure , but theBased on the analysis of probe position in the Affymetrix microarrays and the tag position in MPSS samples, we found that both probe position and tag position, as measured from the 3' end, demonstrated a consistent trend. With the increase in probe position or tag position from the 3' end, the PM value and tag-abundance tended to increase then plateau. One reason for this tendency could be that there are many alternative transcripts for some genes which are not correctly annotated in the current Arabidopsis genome which could be alternative splice products or transcripts utilizing alternative polyadenylation sites. In fact, some probes and tags that we used for this analysis were not located within the most recently annotated genome ,28. To cOur study indicated that the MPSS platform may be more variable in measuring RNA abundance than Affymetrix and Agilent platforms. Our analysis showed that the microarray and MPSS technologies did not correlate well on a quantitative basis for transcript abundance measurements, although within-technology clustering of samples was largely consistent. Expression ratio comparisons were more comparable and consistent across the platform technologies, and ratios for sample pairs involving different tissues were found to correlate quite well across technologies. Therefore, for the purpose of hierarchical clustering of global data sets, the ratios were a more biologically meaningful measure to use for cluster analysis than abundance measurements of gene expression. However, when the difference between the ratios was subtle, as in the case of ratios involving the same pair of tissues but with one derived from a different mutant or treatment, then technological differences often overwhelmed the underlying biological signal. Moreover, the two microarray technologies (Affymetrix and Agilent) were more consistent with each other than with MPSS.The position of the probe or tag in the transcript has previously been shown to affect the measured transcript abundance . Our anaThe microarray data and MPSS data for the manuscript has been submitted to the GEO Omnibus . The SerArabidopsis thaliana tissues , homeotic floral mutants  and two leaf treatments  [Eleven different distinct RNA samples were used in this study Table . The RNActively) . Only onThe Arabidopsis MPSS data has been published previously and is available on our website . In the To generate data that was comparable across technologies, the same Arabidopsis RNA samples used for the MPSS analyses were hybridized to two different high-density Arabidopsis microarray platforms: the Affymetrix ATH1 GeneChip   from TAIR  and focuTo make the data directly comparable, we rescaled the normalized data  into Normalize Expression Unit (NEU) values, as follows:x represented the normalized expression level of gene calculated by each platform. For MPSS platform, NEU has the same meaning as Transcript Per Million (TPM).where log2-transformed. When investigating the correlation relationship between different samples, hierarchical clustering method was used with both Pearson's correlation and Spearman's rank correlation as the distance metrics and using a complete linkage algorithm [r, where r represents correlation coefficient value. Correlation coefficient and total least-square values [Contour-plots were used to represent the density of data. Expression data were lgorithm ,37. The e values  were cale values  was usede values  to obtaie values  to evaluVA and BM initiated the research, provided the MPSS data and finished the first version of analysis on the measurement abundance and ratio comparisons. JC improved the analysis and expanded it to the analysis of the probe-position effect in Affymetrix and tag-position effect in MPSS. JC also evaluated the results and drafted the manuscript. MR helped with the evaluation of the results and manuscript preparation, and provided mentoring. MW provided the Affymetrix experiments and data. RM, MW and DS provided input on experiment design, analysis ideas and Affymetrix data. SC provided the Agilent data. All authors read and approved the final manuscript.Supplementary Figures. An additional pdf file includes supplementary figures.Click here for file


Microarray data has a high dimension of variables but available datasets usually have only a small number of samples, thereby making the study of such datasets interesting and challenging. In the task of analyzing microarray data for the purpose of, e.g., predicting gene-disease association, feature selection is very important because it provides a way to handle the high dimensionality by exploiting information redundancy induced by associations among genetic markers. Judicious feature selection in microarray data analysis can result in significant reduction of cost while maintaining or improving the classification or prediction accuracy of learning machines that are employed to sort out the datasets. In this paper, we propose a gene selection method called Recursive Feature Addition (RFA), which combines supervised learning and statistical similarity measures. We compare our method with the following gene selection methods:Support Vector Machine Recursive Feature Elimination (SVMRFE)Leave-One-Out Calculation Sequential Forward Selection (LOOCSFS)Gradient based Leave-one-out Gene Selection (GLGS)To evaluate the performance of these gene selection methods, we employ several popular learning classifiers on the MicroArray Quality Control phase II on predictive modeling (MAQC-II) breast cancer dataset and the MAQC-II multiple myeloma dataset. Experimental results show that gene selection is strictly paired with learning classifier. Overall, our approach outperforms other compared methods. The biological functional analysis based on the MAQC-II breast cancer dataset convinced us to apply our method for phenotype prediction. Additionally, learning classifiers also play important roles in the classification of microarray data and our experimental results indicate that the Nearest Mean Scale Classifier (NMSC) is a good choice due to its prediction reliability and its stability across the three performance measurements: Testing accuracy, MCC values, and AUC errors. This ability allows scientists to investigate the functional relationship between the cellular and physiological processes of biological organisms and genes at a genome-wide level. The preprocessing procedure for the raw microarray data consists of background correction, normalization, and summarization. After preprocessing, a high level analysis, such as gene selection, classification, or clustering, is applied to profile the gene expression patterns To address the “curse of dimensionality” problem, three basic strategies have been proposed for feature selection: filtering, wrapper, and embedded methods. Filtering methods select subset features independently from the learning classifiers and do not incorporate learning To avoid the weakness of filtering methods, wrapper methods wrap around a particular learning algorithm that can assess the selected feature subsets in terms of the estimated classification errors and then build the final classifier Wrapper methods can noticeably reduce the number of features and significantly improve the classification accuracy et al. designed two methods of gene selection, leave-one-out calculation sequential forward selection (LOOCSFS) and the gradient based leave-one-out gene selection (GLGS) To improve classification of microarray data, Zhou and Mao proposed SFS-LS bound and SFFS-LS bound algorithms for optimal gene selection by combining the sequential forward selection (SFS) and sequential floating forward selection (SFFS) with LS (Least Squares) bound measure In human genetic research, exploiting information redundancy from highly correlated genes can potentially reduce the cost and simultaneously improve the reliability and accuracy of learning classifiers that are employed in data analysis. To exploit the information redundancy that exists among the huge number of variables and improve classification accuracy of microarray data, we propose a gene selection method, Recursive Feature Addition (RFA), which is based on supervised learning and similarity measures. We compare RFA with SVMRFE, LOOCSFS, and GLGS by using the MAQC-II breast cancer dataset to predict pre-operative treatment response (pCR) and estrogen receptor status (erpos) and compare RFA with SVMRFE and LOOCSFS on the MAQC-II multiple myeloma dataset to predict the overall survival milestone outcome  and to predict event-free survival milestone outcome .We compare MSC-based RFA methods with GLGS, LOOCSFS, and SVMRFE on MAQC-II breast cancer dataset. We notice that the performance evaluation using testing accuracy (left column) and MCC (middle column) is consistent, but the performance evaluation using AUC measurement is not always consistent with the evaluation using testing accuracy and MCC. For instance, in applying UDC to the feature sets, RFA methods have the best prediction performance evaluated using testing accuracy and MCC values; however, with respect to performance evaluated using AUC errors RFA is not the best. Regarding the testing performance measured by using AUC errors, the best results have been obtained by using RFA with NMSC classifier. The AUC errors are as low as 0.08, which is much better than the results by using other methods of gene selection.The prediction results on pCR endpoint, shown in Due to a huge number of variables and small sample size, there are complicated interactions and relations among genes as well as high redundancy information with microarray data. The selection of predictive models that depend on selected features and employed classifiers is extremely important for the classification of microarray data and for the further biological function analysis/validation. Machine learning and data mining techniques provide us with a powerful approach to the study of the relationship among genes. Based on supervised learning and similarity measurements, we propose a Recursive Feature Addition (RFA), recursively employ supervised learning to obtain the highest training accuracy and add a subsequent gene based on the similarity between the chosen features and the candidates to minimize the redundancy within the feature set. We believe this RFA method captures more informative and differently expressed genes than other methods. Experimental comparisons are performed by using two MAQC-II microarray datasets, breast cancer and multiple myeloma. Our studies show that the method of gene selection is strictly paired with learning classifier, which determines the final predictive model by using training data. In other words, the best classification models under different learning classifiers are associated with different methods of gene selection. Using several popular learning classifiers including NMSC, NBC, SVM, and UDC, on average, the best method of gene selection is RFA, followed by GLGS, LOOCSFS, and SVMRFE. Regarding compared learning classifiers, NMSC outperforms the others with respect to testing performance, stabilization, and consistency.Biological function analysis based on MAQC-II breast cancer dataset finds that our applied feature selection methods including RFA, GLGS, LOOCSFS, and SVMRFE can generate features containing a significant portion of known cancer related genes for both pCR and erpos endpoints . AlthougIn all results of the five gene selection methods with the four learning classifiers, on average, the combination of gene selections of NMSC-MSC and NBC-MSC with the learning classifier of NMSC has performed the best, regarding the comprehensive evaluation criteria of testing accuracy, MCC values, and AUC errors. It should be noted that the gene selection methods of NMSC-MSC and NBC-MSC are not always the best over the four learning classifiers, in other words, the best models among different learning classifiers are associated with different gene selection methods. The selection of the best model with the use of a specific learning classifier is normally based on the training and the evaluation criteria. Under an evaluation criterion with the use of some learning classifier, the best training model among the five gene selection methods is selected as the best model under the learning classifier. To select the best model among the four learning classifiers, the best models among the four learning classifiers are compared and the model obtaining the highest evaluation score is generally selected the best among the five gene selection methods and the four learning classifiers. For instance, Regarding different evaluation criteria, the gene selection associated with the best model under a learning classifier may be different, as shown in The breast cancer dataset used in the MAQC-II project is used to predict the pre-operative treatment response (pCR) and estrogen receptor status (erpos) We take the MAQC-II multiple myeloma dataset to predict the overall survival milestone outcome  and to predict event-free survival milestone outcome . For OSMO label information, there are 51 positives and 289 negatives in original training set, 27 positives and 187 negatives in original validation set; as for EFSMO label information, there are 84 positives and 256 negatives in original training set, 34 positives and 180 negatives in original validation set Our method of recursive feature addition (RFA) employs supervised learning to achieve the best training accuracy and uses statistical similarity measures to choose the next variable with the least dependence on, or correlation to, the already identified variables as follows:Insignificant genes are removed according to their statistical insignificance. Specifically, a gene with a high p-value is usually not differently expressed and therefore has little contribution to classification of microarray data. To reduce the computational load, those insignificant genes are removed.p-value measured by test-statistics , is identified as the first element. At this point the chosen feature set, G1, contains just one element, g1, corresponding to the feature dimension one.Each individual gene is selected by supervised learning. A gene with highest classification accuracy is chosen as the most important feature or the first element of the feature set. If multiple genes achieve the same highest classification accuracy, the one with the lowest +1)st dimensional feature set, GN+1 = {g1, g2, …, gN, gN+1} is obtained by adding gN+1 to the Nth dimensional feature set, GN = {g1, g2, …, gN}. The choice of gN+1 is processed as follows:The (Nig (i g GN) into GN and have the classification accuracy of the feature set GNig}. The ig (i g GN) associated with the group, GNig} that obtains the highest classification accuracy, is the candidate for gN+1 (not yet gN+1). Considering the large number of variables, it is very likely that multiple features correspond to the same highest classification accuracy, these multiple candidates are placed into the set C, but only one candidate in C will be identified as gN+1. How to make the selection is described next.Add each gene gN+1, we measure the statistical similarity between the chosen features and each candidate. We design a similarity measurement with the use of a widely-used Pearson's correlation coefficient To find a most informative (or least redundant) candidate for ng  is a chosen gene, cg (c g C) is a candidate gene, and cor stands for the function of Pearson's correlation coefficient. The sum of the square of the correlation SC that is calculated as follows:Suppose gN+1 is based on Minimal value of the Square of the Correlation (MSC), that is,Then selection of In the methods mentioned above, a feature is recursively added to the chosen feature set based on supervised learning and the similarity measurement. In our experiments we choose naïve bayes classier (NBC) and nearest means scale classifier (NMSC) Cross-validation is a technique for estimating how accurately a predictive model will perform in practice. Generally, the data are partitioned into complementary subsets, one subset  is used for constructing the predictive model and the other subset  is used for validation. To reduce variability, multiple rounds are performed using different partitions, and the validation results are averaged over all rounds. There are three common types of cross-validation:Repeated random sub-sampling validation. This technique randomly splits the dataset intro training and testing data. The results are then averaged over the splits. The advantage over K-fold cross validation (described below) is that the portion of the training/testing split is not dependent on the number of iterations (folds);K-fold cross-validation. The original sample is partitioned into K subsamples. A single subsample is retained as the validation data for testing the model and the remaining K-1 subsamples are used as training data. The cross-validation process is repeated K times with each of the K subsamples used exactly once as the validation data;Leave-one-out cross-validation. It uses a single observation from the original sample as the validation data and the remaining observations as the training data. It is the same as a K-fold cross validation with K being equal to the number of observations in the original sample. Leave-one-out cross-validation is often computationally expensive.Considering the high computational requirement of leave-one-out cross-validation and the insufficiency of one time K-fold cross-validation, we took the strategy of repeated random sub-sampling validation. In the model implementation, we mixed all the training data points and validation points. In each experiment, we randomly chose 80% of samples for training and the remaining 20% of samples for testing. Twenty experiments were performed . The average testing performances, evaluated in terms of testing accuracy, MCC values, and AUC errors, were compared. The learning classifiers UDC, NBC, NMSC, and SVM were employed for training and testing.