Diagnosis of soft tissue sarcomas (STS) is challenging. Many remain unclassified  or grouped in controversial categories such as malignant fibrous histiocytoma (MFH), with unclear therapeutic value. We analyzed several independent microarray datasets, to identify a predictor, use it to classify unclassifiable sarcomas, and assess oncogenic pathway activation and chemotherapy response.We analyzed 5 independent datasets (325 tumor arrays). We developed and validated a predictor, which was used to reclassify MFH and NOS sarcomas. The molecular “match” between MFH and their predicted subtypes was assessed using genome-wide hierarchical clustering and Subclass-Mapping. Findings were validated in 15 paraffin samples profiled on the DASL platform. Bayesian models of oncogenic pathway activation and chemotherapy response were applied to individual STS samples. A 170-gene predictor was developed and independently validated . Most MFH and NOS tumors were reclassified as leiomyosarcomas, liposarcomas and fibrosarcomas. “Molecular match” between MFH and their predicted STS subtypes was confirmed both within and across datasets. This classification revealed previously unrecognized tissue differentiation lines  and was reproduced in paraffin specimens. Different sarcoma subtypes demonstrated distinct oncogenic pathway activation patterns, and reclassified MFH tumors shared oncogenic pathway activation patterns with their predicted subtypes. These patterns were associated with predicted resistance to chemotherapeutic agents commonly used in sarcomas.STS profiling can aid in diagnosis through a predictor tracking distinct tissue differentiation in unclassified tumors, and in therapeutic management via oncogenic pathway activation and chemotherapy response assessment. Soft tissue sarcomas (STS) are a heterogeneous group of mesenchymal tumors traditionally classified according to their morphological resemblance to presumptive cells of origin such as fibroblasts, muscle cells, adipocytes or peripheral nerve-sheath cells Gene expression profiling has been used in the study of STS We used study cohort 1  in order to train a nearest centroid classifier of the 6 subtypes  frequently presenting differential diagnosis problems, . A singlst and 2nd step predictors are shown in This optimal predictor was 85% accurate for the 6 classes . The genImportantly, the predictor included several genes associated with distinct differentiation states  . ApproprWe mapped the 170-gene set across the different platforms of the 4 datasets in study cohort 2, and despite the many technical differences among them we were able to reproduce its performance. Specifically, its accuracy was 86%, 78%, 79%, and 84% in the MSKCC, Stanford, Japan and UK datasets, respectively by leave one out cross validation  . DetaileWe used the 170-gene predictor in study cohort 3, to reclassify 76 MFH and 10 NOS samples. As noted, the MFH and NOS samples were not used in the development or validation of the predictor.The majority (68 out of 76) MFH tumors were predicted as liposarcomas (46%-35 samples), fibrosarcomas (29%-22 samples) and leiomyosarcomas (14%-11 samples), while 7 out of 10 NOS tumors (in the NCI dataset) were also predicted as liposarcomas (3), leiomyosarcomas (3) and fibrosarcomas (1). The remaining MFH and NOS tumors (11 samples) were predicted as malignant peripheral nerve sheath tumors (4 samples), synovial sarcomas (6 samples) and rhabdomyosarcoma (1 sample).We reasoned that if our proposed MFH reclassification using the 170-gene predictor is valid, it should reflect the overall molecular similarity between reclassified MFH and their corresponding subtypes, above and beyond the prediction by the 170-gene predictor. We addressed this question both within each as well as across datasets.We performed unsupervised hierarchical clustering and assessed whether the MFH samples preferentially grouped with samples from their predicted subtype using the top-33% variant genes, i.e. 4200, 7428, 922 probe-sets in the NCI, Japan and MSKCC, and Stanford datasets, respectively. Indeed, 58 out of the 76 reclassified MFH samples (76%) clustered together with samples from their predicted STS subtypes suggesting that the 170-gene predictor reflects an overall ‘molecular match’ between them. Furthermore, MFH-samples predicted as LIPO (MFH-LIPO) did not cluster exclusively with myxoid or with non-myxoid liposarcomas; rather certain MFH-LIPO clustered with myxoid while other MFH-LIPO clustered with non-myxoid liposarcomas suggesting that our predictor is capturing information associated with adipocyte differentiation irrespective of the myxoid or non-myxoid subclassification .The same analysis was performed for the NOS sample predictions in the NCI dataset, and 6 out of the 10 NOS samples clustered with their predicted subtypes .To strengthen the molecular relevance of the MFH reclassification we investigated whether MFH samples were molecularly similar with samples from their predicted STS subtype across different datasets. To achieve this, we used the Subclass Mapping (SubMap) methodology, specifically developed to assess the commonality of subtypes/subclasses in independent and disparate datasets  overexpressed genes associated with adipocyte differentiation compared to the rest of the MFH tumors. Similarly, MFH tumors predicted as leiomyosarcomas (MFH-LEIO) overexpressed genes associated with smooth muscle differentiation and MFH sarcomas predicted as fibrosarcomas (MFH-FIBRO) overexpressed genes associated with fibroblast differentiation . We coulWe next evaluated the ability of our STS predictor to reclassify formalin fixed paraffin embedded NOS samples in order to assess its broader applicability for clinical practice and future large scale clinical research. These NOS samples had been previously evaluated by a sarcoma pathology expert (J.G) using state of the art current histopathologic methodology and could not possibly be classified into any of the known STS types.Before applying our predictor to the unclassified samples, we verified its accuracy in 10 STS samples with known diagnosis. We trained the predictor  on the combined U 133 datasets and directly applied it on the independent DASL paraffin gene expression dataset. The STS classifier accurately predicted 8 of the 10 samples, demonstrating accuracy identical to that previously estimated in the 5 public frozen-tissue based datasets, thus validating its performance in samples with known diagnosis and in paraffin tissue. We then directly applied our predictor to the 5 unclassifiable (NOS samples) within the paraffin cohort, and 4 of them were classified as liposarcomas and 1 as leiomyosarcoma. We then examined expression of tissue specific genes in the 4 NOS samples classified as LIPO, and found that they appropriately overexpressed genes associated with adipocyte differentiation including adiponectin, insulin-like growth factor 1, and adipocyte fatty acid binding protein 4 , respectively, as compared to the known non-LIPO samples, . These findings confirm the utility of the classifier in real time and routinely collected paraffin sarcoma samples and its capacity to detect previously unrecognized tissue differentiation lineage in truly unclassified sarcoma tumors.In order to evaluate whether STS classification bears potential biologic or therapeutic implications, we estimated the probability of activation of known oncogenic pathways in individual samples, using validated gene expression “read outs” previously generated in vitro as a result of controlled experimental activation of these pathways. We focused on Src, Ras and PI3K pathways, for which pharmacologic inhibitors are currently in clinical development in sarcoma, and we assessed tumors from the 3 Affymetrix oligonucleotide U133A datasets  in our study. Since these gene expression models of pathway activation were generated using oligonucleotide Affymetrix arrays, and most Affymetrix probesets included in these predictors were not present in the cDNA datasets, we did not assess pathway activation in tumors from the 2 cDNA datasets as these predictions would have been less reliable.The probability of activation of the Src, Ras and PI3K pathways was statistically significantly different between different subtypes . More specifically, FIBRO demonstrated higher probability of Ras and PI3K pathway activation (Mann-Whitney p = 0.044 and p = 0.013 respectively) and LIPO demonstrated higher probability of Src pathway activation (p<0.001) and lower probability of PI3K pathway activation (p = 0.06) compared to the rest of the samples . ConversIn order to assess whether MFH reclassification using our 170-gene predictor is tracking specific oncogenic pathway activation patterns, we evaluated the activation status of Src, Ras and PI3K pathways in the 30 MFH samples of the U133 datasets using the aforementioned gene expression “readouts”. Similar to their predicted STS subtypes, MFH sarcomas predicted as fibrosarcomas (MFH-FIBRO) had similarly high average probability of PI3K pathway activation (0.99 vs 0.99 in MFH-FIBRO and FIBRO respectively) and similarly low average probability of Src pathway activation (0.01 vs 0.13 respectively). Furthermore, MFH sarcomas predicted as liposarcomas (MFH-LIPO) had similar average probability of PI3K pathway activation (0.75 vs 0.73 in MFH-LIPO and LIPO respectively). Similar to LEIO, MFH-LEIO did not demonstrate any specific pattern of pathway activation compared to the rest of the samples. There were only two MFH samples predicted as synovial, so no firm conclusions could be reached for this subset.Given the modest predictive value of histology for chemotherapy response, we investigated whether patterns of oncogenic pathway activation are associated with resistance to commonly used chemotherapy agents in sarcoma. For this purpose, we used gene expression readouts (previously developed using Affymetrix arrays from the NCI 60 cancer cell line panel) predicting the probability of resistance to adriamycin, docetaxel and cyclophosphamide, . The probability of resistance to each chemotherapy drug was estimated for each individual sample included in the U133 datasets of our study. As with pathway activation predictions, we did not assess the possibility of chemotherapy resistance in tumors from the two cDNA datasets as these predictions would have been less reliable (most Affymetrix probesets from these predictors were not present in the cDNA datasets).We then performed hierarchical clustering of the samples based on each sample's probability of activation of Src, Ras and PI3K pathways, and observed that samples were classified into seven clusters with distinct pathway activation patterns. As shown in It has been postulated that expression profiles may partly be surrogates for microRNA alterations in sarcoma. Previous in vitro data indicate that Ras is regulated by the let-7 microRNA family Soft tissue sarcomas are heterogeneous neoplasias, thus conceptually well-suited for application of targeted therapies In this study, we integrated 5 publicly available sarcoma microarray datasets A particular challenge for sarcoma classification remains the elusive nature of “MFH” sarcomas. Microarray studies indicated that MFH tumors comprise a complex group not forming a distinct cluster, raising the possibility that MFH does not represent a unique molecular category We showed that our validated multi-gene predictor can reclassify MFH and other uncharacterized tumors into one of the major subtypes, the majority predicted as leiomyosarcomas, liposarcomas and fibrosarcomas. The validity of this reclassification was confirmed by sophisticated bioinformatics approaches , 6 incluIt should be noted, however, that it is possible that certain MFH/NOS tumors may be so undifferentiated that they may not harbor any distinct lineage. In this case, our predictor would result in “overclassification” of these samples, forcing them into one of the conventional classes. The extent of this error is not possible to know with certainty. The fraction of MFH samples that did not co-cluster with their predicted subtype in The second aim of our study was to explore whether genomic reclassification of sarcomas using our predictor bears biologic and therapeutic implications. Several oncogenic pathway inhibitors are currently undergoing clinical trial evaluation in sarcoma including drugs targeting the PI3K/mTOR (deferolimus and everolimus), Src (dasatinib and AZD0530), and the Ras/Raf pathway (sorafenib). However, the activation status of these pathways in individual sarcoma samples and across different histologic subtypes has not been possible to determine, making it difficult to prioritize patients for targeted therapies. For this purpose, we applied previously validated gene expression “read outs” of oncogenic pathway activation to individual samples, and discovered that different subtypes demonstrate distinct patterns of activation of these oncogenic pathways. Interestingly, reclassified MFH demonstrated similar patterns of oncogenic pathway activation as their corresponding predicted subtypes, providing further evidence that the 170-gene predictor reflects the overall molecular program in sarcomas, with therapeutic implications.in vitro demonstration of novel chemoresistance mechanisms were beyond the scope of this study, our findings reveal interesting therapeutic research strategies that can be studied in properly designed prospective studies. For example, prior knowledge of oncogenic pathway activation patterns in individual sarcoma samples may aid in prioritizing patients for novel molecularly targeted agents, conventional chemotherapeutic agents, or combinations thereof. However, our study was limited by the lack of clinical data (i.e. chemotherapy response or outcome data) linked with the public microarray datasets we used, a fact that prevented us from being able to further test the chemoresistance patterns identified in our analysis.Although we showed an association between different histologies and patterns of oncogenic pathway activation, it is also well-known that the association between histology and chemotherapy resistance is only modest, and in many cases unproven, in soft tissue sarcomas. Our findings suggest that oncogenic pathway activation patterns, transcending histologic classes, and assessed by gene expression “read outs”, may serve as useful predictors of resistance to chemotherapy drugs commonly used in sarcoma , perhapsFinally, the finding that the gene expression patterns of Ras-activated tumor samples are enriched for gene targets of the let-7 microRNA family, consistent with compelling experimental evidence of Ras regulation by let- 7 miRNAs Our study demonstrates the power of integrated analysis of multiple and diverse microarray datasets, in order to generate and validate clinically useful models and concepts, in a cost effective manner. We identified a multi-gene STS predictor, reproduced for the first time in multiple independent gene expression datasets, and in routinely collected paraffin tissue, which could serve as an aid to standard histopathologic methods, especially in the diagnosis of pleomorphic tumors that are impossible to classify based on state of the art histopathology. Our findings support the concept that MFH and unclassified (NOS) sarcomas can be reclassified into existing sarcoma subtypes, and proposes a tool for clinical application that reflects previously unrecognized lines of sarcoma differentiation. Finally, our results support the notion that genomic classification may carry potential therapeutic implications, and provide novel therapeutic research hypotheses regarding individualization of targeted therapies and overcoming chemotherapy resistance in soft tissue sarcomas.Our study included five public microarray datasets Paraffin specimens from 15 soft tissue sarcomas including 5 NOS were cut into 1-3 mm cores at the BIDMC Histology Core facility. These included 10 paraffin STS samples with known diagnosis  and 5 paraffin NOS samples that had been previously evaluated by a sarcoma pathology expert (J.G) using state of the art current histopathologic methodology and could not possibly be classified into any of the known STS types. These samples, all archived between 2003 and 2006 at the Beth Israel Deaconess Medical Center Pathology Department, were chosen on the basis of tissue availability and adequate RNA yield for microarray studies. IRB approval for tissue utilization was obtained as per standard institutional protocols. Total RNA was isolated using the Qiagen RNeasy formalin-fixed, paraffin-embedded (FFPE) protocol according to the manufacturer's instructions. Whole genome DASL  arrays , containing 24,000 gene transcripts were used to profile the paraffin specimens on an Illumina BeadStation. The DASL array experiments were carried out at the Children's Hospital (Boston) Microarray Core facility as per manufacturer's instructions and as previously described −7) and demonstrated an accuracy of 85%. Sensitivity analysis using an F-test threshold from p<10−7 (100 genes) to p<5×10−7 (160 genes) demonstrated only minimal loss of performance with accuracy of 79–82%. This classifier was mapped across different platforms using Affymetrix annotation files and applied to the four independent public datasets as well as the paraffin based dataset. The overlap of predictor genes across different platforms was as follows: i) Among the NCI cDNA platform and the 3 U133 datasets there was complete overlap of the 1st and 2nd step predictor genes (i.e. 138 genes and 35 genes respectively), ii) Between the NCI cDNA platform and the DASL Illumina (paraffin) dataset the overlap was 136 genes for the 1st and 34 for the 2nd step predictor) and iii) Between the NCI cDNA platform and the cDNA (Stanford) dataset the overlap was 62 genes for the 1st and 19 genes for the 2nd step predictor).Using the Nearest Centroid algorithm Allowing for partial loss of classifier genes and the technical disparity between the different platforms, classifier accuracy was further assessed using leave one out cross validation and a random permutation test in each of the 4 independent public datasets. Further, for a more direct validation of accuracy, we also collated the 3 U133 datasets into one combined dataset after adjusting for platform effect using empirical Bayes method We used the predictor to reclassify the 76 MFH and the 10 NOS samples into the 6 subtypes after mapping the predictor genes across different platforms (from cDNA to Affymetrix U133) as above. No mapping was necessary for the NOS samples that were all contained in the NCI dataset.In order to assess whether our MFH reclassification reflected true molecular similarity of the reclassified MFH samples with their corresponding STS subtypes, we performed unsupervised hierarchical clustering of all tumors (including MFH samples) within each dataset using the complete linkage method and the one minus centered correlation as a distance metric Hierarchical clustering cannot assess molecular correspondence between phenotypes across different datasets. Thus, we examined the molecular similarity between reclassified MFH samples and their predicted corresponding STS subtypes from different datasets using the Subclass Mapping (SubMap) methodology as previously described in vitrohttp://dig.genome.duke.edu/. Since these models were generated using oligonucleotide Affymetrix arrays and very few Affymetrix probesets were present in the cDNA datasets, our analysis was limited to the 3 Affymetrix oligonucleotide datasets of our study. Non-biological experimental variation between the in vitro arrays and the sarcoma datasets was corrected using a previously described batch effect adjustment algorithm We used publicly available and validated gene expression “read outs” of oncogenic pathway activation previously generated by experimentally controlled activation of these pathways Hierarchical clustering of samples based on their predicted probability values of oncogenic pathway activation was performed using the complete linkage algorithm with the Euclidean distance metric To assess whether the gene expression patterns of STS samples were enriched for targets of microRNAs of interest, we used the functional class scoring method which tests the null hypothesis that the list of differentially expressed genes from each microRNA target set is a random selection from the entire project differentially expressed gene list, implemented in the NCI BRB Array Tools software, as previously described Additional details of the statistical methodology and bioinformatics algorithms described above are found in Data S1Revised Supplementary Data File.(0.05 MB DOC)Click here for additional data file.Figure S1MFH-samples predicted as LIPO (MFH-LIPOs) cluster together with both myxoid and non-myxoid liposarcomas.(0.27 MB TIF)Click here for additional data file.Figure S2Complete Clustering Results.(0.43 MB TIF)Click here for additional data file.Figure S3Fold upregulation of selected genes associated with smooth muscle, fibroblast and adipocyte differentiation in MFH tumors predicted as leio-, lipo or fibrosarcoma respectively (compared to the rest of MFH tumors).(0.19 MB TIF)Click here for additional data file.Table S1Sensitivity and specificity of the 170-gene Nearest Centroid predictor for each STS class in the NCI, UK and JAPAN datasets .(0.02 MB XLS)Click here for additional data file.Table S2Sensitivity and specificity of the 170-gene Nearest Centroid predictor for each STS class in the STANFORD and MSKCC validation datasets.(0.02 MB XLS)Click here for additional data file.Table S3Genes included in 1st step predictor.(0.04 MB XLS)Click here for additional data file.Table S4Genes included in 2nd step predictor.(0.02 MB XLS)Click here for additional data file.Table S5Assessment of MFH reclassification across different datasets with the Sub Class Mapping methodology.(0.02 MB XLS)Click here for additional data file.Table S6Gene expression pattern of the Ras activated samples was enriched for targets of microRNAs of the let-7 family.(0.02 MB XLS)Click here for additional data file.Table S7Detailed clustering results of the classified MFH samples.(0.02 MB XLS)Click here for additional data file.Table S8Patterns of oncogenic pathway activation in different STS histologies.(0.02 MB XLS)Click here for additional data file.


The information from different data sets experimented under different conditions may be inconsistent even though they are performed with the same research objectives. More than that, even when the data sets were generated from the same platform, the data agreement may be affected by the technical variation among the laboratories. In this case, it is necessary to use the combined data set after adjusting the differences between such data sets, for detecting the more reliable information.The proposed method combines data sets posterior to the discretization of data sets based on the ranks of the gene expression ratios, and the statistical method is applied to the combined data set for predictive gene selection. The efficiency of the proposed method was evaluated using five colon cancer related data sets, which were experimented using cDNA microarrays with different RNA sources, and one experiment utilized oligonucleotide arrays. NCI-60 cell lines data sets were used, which were performed with two different platforms of cDNA microarrays and Affymetrix HU6800 oligonucleotide arrays. The combined data set by the proposed method predicted the test data sets more accurately than the separated data sets did. The biological significant genes were detected from the combined data set, which were missed on the separated data sets.By transforming gene expressions using ranks, the proposed method is not influenced by systematic bias among chips and normalization method. The method may be especially more useful to find predictive genes from data sets which have different scale in gene expressions. Data sets that are created for the same purpose in different laboratories have accumulated rapidly. The results are often inconsistent due to the utilization of different platforms, sample preparations, or various technical variations. In this case, if a combined data set were analyzed after adjusting systematic biases that exist among different data sets derived from different experimental conditions, the power of statistical tests would be improved by an increase in the sample size.When the results from different data sets are inconsistent, usually the common portions can be adapted for stability. This type of meta-analysis involves a set of classical statistical techniques  and has Sometimes gene expression ratios may include outliers as a result of incomplete experimental conditions, and these values can cause unreliable results by their strong influence. The usage of the categorized values of gene expression ratios can reduce the influence of outliers in this case and may improve the prediction accuracies in the classification of different experimental classes. The usage of the discrete values has advantage that is more concise to represent and specify, easier to use, and conducive to improved predictive accuracy . The dis2 test to determine when adjacent intervals should be merged. A number of entropy-based methods have recently come to the forefront of work on discretization [The simplest discretization methods are the Equal Interval Width and Equal Frequency Intervals methods. Kerber  suggestetization . Fayyad tization  use a retization  to contrtization , which uIn this study, gene expression ratios were transformed with their ranks for each data set. Next, the transformed data sets were combined and a nonparametric statistical method was applied to the combined data set to detect informative genes with high prediction accuracy. The performance of the proposed method using data sets derived from different platforms and different RNA sources was evaluated.The relationship between the number of genes and OOB (Out of Bag) error rates was investigated using data A, data B and data AB, which represent the data sets with total RNA, amplified RNA, and the combined data, respectively.The OOB error rates were calculated for randomly selected genes 500 times repeatedly with the same size and averaged them. The OOB error rates decreased as the number of informative genes increased.As shown in Figure The prediction accuracies were compared using two original colon cancer data sets as training data sets, which were experimented with different RNA sources.While the prediction accuracy of data B using data A as a train data set was higher than 95%, the accuracy of data A using data B was lower than 80% Figure . This inFigure Tumor 211 and Tumor 86 data sets were predicted more accurately with train data A and data B, respectively. This indicated that the prediction accuracy depends on train data sets in the ME method. Figure Figure The descriptions of six discriminative genes selected from the combined data set, not two separated data sets, are summarized in Table AA485151 was upregulated by over five-fold in colorectal adenocarcinoma . AA42521AA455925 is known as a E2F-1 regulated gene . Xq26 isThe descriptions of 4 colorectal cancer related genes and 9 cancer related genes are summarized in Table The prediction accuracies of combined data sets derived from different platforms were investigated.While the prediction accuracies of data A and data B on affy were low with a small number of genes, it increased as the number of genes was increased. By combining data A and data B, the prediction accuracy on affy was improved, as shown in Figure The proposed method was evaluated using a NCI 60 cell line data set, and the prediction accuracies of the proposed method and the ME method were compared.The prediction accuracies of both methods were compared as the number of genes was increased to 300, and they improved as the number of genes was increased, regardless of train and test data sets Figure . The preWhen Oligo data was used for train, the variation in accuracies was relatively small and prediction accuracy was high. It indicated that the Oligo data set predicted the cDNA data set more stably and accurately than the cDNA data set did. There was rare improvement in prediction after 20 or 30 genes, and this showed that a small informative gene set is sufficient for discrimination. It was also confirmed that the prediction accuracies were robust against the train data sets in the proposed method, while those in the ME method depended on the train data sets and there was significant difference between them Figure .The designed 25-mer oligochips from Affymetrix provide an absolute value of expression in an RNA sample, while cDNA microarrays perform a two-color competitive hybridization that gives the transcript expression in two samples. Also, long oligonucleotide platforms  also use hybridization, the relative measurements resulted in higher precision than did absolute measurements on this platform . TherefoAdditionally, some previous studies indicated that the data sets from different microarray platforms should not be combined straightforwardly -27. HoweThere also exist inter-study biases among several microarray data sets tested with different RNA sources even when they are from the same laboratory and platform. Previous studies showed that there were some differences in results from data sets tested with different RNA sources, and the sensitivity to detect differential gene expression from a microarray data set using amplified RNA was also different compared to using total RNA ,30.An attempt to combine these different types of data sets is the usage of abstraction of expression values such as ranks or discretized values -12. ThesIn colon cancer data sets derived from cDNA microarrays, a data set created with total RNA predicted more accurately a data set created using amplified RNA than vice versa Figure . HoweverIn the colon cancer data set derived from oligonucleotide arrays, the prediction accuracies were improved by combination with cDNA data sets. Although two data sets derived from different experimental conditions have different scales in gene expressions, such a different scale of gene expressions could be compensated by discretizing gene expression. Therefore, no transformation method was required to match these two types of data sets except only the ranking of gene expressions.In the NCI 60 cell line data sets from two different platforms, different types of two data sets were used by alternating train and test data sets. The prediction accuracies in datasets that were transformed by the ME method were greatly different according to train sets, while those by the proposed method were accompanied by stable fluctuation in the prediction accuracies. The Oligo data set predicted the cDNA data set more stably and accurately than vice versa. While the prediction accuracies in the ME method depended on train and test data sets and the significant difference existing between them, they were more robust against train data in the proposed method Figure . In spitIn this study, we transformed microarray data using ranks of gene expressions to combine data sets created in different experimental conditions. The proposed method may be especially more useful to find discriminative genes from data sets that have different scales of gene expression ratios.The data sets used in this study are summarized in Table Two cDNA microarray data sets, data A and data B, experimented with 154 colorectal tissues  were used as train data sets for evaluation of the proposed method. These two cDNA microarray data sets derive from different RNA sources, which were total RNA and amplified RNA. Previous studies have concluded that there were differences between the results from these two types of data sets and the sensitivity to detect differential gene expression from microarray data sets using amplified RNA was also different compared to using total RNA ,30. It wTwo more cDNA data sets, Tumor 86 and Tumor 211, were experimented with amplified RNA and under different batches, and they were used as test data sets. They included only colon tumor tissues. These colon cancer data sets performed with cDNA microarrays were from the Cancer Metastasis Research Center of Yonsei University, Seoul, Korea. One more colon cancer data set was used, which was performed with the Human 6800 Gene Chip Set (Affymetrix). It was obtained from microarray database of Princeton University  and it iTo evaluate the performance of the proposed method in different platforms, NCI 60 cell line data sets derived from different platforms were also used. Gene expression data sets for NCI-60 using 9,706 cloned cDNA microarrays and 6,810 gene Affymetrix HU6800 oligonucleotide arrays were obtained separately from the additional files of Lee et al. , and theGene expression ratios were normalized such that they would have similar distributions across a series of arrays and the normalization process was executed using the 'limma' library of the R package . The cDNFor transformation of the data set, gene expression ratios are rearranged in order of expression ratios by each gene, and the ranks are matched with the corresponding experimental group. If the experimental groups are homogenous, the ranks within the same experimental group would be neighboring. This process can be seen as similar to the first step in the nonparametric Mann-Whitney U test. The process of discretization of gene expressions is summarized in the following steps:(1) Rank the gene expression ratios within a gene for each data set.(2) List in order of the ranks and assign the order of gene expressions to the corresponding experimental groups.(3) Summarize the result of (2) in the form of a contingency table for each gene.(4) Test the relationship between the gene expression patterns and experimental groups for each gene.When there are three data sets to be combined, the data sets can be added by each entry as shown in Table A method for discretizing continuous attributes based on a minimal entropy (ME) heuristic, presented by Catlett  and FayyTmin, which minimizes the entropy function over all possible partition boundaries, is selected as a binary discretization boundary. This method can be applied recursively to both of the partitions induced by Tmin until some stopping condition is achieved, thus creating multiple intervals on feature A. It must be evaluated N-1 times for each attribute with N the number of attribute values [For a given feature A, the boundary e values . The libe values  was usedAfter the summarization of gene expression ratios in the form of a contingency table for each gene, as shown in Table When the sample size for each experiment is small, generally less than five, Fisher's exact test is recommended rather than the Chi-square test.In order to evaluate the predictive accuracy of the selected significant gene set, the Random Forest (RF) test  was used1, B2, ..., Bn} by allowing repetition. (2) Use a Bk to build a tree classifier Tk, and classify Bms (m≠k) data (out-of-bag (OOB) samples). (3) Calculate classification errors of Bms and obtain an average for them which is the overall classification error (OOB error). (4) Calculate the prediction accuracy of test data sets using the classifier built in (2).(1) Generate n data sets of bootstrap samples {BANOVA: Analysis Of Variance; OOB error: Out Of Bag error; ME: Minimal Entropy.KYK participated in the design of algorithms, performed statistical analysis and drafted the manuscript. DHK performed the microarray experiments. HCJ participated in getting the consent form the patients and obtained the clinical data. HCC participated in the study design and data interpretation. SYR conceived of the study, participated in its design and coordination, and finalized manuscript.


Development of efficient analytic methodologies for combining microarray results is a major challenge in gene expression analysis. The widely used effect size models are thought to provide an efficient modeling framework for this purpose, where the measures of association for each study and each gene are combined, weighted by the standard errors. A significant disadvantage of this strategy is that the quality of different data sets may be highly variable, but this information is usually neglected during the integration. Moreover, it is widely known that the estimated standard deviations are probably unstable in the commonly used effect size measures (such as standardized mean difference) when sample sizes in each group are small.We propose a re-parameterization of the traditional mean difference based effect measure by using the log ratio of means as an effect size measure for each gene in each study. The estimated effect sizes for all studies were then combined under two modeling frameworks: the quality-unweighted random effects models and the quality-weighted random effects models. We defined the quality measure as a function of the detection p-value, which indicates whether a transcript is reliably detected or not on the Affymetrix gene chip. The new effect size measure is evaluated and compared under the quality-weighted and quality-unweighted data integration frameworks using simulated data sets, and also in several data sets of prostate cancer patients and controls. We focus on identifying differentially expressed biomarkers for prediction of cancer outcomes.Our results show that the proposed effect size measure (log ratio of means) has better power to identify differentially expressed genes, and that the detected genes have better performance in predicting cancer outcomes than the commonly used effect size measure, the standardized mean difference (SMD), under both quality-weighted and quality-unweighted data integration frameworks. The new effect size measure and the quality-weighted microarray data integration framework provide efficient ways to combine microarray results. Microarray technology has been widely used in identifying differentially expressed genes ,2 and inRecent studies show that systematic integration of gene expression data from different sources can increase statistical power to detect differentially expressed genes while allowing for an assessment of heterogeneity -18, and The first category is a combined analysis of all the data. Each data set is first preprocessed to clean and align the signals, and then these preprocessed datasets are put together so that the integrated data set can be treated as though it comes from a single study. In this way, the effective sample size is greatly increased. Several transformation methods have been proposed to process gene expression measures from different studies ,14,17,20The second strategy is to combine analysis results obtained from each study. The basic idea is to combine evidence of differential expression using a summary statistic, such as the p-value, across multiple gene profiling studies and then to adjust for multiple testing. For example, Rhodes et al. ,12 combiThe third strategy involves taking inter-study variability into account when estimating the overall effect for each gene across studies, and then basing conclusions on the distribution of these overall measures. For example, Choi et al.  focused The widely used effect size measure in this type of models is the standardized mean difference ,26. It hHowever, recent studies show that differentially expressed genes may be best identified using fold-change measures rather than t-like statistics . Fold chMost data integration papers in microarray analysis have not used measures of quality to refine their analyses ,17,20,22In this paper, we reparameterize the effect size measure for each gene in each study as the log ratio of the mean expressions of the two groups being compared. Following the method proposed by Hu et al. , we thenv, that alters the tolerance of the quality weight to the detection p-value significance levels, was set to 0.05.For Affymetrix expression data, we previously developed a quality measure based on the detection p-values  that refg in individual study [rgl represent the raw expression value for gene g and subject l and xgl = log(rgl). The standardized mean difference (SMD) of xgl is given byThere are many ways to measure effect size for gene al study . A commog in treatment group (t) and control group (c) in a given study, respectively. yg1 is given by Cooper and Hedges [where d Hedges (2)n(n = nt+nc) samples, an approximately unbiased estimate of yg1 is given by For a study with given by .Here, we propose an alternative method to measure effect size based on the log ratio of means (ROM), that is, the log fold-change given byg in treatment group (t) and control group (c) in a given study, respectively. The estimated variance yg2 can be estimated using delta method [In contrast to the previous approach, here a method  as followhere yg1or the ROM effect size measure yg2 into a hierarchical model and to test for differences between groups. For either measure, we can write, for study i and measure m (m ∈ SMD or ROM),Any defined quality measure can be incorporated into integrative analysis of gene expression profiles using a quality-adjusted meta-analysis framework . The ratg with effect size measure m, μgm represents the average measure of differential expression across the I studies for gene g. Here, μgm are gene-specific while yigm are gene and study-specific . The quantity ith study. Following Hu et al. [μgm by taking the quality qig for gene g and study i into accountwhere u et al. , we can qig and yigm are quality measure and the estimated effect size based on measure m for gene g in study i, respectively. where iability . Here weg across all I studies can then be computed asA test statistic to evaluate differential expression of gene g by calculating the p-value corresponding to the z statistic; then we estimated the false discovery rates (FDR) for each significance level, to take into account the number of tests performed [We evaluated the statistical significance of gene erformed . A detaizgm using either the log ratio of means (m = 2) or the standardized mean difference (m = 1) as WROM and WSMD, respectively, in the quality-adjusted modeling framework, and as UWROM and UWSMD, respectively, in the quality-unadjusted modeling framework, where qig = 1.We refer the approaches of estimating Following previous studies to generate Affymetrix probe level data ,36, we mYjgk and Wjgk are PM and MM intensities for the probe j in probeset g on array k respectively. O denotes optical noise, independently drawing from PM (XX = PM) and MM (XX = MM), respectively. We set μMM = μPM = 4.6 and assumed that e ~ N, so that PM (XX = PM) and MM (XX = MM), respectively, and the coefficient 0 < Φ < 1 accounts for the fact that for some probe-pairs the MM detects signal; When probe j of gene g is attached by picking up stray signal, Φjg is generated as Φjg~Beta, otherwise, Φgj = 0. Since S follows a power law, we set its base to 2. Therefore, if we denote γg as the baseline log expression level for probeset g, we can select log2(γg) expression levels from 0 to 12, which can be generated from γg~12* Beta+1. δg is the expected differential expression of gene g in covariate X. αjgk is the signal detecting ability of probe j in gene g on array k, which is assumed to follow a normal distribution with mean zero and signal detection variance where rom  and .  represt and control group c with ith study, respectively. We generated G genes and assume the proportion of expressed genes is q and the proportion of differentially expressed genes is d of the G*q expressed genes in each study. We ran three simulation models following the above design by varying treatment effects on the signal between 1.0  and 2 (large) with interval 0.5. The specific parameters used in the five models are summarized in Table We generated two Affymetrix microarray data sets, which are assumed to be from two independent studies. In each of the two data sets, we assume treatment group We used summarized receiver operating characteristic (SROC) curves to compare performance, where the test sensitivities and specificities (true positive and true negative proportions) for a range of p-value cutoffs were averaged over 500 simulated datasets in each study. The SROC curve's overall behavior can be measured by the area under the curve (AUC) .We used gene expression data on prostate tumours and controls from four studies -41. The We evaluated the performance of our method using simulated Affymetrix probe level expression data generated from a model incorporating probe level effects, optical noise, and non-specific binding, as well as true signals ,36. FollTo evaluate the significance of genes identified by quality-adjusted and quality-unadjusted data integration frameworks under ROM and SDM effect size measures, we compared the ranks of a set of known prostate tumor genes. This set of prostate cancer genes are from two sources: The first one is from Welsh study , where tIt should be noted that some of the known tumor genes identified by our new methods have much better ranks than the conventional methods. For example, the ranks of tumor genes FASN and TACSTD1 are 15 and 6 by WROM and 13 and 6 by UWROM while the ranks of these genes are 72 and 289 by WSMD and 231 and 413 by UWSMD.In order to evaluate the overlap between genes identified by our meta-analysis procedures and those identified in a single study, we analyzed each of the three training data sets  as shown in Table To further confirm the validity and biological relevance of the meta-signatures identified by the proposed effect size measures and different data integration frameworks, we evaluated the discriminative power for the top 150 differentially expressed genes identified by the four meta-analysis methods, respectively, using an independent data set listed in Table Many microarray experiments include only a few replications, therefore, it is critical to improve the effect size estimation in meta-analytic procedure. With small sample sizes, the traditional SMD estimates are prone to unpredictable changes, since gene-specific variability can easily be underestimated resulting in large statistics values. In this study, we re-parameterized the traditional SMD-based effect size measure by using a log ratio of means as an effect size measure for each gene in each study. Our results show the new effect size measure has better performance than the traditional one.Traditional wisdom for statistical analysis recommends that highly skewed data should be transformed prior to analysis. It is therefore unexpected, perhaps, that the ROM measure  gives better prediction accuracy than the SMD measure . Since the signals from Affymetrix are expected to be a mixture of background or non-specific binding and true signal, and only the true signal is expected to follow a power law, using the log transformation up front may be introducing variability, in particular for genes with low levels of expression. Furthermore, for genes whose expression levels change dramatically between experimental groups, the apriori log transformation may be inappropriate in the group with low expression levels.We noticed that the ranks of some of the known tumor genes . Some data sources might, for example, be more informative than others. A statistically sound data integration framework should, therefore, take these into account. One approach towards this goal is to develop suitable quality measures for different data types and these measures are then integrated into the statistical models. We used a simple quality measure associated with both log-ratio of means based and standardized mean difference based effect sizes. Our analysis showed this measure works well in the real and simulated data sets.In summary, we combined estimated ROM-based effect sizes for all studies under two data integration frameworks: the quality-unweighted random effects models and the quality-weighted random effects models . CompariROM: ratio of mean; WROM: log ratio of mean used as the effect size measure in weighted meta-analysis Framework; UWROM: log ratio of mean used as the effect size measure in unweighted meta-analysis framework; SMD: standardized mean difference; WSMD: standardized mean difference used as the effect size measure in weighted meta-analysis framework; UWSMD: standardized mean difference used as the effect size measure in unweighted meta-analysis framework; PM: perfect match; MM: mismatch; MLE: maximum likelihood estimation; NSB: non-specific binding; RMA: robust multi-array average; SROC: summarized receiver operating characteristic; AUC: area under the curve; FDR: false discovery rate; SVM: support vector machines; DLDA: diagonal linear discriminant analysis.JB initiated the study and proposed the ratio of means effect size measure and weighted data integration framework. CG proposed the quality weight measure and simulation framework. PH carried out all the data analysis and drafted the manuscript. All authors read, contributed to, and approved the final manuscript.Data integration methods. The document describes the data integration methods in detail.Click here for file


Microarray data is frequently used to characterize the expression profile of a whole genome and to compare the characteristics of that genome under several conditions. Geneset analysis methods have been described previously to analyze the expression values of several genes related by known biological criteria  at the same time and the cost of these methods allows for the use of more values to help discover the underlying biological mechanisms.As several methods assume different null hypotheses, we propose to reformulate the main question that biologists seek to answer. To determine which genesets are associated with expression values that differ between two experiments, we focused on three ad hoc criteria: expression levels, the direction of individual gene expression changes (up or down regulation), and correlations between genes. We introduce the FAERI methodology, tailored from a two-way ANOVA to examine these criteria. The significance of the results was evaluated according to the self-contained null hypothesis, using label sampling or by inferring the null distribution from normally distributed random data. Evaluations performed on simulated data revealed that FAERI outperforms currently available methods for each type of set tested. We then applied the FAERI method to analyze three real-world datasets on hypoxia response. FAERI was able to detect more genesets than other methodologies, and the genesets selected were coherent with current knowledge of cellular response to hypoxia. Moreover, the genesets selected by FAERI were confirmed when the analysis was repeated on two additional related datasets.The expression values of genesets are associated with several biological effects. The underlying mathematical structure of the genesets allows for analysis of data from several genes at the same time. Focusing on expression levels, the direction of the expression changes, and correlations, we showed that two-step data reduction allowed us to significantly improve the performance of geneset analysis using a modified two-way ANOVA procedure, and to detect genesets that current methods fail to detect. The major issue when studying datasets with many tests and few replicates in general and microarray datasets in particular is the decreased power of the analysis. Indeed, to reduce the risk of detecting false positives due to the lack of a sufficient number of replicates, statistical tests allow for higher threshold levels as the number of replicates decreases. This implies an unavoidable increase of the number of false negatives and, thereby, a decrease in power.To compensate for this lack of information about expression data generated by microarrays, some authors have tried to work at the probe level, breaking down probesets into smaller components. Nonetheless, most researchers prefer to work at the probeset level where data can be analyzed in different layers  .Among these methods, the analysis of genesets with regard to known or observed biological criteria holds high potential for fundamental as well as clinical research. For example, members of the same metabolic pathway can be studied simultaneously to assess modifications of the pathway under certain conditions, such as the action of a drug, or to diagnose a pathology, etc. . This la- group discovery: empirical data is used to identify new criteria to discover new genesets. This task uses several different approaches such as data clustering -9 and th- geneset analysis: previously defined genesets are used to guide empirical data analysis, to determine whether the expression values of known genesets are correlated with the conditions compared in the experiments -17.0): "Which known genesets are associated with different expression profiles under the two conditions compared?". We discuss here the ability of current methods to provide an answer to this question.The research presented here focuses on statistical geneset analyses. Our initial motivation was to answer the seemingly trivial question  may be expressed at various levels, over- or under-expressed or correlated (coordinated response of member genes).This main question Q0, we present the FAERI methodology , an ANOVA-2-like procedure tailored to detect genesets with differing expression profiles, regardless of the nature of the individual responses.To provide an answer to question Q0. The methods discussed below were selected to provide an overview of the main strategies followed by the authors, or due to their pioneering approach.Several geneset analysis methods have been published recently. Each available method provides an answer to question QOver-representation analysis methods are not considered in this paper, as they focus on the top list obtained by individual analysis and aim to identify only those groups of genes that are represented by the most differentially expressed probesets. This approach thus facilitates the interpretation of individual analyses, by giving annotation clues about biological meaning. However, such methods require the ad hoc selection of an individual statistic, and the definition of an arbitrary threshold for the detection of individual genes, whatever the individual statistic used. Thus, over-representation analysis does not allow for the detection of genesets associated with subtle individual changes according to the individual statistic chosen -21. For Table comp  methods can be classified based on their competitive or self-contained null hypothesis. Competitive methods sort the groups focusing on question Qmp Table  23-28].-28.comp lf Table . The sigThe GSEA methodology and its derivatives make use of a hybrid strategy providing results that are harder to interpret. The null model used to develop the methods relies on the competitive approach, but is later evaluated by label sampling, thus mixing both strategies ,25,27.The second methodological difference between the methods concerns definition of the geneset score evaluated. Methods using two steps first compute an individual statistic that is then used in a second step to compute a geneset statistic -28,33. C0. Therefore, we propose to reformulate the hypothesis used with the available methods with respect to the biological properties of the sets , data used (4th column), statistic used (5th column) and significance evaluation procedure (6th column). The other columns provide additional information.Table 0 includes all of the other questions  related to the nature of the groups tested. Three criteria must be taken into account to describe the groups: expression levels (a), the direction of the expression change (b) and correlations between the group members (c).Question Qt or SAMGS d statistics characterize expression differences, regardless of the individual expression levels [(a) It is important to study expression levels as highly expressed probesets may mask differences observed for probesets with lower expression levels. Two-step methods such as the SAMGS are not influenced by this criterion. For example, the Student n levels . Moreovet [2d) and expression change variability (2τ) [(b) Individual expression values may reveal up or down regulation of the genes depending on the conditions tested. One way to take the direction of expression changes into account and deliver results with no dependence on this criterion is to avoid studying mixed groups with a global null effect (50% over and 50% under expressed). Two-step methods may be adapted to use the absolute value of a unidirectional individual statistic (Student t ), or itst ). Alternt . Among gity (τ2) ,31.0 takes both possibilities into account. Two-step methods based on a sum or a mean, like the SAMGS, do not need further adaptation since they explicitly rely on the sum of individual pieces of information [uC, see Table (c) Genes may be associated with correlated or variable expression changes. Question Qormation ,23,28,300.It appears that only two-step methods, as well as the SAMGS, may simultaneously provide an answer to the three criteria considered and thus evaluate more completely question QSoftware packages developed thus far are not always able to fine-tune the appropriate combination of steps with regards to the criteria. Moreover, it is not systematic to take the direction of the answer into account or to rely on partial genesets  ,28,30,33Global methods, on the other hand, ignore the possible bias associated with expression level. The SAMGS procedure involves a preliminary step of variance stabilization based on the SAM individual analysis method, which requires that the expression values of all probesets be estimated ,36-38.Competitive procedures aim to identify groups with bigger differences and ignore groups associated with the phenotype but with a moderate answer compared to the others. The self-contained null hypothesis thus provides a more appropriate answer, where each set is evaluated according to its possible distribution range -28,33.0 as completely as possible. Several methodologies may be selected to answer complementary questions. Gentleman et al. previously suggested to separately study sub-groups defined by the direction of expression changes and to define core sets of genes common to several genesets . As a perspective of the GlobalAncova methodology, Mansmann and Meister proposed to study the interaction between the effect of the condition and group composition, to take the variability of the individual responses into account. The hypothesis associated with this test complements the unidirectional null hypothesis, but also provides a partial answer to question Q0 since it ignores unidirectional correlated groups (no interaction).There are several possible strategies to evaluate question Q0 based on a single self-contained test. To optimally analyze expression data, our model relies on a global strategy because two-step methods suffer from a loss of information (correlation) caused by the substitution of data with individual statistics. We introduce the FAERI method that was tailored from the 2-factor ANOVA procedure. Our aim was to group the information associated with individual differences into one single statistic taking direction, correlation and expression level into account and to combine the advantages of the GlobalTest (groups with a variable answer), GlobalAncova  and SAMGS (independence with expression level) models whilst avoiding their respective limitations.The work reported below proposes an alternative strategy to answer question QBosco et al. (2006) described the E-MEXP-445 dataset, available in the ArrayExpress repository. The experiment compares samples extracted from human monocytes. RNA was hybridized on 6 Affymetrix HGU-133a microchips, among which 3 normoxia samples and 3 samples from cultures grown under hypoxic conditions .Vengellur et al. (2005) studied cellular response to hypoxia on human Hep3b hepatocytes cell lines, considering 3 additional treatments known to mimic hypoxic conditions: cobalt chloride (100 μM), nickel chloride (100 μM), and DFO (100 μM). mRNA was extracted separately from two biological replicates, and cRNA was hybridized on Affymetrix GeneChip HG-U95Av1 microchips. We used part of this dataset to compare samples grown under normoxic and hypoxic conditions, and did not use the other samples (used by the authors to compare hypoxia with molecules that mimic hypoxic conditions). This dataset is available on GEO (GSE-1056) .Kim et al. (2006) described the GSE-4086 dataset, available on GEO. The authors studied cellular response to hypoxia on human B lymphocytes (P493-6 cell line). Two biological replicates were used to extract mRNA from separate cultures. cRNA was hybridized on Affymetrix GeneChip HG-U133A microchips .In the work reported here, the three hypoxia datasets were preprocessed using GCRMA to study the effect of hypoxia and compare the ability of each geneset analysis method to detect common pathways from several datasets featuring a limited number of replicates.Xiao et al. (2008) report a study of osteoporosis using microarrays hybridized with 10 low and 10 high BMD samples . B-lymphocyte mRNAs were isolated from 70 ml of blood, and cRNA was hybridized on Affymetrix GeneChip HG-U133A microchips. This dataset is available both on GEO GSE-7429) and ArrayExpress (E-GEOD-7429) [9 and ArrAs several genes are associated with multiple probesets, a bias could be introduced during the analysis at the geneset-level. In the work reported below, we used only one probeset for each gene studied on the microchip, by selecting the probeset associated with the largest variability, to avoid giving too much weight to a gene due to the number of associated probesets. As a result of this selection, the words "probesets" and "genes" are synonyms in this text. The multiplicity of the probesets is not considered in this publication.t test [The analysis of variance or ANOVA relies on the study of the variability of the data with respect to one or more criteria to determine if that(those) criterion(a) is(are) responsible for the values observed. The simplest model, relying on a single criterion is equivalent to the Student t test . When anThe model that we chose to test the validity of this approach states that the expression data observed can be explained by dependence of the intensities measured on the probesets, the condition and the interdependence between those two parameters, since each probeset can potentially be associated with a different response under each of the tested conditions.The mathematical representation of this model is given by Equation 1.i is the contribution of factor a, bj is the contribution of factor b, abij is the contribution of the interaction of factors a and b and E(ij)k is the residual contribution.Where μ is the general mean, aThe comparison between the different criteria relies on the ratio of the mean squares associated with each criterion. The statistic computed for each criterion is evaluated by Fisher's F distribution using the appropriate number of degrees of freedom (equations 2 to 4).x is the F score associated with factor x, MSx is the mean square value associated with factor x, nx is the number of levels for factor x.Where FIt is important to note that when the effect of the interaction is significant, no decisions can be made for the other criteria. The study of the effect associated with the condition is close to the GlobalAncova method ,44. The Geneset analysis of differential expression is influenced by the direction of the expression changes of the members and by possible interactions between the probesets and the conditions ,31,44-46t is unchanged. Note that this procedure renders data closer to the conditions of application of the ANOVA 2 .Probesets belonging to the same geneset can be expressed at various levels. From a biological point of view, a weakly expressed geneset might have a bigger impact on physiology than a strongly expressed one. We propose to standardize the data so that, in principle, all the probesets have the same potential for response under two conditions compared, based on a classical reduction of data producing Z values. The standardization is computed for each probeset using Equation 5 by pooling the data associated with both conditions and computing the observed mean and variance. The resulting reduced data is distributed around a mean value of 0 with a variance of 1. The information about the difference between the conditions is retained. At the end of this operation, the result of an individual analysis based on the Student ij is the expression value associated with the ith probeset in the jth microchip, Mpool, i and Spool, i are the means and standard deviations computed for probeset i from all experiments (step 1). This procedure is repeated for each probeset in the dataset.Where X0, with no dependence on the direction of differential expression, two approaches can be proposed: up- and down-regulated probesets may be studied separately or simultaneously by computing a cumulative value for the absolute difference. The first solution consists of defining "activated" or "repressed" genesets to conduct a proper analysis. This approach relies on current knowledge that is incomplete, but is useful for diagnosis purposes to determine whether a patient has a pathology by analyzing signature genesets. The second solution is addressed by several methods by defining an individual statistic independent of the direction  [Several probesets may be activated or repressed at the same time, but may also be associated with a variable expression change, depending on the biological context studied and the definition of the genesets. To provide a full answer to question Qatistic) ,37,47,48Using a multivariate procedure, we propose to study the absolute response of the genes by directional reduction of the data. This operation involves multiplying the expression values related to down-regulated genes by -1 (Equation 6). The down-regulated genes are identified empirically from the expression data studied.i = MiB - MiA, the difference in means between conditions A and B for the ith probeset, Zij = Z-score associated with the ith probeset in the jth microchip, and ZDij the directionally reduced Z-score associated with the ith probeset in the jth microchip.Where Dt test when using the median as the estimator [It may be interesting to consider the median instead of the mean to evaluate the direction of differential expression. However, the performances tests we conducted previously in individual expression analyses show a reduction in the performance of the Student stimator  on biolot statistic remains unchanged.After this 2-step reduction of the data, FAERI evaluates an F statistic as is done in the ANOVA-2 procedure (Equation 1). The FAERI F* statistic associated with the condition effect is representative of the differences found based on biological reasoning. Each gene provides information on its differential expression, independently of the direction of that difference, and the absolute value of the individual Due to the dependence on geneset size introduced by directional reduction, the significance of the results cannot be evaluated using Fisher's theoretical distribution, which is used by the ANOVA procedure. We will refer to the statistic computed using the FAERI methodology as F*. We tested two solutions to determine the null distribution of the FAERI F* statistic, relying either on permutations or the use of random data.The permutation strategy associated with a self-contained null hypothesis relies on label sampling ,29. Give0 values. Under the null hypothesis, the F* statistic computed on reduced test data should follow the same distribution as the F*0 statistic computed from reduced random data. Conversely, if the expression values of the geneset differ between two conditions, the F* statistic distribution leads to higher values than the F*0 values computed from random data.Due to the Z value reduction step, a second strategy may be used to evaluate the null distribution of the FAERI F* statistic. Indeed, as the null hypothesis states that there are no differences between the two conditions compared and as we postulate that individual expression values follow a normal distribution, Z standardization provides the possibility to establish the null distribution using random data from a normal distribution, and then apply the same 2-step reduction to compute the reference F*This possibility to compute a reference distribution based on random data frees the FAERI method from the limited number of available permutations. Moreover, for each experimental strategy (number of replicates) and each geneset size, the null distribution can be computed once and stored for future analysis.As presented in the introduction, we propose to take the level of individual expression into account, using a first-step reduction of the data. Figure In the second step of FAERI, we used a directional reduction to obtain a cumulated value of absolute expression changes in the genesets. We chose to use an empirical determination of the direction, easier to assess from the data than from prior incomplete knowledge. As the sign of the reduced values is not a feature of the biological mechanisms involved within genesets, the description of the mechanisms involved should be associated with individual analysis of the member genes based on raw data.Figure We tested two procedures to assess the significance of the F* statistic. Figure Mansmann and Meister recommend assessing the significance from permutations instead of the Fisher distribution when analyzing microarray data with a multivariate procedure. Their GlobalAncova procedure evaluates the significance of the test using the label-sampling strategy instead of the classic Fisher distribution . The GloSeveral authors have stressed the need to compare the performances of geneset analysis methods with regard to the correlation between geneset members ,29,50. TWe reproduced the simulation proposed by Ackerman et al. to assess the performances of ANOVA-2, FAERI and several geneset analysis methods currently available, to obtain a comprehensive understanding of the performances with regards to the mathematical structure of the data. Each set was simulated 100 times and Table Comparison of the results illustrates the effect of correlations between members of the sets. For each method tested, the presence of intra-set correlations reduces the performance of the analysis. For the simplest simulated sets , ANOVA-2 provides the best results, followed by FAERI  and GlobalTest, as evaluated using a gamma distribution or permutations (set 5).Moreover, the ANOVA-2 and FAERI procedures are both associated with higher performance compared to all other methods based on the simplest simulation scenario . Indeed, FAERI detects set 1 in 62% of the cases, with a threshold of 0.001 and no false positives (set 3: 0%). At this threshold, FAERI is able to detect roughly 5 times more genesets than the best performing method currently available . As a comparison, SAMGS and GlobalTest require a threshold that is 100 times higher to detect only 60% of set 1. Comparison of sets 4 and 5 highlights the method's abilities to detect genesets where half of the members are not differentially expressed. In this case, FAERI detects 43% of the genesets, compared to 15% when applying GlobalTest with a gamma distribution. The ANOVA-2 and FAERI procedures thus appear to be the most appropriate to analyze unidirectional genesets.The lower part of Table To ease interpretation of method's performances with regard to the mathematical structure of the genesets analyzed, Table Table • ANOVA-2 is the most appropriate method for unidirectional groups;• GlobalTest, which uses a statistic representative of response variability, is suited for the study of both uni- and bidirectional non correlated genesets;• SAMGS, using an individual statistic, appears to be less appropriate for the analysis of correlated genesets, since this information is lost during the procedure;• FAERI, developed to detect sets independently of their mathematical structure, is appropriate for each type of set tested;• ANOVA-2 and FAERI provide an appropriate analysis of sets with correlated members, respectively using unidirectional and bidirectional definition of sets;• Methods making use of a two-step strategy provide lower performance levels (GSA and GSEA).Analysis of a genuine biological dataset offers the opportunity to qualitatively evaluate the results and provide a biological validation of geneset analysis methods through comparison of the results with current biological knowledge. We chose to use three datasets related to response under hypoxic conditions. The definition of the genesets relies on the MSIGDB databank (version 2.5), published by the authors of the GSEA methodology . MSIGDB Table Table This list can be split into 3 categories. First, genesets concerning several metabolic pathways suggest metabolic adaptations occur as a cellular response to oxygen deprivation. Second, several genesets address signaling pathways and are known to be involved in the hypoxic stress response. Genesets of the third category are related to pathologies, mostly involving cancers, for which a hypoxic environment has been observed -54. ThesThese observations should be followed-up by a characterization of the genesets detected by the other methods. Table As Table Setting a significance threshold of 1% for the p-values, the ranking of the methods detecting more genesets common to the two experiments is the following: FAERI.permutations (6 common genesets for at least 14 genesets detected), ANOVA-2 (5/35), GlobalTest.gamma (1/8) and GSA.mean (1/14). The other methods do not intersect. When the significance threshold is set to 5%, the ranking of the methods is the following: FAERI.permutations (34/42), GlobalTest.gamma (20/45), FAERI.null (6/18), ANOVA-2 (15/51), GlobalTest.asymptotic (4/26) and GSA.mean (1/8). The other methods do not intersect. These results suggest that the bidirectional genesets, detected by FAERI and GlobalTest, are an important part of the genesets simultaneously involved in both datasets, compared with the ANOVA-2 and GSA.mean procedures.To quantify the abilities of the methods to provide the same result from different datasets, we propose to compute Pearson's coefficients of correlation on the ranks associated with the genesets analyzed, by pairwise comparison of the three datasets, for each method and each category of genesets defined , and functional class scoring (FCS). This last category of methods relies either on 2-step (post-hoc) or global strategies (using raw data).Functional Analysis: Evaluation of Response Intensities). FAERI is a global methodology tailored from a 2-factor ANOVA procedure by a 2-step reduction of the data, and is evaluated with respect to the self-contained null-hypothesis (using label sampling or random data).In this paper, we address this question by considering the biological properties of genesets with regard to the underlying mathematical properties of the associated expression values. Focusing on expression levels, the direction of regulation and potential correlations between geneset members, we developed the FAERI methodology (0 ("Which known genesets are associated with different expression profiles under the two conditions compared?").Evaluations performed on random data reveal the ability of each method to detect sets simulated according to 8 scenarios, relying on mathematical properties. The ANOVA-2 procedure performs best when analyzing unidirectional genesets and FAERI performs similarly. Both methods outperform each currently available method. When the definition of the sets involves members regulated in both directions, ANOVA-2 performance drops, as expected, and our proposed FAERI methodology outperforms all tested methods. Both methods provide an appropriate solution for the analysis of correlated data . FAERI thus constitutes an improvement over all tested methods, and turns out to be the optimal method for testing question QA real-world example of analysis is reported for three datasets on cellular response to hypoxia. The results obtained from analysis of the E-MEXP-445 dataset illustrate that FAERI  is able to detect relevant results when a strict cut-off is used, compared to the other methodologies. The genesets detected by FAERI are related to three categories, respectively metabolic perturbations, hypoxia signaling/response, and hypoxia-related pathologies. The results of other global methods confirm these results, and post-hoc methods fail to detect any significant genesets. The results provided by FAERI from analysis of the two additional datasets reveal its ability to detect the same sets using several related datasets. Focusing on the top list obtained with several datasets, we showed that FAERI not only detects more sets, but also presents a larger intersection of the results.We proposed to score this last assessment on the whole list of sets by computing the Pearson correlation coefficient on the ranked list of genesets, for each method, using pairwise comparisons between the three datasets. Among all methods tested, FAERI provides the best correlated results between related datasets, regardless of the source of geneset definition used (each category of the MSIGDB databank was used for this purpose). FAERI thus outperforms all methods tested for its ability to attribute similar scores to each geneset from several datasets.Several recommendations emerge from this work. First, we suggest that differential expression analyses of microarray data be studied first by performing a geneset analysis with FAERI, and then that the results be further characterized using either an individual analysis or mathematical characterization of the sets detected. This approach eases interpretation of the results with regard to the correlation and direction of probeset-specific expression changes encountered for each geneset. Alternatively, genesets can be first classified with regard to the mathematical properties of the geneset expression values, and then the data can be analyzed using distinct methodologies .As a perspective for future work, we suggest to adapt FAERI so it would be able to test unequal sample datasets. Several adaptations to the ANOVA-2 procedure have been suggested to this end. To best adapt FAERI for this purpose, performance evaluations are required. Another interesting development would be to develop robust variants of FAERI based on the study of the impact of extreme values.An additional comment may be formulated about the empirical definition of the direction from the difference in means, used in the second step of the FAERI procedure. Indeed, genes slightly differing in one direction may empirically seem to be involved in the opposite direction, as sampling a small number of replicates can lead to a smaller or larger distribution than the population. Thus, the directional reduction step may lead to an incorrect definition of the sign of the expression change with regards to the biological structure of the data. An alternative would be to characterize first the direction associated with each gene based on current knowledge of the biological processes. Similarly, another alternative adaptation of the procedure would be to determine which genes are associated with an expression change, and to use the directional reduction step only on those genes. However, the use of prior knowledge on the expression and directional changes for each gene would be associated with important biases: incorrect prior information would reduce the significance of the geneset analysis, and this bias is more probable for less-studied genesets/biological mechanisms. Thus, the evaluation of the null distribution with realistic background noise would be geneset specific, depending on the number of errors in the annotation. In the future, we also plan to develop a non-parametric equivalent to FAERI, by applying a rank-based multivariate analysis procedure after the 2-step reduction of data. At least three options may be considered for further developments: ranking expression values at the level of the probesets, at the level of the genesets, or at the level of the whole dataset.The evaluation of significance may also lead to several adaptations. First, it would be interesting to add a third significance evaluation strategy based on simultaneous label sampling, where permutations would be performed without breaking the association with the cel file. Such a null distribution would provide an appropriate p-value for the evaluation of highly correlated genesets. However, the small number of available permutations would not allow for discrimination between genesets when the number of replicates is small. Both approaches introduce a bias during the p-value evaluation procedure: (i) the independent model assumes that the genes are not correlated, under-evaluating the p-value of co-occurring events; and (ii) the correlated model accordingly over-evaluates the p-value when the correlation between genes is not perfect. As the correlation pattern between genes is complex, the most appropriate p-value evaluation procedure should be somewhere in between the independent and correlated permutation models, with a geneset-specific balance between these two models.Another interesting perspective for evaluation of the p-value would be to improve the model used to generate the null distribution from random data. By studying several datasets first using individual and geneset analyses, it would be possible to characterize the correlation between genes in several situations. Then the network of associations could be used to count the average number of connections, to catch patterns of association, etc. Using such a strategy, the evaluation of the null hypothesis would consist in generating random networks with similar patterns of association and similar distributions of connectivity, and to use these patterns using multivariate random data built from the simulated correlation matrices. As an extension of this model, it would also be possible to include a supplemental parameter during simulation of the null distribution: the correlation pattern between genes may differ between the two groups.Performance evaluation of geneset analysis methods is not a trivial task, as each method is based on distinct null hypotheses. Previously published methods have been validated following several simulation schemes (using a limited number of scenarios) or running analyses on specific biological datasets (featuring more than 10 replicates for each tested condition). Nevertheless, those validations usually focused on the method developed by the authors, compared with only one method , if any, and a limited number of genesets. Conversely, publication of individual analysis methods requires extensive validations on real-world, simulated and spike-in datasets. The validations reported here reproduce the simulation scheme used by Ackerman et al. (2009) as it represents the most comprehensive scenario we could find in the scientific literature (in a paper that compares the parameterization of current methods). The example analysis reported here was performed on small datasets (3 replicates associated with each condition), as this kind of dataset is far more frequent and more difficult to analyze. We hope that the validation strategy reported here will be reproduced by authors of future methodologies to compare their findings with current methods and to provide a comprehensive evaluation of their methods with regards to several biological scenarios and the underlying mathematical properties of the expression values.In a previous work, we reported a new benchmarking strategy for individual expression analysis on real-world data involving several datasets . In futuhttp://urbm-cluster.urbm.fundp.ac.be/phoenix)FAERI and the ANOVA-2 procedure were implemented using R language, and the source code is included in PEGASE, an R package we designed for differential expression analysis . URL: (hFB scripted the FAERI methodology and performed the analysis presented here. BDM took part in the methodology performance assessment and creation of the methodology. AG took part in analysis and interpretation of the results. SD took part in analysis of the biological data. EB took part in scripting and automation. MP took part in interpretation and collection of the biological data. BDH took part in creation of the methodology and scripting. MD took part in the genesis of the methodology. ED coordinated the work and gave final approval for submission. All authors read and approved the final manuscript.Procedure section, describing the mathematical computing of FAERI. Pearson's correlation coefficients computed on ranks, for each method between 3 datasets, for each geneset definition. Figure Click here for file