DNA microarrays are a potentially powerful technology for improving diagnostic classification, treatment selection and therapeutics development. There are, however, many potential pitfalls in the use of microarrays that result in false leads and erroneous conclusions. This paper provides a review of the key features to be observed in developing diagnostic and prognostic classification systems based on gene expression profiling and some of the pitfalls to be aware of in reading reports of microarray-based studies. They are skeptical of analyses with multiple end points or multiple subsets, knowing that the chances of erroneous conclusions increase rapidly once one leaves the context of a focused single hypothesis clinical trial. Prognostic marker studies are generally performed with no written protocol, no eligibility criteria, no primary end point or hypotheses and no defined analysis plan. The analysis is often much less structured, the patient population more heterogeneous, and there are many subset analyses.The literature of prognostic markers for patients with cancer is vast, but most proposed markers are either not reproducibly established as prognostic in a medically relevant context or not widely used in clinical practice. Broadly accepted good practice standards for the development of prognostic markers and prognostic classification systems do not exist . Some ofMost of the problems that have hindered the development and acceptance of prognostic markers exist also for classification systems based on DNA microarray expression profiles. For example, there are multiple platforms and protocols for measuring expression profiles, and most studies do not evaluate either interlaboratory assay reproducibility or intralaboratory reproducibility on multiple samples of the same tissue specimen. Some of the problems that have hindered the development of reliable prognostic markers are exacerbated in DNA microarray studies. Owing to the number of genes available for analysis, microarray data can be a veritable fountain of false findings unless appropriate statistical methods are utilised.class comparison. A related objective that is often relevant for medical studies is the development of a mathematical function that can accurately predict the biologic group, diagnostic category or prognostic stage of a patient based on an expression profile of the diseased tissue from that patient. This will be referred to as class prediction. With these objectives, the sample classes are defined in advance independently of the gene expression data.DNA microarray experiments require planning . PlanninThe classes used for class comparison may represent different tumour types and the focus may be identifying the gene expression correlates of that tumour type classification. The classes in class prediction studies often represent prognostic or response groups for patients who have received a particular therapy. For example, In some problems of prognostic prediction, the outcome is a continuous measurement rather than a categorical class variable. For example, Class discovery is fundamentally different from class comparison or class prediction in that no classes are predefined. Usually, class discovery in cancer studies is for the purpose of determining whether discrete subsets of a disease entity can be defined based on the gene expression profiles. This is different from determining whether the expression profiles correlate in some way with some already known diagnostic classification. An example of class discovery was the study by unsupervised method because class membership indicators are not utilised. The pairwise distance measures between expression profiles are generally computed with regard to the complete set of genes represented on the array, or those that are well measured with sufficient intensity. The genes that distinguish particular classes may be few in number relative to the full set of genes. Consequently, the pairwise distances used in cluster analysis will often not reflect the influence of these relevant genes. This accounts for the poor results often obtained in attempting to use cluster analysis for class prediction. Even if the classes do group into clusters based on an unsupervised distance metric, the analysis does not provide a useful class predictor that can be used for new cases.Cluster analysis is useful for class discovery, but is generally not appropriate or effective for class comparison or class prediction. Cluster analysis refers to an extensive set of methods of partitioning samples into groups based on the pairwise distances of their expression profiles. Cluster analysis is considered an Our objective here is to highlight important aspects of the process of developing and evaluating gene-expression-based class predictors. Many of the recommended methods for developing and evaluating gene-expression-based classifiers are available in the BRB-ArrayTools software available without charge for noncommercial purposes from the National Cancer Institute .One component of developing a class predictor is determining which genes to include in the predictor. This is generally called ‘feature selection’ or, in the context of microarray prediction, gene selection. Usually, including too many ‘noise variables’ in the predictor reduces the accuracy of prediction. It also makes interpretation and future use of the predictor more difficult. A noise variable is a variable that is not related to the class being predicted. Feature selection is particularly important in microarray studies because the number of noise variables may be orders of magnitude greater than the number of relevant variables. The influence of the genes that actually distinguish the classes may be lost among the noise of the more numerous noise genes unless we select the informative genes to be utilised by the class predictor.The second main component of a class predictor is complete specification of the mathematical function that will provide a predicted class label for any given expression vector. There are many kinds of predictor functions as will be discussed in section ‘Class prediction algorithm’.The third component of developing a class predictor is parameter estimation. Most kinds of predictors have parameters that must be assigned values before the predictor is fully specified. These parameters are in many ways equivalent to the regression coefficients of ordinary linear regression. The machine learning literature calls the process of specifying the parameters ‘learning the data’ but it is equivalent to fitting the parameters of a nonlinear regression model. Even neural network models are really nonlinear regression models, although they are often represented as something more exotic .For many kinds of predictors, there is also a cut-point that must be specified for translating a quantitative predictive index into a predicted class label  for binary class prediction problems. Completely specifying the predictor means specifying all of these aspects of the predictor, the type of predictor, the genes included and the values of all parameters.The most important requirement for a class predictor is that it predict accurately. Neils Bohr supposedly said: ‘Prediction is difficult, particularly the future.’ But it is the future that we want to predict. We want to be able to predict class membership for future samples whose class membership we do not know. For example, How can we develop a proper estimate of the accuracy of class prediction for future samples? For a future sample, we will apply a fully specified predictor developed using the data available today. If we are to emulate the future predictive setting in developing our estimate of predictive accuracy, we must set aside some of our samples and make them completely inaccessible until we have a fully specified predictor that has been developed from scratch without utilising those set aside samples.To estimate properly the accuracy of a predictor for future samples, the current set of samples must be partitioned into a training set and a separate test set. The test set emulates the set of future samples for which class labels are to be predicted. Consequently, the test samples cannot be used in any way for the development of the prediction model. This means that the test samples cannot be used for estimating the parameters of the model and they cannot be used for selecting the gene set to be used in the model. It is this latter point that is often overlooked.split-sample method of partitioning the set of samples into a training set and a test set as described in the previous paragraph. The most straightforward method of estimating the accuracy of future prediction is the Cross-validation is an alternative to the split sample method of estimating prediction accuracy  as depicted in Figure 1M1 the model developed with sample 1 in the test set. When this model is fully developed, it is used to predict the class of sample 1. This prediction is made using the expression profile of sample 1, but obviously without using knowledge of the true class of sample 1. Symbolically, if x1 denotes the complete expression profile of sample 1, then we apply model M1 to x1 to obtain a predicted class ĉ1. This predicted class is compared to the true class label c1 of sample 1. If they disagree, then the prediction is in error. Then a new training set – test set partition is created. This time sample 2 is placed in the test set and all of the other samples, including sample 1, are placed in the training set. A new model is constructed from scratch using the samples in the new training set. Call this model M2. Model M2 will generally not contain the same genes as model M1. Although the same algorithm for gene selection and parameter estimation is used, since model M2 is constructed from scratch on the new training set, it will in general not contain exactly the same gene set as M1. After creating M2, it is applied to the expression profile x2 of the sample in the new test set to obtain a predicted class ĉ2. If this predicted class does not agree with the true class label c2 of the second sample, then the prediction is in error.n times, where n is the number of biologically independent samples. Each time it is applied, a different sample is used to form the single-sample test set. During the steps, n different models are created and each one is used to predict the class of the omitted sample. The number of prediction errors is totalled and reported as the leave-one-out cross-validated estimate of the prediction error.The process described in the previous paragraph is repeated n different models. They were only constructed in order to estimate the prediction error associated with the type of model constructed. The model that would be used for future predictions is the one constructed using all n samples. That is the best model for future prediction and the one that should be reported in the publication. The cross-validated error rate is an estimate of the error rate to be expected in use of this model for future samples assuming that the relationship between class and expression profile is the same for future samples as for the currently available samples. With two classes, one can use a similar approach to obtain cross-validated estimates of the sensitivity, specificity.At the end of the LOOCV procedure, you have constructed resubstitution estimate. You use all the samples to develop a model M. Then you predict the class of each sample i using its expression profile xi; ĉi=M(xi). The predicted class labels are compared to the true class labels and the number of errors are totaled.The cross-validated prediction error is an estimate of the prediction error associated with the algorithm for model building used. It is not an estimate constructed for a specific model. If we use all of the data to select genes and construct a specific model, there is no independent data left to estimate validly prediction error. A commonly used invalid estimate is called the resubstitution estimate of the error rate. In a simulated data set, 20 gene expression profiles of length 6000 were randomly generated from the same distribution. In all, 10 profiles were arbitrarily assigned to ‘Class 1’ and the other 10 to ‘Class 2’, creating an artificial separation of the profiles into two classes. Since no true underlying difference exists between the two classes, class prediction will perform no better than a random guess for future biologically independent samples. Hence, the estimated error rates for simulated data sets should be centred around 0.5 .Without using cross-validation, an astounding 98.2% of the simulated data sets resulted in an estimate of zero misclassifications even though no true underlying difference exists between the two groups. Moreover, the maximum number of misclassified profiles using the resubstitution method was only one. Cross-validating the prediction rule after selection of differentially expressed genes from the full data set does little to correct the bias of the resubstitution estimator: 90.2% of simulated data sets still resulted in zero misclassifications. It is not until gene selection is also subjected to cross-validation that the estimate of error rate was in line with our expectation: the median number of misclassified profiles jumps to 11, although the range is large (0–20).vs disease-free at 5 years) from gene expression profiles, first using the resubstitution method and then using a fully cross-validated approach. The investigators controlled the number of misclassified recurrent cases  in both situations, so here we focus attention on the difference in estimated error rates for the disease-free cases. The improperly cross-validated method and the properly cross-validation result in estimated error rates of 27% (12 out of 44) and 41% (18 out of 44), respectively. The improperly cross-validated method results in a seriously biased underestimate of the error rate, probably largely due to overfitting the predictor to the specific data set. While after selection of genes for inclusion in the predictor from the full data set. As our simulation shows, not subjecting gene selection to cross-validation can result in a large bias. Other examples of incorrect use of cross-validation are described by The simulation results underscore the importance of cross-validating all steps of predictor construction in estimating the error rate. A study of breast cancer also illustrates the point: t-test or a Mann–Whitney test for each gene. The log-ratios or log-signals are generally used as the basis of the statistical significance tests. The genes that are significantly differentially expressed at a specified significance level are selected for inclusion in the class predictor. The stringency of the significance level used controls the number of genes that are included in the model. If one wants a class predictor based on a small number of genes, the threshold significance level is made very small. Issues of multiple testing or false positives are not really relevant, however, because the objective is just to select features for inclusion in the model; no particular claim is made about the selected genes. Similarly, it does not really matter whether the assumptions of the t-test are strictly satisfied, because the P-values are merely used as a convenient index for selecting genes. Some methods do not use P-values at all but merely select the n most differentially expressed genes, and specify n arbitrarily.The most commonly used approach to feature selection is to identify the genes that are differentially expressed among the classes when considered individually. For example, if there are two classes, one can compute a Several authors have developed methods to identify optimal sets of genes that together provide good discrimination of the classes . These aPrincipal components are the orthogonal linear combinations of the genes showing the greatest variability among the cases. The principal components are sometimes referred to as singular values  selected for inclusion in the class predictor. For a two-class problem, there is a threshold value d, and a sample with expression profile defined by a vector x of values is predicted to be in class 1 or 2 depending on whether l(x) as computed from d or greater than d respectively.where l(x) in class 1 is maximally different from the mean value of l(x) in class 2. The squared difference in means divided by the pooled estimate of the within-class variance of l(x) was the specific measure used by Fisher. To compute these weights, one must estimate the correlation between all pairs of genes that were selected in the feature selection step. The study by Dudoit et al indicated that Fisher's linear discriminant analysis did not perform well unless the number of selected genes was small relative to the number of samples. The reason is that in other cases there are too many correlations to estimate and the method tends to be unstable and overfit the data.There are a large number of class predictors based on linear discriminants of the form shown in . They diDiagonal linear discriminant analysis is a special case of Fisher linear discriminant analysis in which the correlation among genes is ignored. By ignoring such correlations, one avoids having to estimate many parameters, and obtains a method that performs better when the number of samples is small. Golub's weighted voting method (Support vector machines (SVMs) are very popular in the machine learning literature. Although they sound very exotic, linear kernel support vector machines do class prediction using a predictor of the form of . The weights are determined by optimising a misclassification rate criterion, however, instead of a least-squares criterion as in linear discriminant analysis  which measures the distance between the expression profiles x and y of two samples. The distance function utilises only the genes in the selected set of features F. To classify a sample with expression profile y, compute d for each sample x in the training set. The predicted class of y is the class of the sample in the training set that is closest to y with regard to the distance function d. A variant of nearest neighbour classification is k-nearest neighbour classification. For example, with 3-nearest neighbour classification, you find the three samples in the training set that are closest to the sample y. The class that is most represented among these three samples is the predicted class for y. Nearest centroid, classification represents each class by a centroid, which is the average expression vector for all the arrays in that class. With nearest centroid classification, the test sample y is assigned to the class whose centroid it is closest to. The PAM method of In the study of et al also studied some more complex methods such as Classification Trees and aggregated classification trees. These methods did not appear to perform any better than diagonal linear discriminant analysis or nearest neighbour classification. Dudoit Complex methods with large numbers of parameters often fit the data used to develop the model well, but provide inaccurate predictions for independent data. This is often referred to as ‘overfitting’ the training data. Complex models have so many parameters that they can fit all of the random variations in the training data well. They find predictors and nonlinear functions that account for the random variations in the training data, but these discovered relationships do not represent real effects that exist in independent data; consequently, predictive ability is poor.Many of the methods described above are for prediction when there are two classes. One strategy for multiple-class prediction is to perform a series of two-class predictions. The multiple-class prediction problem may be represented as a directed binary graph such as shown in Figure 2k-nearest neighbour methods described above are not restricted to two classes. Since this method generally performs well and is one of the simplest, it is probably the most favoured of the non-tree-structured methods.There are many alternative approaches to multi-class prediction. The DNA microarrays are a technology that offers great promise for refining diagnostic classification in ways that will enhance the efficiency of clinical trials and will enable the appropriate treatment to be matched to the appropriate patient. There are, however, major obstacles to the effective use of this technology. Access to tissue with viable RNA complicates research studies because the assay cannot be performed with conventional archived specimens. It also makes tissue acquisition and handling more difficult in the clinic. Another limitation is that most laboratories are not equipped to effectively design studies or analyse the immense amount of noisy data generated by DNA microarrays. The problems go far beyond data management, and include the difficulty organisations have in supporting interdisciplinary collaboration in a manner that brings the best minds together to educate each other and work creatively to make biological discoveries and solve important biomedical problems. The advent of DNA microarray technology has highlighted some of the problems in the area of data analysis. Potentially misleading analyses are published in the best biomedical journals and there is a plethora of algorithms and software of limited utility, sometimes questionable validity, and often based on inadequate understanding of the biological problems being addressed. These are growing pains, however, in a field that is dynamic, destined to make important biological discoveries and potentially revolutionise the process of developing cancer therapeutics.


Combining gene expression microarrays and high resolution magic angle spinning magnetic resonance spectroscopy (HR MAS MRS) of the same tissue samples enables comparison of the transcriptional and metabolic profiles of breast cancer. The aim of this study was to explore the potential of combining these two different types of information.Breast cancer tissue from 46 patients was analyzed by HR MAS MRS followed by gene expression microarrays. Two strategies were used to combine the gene expression and metabolic data; first using multivariate analyses to identify different groups based on gene expression and metabolic data; second correlating levels of specific metabolites to transcripts to suggest new hypotheses of connections between metabolite levels and the underlying biological processes. A parallel study was designed to address experimental issues of combining microarrays and HR MAS MRS.myo-inositol, glycine, taurine, glycerophosphocholine, phosphocholine, choline and creatine and all transcripts in the filtered microarray data were investigated. GO-terms related to the extracellular matrix were enriched among the genes that correlated the most to myo-inositol and taurine, while cell cycle related GO-terms were enriched for the genes that correlated the most to choline. Additionally, a subset of transcripts was identified to have slightly altered expression after HR MAS MRS and was therefore removed from all other analyses.In the first strategy, using the microarray data and previously reported molecular classification methods, the majority of samples were classified as luminal A. Three subgroups of luminal A tumors were identified based on hierarchical clustering of the HR MAS MR spectra. The samples in one of the subgroups, designated A2, showed significantly lower glucose and higher alanine levels than the other luminal A samples, suggesting a higher glycolytic activity in these tumors. This group was also enriched for genes annotated with Gene Ontology (GO) terms related to cell cycle and DNA repair. In the second strategy, the correlations between concentrations of Combining transcriptional and metabolic data from the same breast carcinoma sample is feasible and may contribute to a more refined subclassification of breast cancers as well as reveal relations between metabolic and transcriptional levels.http://www.biomedcentral.com/1741-7015/8/73See Commentary:  Transcriptomics and metabolomics in cancer research have traditionally been considered as two separate fields. Different levels of the molecular processes are studied, aiming at improving cancer treatment by understanding the underlying mechanisms of the disease. Breast cancer treatment decisions today are mainly based on tumor size, histological characterization, grading and receptor status, as well as axillary lymph node status, and age of the patients . HoweverThe field of transcriptomics, using DNA microarrays that enable measurements of thousands of RNA transcripts in a single experiment, has had a huge impact on breast cancer research over the last decade . One of in vivo magnetic resonance spectroscopy (MRS) as a diagnostic, prognostic or predictive tool in the clinic simultaneously with an MR imaging (MRI) examination, makes MRS techniques attractive methods for molecular classification of disease. Metabolic profiling of intact biological samples using high resolution magic angle spinning magnetic resonance spectroscopy (HR MAS MRS) enables measurement of multiple cellular metabolites simultaneously. The method has been utilized in a wide range of biological applications [Metabolomics studies the metabolites and how they are affected by specific cellular processes. The possibility of using ications , and stuications . ImportaProfiling gene expression and metabolite content in the same breast carcinoma samples enables comparisons of molecular findings at different levels. Gene expression data and metabolite data from MRS techniques of different samples from the same breast cancer cell line or xenograft model have been combined previously -12, but The aim of this study was to combine gene expression microarrays and HR MAS MRS for more refined profiling of breast tumors, and to explore some of the potentials and limitations of the experimental procedures. This study focuses on the most prevalent type of breast cancer, invasive ductal carcinomas (IDC) with oestrogen (ER) receptor positive disease, and the largest molecular subgroup within these tumors, luminal A. ER positive breast cancer accounts for approximately 2/3 of the cases, and although they overall have a relatively good prognosis, some patients experience relapse and do not respond to antieostrogen treatment. So far there are no biomarkers available to identify those patients and no targets for therapy, and identification of molecular markers of such tumors, possibly by metabolic profiling, are therefore of high relevance. Identifying subgroups of patients within this group is an important goal to make it possible to further individualize cancer diagnosis and treatment.Tissue samples were selected from a local breast cancer tissue bank, obtained from patients with palpable breast lesions who underwent scheduled surgery for breast cancer at St. Olavs University Hospital in Trondheim, Norway (March 2000 - March 2004). Samples were placed in cryogenic vials and immersed in liquid nitrogen immediately after dissection. In this study we selected samples from patients diagnosed with IDC, who did not receive any treatment prior to surgery. Clinical data of the patients, such as diagnosis, tumor grade, tumor size, hormone receptor status and lymph node involvement were obtained from patient records, including pathology reports, and are summarized in Table An illustration of the experimental workflow as well as the data analysis for this study is shown in Figure 2O containing trimethylsilyl tetradeuteropropionic acid (TSP)). HR MAS MRS experiments were performed on a Bruker Avance DRX600 spectrometer equipped with a 1H/13C MAS probe . Samples were spun at 5 kHz while keeping the temperature at 4°C. Two types of one-dimensional 1H spectra were recorded, a single pulse experiment with water suppression and a spin-echo experiment (CPMG) for suppression of broad signals from macromolecules. Spectral assignments were performed based on previous HR MAS MRS studies of breast cancer lesions [Sample preparation and HR MAS MRS experiments were performed as previously described . In brie lesions .Total RNA was extracted from the biopsies that were used in the HR MAS MRS procedure from each patient, following the protocol for the RNeasy Mini Kit . High quality RNA, measured by BioAnalyzer 2100  was used in microarray experiments, using 44k two-color Agilent Human Whole Genome Oligo Microarrays. The microarray analysis was performed according to the manufacturer's protocol, using Cy5 labeling for the tumor RNA and Cy3 for the reference RNA, Universal Human Reference . Data were extracted from the microarray images using Feature Extraction Software v8.5 . The final dataset included gene expression microarray data from 46 patients, 35 obtained from RNA extracted from the HR MAS MRS analyzed tissue and 11 from adjacent tumor tissue, as illustrated in Figure The two-color microarray data were preprocessed and analyzed in R (v2.8) and Bioconductor, using the Feature Extraction text files. The red (sample) and green (reference) signals were set to be g(r)MeanSignal, and the data were normalized using the Limma package . Each miUsing gene median centered microarray data, the expression of "intrinsic" genes of the tumors from all 46 patients were analyzed for Spearman's correlation to published centroids of the five intrinsic subtypes . The sammyo-inositol, taurine, choline, GPC, PCho, glucose, creatine and glycine using the Voigt (combined Lorentzian and Gaussian) area function in PeakFit . Quantification of lactate was not performed since lipid signals concealed the lactate signals  in two thirds of the spectra. Probes on the microarray with IQR < 0.8 were removed, and Spearman correlation tests between the 8 metabolites and all of the 13875 unique probes were performed. For each metabolite, all transcripts were ranked according to Spearman's rank correlation coefficient. The resulting genelists were analyzed for enriched GO-terms using GOrilla, which searches for GO-terms that are enriched in the top of the list compared to the rest of the list using the mHG statistics [Microarray and HR MAS MRS data from the same samples of the ER+ tumors with tumor cell percentage > 5% (N = 34) were used to test for nonzero correlations between metabolite and gene expression levels . Quantification of eight tissue metabolites was performed based on peak areas in the single-pulse spectra and TSP concentrations as in . After patistics ,21.No apparent differences were observed between the microarray data from the 35 samples that did undergo HR MAS MRS compared to the 11 samples that did not. However, to explore this issue more thoroughly, an additional study was designed to assess the feasibility of performing gene expression microarrays and HR MAS MRS on the same sample as well as identify transcripts that should be filtered out from microarray analyses if they have been systematically affected by the HR MAS MRS procedure. This additional study included 18 pairs of tumor samples, obtained from the local breast cancer tissue bank described above. One sample from each pair was analyzed by HR MAS MRS, RNA was isolated from both samples from each of the 18 pairs, and gene expression microarray analysis was performed. The HR MAS MRS experiments were performed as described above, while the RNA extraction, microarray experiments and preprocessing of the microarray data were performed using slightly different protocols, as described in Additional file Two main strategies of combining information from the microarray data with the HR MAS MR spectra are presented, as shown in the workflow illustrated in Figure myo-inositol, alanine and lipid residues [myo-inositol were significantly lower in A1 than in A2 and A3. Differential gene expression analysis between the three luminal A groups, using Limma, resulted in no significant differences in expression of single genes with fdr < 0.01. However, when analyzing differences in gene expression between sets of genes annotated with the same GO-terms, using GSA, different significantly enriched GO-terms (fdr < 0.01) were revealed by comparing the three luminal A groups  were ranked according to correlation, from high to low, to each of the 8 metabolites. Plots showing the co-variation of myo-inositol and choline concentration with the expression of the transcripts that correlated the most to these metabolites are shown in Figure myo-inositol and taurine. GO-terms related to the cell cycle, such as "cell cycle process" and "chromosome segregation" are enriched towards the top of the list of transcripts ranked according to correlation to choline. The lists of transcripts ranked according to correlation to glucose, creatine and glycine only gave one enriched GO-term each, "immune system process", "mannosyltransferase activity" and "respiratory chain", respectively. There were no significantly enriched GO-terms for the lists of transcripts ranked according to correlation to GPC and PCho.The levels of 8 specific metabolites were compared with the transcriptional activity in each tumor in the ER+ samples (n = 34 with tumor cell percentage >5%). These samples include the 31 luminal A samples mentioned above plus three samples that were classified as normal-like, but also correlated to the luminal A centroid. Tissue concentrations of some of the 8 quantified metabolites co-varied, see Additional file The effect of the HR MAS MRS procedure on RNA integrity and transcription in the breast cancer tissue was assessed separately on 18 pairs of samples. There was not significant evidence that total RNA integrity, quantified by the RIN-value , was affected by HR MAS MRS . The results from the microarray analysis are illustrated in Additional file In this study, we have shown the feasibility of merging transcriptomics and metabolomics data from the very same tumor tissue sample. Two strategies of combining microarray data and HR MAS MR spectra are presented, providing a framework for how information from these different molecular levels can be combined and analyzed. We also identified a set of transcripts which showed slightly altered expression after the HR MAS MRS procedure, but overall this variation was smaller than the biological variation in tumors from one patient to another.In the first strategy to combine gene expression microarray data and HR MAS MR spectra, the expression of "intrinsic" genes was used to classify the samples into established molecular subtypes. The HR MAS MR spectra of the majority of tumor samples, which were classified as luminal A, were further explored to investigate whether metabolic characteristics could define subgroups within a transcriptionally homogenous set of samples. The use of spin echo acquired spectral profiles ensured a more extensive use of metabolic information than using calculated tissue metabolite concentrations, which is limited by several peak areas being non-quantifiable. Three subgroups of luminal A tumors were identified Figure  and 3. Tmyo-inositol were enriched for GO-terms associated to extracellular processes, which could reflect a tumor-stroma interaction. "Cell adhesion" was also one of the enriched GO-terms for the gene transcripts that correlated to myo-inositol, which supports this hypothesis. Taurine and myo-inositol are known to be involved in osmoregulation and volume regulation [myo-inositol correlated negatively to tumor percentage , and findings in a similar evaluation in prostate tissue support this result . The pai MAS MRS . Howevermyo-inositol and taurine, while cell cycle related GO-terms were enriched among the genes that correlated the most to choline. We have shown that combining transcriptional and metabolic data from the same breast carcinoma sample can contribute to a more refined subclassification of breast cancers as well as reveal relationships between these molecular levels. This study has paved the way for further studies in larger patient cohorts of all subtypes, correlating metabolic subgroups to histopathological characteristics, treatment response and clinical outcome.Performing HR MAS MRS and microarray analysis on the same sample is feasible, and the effect of HR MAS MRS on the transcriptome was shown to be subtle. Three subgroups of samples within the most prevalent intrinsic subgroup of breast cancer, luminal A, were found using multivariate analyses of HR MAS MRS spectra. One of the subgroups of luminal A samples, designated A2, had metabolic and transcriptional features indicating a higher Warburg effect and more proliferation than the other luminal A groups. Using a different strategy, enrichment analysis of genes with expression levels that correlated to metabolite concentrations revealed different enriched GO-terms associated with specific metabolites. GO-terms related to the extracellular matrix were enriched among the genes that correlated the most to Abbreviations used: CHO: choline; GPC: glycerophosphocholine; PCHO: phosphocholine; IDC: invasive ductal carcinoma; HR MAS MRS: high resolution magic angle spinning magnetic resonance spectroscopy; PBS: phosphate buffered saline; IQR: inter quartile range; PCA: principal component analysis; TSP: trimethylsilyl tetradeuteropropionic acid; FDR: false discovery rate; GO: Gene Ontolgy; MDS: multidimensional scaling; GSA: Gene Set AnalysisThe authors declare that they have no competing interests.EB carried out HR MAS MRS and microarray experiments and performed microarray preprocessing, data analysis and interpretation of HR MAS MRS and microarray data, and drafted the manuscript. BS performed HR MAS MRS and microarray experiments, did the preprocessing of the HR MAS MRS data, contributed to analysis and interpretation of HR MAS MRS and microarray data and drafting of the manuscript. OCL contributed to analysis and interpretation of HR MAS MRS and microarray data. HJ carried out microarray experiments. TB contributed to interpretation of HR MAS MRS and microarray data. TS contributed to analysis and interpretation of microarray data. SL provided tumor material and clinical data. ALBD conceived of the study, and participated in its design and helped drafting the manuscript. ISG conceived of the study and participated in its design and coordination. All authors revised the manuscript and approved the final version.The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2407/10/628/prepubSupplementary documentation of exploring the effect of HR MAS MRS on RNA integrity and transcription. Additional documentation of the experimental procedure and data analysis involving the 18 pairs of samples used in the parallel study to assess the feasibility of performing HR MAS MRS and gene expression microarrays on the same samples.Click here for fileScatterplot of metabolite concentrations and tumor percentage. The log2 of the tissue concentrations (μmol/gram) of glycerophosphocholine (GPC), glycine, phosphocholine (PCho), creatine, choline, taurine, myo-inositol and glucose as well as the tumor percentage (Tumor) are plotted against each other in the upper diagonal panel. Spearman's rank correlation coefficients are given in the lower diagonal panel. Significant (p < 0.05) correlations are indicated by bold font. The values on the axes represent log2 tissue concentrations (μmol/gram), except for the Tumor axis which represents percentage.Click here for filePlots illustrating the effect of HR MAS MRS on the transcriptome. (A) Hierarchical clustering of the 18 pairs of tumors before (control) and after HR MAS (HRMAS). (B) A plot of the number of significantly differentially expressed (DE) genes caused by HR MAS MRS as a function of false discovery rate. Bonferroni corrected p-value = 0.05 is indicated in the plot. (C) A volcanoplot of significance versus the estimated fold change caused by HR MAS MRS. Transcripts with fdr < 0.01 are colored red or blue to indicate higher or lower expression after HR MAS MRS, respectively. The top Biological Process GO-terms of the differentially expressed genes are listed on each side with the same color-code.Click here for fileSignificantly lower expressed transcripts after HR MAS MRS. A summary of the 865 transcripts with significantly lower expression after HR MAS MRS, identified using Limma (R/Bioconductor). The columns ProbeUID, AgilentProbeID, GeneSymbol, SystematicName are annotations as given by Agilent. The columns logFoldChange, AverageExpression, t.statistic, p.value and adjusted.p.value are the results from the modified paired t-tests performed by Limma.Click here for fileSignificantly higher expressed transcripts after HR MAS MRS. A summary of the 334 transcripts with significantly higher expression after HR MAS MRS, identified using Limma (R/Bioconductor). The columns ProbeUID, AgilentProbeID, GeneSymbol, SystematicName are annotations as given by Agilent. The columns logFoldChange, AverageExpression, t.statistic, p.value and adjusted.p.value are the results from the modified paired t-tests performed by Limma.Click here for file


Normalization is essential in dual-labelled microarray data analysis to remove non-biological variations and systematic biases. Many normalization methods have been used to remove such biases within slides  and across slides . However, all these popular approaches have critical assumptions about data distribution, which is often not valid in practice.In this study, we propose a novel assumption-free normalization method based on the Generalized Procrustes Analysis (GPA) algorithm. Using experimental and simulated normal microarray data and boutique array data, we systemically evaluate the ability of the GPA method in normalization compared with six other popular normalization methods including Global, Lowess, Scale, Quantile, VSN, and one boutique array-specific housekeeping gene method. The assessment of these methods is based on three different empirical criteria: across-slide variability, the Kolmogorov-Smirnov (K-S) statistic and the mean square error (MSE). Compared with other methods, the GPA method performs effectively and consistently better in reducing across-slide variability and removing systematic bias.The GPA method is an effective normalization approach for microarray data analysis. In particular, it is free from the statistical and biological assumptions inherent in other normalization methods that are often difficult to validate. Therefore, the GPA method has a major advantage in that it can be applied to diverse types of array sets, especially to the boutique array where the majority of genes may be differentially expressed. The cDNA microarray is a widely used high-throughput technique for gene expression profiling, especially for organisms whose genome sequences are unavailable. However, in microarray experiments, there exist many non-random variations and systematic biases, which can confound the extraction of the true fluorescence intensity signals, and thus compromise downstream data analysis and interpretation of the experimental data. Therefore, proper data normalization is required to remove these biases before accurate identification of differential gene expression -4.The main objective of normalization is to ensure that measured intensities within and across slides are comparable. Based on different biological or statistical assumptions about data distribution or experimental design, various normalization methods have been proposed. The housekeeping gene method  is an eaThese various methods can effectively remove the intensity-dependent or spatially-dependent biases within each slide. However, they do not account for the intensity-dependent differences across multiple slides, which can introduce undue weighting of some slides to an average of log-ratios across slides in the subsequent data analysis . Scale nWhile many different normalization methods are available and diverse strategies are implicated, most of them require certain critical biological or statistical assumptions about data distribution. For example, one usual assumption underlying the Global, Scale, Lowess and VSN methods is that the array contains many non-differentially expressed genes. The assumption on data distribution inherent in these methods may not be valid in practice. For example, in custom-made boutique arrays most of genes are expected to be differentially expressed ,16. MostHere we present evidence to show that the Generalized Procrustes Analysis -21, or GGPA is a standard multivariate statistical method widely applied in shape analysis to find the optimal superimposition of two or multiple configurations -21. The Based on the different data sets, we compared our GPA-based method with six other normalization methods, including Global , Lowess We first evaluated the performance of these normalization methods on two real microarray data sets through comparing data variability and similarity of data distribution among replicated slides. The rationale behind these two criteria is that an effective normalization method should result in lower replicate variability, and more similar  data distributions for replicated slides. Then we applied two simulation methods to simulate several types of microarray data. Mean square error (MSE) was used to evaluate different normalization methods through calculating the true difference between simulated data and normalized data. For the boutique array type, both real data and simulated data were utilized and subjected to these three empirical criteria. In addition, the differential effects of these normalization methods on M-A plots for both real data and simulated data are showed in Additional files N replicated slides, the variability of N values for each gene can therefore be used to compare normalization methods [g (σg) can be estimated asThe criterion of replicate variability is based on the rationale that the expression level of a gene should ideally remain the same across multiple replicated slides. For  methods ,22. The N is the number of replicates in the data set, log2(R/G) value for gene g in slide i, while g. A smaller where Figure When we compare the across-slide normalization methods based on the same within-slide normalization background , we can see (1) different combinations of within- and across-slide normalization methods can further reduce the variability values Figure , and (2)The Kolmogorov-Smirnov (K-S) test is a goodness-of-fit test of two continuous distributions. It is used as a criterion for assessing normalization methods and is based on the rationale that an effective normalization procedure should result in two similar  distributions with a small, ideally zero-valued, K-S statistic ,23. In cFigure Mji as the true expression log-ratio for the j-th gene of i-th replicate, Mji, and i = 1,2, ...N, j = 1, 2, ..., the number of genes) in N replicates. MSE for the j-th gene is defined asWe used a data simulation study to further test the GPA method. The advantage of using a simulated data set is that the true intensities are known, so we can assess the accuracy and precision of normalized data more systematically. Mean square error (MSE) is a widely used comparison criterion, which can calculate the true difference between simulated and normalized data ,24. Denoν) is an index of precision and the bias component (β) is an index of accuracy. Obtaining data with satisfactory precision and accuracy has been one of the biggest challenges in the application of microarrays. For this reason, MSE is an excellent criterion for evaluating alternative normalization methods [ν and β values indicating better normalization.Note that MSE is decomposed into variance and squared bias. The variance component  is several times lower with the GPA normalization individually than with the other normalization methods. The median of bias (β) with GPA normalization is also significantly lower than with the other methods (p < 0.01 by Wilcoxon test). This trend is observed for other data sets with 3, 10, 30% differentially expressed genes . These results indicate that the GPA method performs effectively and consistently better in reducing across-slide variability and removing systematic bias.Two different methods to simulate microarray data were used: one is a parameterized random signal model from the study of , which iSince the accuracy and efficiency of across-normalization method may depend on the number of replicate slides, we also conducted a series of simulation studies to verify the effect of number of replicate slides on the performance of the GPA method compared with other methods. In practice, four slides are commonly considered as the minimum number for one reliable cDNA array experiment. For this reason we have simulated microarray data from four to eight replicates. We found that, with replicate slides from 4 to 8, the GPA method yields lower MSE (sum of variance and squared bias) than other methods. In general, GPA has the best and most consistent performance in decreasing variance and bias across slides . Given these results, we recommend using the GPA method with within-slide normalization methods such as Lowess whenever feasible, especially when the number of replicate slides is smaller than four.ν and β are smaller with GPA normalization than with the housekeeping gene normalization for diverse cases. These results demonstrate that the GPA method performs better, with consistently smaller ν and β values, than the housekeeping gene method.Being relatively assumption-free, the GPA method may be particularly useful in analyzing boutique arrays where the majority of genes may be differentially expressed. Here we show that the GPA method is superior to the popular housekeeping gene normalization approach. We used one real apoptosis microarray data set to evaluate these two methods. The hypothesis behind this type of microarray data is that the expression of most genes changed significantly, so most normalization methods are not appropriate. Table In order to further test the ability of GPA on boutique arrays and illustrate its advantage of being assumption-free, we simulate another extreme example of boutique arrays with 90% up-regulated genes at 10 fold and 10% down-regulated genes at 2 fold. In this case, the housekeeping gene normalization method cannot work since there are no assumed prior housekeeping genes in the experiment, whereas the GPA method can solve this problem. Figure We demonstrate the potential of a GPA-based method for normalizing microarray data. Data normalization is an essential step for the spotted cDNA array and exerts important effects on the subsequent data analysis leading to identification of differentially expressed genes, and clustering and pathway analyses. Many normalization methods have been proposed, however, almost all the available normalization methods are based on biological or statistical assumptions about data distribution which are often not valid in practice. For example, the usual assumption that the array contains a large enough assortment of random genes, most of which are not differentially expressed across the experiments, is inherent in many existing methods including the Global, Scale, Lowess and VSN methods considered here. The assumption is particularly problematic for custom-made boutique arrays where most genes are expected to be differentially expressed, or are directly implicated in the biological process being studied. Although some novel methods have been proposed for such boutique arrays, including the housekeeping gene method , Zipf's Procrustes analysis is a powerful least-squares approach by translating, rotating and isotropic scaling to achieve a better fit among matrices of similar configurations. It has been widely applied in many fields such as statistical analysis of shapes , analytiWe utilized both the real and simulated data to compare the GPA-based approach with six other normalization methods: Global, Lowess, Scale, Quantile, VSN, and one boutique array specific housekeeping gene method. For both real data sets, the GPA method showed a relatively better performance in decreasing replicate variability and increasing similarity of data distribution than other methods. Global and Scale methods performed worse in both criteria. Both VSN and Quantile performed best only in one case of variance or K-S statistic because their assumptions specifically favour one of them. Although the Lowess method can decrease variability, it showed no advantage in decreasing discrepancy of data distribution. A better performance of GPA than other across-slide based methods can still be observed on the same within-slide normalization background. The Lowess-GPA pair performed better than other dual normalizations, implying a better combination potential. Furthermore, we utilized two different parameter models to simulate several types of microarray data, which were used to accurately evaluate normalization effects through assessing true difference between known true data and normalized data. The results indicate that our GPA method outperformed other methods in reducing across-slide variability and removing systematic bias in microarray data. Overall the GPA normalization can effectively decrease the replicate variability, discrepancy of data distribution among slides and retrieve underlying biological information. It not only can be used individually to balance the bias across slides, but also in practice can be combined with other methods such as Lowess to produce better results. Moreover, the combination of GPA with Lowess can reduce discrepancies in data distribution as a result of Lowess. This is logical because Lowess and GPA have different within-slide based and across-slide based strategies. Furthermore, the application of GPA normalization in analyzing the boutique array was demonstrated in both the real and simulated boutique arrays. The GPA method not only performs consistently better than the popular boutique array-specific housekeeping gene normalization method, but can also be used where other methods are inapplicable.We have shown that GPA is a promising normalization method for microarray data analysis. In particular, GPA method is free of assumptions about data distribution inherent in other existing approaches. This makes GPA versatile and robust for diverse types of array sets, especially for custom-made boutique arrays where the majority of genes may be differentially expressed.The swirl zebrafish data set is available at Bioconductor . The datThe microaray data set of HCT116 cancer cell line was retrieved from  which idThe custom boutique array data set was retrieved from . The micThe generation of our first simulation is based on a parameterized random signal model , which h1. Simulation data 1 is a 1000 × 10 log ratio intensity of expression matrix. Based on the general assumption that most genes are not differentially expressed , our data was simulated to hold 3, 5, 10, and 30 percent levels of differentially expressed genes, at the same time the proportion of up-regulated genes equals that of down-regulated genes without dye bias.2. Simulation data 2 is the same as Simulation data 1, but with dye bias which generates the banana shaped MA plot which is commonly observed in real cDNA microarray data sets.3. Simulation data 3 is a 1000 × 10 log ratio intensity of expression matrix which is simulated as the boutique arrays in which 60 percent of genes are differentially expressed. The ratios of the up-regulated genes to the down-regulated genes are 5:5, 7:3, and 9:1.With the SIMAGE method , simulatThe Procrustes analysis method described in this paper is available as the package vegan in the R project . ScriptsM = log2(R/G) and the mean log intensity N replicated arrays as N series of matrices with g rows and two columns where g is the number of gene probes on the slide and the two columns are the log ratio intensity and mean log intensity on a scale . For each cDNA spot j in the i-th slide, it corresponds to a vector j = 1,2,...,g; i = 1,2,...N). Mji and Aji are the measurement of M and A for gene j in replication i. Our expression level matrix of the i-th slide becomes the matrix i = 1, 2, ..., N. Firstly, a reference array S0 is generated through computing the median intensity of each gene over all slides. Then each slide Si  is translated so that its centroid is at the centroid point of the reference array S0. Let Si be rotated and scaled such that the residual discrepancy between Si and S0 is minimized. We wish to find the orthogonal matrix Hi with (HiHiT = I) and scale factor ci so as to minimize the sum of squared distances between the corresponding points in Si and S0, i.e.We consider the log-transformed data, log ratio intensity where M2 = 0. To minimize M2, we needA perfect match gives Hi = VUT, where V and U are products of the singular value decomposition of SiULVT [T = ULVT .Si is Si* = ciSiHi.Therefore, the output data after our Procrustes normalization method on In this study GPA normalization employs a reference array which is established from median values across all slides. This idea of a reference array is popularly utilized in several other normalization methods including Qspline ,22, IsetThe GPA method needs approximately a few seconds on a regular PC platform to normalize the data. Since the GPA algorithm computes the matrix from all the spots, it requires that all the spots have values. Here we used K-nearest neighbor averaging scheme  to imputIn our study six other normalization methods were compared against our GPA-based method. These include two within-slide methods . Combinations of different methods produce combinatorial effect for plots.Click here for fileThe M-A plots for SIMAGE simulated microarray data after different normalization methods. This data set includes 1000 genes in 50 slides. 5 percent genes are differentially expressed. The ratio of up-regulated to down-regulated genes is 1:1. Here we showed M-A plots for first 10 slides in this data set.Click here for fileβ) in the simulated data without and with dye bias based on [The median of the two components of MSE: variance (v) and bias .Click here for fileInput parameters used in SIMAGE simulation.Click here for fileThe effect of choice of a reference array on GPA normalizations.Click here for fileReplicate variability and K-S statistic for the swirl zebrafish data set after GPA normalizations with different reference arrays. The upper dashed and lower straight lines indicate variability and K-S value for Lowess method, respecitively.Click here for fileReplicate variability and K-S value for swirl zebrafish data after GPA normalizations based on different reference slides, and other normalization methods. Although the GPA normalizations differ based on different reference slides, the overall better performance compared to other methods can be still observed here.Click here for fileThe MSE results for simulated data after GPA normalizations with different reference arrays. The data is simulated by SIMAGE method, include 1000 genes in 50 slides with 5% differentially expressed genes and ratio of up-regulated to down-regulated genes is 1:1. Med represents median reference slide and Mea represents mean reference slide.Click here for file


Molecular signatures are computational or mathematical models created to diagnose disease and other phenotypes and to predict clinical outcomes and response to treatment. It is widely recognized that molecular signatures constitute one of the most important translational and basic science developments enabled by recent high-throughput molecular assays. A perplexing phenomenon that characterizes high-throughput data analysis is the ubiquitous multiplicity of molecular signatures. Multiplicity is a special form of data analysis instability in which different analysis methods used on the same data, or different samples from the same population lead to different but apparently maximally predictive signatures. This phenomenon has far-reaching implications for biological discovery and development of next generation patient diagnostics and personalized treatments. Currently the causes and interpretation of signature multiplicity are unknown, and several, often contradictory, conjectures have been made to explain it. We present a formal characterization of signature multiplicity and a new efficient algorithm that offers theoretical guarantees for extracting the set of maximally predictive and non-redundant signatures independent of distribution. The new algorithm identifies exactly the set of optimal signatures in controlled experiments and yields signatures with significantly better predictivity and reproducibility than previous algorithms in human microarray gene expression datasets. Our results shed light on the causes of signature multiplicity, provide computational tools for studying it empirically and introduce a framework for in silico bioequivalence of this important new class of diagnostic and personalized medicine modalities. One of the promises of personalized medicine is to use molecular information to better diagnose, manage, and treat disease. This promise is enabled through the use of molecular signatures that are computational models to predict a phenotype of interest from high-throughput assay data. Many molecular signatures have been developed to date, and some passed regulatory approval and are currently used in clinical practice. However, researchers have noted that it is possible to develop many different and equivalently accurate molecular signatures for the same phenotype and population. This phenomenon of signature multiplicity has far-reaching implications for biological discovery and development of next generation patient diagnostics and personalized treatments. Currently the causes and interpretation of signature multiplicity are unknown, and several, often contradictory, conjectures have been made to explain it. Our results shed light on the causes of signature multiplicity and provide a method for extracting all equivalently accurate signatures from high-throughput data. In the worst case this phenomenon entails that the produced signatures are not statistically generalizable to new cases, and thus not reliable enough for translation to clinical practice.A Some authors motivated by classical statistical considerations, attribute signature multiplicity solely to the small sample size of typical microarray gene expression studies resampling-based signature extraction. It operates by repeated application of a signature extraction algorithm to resampled data  iterative removal, that is repeating signature extraction after removing from the data all genes that have been found in the previously discovered molecular signatures stochastic gene selection techniques brute force exhaustive searchCritical to the ability to study the phenomenon empirically is the availability of computational methods capable of extracting multiple signatures from the data. Several methods have been introduced with this intent. The available methods encompass four algorithm families. The first family is in-silico equivalence to previously validated signatures in a manner similar to bioequivalence of drugs.The above methods, useful first attempts as they may be, are either heuristic or computationally intractable, are based on currently unvalidated conjectures about what causes multiplicity, and output incomplete sets of signatures with currently unknown generalizability. The practical benefits of an algorithm that could systematically extract the set of truly maximally predictive and non-redundant signatures include: (i) a deeper understanding of the signature multiplicity phenomenon and how it affects reproducibility of signatures; (ii) improving discovery of the underlying biological mechanisms by not missing genes that are implicated mechanistically in the disease processes; and (iii) catalyzing regulatory approval by establishing To achieve these goals we provide a theoretical framework based on Markov boundary induction that enables probabilistic modeling of multiple signatures and formally connects it with the causal graph  of the data generating process relative to the genes (variables) contained in each signature. In other words, we exclude from consideration signatures that do not utilize all predictive information about the phenotypic response variable contained in their genes. This allows us to study signature classes by reference only to the genes contained in each class. Specifically, for a gene set X there can be an infinite number of classifiers that achieve maximal predictivity for the phenotype relative to the information contained in X. Thus, when we say “signature X” we refer to one of these predictively equivalent classifiers. This reduction is justified, for example, whenever the classifiers used can learn the minimum error decision function given sufficient sample . Most practical classifiers employed in this domain as well as classifiers used in our experiments (SVMs) satisfy the above requirement either on theoretical To simplify analysis, and without loss of generality, instead of considering all possible signatures derivable from a given dataset  we only consider the signatures that have maximal predictivity for the phenotypic response variable maximally predictive of the phenotype , and at the same time (b) do not contain predictively redundant genes . Every suboptimal signature  can be discarded from consideration when studying multiplicity.Given the above reduction of signatures to equivalence classes, the focus of this work is in extracting signatures that satisfy two desirable optimality properties: (a) X and Y of the phenotypic response variable T are maximally predictive and non-redundant if and only if X and Y are Markov boundaries of T. A Markov boundary M of T is a set of variables that (i) renders all other variables outside M and T independent of T conditioned on M  and (ii) no proper subset of M is a Markov blanket of TM under distributional assumptions intersection property of probability distributions is a sufficient condition for uniqueness of Markov boundaries As is proved in A, B, C and a phenotypic response variable T) and constraints on their joint probability distributions. Each member of this class is derived by parameterizing the joint probability distribution subject to the constraints. An example of a parameterized Bayesian network is provided in T: {A, C} and {B, C}. Furthermore, {A, C} and {B, C} remain maximally predictive and non-redundant signatures even in infinite samples from that distribution .There exist two maximally predictive and non-redundant signatures (Markov boundaries) of multiplicity does not require very dense connectivity).The pathway structure has very low connectivity  .B, C}), then there is danger of missing biologically important (causative) genes  and focusing instead on confounded genes  .If an algorithm selects only one maximally predictive and non-redundant signature .The union of all maximally predictive and non-redundant signatures includes all genes located in the local pathway around T .In this example, the intersection of all maximally predictive and non-redundant signatures contains only genes in the The above example is concerned with the large sample case. In practice, one deals with small samples where statistical inferences have to be made about large sample predictivity and redundancy. This creates an additional source of error and concomitant multiplicity. An example of this is given in M of the phenotype with maximal predictivity and no redundancy (Markov boundary). The same base algorithm is applied repeatedly to versions of the original dataset in which some subset of variables G has been removed (step 4). If a new signature Mnew has the same predictivity for the phenotype as M, then it is a Markov boundary and it is output (step 5). Steps 3–5 are repeated until no subset G can be generated in step 3.Generalized Local LearningThe base Markov boundary induction algorithm must be suitable for the distribution at hand. Thus, TIE* is a generative algorithm that is instantiated differently for different distributions. In the experiments reported in this paper, we use as the base algorithm HITON-PC , which iTIE* is guaranteed to be correct in the large sample under its stated assumptions. In the small sample some signatures that are not maximally predictive and/or redundant will be statistically indistinguishable from the maximally predictive and non-redundant ones. This indistinguishability occurs at two different levels: one is estimation of predictivity and testing for statistical significance of differences in predictivity among signatures. The second level is the performance of tests of conditional independence  with small samples inside the base algorithm which incurs errors of type I and II. As the sample size grows, the algorithm will output only truly maximally predictive and non-redundant signatures.We present several experiments testing the new algorithm and comparing it against 8 previously described multiple signature extraction methods. The methods comprise of four resampling-based algorithms, one iterative removal method, and three stochastic gene selection methods  experiments where generative models are known and in the case of simulated data all maximally predictive and non-redundant signatures are known as well (details about data generation are provided in discovery dataset”) is used for identification of signatures and estimation of their predictivity by holdout validation validation dataset”) originating either from a different laboratory or from a different microarray platform is used for validation of predictivity of the signatures. No overlap of subjects between discovery and validation dataset analyses occurs in this design. The criteria for dataset admissibility and exact protocol for quality assurance and processing of pairs of datasets is described in To test reproducibility of molecular signatures, we adopt an experimental design where one microarray dataset  measured by area under ROC curve (AUC); finally we compute the maximum value of the above average predictivity estimates and refer to it as “maximal predictivity”.Statistical comparisons of predictivity between methods in the same dataset are accomplished by Wilcoxon rank sum test with α = 0.05 non-reducible if it is not properly included in any other signature output by this method . For example, if a method outputs 3 signatures with the following genes: {A, B, C}, {A, B, X}, and {A, B}, only signature {A, B} is non-reducible. The number of maximally predictive signatures  and the number of maximally predictive and non-reducible signatures output by the algorithm for each sample size in resimulated data is shown in This is consistent with the existence of at least two sources of multiplicity: one is small sample size and the other is multiplicity intrinsic to the nature of gene-gene and gene-phenotype relations. As sample size grows, the first source vanishes and only the second one remains. Since the resimulated data distribution closely mimics the real-life distribution , the magnitude of this difference is <0.009 AUC on average over all discovered signatures; thus this particular deviation from maximal predictivity may be considered negligible for most practical purposes.Leukemia 5 Yr. Prognosis task is provided in A detailed example of application of multiple signature extraction methods to the averaged over all datasets and all discovered signatures. Recall that validation datasets originate from different laboratories and/or using different microarray platforms than discovery datasets. The horizontal distance of each method to the diagonal measures the magnitude of overfitting defined as the difference (ε1-ε2), where ε1  =  expected performance in the validation data obtained by holdout validation in the discovery dataset, and ε2  =  observed validation dataset performance. TIE* rests slightly right of the diagonal denoting no overfitting, or equivalently perfect statistical reproducibility on average. However all other methods exhibit varying degrees of non-reproducibility. Depending on method the average magnitude of overfitting varies from 0.02 to 0.03 AUC.Analysis of the signatures output by TIE* reveals that they share many genes in common. The properties of the data-generative process affect computational feasibility of the signature discovery. In the worst case, it is computationally infeasible to discover even one of all optimal signatures with all known sound algorithms . However, there exist several sound algorithms for extracting an optimal signature that run in low-order polynomial time in real high-throughput data . Even if the computational cost of discovery of one signature was constant, the number of all optimal signatures can grow exponentially large in the number of genes measured  in the complex regulatory network of the underlying biological system. Second, the variability in the output of gene selection and classifier algorithms especially in small sample sizes. Third, the small sample statistical indistinguishability of signatures that have different large sample predictivity and/or redundancy characteristics  Sixth, RNA amplification techniques that systematically distort measurements of transcript ratios  Seventh, cellular aggregation and sampling from mixtures of distributions that affect inference of conditional independence relations that are needed to establish model equivalence according to our framework for multiplicity Eighth, normalization and other data pre-processing methods that artificially increase correlations among genes  ninth, the engineered redundancy in the assay technology platforms . In datasets produced by dissimilar underlying biological mechanisms, assayed with different platforms and pre-processed and modeled with a variety of algorithms, the relative contributions of the above factors to multiplicity will vary. As a result, methods that rely on a specific cause of multiplicity or combination of causes will not output all maximally predictive and non-redundant signatures in all types of high-throughput data.Our results are consistent with the hypothesis that signature multiplicity in real-life datasets is created by a combination of several factors that include the following: With regard to non-TIE* baseline comparison algorithms, we note that resampling-based methods that use bootstrap samples to extract signatures may stop producing multiple signatures in large sample sizes. This is expected because resampling methods are designed to address directly only the small sample multiplicity and not the intrinsic multiplicity which persists in large samples. Iterative removal, on the other hand, by its design always fails to identify all maximally predictive and non-redundant signatures that have genes in common. KIAMB among the baseline algorithms has the strongest theoretical motivation because it can be shown to discover all Markov boundaries for a subset of distributions. However, a major limitation of KIAMB is that it has sample size requirements that range from at least linear to exponential to the number of genes in a signature (depending on the test of independence employed). This makes the algorithm not only computationally inefficient but also prone to statistical errors in small sample sizes. This leads to its substantial observed overfitting in the experiments with real data and its inability to find the maximally predictive and non-redundant signatures in simulated data. KIAMB, being a randomized search algorithm, also guarantees to output all optimal signatures that satisfy its distributional requirements, but only after infinite number of runs. The method by design will discover the same signatures over and over again further compounding its computational inefficiency.Dealing with molecular signature multiplicity using a Markov boundary framework and the TIE* algorithm does not require a particular combination of factors causing signature multiplicity in order to be able to discover all maximally predictive and non-redundant signatures. Because of efficient heuristics, TIE* can extract the signature set very quickly when the connectivity is locally sparse, and the number of true optimal signatures is low-order polynomial or smaller in the number of variables. A very important factor for the performance of TIE* is the choice of the base algorithm to discover non-redundant and maximally predictive signatures in the distribution at hand. Latest developments in Markov boundary discovery provide such tools for high-throughput data. One of the key advantages of these methods is their ability to implicitly control for false discovery rate Our experiments used real data exclusively from human cancer gene expression microarrays because of pragmatic reasons: known identity of observed variables, number and size of datasets, and maturity of standardization protocols that allows for multiple independent dataset validation experiments. The methods introduced here are however directly applicable to any high-throughput data, and future research in this direction is warranted. As an example of applicability of TIE* to high-throughput data beyond gene expression microarrays, we applied the method to proteomics mass-spectrometry data where TIE* identified hundreds of signatures of ovarian cancer with AUC = 0.95−0.98  and phenotypic response variable T.Example pathway structure with 3 gene variables ((0.02 MB PDF)Click here for additional data file.Figure S2HITON-PC algorithm.(0.05 MB PDF)Click here for additional data file.Figure S3Number of signatures output by TIE* as sample size grows.(0.01 MB PDF)Click here for additional data file.Table S1Gene expression microarray datasets used in this work.(0.03 MB PDF)Click here for additional data file.Table S2Results of experiments with artificial dataset with 30 variables.(0.02 MB PDF)Click here for additional data file.Table S3Results of experiments with artificial dataset with 1,000 variables.(0.02 MB PDF)Click here for additional data file.Table S4Results for the number of output signatures , number of genes in a signature, and classification performance in discovery and validation microarray datasets.(0.06 MB PDF)Click here for additional data file.Table S5Number of common genes in 50%, 60%, …, 100% of signatures discovered by TIE* algorithm for each dataset.(0.04 MB PDF)Click here for additional data file.Text S1Maximally predictive and non-redundant molecular signatures are precisely the Markov boundaries and vice-versa.(0.07 MB PDF)Click here for additional data file.Text S2An example of signature multiplicity due to small samples.(0.08 MB PDF)Click here for additional data file.Text S3Details about the TIE* algorithm.(0.15 MB PDF)Click here for additional data file.Text S4Previous algorithms for multiple signature identification used in experiments.(0.08 MB PDF)Click here for additional data file.Text S5Data generation details.(0.13 MB PDF)Click here for additional data file.Text S6Criteria for microarray gene expression dataset admissibility and protocol for quality assurance and processing.(0.04 MB PDF)Click here for additional data file.Text S7The number of maximally predictive and non-redundant signatures is worst-case exponential to the number of variables.(0.08 MB PDF)Click here for additional data file.Text S8An example of signature multiplicity due to hidden variables.(0.02 MB PDF)Click here for additional data file.Text S9Preliminary experiments with the TIE* algorithm in mass-spectrometry proteomics data.(0.02 MB PDF)Click here for additional data file.


A method for the comparison of mRNA expression levels of most human genes across gene expression array experiments, and a database of the results, are presented. Our knowledge on tissue- and disease-specific functions of human genes is rather limited and highly context-specific. Here, we have developed a method for the comparison of mRNA expression levels of most human genes across 9,783 Affymetrix gene expression array experiments representing 43 normal human tissue types, 68 cancer types, and 64 other diseases. This database of gene expression patterns in normal human tissues and pathological conditions covers 113 million datapoints and is available from the GeneSapiens website. A fundamental challenge in the post-genome era is the identification of the context-specific functions of human genes across healthy and disease tissues. Thousands of gene expression microarray measurements are performed each year by the scientific community and many of the data are made publicly available. In order to make use of this resource, integration of large collections of gene expression data from different tissues and microarray platforms is required. Available datasets, however, are often discordant and challenging to integrate due to the variety of the technologies used. Nevertheless, meta-analyses have already been shown to facilitate the analysis of gene expression across healthy and disease states -3. Due tin silico transcriptomics data that currently integrates 157 separate studies involving 9,783 human specimens, from 43 normal tissue types, 68 cancer types and 64 other disease types. The launch of the database was made possible by the development and validation of a novel method to normalize data arising from different Affymetrix microarray generations. The array data are linked with detailed clinical classifications and endpoints and are available through an interactive web interface designed for exploration by biologists and available at the GeneSapiens website , where ak is the lowest 2% value and bk is the largest 2% value of gene k. AGC values should not go over this range. However, if the new centered value exceeds the range, the difference is diminished towards the range limits with coefficient c, 0 ≤ c ≤ 1. Here, the coefficient is set to c = 1/5. Coefficient c is necessary to prevent some extremely tissue-specific genes from having arbitrarily large correction factors, which is possible if the specific tissue is absent from one or more array generation. The coefficient c affects 2.9% of all correction factors. Of those cases, the proportion of the correction factor modified by coefficient c was, on the average, 7.6%. Thus, the coefficient c affected an extreme minority of the corrections in a significant manner, but nevertheless, it was found to be crucial for the AGC method. The centered values can now be obtained with:To be able to compare the samples of xi, j, k is the value of gene k in sample j from array generation i, μi, k is the mean of the values of gene k across array generation I, and μall, k is the mean of the values of gene k across all array generations. Further, the adjusted values are computed based on the equation:where y.The resulting AGC values are now AGCvalue = 2Some other methods ,59 are uAnnotation of the samples is important to make biological and medical sense of the data. Since not all sources of CEL files come with annotations following the MIAME standards , we perfGene annotation is based on Ensembl. The database has data for each Ensembl gene, even those not featured on any arrays. Gene data include transcript and protein product information, chromosome name and position (band and nucleotide count), biotype , and Hugo and Entrez IDs for each gene. These data were downloaded from the Ensembl web site, using the same Ensembl genome build version (release 46) as that used for the construction of the used alternative CDF files .We utilized classic MDS in order to diminish the number of the dimensions within the data . With MDK-means clustering was performed with default parameters in R. The initial centroids were given as the median value of each gene in array generations or tissues. The algorithm was allowed to run for a maximum of 100,000 iterations for each clustering. The corrected rand index  was calcReplicate analysis was performed by comparing the correlation coefficients of the logarithmic values of two or three hybridizations from a single biological sample using standard methods of computing the Pearson correlation coefficient. This was done for all samples described in ,26,27.We visualize the expression profile of a single gene across all human tissues with boxplots and with custom designed body-wide expression plots. In the boxplots, the expression profiles of a single gene are displayed and grouped into healthy samples (green boxes) and malignant samples (red boxes). Both types are in anatomically meaningful order, allowing easy comparison of related tissue types. Numbers of samples in each tissue type are in parentheses.Custom designed body-wide expression profiles show the expression pattern of a single gene at the level of individual samples, while its layout allows easy analysis of the biological or medical significance of the profile. The y-axis provides the expression level of the gene and the x-axis contains all samples arranged into a fixed order by the type of the sample  and subsequently by the tissue type. Thus, each dot describes the expression level of a particular gene in one sample. The anatomical origin of each sample can be seen from the color bar at the bottom of the image. Tissues expressing the gene at a high level (more than one standard deviation higher than the baseline for that gene or having a group of outlier data points) are colored.in vivo tissues. The number of samples per tissue type is given in parentheses. Values for each gene are mean-centered at 0 with a standard deviation of 1.Bodywide expression maps of genes are done with hierarchical clustering (Euclidean distance with Ward linkage) of mean expression profile for 342 genes across 110 in silico transcriptomics data of this project are composed of custom integration of already public microarray data we provide a table describing the origins of the data used to construct GeneSapiens . We have set up a website [As the  website  to allowAGC: array-generation-based gene centering; GIST: gastrointestinal stromal tumor; MDS: multi-dimensional scaling; PLAP: placental alkaline phosphatase; Q: quantile; QAGC: Q normalized data to which AGC correction has been applied.The institute has filed a patent application regarding the normalization methodology.SK contributed to the majority of data analysis, database construction and development of normalization and writing of the manuscript. RA and MS contributed to the development and testing of the normalization. KO and EB contributed to data collection and the annotation process. KO also contributed to data mining methods and checking of all annotations. KI had a major contribution to the annotation. SH contributed to the development of normalization and supervised the comparison and validation of the normalization methods. OK supervised the entire project for database construction, data mining and annotation efforts and participated in manuscript writing and editing. The remaining authors contributed towards annotation, data visualization and other methods as well as editing the manuscript.N) estimated from it (dashed line). Additional data file FEV has clearly elevated expression in several malignancies, when compared to any healthy tissue. Most interestingly this ETS-factor family member appears to have slightly elevated expression in prostate cancer when compared to healthy prostate. Additional data file C1orf56 gene, also known as AF1Q or MLLT11, shows extreme expression in several cancers, especially in neuroblastoma.The following additional data are available with the online version of this paper. Additional data file N8, ) estimatN) estimated from it (dashed line).Distribution of preprocessed datapoints across the entire database (solid line) and normal distribution (Click here for file(a) KLK3 (PSA) is a known prostate specific gene. This specificity is perfectly shown in its expression profile. (b) GFAP, a gene coding for glial fibrillary acidic protein, is known to be expressed in central nervous system. Its expression profile perfectly confirms this prior knowledge.Click here for fileSources for all the raw expression data files used in this study.Click here for fileVarious healthy tissues, cancers and non-cancer diseases represented by the samples in the database and the amounts of samples in each of these categories.Click here for fileRand indices for the different normalizations, and the distribution of array generations and tissues into clusters with Q and QAGC normalized data.Click here for file(a) Insulin (INS) has pancreas specific expression, as one expects it to have. (b) LDHC is a known testis-specific gene and it is expressed above background only in healthy testis.Click here for fileMost interestingly this ETS-factor family member appears to have slightly elevated expression in prostate cancer when compared to healthy prostate.Click here for fileC1orf56 gene, also known as AF1Q or MLLT11, shows extreme expression in several cancers, especially in neuroblastoma.Expression of the Click here for file