RNA-seq, a massive parallel-sequencing-based transcriptome profiling method, provides digital data in the form of aligned sequence read counts. The comparative analyses of the data require appropriate statistical methods to estimate the differential expression of transcript variants across different cell/tissue types and disease conditions.We developed a novel nonparametric empirical Bayesian-based approach (NPEBseq) to model the RNA-seq data. The prior distribution of the Bayesian model is empirically estimated from the data without any parametric assumption, and hence the method is “nonparametric” in nature. Based on this model, we proposed a method for detecting differentially expressed genes across different conditions. We also extended this method to detect differential usage of exons from RNA-seq data. The evaluation of NPEBseq on both simulated and publicly available RNA-seq datasets and comparison with three popular methods showed improved results for experiments with or without biological replicates.http://bioinformatics.wistar.upenn.edu/NPEBseq.NPEBseq can successfully detect differential expression between different conditions not only at gene level but also at exon level from RNA-seq datasets. In addition, NPEBSeq performs significantly better than current methods and can be applied to genome-wide RNA-seq datasets. Sample datasets and R package are available at  The advent of massive parallel sequencing, popularly known as Next-Generation Sequencing (NGS), is allowing whole genomes and transcriptomes to be sequenced with extraordinary speed and accuracy, providing insights into the bewildering complexity of gene expression at both gene and isoform levels . With deRNA-seq experiments are now frequently employed for identifying genes and alternatively spliced gene isoforms that are differentially expressed across distinct tissue/cell types and disease conditions . This amSimilarly, several tools have been developed for DE analysis of RNA-seq data. The Poisson model has been successfully used to account for technical variations in RNA-seq data . When biHere, we developed a novel method to model the RNA-seq data and detect differentially expressed genes and exons across different conditions. To mitigate the biases caused by the nature of sampling and reliably estimate the expression levels of those unseen and lowly expressed genes, we adopted a previously developed Poisson mixture model to empirically estimate the prior distributions of read counts completely from the data . We propA few of the earlier RNA-seq assessment studies have reported highly reproducible results with little technical variation ,28, suggG(x) = ∫ λx/x ! eλ‒ dG(λ) is a G-mixture of Poisson.Let γ be the expression level of one gene under one condition and x be the number of observed reads mapped to this gene. It is well known that x follows a binomial distribution and can be approximated well by a Poisson distribution with mean λ = γdl, where 1 is the gene length and d is the normalization constant reflecting the sequencing depth. Given a prior mixing distribution G (with probability density function g(λ)) on λ, the posterior distribution of λ is G(x)/(1 − hG(0)) or a mixture fQ( x ) of truncated Poisson, where A gene is expressed if, and only if, x ≥ 1. Conditioning on x ≥ 1, x follows a Q-mixture of zero-truncated Poisson hx denote the number of genes with exactly x reads in the sample. The conditional nonparametric maximum likelihood estimator Let nussed in ,30 and tA and λB denote the read counts that represent the true expression level of a gene and GA and GB denote the corresponding prior distributions, under conditions A and B, respectively. However, as mentioned previously, since NGS is like sequencing a set of sampled reads from a pool of expressed sequences of gene, the read counts that are obtained, say xA and xB, denote the corresponding reads counts obtained in conditions A and B. The posterior distribution of log Let λThe normalization constant d can be inferred from some previous available methods, for example the methods proposed in DESeq or PoissonSeq . This caThus, we rank the genes by the values of NPEBseq tests the hypothesis that the difference in the gene expression level between conditions A and B is above a user-defined cutoff Δ, i.e., the probability that The default value for Δ is log(2). We consider this as our own pre-defined p-value. The false discovery rate is controlled with Benjamini-Hochberg adjustment.RNA-seq datasets with large numbers of biological replicates are increasingly generated by many laboratories and consortia, for example, HapMap , ENCODE Let c denote the number of biological replicates for one condition and we assume thatij is the number of reads for gene i and replicate j; eij is the expression index; λi is the expression level of gene i under this condition A; θ is the scale parameter of Gamma distribution; and dj is normalization constant for replicate j. The prior distribution G is inferred as before by using the sample that has the largest data depth under each condition.where xl=1 does not change the calculation. Let ij, λi is given by Here we are interested in inferring the posterior distribution of fold change for each gene, in which the 1 will be cancelled out, so simply letting i is given byIt is known that marginal conditional The p-values and FDR can be computed by equation (4).ij|λi follows negative binomial distribution (mixture of Poisson with Gamma prior) and the expected value and variance of it are given byEmpirical Bayes methods have been used to estimate the degrees of over-dispersion in the data. Based on our hierarchical model, we also propose an empirical Bayes method to estimate the dispersion parameter θ. It is known that the conditional variable xij is unclear, the expected value and variance can be computed in the following ways:Although the marginal distribution of xSo we estimate θ by,Similar to the estimation of G, θ is also estimated by using the sample that has the largest data depth under each condition.RNA-seq also provides information for the study of alternative splicing. DE analysis of individual transcripts is essential in many comparative studies because of isoform-level changes in gene expression between conditions . RecentlAk and tBk denote the expression level of counting bin k of this gene and tAk and tBk denote the observed read counts overlapping with it. The differential exon detection method involves the following steps:DEXSeq tests if each counting bin is differentially used between conditions. Inspired by this, we propose a method to detect different exon usage based on our Bayesian hierarchal model. Assuming that a gene is expressed under two different conditions, A and B, let tAk|yAk and tBk|yBk is derived based on our model applied to counting bin read count data.1. The posterior distribution of t2. Test the fitness of the distribution against the null hypothesis: the proportion of the number of reads overlapping with a counting bin to that of all the reads overlapping with the gene does not change between conditions (same as in DEXSeq).3. Finally, define the p-value as the probability of j can be computed by the method proposed in DESeq or the trimmed mean of log-folder change method [For cases without replicates, the normalization constant can be computed by equation (3). With biological replicates, de method . Here weGA and GB follow different statistical distributions. Dataset2 consists of no replicates and is generated by following the simulation scheme adopted by baySeq and edgeR. Dataset3 is generated by the same scheme as dataset2 but with biological replicates.We generated three simulation datasets to evaluate the proposed method for identifying differentially expressed genes. Dataset1 is generated with biological replicates by assuming different priors across conditions, which means NPEBSeq is a nonparametric empirical Bayesian-based approach to model the RNA-seq data. The expression level of genes with low read counts is estimated by borrowing information from the gene expression in the whole sample. The non-parametric form of the prior distribution avoids any unrealistic assumption. The parametric assumption for the prior distribution is usually not fulfilled for the RNA-seq read count data. The fact that there are many genes expressed at low levels in one sample is illustrated in Figure To evaluate the proposed method for identifying differentially expressed genes, we first conducted a simulation study.To generate data similar to those produced by real RNA-seq experiments, we first applied the empirical Bayes method on publicly available RNA-seq datasets, which were generated to compare liver and kidney transcriptomes . The priDataset1 consist of 20 independent simulations with seven samples each for two conditions. The library size of each sample is uniformly sampled from 300,000 to 900,000. Each sample was generated by a mixture of negative binomial model with both the prior distributions and dispersion parameters estimated from Marioni’s data. Each sample consists of 10,000 genes for computational efficiency.http://bioconductor.org[We performed a comparative analysis of our method with four popular methods, DESeq, edgeR, baySeq and NOISeq, which are available as part of Bioconductor packages at uctor.org. The edgτ, the expected percentage of the true first τ DE genes being correctly declared as the first τ DE genes. The average of estimated pτ is calculated from the 20 replicates. The simulation results for dataset1 are shown in Figure We used the following criteria to compare the performance of different methods. Given a cutoff point τ (e.g. the number of genes declared significantly expressed), the efficiency of a statistical method is measured by pA simulation scheme similar to the one suggested by Robinson and Smyth  is appliA = bλB. In order to produce both over- and under-expression in our simulated data, 500 randomly selected genes were set to have b=4 and the remaining 500 genes were set to have b=1/4. Both dataset2 and dataset3 consist of 20 independent simulations. Dataset2 was generated without replicates. Similar to dataset1 seven samples per condition per simulation were generated for dataset3. The full ROC curves for dataset2 and dataset3 are shown in Figures Dataset2 was generated by Poisson distribution and dataset3 by negative binomial distribution, with the dispersion parameter estimated from the liver data. The simulated data consists of 10,000 genes, and one-tenth of those genes were set to be differentially expressed (between condition A and condition B) with λTo further evaluate our method, we tested it on two published RNA-seq datasets.http://www.ncbi.nlm.nih.gov/sra, which contains seven technical replicates of brain reference RNA samples and seven technical replicates of UHR RNA samples. Tophat [http://www-huber.embl.de/users/anders/HTSeq/), using the annotation of the Ensembl genes and only reads that mapped to exons.We first applied NPEBSeq on the MicroArray quality control (MAQC) dataset ,41 and c. Tophat  was usedt methods [As part of the original MAQC project, around 1,000 genes were also chosen to be assayed by Taqman qRT-PCR. Those qRT-PCR data were obtained from GEO database, which contains four technical replicates for each of the two samples. The qRT-PCR data were used as a gold standard to benchmark the gene expression values by RNA-seq. We analysed the qRT-PCR data using the comparative C methods . Finally methods  to definand the false positive rate (FPR) is defined asNote that these are not the standard definitions of TPR and FPR.qRT-PCR data were annotated by RefSeq. The BioMart R package  was usedThe ROC curves from all the compared methods are shown in Figure We also analysed the data by Brook et al. , where tWe then analysed Brook’s data at exon level. NPEBseq found differential exon usage for 2,370 counting bins at FDR 0.01 for between-condition comparison and 225 counting bins for within-condition comparison. We also applied the newest version of DEXSeq on the exon data, which reported 120 counting bins as DE at FDR 0.1. We checked whether NPEBseq and DEXSeq could achieve comparable results by computing the percentage of DE called exons that are common in the two ranked lists of exons generated by both programs. The results are shown in Figure In this paper we developed a novel empirical Bayesian-based approach to model the RNA-seq data. This method has been widely used in ecology to estimate species diversity . The nonThere are many challenges still present in the processing and analyses of RNA-seq data. For example, it has been empirically observed that quantification of expression depends on the length of the biological features under study , as longer features tend to have more significant statistics than shorter ones . Also, iDelineating the gene expression at an alternative transcript-level from RNA-seq data is still a very challenging problem. Our recently published IsoformEx method , based oNPEBseq can be applied to not only detect differential gene expressions from the RNA-seq dataset with technical and biological replicates for both studied conditions, but also to detect differential usage of exons. It is robust, since it requires no limited assumptions to be made about the prior distribution of the data. NPEBseq also provides the closed form of posterior distribution of the fold change, which is useful for further analysis.Project name: NPEBseqProject home page:http://bioinformatics.wistar.upenn.edu/NPEBseqOperating system and R version: The R package is platform independent and is compatible with all the versions of R same as or higher than 2.15.1.Other requirements: No.License: GPL (≥ 2)Any restrictions to use: It is available for free download.Both authors declare that they have no competing interests.YB and RVD conceived the initial approach. YB designed and implemented the methods and performed the analyses. YB and RVD wrote the manuscript. All authors read and approved the final manuscript.This file describes the procedure to derive the marginal posterior distribution of λi and the procedure to infer the prior distribution of our proposed model.Click here for fileFigure S2 Partial ROC curves based on simulated dataset3.Partial ROC curves based on simulated dataset2. Click here for file


E. coli and S. cerevisiae. The flux predictions made by different methods using transcriptomic data are compared against experimentally determined extracellular and intracellular fluxes (from 13C-labeling data). The sensitivity of the results to method-specific parameters is also evaluated, as well as their robustness to noise in the data. The results show that none of the methods outperforms the others for all cases. Also, it is observed that for many conditions, the predictions obtained by simple flux balance analysis using growth maximization and parsimony criteria are as good or better than those obtained using methods that incorporate transcriptomic data. We further discuss the differences in the mathematical formulation of the methods, and their relation to the results we have obtained, as well as the connection to the underlying biological principles of metabolic regulation.Constraint-based models of metabolism are a widely used framework for predicting flux distributions in genome-scale biochemical networks. The number of published methods for integration of transcriptomic data into constraint-based models has been rapidly increasing. So far the predictive capability of these methods has not been critically evaluated and compared. This work presents a survey of recently published methods that use transcript levels to try to improve metabolic flux predictions either by generating flux distributions or by creating context-specific models. A subset of these methods is then systematically evaluated using published data from three different case studies in  Constraint-based modeling has become one of the most successful approaches for modeling large-scale biochemical networks. There are nowadays hundreds of genome-scale reconstructions of metabolic networks available for a wide variety of organisms ranging from bacteria to human cells. One of the limitations of the constraint-based approach is that it describes the cellular phenotype simply in terms of biochemical reaction rates, in a way that is disconnected from other biological processes such as genetic regulation. In order to overcome this limitation, different approaches for integration of gene expression data into constraint-based models have been developed during the past few years. However, all the methods developed so far have only been tested using isolated case studies. In this work, we elaborate a detailed survey of these methods, and perform a critical and quantitative evaluation of a selected subset of methods, using experimental datasets that include different organisms and conditions. This study highlights some of the current limitations in many of these methods, and reveals that no method published so far systematically outperforms the others. During the past years, there have been successful applications of computational modeling of cellular metabolism in biotechnology omics data into constraint-based models are being developed In order to study the effects of environmental perturbations or genetic manipulations on cellular metabolism, one can measure changes at the transcript, protein, metabolite, and flux levels. However, analyzing the coordinated behavior of the different biological processes requires the integration of this information under one common framework. Although flux measurements can be easily integrated into constraint-based models The advancements in high-throughput sequencing methods have increased the speed and decreased the cost of DNA and RNA sequencing Novel methods in computational biology are usually evaluated with dedicated case studies and data sets in the publications that introduce them. However, it is difficult to systematically evaluate and compare methods as the cases and data sets used in different publications are usually different. For this reason, there has been increased attention recently on performing dedicated studies that compare multiple methods using common cases and data sets This work presents a comprehensive survey and a critical evaluation of the methods for integrating transcriptomics data to genome-scale metabolic models published thus far, with the purpose of guiding a careful methods selection and discussing limitations of the existing methods. In the spirit of reproducible research, we also aim to make it easier to evaluate new methods developed in the future by making all of our code and preprocessed datasets publicly available to the scientific community.i.e. they return both a context-specific model and a metabolic flux distribution for the complete model consistent with the gene expression data. At the implementation level, the methods differ mainly in the way they use the expression data, by integrating either discrete or continuous expression levels, and by using absolute values for a single condition, or relative expression levels between different conditions  specific models from more generic organism-specific models. A typical example is the creation of models for different kinds of human cells using the global human metabolic reconstruction, which can be used for the study of tissue specific diseases nditions .S. cerevisiae.One of the earliest approaches to integrate gene expression data into constraint-based models was proposed by Åkesson and co-workers with the purpose of improving flux predictions E. coli strains, and to create tissue-specific human cell models. A recent extension of the method, GIM3E, extends the previous method in order to force a minimum turnover rate for experimentally measured metabolites Gene Inactivity Moderated by Metabolism and Expression (GIMME), uses gene expression data to build context-specific models The integrative metabolic analysis tool (iMAT) Moxley and coworkers developed a method that predicts flux variation as a function of variation in gene expression E–Flux is a method that directly maps normalized gene expression levels into flux bound constraints using a “pipe capacity” analogy Probabilistic regulation of metabolism (PROM) is a method for integration of regulatory and metabolic networks Metabolic Adjustment by Differential Expression (MADE) aims to overcome the problem of selecting arbitrary thresholds by comparing measurements across multiple conditions Transcriptionally controlled FBA (tFBA) follows the same principles as MADE, albeit with a different formulation The Integrative Network Inference for Tissues algorithm (INIT) is a method for building tissue-specific models from genome-scale reconstructions S. cerevisiae under two growth conditions.Lee and coworkers proposed a method that integrates absolute gene expression data directly into the objective function of a constraint-based model, instead of manipulating the flux constraints The method developed by Fang and co–workers is based on the hypothesis that the flux distribution for a reference condition can be calculated with existing methods, and that the differential gene expression between a perturbed and reference condition can be used to predict the flux distribution for the perturbed state RELATive CHange (RELATCH) is a two-step method that uses flux and gene expression data from a reference state to predict metabolic responses in a genetically or environmentally perturbed state Temporal Expression-based Analysis of Metabolism (TEAM) is a method that combines dynamic Flux Balance Analysis (dFBA) Adaptation of Metabolism (AdaM) is a method for integration of temporal gene expression data Gene-expression FBA (GX–FBA) is a flux prediction method that incorporates gene expression data into flux balance analysis Metabolic Context-specificity Assessed by Deterministic Reaction Evaluation (mCADRE) is a context-specific model building method Flux-coupled genes (FCGs) is a method developed under the hypothesis that, although gene expression and flux levels are not always correlated, there might exist a subset of genes whose expression levels are consistent with the respective fluxes The EXploration of Alternative Metabolic Optima (EXAMO) method for context-specific model building extends the iMAT approach by searching for multiple optima with the same agreement score In order to compare the predictive capability of the different methods, we tested their predictions using experimental datasets taken from the literature. From the initial survey of methods, we evaluated those that provide an implementation and that can be directly used for flux prediction. Note that although some of the methods not evaluated did provide an implementation, it was specific to a particular case study and not readily usable for general-purpose application.13C-labeling) obtained using the same exact conditions for the same strain. However multi-omics studies that contain these two kinds of data are not common to find even for widely studied microbes such as E. coli and S. cerevisiae. A compilation of suitable multi-omic datasets is presented as supplementary material  influence the prediction of the intracellular flux distribution. Also, in order to understand how the integration of gene expression data can improve phenotype prediction, we compare the results with those obtained by standard FBA simulation. To avoid the typical degeneracy of FBA solutions, the parsimonious version of FBA (pFBA) was used. This approach finds a flux distribution with minimum absolute flux values among the alternative optima, assuming that the cell attempts to achieve its objective (here assumed to be maximization of biomass production) while allocating the minimum amount of resources.et alE. coli strains growing aerobically in a chemostat at a dilution rate of 0.2 h−1. The different experiments include variations of the dilution rate (from 0.1 to 0.7 h−1), and several single-gene knockout mutants growing at the reference dilution rate. For this dataset, the gene expression data is limited to the central carbon metabolism and is measured by microarray analysis.This case study uses a comprehensive omics dataset published by Ishii E. coliThe assessed methods were applied to a genome-scale metabolic reconstruction of −1). This is a typical case where FBA simulations are less accurate, since the assumption of growth yield maximization no longer holds true due to overflow metabolism. This is one of the experiments where pFBA gives a higher prediction error, and a likely scenario where alternative methods, such as those studied herein, will be most useful.In order to understand how the phenotype predictions vary across the different methods, we analyze in detail a particular case, namely the experiment at the highest dilution rate  and the methods that impose some predefined threshold of biomass production , predict values close to the maximum theoretical level. On the other hand, methods that do not impose any constraints regarding the growth rate simply predict no growth at all .In order to understand the influence of imposing experimental measurement constraints on the predictive ability of the methods, all the simulations were repeated using the complete set of measured uptake, growth and secretion rates as constraints . As expeA comparison between the predicted and measured fluxes across conditions for all the methods is given as supplementary material . The expFinally, we test whether integration of proteomic data  results in more accurate predictions than the use of gene expression data . Despiteet alE. coli strains growing aerobically in batch cultures. The study compares the phenotype of the wild-type strain with two over-expression mutants, nox (NADH oxidase) and atpAGD (F1-ATPase), with the goal of understanding global transcriptional responses to lowered levels of NADH and ATP. The dataset contains gene expression data measured at the genome scale using microarray analysis and 13C-flux data.This case study uses a dataset from Holm The methods were tested using the same metabolic model as in the previous case study. In this dataset glucose uptake is the only measured uptake rate. The error distributions are shown in The predicted phenotypes for the over-expression mutants are analyzed in more detail . Unlike The impact of including the measured growth and secretion rates as constraints was also measured . As expeA comparison between the predicted and measured fluxes for all conditions is given as supplementary material . It is iet alS. cerevisiae strains growing in a glucose-limited chemostat at a dilution rate of 0.1 h−1 with different oxygenation levels. These include intermediate levels from fully anaerobic to fully aerobic. The dataset contains genome-wide gene expression data. Fluxomic data for the same conditions could be obtained from a separate publication This case study uses a dataset from Rintala S. cerevisiaeThe assessed methods were used to integrate the gene expression data into a recent genome-scale metabolic reconstruction of We analyze in more detail the results for the two extreme conditions, complete aerobiosis and complete anaerobiosis . For theAs in the previous case studies, we analyze the impact of including the complete physiological measurements  as constraints . A decreFinally, we tested whether integration of proteomic data  results in more accurate predictions than the use of gene expression data . Since tThree of the methods evaluated, namely GIMME, iMAT and MADE are parameterized, which makes the results presented so far dependent on the particular choice of the parameter configuration. Therefore, the sensitivity of the prediction error with respect to the parameter values was analyzed. For each case, one parameter was varied at a time while the others remained fixed see . In ordeThe results show that for most parameters the variation is not monotonic with respect to the parameter value, and that the variance for one particular value can be larger than the average variation across the whole parameter range , S6. NevFinally, we tested the robustness of all the methods towards noise in the data . The levOne would expect a smooth increase in the average prediction error with increasing noise level as an indicator of robustness. This increase in the error should also be accompanied by a gradual increase in the variance in flux predictions (made using different noisy transcript patterns generated at the same level of noise) as an indicator of the absence of systematic bias in flux predictions. However, only E–Flux exhibited this pattern. GIMME and Lee–12 show a gradual increase in the variance, although the average prediction error is the same for the original and the random data. MADE and iMAT show small changes in the average prediction error, coupled with a mostly constant level of variance. GX–FBA shows a smooth increase in the average prediction error, coupled with a sharp increase in variance, and fails to compute for very high levels of noise. RELATCH* shows an apparent constant level of the prediction error, with an increasing variation that is many orders of magnitude lower compared to the other methods. Hence the solution is biased regardless of the gene expression levels.In this work we surveyed a wide variety of methods that integrate gene expression data into constraint-based models of metabolism. The publication rate of these types of methods seems to be rapidly increasing, which shows that the solution to this problem is far from trivial.In general, these methods fall into one of two categories. They have been used to improve the prediction of metabolic flux distributions based on transcript levels, a useful tool for metabolic engineering of microbial cell factories E. coli) and an eukaryotic organism (S. cerevisiae). The experimental conditions included batch and chemostat fermentations, aerobic and anaerobic growth, as well as single gene deletions and over-expression mutants.From the initial survey, seven methods were evaluated in detail, and their predictive ability was compared to that of FBA (in its parsimonious version). The experimental datasets used were selected in order to provide a variety of test scenarios, including a prokaryotic (13C-labeling) for the central carbon metabolism. It is known that the central metabolic pathways are more heavily regulated at post-transcriptional levels All the methods have a lower overall predictive capability compared to pFBA. At first sight this might indicate that the integration of gene expression data is hampering, rather than improving, the prediction of flux distributions. Given the variety of case studies it is unlikely that the selection of experiments was coincidentally favorable towards the pFBA approach. However, our evaluation could still be biased due to the fact that it is only possible to compare the simulated fluxes with intracellular flux measurements  provide more accurate flux predictions than the ones that discretize the expression levels .Discretization also presents a few advantages, such as robustness to noise in the data, seamless integration with the logic-based gene-protein-reaction (GPR) associations, and avoiding data normalization issues. Furthermore, coarse-graining the gene expression data reduces the reliance on a direct proportionality between the fluxes and the transcript levels.Another major distinction between the surveyed methods is the choice between using absolute gene expression levels for one condition, or using differential gene expression between two or more conditions. One of the limitations of using absolute expression levels is the lack of proportionality between transcript and flux levels. A recent review from Hoppe highlights the multiple steps between gene expression and reaction rates Altogether it seems that enforcing a correspondence between absolute transcript and flux levels does not reflect the underlying biochemical mechanisms. In that sense, accounting for relative expression changes as an indicator of the intended flux reconfiguration may provide a more meaningful description. However, the methods that use relative expression levels (MADE and GX–FBA), did not generally give more accurate flux predictions.Another distinction among the presented methods is the utilization of a biological objective function. The mathematical definition of a biological objective is the key step that transforms a metabolic network reconstruction into a model that can simulate the cellular phenotype. The maximization of growth yield, determined from the cellular biomass composition, has been a commonly assumed objective for microbial organisms. Although the validity of this assumption has been experimentally confirmed under some conditions All of the methods evaluated, with the exception of E–Flux, replace the biological objective function with a function that relies on the gene expression data. Nevertheless, some of these methods still use the original objective to define a minimum growth requirement constraint  or to calculate a reference flux distribution (GX–FBA). Methods that do not make any assumptions regarding a biological objective  should be suitable for a larger scope of organisms and experimental conditions. However, these methods incorrectly predicted a zero growth rate in all cases, with the exception of RELATCH* for the yeast case study.In order to evaluate the effect of imposing a biological objective on all methods, we repeated all the tests, adding a minimum growth rate constraint, corresponding to 90% of the maximum theoretical growth rate, to all simulations . We obseDespite the high number of proposed methods, the prediction of flux levels from gene expression data is far from being solved. Although some of the methods evaluated give reasonable predictions under certain conditions, there is no universal method that performs well under all scenarios. Regardless of the mathematical formulation proposed to address the problem, the mapping of transcripts to fluxes is intrinsically hampered by the fact that gene expression levels do not necessarily reflect flux levels, which are systemic properties of the cellular metabolism. Nonetheless, the transcriptome should provide cues to guide the determination of the correct phenotype among the space of solutions that results from the large number of degrees of freedom in metabolic networks.It has been proposed that the metabolic phenotype of microbial cells results from a trade-off between optimality and flexibility towards adaptation https://github.com/cdanielmachado/transcript2flux.Finally, we would like to acknowledge the authors who published their source code with the respective articles. We would like to reiterate the importance of providing published methods in a usable format, a fundamental step for reproducible research E. coli and S. cerevisiae case studies were performed using, respectively, the iAF1260 and iTO977 genome-scale models The simulations for the All method-specific configuration details are given in the following. All methods evaluated in this study have available implementations in MATLAB . These were tested using MATLAB R2012b with Gurobi Optimizer 5.5  running on a 1.7 GHz Intel Core i5 processor.FBA simulations were performed using the available implementation in the COBRA Toolbox The GIMME implementation in the COBRA Toolbox was used. However, this implementation discretizes the gene expression levels, which cannot then be used as weights in the objective function. This was changed to use continuous values as in the original publication The iMAT implementation in the COBRA Toolbox was used. However, this implementation does not use the tri-valued logic used in the original formulation. Therefore, this was changed to mapping scheme described in the original publication An implementation of MADE is provided with the original publication. MADE integrates relative gene expression data for a series of sequential experiments into one large MILP formulation. This can create computationally intractable problems if the number of experiments is too large. Also, most datasets used in this study do not represent sequential experiments. Therefore, each experiment was individually coupled with a given reference condition. . RELATCH uses the gene expression data directly in the objective formulation without any normalization. This makes the results dependent in the particular choice of units for the expression vector. Therefore, the expression vectors were divided by their mean prior to computation.An implementation for GX–FBA is provided with the original publication. This method uses flux variability analysis (FVA) 13C-labeling experiments, some of the experimental flux distributions do not lie precisely within the solution space of the genome-scale models used in this study. Since this would cause a systematic error in the evaluation of all methods, the experimental measurements were adjusted to the respective model by determining the feasible flux distribution with the smallest Euclidean distance to the original values.Due to experimental error and differences in the models used in the and operator) is set to the minimum expression level of the associated genes, and the expression level of reactions catalyzed by isozymes (or operator) is set to either the maximum or the sum of the expression levels of the associated genes. In order to deal with missing gene measurements, we opted to simplify the formula, removing the respective gene, rather than using an arbitrary expression value.For the methods that use continuous gene expression, the expression levels were mapped from genes to reactions using the gene-protein-reaction (GPR) association rules in the models. In each case, the mapping was performed as described in the respective publication. In general, the expression level of reactions catalyzed by enzyme complexes . The error of the estimation is given by the normalized Euclidean distance:Each parameter for GIMME, iMAT and MADE was varied independently, while the others remained fix at the default value described earlier. The required fraction of the biological objective for GIMME and MADE was varied linearly from 0 to 1. The flux activation threshold of iMAT varied from 0.1 to 10 in log-scale. The gene expression thresholds for GIMME and iMAT were defined in terms of the percentile of the given expression data, in order to make the results independent of the given units. The expression threshold for GIMME varied between the 0th and 100th percentile. For iMAT the low expression threshold varied from the 0th to the 75th percentile, whereas the high expression threshold varied from the 25th to the 100th percentile. For each range, a total of 20 equally spaced values were evaluated for the multiple experiments on each dataset.x) was randomly shuffled in order to generate a random expression vector (r) that follows the same distribution as the original data. The noisy expression data (y) is given by:λ parameter it is possible to vary between intermediate noise levels that range from the original data (λ = 0) to completely random data (λ = 1). For each test, λ is varied by 10 equally spaced steps and, for each step, a total of 100 evaluations are performed and the average prediction error is measured.All the methods were analyzed regarding their robustness to noise in the data. The noisy data was generated as follows. For a given experimental condition, the gene expression vector . Difference between predicted and measured fluxes (mmol/gDW/h) for all the evaluated methods, across all conditions from the Ishii dataset for E. coli. All the conditions are sorted by increasing error of pFBA simulation. The error distribution is individually scaled for each method. Missing columns represent failed computations.(TIF)Click here for additional data file.Figure S2Individual flux predictions (Holm). Difference between predicted and measured fluxes (mmol/gDW/h) for all the evaluated methods, across all conditions from the Holm dataset for E. coli. All the conditions are sorted by increasing error of pFBA simulation. The error distribution is individually scaled for each method. Missing columns represent failed computations.(TIF)Click here for additional data file.Figure S3Individual flux predictions . Difference between predicted and measured fluxes (mmol/gDW/h) for all the evaluated methods, across all conditions from the Rintala dataset for S. cerevisiae. All the conditions are sorted by increasing error of pFBA simulation. The error distribution is individually scaled for each method. Missing columns represent failed computations.(TIF)Click here for additional data file.Figure S4Transcriptomics vs proteomics. Comparison of the normalized prediction error for each method across multiple conditions using either transcriptomic or proteomic data. Two scenarios are evaluated: prediction of the complete metabolic phenotype  from measured uptake rates (a–d); and prediction of the intracellular fluxes from the measured physiology  (e–h).(TIF)Click here for additional data file.Figure S5Sensitivity analysis (Holm). Sensitivity analysis of the parameterized methods using the Holm dataset. The averaged normalized prediction errors across all conditions are presented.(TIF)Click here for additional data file.Figure S6Sensitivity analysis . Sensitivity analysis of the parameterized methods using the Holm dataset. The averaged normalized prediction errors across all conditions are presented.(TIF)Click here for additional data file.Figure S7Robustness analysis . Robustness analysis of the different methods towards increasing levels of noise in the data. The noise level varies from 0  to 1 (completely random data). Analysis performed using the anaerobic condition from the Rintala dataset.(TIF)Click here for additional data file.Figure S8Comparison of the prediction errors for the original formulation of the methods (a–c) to the scenario where a minimum growth rate of 90% is enforced for all methods (d–f). The absence of results for iMAT in the Holm dataset results from infeasibility of solutions using the imposed constraints.(TIF)Click here for additional data file.Table S1Original methods validation. Description of the validation approach and data sources for each method in their original publication.(PDF)Click here for additional data file.Table S2Compilation of multiomics dataset references. Compilation of literature references containing multi-omics datasets including transcriptomic and fluxomic data for E. coli and the yeast S. cerevisiae. The datasets used in this study are highlighted.(PDF)Click here for additional data file.


It frequently harbors alterations in DNA and chromosome copy numbers, and can be divided into two major subtypes, hyperdiploid (HMM) and non-hyperdiploid multiple myeloma (NHMM). The two subtypes have different survival prognosis, possibly due to different but converging paths to oncogenesis. Existing methods for identifying the two subtypes are fluorescence in situ hybridization (FISH) and copy number microarrays, with increased cost and sample requirements. We hypothesize that chromosome alterations have their imprint in gene expression through dosage effect. Using five MM expression datasets that have HMM status measured by FISH and copy number microarrays, we have developed and validated a K-nearest-neighbor method to classify MM into HMM and NHMM based on gene expression profiles. Classification accuracy for test datasets ranges from 0.83 to 0.88. This classification will enable researchers to study differences and commonalities of the two MM subtypes in disease biology and prognosis using expression datasets without need for additional subtype measurements. Our study also supports the advantages of using cancer specific characteristics in feature design and pooling multiple rounds of classification results to improve accuracy. We provide R source code and processed datasets at  Multiple myeloma (MM) is a plasma cell malignancy characterized by complex and heterogeneous cytogenetic abnormalities The two MM subtypes have different prognosis and survival outcome. Patients with hyperdiploid multiple myeloma tend to have a better prognosis than those with non-hyperdiploid subtype However, most genomics profiling myeloma studies do not have HMM status measured. The existing standard method for distinguishing the two subtypes is FISH, which measures a few chromosomes' trisomy status and can be inaccurate for determining the hyperdiploidy status of cancer samples. Using array comparative genomics hybridization (aCGH) or SNP arrays to detect genome-wide copy number changes can also readily identify the two major subtypes http://www.ncbi.nlm.nih.gov/geo/) GSE6477, GSE19784, GSE6401 are 3 gene expression profiling datasets from GEO . We use GSE19784 to improve the method for cross platform/dataset classification and as validation data to minimize over-fitting. The rest datasets are used for testing the classification accuracy.http://brainarray.mbni.med.umich.edu/). The probe level data was converted to expression values using the BioConductor project with the Robust Multi-Array average (RMA) procedure, in which perfect match intensities were background adjusted, quantile-quantile normalized, and log2-transformed http://cran.r-project.org/web/packages/kknn/).All gene expression profile analysis procedures were performed using R (version 2.13.1). Package ‘affy’ was used for primary process http://genome.ucsc.edu/).Based on frequent trisomy status of eight chromosomes in HMM, we separate genes into two groups: trisomy-chromosome genes  and nontrisomy-chromosome genes (genes on the rest chromosomes) We used two test datasets with paired copy number microarray samples as gold standard for HMM status. In GSE29023, for each sample we computed the ratio between the median copy number of the trisomy chromosomes and that of the non-trisomy chromosomes, and plotted the histogram of the ratios. We expected this ratio to be around 1.5 (3 copy/2 copy) for HMM samples and 1 (2 copy/2 copy) for NHMM samples, since HMM samples usually have 3 copies in most of the 8 trisomy chromosomes, but NHMM samples have 2 copies in these chromosomes. With the expectation that the patients should be separated into two groups, we observed the histogram and chose the boundary 1.125 as half-way between the two peaks in the distribution. The fact that we didn't observe a cutoff midway between 1 and 1.5 could be due to sample normalization during microarray analysis, where the overall signal distribution in one sample is adjusted to be similar across all samples. This may lead to a downward shifting of all signals in a HMM sample, where 8 chromosomes are usually in trisomy. A cutoff of 1.10 or 1.15 is also possible from this histogram, but this leads to only 3 samples to be assigned differently and a difference in accuracy of <2.6% (3/115 of samples in GSE29023).In GSE12896, we plotted the distribution of the median copy number of the trisomy chromosomes and expected the median to be around 3 for HMM and around 2 for NHMM. We observed the corresponding two peaks, although the HMM samples peak at 2.6 due to possible normalization effect as above. We chose the midway value (2.3) between the two peaks as the cutoff to call HMM and NHMM samples from the copy number data.We used K-nearest neighbor (KNN) method and leave-one-out cross-validation (LOOCV) to develop the classification method for HMM status. Based on this method, we leave one sample out of GSE6477 and use the remaining data as training samples for fitting the classification model. A list of genes differentially expressed between HMM and NHMM of the training samples are used to construct a two-dimensional feature vector to classify the left-out sample. The overall classification accuracy from cross-validation is used to select the optimal model parameters. An overview of the procedure is in First, using GES6477, we left one sample out and used the rest 139 samples to perform Student's t-test between HMM and NHMM using each gene's expression data. Controlling for FDR  that is less than 0.01, we obtained a list of genes that are differentially expressed between HMM and NHMM. We subdivided these genes into trisomy-chromosome genes (TC genes) and nontrisomy-chromosome genes (NTC genes) according whether a gene belongs to trisomy chromosomes or nontrisomy chromosomes (see Methods). Next, we standardized GSE6477 to make the mean of each sample's expression values of all genes to be 0 and the variance of each sample to be 1, and got the GSE6477_scale data. This step is necessary for cross-platform/dataset classification in the next section. Then, for each sample in the training and validation sets of GSE6477_scale, we computed the mean expression of the TC genes and NTC genes, respectively, and denote the two means as TC mean and NTC mean. Last, we used TC mean and NTC means as the feature values of each sample to perform K-nearest neighbor algorithm (KNN) to classify the left-out sample using the 139 training samples. KNN is a method for classifying objects based on closest training examples in the feature space, which is our core algorithm for classification We next used GSE19784 to check the applicability of the classification method developed from GSE6477 when applied to an independent dataset and patient cohort. GSE19784 uses a different array platform, comes from a different research study and has more MM samples (201 MM samples with FISH status). These factors may affect the accuracy of cross-dataset classification. To minimize platform and batch differences between the two datasets, we also standardized GSE19784 to make the mean expression values of all genes in each sample to be 0 and the variance of each sample to be 1, and obtained the GSE19784_scale data. To apply a classification model from one cross-validation run of GSE6477 , we retrOn a closer look, we observed that the relative locations and overall shape of HMM and NHMM samples within a dataset are similar for the two datasets . In ordeLast, we used the adjusted TC mean and NTC mean as sample features to classify samples in GSE19784_scale using models trained from GSE6477. We classified each sample in GSE19784 140 times using each of the leave-one-out cross-validation models from GSE6477. The subtype classified as the majority of 140 times is the final classified subtype of the sample. We then compared the classified subtype of GSE19784 samples to their FISH-based subtype to check the cross-dataset classification accuracy. Using the classification results of GSE6477  and GSE19784 (cross-dataset classification), we explored values of k from 1 to 69 that resulted in high classification accuracy . We seleWe also tested the classification method using two MM expression datasets with paired copy-number profiling samples. The first is GSE29023, consisting of paired gene expression and array CGH samples The second MM dataset is GSE12896 and GSE39754 with 170 paired gene expression and SNP array profiling samples When we applied our classification method trained from GSE6477 to other datasets, we classified each new sample 140 times using the 140 cross-validation models that built from GSE6477. In the above we assigned the final classified subtype using majority voting from the 140 classified subtype calls. We also computed a confidence score for each sample as the proportion of 140 classified subtype calls for the winning majority. When we classified only those samples with confidence score >0.9, the classification accuracy for GSE39754 improved from 0.876 to 0.913 (147/161). This is due to that in the 9 unclassified samples with confidence score <0.9, 7 were misclassified previously. For GSE6401, If we only classified samples whose confidence score is more than 0.9, the accuracy increased from 0.88 (66/75) to 0.93 (62/67).www.ChengLiLab.org/software.Multiple myeloma is a malignant cancer of bone marrow plasma cells. Researchers are making progress in diagnosing and treating multiple myeloma in innovative ways; however, it cannot be fully cured by current treatment regimes. The two main aneuploidy-based subtypes of multiple myeloma, HMM and NHMM, correspond to different survival prognosis and potential underlying different pathogenic pathways and altered regulatory networks. Since most expression profiling studies of MM do not measure HMM status, we have developed an expression-based method to classify HMM status of patients with high accuracy across datasets and array platforms. Compared with FISH and copy number based HMM status, the test accuracy in three independent datasets ranges from 0.83 and 0.88. Our method opens up the opportunity for meta-analysis of many MM expression datasets for the disease biology of HMM and NHMM subtypes, and for incorporating HMM status as a covariate in genomics-based survival prognosis models. The lessons we learned in this study, such as using cancer specific characteristics in feature design and pooling multiple rounds of classification results to improve accuracy, are applicable to similar genomics-based classification. We also provide R package and processed datasets publicly at Inferring chromosome abnormalities using gene expression profiles can help predict clinical outcomes and identify causal genomic alterations Importantly, we found that the carefully selected gene features considering trisomy and non-trisomy chromosomes resulted in more classification power than using the whole list of differentially expressed genes between the two subtypes. We decided that the adjusted TC mean and NTC mean are good features for cross-dataset KNN classification . The claAlthough we developed an effective method to classify multiple myeloma subtypes, we did not achieve 100% accuracy. There are both biological causes and analytical causes for misclassified samples. Biologically, multiple myeloma genome is not strictly altered following main HMM and NHMM characteristics. Some MM samples could have both trisomy chromosomes and deletion of chromosome 13 , renderiAnother biological cause is inaccuracy of FISH-based HMM status, which may probe only a few chromosomes to determine hyperdiploidy. A hypothetical example is a tetraploid sample with 4 copies for most chromosomes. Its transcriptomic profiles may be similar to NHMM due to equal copy numbers of most genes, but FISH could report it as hyperdiploidy.Analytically, as we used all sample points to adjust the center of a dataset's sample points in the NTC mean vs. TC mean plot , the proWe have also compared to two other classification methods, support vector machine (SVM) and weighted k-nearest neighbors (KKNN). We set GSE6477 as training data and GSE6401 as test data in the same way as the original KNN classification, and k = 16 is used for both KNN and KKNN. SVM classifies only one more sample correctly than KNN (the accuracy increases from 0.88 to 0.89), and KKNN achieves the same accuracy as KNN. Due to potential better performance of KKNN for unbalanced datasets, we removed various number of HMM or NHMM samples from GSE6401 to create unbalanced datasets, and used KKNN to classify HMM status. Computing a confidence score by pooling results of multiple rounds of classification could improve the accuracy, if samples with low confidence score tend to be misclassified. In our analysis of GSE39754 and GSE6401, not classifying samples with confidence score <0.9 improved classification accuracy from 0.876 to 0.913 and from 0.88 to 0.93, respectively. In the TC mean vs. NTC mean plots , most saTable S1The 675 genes that are intersection gene features among the 140 leave-one-out models of the training dataset GSE6477. Chro Type: TC: Trisomy chromosomes, NTC: Non-trisomy chromosomes. FDR: the median FDR of all comparisons in the 140 models. HMM.Mean and NHMM.Mean are the mean value of the scaled gene expression in HMM and NHMM samples, respectively.(XLS)Click here for additional data file.


The suggested approach makes it feasible to screen large metabolomics data, sample sets with retained data quality or to retrieve significant metabolic information from small sample sets that can be verified over multiple studies. Hierarchical multivariate curve resolution (H-MCR), followed by orthogonal partial least squares discriminant analysis (OPLS-DA) was used for processing and classification of gas chromatography/time of flight mass spectrometry (GC/TOFMS) data characterizing human serum samples collected in a study of strenuous physical exercise. The efficiency of predictive H-MCR processing of representative sample subsets, selected by chemometric approaches, for generating high quality data was proven. Extensive model validation by means of cross-validation and external predictions verified the robustness of the extracted metabolite patterns in the data. Comparisons of extracted metabolite patterns between models emphasized the reliability of the methodology in a biological information context. Furthermore, the high predictive power in longitudinal data provided proof for the potential use in clinical diagnosis. Finally, the predictive metabolite pattern was interpreted physiologically, highlighting the biological relevance of the diagnostic pattern.  CVCross validation or Cross validatedGC/TOFMSGas chromatography time of flight mass spectrometryH-MCRHierarchical multivariate curve resolutionMCRMultivariate curve resolutionOPLSOrthogonal projections to latent structuresOPLS-DAOrthogonal projections to latent structures discriminant analysisOSCOrthogonal signal correctionPCAPrincipal component analysisPLSPartial least squares projections to latent structuresUPLC/MSUltra performance liquid chromatography mass spectrometryOne issue of major concern for the progression of metabolomics ,2 is effi.e., a reference table of putative metabolites in the analyzed samples. In that way, multiple sample comparisons and biomarker or biomarker pattern extraction can be efficiently carried out by means of multivariate data analysis. Verification of the findings in independent sample sets, in the case of one or a few detected selective biomarkers, could be carried out by setting up biological assays for the detected metabolites. However, in the case of using a metabolite pattern or profile as the indicator of a specific physiological state, the data processing and analysis must work in a predictive way so that this pattern or profile can be verified in new samples, and thus work as a diagnostic tool. ‘Predictive’ in this case means a processing algorithm that can efficiently detect and quantify metabolites in the generated reference table in independently analyzed samples. Mining of sample banks is becoming increasingly important in trying to understand the complex biological interactions behind, or finding diagnostic or prognostic biomarkers for, various disease states. Usually, these sample banks contain a large number of human samples collected continuously over a long period of time, often extremely well-characterized in terms of property data (metadata). Samples of this type are very attractive for research purposes. However, a problem is that regulations regarding availability, for obvious and valid reasons, are very strict, and also, sample volumes might be limited for specific applications. For this reason, it will be of high relevance to be able to select a representative subset of samples for analysis and method development. A way to address this would be to select samples based on the metadata characterization to make sure to create a sample set consisting of samples relevant for the question. The selected sample set could then be characterized using an appropriate analytical technique, and the acquired data processed to obtain a reliable quantification and identification of detected metabolites, i.e., generation of a reference table of putative metabolites. Again, a key here is for the sophisticated processing to work predictively for new samples. If this is the case, then the samples not selected for processing, as well as additional samples measured at a later point in time, can be predictively processed to detect and quantify the metabolites in the reference table. GC/MS has proven to be a valuable tool for the global detection of metabolites in biofluids and tissues [To obtain an efficient screening of large sample sets where the aim is to acquire data for all samples, the key issue will be the data processing step. A sophisticated processing of GC/MS data, such as curve resolution, ,15,16 is tissues ,18,19,20 tissues , allows  tissues . By comb tissues ,24,25. T tissues , ChromaT tissues , and ADA tissues , H-MCR pIn this paper, we show that by selecting representative sample subsets to generate a reference table with reliably quantified and identified metabolites, by means of H-MCR, and performing multivariate regression analysis, using orthogonal projections to latent structures discriminant analysis (OPLS-DA),30, an ei.e., generation of a reference table of putative metabolites in the analyzed samples. The generated reference table was used for OPLS-DA classification modeling of the systematic metabolic variation related to the acute effect of strenuous exercise. Three samples were excluded prior to data processing and analysis due to insufficient derivatization, giving a total of 93 samples for further investigations. The H-MCR processing of the 93 samples resulted in reference table containing 167 resolved metabolites. The area under the resolved chromatographic profiles was used as sample descriptors in multiple sample comparison between samples taken pre- and post- exercise. The cross validated OPLS-DA score plot revealing the separation between pre- and post- exercise samples is shown in , and the resolved metabolite profiles responsible for the separation are shown in the corresponding covariance loading plot . The resulting data were subjected to H-MCR processing to obtain a reliable quantification and identification of detected metabolites, ing plot b. A geneA total of 34 metadata variables characterizing the 24 male subjects included in the study were subjected to PCA analysis. Two principal components were extracted describing 78.8% of the variation in the data (R2X = 0.788), and the resulting score vectors were used for a diversity-based selection of a representative sample subset. Six subjects were selected creating a subset of twelve samples  that spanned the score space maximally. The acquired GC/TOFMS data for the selected subset was subjected to H-MCR processing resulting in a reference table containing 233 resolved putative metabolites. The area under the resolved chromatographic profiles were used as sample descriptors in OPLS-DA analysis for multiple sample comparison that revealed a clear separation between pre- and post- exercise samples in relation to metabolic composition. The metabolic information content in the subset was compared to the information content in the total dataset . The percentage of shared spectral metabolite profiles in the two reference tables was 87.4% (146/167). The percentage of shared metabolite profiles significantly separating pre- and post- exercise samples between the subset and the total data set, identified by a permutation test, was 94.1% (32/34). In addition, the remaining samples from test occasions one and two were predictively processed to detect and quantify the metabolites in the reference table, followed by predictive classification into the OPLS-DA model. This resulted in a cross-validated classification accuracy for the model samples (n = 24) of 100% (Class prediction (CV)) and a predictive classification accuracy of 97.1% (Class prediction (Test Set)) for the test samples (n = 69). The representative subset selection was evaluated by repeating the procedure above for three additional selections, where each subject was included in one subset only. The results are presented in the supporting information . Human serum GC/TOFMS data of the 93 samples from exercise occasions one and two were processed using a fast processing method called hierarchical data compression . The 230The metabolic information content in the model samples was compared to the information in the total dataset . The percentage of shared spectral metabolite profiles was 82.6% (138/167). The percentage of shared mass spectral metabolite profiles significantly separating pre- and post- exercise samples between the subset and the total data set, identified by the permutation test, was 88.2% (30/34). In addition, the remaining samples from test occasions one and two were predictively processed to detect and quantify the metabolites in the reference table, followed by predictive classification into the OPLS-DA model. This resulted in a cross-validated classification accuracy for the model samples (n=16) of 93.8% (Class prediction (CV)) and a predictive classification accuracy of 96.1% (Class prediction (Test Set)) for the test samples (n=77) . The timIn order to compare the predictive ability of the models generated by the two subset selection strategies, we formed a test set including the samples that were outside both selections. The test set, including 57 samples (29 pre- exercise (0) and 28 post- exercise (1)), were used to show the differences/similarities in prediction power for the two different models  . Samples from two additional exercise sessions (referred to as exercise occasions three and four) that were analytically characterized eight months later compared to the model samples were predictively processed to detect and quantify the metabolites in the reference tables. The updated OPLS-DA models based on significantly separating metabolic marker patterns, extracted using permutation tests, showed an evident separation between the samples taken pre- and post- exercise, in addition to a high predictive ability of the longitudinal samples (n = 64). This is shown for the OPLS-DA model based on the subset selected from metadata , the subThe results highlight that, by selecting a representative subset of samples, the metabolic information from, for example, a sample bank can be extracted and evaluated in a reliable fashion. Also, the predictive features of the strategy made it possible to process and classify new samples based on the information extracted from the selected subsets. This was shown by the fact that the H-MCR processing resolved a similar number of profiles (n = 206–233) for each of the four subsets selected based on the variation in metadata. OPLS-DA models based on each individual subset gave similar classification results, both in terms of cross validation (91.3%–100%) and predictive classification of the samples from the other three subsets (93%–97.1%). In addition, comparison of the results for the subset models to the results from when all samples were processed and modeled together showed that the subset models contained the same metabolic information as the model based on all samplesThe method’s ability for efficient processing and classification of large data sets by selecting representative subsets was exemplified by the results based on the 16 samples selected from already acquired GC/MS data. In this case the predictive H-MCR processing and OPLS-DA classification was applied to the remaining samples in order to allow a high-throughput processing of many samples with retained data quality for interpretation and biomarker pattern identification. By using the selected subset to create a reference table of metabolites and predictively processed remaining samples to detect and quantify the metabolites in the reference table, an efficient strategy for screening of large data sets for producing representative and high quality data was offered. The proof for this was given by the prediction results for the OPLS model based on the subset, which correctly classified 96.1% of the remaining samples. Investigation of the metabolic information content revealed that 138 out of 167 metabolite profiles and 30 out of 34 metabolite profiles significantly discriminating the sample was detected in the reference table as compared to the data where all samples were resolved. This indicates that a small subset selected based on acquired GC/MS data, if done in a systematic fashion, will efficiently retain the variation in the original data. Here, the importance of a feasible sample selection approach that retains the systematic variation in the data must be highlighted. In high complexity data, such as the data presented here, this can be achieved by chemometric approaches, where selections are made in the compressed multivariate space, as offered by PCA for example. These results also highlight an interesting and important issue, namely that by selecting a representative subset of samples, significant metabolic information can be retrieved from a small number of samples, information by the presented method that can be predictively verified in follow up studies. This is important since it is not reasonable to believe that all future studies of value should include thousands of samples. Instead, there could be value in detecting potentially relevant information in small, well designed studies. The benefits of this will be many, including an efficient use of biobank samples, as well as better possibilities for maintaining a high analytical data quality, which is a major problem when analyzing large sample sets over longer times with mass spectrometry.It is also worth noting that the same strategy as above, namely selecting representative sample subsets from acquired GC/MS data and utilizing the predictive feature of the H-MCR method, was used as an internal cross-validation procedure for the H-MCR curve resolution as a means to obtain a robust and reliable metabolite pattern on which to base further modeling and interpretations. Thus, the whole chain of events, including multivariate curve resolution and sample classification, is subject to an internal cross-validation procedure, as well as providing the possibility for prediction of new independent samples, which to our knowledge makes the proposed strategy unique.Of high importance for an efficient screening of large sample sets is the time and feasibility for producing representative data. Curve resolution methods are, in general, very time-consuming and demanding in terms of computer capacity, which limits the number of samples that can actually be processed. However, the predictive feature of the H-MCR method can resolve this issue. In the given example, H-MCR processing of the 16 samples took 6h and 29 min to resolve, while predictive processing of the remaining 77 samples took <10sec/sample. This indicates that as long as the selected subset is representative in terms, retained variation large sample quantities can be efficiently processed, providing data of high quality for sample comparisons, biomarker detection and identification, and predictions. It should be pointed out that the number of samples used in this example cannot be considered as a particularly large sample or data set. However, the point was proven that samples could be efficiently processed, with retained data quality, based on a selected set of representative samples, and as far as the method goes, there is no limitation to the number of samples that can be predictively processed in the same way as shown here. This high-throughput property of the methodology makes it interesting as a contribution to the recently presented metabolome-wide association studies (MWAS), where the aim is to analyze several thousands of samples in the search for metabolic phenotypes that can aid in, for example, disease treatment or prevention ,34. MWASThe ultimate goal for an efficient screening method in terms of a diagnostic is to perform in a robust and reliable fashion over time. We believe that the presented approach has the ability do so, however this is still something that needs to, and will be, evaluated in much more detail. As proof of the capability of the method in terms of providing correct classifications over time, predictions were made of samples from two exercise sessions that were analytically characterized eight months later compared to the model samples. Predictive processing and classification was carried out using all subset models, as well as all model samples. The results showed that, irrespective of subset model, a high classification accuracy was obtained for the new samples, and this accuracy was comparable to the one obtained when using all model samples for carrying out the predictions. The interpretation of this is that the method is efficient in carrying out longitudinal predictions based on a biomarker pattern extracted eight months earlier in time. Hence, we can conclude that the proposed method is showing promising results as a means for predictive screening in terms of biomarker pattern verification and diagnosis.Although the presented strategy is promising, there are still challenges ahead in order to reach the stage where a complete and robust method for screening and predictions over time is in place. For instance, to further evaluate the strategy, it would be of value to perform a separate study applying the same test to a completely independent study population and make predictions for these subjects into the existing model. Also, this study was carried out on a homogenous human population, which is not representative for the whole human population, which is obviously something that needs to be considered, for example, in disease diagnosis modeling. One extremely important factor for these types of approaches to be successful will always be properly designed studies carried out according to a standardized protocol under as standardized conditions as possible, which is the case for the presented study. Dealing with human subjects, and that is eventually where many research projects are aiming today, there is a need to minimize confounding variation, or at least to be able to detect and handle this variation. In this, the chemometric methodology in terms of design of experiments and multivariate projections can bring a valuable contribution together with experience and knowledge related to the study itself . Other iInterpreting the metabolite pattern reveals that the apparent increase in fatty acids in blood serum following exercise could be expected and does reflect an increased lipolysis and release of fatty acids from the adipose tissue. This is stimulated by catecholamines and other stress-induced hormones during exercise ,41. It i2peak at 59.1 ± 7.3mL kg−1min −1) volunteered to participate in the study. The study was approved by the regional ethical committee (Dnr 05–069M). Each subject participated in a pre-experimental VO2peak test and four identical experimental tests performed one week apart. Samples from two test occasions were used in the primary analysis, while samples from the remaining two test occasions were used for longitudinal predictions and for that reason, characterized analytically by GC/TOFMS eight months later. The dataset included in total 160 samples, i.e., 96 samples used in the primary analysis (24 subjects at two occasions and two time points) and 64 additional samples characterized eight months later . The data have been previously used for evaluating physiological variation related to the acute effect of strenuous exercise [24 healthy and regularly training male subjects  [The included subjects performed a pre-experiment incremental test on an ergometer cycle (Monark 839E) to exhaustion in order to determine the maximum oxygen uptake as a mean of 60 seconds (VO2peak) . At the ®2, Medex) in a forearm vein, a transmitter belt (Polar WearLinkTM31) and a heart frequency monitor Polar S610iTM). Subjects then performed 90 min of ergometer cycling, using an electronically braked bicycle . Each 90 min test session consisted of nine equal 10 minutes sections. The workloads during the sections were loads that corresponded to 40% (2 min), 60% (6 min) and 85% (2 min) of the VO2peak value from the pre-experimental test. 100ml of water was ingested after every 10 min of cycling. Immediately after 90 min completed cycling, blood was collected from the vein catheter into vacutainer tubes. Serum was extracted from the collected blood samples following 8 min centrifugation (+4 °C at 3000g) and immediately frozen and stored in -80 °C. Prior to GC/TOFMS analysis, the serum samples were extracted and derivatized according to A et al. [µm DB 5-MS stationary phase . The column effluent was introduced into the ion source of a Pegasus III time-of-flight mass spectrometer, GC/TOFMS . Detailed description of the pre-experimental procedures, blood sampling, sample preparation, derivatization and GC/TOFMS protocol are found in the supporting text.Venous blood samples were taken after 15 min of bed rest by using a vacutainer system . Thereafter, subjects were equipped with an intravenous catheter  to base the selection on metadata and (2) to base the selection on already acquired analytical data (GC/TOFMS). The selection was based on the systematic variation captured in the meta- or analytical data by principal component analysis (PCA). Each sample subsets was selected so that the systematic variation in the original set was maintained in the best possible way ,55,56.2peak, load at different percentage of VO2peak, serum glucose and hemoglobin levels .GC/TOFMS data files were exported to MATLAB software , where all data processing procedures and the space-filling design were performed using in-house scripts. The multivariate analysis was carried out in the SIMCA-P+ software . NIST MS Search 2.0  was used for compound identification based on comparison between resolved spectra and standard spectra from NIST 08 mass spectra library, in-house mass spectra library or the MaxPlanck Institute mass spectra library (Multivariate Curve Resolution (MCR)  is a metDue to the size and complexity of metabolic data, MCR decomposition of data into spectral and chromatographic profiles cannot be done for the complete data set simultaneously. To cope with this the data is divided into smaller parts. This is done in by splitting the data in the chromatographic dimension into a set of time window. Prior to this division, the samples are aligned in the chromatographic dimension. Each time window is then resolved separately using MCR. This procedure is called H-MCR . For newsetA, CsetA, SsetB and CsetB are obtained. Samples in set 1 are then predicatively resolved using SsetB to get CsetA_pred and set 2 are then predictively resolved using SsetA to get CsetB_pred. By comparing the similarity between SsetA and SsetB, CsetA and CsetA_pred and CsetB and CsetB_pred, respectively, it is possible to identify the profiles that are stable across samples. Here we use Pearson correlation above 0.95 as the criterion for stability. Only profiles that meet this criterion for all comparisons are used. In this way a reference table consisting of verified and stable spectral profiles is created. To ensure that the reference list is created using representative samples, the selection of “set A” and “set B” are made independently for each time window using principal component analysis (PCA) and space-filling design [This predictive feature of MCR also made it possible to integrate an internal validation step in the processing. By dividing the samples to be resolved into two sets and performing independent resolution of the two sets, interchanging Sg design , where evs. post- exercise) and predicting independent samples with known phase into existing models. Data were mean-centered and scaled to unit variance prior to modeling, and the number of significant OPLS-DA components was decided by seven-fold full cross validation [Prior to multivariate analysis, the data of all putative metabolites (integrated areas under the metabolites chromatographic profiles) were normalized using the weighted sum of the concentrations of 11 labeled internal standards  and the binary class identity variable , respectively, explained by the extracted OPLS-DA components and Q2 corresponds to the fraction of the total variation in y predicted by the model, according to 7-fold full cross-validation .i.e., variables with elevated OPLS weight values , compared to the permuted y models were extracted, and the percentage of significantly separating metabolite profiles shared between the entire dataset and each subset was calculated. The metabolic information in the sample subsets was compared to the information present in the entire sample set by matching of resolved metabolite profiles. The reference table from the H-MCR processing of the entire sample set was compared to the attained reference table for the subsets and the spectral similarity was decided by comparing retention time and the match factor obtained in NIST MS Search 2.0 . The factors range from 999 for a perfect match to zero for spectra having no peaks in common. Resolved mass spectral profiles were considered to be equivalent if the match factor was above 700 and the retention times differed less then ± 1 second. Subsequently, the percentage of the overall shared resolved spectral profiles in the reference tables was calculated. The metabolic information in the processed data was further assessed by extracting metabolite profiles that significantly separated the two exercise states (pre- or post- exercise) by a permutation test. In the permutation test, the y-vector (in this case a vector containing information about class identity (pre- or post- exercise)) was permuted randomly 10 000 times, and for every permutation, a OPLS model  was creaThe predictive ability of the multivariate models was investigated by the number of model samples that was correctly classified according to seven-fold cross validation (CV) (Class Prediction (CV)), as well as the number of independent samples (Test Set) predicted into the right class by the OPLS-DA model (Class Prediction (Test Set)). Samples in the Test Set are predictive both in the case of the resolving of metabolites H-MCR and the OPLS-DA classification.i.e., its potential as a diagnostic tool. Exercise occasions three and four were performed by the same male subjects in conjunction with the other tests, but the samples were characterized analytically by GC/TOFMS eight months later. The idea here was to use each subset to extract a significantly separating metabolic marker pattern using the previously described permutation test and verify the pattern by means of classification of the samples from the additional exercise occasions. This was done by first assigning a predicted class membership (pre- or post- exercise) to each sample from exercise occasions one and two, which was used to update the existing models. The samples from exercise occasions three and four were then predictively resolved, using the reference table for the model samples, and classified by prediction into the new OPLS-DA models based on the significantly separating metabolic marker patterns. Additional samples from exercise occasions three and four (n = 64) were used to investigate the methods ability as a means for predictively verifying a detected metabolic marker pattern in longitudinal studies, We show that by using chemometric strategies for selecting representative sample subsets, H-MCR curve resolution and multivariate classification can be used to efficiently screen large metabolomics data or sample sets with retained data quality, or to retrieve significant metabolic information from smaller sample sets that can be verified over multiple studies.


MicroRNAs (miRNAs) are short non-coding RNA molecules that regulate mRNA transcript levels and translation. Deregulation of microRNAs is indicated in a number of diseases and microRNAs are seen as a promising target for biomarker identification and drug development. miRNA expression is commonly measured by microarray or real-time polymerase chain reaction (RT-PCR). The findings of RT-PCR data are highly dependent on the normalization techniques used during preprocessing of the Cycle Threshold readings from RT-PCR. Some of the commonly used endogenous controls themselves have been discovered to be differentially expressed in various conditions such as cancer, making them inappropriate internal controls.We demonstrate that RT-PCR data contains a systematic bias resulting in large variations in the Cycle Threshold (CT) values of the low-abundant miRNA samples. We propose a new data normalization method that considers all available microRNAs as endogenous controls. A weighted normalization approach is utilized to allow contribution from all microRNAs, weighted by their empirical stability.The systematic bias in RT-PCR data is illustrated on a microRNA dataset obtained from primary cutaneous melanocytic neoplasms. We show that through a single control parameter, this method is able to emulate other commonly used normalization methods and thus provides a more general approach. We explore the consistency of RT-PCR expression data with microarray expression by utilizing a dataset where both RT-PCR and microarray profiling data is available for the same miRNA samples.A weighted normalization method allows the contribution of all of the miRNAs, whether they are highly abundant or have low expression levels. Our findings further suggest that the normalization of a particular miRNA should rely on only miRNAs that have comparable expression levels. MicroRNAs (miRNAs) are short non-coding RNA sequences that average 22 nucleotides in length -3. This miRNAs have been discovered to play a role in many diseases and pathologies ,13,15,160CT. The ΔΔCT is calculated by subtracting the ΔCT of an experimental sample from a control sample. Fold change is calculated by raising 2 to the power of the negative ΔΔCT value, since CT values are related to the amount of miRNA or gene logarithmically [There are two main tools used to quantify the expression of miRNAs: microarrays and real-time polymerase chain reaction (RT-PCR). RT-PCR returns the number of cycles that the samples underwent before they were detected, reported as a value known as the Cycle Threshold (CT). The CT values vary logarithmically with expression levels. There are several methods of normalizing the data and calculating the fold-change of each gene between samples. For convenience, in this presentation the terms, "miRNA" and "gene," are used interchangeably in the context of RT-PCR. ΔCT values are calculated by subtracting the CT value of the endogenous control for a given sample  from the CT value of the gene for the given sample. In the calculation of ΔCT values we refer to the number subtracted from the raw CT values of each gene as the hmically . The relTheoretically, endogenous controls are selected because they have low variance in their expression levels across samples. In the case of miRNAs, the endogenous controls are typically recommended by the manufacturer of the miRNA kit used in the PCR. Some of the most commonly used endogenous controls are RNU44, RNU48, and U6 . HoweverDirectly applying this method can lead to misleading results if the CT values in the data are not normalized. There are several commonly used methods for miRNA normalization, including: quantile normalization, median normalization, and cyclic loess. Quantile normalization involves sorting the expression values of each gene in a given sample in order from least to greatest. This is done for each sample in the study. The vectors of the sorted CT values for each sample are combined into a matrix. The mean of each row of the matrix is calculated. The CT value in each element in each row is replaced with the mean of the entire row. In the case of median quantile normalization the median of the row is used instead of the mean. The CT values in each sample are then rearranged back into their original order. This causes the distribution of CT values across all samples to assume the same shape, which will minimize the variance except for that resulting from the experimental condition beings studied ,22.Median normalization shifts the CT values in each sample such that the median CT value of each sample is the same. The median of each plate should be determined, and the medians of all plates should be arranged in a vector and sorted to determine the median of the medians. In each plate the difference between the median of the sample and the overall median should be subtracted from the CT value of each gene .M, and the average of each gene of the log of the expression values is represented by A. Then a loess curve is fit by regression of M on A which results in a fitting vector F. The genes in the first sample are adjusted by adding half the F value corresponding to the log of the CT for each gene. In the second sample half the F value is subtracted from the log CT of the gene [In cyclic loess normalization, pairs of plates are considered. For all pairs of plates the difference of the log of the CT for each gene is represented by the gene ,21.A number of normalization methods developed for microarrays have been applied to RT-PCR experiments. These methods assume that all miRNAs present in the organism are being profiled in the experiment. While microarrays can profile all miRNAs encoded in a genome, this assumption does not hold for RT-PCR experiments which typically only profile a few hundred miRNAs at a given time . Mar et One of the main problems with RT-PCR that remains as yet unaddressed by current normalization methods is the systematic bias present within the data. We observe that standard deviation increases as CT values increase. We believe that the most likely cause of this observation is the assumption that the PCR magnification at each cycle is an exact doubling of the expression levels is inaccurate. There seems to be an accumulation of an expression-level specific rate-limiting effect. As a result, a small difference in the size of the initial sample being amplified causes larger variations in the CT values of the less abundant microRNA molecules. Consequently, using endogenous controls, which are usually chosen from highly expressed microRNAs, for normalization becomes inappropriate for the less-abundant microRNAs. Even quantile normalization has been observed to produce more variance at high CT values than was present in the original raw data . One potThe primary dataset used in this study was obtained from a recently deposited microRNA RT-PCR dataset in the Gene Expression Omnibus (GEO) . The dat0CT. For identification of endogenous controls, we calculate the standard deviation of each microRNA across all samples, and rank them in the order of increasing standard deviation. The CT values of the top-k microRNAs are averaged in each sample to provide the 0 CTvalues.We have investigated several normalization methods, including quantile, mean, and median normalization methods, and endogenous controls identified using various stability criteria. In mean and median normalization, the mean and median of all of the genes in a given sample are used as the value for A new weighted mean metric is proposed using the standard deviations of the microRNAs as weights. For a given gene, the weighted average is calculated using the following equation:wmp is the weighted mean power, which can be adjusted to shift the dominance between stable and unstable microRNAs, n is the number of genes or microRNAs, and STD is a function that returns the standard deviation. The weighted mean calculation involves raising the inverse of the standard deviation of a given gene across all samples to the weighted mean power, which is usually specified as 1, and dividing by the sum of the inverses of the standard deviations for all genes. 0 CTis calculated for each sample by taking the sum of the product of all the raw CT values in the sample and the previous number. When the ΔCT is calculated the CT of each gene is subtracted by the above value. This method gives a higher weight to genes with a lower standard deviation.where We also examined the reproducibility of miRNA expression experiments between RT-PCR and microarray. To explore this topic we utilized data from Chen et al. . They evIn order to test the hypothesis that increasing CT values magnifies the natural variation between the initial amounts of samples loaded in each well during RT-PCR, we examined the standard deviation of the genes against their mean CT values, as shown in Figure As expected, the CT values of most genes are well correlated with the mean expression of all the genes. This is illustrated in Figure The correlation with the mean expression level extends to low-abundant miRNAs. We demonstrate this in Figure In order to quantify the sensitivity of the microRNA expression levels to the initial loaded sample size, a regression line is fitted to the fluctuation of each miRNA against the fluctuation of mean expression. Fluctuation is determined by subtracting the value of the expression of a miRNA in a given sample from the mean expression of that miRNA for all samples; the overall mean of the expression of all miRNAs in all samples can be subtracted from the sample means to determine the fluctuation of a sample mean. In Figure In conclusion, the fluctuations of the low-abundant miRNAs are not random. The changes in their expression levels are correlated well with the overall changes in all miRNAs, which is assumed to be due to different starting sample sizes for the PCR reactions. We see that there is a systematic bias in the CT values that causes the expression levels of the low-abundant miRNAs to be more sensitive to the initial sample sizes.0 CTfor several different methods including using the mean of all raw CT values in the bottom line (top-k = 0), the means of the top-k miRNAs for different values of k, and the weighted mean for different values for the weighted mean power. The plot demonstrates that varying the weighted mean power enables the shifting of the curve upwards or downwards. In Table k calculations, but the weighted means' 0 CTvalues are determined to be more stable by geNorm . In Table 0 CTcalculations using either the top-k miRNAs or the weighted mean. This indicates that it is better to use these values in the ΔΔCT calculation than any endogenous control. We also investigated the use of the geometric mean of the endogenous controls, all of the miRNAs, or just the top-k miRNAs. In all cases using the geometric mean resulted in similar but slightly lower values for CT0 as using the mean and almost exactly the same standard deviations and geNorm stability values (data not shown), thus using the geometric mean had no advantage over using the mean.We then investigated the suitability of our weighted mean metric. In Figure 0 which could be evaluated for stability and compared to weighted mean normalization. However, the normalized data resulting from each method could be visualized and compared as boxplots. Figure The proposed weighted mean normalization method could not be compared to quantile normalization in the same fashion as the other methods because quantile normalization does not have a value analogous to CTHaving explored the problems of RT-PCR normalization, the consistency of miRNA expression experiments between RT-PCR and microarray technologies was of further interest to us. In order to explore this issue we used a dataset from Chen et al. , to explFigure We explored the phenomenon whereby differences in the initial sample size of miRNA in an RT-PCR experiment were magnified with increasing CT levels. This was illustrated by the strong correlation of the CT values of the individual miRNAs with the average CT values of all miRNAs and by the increased sensitivity in the CT values of the low-abundant miRNAs to the average CT values. We conclude that a systematic bias in RT-PCR exists in which the fluctuations in the CT are dependent on the expression levels of the particular miRNAs. We further proposed a novel data-driven method of addressing this bias by using the weighted mean instead of an endogenous control in the calculation of ΔCT. We demonstrated that the new normalization method produces lower standard deviations and is more stable than other methods.0 values can be calculated for different values for the weighted mean power, subsequently the value of the power that produces the lowest standard deviation or is determined to be the most stable by geNorm can be used for normalization. The standard deviation or geNorm stability calculations are two methods to quantitatively determine the ideal weighted mean power. Other criteria, such as significance of the differentially expressed microRNAs can be utilized in this optimization. Furthermore, a different custom 0 CTvalue for each microRNA may be used, such that each microRNA is normalized differently, dependent on its average expression level.Note that, while the power parameter in the weighted mean normalization method provides a convenient way of adjusting how much one wishes to let the less stable microRNAs influence the normalization of other microRNAs, its optimization currently requires enumeration of different values and using the one with the best overall stability. Several CTWe further examined the reproducibility of miRNA expression experiments across two different platforms by comparing RT-PCR and microarray results. We explored the relationship between the CT value and the consistency of the expression of a miRNA between RT-PCR and microarray. We leave as a future work the comparison of the ability of different normalization methods to detect differentially expressed genes.The authors declare that they have no competing interests.AS conceived of the study and coordinated the project. RQ implemented the method and performed the experiments. RQ contributed to the design and testing of the method. All authors participated in the analysis of the results. RQ and AS contributed to the writing of the manuscript. All authors read and approved of the final draft.