Mouse embryonic stem cells (mESCs) are derived from the inner cell mass of a developing blastocyst and can be cultured indefinitely in-vitro. Their distinct features are their ability to self-renew and to differentiate to all adult cell types. Genes that maintain mESCs self-renewal and pluripotency identity are of interest to stem cell biologists. Although significant steps have been made toward the identification and characterization of such genes, the list is still incomplete and controversial. For example, the overlap among candidate self-renewal and pluripotency genes across different RNAi screens is surprisingly small. Meanwhile, machine learning approaches have been used to analyze multi-dimensional experimental data and integrate results from many studies, yet they have not been applied to specifically tackle the task of predicting and classifying self-renewal and pluripotency gene membership.For this study we developed a classifier, a supervised machine learning framework for predicting self-renewal and pluripotency mESCs stemness membership genes (MSMG) using support vector machines (SVM). The data used to train the classifier was derived from mESCs-related studies using mRNA microarrays, measuring gene expression in various stages of early differentiation, as well as ChIP-seq studies applied to mESCs profiling genome-wide binding of key transcription factors, such as Nanog, Oct4, and Sox2, to the regulatory regions of other genes. Comparison to other classification methods using the leave-one-out cross-validation method was employed to evaluate the accuracy and generality of the classification. Finally, two sets of candidate genes from genome-wide RNA interference screens are used to test the generality and potential application of the classifier.Our results reveal that an SVM approach can be useful for prioritizing genes for functional validation experiments and complement the analyses of high-throughput profiling experimental data in stem cell research. Mouse embryonic stem cells (mESCs) are derived from the inner cell mass of a developing blastocyst and can be cultured indefinitely in-vitro. Their distinct features are their ability to self-renewal as well as to differentiate into all adult cell types including the germ-line. These features render mESCs ideal for applications in basic scientific research and translational medicine. To harness their full potential, better understanding of the molecular mechanisms of mESCs self-renewal maintenance and pluripotency is critical. Therefore, genes that are critical to mESCs self-renewal maintenance are of interest to the stem cell research field. In the past decade, significant steps have been made toward identifying and characterizing the genes and regulatory networks that compose the self-renewal machinery. A mESCs stemness membership gene (MSMG) signature has been proposed through application of high-throughput profiling approaches such as mRNA expression microarrays combined with advanced computational analyses as well as through low-throughput detailed functional studies -3. GenesFuelled by the growing volume, diversity and complexity of genome-wide profiling data generated from high-throughput biotechnologies, advanced computational approaches such as machine learning have been used to analyze multi-dimensional experimental data and integrate results from many studies -10. SuppWe hypothesized that we can utilize data from mESCs-related mRNA microarrays profiling and genome-wide transcription factor binding profiling (ChIP-seq) applied to characterize mESCs to classify genes important for ES cell self-renewal and pluripotency (MSMGs). We believe that within these datasets there are subtle patterns from which a gene's functional characteristic, in regards to the self-renewal and pluripotency involvement, Yes or No question, can be inferred. We employed an SVM-based approach to construct a classifier that can be used to predict the class membership as being MSMG or not-MSMG for genes by combining genome-wide mRNA expression profiling data and ChIP-seq data. The accuracy and generality of the classifier are evaluated using the leave-one-out-cross-validation (LOOCV) approach. We also compared the SVM classifier with other machine learning classification methods, including linear discriminant classifier, decision trees, and artificial neural networks. Furthermore, we tested the ability of the SVM classifiers to predict the class membership of positive and negative lists of genes resulting from two genome-wide RNAi screen studies to demonstrate how such classification approach can be useful for helping in prioritizing hits from such screens.We extracted 91 features/attributes from mRNA gene expression and ChIP-seq experiments for each gene (vector) from mESCs-related studies (detailed description is provided in the Methods section). 79 features/attributes were created from mRNA expression microarray profiling data extracted from the Gene Expression Omnibus (GEO) database  referencij ais the binding value of the transcription factor j on gene i, k gis the intensity of the kth binding peak of transcription factor j, dk is the distance between the TSS of gene i and the kth binding peak, and d0 is a constant. ij ais then log-transformed and quantile-normalized. This method was previously introduced by Ouyang et al.  ,31. The This method for preprocessing microarray data for SVM training was borrowed from Brown et al. .The SVM classifiers we implemented to map the data from the input space to a high-dimensional space in which classification can be performed by locating data points with respect to a hyperplane that separates binary classes. The feature space can be adjusted by selecting a kernel function, which is used to transform the data for optimization of the classification ,19. In t(1) Linear kernel:(2) Polynomial kernel:(3) Gaussian radial basis kernel-2 to 101), using LOOCV, to select the best σ value that maximizes the prediction accuracy.It is common practice to set σ to be equal to the median of the Euclidean distances from each positive sample vector to its closest negative sample vector , we noteWe integrated the two data types, mRNA expression and ChIP-seq, simply by concatenation. We then trained the SVM classifier using one of three kernel functions. Alternatively, motivated by the hypothesis that the classification would be better for treating each data type separately, we employed a strategy of integrating two kernel functions each applied to one of the two different data types. The weights for each classifier are determined by an F1 score.F1 score is a measure of accuracy that takes into accounts both precision and recall.As a result, we can obtain a classifier that is made of two weighted kernels.m Kand c Krepresent the kernel matrix measured from the microarray data and the ChIP-seq data, respectively. Hence, the more accurate, the more weight each data type in the kernel matrix would be. The motivation for combining kernel matrices by their weights is that each kernel matrix (from single data type) should exert their effects on the final training of the SVM according to their performance. We named these strategies "simple" and "weighted" in the figures and tables.Analyses for LDA, Decision Trees and ANN were performed with the default settings using the Statistics Toolbox in MATLAB, Natick, MA. For ANN, we used the Neural Network Toolbox in MATLAB implementing back-propagation to learn a two-layered-feed-forward network with five neurons in the hidden layer. To increase the reliability of the ANN results, we trained 30 ANNs and the final result is computed as the average accuracy. The simple fold-change-based predictor/classifier classifies genes as MSMG if the ESC-to-EB gene expression ratio is more than one. Such ratio was extracted from studies that compared gene expression in mESCs and EBs . The accuracy of this simple predictor based on these two independent datasets is 0.73 and 0.79, respectively.The performance of SVM classifiers and other machine learning classification methods is evaluated by LOOCV. Each classifier is trained on n-1 of the total n training samples and tested on the one left out. This step iterates n times to calculate the average performance of the trained classifier as an estimation of prediction error for unseen samples. We measured the accuracy to assess the learning performance:We also evaluated the performance of various SVM classifiers by measuring the average area under the curve (AUC) using a receiver operating characteristic curve (ROC). Each time we left one fold of training samples out as a testing set and trained the SVM on the other two folds. This step is iterated three times and the average performance can be calibrated as AUC. The ROC curves were generated by varying the decision threshold of each SVM classifier.To evaluate the SVM classifier, we chose two independent sets of genes from genome-wide RNAi screens. One screen identified genes important for stem cell self-renewal and pluripotency, whereas the other screen identified genes important to insulin signaling. We were able to match the IDs of 126 genes from the screen that identified stem cell self-renewal out of 148 genes identified in the study . These gH.X. and A.M. designed the project. H.X performed the data processing and analysis. H.X, I.R.L and A.M participated in several discussions. H.X and A.M wrote the paper. All authors read and approved the final manuscript.Feature description and reference. This file contains a table listing all features/attributes with a description and the GEO accession numbers.Click here for fileProcessed microarray and binary ChIP-seq features for all training and RNAi test samples. This file contains the normalized microarray features (1 to 79) and binary ChIP-seq features (Feature 80 to 91) for all training and RNAi test samples.Click here for fileProcessed microarray and continuous ChIP-seq features for all training and RNAi test samples. This file contains the normalized microarray features (Feature 1 to 79) and continuous ChIP-seq features (Feature 80 to 91) for all training and RNAi test samples.Click here for fileTop 20 features. This file contains two lists that are the top 20 features using the weight magnitudes in a RBF-SVM classifier and a Poly-SVM classifier as criteria for inclusion in the list. This means that these features are most useful for separating the data into the correct classes.Click here for fileResults of gene membership predictions of SVM classifiers. This file contains the classifications results across all SVM classifiers for all training samples and RNAi test samples. The column labels correspond to the same labels as in Table Click here for file


Lung cancer is the leading cause of death from cancer in the world and its treatment is dependant on the type and stage of cancer detected in the patient. Molecular biomarkers that can characterize the cancer phenotype are thus a key tool in planning a therapeutic response. A common protocol for identifying such biomarkers is to employ genomic microarray analysis to find genes that show differential expression according to disease state or type. Data-mining techniques such as feature selection are often used to isolate, from among a large manifold of genes with differential expression, those specific genes whose differential expression patterns are of optimal value in phenotypic differentiation. One such technique, Biomarker Identifier (BMI), has been developed to identify features with the ability to distinguish between two data groups of interest, which is thus highly applicable for such studies.Microarray data with validated genes was used to evaluate the utility of BMI in identifying markers for lung cancer. This data set contains a set of 129 gene expression profiles from large-airway epithelial cells (60 samples from smokers with lung cancer and 69 from smokers without lung cancer) and 7 genes from this data have been confirmed to be differentially expressed by quantitative PCR. Using this data set, BMI was compared with various well-known feature selection methods and was found to be more successful than other methods in finding useful genes to classify cancerous samples. Also it is evident that genes selected by BMI  showed better discriminative power than those from the original study. After pathway analysis on the selected genes by BMI, we have been able to correlate the selected genes with well-known cancer-related pathways.Our results show that BMI can be used to analyze microarray data and to find useful genes for classifying samples. Pathway analysis suggests that BMI is successful in identifying biomarker-quality cancer-related genes from the data. Lung cancer accounts for large portion of cancer deaths 29%) in the United States for men as well as woman . The maj9% in theWith advance of microarray technologies, it is possible to conduct high throughput determination of the relative rates with which genes are expressed in a given cell or tissue type. This can help researchers better understand a disease at the genomic level and has become an important tool in biological sciences as well as medical and pharmaceutical research. In the context of lung cancer, microarray technology can be used to identify genes whose expression profile in a type of cancer differs from normal tissues or from other types of cancer. Such biomarkers are important since they can provide the basis for improving a diagnostic classifier or for enhancing the prediction of patient-specific prognosis or therapeutic response . From anFeature selection methods, in general, can be categorized into three types, depending on how they are combined with other analysis steps: filter methods, wrapper methods and embedded methods . Filter In this paper, a filter-based feature selection method, biomarker identifier (BMI), is adopted to analyze gene expression data that might be used to discriminate between samples with and without lung cancer. The data consists of gene expression patterns in histologically normal large-airway epithelial cells obtained via bronchoscopy from smokers. Genes identified using this data set can be used to diagnosing lung cancer among smokers with suspected lung cancer. The genes selected by BMI were compared with those from various other feature selection algorithms and those identified from the original experimental study. Pathway analysis for the genes selected by BMI was also performed.D consisting of two groups 'control (ctr)' and 'experiment (exp)'. BMI assigns a score for a feature x defined as follows:The biomarker identifier (BMI) ,6 methodwhereTP2 is the product of the true positive (TP) rates determined for each groups using logistic regression of the form 'outcome ~ feature'. ctr CVand CV denote the coefficient of variance for the feature x in the 'control' group and in both groups, respectively. Also, Δ = x in 'control' and in both groups, respectively. For biological data such as microarray, the sign of Δdiff for a particular gene can be interpreted as over-expression or under-expression in 'experiment' compared to 'control'; positive as over-expression and negative as under-expression.Here, λ is a scaling factor and BMI has shown promising results on various data sets such as mass spectrometry data of metabolites , liver dFor comparison with BMI, we used 6 different popular feature selection methods: information gain (IG), Relief-F (RF), t-test (T) and its two variants (moderated t-test (MT) and window t-test (WT)), and chi-squared test (CS).x by measuring the amount of information gained with respect to the class (or group) variable y, defined as follows:Information gain , in probability theory and information theory, is a measure of the difference between two probability distributions. It evaluates a feature y assuming that it is independent of feature x (P(y)) and the conditional distribution of y assuming that it is dependent of x (P(y|x)). If x is not differentially expressed, y will be independent of x, thus x will have small information gain value, and vice versa.Specifically, it measures the difference between the marginal distribution of observable x, Relief-F selects a random sample and k of its nearest neighbors from the same class and each of different classes. Then x is scored as the sum of weighted differences in different classes and the same class. If x is differentially expressed, it will show greater differences for samples from different classes, thus it will receive higher score (or vice versa).Relief-F  is an ina priori information to yield more robust estimates. Berger et al. [The Student's t-test  is tradir et al.  suggesteChi-squared test is another popular statistical test of the divergence between the observed and expected distribution of a feature. In feature selection, it tests whether the distribution of a feature differs between groups. The chi-square score uses the summation of squared differences between observed and expected values divided by expected values.et al. reported gene expression data from large airway epithelial cells by microarray analysis [http://www.ncbi.nlm.nih.gov/geo/) under accession number of GSE4115. This data set was chosen since it consisted of a significant number of replicates and some of the genes in the data set were confirmed by quantitative PCR, which provides a good basis for preliminary validation.Spira analysis . This daanalysis . The Robanalysis  was usedanalysis  for detaTo contrast performance among feature selection methods, we also used the dataset published through MicroArray Quality Control project phase II (MAQC-II). Among 9 non-control data sets from MAQC-II, the data set with the most balanced number of positive/negative samples (breast cancer data with estrogen receptor status as class) was chosen. The data set consists of training (130 samples) and validation (100 samples) sets. The processed data was obtained through GEO under accession number GSE20194.Feature selection methods can be evaluated in various ways. One popular way is to observe the classification performance using the features selected by the method. If a feature selection method is able to choose truly significant features, the classifier trained using those features should show good performance with a small number of features. If important features are already known, on the other hand, we can evaluate feature selection methods by how they rank those known features. Since important features have not been reported for the MAQC-II data set, it can be approached only via the first evaluation strategy, but the airway data set is amenable to both modes of evaluation since some of genes have been experimentally confirmed to be differentially expressed.Since a separate validation set is available within the MAQC-II data, we used the training set for feature selection and validation set for classification. That is, feature selection methods are first applied to training set to obtain feature subsets. Then, for each feature selection method/classification algorithm pairing, classification performances are evaluated on the validation set through 10-fold cross-validation with varying number of features (from 1 to 60). AUC values  have been used herein to measure classification performance. Larger AUC values imply more precise classification. For implementation, we used Weka , a populFor the airway data set, we applied a similar ten-fold cross-validation approach as with the MAQC-II data to compare classification performance of different feature selection methods. Here, the data was divided into 10-folds, whereby 9 folds are used for both selecting features and training classifiers, and the reserved fold was used to calculate AUC value of trained classifiers. For each combination of feature selection methods and classification algorithms, this process was repeated 10 times with a different reserved fold, while varying number of features (from 1 to 60) and the AUC values were averaged over the ten distinct reserved-fold cases. The parameter setting for each classification algorithm was the same as in MAQC-II data set. Table Next, for the airway data set, we investigated how the genes confirmed in the literature  are ranked by BMI compared to other feature selection methods. If these genes are generally ranked highly, a feature selection method could be said to corroborate the given data. As before, we divided the data into 10 folds and used only 9 folds in feature selection, repeating the feature selection for each distinct reserved fold. For each of these ten fold cases, we recorded gene ranks as determined by each method and calculated the median value for each gene. Figure From these results, it can be said that BMI shows competitive performance in identifying useful features for classification and shows high consistency with actual differential expression.For the airway data set, we further compared the genes selected by BMI and the biomarkers from original literature . In origTable Although a set of genes is useful for training classifier, the constituent genes may be useless as biomarkers if their biological roles are not related to the target disease or process. Thus we analyzed the pathways associated with 80 highly-ranked genes to investigate their biological roles. For pathway analysis, we investigated associated terms in KEGG pathways , NCI-Natp-values (< 0.05). We can observe that there are some genes  associated with mitogen-activated protein kinase (MAPK) pathways, which is a well-known target in the oncology drug discovery [p-value of 0.0019) with our set of 80 genes. One also finds other pathways related with known oncogenes such as c-Met [Tables iscovery . Also, tiscovery . We noteas c-Met  and epidas c-Met  within oIn this work, a filter-based feature selection method, biomarker identifier (BMI), has been applied to find potential biomarkers for lung cancer from microarray data. BMI measures the potential value of each gene as a biomarker candidate by combining various statistical measures to assess its ability to distinguish between two data groups of interest. We evaluated BMI performance on two public microarray data sets: one from the MicroArray Quality Control project and the other from smokers with and without lung cancer. BMI was compared with other popular filter-based feature selection methods on both data set and showed competitive performance in selecting useful features for various classification algorithms. Since of the latter data set includes information regarding specific genes whose tissue differentiation relevance has been validated by quantitative RT-PCR, we also compared how these genes were ranked by different feature selection algorithm. The validated genes generally were assigned higher ranks by BMI than by other methods, implying that BMI should be effective in identifying biomarkers that show differential expression in cancerous samples. We also compared BMI with the approach in the original analysis conducted on the lung cancer microarray data  by contrThe authors declare that they have no competing interests.IL participated in the design of the study, performed the statistical analysis and drafted the manuscript. GL and MV conceived of the study, and participated in its design and coordination. All authors read and approved the final manuscript.


Sustained research on the problem of determining which genes are differentially expressed on the basis of microarray data has yielded a plethora of statistical algorithms, each justified by theory, simulation, or ad hoc validation and yet differing in practical results from equally justified algorithms. Recently, a concordance method that measures agreement among gene lists have been introduced to assess various aspects of differential gene expression detection. This method has the advantage of basing its assessment solely on the results of real data analyses, but as it requires examining gene lists of given sizes, it may be unstable.Two methodologies for assessing predictive error are described: a cross-validation method and a posterior predictive method. As a nonparametric method of estimating prediction error from observed expression levels, cross validation provides an empirical approach to assessing algorithms for detecting differential gene expression that is fully justified for large numbers of biological replicates. Because it leverages the knowledge that only a small portion of genes are differentially expressed, the posterior predictive method is expected to provide more reliable estimates of algorithm performance, allaying concerns about limited biological replication. In practice, the posterior predictive method can assess when its approximations are valid and when they are inaccurate. Under conditions in which its approximations are valid, it corroborates the results of cross validation. Both comparison methodologies are applicable to both single-channel and dual-channel microarrays. For the data sets considered, estimating prediction error by cross validation demonstrates that empirical Bayes methods based on hierarchical models tend to outperform algorithms based on selecting genes by their fold changes or by non-hierarchical model-selection criteria. (The latter two approaches have comparable performance.) The posterior predictive assessment corroborates these findings.Algorithms for detecting differential gene expression may be compared by estimating each algorithm's error in predicting expression ratios, whether such ratios are defined across microarray channels or between two independent groups.According to two distinct estimators of prediction error, algorithms using hierarchical models outperform the other algorithms of the study. The fact that fold-change shrinkage performed as well as conventional model selection criteria calls for investigating algorithms that combine the strengths of significance testing and fold-change estimation. Continual invention of new microarray data analysis algorithms for the identification of which genes express differently across two groups calls for objectively comparing the performance of existing algorithms . While tet al. 2) to the squared prediction error. Under the "all nulls false" reference algorithm, the best prediction of future values of The squared prediction error criterion therefore quantifies the relative costs of false positives and false negatives in terms of the bias-variance trade-off. To estimate the squared prediction error, we used leave-one-out cross validation,n = n' and xi,j orif Xi are independent, where (-j) means the jth replicate is omitted:if xi,,j and the data for gene i were xi = . For j from 1 to 3, πα (Hi| xj)(-) = . (-).) The individual terms in the sum in equation 28 are (-2 - 0.95 - 2)2, (1 - 0.78 - 0.5)2, and (3 - 0.39 - (-0.5))2, and their mean is 8.6. If the given data were independent instead of paired, the calculation would involve each of the 9 subsets obtained by leaving out one perturbation data point and one control data point. For example, suppose that Xi = 0 on a gene-wise basis using the ratioWe considered measuring error relative to always predicting that with two measures of central tendency,hsm function in the modeest package of R.) We also considered an absolute error criterion,  is a fasthis measure is relative to a base model such as the "all nulls true" model or the "all nulls false" model because we expect only the relative performances of the estimators to be meaningful. We found that the relative error mean essentially reproduced the absolute error relative to the "all nulls true" model, and the relative error mode often evaluated estimators as not practically different from the "all nulls true" benchmark. Therefore, we show only the results for the absolute error measure.et al. [The use of cross-validation for estimation of classification error, appropriate for the problem of categorizing samples or microarrays given known classifications for use in the training and test sets, differs from the use cross-validation for estimation of squared prediction error, appropriate for the distinct problem of determining which genes are differentially expressed without knowledge of which genes are differentially expressed for use in the training and test sets. Jeffery et al.  used a cet al. .The local FDR shrinkage algorithm can be used to define an estimator's posterior predictive expected squared error. In general, the posterior predictive expected squared error isXi,new are random variables for new observations, α's point prediction for Xi,new, and Eposterior and varposterior are the expectation and variance with respect to the posterior distribution. The effective posterior distribution that leads to estimators of the form (26) iswhere which has variance,πα ; this model will be accurate under the reasonable assumption that few genes are differentially expressed at appreciable levels.We use the local FDR estimator with t-statistics and theoretical null distribution as our gold standard model for the computation of Xnew under both the null and alternative hypotheses for both paired and non-paired data. Conditional on each hypothesis, we use improper prior distributions for convenience. Strictly speaking, this is inconsistent with our choice of πα , an empirical Bayes approximation to a posterior probability; under a full Bayesian analysis, posterior probabilities of hypotheses can only be computed under proper priors for the parameters conditional on each hypothesis, as in the Bayes factor algorithm of equation (11). Our choice of πα  enables sharing information across genes to give a sensible empirical Bayes posterior probability for the hypotheses but otherwise relies on the same assumptions as our conditional prior distributions.To fully express the posterior predictive loss, we must define the posterior predictive distribution for Xnew has a normal sampling distribution with zero mean and sampling variance estimated from the data. Under the usual improper prior for the sampling variance ∝ σ-2), the posterior distribution for the sampling variance is a scaled-inverse-χ2 distribution with degrees of freedom n and scale For paired data under the null hypotheses, tv  is a shifted, scaled version of the t distribution with v degrees of freedom, center c, and scale factor s. , then (Y - c)/s is distributed as the usual tv distribution.)where N  is the normal distribution parameterized in terms of mean and variance, and Xnew has a normal sampling distribution with both mean and sampling variance estimated from the data. It can be shown that under the usual improper joint prior for mean μ (μ = E(X' - X)) and the sampling variance  ∝ σ-2), the posterior predictive distribution for Xnew is,For paired data under the alternative hypothesis, s2 is the usual unbiased variance estimator.where πα ) then the posterior predictive distribution isFor non-paired data under the null hypothesis, if the treatment and control data are modeled as having distinct sampling variances (consistent with the assumptions used to specify σ')2 and σ2 are the sampling variance for treatment and control data respectively. This integral is intractable because πposterior 2)has a non-standard form 2) using Markov chain Monte Carlo (MCMC) [where (o (MCMC)  and thenk indicates the kth MCMC draw of parameter values (after suitable burn-in) and K is the total number of draws. In the present case, the MCMC algorithm we use is an inherently multi-chain procedure; we used 10 chains. We used a burn-in of 20 samples per chain, followed by 100 samples per chain, for a total of K = 1000 samples. For each gene in a randomly chosen subset of genes from the complete data set, a contour plot of the posterior density was superimposed on a scatter plot of the MCMC draws of parameter values. The scatter plots visually conformed to the contours of the posterior densities, verifying that the MCMC draws of parameter values provided a good approximation to the posterior distributions.where the subscript Xnew each have a normal sampling distribution with both mean and sampling variance estimated from the data. It can be shown that under the usual improper joint prior for the individual means and sampling variances, the posterior predictive distributions for Xnew areFor non-paired data under the alternative hypothesis, and thereforeTo summarize gene-wise posterior predictive expected squared error over all genes in a data set, we considered quantities analogous to the relative errors and absolute errors of equations (32)-(34), with gene-wise posterior predictive expected squared errors replacing cross-validation-derived prediction errors. Again, we found that the relative error mean essentially replicated the results of the absolute error relative to the "all nulls true" benchmark; relative error mode evaluated the performance of all estimators as identical to the "all nulls true" benchmark. Therefore, we show only the results for the absolute error measure for posterior predictive expected squared error.To illustrate the proposed methods of quantifying the performance of gene selection algorithms, we applied them to two example data sets, one relevant to agriculture and the other to medicine. Since this study is limited to the evaluation of high-level algorithms of detecting differentially expression, we did not consider multiple pre-processing schemes. The agricultural data sets were processed as described in ; the medm = 13,440 genes at the breaker stage of ripening and at 3 and 10 days thereafter [n = 6), but one of the biological replicates is missing at the breaker stage of ripening (n = 5). The next subsection compares algorithms of determining which genes are differentially expressed between mutant and wild type at each point in time, whereas Subsection 3.2 uses the same data to instead compare algorithms of determining which genes are differentially expressed between one point in time and another point in time.Dual-channel microarrays were used to measure in tomatoes the expression ratios (mutant/wild type) of ereafter . Each ofxi,j at 0, 3, or 10 days after the breaker stage. Then xi,j is the logarithm of the observed ratio for the ith gene and jth microarray. Due to this dependence structure, paired (1-sample) t-tests and Wilcoxon signed-rank tests were used to obtain p-values, and equation (34) was used to estimate prediction error. We measured absolute error relative to the local FDR using the t-statistic and the theoretical null (labelled "t stat. with locfdr") because this model had the best or near-best performance in seven out of the nine data sets considered in this paper. Thus, its use as the base model facilitated the plotting of multiple data sets in a single figure; this same model was used as the base model for all subsequent figure. The estimated prediction errors for all algorithms mentioned above are displayed as Figure In order to determine the genes for which expected values of logarithms of mutant-to-wild-type ratios differ from 0, let xi,j denote the logarithms of ratios observed at two different points in time for gene i and for microarrays j' and j. Since the measurement errors of observations made at one time point are independent of those made at the other time point, 2-sample t-tests and Wilcoxon rank-sum tests were used to obtain p-values, and equation (34) was used to average estimated prediction error  and the two Affymetrix data sets  were used to assess gene selection criteria on the basis of prediction error  model selection criteria except when it is penalized by the imposition of a hard threshold. Algorithms based on hard thresholds for fold change are outperformed by shrinkage fold-change and by all other non-trivial algorithms that are not restricted by arbitrary thresholds: Tables Herein we examined only algorithms that fall into one of two distinct categories:1. The shrinkage and hard-threshold fold-change algorithms are based on estimated fold change without regard for statistical significance or estimates of variance.2. All other algorithms of the present study compute levels of significance without regard for fold change estimates. Since these categories represent opposite extremes, their algorithms might be outperformed by those that instead employ both fold-change information and variance/significance information. Our observation that fold change performs as well as simple model-selection criteria suggests consideration of less extreme algorithms that combine the advantages of the ones studied herein. Investigators reported that the estimation of fold-change following a non-stringent significance filter performs better than does either type of algorithm alone ,40, and Adjusting fold-change estimates according to significance levels is not the only way to combine the two types of information. A complementary strategy instead adjusts significance levels according to fold-change thresholds. In fact, the seemingly inferior performance of statistical methods that do not make use of fold-change estimates has been explained in terms of a distinction between statistical and biological significance , which wIn general, cross-validation is subject to high variance when sample sizes are small. If each of the features had independent data of finite variance, the central limit theorem would nonetheless guarantee a small variance in the overall measure of performance (34). In the present case, however, due to gene-gene interactions, the numerator and denominator of the overall measure of performance are sums of positively correlated quantities. To address this concern, we performed an additional assessment of the differential-expression-detection algorithms using posterior predictive expected squared error methodology. The posterior predictive expected squared error requires the choice of a particular gold standard model, a Bayesian model consisting of a family of sampling distributions refecting knowledge about the biological system and a prior distribution. Here, we based our posterior predictive expected squared error on the implicit Bayesian model approximated by the local-FDR mean expression estimator calculated using t-statistics and the theoretical null distribution. The key assumption of the model is that few genes are differentially expressed at any notable level; the model also assumes that gene expression ratios are lognormally distributed. The model accommodates unequal variances for non-paired data using conventional improper priors under each hypothesis since we have little prior information about the specific parameter values.  Naturally, the model's corresponding estimator had the lowest posterior predictive expected squared error, but provided the assumptions encoded in the model hold, the posterior predictive expected squared error will nonetheless be a good way to rank the performance of the estimators.The fitting of the gold-standard model generated estimates for the proportions of equivalently expressed genes, allowing the verification of the assumption that most genes were equivalently expressed. For the 0 days, 3 days, and 10 days data sets, the estimated proportion of equivalently expressed genes were 0.91, 0.89, and 0.73, respectively; for the 10 days vs. 3 days, 10 days vs. 0 days, and 3 days vs. 0 days data sets, they were 0.83, 0.83, and 1.00, respectively; and for the Applied Biosystems, Affymetrix 1, and Affymetrix 2 data sets, they were 0.62, 0.59, and 0.60, respectively. The six tomato data sets have relatively high proportions, showing that these data sets more closely satisfy the assumption of a proportion close to 1. Therefore, the local-FDR-based rankings for the estimators in these data sets should be accurate. The MAQC data sets have lower proportions, indicating that the model assumption is a poor approximation. It is not surprising that the MAQC data sets have many differentially expressed genes, as they are derived from liver tissue treated with a potent toxin.c, which was rated highly by posterior predictive expected loss but poorly by cross-validation for the 0 days and 3 days data sets , and the moderated t-statistic with limma, which was rated highly by cross-validation but poorly by posterior predictive expected loss for the 0 days, 3 days, 10 days vs. 3 days, and 3 days vs. 0 days data sets (Figures c estimator are essentially tied for best performance (Figure As noted before, the cross-validation performance measure ranks the gold standard model highly for the tomato data sets, that is, for the data sets that we expect good estimation from the gold standard model. Furthermore, a careful inspection of Figures  Figures -2, and te Figure  These obThe posterior predictive methodology helped to confirm that the cross-validation methodology was effective for measuring estimators' relative performances. The results support the use of local-FDR-based statistical algorithms over both conventional model-select ion criteria and over algorithms based only on fold change. In particular, the estimator based on the local FDR calculated using t-statistics and the theoretical null had the overall best performance when the proportion of equivalently expressed genes was high. As a second choice, the estimator based on the local FDR calculated using moderated t-statistics also performed quite well. Tables The fact that fold-change shrinkage performed as well as conventional model selection criteria calls for investigating algorithms that combine the strengths of significance testing and fold-change estimation.c algorithms, and implemented the posterior predictive expected squared loss assessment. DRB conceived the study, selected the data sets, and applied the fold change and empirical Bayes algorithms. Each author made substantial contributions to writing the paper. Both authors read and approved the final manuscript.CMY selected the Bayes factor algorithm, implemented the Bayes factor, BIC, and AICThis file contains a heuristic overview and detailed derivation of our Bayes factor approach to calculating probabilities of differential expression.Click here for file


Deregulation of biological pathways has been shown to be involved in the turmorigenesis of a variety of cancers. The co-regulation of pathways in tumor and normal tissues has not been studied in a systematic manner.In this study we propose a novel statistic named AR-score (average rank based score) to measure pathway activities based on microarray gene expression profiles. We calculate and compare the AR-scores of pathways in microarray datasets containing expression profiles for a wide range of cancer types as well as the corresponding normal tissues. We find that many pathways undergo significant activity changes in tumors with respect to normal tissues. AR-scores for a small subset of pathways are capable of distinguishing tumor from normal tissues or classifying tumor subtypes. In normal tissues many pathways are highly correlated in their activities, whereas their correlations reduce significantly in tumors and cancer cell lines. The co-expression of genes in the same pathways was also significantly perturbed in tumors.The co-regulation of genes in the same pathways and co-regulation of different pathways are significantly perturbed in tumors versus normal tissues. Our method provides a useful tool for better understanding the mechanistic changes in tumors, which can also be used for exploring other biological problems. Cell behavior is under precise regulation by biological pathways, which consist of a series of biochemical reactions (metabolic pathways) or signal transduction events (regulatory pathways) Microarray experiments provide the expression levels of tens of thousands of genes simultaneously, and have been widely used to understand the cancer mechanisms In this study, we calculated the normalized average rank of genes in pathways in a gene expression profile, denoted as AR-score (average rank based score), to represent the activity of the pathway. Similar rank-based statistics have been proposed in previous studies The human genome contains more than 23,000 protein-coding genes. Their protein products are organized into a complex network, in which they function coordinately to regulate many important biological processes. Specifically, the network is formed by a variety of intertwined biological pathways, which consist of a series of chemical reactions or signal transduction events. The activities of many pathways vary substantially among different tissues/cell-types, and are often subjects to considerable modifications in tumors with respect to the corresponding normal tissues.We defined a new measurement called AR-score (AS) to quantify the activity of pathways in a biological sample based on its gene expression profile from microarray experiment. The AR-score for a pathway is calculated as the average rank of the relative expression levels of all genes in this pathway normalized by the total gene number. It takes a value from 0 to 1. A higher score indicates that genes in the pathway are overall highly expressed and therefore this pathway is highly active. The rank based score is robust by nature and can be directly used to compare pathway activities between different samples, even though they are not from the same microarray dataset. Moreover, the activities of pathways with different sizes (i.e. different in their gene numbers) are directly comparable.We calculated the AR-scores of 186 KEGG pathways in 79 human tissues for which the expression profiles were measured by microarrays On the other hand, the scores for the 186 KEGG pathways in a tissue give rise to an activity profile of the tissue. The pathway activity profile characterizes a tissue and can be used to investigate the similarity of different tissues in their metabolic/regulatory states. We performed the hierarchical clustering analysis for the 79 human tissues based on their pathway activity profiles. As shown in As the components of a whole network, different pathways often function in a coordinated manner to participate in many critical biological processes. To understand the inter-relationships among the KEGG pathways, we calculated the correlations of their activity profiles across the normal human tissues based on the dataset by Su et al −8 (see −14). As shown, many pathways were positively correlated in their activity profiles (red edges), namely they are presumably coupled with each other. For example, the activity profile of the “CELL_CYCLE” pathway is positively correlated with those of the “DNA_REPLICATION”, the “MISMATCH_REPAIR”, the “NUCLEOTIDE_EXCISION_REPAIR” and 4 other KEGG pathways. More interestingly, we found that the “CITRATE_CYCLE_TCA_CYCLE” pathway was highly correlated with the “PARKINSONS_DISEASE”, “ALZHEIMERS_DISEASE” and “HUNTINGTONS_DISEASE” pathways. The high correlation of the pathway activity profiles indicated that different pathways were coupled with one another to achieve normal biological processes.Our results indicated that the pathways were highly correlated in their activity in normal tissues. Among the 17,205 possible pathway pairs, 809 have a Spearman correlation coefficient ≥0.6 and 221 have a correlation ≤−0.6, corresponding to a P−value ≤10To investigate the pathway modifications in tumor cells, we applied the pathway analysis to 9 cancer microarray data sets, representing 7 different cancer types: liver, lung, kidney, pancreas, prostate, stomach and thyroid cancers (see −5) were used as the classifiers, the SVM model achieved 95% accuracy (83 out of the 87 samples). The same accuracy was obtained when the 11 most significant pathways were used (P<10−10). Impressively, a SVM model based on only 5 pathways (P<10−12) could still result in correct classification for 82 samples (94%). In contrast, the average classification accuracy was 0.85 when 5 randomly selected pathways were used as the classifiers. The classification power of these 5 pathways is also shown Given the substantial difference in the activities of many pathways between tumor and normal tissues, we then examined the effectiveness of classifying normal and tumor samples based on their activity profiles. Here we used the “Liver_Tsuchiya” data set as the example. The data set contained the expression profiles for 43 tumor and 44 non-tumor liver tissues surgically resected from patients with HCV-associated hepatocellular carcinoma. We established a support vector machine (SVM) to classify the tumor and non-tumor samples using the activity profiles for KEGG pathways as the classifiers, and examined the accuracy of the model based on the leave-one-out cross-validation method (see “−5), the model still gave rise to a classification accuracy of 85% (83 out of 97 samples). The hierarchical clustering results based on these 10 pathways were shown in We then examined the effectiveness of classifying tumor subtypes based on the pathway activity profiles. As an example, we applied the SVM method to classify estrogen receptor positive (ER+) and negative (ER−) samples in the breast cancer data We also examined the capability of the pathway activity profiles for predicting prognosis. In the breast cancer data from van’t Veer et al Gene-based classification of tumors has been intensively described in previous studies In normal cells, behavior is regulated by a number of pathways that cross-talk and are highly coordinated with each other. Cancer, in many ways, can be regarded as a disease of mis-regulated signal transduction. Thus, we would expect to see the decoupling of pathways in cancer with respect to normal cells. To investigate this issue, we carefully selected five microarray gene expression data sets, three for normal tissues  −308). In addition, we examined the correlations of pathways in tumor and normal samples from the same dataset , which confirmed the conclusion that pathways were more correlated in normal tissues than in cancers. For example in the “Kidney_Dalgliesh” data, correlations  between pathways are significantly higher in kidney cancers than in the normal kidney controls .We calculated the Spearman correlation coefficients for all pairs of the 186 KEGG pathways in the 5 data sets. Our results showed that the activity profiles of pathways were more correlated (either positively or negatively) in the normal data sets  than in the tumor and the NCI-60 data sets (NCI-60). −11), Normal_Su vs. NCI-60 (P = 5×10−18), Normal_Roth vs. tumor (P = 2×10−13), Normal_Roth vs. NCI-60 (P = 2×10−17), Normal_Ge vs. tumor (P = 2×10−11) and Normal_Ge vs. NCI-60 (P = 3×10−15).We have shown that the correlations of activity profiles between the pathways were reduced significantly in tumor samples or cell lines. We then asked: do the correlations of the expression profiles for genes in the same pathway also reduced in tumors? For each pathway we calculated the Spearman correlation coefficients for all gene pairs in the three normal data sets as well as in the tumor and NCI-60 data sets. The average of these pairwise correlations was then computed to represent the co-regulation of genes in a pathway. As shown in To further validate our results, we repeated the above-described analysis using the ES scores (enrichment scores) introduced in the GSEA method. Specifically, for each sample in a microarray data we calculated the ES-scores of all KEGG pathways. We compared the ES-scores of pathways in cancer versus normal samples to identify the differentially expressed pathways using the Wilcoxon test. The results we achieved are very consistent with those obtained from the AR-score based analysis (note that the t−test was used for comparing AR-scores). As an example, in the Liver cancer data we identified 88 significant pathways based on AR-scores and 85 significant pathways based on ES−scores (P<0.001). Among these pathways, 64 are identified by both methods. More importantly, among them the up-regulated and down-regulated pathways identified by the two methods are perfectly matched see .−208), Normal_Su vs. NCI-60 , Normal_Roth vs. tumor , Normal_Roth vs. NCI-60 , Normal_Ge vs. tumor  and Normal_Ge vs. NCI-60 .We also examined the correlation between different pathways in normal and cancer data sets based on their ES−scores. This analysis further validated our conclusion that the co-regulation of pathways tended to be decoupled in cancers with respect to normal tissues see . The corPrevious studies have reported the association of a number of pathways with cancers Several methods for pathway analysis have been previously proposed, such as Gene Set Enrichment Analysis (GSEA) The aforementioned limitations, however, are not likely to affect our main conclusions. Although genes in a pathway could be either up- or down- regulated in cancer versus normal, our results suggest that they tend to be regulated into the same direction. First, genes in the same pathway are more likely to be positively correlated in their expression profiles, which is particularly true in normal tissues . Second,During human tumor development, the tumor cells have to obtain several capabilities, such as sustaining proliferative signaling and resisting cell death . Acquirement of these capabilities is often involved in the de-regulation of related pathways such as the apoptosis pathway. To de-regulate a pathway, the expression of a different subset of its genes might be perturbed in different tumor samples even for the same type of cancer. For this reason, the alteration of pathways is often more consistent than that of genes in tumor samples of the same type. For example, we compared the differentiation of pathways and genes between tumor and normal samples in two independent kidney cancer data sets. Specifically, the t-scores were calculated by comparing the AR-scores of pathways or expression levels of genes in tumors versus normal. The t-scores for pathways are more correlated (R = 0.81) between the two data sets than those for genes (R = 0.42), indicating high consistency between tumor samples at the pathway level. Such a consistency is only valid for those well-defined biological pathways, not for the random gene sets, which are mechanistically less informative. We examined the classification power of a number of random gene sets that are significantly different between tumor and normal samples. These random gene sets resulted in a classification accuracy  that is only slightly lower than that of the real pathways. When applied to another data set, these gene sets are no longer effective for classification.In brief, we propose a statistic to measure the pathway activities in gene expression profiles. We applied the method to a large number of microarray gene expression data sets to investigate changes of pathways in their activities and relationships in cancers with respect to normal tissues. We found that the AR-score we defined is capable of classifying samples into biological meaningful groups, e.g. normal vs tumor liver, and ER+ vs ER- breast cancer subgroups. We also found that in tumor samples and cell lines, the co-regulation of genes in the same pathways and between pathways was significantly perturbed with respect to normal tissues. More intensive investigation of the relationship changes between pathways would be helpful for further understanding the mechanisms of carcinogenesis.A total of 14 microarray data sets were used in this study, among which three are for normal tissues, one is for cancer cell lines and ten are for tumor/normal samples. The breast cancer data was from van’t Veer et al http://www.genome.jp/kegg/The pathway information was downloaded from the database called Kyoto Encyclopedia of Genes and Genomes (KEGG) at Except for the breast cancer data from i is the rank of the ith in the pathway, m is the number of genes in this pathway and n is total number of genes in the gene expression profile. The AR-score takes a value within , with a larger value indicating relatively higher expression levels of genes in a pathway and therefore a higher pathway activity.Given the expression profile for a specific biological sample, we first sorted the expression levels for all genes in decreasing order. Then, we calculated the AR-score of a pathway by averaging the ranks of all genes in this pathway and then normalizing the result with the total gene numbers in the expression profile. Namely, i, i = 1,2, …, p} and {yi, i = 1,2, …, p} as the rank vectors (p is the number of samples) for the AR-scores of two pathways, the co-regulation is calculated asThe co-regulation of two pathways can be inferred based on the similarity of their activity profiles- their AR-scores in all of the samples. Simply, the Spearman Correlation Coefficient (ρ) between the activity profiles of two pathways can be used to measure the similarity. However, this would over-estimate the correlation if two pathways share common genes. To overcome this problem, we first removed genes shared by two pathways and calculated their AR-scores based on the remaining unique ones. The resulting AR-score profiles were subsequently used for estimating the correlation between the two pathways. Denote {xWe calculated the Spearman Correlation Coefficient for all possible pairs of the 186 KEGG pathways, while taking into account the shared genes between them (see Section “Correlation of activity profiles between pathways”). A total of 17,205 correlations were calculated, among which we observed much more positively correlated pathway pairs than negatively correlated ones. We then set |ρ|>0.75 as the cut-off value for correlations to select pathway pairs with similar AR-score profiles. Finally, all the pathways with |ρ|>0.75 were connected to form a co-regulation network for the pathways. The network contains both positive (correlated pathways) and negative (anti-correlated pathways) interactions. In normal tissues , the above analysis resulted in a network with 59 nodes (pathways) and 127 edges , among which 110 are positive correlations and 17 are negative correlations.We used the t−test to identify the pathways that show differential activities between two sample groups . For instance, to identify pathways related to lung cancer, we compared the AR-scores of all pathways in lung tumor samples with those in normal lung tissues using the t−test, resulting a P−value for each pathway. To correct for multiple testing, we calculated the corresponding Q−value  for each of the P−values using the method proposed by Storey et al We performed hierarchical clustering to investigate the similarity of normal human tissues based on the AR-scores of the 186 KEGG pathways. Specifically, the “complete linkage” method was applied and the “Euclidian distance” was used as the dissimilarity metric in the hierarchical clustering analysis. Hierarchical clustering was also used to cluster tumor and normal samples or to cluster different tumor subtypes. For this purpose, we usually selected a number of pathways that showed differential activities and performed hierarchical clustering based only on these pathways. For example, We constructed support vector machine (SVM) models The SVM models were also used to classify normal versus tumor samples, and to predict the clinical outcome (good- or poor-prognosis groups) of patients.We calculated the Spearman correlation coefficient of the expression profiles for all possible pairs of genes in the same pathway. Then these correlations were averaged to represent the strength of co-expression of genes in this pathway. The average correlations were calculated for all of the 186 KEGG pathways. The average correlations for most pathways are positive values, particularly in normal tissues, indicating that genes in the same pathway tend to be co-expressed.To confirm our findings using the AR-score based method, we also performed all analysis using the statistic, “enrichment score” (ES), proposed by the GSEA method http://www.r-project.org.All the calculation described above was performed using the R language and packages available from Figure S1Correlation patterns of pathway activities (AR-scores) in five microarray data sets. The figure shows pairwise correlations of all pathways in three normal tissue data sets, one multiple-tumor data set and one NCI-60 cell line data set.(PDF)Click here for additional data file.Figure S2Correlation patterns of pathway activities (ES-scores) in five microarray data sets. The figure shows pairwise correlations of all pathways in three normal tissue data sets, one multiple-tumor data set and one NCI-60 cell line data set. Note that ES-scores are used to represent the pathway activities, whereas in (PDF)Click here for additional data file.Figure S3Distribution of AR-scores and ES-scores. Distributions of the AR-scores (left panel) and ES-scores (right panel) for pathways in Normal_Su data were shown. As shown, AR-scores approximately follow a normal distribution, while ES-scores follow a bimodal distribution with a positive and a negative peak.(PDF)Click here for additional data file.Table S1Correlations for all possible pairs of the KEGG pathways. Spearman correlation coefficients of all the 17,205 pairs of the 186 KEGG pathways were calculated.(XLS)Click here for additional data file.Table S2Information about the nine tumor vs normal data sets.(XLS)Click here for additional data file.Table S3Pathways showing differential activities between tumor and normal tissues in nine data sets. T-test was used to compare the activities of all the 186 KEGG pathways between tumor and normal samples in nine microarray data sets.(XLS)Click here for additional data file.Table S4Average correlations of expression profiles across all gene pairs in the same pathways. The table contains the average Sperman correlation coefficients for all the 186 KEGG pathways in three normal tissue data sets, one multiple-tumor data set and one NCI-60 cell line data set. For each pathway, the correlations for all possible pairs of genes in the pathway were calculated and averaged.(XLS)Click here for additional data file.Table S5Comparison of the differential pathways identified by the AR-score based and the ES-score based methods. The table contains the pathways that show significantly activities in Liver cancer and normal liver samples. The AR-scores of pathway in cancer and normal are compared using the t-test, whereas the ES-scores are compared using the Wilcoxon test. Note that pathways identified by the two methods are highly consistent.(XLS)Click here for additional data file.Table S6Percentage of up-regulated genes in pathways. The table contains the numbers of genes in each pathway, the fraction of up-regulated genes (t-score>0) in all genes or in the differentially expressed genes (P<0.001). Calculation is based on the Dalgliesh kidney cancer data.(XLS)Click here for additional data file.


Transcriptional regulation by transcription factor (TF) controls the time and abundance of mRNA transcription. Due to the limitation of current proteomics technologies, large scale measurements of protein level activities of TFs is usually infeasible, making computational reconstruction of transcriptional regulatory network a difficult task.We proposed here a novel Bayesian non-negative factor model for TF mediated regulatory networks. Particularly, the non-negative TF activities and sample clustering effect are modeled as the factors from a Dirichlet process mixture of rectified Gaussian distributions, and the sparse regulatory coefficients are modeled as the loadings from a sparse distribution that constrains its sparsity using knowledge from database; meantime, a Gibbs sampling solution was developed to infer the underlying network structure and the unknown TF activities simultaneously. The developed approach has been applied to simulated system and breast cancer gene expression data. Result shows that, the proposed method was able to systematically uncover TF mediated transcriptional regulatory network structure, the regulatory coefficients, the TF protein level activities and the sample clustering effect. The regulation target prediction result is highly coordinated with the prior knowledge, and sample clustering result shows superior performance over previous molecular based clustering method.The results demonstrated the validity and effectiveness of the proposed approach in reconstructing transcriptional networks mediated by TFs through simulated systems and real data. UnderstIn contrast, works based on factor models -10 pointIn a response to these requirements of modeling TF mediated regulatory networks, we propose here a novel Bayesian non-negative factor model (BNFM). Different from conventional factor analysis models, BNFM consists of a sparse loading matrix and a set of correlated non-negative factors. The sparsity of the loading matrix is constrained by a sparse prior  that dirn-th microarray mRNA expression profile of G genes under a specific context. In practice, microarray data yn register the log2 scaled (fold change of) gene expression levels under the context of interest relative to a background often obtained as the average expression levels among a variety of contexts, such as different cell lines and tumors ⊺ can be approximated with the mean of prior distribution (4)(5), which can be calculated numerically. We can also see that the corresponding centralized factors are a shifted version of the original factors, and different samples shift the same amount, so sample clustering effect is still retained. On the other hand, the removed term from data centralization is no longer an estimator of the constant term c, but,Given sufficient number of samples, the sample mean A, xn∀n, nγ∀n, given yn∀n and gπ,l ∀g, l, which is the TF binding prior information extracted from existing database. For convenience, we let Θ denote all the known and unknown variables.The goal is to obtain the posterior distributions and hence the estimates of The proposed BNFM model is high dimensional and analytically intractable, so a Gibbs sampling solution is proposed. Gibbs sampling devises a Markov Chain Monte Carlo scheme to generate random samples of the unknowns from the desired but intractable posterior distributions and then approximate the  posterior distributions with these samples. The key of Gibbs sampling is to derive the conditional posterior distributions and then draw samples from them iteratively until convergence is reached. The proposed Gibbs sampler can be summarized as follows:t-th iteration:Iterate the following steps and for the 1. Sample 2. Sample p;3. Sample n = 1 to N4. for Sample Sample p(sn|Θsn–);Sample µx can be approximated and calculated numerically, the factor xn can be recovered from the centralized factor Note that ribed in , T, then,Let ŷg,l = xlag,l + eg, xl = T and eg = T. The posterior distribution of ga,l,where, Z0 is a normalizing constant, ga,l ≠ 0 and BF01 is the Bayes factor of model ga,l = 0 versus model ga,l ≠ 0where ga,l ≠ 0 and defined bywith gπ,l is the prior knowledge of the probability of ga,l to be non-zero. When gπ,l = 0.5, i.e, a noninformative prior on sparsity is assumed, BF01, and BF01 > 1. Since model selection based BF01 favors ga,l = 0, it suggests that this Bayesian solution favors sparse model even when gπ,l = 0.5.where, nγ does not depend on xn in the distribution. It is intended that samples of nγ from this distribution are not affected by the immediate sample of xn, thus achieving faster convergence of the sample Markov chains. To derive this distribution, first let ŷl,n = alxl,n + en with al being the l-th column of A and hence It should be noted that ls that also belong to cluster k, Nl,k– is size of where with,where,and,–lN,k = 0, and kg for Noted that, for a new cluster, This distribution can be expressed asxµ,+∞), and,where, lx,n, the conditional distribution of ls,n does not depend on y1:N; As the predictive density p is shown to be a Student-t distribution, which can be conveniently approximated as a normal distribution when –lN,k is large:According to the graphical model, given and conditional distribution can be expressed asπsl,n = 1 – sgn where, E = Y – AX, and we have, eg = ⊺ Given the conjugate Inverse-Gamma prior, we haveLet the residuals where Inv-Gamma represents the Inverse-Gamma distribution andWith the prior distribution aN,l is the size of where, The proposed BNFM model was first tested on a simulated system, in which the microarray data consists of the expression profiles of 150 genes with 40 samples. The samples form 5 clusters and the 150 genes were assumed to be regulated by 10 TFs. The sparsity of loading matrix was set at 10%, which means that on average each gene is regulated by 1 TFs, and each TF regulates 15 genes. To simulate a practical imperfect database, the precision and recall of the prior knowledge were both set equal to 0.9 each, i,e., 90% of the database recorded regulations indeed happened in this specific data set (10% of the database recorded regulations may be context-specific and didn’t happen in the data); and 90% of the true regulations was recorded in the database (10% of true regulations are not in the database). This setting indicates that the recorded prior regulations may not exist in the experiment, and the unknown regulations could exist. Since this is a relatively large data set involving sampling of many variables, instead of examining convergence based on , chap. .F metric [L(e) and C(e) be the category and the cluster of an item e. Then, the correctness of the relation between e and ′e is defined bySince the algorithm estimates the loading matrix, the factors, the clustering result, and TF regulatory targets, to evaluate the performance, four respective metrics were computed. Particularly, in order to systematically evaluate the clustering result, a Van Rijsbergen’s F metric  that comF metric  was implF metric . More spThat is, two items are correctly related when they share the same cluster. Moreover, the BCubed precision and recall are formally defined asF metrics:These two metrics can be further combined using Van Rijsbergen’s F metrics satisfy all the 4 formal constraints defined in [F metrics to evaluate the clustering result in the following tests. Similarly, a Van Rijsbergen’s F metric that combines the target prediction precision and recall is used to measure the target prediction result. Since our model can avoid sign ambiguity problem, the loading and factor estimations were evaluated using its Pearson’s correlation with their true values.The fined in  includinExperiments were carried out to test the impact of noise Fig. , databas+ breast cancer. All samples came with gene microarray expression, ER status and survival time information. For the settings of the algorithm, we first manually selected a total of 15 TFs that are reported to be relevant to breast cancer  as shown in Fig.Along with the estimates of regulatory coefficients, the transcription factor activities and the sample cluster attributes were also obtained. Fig.We proposed a new approach to uncover the transcriptional regulatory networks from microarray gene expression profiles. We discuss next a few distinct features of it.First, to reflect the fact that a TF only regulates a small number of genes among the whole genome, the loading matrix of the factor model is constrained by a sparse prior , which dSecond, since the activities of TFs cannot be negative, the factors are modeled by a non-negative rectified Gaussian distribution , which nindicating that the rectified Gaussian model can also describe the possible suppressed state of TFs, which nevertheless cannot be modeled by the truncated Gaussian distribution. A comparison of Gaussian, rectified Gaussian, and truncated Gaussian is shown as Fig.To model the samples correlation due to, for instance, cancer subtypes, the samples are modeled by a Dirichlet process mixture (DPM), which imposes clustering effect among samples and can automatically determine the optimal number of clusters from data rather than be predefined in the algorithm. Forth, other types of data, such as ChIP-chip data -41 and DHowever, the proposed model is not without shortcomings. First, this model can not yet capture regulations from TFs that are not specified in the prior knowledge database. In reality, it is possible that some TFs that are not specified in the prior knowledge actually regulate the gene transcription. Second, the algorithm may not converge in a reasonable number of iterations on a large data set, thus cannot yet be applied to genome wide data set. Because the model parameters are high dimensional and highly correlated, the speed of convergence may significantly slow down on a large data set ,45. Howep, recall r of prior information and the sparsity of the loading matrix s is given, the prior probability of the g-th gene to be a target of the l-th TF gπ,l can be calculated as follows:Thirdly, the prior knowledge may still need to be properly evaluated. If the prior knowledge is considered an estimation of the true TRN, when the precision However, the precision or recall of the prior knowledge database are only arbitrarily specified (both 90%). In practice, the quality of prior knowledge should be evaluated first before more reasonable prior probabilities of regulations can be assigned.A Bayesian factor model that has sparse loading matrix, non-negative factors, and correlated samples, was proposed to unveil the latent activities of transcription factors and their targeted genes from observed gene mRNA expression profiles. By naturally incorporating the prior knowledge of TF regulated genes, the sparsity constraint of the loading matrix, the non-negativity constraints of TF activities, the regulation coefficients and TF activities can be estimated. A Gibbs sampling solution was proposed and model inference. The effectiveness and validity of the model and the proposed Gibbs sampler were evaluated on simulated systems. The proposed method was applied to the breast cancer microarray data and a TF regulated network for breast cancer data was reconstructed. The inferred TF activities indicates 3 patients clusters, two of which possess significant differences in survival time after treatment. These results demonstrated that the BNFM provides a viable approach to reconstruct TF mediated regulatory networks and estimate TF activities from mRNA expression profiles. The BNFM will be an important tool for not only understanding the transcriptional regulation but also predicting the clinical outcomes of treatment.


For RNA-seq data, the aggregated counts of the short reads from the same gene is used to approximate the gene expression level. The count data can be modelled as samples from Poisson distributions with possible different parameters. To detect differentially expressed genes under two situations, statistical methods for detecting the difference of two Poisson means are used. When the expression level of a gene is low, i.e., the number of count is small, it is usually more difficult to detect the mean differences, and therefore statistical methods which are more powerful for low expression level are particularly desirable. In statistical literature, several methods have been proposed to compare two Poisson means (rates). In this paper, we compare these methods by using simulated and real RNA-seq data.Through simulation study and real data analysis, we find that the Wald test with the data being log-transformed is more powerful than other methods, including the likelihood ratio test, which has similar power as the variance stabilizing transformation test; both are more powerful than the conditional exact test and Fisher exact test.When the count data in RNA-seq can be reasonably modelled as Poisson distribution, the Wald-Log test is more powerful and should be used to detect the differentially expressed genes. Recent advancements in deep sequencing technique enable the ultra-high-throughput sequencing  approaches, to be used for transcriptome, including gene expression, analyses -4. RNA-sIf there are biological replicates, the count data may have larger variances than expected from the Poisson distribution as biological replicates bring extra variances. Under this kind of situations, Poisson distribution with over-dispersed variances or negative binomial distributions are thought to be more appropriate ,9. Howev1 and X2 are both from Poi(λ), then given k = X1+X2, X1 has a binomial distribution Bin, where p = X1/k. Therefore the conditional exact test can be used to detect the difference of the two Poisson means. One of the advantages of the GLM is that it can incorporate some covariates that we are interested in. If no covariates are considered due to the small sample size and large number of genes, the LRT can be constructed directly. Similar to LRT, Wald test also has an asymptotical Chi-square distribution with df = 1 [Besides Fisher exact test and LRT from GLM, several other test approaches in statistical literature have been proposed to detect the difference of two Poisson means. For example, under the null hypothesis that Xh df = 1 -12. A moh df = 1 . Anotherh df = 1 ,13.In this paper, we first compare the above mentioned methods by using simulations. Since for those methods having the same asymptotic distribution, they are expected to have similar performances for large sample sizes , we focus on simulating Poisson distributions with small or moderate means. We then use a real RNA-seq data to show the performances of those methods.-3 to reflect the situation where in a study with a large number of significant variable (genes), a stringent p-value is needed. The estimated size and power are calculated from 10,000 runs for each setting.In the simulations, we first assume there are no replicates, i.e., each of the two conditions has only one sample. We assume the data for condition 1 is from a Poisson distribution with fixed mean 5, 10, 15, or 30. For condition 2, we assume the Poisson mean is the same as or greater than that for condition 1. We use the nominal significance level 10Table In the second simulation, we assume there are three replicates for each condition. Table From the results of the two simulation studies, we can see that the Wald-Log method is usually more powerful than the LRT, which has similar power as that of VST; both are slightly better than the Wald test, which in turn outperforms Fisher test and conditional binomial test. It is noticeable that both Fisher and conditional tests are exact conditional tests and have almost identical powers.We also apply these statistical methods to a real RNA-seq data generated by Illumina's sequencing technology . In thisWe are particularly interested in seeing how these statistical methods perform for low expression data. To this purpose, we use the data from genes with average count per lane between 1 and 10. There are 2657 such genes from the data. Figure To see how statistical significance  related to biological significance (fold changes), we plot the p-values  vs. absolute fold change  in Figure -3. Of the 581 selected genes by Wald-Log, 498 are also selected by LRT. Each of the selected gene has at least 20 ounts in total for at least one sample (liver or kidney). The fold changes  of all the 581 selected genes are all at least 2. For those 83 genes selected by Wald-Log but not by LRT, some even have fold changes as large as 4.Figure Table Some studies have shown that for RNA-seq data without biological replicates, the count data can be reasonably modelled as Poisson distributed ,3. SinceIt is interesting to see that the Wald-Log test is also related to the fold changes (see method section for more details); therefore it is also more biologically meaningful. The test statistic of Wald-Log not only uses the fold change , but also considers the variances associated with FC. For low expression level, the FC has larger variances, indicating it is more difficult to identify DEGs for low expression genes. This is consistent with the observation from other studies ,15: the k→∞, it becomes the fold change criterion as the denominator equals to 1. When the count data has an over-dispersed Poisson distribution, the above estimator underestimates the standard error and we will have inflated type I error rate. Under this situation, we need to derive a new estimator for the standard error; or a Bayesian approach assuming the standard error follows a specific distribution can be built. It should be noticed that for the Bayesian approaches, choosing appropriate prior distributions are critical and may have huge impacts on the results. We may also choose models other than the Poisson model. For example, if there are biological replicates, the count data may have over-dispersed variances, other models such as over-dispersed Poisson distribution or negative binomial distribution can be used [The denominator of the Wald-Log test statistic is:  be used ,9.When detecting the difference of two Poisson means, if the Poisson means are large, all the statistical tests mentioned above have very similar power. However, their powers may differ a lot when the Poisson means are small. When the count data in RNA-seq can be reasonably modelled as Poisson distribution, it is desirable to choose a statistical test which outperforms others for low expression genes. Through simulation study and real RNA-seq data analysis, we have shown that the Wald-Log test is more attractive than other methods and should be used to identify DEGs.1 and n2 technical replicates for condition 1 and condition 2, respectively; denote the count of short reads of a gene of the th jreplicate under condition i by xij, where i = 1,2, j = 1,2, ..., ni. We also assume random variables Xij~Poi(iλ).Suppose there are nThe test statistic of LRT is:where Under the null hypothesis that the two Poisson means are the same, the test statistic has an asymptotic Chi-square distribution with degree of freedom (df) equals to 1.LRT has been used by Marioni et al  based on1j given the total sum of Xij has a binomial distribution [It can be shown that the conditional distribution of the sum of Xribution ,16. More1+n2, p0).Under the null hypothesis, The Wald test statistic is :Wald has an asymptotic standard normal distribution.Under the null hypothesis, TThe Wald test can be also applied to the logarithm transformed count data . We callThe VST statistic ,13 has tFisher exact test is applied to the following 2 by 2 table Table :The exact p-value is then calculated based on the conditional hypergeometric distribution.The authors declare that they have no competing interests.C designed the algorithm, conducted the study, and drafted the manuscript; JL analyzed the data and assisted in programming; YD coordinated and directed the whole project. HKTN, SN, HLK, JYY and YD participated in the analysis and discussion. All authors read and approved the final manuscript.