As the barriers to incorporating RNA sequencing (RNA-Seq) into biomedical studies continue to decrease, the complexity and size of RNA-Seq experiments are rapidly growing. Paired, longitudinal, and other correlated designs are becoming commonplace, and these studies offer immense potential for understanding how transcriptional changes within an individual over time differ depending on treatment or environmental conditions. While several methods have been proposed for dealing with repeated measures within RNA-Seq analyses, they are either restricted to handling only paired measurements, can only test for differences between two groups, and/or have issues with maintaining nominal false positive and false discovery rates. In this work, we propose a Bayesian hierarchical negative binomial generalized linear mixed model framework that can flexibly model RNA-Seq counts from studies with arbitrarily many repeated observations, can include covariates, and also maintains nominal false positive and false discovery rates in its posterior inference.In simulation studies, we showed that our proposed method (MCMSeq) best combines high statistical power  with maintenance of nominal false positive and false discovery rates compared the other available strategies, especially at the smaller sample sizes investigated. This behavior was then replicated in an application to real RNA-Seq data where MCMSeq was able to find previously reported genes associated with tuberculosis infection in a cohort with longitudinal measurements.https://github.com/stop-pre16/mcmseq) best combined sensitivity and nominal error rate control.Failing to account for repeated measurements when analyzing RNA-Seq experiments can result in significantly inflated false positive and false discovery rates. Of the methods we investigated, whether they model RNA-Seq counts directly or worked on transformed values, the Bayesian hierarchical model implemented in the mcmseq R package (available at  As the cost of sequencing decreases, the complexity and size of RNA sequencing (RNA-Seq) experiments are rapidly growing. In particular, paired, longitudinal and other correlated study designs are becoming commonplace , 2. Longet al (2016) also noted that this strategy led to inflated false positive rates [Failing to account for correlation between repeated measurements can result in misleading inferences . For exave rates . In addi,9–14–As RNA-Seq data are over-dispersed counts, traditional linear mixed models, which assume a normal distribution, are inappropriate. In non-correlated settings, RNA-Seq data are typically modeled using a negative binomial (NB) distribution in a generalized linear model framework ,8. Since,The NB generalized linear mixed model (NBGLMM) is an extension of NB regression that allows for the inclusion of random effects to account for correlation between observations ,17, and There have been several recent developments in the analysis of correlated RNA-Seq data. For studies of related individuals, MACAU (Mixed model Association for Count data via data AUgmentation)  uses a P,26–PairedFB is a Bayesian model that can test for differential expression between two groups when there are paired observations . However22p-value, which can undergo common FDR corrections. These methods are implemented in the mcmseq R package, which fits models with a Markov-chain Monte Carlo (MCMC) algorithm that has been optimized in C++ and can be run in parallel to speed computation time. We illustrate our methods in the analysis of publicly available longitudinal whole-blood RNA-Seq data from subjects with latent tuberculosis (TB) infection.In this work, we propose to model correlated and longitudinal RNA-Seq data using a simple Bayesian NB hierarchical model. This model allows for the inclusion of random effects that can be used to account for correlation between samples, which may occur in studies where subjects are followed over time, in studies of related individuals, such as paired siblings, or when multiple samples are taken from the same subject . We use non-informative priors with large variances, making prior selection straightforward. If desired, information can be borrowed across genes when developing the prior for the dispersion parameter in a way that mimics the information sharing in edgeR and DESeq2. Unlike frequentist methods of estimation, estimates of variability for parameters and statistical tests account for the fact that dispersion parameters are estimated, resulting in better controlled false positive rates than frequentist NBGLMMs. We calculate a contour probability from the posterior distribution, a corollary to a two-sided Ygij be the expression of gene g from subject i at time point or observation j, thenSimilar to other commonly used RNA-Seq analysis tools, we model the observed data with a NB distribution using a log link function. To account for correlation, we allow random effects to be included in the model in a NBGLMM framework. Let μgij is the mean expression of gene g∈{1,⋯,G} in subject i∈{1,⋯,I} at observation j∈{1,⋯,ni} and αg is the dispersion parameter for gene g, which is related to the variance of Ygij by Var (Ygij) = βg is a p x 1 vector of fixed effect regression coefficients for gene g, Xij is a 1 x p row vector of fixed effects covariates for subject i at observation j, bgi is a q x 1 vector of random effects for gene g and subject i, Zij is a 1 x q row vector of random effect covariates for subject i at observation j, and ρij is an offset for subject i at observation j, which can be used to account for differences in sequencing depth between samples. A graphical representation of the hierarchical model is presented in Fig. Σg. In addition to accounting for repeated measures made on subjects in longitudinal studies, random effects can be used to account for correlation in other study designs, such as studies of related family members or paired siblings or studies in which RNA-Seq is performed on multiple types of samples (e.g. blood and biopsied tissue) from the same subject. This flexible framework also allows for the inclusion of both categorical and continuous covariates, where the latter can be modeled in a number of different ways .where βg, Σg, and αg:We fit the NBGLMM as a Bayesian hierarchical model using MCMC. The Bayesian NBGLMM is completed by specifying priors (and hyper-priors) for Mg is a vector of length p containing the prior means for the regression coefficients for gene g, T is a p x p covariance matrix, q x q scale matrix U and V>q−1 degrees of freedom. Ag is the prior mean of the log normal distribution for the dispersion parameter for gene g and B is the prior variance, which is common across genes.where Mg (the prior mean for the intercept) to be the log of the mean number of counts across samples for that gene; all other elements of Mg are set to 0. This prior is uninformative, as it assumes that observations come from a common distribution with a single mean and that there are no differences between groups or time points. For T, we use a diagonal matrix with s2 on the diagonal, and choose s2, the prior variance, to be large. We have used a prior variance of 72=49 for all analyses in this paper. The influence of these hyperparameters on inference is well known. Choosing non-zero means for regression coefficients is uncommon and we do not recommend it; this could result in increases in type 1 errors for truly null genes and could also increase type 2 errors for differentially expressed genes if the differential expression is in the opposite direction of the assumed prior mean. For the prior variance T, choosing smaller values increases the influence of the prior on inference; assuming the priors for the regression coefficients are centered at zero, this could lead to a loss of power. The influence of prior hyper-parameters on inference diminishes with increasing sample size.In practice, we set the first element of U as a q x q identity matrix and V=q+1. For models including a single random effect for each subject (i.e. q=1 as in our simulation study using a random intercept for each subject), we assume Σg has an inverse gamma prior with parameters U and V. The prior becomes more diffuse and uninformative as U and V decrease; therefore, we set these parameters to small values (e.g. 0.01) in order to be uninformative. We investigated the influence of choice of U and V in our simulation study in We specify mg, and variance, vg of the scaled counts for each gene. The naïve dispersion estimate for each gene is given by Ag is then set to the expected log(dispersion) at the average log(CPM) for gene g based on the LOESS regression. B is set to (k·τ)2 where k is an inflation factor and τ is the residual standard deviation from the LOESS fit. The method-of-moments dispersion estimates are over-estimates since they do not take into account fixed or random effects that explain variability in the data. Therefore, we typically choose k>1 in order to include a larger range of potential dispersion values in our prior. In practice, we have found a k of 2 tends to work well and use k=2 for all analyses presented in this paper (See k on testing characteristics). Type 1 error rates and FDR increase with k, such that values of k<2 tend to be more conservative and values of k>2 are more liberal. As sample size increases, the dispersion prior has less influence and all values of k perform similarly.To stabilize estimation of the dispersion parameter at small sample sizes, we share information across genes when specifying the log normal prior for the dispersion parameter in a similar spirit to the dispersion estimation in the DESeq2 pipeline . First, βg and bg; αg is updated using a random walk proposal and Σg can be updated using an appropriate Gibbs step. In this framework, the posterior distribution of regression coefficients naturally incorporates the uncertainty associated with simultaneous estimation of the dispersion parameter. Access to the full posterior distribution also makes inference about quantities that involve complex combinations of model parameters  straightforward [Posterior estimation is performed with a hybrid Metropolis-Hastings/Gibbs MCMC sampler  are calculated for all model parameters, and are then adjusted for multiple comparisons across genes using the Benjamini-Hochberg method [Convergence of the chain for each gene is examined in two ways: acceptance rate and Geweke statistics comparing the first 20% of the chain (after discarding burn-in) to the final 50% of the chain . For eacg method . We deemp-value for the null hypothesis that a parameter θ equals a given value θ0 , there are multiple potential approaches. One option is to use Bayes factors to compare the likelihood of a given parameter value in the prior versus the posterior distribution. A major drawback of this approach is that adjusting for multiple comparisons when using Bayes factors is not well characterized. Alternatively, we use the contour probability originally proposed by Box and Tiao , which ce θ0 See  Fig. 1. pc in this scenario is to construct a smoothed approximation of the posterior distribution using kernel density estimation with the MCMC sample {θ(1),⋯,θk}, and to then estimate pc as the proportion of samples for which p(θ0|Y)>p(θk)(|Y). However, the results are dependent on the choice of kernel function and smoothing bandwidth [,Since the NBGLMM does not have a closed-form posterior distribution, Monte Carlo estimation is required. One strategy for computing andwidth ,35. To aI is an indicator function and θ. That is, we estimate the contour probability as twice the proportion of MCMC samples that lie on the opposite side of 0 as the posterior median, which is twice the posterior probability of the null in a one-sided hypothesis testing framework  for which the average posterior probability of being differentially expressed is above some threshold  . In thistion see  Table 2,d Berger . Moreove uniform  Fig. 3. –The MCMSeq method is implemented in the mcmseq R package, which contains a detailed vignette that illustrates these methods on a simulated data set. The package is written in C++ via the Rcpp, RcppArmadillo, and RcppParallel R packages, the last of which allows the chains for each gene to be automatically run in parallel to shorten computation time –40. In tWhile MCMC samplers for Bayesian hierarchical negative binomial models can be developed and fit with other packages or software, such as STAN , OpenBugTo evaluate the testing characteristics of MCMSeq and to compare to other potential analysis strategies, we performed a simulation study. Datasets were simulated using a two group design (e.g. treatment and control) with paired observations (e.g. baseline and follow-up) on each subject. Count data were simulated from a NB distribution as follows:i is in the treatment group, j is the follow-up, and bgi is the random intercept for gene g and subject i. Simulated data sets were generated with a combination of null and differentially expressed (mixed effect sizes) features, as summarized in Table βg0, αg, and αg), and random intercept variance ,where mentclass2pt{minimmentclass2pt{minimm samples must have a CPM >1 where m is the number of samples in the smallest experimental unit of interest). We also tested the sensitivity of our simulation results to additional filtering schemes and found similar testing characteristics and relative performance of the methods. See bgi from Eq. βg0) and the main effect of treatment (βg1) from the model, as they are not estimable. For all other methods, both fixed and random effects were included in the model as specified in Eq. We analyzed each simulated data set using the methods described in Table βg1+βg3=0 tests for differences in expression between the treatment and the control group at follow-up. We refer to this as a “between-subject” test, as inference involves only independent observations. Second, βg2+βg3=0 tests for a change in expression over time in the treatment group. We refer to this as a “within-subject” test as inference involves the paired observations. Lastly βg3=0 tests if there is a difference in the change in expression over time between the treatment and control groups. We refer to this as an “interaction” test, which depends on both between- and within-subject information. For each simulation scenario, we calculate type 1 error rate, FDR, and power for each contrast within each dataset and then average across the 10 datasets. We evaluate these characteristics over a range of significance thresholds  for three statistical tests. First, h dataset. HoweverWe apply the analysis strategies described above to a publicly available longitudinal RNA-Seq dataset of 112 whole blood samples from 40 subjects in recent contact with patients with active TB infection . The cohort was recruited between September 2015 and September 2016 at the Glenfield Hospital, University Hospitals of Leicester NHS Trust, Leicester, UK. The study design and methods have been described in detail elsewhere [γ) release assay (IGRA) and were classified as having latent TB infection (LTBI). The remaining 15 subjects tested negative on the IGRA and were classified as controls. Previous analyses of the data evaluated a risk score based on a panel of 20 genes designed to distinguish LTBI from TBP and found that among subjects that remained healthy over the course of the study, risk scores were stable over time, while for most subjects that developed active TB, risk scores increased over time. In addition, it was noted that at baseline (first study observation after recruitment) the risk score was, on average, slightly elevated in the LTBI and TBP groups compared to the control group [Over the course of the study, 9 subjects developed active TB infection and were classified as TB progressors (TBP). Of the remaining 31 subjects, 16 tested positive on the M. tuberculosis antigen-specific interferon-gamma  were dropped, which reduced the total number of genes analyzed to 15,993.The goal of our analysis is to compare longitudinal changes in gene expression between subjects that develop active TB infection and others. As the number and timing of observations differed greatly between subjects, we chose to include time as a continuous predictor in our models. Since there were relatively few observations per subject, with 80% of all subjects having 3 or fewer observations and with 20% of controls, 56% of progressors and 44% of LTBI having 2 or fewer observations, we decided to model time using a linear effect. For studies with larger numbers of repeated measures, regression splines or polynomials could be used to model potentially non-linear changes over time. We included group by time interactions in the model to allow the 3 groups to have different changes in expression over time, in addition to different expression at baseline. We used the following basic regression model:tij is the number of days since baseline for subject i at observation j, i is in the LTBI group, i is in the TBP group, bgi is the random intercept for gene g and subject i, and ρij is an offset. In this model, g in the control group at baseline, βg1 is the log(fold change) comparing the LTBI group to the control group at baseline, and βg2 is the log(fold change) comparing the TBP group to the control group at baseline. βg3 is the change in log expression per day for the control group, βg4 is the difference in the change in log(expression) per day between the LTBI and controls groups, and βg5 is the difference in the change in log(expression) per day between the TBP and controls groups. Additional contrasts of interest include βg3+βg4 (change per day in the LTBI group), βg3+βg5 (change per day in the TBP group), βg2−βg1 (difference between TBP and LTBI groups at baseline), and βg5−βg4 (the difference in change per day between TBP and LTBI groups). All models were fit as described in the “p-values for all methods except ShrinkBayes, which reports a Bayesian FDR [where sian FDR . Numbersp-value of less than 0.001.To better understand the genes that significantly changed over time in the TBP group, we performed functional enrichment analysis on the differentially expressed genes (DEGs) identified with the MCMSeq method using Panther 14.0 with the GO Biologic Process Complete annotation database. We used a 0.05 FDR level to identify GO terms that were significantly enriched. We further filtered results to include only GO terms with at least 50 genes, ≥ 10% overlap of the GO terms with the DEG list, and a raw ,In addition, cross-sectional microarray ,48 and R% to ≈20% of all genes tested at a given sample size. DESeq2 ignoring correlation had minor convergence issues at smaller sample sizes. MCMSeq had minor issues at the largest sample size, but the number of genes affected was <1%. Features that failed to converge in both MCMSeq and DESeq2 ignoring correlation shared similar characteristics: small counts, large dispersion, large random intercept variance, repeated zero values, and fewer unique count values. The NBGLMM had the opposite pattern, with the genes that failed to converge tending to have larger counts, smaller dispersion, and smaller random intercept variance. For the LMM, there were no obvious distinguishing features of the genes that failed to converge , and brings the type 1 error rates very close to nominal levels for the MCMSeq and LMM methods. Overall, similar results were seen for the interaction term.For “within-subject effects,” such as changes over time within a single group, NBGLMM and DESeq2 including subject as a fixed effect have severely inflated type 1 error rates. Conversely, edgeR and DESeq2 ignoring correlation are overly conservative, with observed rates below the nominal level. edgeR including subjects as fixed effects shows a mix of conservative and anti-conservative behavior. Again, MACAU has inflated rates at smaller sample sizes but is overly conservative at The results for CPGLMM and VarSeq are presented only in the supplementary materials as the results for the CPGLMM were virtually identical to the NBGLMM, and VarSeq either found no significant features or had highly inflated error rates.Figure n=3 per group, but conservative behavior at n=10 per group. At all sample sizes and testing thresholds, the observed FDRs for the NBGLMM were inflated. ShrinkBayes is conservative at small sample sizes, but has FDR inflation at 10 subjects per group, showing worse performance with increasing sample size. MACAU has similar behavior to its type 1 error profile, with FDR inflation at n=3 and very conservative behavior at the largest sample size. Again, limma performs best at n=3 and has worsening FDR inflation with increasing sample size. The LMM and MCMSeq again came closest to the nominal levels with both tending to be slightly conservative at the lower sample sizes.When testing within-subject and interaction effects, DESeq2 and edgeR ignoring correlation are conservative, especially at the larger sample sizes. However, when including subjects as fixed effects, DESeq2 has severely inflated FDRs, while edgeR displays inflation at Power  for the various methods can be seen in Fig. n=3 to n=10. For limma, power is typically about average for all of the methods, in spite of having significant type 1 error and FDR inflation at N=5 and N=10. For both the within subject and interaction tests, the relative performance of the MCMSeq method improves with increasing sample size. Again, at 10 subjects per group, the only methods with higher power than MCMseq fail to control type 1 error rates and/or FDRs at their nominal levels.For within-subject tests, ShrinkBayes has the lowest power, followed by edgeR ignoring correlation and the LMM. The conservative behavior of edgeR may be driven by failing to account for the paired structure of the data, while the poor performance of the LMM may be driven by the large number of models that failed to converge. For the interaction test, at 3 and 5 subjects per group, ShrinkBayes has the lowest power, but at N=10 per group has the highest power . The power for MACAU remains relatively constant across the various sample sizes, increasing only marginally from n=5 simulated datasets at the commonly used 0.05 FDR level (See N=3 and N=10). Here we see that for each type of test, MCMSeq has the highest power of the methods that maintain the nominal FDR level. Again, the only methods that have higher power also have substantially inflated FDRs.Figure % of MCMSeq models failing to converge. For MCMSeq, we also ran the sampler four separate times to evaluate the consistency of the results, as this method is based on a stochastic estimation process. In general, we found the results were highly consistent between the multiple runs, as the Spearman correlation of p-values was at least 0.9996 between any combination of runs, and typically over 99% of differential expression calls for the contrasts we tested were congruent. More details can be found in The NBGLMM had the largest number of models that failed to converge (≈8.4%), followed by the LMM (≈2.5%). Convergence rates were higher in the TB dataset compared to our simulation studies. This difference is likely due to the larger number of observations (112) and subjects  compared to our simulations, which included a maximum of 40 observations. To explore this, we subsampled 2 observations from 5 subjects in both the Control and Latent groups in the TB dataset to create a scenario similar to our 5 subject per group simulations. We found that convergence rates were similar to our simulation study, with 18.1% of LMMs, 17.6% of NBGLMMs, 0.4% of DESeq models, and <1For brevity, we focus our discussion on the edgeR, edgeR*, limma, LMM, MCMSeq, NBGLMM and ShrinkBayes methods; however, full results for all methods can be found in the For within-subject tests of changes in gene expression over time, changes in gene expression were expected in the TBP group, which developed active TB over the course of the study. Expression was expected to be relatively stable in the control and LTBI groups. As expected, most methods identified few or no genes that significantly changed over time in both the control and LTBI groups. However, ShrinkBayes found nearly 3,000 genes with significant changes over time in these groups. For the TBP group, ShrinkBayes found the largest number of significant genes, with over 5,000 more DEGs than NBGLMM and limma, which had the next highest numbers. In our simulation studies, the NBGLMM and limma had inflated FDRs, even at 10 subjects per group. The large number of additional significant genes found with ShrinkBayes may reflect the pattern of increased FDR with increasing sample sizes noted in the simulation study. For the TBP group, edgeR identified fewer genes than the MCMSeq and LMM methods. edgeR ignoring correlation identified fewer genes than when subjects were included as fixed effects, which also aligns with our simulation results showing that failing to account for correlation caused DESeq2 and edgeR to produce conservative results for tests of within-subject effects. The results for the interaction effects (differences in change over time between groups) largely follow the same patterns as the within subject tests for change in gene expression over time.We also calculated pairwise Jaccard indices and Kappa agreement statistics to compare the similarity of differential expression calls between methods  or inflated type 1 error rates and FDRs (including subjects as fixed effects). While limma performed relatively well at small sample sizes, false discovery and type 1 error rates became increasingly inflated as sample size increased, having one of the highest levels of error inflation in our Both the NBGLMM and CPGLMM consistently had highly inflated type 1 error rates and FDRs when fit under a frequentist paradigm. This could, in part, be a result of the statistical tests for these models being designed for larger sample sizes, while many of the other RNA-Seq specific methods we examined have some modifications due to the expectation of low sample sizes. Lastly, ShrinkBayes is similar to MCMSeq in that both take a Bayesian approach to fitting hierarchical NB models. However, ShrinkBayes had mixed behavior, with a lack of power at the smallest sample size investigated that transitioned into inflated FDRs at the largest sample size. The Bayesian FDR used in ShrinkBayes was originally validated in a simulation using a 2 group comparison with N=8 per group, without correlation or repeated measures . While tMCMSeq did struggle with power at the smaller sample sizes for the between-subject test. This is not unexpected as this type of test does not require the extra complexity of the mixed model since there are no repeated measures involved in this contrast. Of the methods examined, MCMSeq was best able to adequately control type 1 error rates and FDRs while maintaining good power for the tests that involved changes over time.The results from applying the various methods to the TB data set largely mirror the behavior observed in the simulations as the methods that generally had inflated error rates tended to find the largest number of significant features . Methods that showed conservative behavior at larger sample sizes, such as VarSeq and MACAU, failed to find any genes that significantly changed over time. For the two methods that were best able to maintain type 1 error rates in our simulation study (MCMSeq and the LMM), MCMSeq found more significant genes for tests of changes over time and differences in changes over time between groups, which aligns with the results of our simulation study, where MCMSeq had more power than the LMM.In our application, ShrinkBayes was an outlier, especially for the “changes over time” and “differences in changes over time” contrasts. For changes over time in both the control and latent groups, all of the other methods find no or very few genes showing significant changes. However, ShrinkBayes finds about 3,000 genes for both tests. The same is true for comparing the change over time in the latent group to the change over time in the control group. ShrinkBayes also finds several thousand more significant genes than all the other methods for changes over time in the TB progression group. This could be a continuation of the behavior seen in the simulation study where ShrinkBayes had significant FDR inflation at the largest sample size, which is smaller than the number of subjects in the TB dataset. However, as others have noted, ShrinkBayes produces a large number of integrated nested Laplace approximation (INLA) errors during model fitting which the ShrinkBayes documentation instructs users to ignore . Therefon=10 simulated data set), our implementation available in the mcmseq R package (196 min) was actually faster than fitting the frequentist NBGLMM (803 min) or the CPGLMM (492 min). It is also faster than ShrinkBayes (474 min), which uses Laplace approximations rather than MCMC, but involves computationally expensive steps needed to estimate the more complicated priors utilized in their model. Moreover, the fitting function in mcmseq allows the user to specify contrasts so as to test combinations of the regression parameters. The frequentist NBGLMM and CPGLMM both had to be fit multiple times to test all of the contrasts we were interested in for both the relatively simple design of the simulated data sets and the more complicated study design of the TB data. ShrinkBayes also suffers from this issue and had to be fit twice for the simulations and three times for the TB data, leading to much longer run times to complete the whole analysis. Additionally, due to the way genes are analyzed and summarized in parallel, MCMSeq does not require large amounts of RAM to be run. For example, the analysis of the TB dataset required only around 300 MB of RAM at its peak consumption  can be calculated. Since a Monte Carlo estimate is used to compute the tail probability, the precision is determined by the number of MCMC iterations that are run. We found that chains of length 30k were generally sufficient for type 1 error and FDR control, but if greater accuracy is needed the only solution is to run longer chains. Using normal approximations for the posteriors of regression coefficients was investigated, but this strategy was unable to maintain nominal error rates, even at n=10. We also explored using Integrated Nested Laplace Approximations (INLA) as an alternative approach to fitting the Bayesian hierarchical model utilized in MCMSeq. Using the INLA R package we found that both type 1 errors and FDR rates were inflated, similar to the inflation we noted when using normal approximations based on the MCMC output. This is not surprising as INLA is an approximate method as compared to MCMC, which is asymptotically exact. While the custom sampler implemented in the mcmseq R package was faster than INLA at the sample sizes we tested, we are interested in further exploring both INLA and variational methods in future work. Though our motivation for developing MCMSeq was to create a tool for analyzing longitudinal RNA-Seq experiments, we have also included functions in the mcmseq R package that can fit NBGLMs that do not include random effects. In a similar set of simulations that did not include random effects, we found that using MCMC to fit a NBGLM gave similar improvements for maintaining nominal FDRs and type 1 error rates, likely due to incorporating uncertainty in the dispersion parameter into the posteriors of the regression coefficients (data not shown).While in the future we intend to extend the package to allow for multiple levels of clustering and additional random effects, such as a random slope, the current version should prove adequate for many studies with repeated or paired measurements. We also hope to add features allowing users to simultaneously test multiple fixed effects regression parameters (i.e. ANOVA type tests). Another limitation is the precision with which the MCMC Failing to account for repeated measurements when analyzing RNA-Seq experiments can result in significantly inflated false positive and false discovery rates. The most used RNA-Seq analysis software cannot inherently account for repeated measures, and though some workarounds have been proposed, our simulations show they are not particularly effective. Ultimately, the Bayesian hierarchical model underlying MCMSeq was best able model RNA-Seq expression from study designs with repeated/clustered measurements based on its ability to combine good power with maintenance of nominal false positive and false discovery rates. Additionally, MCMSeq’s NBGLMM framework is able to flexibly model mean expression with the inclusion of covariates, and it can handle arbitrarily many repeated measurements which can also vary in number between individuals. Finally, the implementation found in the mcmseq R package makes utilizing Bayesian NBGLMMs more accessible by automating and parallelizing the process of fitting and summarizing results from these models over the tens of thousands of genes typically analyzed in an RNA-Seq experiment.Additional file 1 Supplementary materials. Supplementary methods, results, tables, and figures.


A novel multi-classification method, which integrates the elastic net and probabilistic support vector machine, was proposed to solve this problem in cancer detection with gene expression profile data of platelets, whose problems mainly are a kind of multi-class classification problem with high dimension, small samples, and collinear data. The strategy of one-against-all (OVA) was employed to decompose the multi-classification problem into a series of binary classification problems. The elastic net was used to select class-specific features for the binary classification problems, and the probabilistic support vector machine was used to make the outputs of the binary classifiers with class-specific features comparable. Simulation data and gene expression profile data were intended to verify the effectiveness of the proposed method. Results indicate that the proposed method can automatically select class-specific features and obtain better performance of classification than that of the conventional multi-class classification methods, which are mainly based on global feature selection methods. This study indicates the proposed method is suitable for general multi-classification problems featured with high-dimension, small samples, and collinear data. Cancer detection plays a vital role in improving the survival rate and living quality of patients. Methods of detecting cancer mainly include clinical symptom analysis, pathological analysis of cancer tissue, radiology, biochemical examination and molecular diagnosis, etc. At present, using the approach to detect the gene expression profile data has emerged as a potential method for detecting cancer. For example, it is reported that the gene expression profile data of platelets can be used to detect cancer ,2. Platep, small n” problem. Besides, the gene expression profile data is collinear as multiple genes tend to function together, e.g., in the shared biological pathways, and thus the correlations in expression level among those genes can be high. As for collinear data, the elastic net is able to select groups of features, rather than individual features, which are highly correlated.Cancers are complex diseases, and are related to many etiological factors, especially in the early stage of cancer development ,5. It is problem ,9,10,11,assifier ,17,18,19assifier ,21 naturIn the field of machine learning, researches on the feature selection method for the multi-classification problem is mostly based on global features. It is actually the same case for cancer detection with gene expression profile data. In addition to the global feature selection method, there is another way to select features for the multi-classification problem, which is the class-specific feature selection method. There are few studies investigating on solving multi-class classification problems from the angle of the class-specific feature selection method. The class-specific feature selection method is related to the one-vs-all (OVA) strategy , which iIn this study, it was focused on the problem of how to use the class-specific feature selection method to enhance the performance of the classification algorithm in cancer detection with gene expression profile data. In this work, it was presented a new multi-class classification algorithm, which integrates the elastic net and probabilistic support vector machine. The OVA strategy was used to decompose the multi-class classification problem of cancer detection with gene expression profile data into multiple binary classification problems. The elastic net was conducted to select class-specific features for the obtained binary classification problems, on consideration of the characteristics of the elastic net and gene expression profile data. Besides, the probabilistic support vector machine was employed to construct the binary classifiers with the selected class-specific features. The probabilistic support vector machine inherits advantages of support vector machine (SVM), such as excellent generalization ability for the problem with nonlinear, small sample and high dimension. The probabilistic support vector machine converts the decision output of SVM into the form of probability, which makes the outputs of the binary classifiers based on the class-specific features comparable. The simulation data and gene expression profile data were obtained to verify the effectiveness of the proposed method and the results were compared with the conventional methods, such as SVM classifier combined with global feature selection methods, including Relieff and SVM-RFE ,24,25.The blood-based liquid biopsies belong to a noninvasive cancer detection technology receiving increasing attention nowadays. The blood plate is an excellent biomarker among various biomarker sources for blood-based liquid biopsies. The mRNA profiles of tumor-educated blood plates have been used to train classifiers to detect cancers. This paper proposed a new idea that embedding elastic net in classifiers for solving the problem of detecting cancer using tumor-educated platelets dataset. We adopt classifiers of SVM and probabilistic SVM to implement this idea, named as ESVM and EPSVM respectively.The proposed classifiers are applied to detection cancer with the transcriptomic data derived from tumor-educated platelets from individuals. The experiment results show that the proposed methods have the advantage of automatically selecting the feature subsets to improve the generalization ability of classifiers comparing to the conventional multi-classification methods. The rest of the context is organized as follows: the related theories are described in SVM ,27,28 isIn doing that, it seeks the support vectors to support the hyperplane, using training data However, the standard SVM classifier does not provide such probabilistic output. To address this need, the output of the SVM classifier has been calibrated into a posterior probability. Here, a typical probabilistic SVM algorithm (PSVM) was introduced in briefly ,30.With the data The feature selection methods can be divided into two categories: filter algorithms and wrap algorithms ,13,14,31Relieff ,33 is a SVM-RFE  is a repLet lty term .With  (LASSO) ,36, whicgression , which uRelieff and SVM-RFE are based on ranking methods, which produce a series of candidate nested feature subsets according to the threshold values. The elastic net is based on the sparse attribute of the L1 penalty term, which can form a set of candidate feature subset directly according to the values of The multi-class classification problems can be decomposed into a series of binary classification problems by OVA strategy, where one class is treated as the positive class and the rest classes are treated as the negative class. These binary classification problems are solved with global features in general. However, for a special positive class, the global features may not be the best class-specific features. If the class-specific features of positive classes are similar, then using the global features are suitable for these binary classification problems. However, if the class-specific features of positive classes are very different from one another, then using class-specific features to solve these binary classification problems will achieve better results than using global features. The idea of solving the multi-class classification problems was demonstrated by using class-specific features by the OVA strategy in As shown in Using gene expression profile data for detecting cancers is a kind of multi-class classification and these data encounter the following problems: a large number of highly correlated features and limited data samples. Considering the characteristics of expression profile data and class-specific feature selection method, an idea of embedding the elastic net was proposed in the probabilistic SVM to improve the effect of cancer detection, and the working principle is discussed as follows. For the multi-classification problem, the OVA strategy was used to decompose the initial problem into multiple binary classification problems. For each binary classification problem, the elastic net is built using the class label as the dependent variable and features as independent variables. A series of Using the OVA strategy to deal with the multi-class classification problem, the predicted class is taken from the binary classifier with maximum confidence:The simulation data and gene expression profile data were provided here, to verify the effect of proposed EPSVM. The SVM classifier combined with global feature selection methods, such as SVM-RFE and Relieff is used to evaluate the performance of EPSVM. The SVM and PSVM are taken from LIBSVM . All calThe performance of the proposed algorithm is related to the number of classes. Therefore, the simulation dataset is designed to gradually increase the number of classes to continuously show the advantage of the proposed algorithm. The simulation dataset includes nine data subsets produced in the following way. The data p > 0.001) [The gene expression profile data of platelets was measured by RNA-Sequencing technology ,40, whic> 0.001) ,43. Tablhttp://www.gems-system.org/ and their information is shown in The microarray technology  is widelNER) is taken as an evaluation index, which is defined asi-th binary classification problem obtained by the OVA strategy and For each data, two-thirds of the samples in each class are randomly chosen to form the training set and the rest samples are reserved for independent testing. In the training process, the parameters are set as follows. We adopt the Gaussian kernel for the classifiers of SVM and PSVM and use the grid search method to find the best parameter setting of sitivity  and specsitivity  of the iAccording to the generating method of simulation data, the features of the data In this work, it uses one experiment of data 1 to show the feature selection results of the three methods. The classification results of the simulation data are shown in The performance of the elastic net for the binary classification problem with gene expression profile data is a key factor for the success of the proposed method. We choose the dataset 10 (HD+BrCa) to show the effect of the elastic net comparing with the other two feature selection methods in detail. According to the ranking of p-values, we gradually increase the dimensionality of the feature dimension to test the performance of the three feature selection methods. We plot the first two principal component scores of dataset 11 with raw features and selected features in one experiment in The OVA strategy comes with an inherent flaw that the positive class and negative class are imbalanced ,46. TablBesides, the penalty parameter We also use microarray data to further demonstrate the validity of the proposed method. In general, the gene expression profile data has the problems of high dimension, small sample and collinearity. With more classes, the classification problem becomes harder to be dealt with by the global features. The proposed EPSVM can achieve better effects of cancer detection with gene expression profile data than the traditional method based on global feature selection methods.In this work, a novel multi-class classification method was proposed to detect cancer diseases with the gene expression profile data, which has the problem of high dimension, small sample, and collinearity. In the proposed method, the class-specific feature selection, OVA strategy and probabilistic support vector machine were innovatively combined to enhance the effect of cancer detection with gene expression profile data. The experiment results show that the proposed method can achieve better classification results than conventional methods based on global feature selection methods. So that the proposed method is suitable for the problem of cancer detection with gene expression profile data.In general, the proposed method is suitable for multi-class classification problems, where the class-feature set of each class is significantly different from others. Besides, the idea behind the proposed method in this paper can be easily extended to the combination of other feature selection methods and classifiers, which is flexibly in solving the multi-feature and multi-class classification problems encountered in practice. However, for the adopted classifiers, how to normalize the outputs of classifiers with different feature subsets is a key problem. The following work will further study the effect of different normalization methods on the class-feature based multi-class classification method.


Gene expression data obtained in large studies hold great promises for discovering disease signatures or subtypes through data analysis. It is also prone to technical variation, whose removal is essential to avoid spurious discoveries. Because this variation is not always known and can be confounded with biological signals, its removal is a challenging task. Here we provide a step-wise procedure and comprehensive analysis of the MINDACT microarray dataset. The MINDACT trial enrolled 6693 breast cancer patients and prospectively validated the gene expression signature MammaPrint for outcome prediction. The study also yielded a full-transcriptome microarray for each tumor. We show for the first time in such a large dataset how technical variation can be removed while retaining expected biological signals. Because of its unprecedented size, we hope the resulting adjusted dataset will be an invaluable tool to discover or test gene expression signatures and to advance our understanding of breast cancer. Laurent Jacob et al. develop a workflow and analytical pipeline to remove technical variation from the MINDACT microarray dataset. Their method preserved biological signals and the normalized datasets can be repurposed for the discovery of other biomarkers and signatures for breast cancer. Transparency and managing the potential influence of these technical factors on expression profiles are critical to avoid spurious conclusions3, an international, multi-center, prospective, phase III randomized trial whose main objective was to prospectively validate the 70-gene MammaPrint signature as a prognostic tool for the clinical evaluation of patients to help avoid chemotherapy4. Trial results showed that the 70-gene expression signature enables a reduction of 46% in the use of chemotherapy for clinically high-risk patients4, a finding that was adopted in various international breast cancer guideline recommendations9.The largest trial that collected full-transcriptome expression data is the MINDACT trial12. In addition, the risk classification using the 70 genes is binary and is a result of the combination of the expression of the 70 genes. Here, we are studying all other genes and the procedures to monitor and guarantee the risk classification of MammaPrint, however, did not monitor or guarantee that the full-transcriptome data was unaffected by technical variation.During the 5-year enrollment phase of MINDACT, the measurement of the 70-gene expression signature was performed under US Food and Drug Administration Quality System Regulation (FDA/QSR)-compliant protocols, which included technical adjustments and improvements Fig. . As such16 could be used that handle the unwanted technical variation when studying a factor of interest , however, one should be aware that analyses  can be misled by unwanted variation, and prior adjustment of the data is warranted. Deciding what should be adjusted for is, however, a challenge, as yet unknown subtypes may be confounded with some technical factors and adjusting for the technical factors may discard the unknown subtypes. Identifying the technical variation is a non-trivial task, as part of it may be caused by undetected factors26. Accordingly, the objective of this paper is twofold. First, we aim to describe the unwanted variation affecting the expression data of this large MINDACT dataset. Principal component analyses suggested that time-varying technical factors affected the measured gene expression, but still showed the expected biological signals related to the main breast cancer subtypes. Secondly, our aim was to build an adjusted version of the expression dataset for future analysis purposes. We relied on an adaptation of the random naive RUV algorithm27 to remove variation related to time-localized technical factors and mediate the variation caused by unobserved technical factors, without losing biological signals.Initial steps in standard microarray procedures and as employed for this dataset based on a two-color method, convert raw feature extracted image data to biological significant data on a per gene-probe measurement at full-transcriptome level. This includes a multi-step process involving quality control per probe, as well as normalization steps that correct for differences in sample labeling efficiency or the amount of RNA used. These processes are performed per array , however, they do not correct for variation caused by technical changes over time . Statistical methodsAfter adjustment, the top principal components are stable over time, and correlate with expected biological signals. The analysis pipeline is entirely reproducible as we provide the R scripts that were used to generate the tables and figures of this paper. The adjusted and unadjusted breast cancer datasets with unprecedented sample sizes are available through the trial sponsor, the MINDACT committee of European Organization for Research and Treatment of Cancer (EORTC), and ready-to-use for translational research, including discovery of new classifiers or testing of proposed signatures. Additionally, the step-wise analysis pipeline we describe provides guiding principles for the analysis and management of technical variation and big data processing of other large cohort studies spanning extended time periods of patient enrollment.4 (6693 patients) were processed at Agendia between February 2007 and September 2011, using customized full-transcriptome two-color Agilent microarrays that contained the MammaPrint probes , and successive steps as depicted in Fig. 10-ratio values. During the MINDACT study three different RNA pools were used as reference . The full-transcriptome expression data of five patients was excluded based on poor quality, resulting in 6688 full-transcriptome study samples. In addition, between patient sample correlations were calculated to assess whether some samples may have been accidentally repeated . Expression values of the dual channel patient plus reference sample microarray hybridizations are presented as logn = 5182); more precisely as three different RNA references were used, missing values were determined per RNA reference group (see “Methods”), leaving 35,611 probes per patient for further analysis. The remaining missing values were imputed, and the data were zero-centered for each probe in each of the three reference groups to allow for combining the three sets of samples with different references for further analyses (see “Methods”). The total dataset after missing value imputation consists of 6688 samples with 35,611 data points.For the 6688 patient samples , the number of missing values was determined  of the unadjusted data showed a sharp variation over time Fig.  and thisER, HER2, CLDN, and BRCA (see “Methods”). PC2 separated samples with low and high ER signals, thereby indicating the presence of the expected ER- and ER + subgroups28 .PC3 projections, similarly to PC1, showed sharp variation over time Fig.  that wasnal Fig.  with a PBRCA signal, and −0.29 with the CLDN signal.Projections of PC4 suggest an association with mainly biological factors Figs. . This PC29 (see “Methods”).Overall, the distribution of both PC1 and PC3 fluctuated sharply over time. While projections for PC2 and PC4 also fluctuated somewhat over time, their variation was smoother than for PC1 and PC3. The three time-localized technical factors explaining PC1 and PC3  were also the ones with largest overall association with the four PCs  and RNA integrity number (r = 0.42). Importantly, these other factors were not time-localized, making it impossible to decide if they were confounded with unobserved biological factors of interest (tumor cell percentage for example could be higher in subtypes characterized by denser tumors). In general, PC1 and PC3 mainly corresponded to known technical variations and to some extent to biological signals, while PC2 and PC4 mainly corresponded to expected biological variations and seemed to be affected by smaller time-localized signals.The first four PCs were also associated with a milder extent with other technical factors . Since these technical factors clearly fluctuated over time, removing their effect was unlikely to accidentally remove biological signals. For the same reason, we chose not to explicitly adjust for other technical factors, which were not time-localized. In comparison, the regular method27 led to less satisfactory results and poor adjustment of data  were still present   of the unadjusted dataset was used as a proxy for the immunohistochemical staining. We compared TP-readout of the dataset with ER and HER2 signals before and after data adjustment. Analyses showed high and similar correlations  showed increased association with the top PCs. The Pearson correlation of ER signal and PC1 increased >4-fold to 0.85 in the adjusted data  and the expression signals of CLDN-associated genes (−0.36). The CLDN signal is even stronger on PC2 (−0.41) as illustrated in Fig. ER+/ER–, HER2+/HER2–, CLDN low/high). The expression of BRCA is associated with PC2 (0.34) and PC3 (0.35) as illustrated in Fig. TP53 is associated with PC3 (−0.43). Projections of the 6688 samples against the first four PCs and color-coded for BRCA, CLDN, ER, HER2, and TP53 are presented in Supplementary Fig. After adjusting for technical variation, we expected the known biological signals to explain most of the variance, which should be reflected by an increased association with the top PCs in the adjusted dataset. Consistent with this, we observed an increased Pearson correlation between PC1 and both ts Table ). Similanal Fig. , correspset Fig. , therebyIn summary, variation introduced by time-localized technical factors was largely removed after adjustment while biological signals were preserved.Gene expression data from studies that span multiple years naturally include variation due to technical factors, even when sources of variation are minimized by standardizing laboratory procedures and using the same batch of reagents for the duration of the study. The MINDACT study spanned several years and here we describe the MINDACT full-transcriptome microarray expression dataset and show for the first time that such a large dataset is adjusted for technical variation while preserving biological signals.12 that are under diagnostic quality schemes, by using specific procedures and validations. However, its management for gene expression of the full-transcriptome is of utmost importance to avoid spurious discoveries in future analysis that may be the result of technical variation. Accordingly, the goal of this paper was to describe this technical variation, and to prepare a ready-to-use dataset for future breast cancer relevant projects by adjusting explicitly for the technical variation that could be safely modeled. Adjustment using a modified version of the naive random RUV method27 implicitly removes technical variation due to known as well as unidentified sources, while preserving biological signals.This technical variation is corrected for in the result of diagnostic tests such as MammaPrint or BluePrintER+, ER-/HER2 + and ER–/HER2-groups along the PC1, and the next principal components had a high correlation with the expression of TP53, BRCA, and CLDN-associated genes. While it is impossible to guarantee that all unknown signals of interest were preserved by the adjustment, it is encouraging to see that known important signals, which were not modeled in the adjustment were preserved nonetheless. The top four principal components of the adjusted data were also much more stationary than the top PCs of the unadjusted data, suggesting that no unknown-for time-localized event had a strong influence on variance after adjustment.The adjusted data revealed a clear clustering of the samples in https://www.eortc.org/data-sharing/), and if granted can be used for analysis and discovery science32, datamining or creating new signatures of clinically relevant questions33.The resulting dataset is representative of an early stage breast cancer population and is now available (unadjusted and adjusted) together with patient characteristics and clinical outcomes for future translational research by submitting proposals to the EORTC for access to the gene expression data as well as the clinical and patient data  distributions, is also possible. Gaussian linear models are typically used for microarray intensities, allowing the presented algorithm to rely on a least square objective, which can be derived as the maximum likelihood estimator of the linear parameter in the Gaussian model. Poisson or NB models would lead to different objectives, which could be minimized using existing toolboxes assuming the unwanted variation factor W was known. In the Gaussian case, the maximum likelihood estimator of W can be obtained by taking the singular value decomposition (SVD) of the expression matrix restricted to its control genes. When dealing with sequencing data, a simple option to estimate W is to use SVD on the log-transformed counts for the negative control genes40. Alternatively, methods to compute the maximum likelihood estimate of W under non-Gaussian distributions were recently introduced41 and can be used instead.Our adjusted dataset is an ideal base for tasks such as subtype discoveryFor the gene expression dataset described in this paper, we expect the exceptional sample size and homogeneity of the data to bring an unprecedented power to differential and survival analysis, to help reveal finer subtypes and create new genomic signatures relevant to management of breast cancer.4, sponsored by the European Organization of Research and Treatment of Cancer (EORTC) and country-associated clinical trial groups, enrolled women aged ≥18 years that were diagnosed with histologically proven operable TNM stage I–II invasive breast cancer and 0–3 positive lymph nodes. Patients were eligible to enroll in the MINDACT trial when a frozen tumor sample, containing at least 30% tumor cells, was available for gene expression analyses by microarray. For further clinical eligibility and exclusion criteria see Viale et al.42 and Cardoso et al.4. Patients were enrolled in 112 participating institutions in nine European countries between February 2007 and September 2011. Tumor samples were processed in Agendia’s central laboratory . Subsequent MammaPrint test failure rate, defined as the percentage of samples that did not finish the laboratory test procedure due to quality control failure was 1.7%.The MINDACT trial4.The MINDACT trial protocol was approved by the protocol review committee of the European Organization for Research and Treatment of Cancer (EORTC) and the ethics committee at each participating site. The trial was conducted in accordance with the Declaration of Helsinki and good clinical practice guidelines. A written informed consent was obtained from all patients12 and used for quality control assessment. This included the use for positional monitoring of array hybridization to control for potential mix-up of samples: control samples with known outcome were hybridized in a certain order and positioned on the arrays within the patient sample batches. During the time of the MINDACT trial three different MammaPrint low-risk controls and five different MammaPrint high-risk controls were used for the purpose of assessment and control of technical variations of MammaPrint procedures over time. The full-transcriptome data of the n = 562 control samples were only used to support visualization of the results, e.g., to show that the biological signal was still present after removal of technical variation from the dataset , multiple control samples with previously defined MammaPrint scores were used as technical and experimental controls within each batch run of samples. Data from these MammaPrint control samples were continuously monitored in the clinical diagnostic setting to ensure quality and safetyFor the MINDACT trial, a full-transcriptome microarray design was used that is based on Agilent’s 4 × 44 k format . This customized Agilent microarray (Agendia array 15746) was designed using annotation from genome build NCBI35. The array was customized in such a way that the MammaPrint 70 probes were added as well as additional Agendia features, counting to a total of 40,793 unique probes targeting 28,655 transcripts. The 40,793 unique probes are a combination of Agendia probes and Agilent human genome catalog probes available at the time of platform design (January 2007).11. As shown in Fig. 4) from May 24, 2009, to January 30, 2010, at which time the issue rectified with the use of a new reference RNA (see below). All samples of study patients (n = 6693) were evaluated and passed all criteria for RNA quality and RNA yield following the diagnostic MammaPrint assay-specific quality control assessment.RNA was isolated from a patient’s fresh frozen tumor sample and was amplified as described previously11. cRNA labeling was either performed manually or automated by use of a robot  and BP1 (Breast Pool 1), both locally assembled reference pools, was followed by the use of a commercially available UHR . MRP and BP1 consisted of pooled and amplified RNA of >100 primary breast tumors11. Samples used for MRP were isolated using RNA-Bee batch 1 and samples that were used to create BP1 consisted of samples isolated with RNA-Bee batch 2  to Agendia array 15746 as described previouslysee Fig. . Arrays see Fig. . The usesee Fig. . BP1 wasAgendia maintains a quality system in compliance with international regulations as FDA and the EU in vitro diagnostics directives.10-ratio expression values (reverted to sample/reference format). Positive and negative controls were excluded, and only positive and significant signals were used without background correction in the normalization procedure.RNA expression was measured by quantifying fluorescent intensities of the scanned array TIFF-images using Feature Extraction software, version 9.5 (Agilent Technologies). Subsequent within-array normalization was performed with the lowess correction method using a linear polynomial , which was the default method for normalizing dual color Agilent microarrays. Expression values are presented as the logp-values, errors and non-uniformity indicators, and are provided in the individual sample text files.Only fluorescent intensities of high-quality probe measurements were used in the analysis pipeline. Probe quality was calculated by the standard Feature Extraction Software using the built-in “universal error model”, and indicated by several factors including 10-ratios were excluded from further analysis as these probes have a high inaccuracy level due to the negative value in either channel. Some probes were flagged as “Feature Non-uniformity Outliers” (FNO) in the individual sample text files and subsequently excluded from further analysis. Probes that are labeled by the software as FNO usually have a high and uneven intensity. This can be caused by a scratch, contaminant, or some manufacturer imperfection on the array, which can cause deviating ratios and is an indicator of poor quality.In case the normalized red or green signals are ≤0 then the normalized Log12. As a result, the full-transcriptome dataset contains data for 6,688 patients.Hybridization quality was assessed, and five study samples were excluded from further analysis based on poor quality full-transcriptome hybridizations (non-uniform hybridization). Diagnostic MammaPrint readout was not affected and passed the stringent quality assessment that was FDA/QSR controlled for accurate MammaPrint readoutFor the 6688 patient dataset containing 40,793 probes, the number of missing values was determined. A missing value is the result of probes flagged by a poor quality as described above.10-ratios) are not meant to be compared across the different references. Expression values were calculated based on the ratio of the sample-/reference-intensities and can give different log10-ratios. The number of missing values was determined and imputed for the three references  separately.During the MINDACT study three different standard references were used  and no values were imputed, leaving 35,611 probes per patient for further analysis. Missing values were imputed for the remaining probes using a k-nearest neighbor approach based on other probes that behave similarly across the patient samples43. A separate imputation procedure was done specifically for control arrays  to avoid potential introduction of artifacts from the missing value imputation procedure. To determine if a high correlation was caused by repeated processing of the same sample we reprocessed available RNA for a selected number of samples. Their MammaPrint indices were compared to the original indices  signals were calculated as the average of log10-ratios for all probes corresponding to ESR1, FOXA1, SCUBE2, and PTPRT genes. Claudin (CLDN) signals were calculated as the average expression of all probes corresponding to CD14, VAV1, IL6, and VEGFC genes. Erb-b2 receptor tyrosine kinase 2 (HER2) signals and breast cancer gene signals were calculated as the average expression of all probes corresponding to the HER2,or BRCA1, and BRCA2 genes, respectively.Throughout the paper, we measure the association between principal components and expected biological signals related to the main breast cancer subtypes. The estrogen receptor (10-ratios of dedicated ER and HER2 probes30) of the unadjusted dataset was used as a proxy for the immunohistochemical staining. This TP-readout showed a concordance of 95% with the immunochemistry status of the unadjusted samples after binarization in Roepman et al.30. Viale et al.42 showed in the first 800 MINDACT samples that the concordance between TargetPrint and immunohistochemistry was 98%, and 96% for ER, and HER2, respectively. Therefore, TP-read out is a suitable substitute for the immunohistochemistry status that was not available to us at the time of analyses.TargetPrint (TP)-readout (loghttps://www.r-project.org/44).Analyses and visualization of data was performed in R  analysis29 were used to evaluate the association between technical or biological factors and the four PCs. Canonical correlations are a multivariate generalization of Pearson correlations that define an association between sets of variables rather than single variables. Although all associations were significant, we chose not to provide p-values. Indeed, the large sample size of this study makes all observed associations significant, even for very small size effects. We are interested in identifying factors, which explain a large proportion of the variance rather than testing for non-zero associations. Since we do not perform any inference, we do not assess the normality of the factors, which is only a necessary condition to build valid inference procedures, not to use Pearson correlations as a measure of linear correlation. It is important however to assess that these associations are not affected by outliers, and that no major non-linear effect is missed. The scatter plots that we provide and discuss in the manuscript show no evidence of strong outliers. We specifically address some cases of notable non-linear associations, such as the peaks in the PC projections ), and the association between PCs and HER2 expression ).Pearson correlations and canonical correlationsWe encoded biological signals and labeling yield using shades of blue in Figs. 27. RUV relies on a linear model of the expression data, including a factor of interest X and k unwanted variation factors stored in a matrix W:α, β) are the effects of the factor of interest and unwanted variation, respectively, and ε is a Normal centered random variable. To estimate W, RUV methods rely on negative control genes, defined as genes whose expressions are not expected to be influenced by biological signals of interest. Jacob et al.27 describe a procedure to estimate Wα when X is unobserved, e.g., to adjust a dataset for future analyses with no specific factor of interest in mind. The “naive” procedure uses Xβ = 0. The “naive random” method additionally does a ridge regression of the gene expression matrix Y against these k factors instead of an ordinary regression to estimate α. It refers to the fact that ridge regression provides a maximum a posteriori estimator of α when it is modeled as a random effect. Ridge regression limits the amount of variation captured by α, which is crucial if there is a risk that W is confounded with some unobserved factor of interest X, especially in the naive procedure, which models Xβ = 0. In this study, we further regress the expression matrix against a few identified technical factors stored in a matrix Z: specifically, Z includes an interaction factor representing all combinations of reference RNA and isolation reagent  described by Jacob et al.Z represents the interaction of automated labeling with isolation reagent: as the two groups of arrays for which automated labeling was used were isolated at different time periods and with different reagents, the latter was more suited for removing the time-localized effect on gene expression.Of note, Fig. k = 100 and tried several values of the ν parameter of the ridge regression by exploring a grid of values nu.coeff  multiplied by the squared largest singular value of W. We then picked the nu.coeff, leading to the largest correlation between TargetPrint–ER30 (see previous section) computed before adjustment and the corresponding ER probe after adjustment (nu.coeff = 0.2). We emphasize that the TargetPrint readouts were only used to choose the amount of variance removed, not to determine the unwanted factors W and Z in our adjustment procedure. A more detailed explanation of the choice of nu.coeff is provided in Supplementary Note 3.We utilized the 1000 genes of lowest interquartile range as negative controls. We used W matrix was estimated on both patient and control samples, as we wanted the adjustment of the control sample data to be closer to the one applied to the patient sample data. By construction of the Z matrix, both the 6688 patient sample data and 562 control sample data are mean centered by the adjustment procedure.The data adjustment procedure was applied to the 6688 patient samples but without the 562 control samples as the latter could bias the estimation. The data of the 562 control samples were adjusted separately using the same procedure, except that their Further information on research design is available in the Supplementary InformationSupplementary Data 1Supplementary Code 1Description of Additional Supplementary FilesReporting SummaryPeer Review File


Topic modeling was introduced to classify texts of natural language by inferring their topic structure from the frequency of words. This paper assumes that analogously the cancer subtype identity, which is crucial for the correct diagnosis and treatment plan, can be extracted from gene expression patterns with similar techniques. Focusing on breast and lung cancer, we show that state-of-the-art topic modeling techniques can successfully classify known subtypes and identify cohorts of patients with different survival probabilities. The topic structure hidden in expression data can be looked at as a biologically relevant low-dimensional data representation that can be used to build efficient classifiers of expression patterns.Topic modeling is a widely used technique to extract relevant information from large arrays of data. The problem of finding a topic structure in a dataset was recently recognized to be analogous to the community detection problem in network theory. Leveraging on this analogy, a new class of topic modeling strategies has been introduced to overcome some of the limitations of classical methods. This paper applies these recent ideas to TCGA transcriptomic data on breast and lung cancer. The established cancer subtype organization is well reconstructed in the inferred latent topic structure. Moreover, we identify specific topics that are enriched in genes known to play a role in the corresponding disease and are strongly related to the survival probability of patients. Finally, we show that a simple neural network classifier operating in the low dimensional topic space is able to predict with high accuracy the cancer subtype of a test expression sample. TCGA SubtypeSeLet us stress again that we obtained these results without resorting to the marker genes HER2, KI-67 and hormone receptors, which, except for HER2, were actually excluded from the gene set used in the analysis. Our results show that the expression levels of these genes are crucial to distinguish between the two Luminal types, but they are not mandatory to identify the remaining subtypes and separate them from Luminal A/B.One of the advantages of using a topic modeling approach is that, as we mentioned above, each sample is a mixtures of topics described by the probability distribution Equation ). The ceTopics are nothing but lists of genes. A common way to investigate their properties is to perform enrichment tests using tools like GSEA . The enrdepleted in the topic space  starting from a space with almost 20,000 genes. We used the topic space as an embedding space to train a neural network model which can then be used as an efficient classifier to associate samples to their specific subtype. Using topics as features and To evaluate the performances of our predictor, we performed the same analysis using K-Nearest Neighbors which is a standard and very popular tool in this context. It turns out that the performances of our predictor model  represents a much easier task for a clustering algorithm and indeed, as we shall see, hSBM is able to correctly separate LUAD from LUSC. TCGA repository on lung cancer data allows for a non-trivial test of clustering algorithms. Indeed, Cline et al. in  observedprovided . We analprovided , we may Running hSBM on the above data we found three different layers of resolution with 2, 8 and 58 clusters and 5, 12 and 42 topics, respectively.Discordant LUSC samples are clustered together with the LUAD ones, capturing the fact that they are more similar to LUAD than LUSC. It is instructive to follow the hierarchical organization of clusters  in lifelines Python package was used to perform the survival analysis. We performed an estimation of the survival probability cox [The (t), cox  was usedevel see . None ofevel see .At this point we extracted the list of genes contained in Topic 3 to see if we could understand the origin of this different survival probability. As we mentioned above, one of the advantages of topic modeling is that groups are not hard-constrained but are actually mixtures weighted by We sorted the genes in this topic by Interestingly, the list of the genes contained in the topic has a very small overlap with the one found by Lucchetta et al.  even if C of labeled samples and a partition K in clusters of these samples, the NMI is defined as the harmonic average of homogeneity h and completeness We used the Normalized Mutual Information (NMI)  to evaluC and K; c, k and c in the cluster k. With this definition it is easy to understand the meaning of homogeneity and completeness. If all the samples in cluster k belong to the cancer subtype c then c are in the same cluster k. A similar approach involving NMI was proposed in [In these definitions posed in  to evaluRandom clusters can have a non-zero NMI value given the way this score is defined. The score value will be a steadily increasing function of the number of clusters. To keep this effect into account, we normalized the empirical NMI values with the score NMIhttp://cancergenome.nih.gov.The results published here are in part based upon data generated by The Cancer Genome Atlas (TCGA) managed by the NCI and NHGRI. Information about TCGA can be found at Gene Expression Quantification data type in transcriptome profiling category. We choose RNA - Seq with HTSeq - FPKM as workflow type. We downloaded the 1222 samples from TCGA-BRCA project and 1145 from TCGA-LUSC and TCGA-LUAD projects.We downloaded data from TCGA using tools provided by Genomic Data Commons (GDC) . We downDuring the analysis of lung cancer, we considered only the 408 samples with a subtype annotation available in the TCGABiolinks GUI .Version:2.17.1 [We downloaded metadata for TCGA’s samples using the TCGABiolinks GUI at n:2.17.1 ,47.SubtypePam50 classification labels provided by [SubtypeSelected obtained via the PanCancerAtlas_subtypes function of TCGABiolinks [During the analysis of breast cancer, we downloaded both the vided by  and avaiBiolinks . TCGABioBiolinks . We discWe downloaded data for lung from a dataset prepared by Wang et al. . They prE and let the algorithm to be faster even with a large number of nodes N.Only samples with a valid annotation were considered and this left us with 1415 samples from LUAD, LUSC and GTEx. We applied a In our repository, we provide the code to correctly preprocess the data in order to be loaded by the model.As mentioned above, we filtered genes with two different strategies.Tissue specific genesg in tissue T  s the samples and R the total number of samples. Similarly, we estimated the variance of breast samples We searched genes whose behavior was different in one tissue with respect to all the others. We used the following procedure. Firstly we estimated the mean expression of gene ished in .Highly Variable Genesscanpy Python package [Another approach we considered is the standard selection for the so-called highly variable genes. We selected them using  package  and keptWe report in https://github.com/martingerlach/hSBM_Topicmodel/tree/develop.We adapted hierarchical stochastic block model (hSBM) to gene expression data. The original code to run hierarchical stochastic block model on a bipartite network was provided by  in the ri in sample j. In other words, Hierarchical stochastic block model is a kind of generative model that tries to maximize the probability that the model  package  minimizethe data ,51,52.We set the algorithm to minimize the description length As output of the model, we find the probability distributions V is the number of vertices (samples and genes) and E the number of edges. For E.The complexity of hSBM is if E∼O(V), where VIn our setting, we have R the total number of samples. This centered We centered the A topic is nothing but a list of genes, it can be investigated using hypergeometric tests. The results shown in this paper are computed using the GSEA  tool.We performed survival analysis on lung, we wanted to find how topics are related to the survival probability of a patient.stage ia became stage i) and ended up with four stages i, ii, iii and iv.Our analysis began with the list of the mixtures p labeled stage. We subtracted the mean to normalize the data and obtained We averaged each table over stages and obtained vital_status, days_to_last_follow_up and days_to_death. We estimated the lifetime or the number of days the patient survived after the diagnosis, using days_to_last_follow_up if the patient was Alive and days_to_death for Dead patients. A similar approach was recently utilized by [Using GDC tools we downloaded TCGA metadata and in particular: lized by .In order to estimate whether a topic is up regulated in a patient, we evaluated the 50th percentile of cox model using the COXPHFitter module. We used the lifetime, the vital status and the new feature as input for the fit function.We used these data to fit the hazard with a x is the new feature described above. The hazard is defined as the ratio of the derivative of the survival and the survival itself t, namely the number of patient alive at time t divided by the total number of patients. The package estimated the ratio between the hazard of samples with topic up-regulated and hazard of samples with topic not up-regulated. Therefore, we were able to estimate the coef does not depend on time, but it is a sort of weighted average of period-specific hazard ratios. In our test, we obtained an p-value reported in gender variable and we obtained The Cox model quantified how the up regulation of the topic affected the survival probability. Cox fits the hazard function conditioned to a variable lifelines Python package [These analyses were performed using  package .X with samples on the rows and topics on columns. The element In order to build the predictor for cancer subtypes, we prepared the design matrix crossentropy. The model was built using keras [The model consisted of a neural network with one hidden layer. In order to keep the model as simple as possible, Stochastic Gradient Descent was selected as the optimizer in both cases. We set the loss function to be the ng keras .ReLU and the output layer consisted of 4  neurons activated by the softmax function.In case of breast, the input layer had a dimension of 399, namely the number of topics; the hidden layer was built with 50 neurons activated by Hyper-parameters were searched maximizing the We obtained an accuracy of X with the same process described above. The input layer consisted of 326 neurons, the hidden layer was instead built with 20 neurons activated by ReLU functions and finally, the output layer consisted of a single neuron activated by a sigmoid function. The output was a binary array that distinguished LUAD and LUSC.In the analysis of lung, we prepared the design matrix We obtained an accuracy of Z that defines the classes given the outputs of the hidden layer z. In lung, Z is the value of the sigmoid function in the last layer c, In both cases a confusion matrix and a Receiving Operating Characteristic curve were constructed. The confusion matrix and the ROC curves were estimated using scikit learn . The ROChttps://cran.r-project.org/web/packages/WGCNA/index.html. This was run using default parameters: power was set to the lowest for which the scale-free topology fit index curve flattens out, minModuleSize was set to 5 and mergeCutHeight to modules of genes, we considered them as topics. In order to obtain clusters we cut the tree built using modules to estimate distances between samples. We reported in In this work, some of the analysis required other clustering methods. Weighted Gene Correlation Network Analysis was performed using the dedicated R package available at K the number of topics. K was set based on the number of clusters in output from hSBM. When managing LDA output, we selected the argmax of Latent Dirichlet Allocation (LDA)  is a stasklearn and we set the model to use euclidean metric and complete linkage. In this case, we cut the tree to match the hSBM number of clusters. In this case it has not been possible extract any information on the genes.Hierarchical clustering was performed using https://github.com/fvalle1/topicTCGA.The codes, notebooks and data to reproduce this work are available on a GitHub repository at In conclusion, there are three properties of the hSBM topic modeling analysis that we think are at the basis of the effectiveness of this approach and correspondingly there are three lessons that we can learn from our work.hSBM imposes a minimal amount of assumptions on the nature of the statistical distributions of gene expression values across samples . The absCell phenotypes are driven by the expression pattern of the whole set of genes and not by a handful of markers. This is probably true in general, but it is even more plausible for complex diseases like cancer. The breast subtype classification is based on the expression levels of a handful of genes but, notwithstanding this, hSBM was able to reproduce the classification to a large extent without resorting to marker genes. Clearly this is telling us that breast cancer subtypes are driven not only by the expression level of one or two genes but by a whole pattern of pathway alterations as highlighted by the topic distribution in the different clusters. This can have important consequences on the way we approach the search for gene signatures.While it is certainly true that gene signatures play an important role in cancer medicine, it is also clear that they are not the end of the story. It is not strange to find different gene signatures aimed to identify the same cancer subtype that have almost no overlap . This supports the importance of data mining tools able to address the overall behavior of the transcriptome and not driven by the gene expression pattern of a single gene signature.Looking at all our findings, we see that the actual information on the complex disease is well encoded in the topic space and in particular in the probability distributions We think that these lessons could be useful beyond the specific tool (hSBM) that we discussed in this paper. They should drive the conception of a new generation of innovative and network-based data mining approaches, possibly combining the best features of the different tools that we compared in this paper, such as LDA, WGCNA and hierarchical clustering. The development of new statistical tools is a mandatory task if one wants to address the challenging issue of combining the different layers of information which are available in complex databases like TCGA, say, the microRNA expression levels, the mutational content of genes or the epigenetic layer of regulation. A fruitful combination of these different sources of information could greatly enhance our ability to address a personalized therapeutic approach to complex diseases like cancer.


GSE34095, GSE15227, GSE63492GSE116726, GSE56081 and GSE67566) associated with IDD from the GEO database. Using differential expression analysis and recursive feature elimination, we extracted four optimal feature genes. We then used the support vector machine (SVM) to make a classification model with the four optimal feature genes. The ROC curve was used to evaluate the model’s performance, and the expression profiles  were used to construct a competitive endogenous RNA (ceRNA) regulatory network and explore the underlying mechanisms of the feature genes. We found that three miRNAs  and three circRNAs  were important regulators with more interactions than the other RNAs across the whole network. The expression level analysis of the three datasets revealed that BCAS4 and SCRG1 were key genes involved in IDD development. Ultimately, our study proposes a novel approach to determining reliable and effective targets in IDD diagnosis and treatment.Intervertebral disc degeneration (IDD), a major cause of lower back pain, has multiple contributing factors including genetics, environment, age, and loading history. Bioinformatics analysis has been extensively used to identify diagnostic biomarkers and therapeutic targets for IDD diagnosis and treatment. However, multiple microarray dataset analysis and machine learning methods have not been integrated. In this study, we downloaded the mRNA, microRNA (miRNA), long noncoding RNA (lncRNA), and circular RNA (circRNA) expression profiles ( Lower back pain, commonly caused by intervertebral disc degeneration (IDD), can be a significant socioeconomic burden on patients . IDD is thrombospondin-2, vitamin D receptor, COL2A1, ACAN, interleukins , matrix metalloproteinases (MMP-3 and MMP-9), and growth/differentiation factor 5, have been associated with the pathophysiological process of IDD development , long noncoding RNAs (lncRNAs), and circular RNAs (circRNAs), are important gene expression regulators that influence cellular function and disease states . lncRNAsradation . The divIn this study, we aimed to identify the key genes and underlying mechanisms of IDD development by constructing an lncRNA/circRNA-miRNA-mRNA network using multiple microarray datasets and machine learning methods. http://www.ncbi.nlm.nih.gov/geo): two mRNA expression profiles , two miGSE63492 ) and GSESE116726 , one mRNGSE56081 ), and onGSE67566 ). Table p-value <0.05 for the GSE34095 dataset, p-value <0.01 and —log2FC—≥ 2 for the GSE15227 dataset, and p-value <0.01 and —log2 FC—>2 for the GSE116726 dataset. We obtained the GSE67566, GSE63492, and GSE56081 datasets from the same tissue samples, and provided the differential expression analysis results and thresholds as Supplemental Materials , differentially expressed miRNAs (DEMs), differentially expressed lncRNAs (DELs), and differentially expressed circRNAs (DECs) in IDD and normal disc tissue using the “Limma” R package . Since eaterials  and S2. aterials .http://bioinfogp.cnb.csic.es/tools/venny/index.html), we extracted the differential gene intersections of the GSE34095 and GSE15227 datasets to use as IDD-related DEGs  to build an RBF kernel SVM model to identify the optimal feature genes, and we selected the GSE15227 dataset to train the machine. We then evaluated its performance in the GSE15227 training set using the R pROC package  (GSE34095 and GSE56081) to further verify the model’s performance.The support vector machine (SVM) is particularly effective for binary classification in a supervised learning manner, and is better than other machine learning methods at identifying subtle patterns in complex datasets . Radial /12/77/) . The optGSE116726 and GSE63492 datasets as candidate DEMs. Intersecting target miRNAs and candidate DEMs were selected as optimal feature gene interaction pairs.We used miRWalk 2.0 with 12 prediction programs  to predict the optimal feature genes’ target miRNAs . miRNAs We obtained the miRNA sequences interacting with optimal feature genes from the miRbase  and extrhttp://www.cytoscape.org/).We then used the miRanda tool to analyze the combinations of miRNAs, DELs, and DECs. We set the analysis parameters for the miRNA-DELs and miRNA-DECs as sc:120, en:-20 and sc:150, en:-7, respectively, and processed the results using python script  and S4. t-test in GraphPad Prism 7.0 . P values < 0.05 were considered statistically significant.All data were presented as the mean ± SEM. Differential expression levels were compared using the Student’s GSE34095 dataset  via the intersection between the two datasets, and considered these as IDD-related DEGs  in the  dataset  and 188  dataset . Hierarc dataset  and 1D. ted DEGs .WDR46, BCAS4, CRNKL1, and SCRG1) as optimal feature genes associated with IDD  of the SVM model was 100 percent, suggesting that the model could accurately distinguish between IDD and normal samples  could be used as effective and accurate IDD diagnostic biomarkers.A classification model was constructed using the RBF kernel SVM, the four genes as features, and the  samples . We used dataset . However percent . These rGSE116726 dataset  from the  dataset  and 3B aThe differentially expressed lncRNAs and circRNAs between the degenerated disc samples and the control disc samples were downloaded from the GEO database. We analyzed the miRNA-DEL interactions and the miRNA-DEC interactions using the miRanda tool with ≥5 miRNA binding sites. The miRNA-DEL pairs, miRNA-DEC pairs, and miRNA-optimal feature gene pairs were combined to build a DELs/DECs-DEMs-optimal feature gene regulatory network, which included four mRNA nodes, 12 miRNA nodes, 10 lncRNA nodes, and 75 circRNA nodes . Out of GSE15227 and GSE56081 datasets, BCAS4 expression levels were significantly down-regulated  were identified using machine-learning methods. The construction of a DELs/DECs-miRNA-optimal feature genes network revealed that three miRNAs  and three circRNAs  may be important mediators for optimal feature genes. To further explore the different optimal feature genes in normal and degenerative NP tissues, we also investigated their expression levels across three datasets. The results indicated that BCAS4 and SCRG1 were key genes related to IDD.Recent developments in bioinformatics and computational biology have led to the identification of several key genetic targets related to IDD and the prediction of potential molecular mechanisms . In thisBCAS4), a novel gene cloned from breast cancer cells, encodes a 211-amino acid cytoplasmic protein with no significant homologies to any known protein  was first found in the genes associated with, or responsible for, the neurodegenerative changes observed in transmissible spongiform encephalopathies  growth suppression and differentiation during dexamethasone-dependent chondrogenesis  and DELs (D) of the degenerative disc samples and control disc samples, respectively, in the GSE56081 dataset. Hierarchical cluster heatmaps represent the DEMs in the GSE63492 dataset (E) and the DECs (F) in the GSE67566 dataset. Blue represents the downregulated and red represents the upregulated.Volcano plots represent the DEGs (A) and DELs (C) of the degenerative disc samples and control disc samples, respectively, in the Click here for additional data file.10.7717/peerj.10120/supp-3Supplemental Information 3Click here for additional data file.10.7717/peerj.10120/supp-4Supplemental Information 4Click here for additional data file.10.7717/peerj.10120/supp-5Supplemental Information 5Click here for additional data file.10.7717/peerj.10120/supp-6Supplemental Information 6Click here for additional data file.10.7717/peerj.10120/supp-7Supplemental Information 7Click here for additional data file.10.7717/peerj.10120/supp-8Supplemental Information 8Click here for additional data file.10.7717/peerj.10120/supp-9Supplemental Information 9Click here for additional data file.10.7717/peerj.10120/supp-10Supplemental Information 10Click here for additional data file.10.7717/peerj.10120/supp-11Supplemental Information 11Click here for additional data file.