TB) pandemic. Gene expression profiles based on blood signatures have proved useful in identifying genes that enable classification of TB patients, but have thus far been complex. Using real‐time PCR analysis, we evaluated the expression profiles from a large panel of genes in TB patients and healthy individuals in an Indian cohort. Classification models were built and validated for their capacity to discriminate samples from TB patients and controls within this cohort and on external independent gene expression datasets. A combination of only four genes distinguished TB patients from healthy individuals in both cross‐validations and on separate validation datasets with very high accuracy. An external validation on two distinct cohorts using a real‐time PCR setting confirmed the predictive power of this 4‐gene tool reaching sensitivity scores of 88% with a specificity of around 75%. Moreover, this gene signature demonstrated good classification power in HIV+ populations and also between TB and several other pulmonary diseases. Here we present proof of concept that our 4‐gene signature and the top classifier genes from our models provide excellent candidates for the development of molecular point‐of‐care TB diagnosis in endemic areas.There is an urgent need for new tools to combat the ongoing tuberculosis ( Although molecular assays have their drawbacks for routine diagnosis as compared to more basic laboratory tests, RT–PCR assays are widely approved for detection of influenza viruses  was applied to the normalized gene expression levels to inspect the variation in the dataset. The results indicate that the gene expression differences between TB patients and healthy controls constitute the majority of the variance within the dataset as indicated by their separation in the first two principal components . For traThe overall sample assignment and analysis steps applied in this study were as follows: (i) training set used to generate models and test them on the test set, (ii) determine optimal biosignature sizes and model training approaches, (iii) merge test and training set to maximize model performance, (iv) test full models using internal cross‐validation, (v) test full models on qPCR data from external cohorts, and (vi) test full models on independent (microarray) data Fig .To initially investigate the number of genes that are required for good discrimination of TB patients from healthy controls in our dataset, RF models were built multiple times on the training set Fig  and eachTo identify the top classifier genes within our dataset we used RF models to rank genes according to their importance in partitioning the samples into the defined groups . Generating 25 replicates of an RF model on the whole dataset we counted how many times each gene was among the most important predictors. Twelve genes ranked within the top 15 in all RF models Fig A. VariabBased on the above results Fig , one canWhereas random forests can output a list of genes ordered by their importance as classifiers, CI trees output a set of top predictor genes, a single simple decision tree and also have the advantage of generating easily interpretable predictor values. Intriguingly, running such a CI decision tree on samples in the training set resulted in a very small set of only three genes. Calculation of the classification power of these genes on the test set RT–PCR data gave a performance of AUC = 0.95 (95% CI: 0.90–0.99).In the following step, we merged the training and tests sets from the Indian cohort to generate models to be tested both using internal cross‐validation, and by applying them to external datasets.Since large training sets usually provide better classification power, we also evaluated a 15‐gene signature built on the whole 189 sample dataset . A grapTo validate our signature in a RT–PCR‐based experimental setting, we tested the performance of our 4‐gene model on whole‐blood RNA samples collected from two external cohorts. RT–PCR of the 4‐gene signature was carried out on a total of 75 samples from The Gambia and 62 samples from Uganda (see From the receiver operator characteristic (ROC) curves for the 4‐gene model performance in the Ugandan and Gambian cohort Fig  one can et al, et al, et al, et al, et al, – and HIV+ populations.A robust gene signature that can be efficiently used as a diagnostic triage tool at point of care should be able to accurately discriminate TB patients from healthy individuals across a wide range of ethnic backgrounds and also across different technology platforms. Ideally, it should also not be confounded by other diseases or environmental factors. To this end, we tested our gene models for their classification power in several independent, publicly available microarray expression datasets  or whether higher specificity would be preferred (at the cost of sensitivity).et al, et al, Overall, the 4‐gene model displayed the best performance, with similar AUCs to the 15‐gene model for discriminating between TB patients and healthy individuals, but far better performance when other diseases were considered. This small set of four genes consisted of GBP1, IFITM3, P2RY14 and ID3. Both GBP1 and IFITM3 are interferon‐induced genes, corresponding to the strongly upregulated interferon signatures observed in TB patients  diseases is unlikely to emerge. We rather envision an easy two parallel signature setup that when (i) the signature classifying healthy from probable TB tests positive, then (ii) a second differential diagnosis signature (yet to be defined) would distinguish TB from other diseases.et al, et al . 29% of skin test‐negative individuals were tested IGRA+, while 15% of skin test‐positives tested IGRA−. LTBI individuals were defined based on positive skin test to match the definition for latency in the other validation cohorts. Numbers of patients, LTBI and uninfected individuals are given in A prespecified total of 200 adult participants  were recruited at St. John's hospital in Bangalore, India. Inclusion criteria for patients were cases of newly diagnosed active TB, confirmed by a positive GeneXpert sputum test . Samples were randomly assigned to a training set of 60 TB cases and 40 controls, maintaining the study design's 3:2 ratio of TB patients to controls. The remaining samples were assigned to a separate test set. No predefined analysis protocol was used in this study.Peripheral blood (2.5 ml) was drawn into PAXgene tubes and RNA was extracted using the PAXgene Blood miRNA extraction kit (Qiagen) following the manufacturer's instructions. Low quality RNA samples were excluded from the study (seven samples with A260/280 < 1.6). For this validation study, 360 target genes were selected based on our previously published microarray datasets from South Africa and The Gambia, using a ranking strategy based on the original analysis and RF variable importance  was calculated for each sample as its normalization factor.The entire raw dataset was evaluated for quality control. Genes with Ct‐test in R, with correction for multiple testing controlling the false discovery rate  database repository.‐RT–PCR data from the Indian cohort generated in this study: GSE74092‐et al, et al, et al, et al, et al, External microarray validations sets: GSE42834  database, were used in this study: Ethical approval for gene expression analysis in peripheral blood in this study was issued by St. John's Medical College and Hospital Institutional Ethical Review Board, Bangalore, India (Ref No. 173/2012) and by the Ethical Board of the Charité Berlin (No. EA1/200/08). Informed consent was obtained from all study participants prior to inclusion in the study and experiments conform to the Declaration of Helsinki principles.This study was designed by JM, ST, EL, US, JK, and SHEK. JM and GME performed most of the research. Clinical samples and data were provided by HMK, MO, and JK. Data analysis was performed by JM, GME, JW, ST, and EL. The manuscript was written by JM, ST, JW, GME, EL, and SHEK. All authors approved the manuscript.The authors declare that they have no conflict of interest.Tuberculosis (TB) continues to have a major impact on global health, particularly in resource‐poor countries. There is a great need for efficient clinical tools for rapid diagnosis in order to better control the ongoing disease burden. Multiple studies have shown that gene expression signatures can be used to discriminate TB patients from healthy individuals but, to date, these signatures have been large and not applicable for simple diagnostic application.We show that a gene signature containing as few as four genes can be used to rapidly diagnose probable TB cases. By using a standard RT–PCR assay, this signature could be directly applicable as a simple triage tool. Using cohorts from multiple geographical regions and ethnicities, we show that our signature can predict TB cases with high sensitivity and specificity, and even to some extent differentially diagnose them from several other pulmonary diseases.The classification signatures we describe in this work could provide a valuable simple and rapid new diagnostic tool to help clinicians decide to exclude or include probable TB in suspected new cases.Such new diagnostics could potentially have a great impact on reducing TB disease incidence in high endemic countries.AppendixClick here for additional data file.Expanded View Figures PDFClick here for additional data file.Review Process FileClick here for additional data file.


The recent advances in next generation sequencing technology have made the sequencing of RNA  an extemely popular approach for gene expression analysis. Identification of significant differential expression represents a crucial initial step in these analyses, on which most subsequent inferences of biological functions are built. Yet, for identification of these subsequently analysed genes, most studies use an additional minimal threshold of differential expression that is not captured by the applied statistical procedures.Here we introduce a new analysis approach, ABSSeq, which uses a negative binomal distribution to model absolute expression differences between conditions, taking into account variations across genes and samples as well as magnitude of differences. In comparison to alternative methods, ABSSeq shows higher performance on controling type I error rate and at least a similar ability to correctly identify differentially expressed genes.ABSSeq specifically considers the overall magnitude of expression differences, which enhances the power in detecting truly differentially expressed genes by reducing false positives at both very low and high expression level. In addition, ABSSeq offers to calculate shrinkage of fold change to facilitate gene ranking and effective outlier detection.The online version of this article (doi:10.1186/s12864-016-2848-2) contains supplementary material, which is available to authorized users. Since relatively recently, the variation in gene expression is commonly studied through RNA sequencing or RNA-Seq, based on next generation sequencing (NGS) technologies. In these study approaches, DE is usually inferred from comparison of two different treatments, developmental stages, or different tissues. A key step in these analyses is the reliable identification of significant DE. Most current statistical approaches employ a probabilistic model, such as the Negative Binomial (NB) –3, Poissd change , 9. ThisAn alternative approach has so far only been established for ChIP-Seq data and relies on an analysis of count differences between test and reference conditions , 11. In Here, we introduce ABSSeq , which employs an NB distribution to model count differences between conditions. It permits testing the magnitude of observed count differences taking into consideration background expression level variation. In particular, ABSSeq accounts for heterogeneous dispersions in expression level across genes by adding expected values (pseudocounts) to reads count according to the smoothed mean-variance relationship , which thttp://bioconductor.org/packages/devel/bioc/html/ABSSeq.html.ABSSeq has been implemented in the software package ABSSeq for the cross-platform environment R . ABSSeq We firstly introduce our approach with the help of the modencodefly and ABRF datasets see . Thereafp-value cutoff of 0.05 for each method except baySeq and EBSeq, which are excluded from this comparison since they report DE by posterior probalilities instead of a p-value. The simulated datasets are assumed to lack DE genes, facilitating computation of the type I error rate by dividing the number of DE genes identified by each method with the total number of genes. Figure Minimizing the type I error rate  or false positive rate is a primary goal of differential expression analysis , 25. TypDrosophila melanogaster, the modencodefly data set [p-value cutoff of 0.05, ABSSeq identifies an average of 17 DE genes and thus significantly fewer DE genes than all alternative methods . DESeq2 also performs well on this real data set, while the highest type I error rate is obtained for limma (873 identified cases of DE).The first comparison is based on a real data set for the fruitfly data set , which cNext, we examined the distribution of false positives along absolute log2 fold change and expression level  using the data from Fig. As the modencodefly data set only allows us to consider two replicates per group and condition, the resulting statistical power may be limited. Therefore, we repeated this assessment using another real data set, the ABRF data set  see Dat section,Overall, the results from the two real data sets suggest that ABSSeq has the ability to handle very small expression changes by considering the magnitude of absolute differences and penalizing the estimated dispersion See . Our res00, 0 indicates the number of up or down-regulated genes) but exhibit high diversity on the NB distributed data with randomly introduced outliers . ABSSeq performs best in both cases , but slightly decreases its performance with increasing sample size (n = 10).In addition to the two real data sets, we also compare the ability of the alternative approaches to control type I error rates on simulated data Fig. . GeneralTaken together, ABSSeq is able to efficiently control type I error rates for the real and simulated data sets Fig. -c and itp-value or posterior probability (baySeq) threshold. Thereafter, the obtained genes are divided into a truly positive group and a truly negative group according to pre-defined DE genes in the simulated data. This information then allows us to calculate the true positive and the false positive rate for all possible thresholds, construct ROC curves and compute AUCs using the ROC package in Bioconductor [An ideal DE inference method should be more sensitive to DE than non-DE genes, that is, it should be able to discriminate true DE genes against non-DE ones. Here, we evaluate the discriminative power of ABSSeq and other selected methods in terms of the true and false positive rates and also the area under Receiver Operating Characteristic (ROC) curve (AUC), using again the simulated data and general approach of Soneson et al. . The AUConductor . For all625625, where the super- and subscripts indicate the number of up- and down-regulated genes, respectively; Fig. n = 2). The performance of ABSSeq and the other methods are generally improved as the sample size increases, revealing a positive power of sample size on identifying true DEs. Overall, these results suggest that our NB model fits the over-dispersion data at least as well as the NB model implemented in other methods.When applied on the data set simulated using the NB distribution  at all replicate sizes, especially for the R625625 data set  whose AUC is even greater than 0.9 at n = 10 , but similar at small sample size with other methods; DESeq2 performs better at n = 10 due to outlier detection but worse at n = 2 and n = 5 (n ≥ 7 required for outlier detection); baySeq shows little improvement in performance as the sample size increases for the R625625 data set; EBSeq shows lowest AUC at n = 2 and improves performance at large sample sizes (n = 5 or 10); edgeR-robust (denoted by edgeR-r) shows an improved ability to handle outliers at small sample size (n = 2 or 5) compared to edgeR.We next test the influence of outliers, which we introduce into the NB distributed data using a similar approach as above  in outlier detection even at small sample size see . TogetheAs simulated data are by nature artificial, we further evaluate method performance on real data sets. The first of these relate to the MAQC study, for which RNA-Seq-identified DE genes were validated by quantitative reverse transcription PCR (qRT-PCR)  based onThe results for both data sets are essentially identical Figs.  and 3b. p-value cutoff in the alternative approaches, even if the cutoff is below 1.0e-8. Therefore, other cutoff criteria are required such as fold change, which has the problem that the biologically relevant cutoff point is not clear. The ABSSeq-based analysis instead produces high correlation between the minimal fold change and the inferred adjusted p-value, indicating that the p-value alone will select against DE genes with small fold change. Additional cutoff criteria therefore do not seem to be necessary for reliable DE gene identification.Analysis approaches, which do not consider the magnitude of expression differences, might yield highly statistically significant DE for genes with only small fold change  might lead to low senstitivity . To evaluate the ability of ABSSeq to detect true positives, we apply ABSSeq and alternative methods on the ABRF data set whereby in this case we focus on the comparison of the two considered conditions  either within the considered laboratories  or across the laboratories . The results are shown in Fig. Confounding variation can originate from library preparation or other kinds of batch effects. To remove its influence on DE detection, it can be modeled and thus integrated into the statistical analysis , as implp-value [p-value might lead to reduced sensitivity (type II error) by assigning a large p-value to small SNR ratio (i.e. high variance). The monotonic dependency between predictor (SNR ratio) and response  is inferred through an isotonic regression on 1514 paired variables (genes). Results are shown in Fig. p-value for one specific gene with high SNR ratio but low expression , suggesting Voom is more sensitive to DE at low expression level. Since DESeq2 and Voom test DE on log fold change, we postulate that the closer correlation between SNR ratio and adjusted p-value of ABSSeq is due to modelling directly the magnitude of DE difference. Overall, these results suggest that ABSSeq seems to model the magnitude of count difference with higher accuracy, which might help DE inference by reducing false positives.To evaluate the statistical power of each method in measuring the magnitude of DE in dependence of its variance, we repeated above comparison using genes that are exclusively expressed in only one condition of the MAQC-II data set following the approach from . The magp-value . A poor Another real data set (HapMap-CEU) is taken from , consistFold change often serves as a more informative indicator for biologists to identify DEs. It is also utilized in gene ranking to select candidates for further investigation and visualization . However, the fold change neglects variance across samples and might not necessarily be informative, especially for genes with low counts . To overcome this problem, DESeq2 introduces an empirical Bayes shrinkage for fold change estimation, which moderates the log fold change according to gene-specific dispersion . Fold chpvalue <0.05) and produce nearly evenly distributed fold change values, apparently improving gene ranking and visualization. Notably, shrinkage with gene-specific dispersion only influences a small part of genes with high dispersion. Unlike the approach in DESeq2, our shrinkage method on gene-specific dispersion is based on p-values and therefore does not change the number of inferred significant DE genes. In practice, users can obtain all three types of fold change values  in ABSSeq.However, this shrinkage approach neglects the gene-specific dispersion and thus shows no effects on non-DE genes with high dispersion as well as high expression Fig. , the folp-values correlate with the magnitude of count differences, thus producing a linear relationship between SNR ratio and p-value. As a result, it reduces type I error rates at both very low and high expression level and it also leads to a smaller number of highly significant DE genes with small fold change.Here we introduce a new method for differiential expression analysis of RNA-Seq count data, ABSSeq. Distinct from other current methods, ABSSeq infers DE genes through the absolute differences in gene expression and assumes the differences to be influenced by two sources of variation: that found for average gene expression levels and that found for the magnitude of differential expression. Our approach employs a NB distribution to model these two parts and, as a consequence, it is able to detect DE genes more effectively than existing methods, as demonstrated by our analysis of both real and simulated data. In particular, ABSSeq shows an advantage in discriminating DE genes against non-DE ones, it applies an efficient outlier detection approach and is thus robust against outliers. Morever, ABSSeq inferred In addition, ABSSeq introduces a procedure to shrink fold change according to the smoothed dispersion across expression level and observed dispersion (gene-specific), which permits fold change comparisons across genes and thus might favor downstream analyses, such as gene set enrichment analysis by ranking , clusterIn this study, the performance of methods is assessed with the help of two types of data sets: simulated and real. The simulated data sets are derived from the study of Soneson et al. . Followihttp://bowtie-bio.sourceforge.net/recount/ [Five real data sets were considered. Four of these  were downloaded from recount/ . The Micrecount/ . It is brecount/ , 45. We Drosophila melanogaster [The modencodefly data set served to study gene expression during the development of nogaster , coverinThe HapMap-CEU data set  includesThe ABRF data set is the Association of Biomolecular Resource Facilities next-generation sequencing (ABRF-NGS) study on RNA-seq, which aims to assess RNA-Seq data across laboratory sites and platforms  and reliThe Bottomly data set is from a study that characterized transcriptomic differences between two inbred mouse strains (C57BL/6J and DBA/2J) with 10 and 11 replicates each, respectively . We filtcij) for genes (i) and samples (j) at different conditions , which are discrete. Due to technical and other reasons, the total number of reads varies between samples or even sequencing lanes. The read count must thus be normalized before comparison across samples. The most common practice is to scale the counts according to the total number of reads of each sample [RNA-Seq data is represented as count of reads , and also provide an interface for the user to change these two values. Thus, the outliers are defined asOutliers mask the statistical significance by influencing the estimation of mean and variance. Given extreme high read counts outliers are often present in one or more RNA-Seq samples and thus it is essential for DE inference to reduce the impact of outliers , 13, 48.DE inference relies on an assessment of the difference of expression levels between two conditions (or more) as well as the variance across replicate samples. The popular null hypothesis for testing DE is that the mean read count for a particular gene is identical between conditions. However, the standard analysis of such a hypothesis neglects the magnitude of encountered differences. Here we use a distinct test statistic: the absolute difference of read counts between conditions , which was firstly applied to detect differential expression, epigenetics changes and transcription factors binding sites in the program EpiCenter , that isDi introduces a bias by favoring the larger group, which has a higher likelihood to reach higher sum counts by chance, thus more likely resulting in non-zero Di). For this reason, ABSSeq compensates the smaller group with the most likely read counts: the mean. In these cases, Di might not be an integer and needs to be rounded to the nearest integer.When the sample sizes between groups are not equal, Di is always discrete and apparently overdispersed as Di inherits variance from Di, which has two parameters, the mean mi and size factor ri, that ismentclass2pt{minimispersed ), which mi can be treated as the expected value or baseline of Di which is proportional to average expression level  or determined using the coefficient of variation (CV) in the tested data. Therefore, mi isci and α are larger value of sum counts, general CV. ri as the size factor is dependent on the mean-variance relationship and determines the scale of information contained byDi. We assumeDi to inherit dispersion from ci . As ci could be written as ci = nμi n = max, μi = max (the μA||B denotes mean of each condition), we assume ci has the same dispersion under μi. Therefore, the dispersion of ci becomesvi and siA||B2 denote the pooled dispersion factor, the mean and variance of each condition, respectively. ri is then given asDi against its expected value mi and dispersion ri. ABSSeq also allows DE detection on paired samples by replacing siA2 + siB2 with variance drived directly from paired differences.As a result, DE detection is based on the magnitude of ci (higher mi) for genes with low expression levels. To account for the dynamic uncertainty, we moderated mi by adding pseudocounts to cij according to the mean-variance relationship, which has no influence onDi and siA||B2 but μi and ri, that isIt is well-known that the mean-variance relationship of RNA-Seq data is basically quadratic , which sμi. To estimate μi0, we firstly construct the mean-variance relationship by applying local regression [locfit package from R, which has been introduced by DESeq. That isgression  on the gThen the smoothed or expected variance for each gene is given byμi always decreases as the expression level or sample size increases, we assume that θ serves as background of uncertainty across all genes.Since the uncertainty of cij is the same in all samples), the dispersion of sum counts ci simply becomes vi. However, vi > 0), indicating underestimation of v0 to vi, which becomesWhen the observed variance is 0  is not caused by ri. However, neglecting this variance will introduce false positives at low expression level since the small variance has a higher impact on μi when μi is small. On the other hand, this small variance could be treated as noise for μi or ci. In light of this, we add a small value to ci. mi becomesNotably, the small variance of α byσi is obtained by fitting a linear model to log-transformed counts from limma. In practice, the estimated α usually ranges from 0.1 to 0,3. α could also be provided by the user .After shifting read counts according to the mean-variance relationship, we simply assume that CVs of all genes are identical. While the SD of log-transformed data stands for the CV at original scale, we get p-value for each gene by the cumulative distribution function of Following , we can The log fold change can be described asci or Di. Indeed, in . The gene-specific dispersion Di, indicating a possible moderation of Di as well as fold change by shrinkage of Di and thus fold change. Based on this idea, we obain a new Di by replacing pi is the pvalue for gene i. Notably, the moderation is only applied on genes with r0. Using this approach the log fold change is then calculated by  sequencing of RNA; ROC, receiver operating characteristic; SEQC, sequencing quality control; TPR, true positive rate


Quantifying heterogeneity in gene expression among single cells can reveal information inaccessible to cell-population averaged measurements. However, the expression level of many genes in single cells fall below the detection limit of even the most sensitive technologies currently available. One proposed approach to overcome this challenge is to measure random pools of k cells  to increase sensitivity, followed by computational “deconvolution” of cellular heterogeneity parameters (CHPs), such as the biological variance of single-cell expression levels. Existing approaches infer CHPs using either single-cell or k-cell data alone, and typically within a single population of cells. However, integrating both single- and k-cell data may reap additional benefits, and quantifying differences in CHPs across cell populations or conditions could reveal novel biological information. Here we present a Bayesian approach that can utilize single-cell, k-cell, or both simultaneously to infer CHPs within a single condition or their differences across two conditions. Using simulated as well as experimentally generated single- and k-cell data, we found situations where each data type would offer advantages, but using both together can improve precision and better reconcile CHP information contained in single- and k-cell data. We illustrate the utility of our approach by applying it to jointly generated single- and k-cell data to reveal CHP differences in several key inflammatory genes between resting and inflammatory cytokine-activated human macrophages, delineating differences in the distribution of ‘ON’ versus ‘OFF’ cells and in continuous variation of expression level among cells. Our approach thus offers a practical and robust framework to assess and compare cellular heterogeneity within and across biological conditions using modern multiplexed technologies. Different cells can make different amounts of biomolecules such as RNA transcripts of genes. New technologies are emerging to measure the transcript level of many genes in single cells. However, accurate quantification of the biological variation from cell to cell can be challenging due to the low transcript level of many genes and the presence of substantial measurement noise. Here we present a flexible, novel computational approach to quantify biological cell-to-cell variation that can use different types of data, namely measurements directly obtained from single cells, and/or those from random pools of k-cells . Assessment of these different inputs using simulated and real data revealed that each data type can offer advantages under different scenarios, but combining both single- and k-cell measurements tend to offer the best of both. Application of our approach to single- and k-cell data obtained from resting and inflammatory macrophages, an important type of immune cells implicated in diverse diseases, revealed interesting changes in cell-to-cell variation in transcript levels upon inflammatory stimulation, thus suggesting that inflammation can shape not only the average expression level of a gene but also the gene’s degree of expression variation among single cells. In the case of qPCR expression data in Et = 40-Ct units  Constant white noise sampled from a Gaussian distribution with a mean 0 and standard deviation (std) of 0.5 Et is added to the simulated Et value of detected single/k-cell expression samples . This scenario is representative of noise characteristics of highly expressed genes in our experiment, whose measurements exhibit similar noise levels in both single- and k-cell measurements  under any of the above “3 biological variation x 3 assay sensitivity x 5 noise configuration” scenarios = 45 settings using our model, and infer back the true values using only the simulated data. We could have used our MCMC procedure for inference, but wanted to rule out potential differences in the MCMC convergence behavior of the SC, KC or SCKC approaches from confounding the intrinsic differences among these methods. So we opted for a simpler posterior surface scanning procedure that involved computing the posterior in each combination of the parameter values listed below and using this coarse grid of likelihood values to approximate the posterior distribution and calculate its mode and 90% CrI  reported in Vary π from 0 to 1 in steps of 0.02.Vary μ from 0 to 40 in in steps of 1 Et unit.Vary σ from 0 to 5 in steps of 0.1 Et unit.Set c,m to the true value, since we assume it is known to the inference procedure. This is a realistic assumption since data from titration experiments of known mRNA inputs can be used to independently determine these parameters.Note that the posterior (derived from single-cell or Fenton-Wilkinson approximated k-cell likelihoods) is a function of just the five parameters above and not of any additional measurement noise configuration parameter, since our model is unaware of the additional measurement noise added to simulated data.We performed the comparative two-condition simulations similar to the single-condition simulations described above , but with the following changes. Since we need to assess the performance of different methods in identifying changes in heterogeneity parameters between two conditions , we now tested three comparative scenarios. A gene hasπ of 0.8 vs. 0.5) between the two conditions,80% vs. 50% ON cells  of 10 Et and a standard deviation (σ) of 1 Et in both conditions, so that both conditions are simulated using an assay with the same logistic detection behavior . Grid-based posterior scan is now run separately on both conditions of a simulated gene to approximate the posterior distribution of a parameter in each condition, and the two resulting distributions jointly analyzed to approximate the posterior of the difference in the parameter between the conditions. Relevant code snippets implementing the single/two-condition simulations are provided (see Data Availability for download URL).In all these scenarios, the ON cells have a mean .Three levels of inference difficulty were used for single-condition simulations and four configurations of measurement noise (and one more configuration of no noise) were used for both single- and two-condition simulations. These configurations, in combination with different assay sensitivity (detection) settings, yield the diverse simulation scenarios employed to compare the performance of SC, KC and SCKC methods. A. Levels of inference difficulty when simulating gene expression profiles. Low inference difficulty simulation scenario for instance corresponds to a gene with a high fraction of ON cells with tight expression distribution. B. Measurement noise configurations when simulating gene expression profiles. Technical noise vs. average expression relationship observed in a dilution-series experiment was used to define the four realistic noise configurations for the simulation analyses. The estimate of total technical noise  when measuring different genes (different colored lines) at different dilutions of a standard bulk mRNA sample is shown. The average expression of the gene (x axis) is plotted against the standard deviation of the technical replicate measurements (y axis). The dilution series experiment was done using bulk mRNA pooled from human macrophages residing in diverse conditions in a related study—a total of seven dilutions were performed spanning a range of medium to high mRNA concentrations, and each dilution had eight technical replicates (except for one dilution which had only seven replicates due to an outlying measurement). Only genes that pass our quality control criteria are shown here: 1) The gene must exhibit a range of detection behaviors along the standard curve, or specifically its non-detect frequency should be at least 0.7 at the lowest concentration and at most 0.1 at the highest concentration , and 2) the measured non-detect frequency and Et value of the gene at different concentrations should be concordant, or specifically the baseline Et value corresponding to single transcript copy detection  should (TIF)Click here for additional data file.S2 FigSingle-condition simulation results for sample size n = 1000 shown in main text (TIF)Click here for additional data file.S3 FigSimulation results for sample size n = 1000 on assessing CHP differences between two conditions under different assay sensitivity and noise configurations. This figure is similar to the above (TIF)Click here for additional data file.S4 FigError in parameter estimates is significantly mitigated in two-condition compared to one-condition simulation settings, particularly for the σ parameter. The x-axis is showing the absolute error of a parameter , averaged across both conditions and all ten simulation runs; the y-axis is showing the absolute error of the parameter difference between the two conditions, averaged across all ten simulation runs. Both x- and y- axis are divided  by the true value of the parameter difference between the two conditions (if non-zero).(TIF)Click here for additional data file.S5 FigExperimental data were generated from untreated (CNT) and IFN+TNF-treated (IFNT) human macrophages, and the three methods assessed on this data. Density plots of parameter inferences for all gene-condition combinations (GCCs) successfully modeled by all three assessed methods are shown. Each plot compares the performance of the two indicated methods as follows: The x-axis indicates the difference in the two methods’ estimates (posterior means); the y-axis is the difference in the CrI width of the two methods’ estimates; and both x- and y- axis values are divided  by the average parameter estimate if non-zero . To compare two methods when ground truth is not known , we only consider GCCs whose parameter estimates are similar between the compared methods . Shown alongside are the histogram of red dots and the (method1 vs method2) percentages indicating the percentage of GCCs inside the red band whose method1 (or method2) CrI is tighter than that of the other method by at least the same band width of 0.2 units Click here for additional data file.S6 Fig2-fold-change (log2(FC)) in average gene expression between IFNT vs. CNT conditions are correlated (A-B) or close (C-D) to the “ground truth” log2(FC) estimated from bulk RNA-Seq data. These correlations are shown for different values of bulk RNA-Seq log2(FC) cutoffs (x axis) ) ≤ cutoff”). We similarly compute the closeness to true values, quantified conversely by the average squared error (square of the difference between the bulk RNA-Seq log2(FC) value and the log2(FC) value estimated by a given method, averaged across the genes under consideration). We first evaluate all methods using a starting set of 55 genes that have more than 5 observed ON cells in both conditions (the “ON-cells filter” that ensures sufficient data is available to estimate QVARKS parameters). We also evaluate our QVARKS methods on other starting gene sets obtained by combining the “ON-cells filter” with two other filters: i) genes whose log2(FC) CrI width estimated by our method is at most 5 units, so that the point estimate of DE used in these comparisons are reliable, and ii) genes that passed our AD-test based model assessment criteria. A. QVARKS SCKC and KC correlation to bulk RNA-Seq DE is comparable to that of MAST and SCDE across all scenarios, and QVARKS SC performs comparable to other methods once CrI filter is used to remove genes with large uncertainty in their estimates. B. QVARKS SCKC correlation to bulk RNA-Seq DE using ten random 50% downsamplings of the dataset Click here for additional data file.S7 FigEt measurement values, after assuming non-detects as zero expression .Average expression derived from single-cell data tended to be consistently lower than that obtained from 10-cell data (divided by 10) for transcripts expressed at medium or low levels. Note that average expression here refers to average of the 2(TIF)Click here for additional data file.S8 Fig2 fold change of their bulk/overall mean expression between the two conditions and the list of significant differentially expressed (DE) genes at adjusted P < 0.05, as determined by two methods. Though the DE results from both methods are not expected to be exactly equal (as explained in Methods), the concordance between them as shown here reinforces each method’s discoveries.For the genes successfully modeled in both IFNT and CNT conditions by our approach, we show the log(TIF)Click here for additional data file.S9 Figggplot2’s stat_smooth function with “rlm” method and default options). B. The changes in parameter estimates between IFNT vs. CNT are largely similar before or after performing the new per-well normalization designed to remove well-to-well variations shown in (A) . This comparison is done using the 39 overlapping genes out of the 41 and 47 genes passing model assessment (in both CNT and IFNT conditions) in our macrophage data before and after performing the per-well normalization respectively. Note that the per-well normalization factor subtracted from each gene measurement within a well is a weighted sum of the three ERCC levels in the well, where the weights are the corresponding coefficients of a linear regression model fitted separately for single/k-cell data in each condition (in R notation: “Median expression of highly expressed genes ∼ ERCC-0002 + ERCC-0003 + ERCC-0044”).We augmented our method with an additional pre-processing/normalization step that accounts for one of the key noise sources: well-to-well variation in sampling efficiency ). This (TIF)Click here for additional data file.S10 Fig2(π) − (1 − π)log2(1 − π)) captures the intuitive notion that a cell population containing a balanced number of ON and OFF cells is more heterogeneous than another where almost all cells are ON or almost all are OFF. A. Inferred alterations in entropy in IFNT treatment relative to CNT is shown for the genes successfully modeled in both conditions. The 23 differentially expressed genes identified using overall expression changes, along with whether they are increased or decreased, is also indicated. The 90% Crl is indicated by the lightly shaded lines around the mean of the posterior distribution. B. Genes with significant (adjP < 0.05) changes in entropy between the conditions (and gene RELA) are shown along the entropy function’s curve.The Shannon entropy function computed using π (entropy(π) = −π log(TIF)Click here for additional data file.S11 Figggplot2’s stat_smooth function with “lm” method and default options).Linear regression fit of changes in continuous variation (σ) as a function of changes in the average expression among ON cells  or ON rate  between IFNT and CNT conditions using the genes successfully modeled in both conditions. For genes with significant alterations in σ (from main text (TIF)Click here for additional data file.S12 FigFor genes with significant alterations in σ (DH genes from main text (TIF)Click here for additional data file.


RNA-Seq provides the capability to characterize the entire transcriptome in multiple levels including gene expression, allele specific expression, alternative splicing, fusion gene detection, and etc. The US FDA-led SEQC  project conducted a comprehensive study focused on the transcriptome profiling of rat liver samples treated with 27 chemicals to evaluate the utility of RNA-Seq in safety assessment and toxicity mechanism elucidation. The chemicals represented multiple chemogenomic modes of action (MOA) and exhibited varying degrees of transcriptional response. The paired-end 100 bp sequencing data were generated using Illumina HiScanSQ and/or HiSeq 2000. In addition to the core study, six animals  were sequenced three times, with two separate library preparations on two sequencing machines. This large toxicogenomics dataset can serve as a resource to characterize various aspects of transcriptomic changes  that are byproduct of chemical perturbation. Recent studies using deep sequencing technologies have provided a more comprehensive understanding of the processes that govern the transcriptome3. The rapid advancement in this field is in part enabled by various collaborative efforts that have employed a crowd sourcing model to assess data utility as it pertains to challenges in different application domains. One such effort is the FDA-led MicroArray Quality Control (MAQC)4–64–64–6 project to assess reliability and applicability of emerging technologies including RNA-Seq in the clinical and regulatory settings.Whole-transcriptome sequencing technologies, i.e., RNA-Seq, offer the capability to capture a high resolution picture of the transcriptomic landscape. With such data it is possible to evaluate the transcriptome at multiple levels including differential gene expression, allele specific expression, alternative splicing, gene fusion, and etc6, an RNA-Seq profiling dataset of rat liver samples was generated to assess the utility of RNA-Seq in safety assessment and toxicity mechanism elucidation. The liver was chosen because it is one of the most common target organ sites for toxicity and carcinogenicity in the rodent toxicological assessments7 and the regulatory community often considers mechanistic data (such as transcriptomics) in order to gauge the relevance of findings in rat liver to human health. The evaluation of such data takes place through what is call a mode of action framework. Drugs, toxicants and other environmental agents elicit toxicity in the liver by interacting with a variety of molecular targets that then elicit a complex cascade of biological processes that can either enhance or attenuate toxicity. The composite of these biological processes is referred to as a toxicological mode of action (MOA) which, as noted above, can, in part, be reflected in the agent-elicited transcriptional response. The SEQC toxicogenomics study evaluated the downstream genomic biology of chemicals representing seven MOAs in the rat liver with the purpose of characterizing the commonalities and distinctions of the different MOAs as measured by RNA-Seq and microarray.As a part of the third phase of the MAQC project (MAQC-III), also called SEquencing Quality Control (SEQC)As illustrated in 8. In this Data Descriptor, we provide additional information aimed at helping others reuse and interpret these data within their own research, including more detailed methods descriptions and fuller release of ancillary datasets .In average, about 20 million paired end reads (100 bp x2) were produced for each ratliver sample from one library preparation. In addition, six samples, including three rats treated by aflatoxin B1 and three vehicle control rats, were sequenced three times, with two separate library preparations on two different Illumina sequencing machines (HiScanSQ and HiSeq 2000). These technical replicates were used to evaluate the variability introduced by different batches of library preparation and sequencing. Sample annotation, read depth, and sequencing data mapping statistics were listed in 8. All biological samples employed in the studies described here were derived from the DrugMatrix tissue/RNA bank that is now owned by the National Toxicology Program (NTP). Details on the design and in life portion of these studies can be found elsewhere9. In short, male Sprague-Dawley [Crl:CD® (SD)|GS BR] rats  were housed individually in hanging, stainless steel, wire-bottom cages in a ventilated room  and received Certified Rodent Diet #5002 ; chlorinated tap water was available ad libidum. Animals were acclimated for one week to the testing laboratory prior to dosing. Test chemicals were administered orally (10 ml/kg body weight in corn oil or water) or via intraperitoneal, intravenous or subcutaneous injection . In order to ensure a maximal transcriptional response, 5 day maximum tolerated doses (MTD) of test chemical were administered to the study animals. The MTD was determined in a 5-day range finding study in which an MTD was determined as a 5 to 10% reduction in body weight relative to control. Animals were dosed once daily for 3, 5, or 7 days, depending on the chemical, and livers were harvested 24 h after the last dose. For example, a 3 day study would start with a dose at time 0 followed by subsequent dosing at the 24 and 48 h time points. The animals would be euthanized at 72 h having received 3 repeat doses of the test chemical. Hence, animals from the 5 day studies received 5 repeated doses of chemical and animals in the 7 day studies received 7 repeated doses with each dose separated by 24 h. Animals were randomly assigned to test chemicals using a computerized body-weight stratification procedure. All toxicity studies described here were performed in a contract laboratory and therefore investigators from the sponsor were blinded to group allocation during the study. Animals were handled in accordance with United States Department of Agriculture and Code of Federal Regulations Animal Welfare Act . After dosing, all liver samples were harvested as 100 mg punches using six millimeter disposable biopsy punches. An appropriately staggered schedule was employed so that the harvest times were accurate to ±30 min of the designed dose-to-harvest interval.This description on the housing of animals is expanded from descriptions in the related research manuscript8. RNA samples for RNA-Seq were derived from the NTP DrugMatrix RNA bank. The following criteria were applied in the selection of treated samples: act via one of 7 modes of toxicological action, have existing Affymetrix whole genome GeneChip® Rat Genome 230 2.0 array data in the DrugMatrix Database for the same RNA sample, be derived from repeat dose toxicity studies  and be from liver. Dosing level and duration for each chemical are listed in This description on sample selection is expanded from descriptions in the related research manuscriptThe samples were split into 2 sets, training and test, which were sequenced in two separate batches and run on different days. The splitting of the samples into 2 sets was done to allow for the evaluation of classifiers derived from the training data. There were 63 samples in the training set and 42 in the test set. Of the 63 samples in the training set 45 were derived from rats treated with chemicals and 18 were control samples (6 samples for each of 3 employed combinations of vehicle type and delivery route). Thirty six of the test set samples were derived from chemical treated animals and 6 were from vehicle and route matched controls. There was only one combination of vehicle type and delivery route employed in the test set. For each chemical there were three rats treated at the same dose level and there were three chemicals chosen for each MOA for the training and test set. All chemical treatments were different between the training and test set. Five MOAs  were represented in the training set and 4 MOAs  were in the test set. Two of the MOAs (PPARA and CAR/PXR) were duplicated from the test set with different test chemicals and two were without representation in the training set .8. Prior to library preparation, RNA quality was checked with an Agilent Bioanalyzer  and all of the RINs were between 8.7 and 10 except four samples with RINs between 7.4 and 7.9 , followed by polyA mRNA selection and fragmentation, first and second strand synthesis, end repair, adenylation of 3′ ends, and index adapter ligation. Each library was subject to 15 cycles of PCR amplification, and the size distribution was examined on an Agilent Bioanalyzer  using a DNA 1000 chip. Each sample used one out of 12 unique indices (Illumina). All libraries displayed a band between 200–500 bp with a peak at approximately 260 bp. The libraries were quantitated with Qubit 2.0 Fluorometer , loaded at a concentration of 8.6 pM with four or six libraries pooled in one lane of Illumina V3 flow cell, and sequenced on either Illumina HiScanSQ or HiSeq 2000 Sequencing Systems with paired-end 100 bp reads.These descriptions are slightly expanded from descriptions in the related researchmanuscript and 7.9 . LibraryLibrary preparation was performed in two batches with several months separating the batches. One batch contained samples in the training set and the other batch contained the test set samples. In order to assess the variation due to library preparation, six samples from the training set (three with aflatoxin B1 treatment and three matched controls) were included in the batch for the test set to track the technical reproducibility. A total of 111 libraries were constructed.For the training set, sequencing was separated in two batches with four libraries pooled perlane, one on an Illumina HiScanSQ machine and the other on an Illumina HiSeq 2000 machine. In order to assess the variation introduced by different sequencing instruments, libraries of the six samples described above (three treated with aflatoxin B1 and three matched controls) were sequenced in both batches. For the test set, all libraries were sequenced in one batch with six libraries pooled per lane on an Illumina HiScanSQ. There were six libraries failed in a lane (due to a clogging of the fluidic system associated with that lane) that were re-sequenced with the read length reduced to paired-end 50 bp. Thus a total of 117 paired-end sequencing runs were obtained on the 111 libraries constructed from 105 RNA samples. Depths of ~23–25 million paired-end 100 bp reads were generated for each library sequencing run . One libAfter base calling, adapter trimming, and barcode demultiplexing using the sequencer manufacturer's software, sequence data with quality scores were transferred to the FDA’s National Center for Toxicological Research (NCTR) for integrity check and reformatting. A naming convention was then applied to the uniformly reformatted data with a compact digital signature (MD5 checksum) computed for each data file. The name of each data file coded the following fields with valid values in parentheses: the sequencing facility , the facility director (‘WANG’), flow-cell ID, RNA sample ID (5 digit number), sample origin (‘L’ for liver), chemical name or control abbreviation , vehicleRNA-Seq data from 116 library sequencing runs were deposited in the Gene Expression Omnibus(NCBI) under accession number GSE55347 (Data Citation 1). GSE55347 contains a processed data file listing the expression measurements for each of the rat and ERCC transcripts, which is in a tab-separated text format with the following columns: Gene_Name, Annotation_Method, NCBI_Gene_ID, RefSeq_Transcript_ID, Chromosome, Strand, Start_Base, End_Base, Gene_Description, and gene expression index for each of the 116 library sequencing runs. The complex mapping relationship between the microarray probesets and Refseq/Aceview genes was also embedded in the processed data file (‘GSE55347_TGxSEQC_GeneExpressionIndex_Magic_20120831_116samples.txt.gz’) with the first column ‘Gene_Name’ as the unique gene identifier. One gene may be mapped to multiple microarray probesets or Refseq genes. A Supplementary File was also included to explain the mapping and quantification procedure. The GEO dataset provides a link to the corresponding NCBI Sequence Read Archive (SRA) accession that contains the raw sequencing data from all sequencing sites. The corresponding microarray data were deposited in GEO under accession number GSE47875 (Data Citation 2), and qPCR data with descriptions in figshare (Data Citation 3). All SEQC (MAQC-III) data sets are available through GEO (Data Citation 4).The table column headers for RNA ID,Sample Description,RNA-Seq Run ID,Animal ID,Affy_ID (Microarray),Chemical Name,Chemical Name Abbreviation,Vehicle Type,Route,Duration (days),Dose (mg/kg/day),Mode of action (MOA),Original Set Annotation,RNA Extraction Date,Microarray Hybridization Date,RNA-Seq Run Date,RIN Measured Before RNA-Seq Library Preparation,ERCC_Mix Spike-in,Machine,Sequencing_Lab,Sequencing_Lane,Number_of_Reads,Read_Length (nt),GC_Content_in_the_reads,%Reads_Aligned,Number of Aceview genes expressed (with at least 4 reads mapped),Average insert length measured after alignment (bp),Number of mismatches per kilobase aligned,%Reads aligned to ERCC transcripts,%Reads aligned to rat rRNA,%Reads aligned to rat mitochondria,%Reads aligned to rat RefSeq genes,%Reads aligned to rat genome,fastq file_name_1,fastq file_name_2.11. Since the relative change was about 10%, we assumed the effect may not be pronounced. The sequence error rates were plotted in 12. The HiSeq 2000 machine performed better than both HiScanSQ machines. A technical glitch associated with a specific lane caused higher error rates for six libraries sequenced on a HiScanSQ machine.The first aim of QC and validation was to assess the variation introduced by library preparation and sequencing batches by taking advantage of the ERCC spike-ins and technical replicates built into different batches. ERCC mixes 1 and 2 were spiked in to RNA samples prior to library preparation. After aligning the reads to ERCC transcripts, RefSeq rat genes, and rat genome, we chose four different quality metrics including percentage of reads aligned to ERCC transcripts, percentage of reads aligned to RefSeq rat genes, average insert length measured after alignment, and base error rate measured after alignment. As shown in 2 greater than 0.98 for each sample). According to principle component analysis (PCA) of these six samples  and found out that the expression levels were highly correlated  with RNA_ID ‘98912’, was discovered as an outliner, probably due to mislabeling. The sample was then excluded from data deposition and any further analysis.Finally, we computed the correlation of RNA-Seq expression levels between samples treated by the same chemical and compared the results with the corresponding ones calculated from the microarray data. The calculation of Pearson correlation was repeated multiple times and plotted with different numbers of genes selected by descending expression order . The res15, TopHat16, and Cufflinks17, could be used for mapping and quantification of RNA-Seq data. Several reference genomes of rat could be used as the mapping template, such as RefSeq, UCSC, Aceview and Ensembl. Many publicly available software packages, e.g., limma18, edgeR19, and DESeq20, could be used for differential gene expression analysis of the read count data. Six pipelines were built for mapping and quantification of RNA-Seq data and documented in our related research manuscript8.Many publicly available software packages, e.g., BowtieThe three technical replicates were included for each of three samples treated with aflatoxin B1 and three matched controls, with the same library run on different sequencing machines or separate libraries construction derived from the same sample. These replicate library/sequencing runs could be used for library preparation and sequencing platform quality control or as super deep sequence data for aflatoxin B1 study by merging the reads for each sample from all the sequencing runs.How to cite this article: Gong, B. et al. Transcriptomic profiling of rat liver samples in a comprehensive study design by RNA-Seq. Sci. Data 1:140021 doi: 10.1038/sdata.2014.21 (2014).


Transcriptome-based biosensors are expected to have a large impact on the future of biotechnology. However, a central aspect of transcriptomics is differential expression analysis, where, currently, deep RNA sequencing (RNA-seq) has the potential to replace the microarray as the standard assay for RNA quantification. Our contributions here to RNA-seq differential expression analysis are two-fold. First, given the high cost of an RNA-seq run, biological replicates are rare, and therefore, information sharing across genes to obtain variance estimates is crucial. To handle such information sharing in a rigorous manner, we propose an hierarchical, empirical Bayes approach (R-EBSeq) that combines the Cufflinks model for generating relative transcript abundance measurements, known as FPKM (fragments per kilobase of transcript length per million mapped reads) with the EBArrays framework, which was previously developed for empirical Bayes analysis of microarray data. A desirable feature of R-EBSeq is easy-to-implement analysis of more than pairwise comparisons, as we illustrate with experimental data. Secondly, we develop the standard RNA-seq test data set, on the level of reads, where 79 transcripts are artificially differentially expressed and, therefore, explicitly known. This test data set allows us to compare the performance, in terms of the true discovery rate, of R-EBSeq to three other widely used RNAseq data analysis packages: Cuffdiff, DEseq and BaySeq. Our analysis indicates that DESeq identifies the first half of the differentially expressed transcripts well, but then is outperformed by Cuffdiff and R-EBSeq. Cuffdiff and R-EBSeq are the two top performers. Thus, R-EBSeq offers good performance, while allowing flexible and rigorous comparison of multiple biological conditions. Obtaining the sequence of a human genome once cost on the order of $100,000,000 dollars and took years; now, with next generation sequencing techniques, we obtain such a sequence for on the order of $10,000 to $1,000 in days/weeks ,3,4,5. TOne application of next generation sequencing is transcriptomics, where an entire set of mRNAs is sequenced ,7. This The process, of course, starts with cells, from which mRNA is extracted. Note that, due to the biological variability, the extracted mRNA levels may differ considerably among different biological samples. Then, a so-called library is created from the isolated mRNA. The library typically consists of nucleic acid fragments, but can also consist of full-length nucleic acid molecules ,8. It isA subset of the library molecules is then sequenced, producing what are called reads or DNA sequences. Because only a subset of the library molecules is chosen for sequencing, there is variability and potential bias in the selection process . Furtherde novo assembly and, subsequently, count the number of reads that align to particular genes. Such aligned reads are also termed mapped reads. Regardless of which of these two are done ; however, there will inevitably be uncertainty in assigning a read to a particular gene; a single read may well align to multiple genes.With this sample of reads, one can either align them to a reference genome or perform a Some researchers consider these count data to be the final product of an RNA-seq experiment. However, such count data are not usually linearly proportional to the original number of full length transcripts for two main reasons. First, longer transcripts generate more reads simply due to their length. Second, because a single gene typically codes for multiple transcripts with different lengths, a gene’s expression in terms of the expected number of counts may remain constant even while the number of transcripts being produced from that gene changes. Moreover, since one eventually wants to compare the results of two or more different sequencing runs, it is also necessary to correct for the total sequencing depth, which is related to how large the subset of the sequenced library molecules is. These reasons are why many people take an additional step to transform these gene counts to units of reads (R) or fragments (F) per kilobase of transcript length per million mapped reads (RPKM or FPKM), which are linearly proportional to original transcript levels ,7.Finally, one can assign an FPKM value to each individual transcript for every gene. This transcript assignment process also introduces uncertainty, because only reads that happen to contain exons or exon-exon boundaries that are unique to particular transcripts can inform this transcript assignment process and such reads may be rare . It is wIn what follows, we first analyze the RNA-seq process described by 2.2.1.The method developed in ,11 charaj indicate j-th transcript and xj = jx1,...,jIx be the expressions in the particular condition denoted: x, where I is the number of expression measurements we can take in this condition. The method treats jx as a random sample with a transcript- and condition-specific mean. Thus, we have that ij|μjX are iid with a pdf given as fXij|μj (ijx). To allow information sharing among all transcripts and conditions, a prior distribution, π, is assumed on jμ.Let I} is partitioned into two subsets, s1,s2, each containing indices for a corresponding condition ). For each transcript, we aim to investigate a null hypothesis that assumes equivalent expression (jEE) among conditions, against the alternative that a transcript is differentially expressed (jDE) in one of conditions.In a simple setting, if we investigate differences in expression between only two conditions, the index set {1,...,The pdf in each expression pattern (’EE’ or ’DE’) can be written as follows:jEE:jDE:p and 1 − p denote the fraction of all genes that are differentially expressed and equivalently expressed among conditions. Then, the pdf for xj takes the following form:Let Using Bayes’ formula, we can write the pdfs for each hypothesis:jx and its conjugate prior to the mean are usually known. EBarrays uses the EM algorithm to compute the estimates of parameters for both distributions, as well as mixing proportions of genes supporting the investigated hypothesis.The form of both the conditional distribution of m DE patterns, which yield a total of m+1 patterns .This method is easily generalized for more than two expression patterns. A pattern corresponding to the null hypothesis, where each sample has the same underlying mean, is always distinguished. Consider jx, corresponding to kf is density corresponding to the k-th pattern, as in [kp is the fraction of genes supporting the corresponding pattern (hypothesis). Consequently, the posterior probability of expression pattern, k, for gene, j, is:The formula for the marginal distribution of n, as in , and pk 2.2.Usually, Cufflinks-processed data does not have replicates. To use the EBarray framework, replicates need to be generated from the Cufflinks-specified normal distribution  for geneπ ∼ N and an n-dimensional input for gene j, after evaluating the integral as detailed in [f, which is the pdf of a normal distribution with mean, μ = , and covariance matrix:n×n is an identity matrix and n×nM is a matrix of ones.In this setting, ailed in , followsWe adapted the EBarrays R code to account for the normal-normal model with modified variances by modifying the previously built-in LNNMV model  is a bivariate normal random variate with zero marginal means, marginal standard deviations, σ1 and σ2, and the correlation coefficient, ρ. In the case of the abundance model, tA considered in the paper, the vector,  = , is a joined MLE and the convergence above follows from the general theory of the likelihood-based estimation . Sincenη̂ → η in probability, therefore, due to Slutsky’s theorem  is a zero mean normal variate with the standard deviation, γ, such thatAccording to e, e.g.,  Chapter e, e.g.,  chapter 2.4.× n measurement matrix, according to the Normal-Normal model—reflecting 1,000 transcripts under one condition with n replicates. The number of replicates is also referred to as the sample size.In order to evaluate the performance of our method, we investigated its ROC plots for several different parameter settings under the normal-normal model assumption. The control dataset was created by generating a 1, 000 The treatment dataset was created in the same way, but expression values for 10% of the transcripts (randomly chosen) were generated with altered parameters in order to simulate the differential expression. Only over-expression was considered.To make the simulations as close to real data as possible, the parameters in the control setting (the mean and the variance) for the generation from the normal-normal model were taken from real data .Mean difference in expression:Fixed: variance—taken from Cufflinks, sample size: 10x mean difference, and the remaining 5% had 2x mean difference in expression from the “control” group. The value of x was varied from 0, 5, 10,..., 50, and these different values yield ten ROCs visible in Investigated: influence of difference in expression in 10% of transcripts (“treatment” group). Different levels of differential expression were generated according to the following: 5% of transcripts had Variance:Fixed: mean difference in expression for DE pattern: 5% of transcripts: 50, 5% of transcripts: 100. Sample size: 10x ∈{0, 5,..., 50}, yielding 10 ROCs in Investigated: influence of variance. For the whole “treatment” dataset, we created the ROCs for different additional variance levels. The cufflinks variance was increased by Sample size:Fixed: Mean difference in expression for DE pattern: 5% of transcripts 50, 5% of transcripts 100. Variance: Cufflinks variancen was investigated at 10 values: n ∈ {2, 4,..., 22}, each corresponding to a line in Investigated: Influence of the sample size. The sample size The following three settings were investigated:2.5.In order to make comparisons between methods, we needed a benchmark dataset that would provide a common base for all methods. To ensure similarity with experimental conditions, we started with real data published in ,16. The The treatment (sam file format) dataset was created by first randomly choosing 100 reads from the control (sam) file. Each of those reads was replicated a number of times corresponding to its order: the first read was copied once, the second twice, and so on, until the 100th was copied 100 times.To retrieve count datasets for DESeq and BaySeq methods, the created control and treatment files (sam file format) have to be further processed. We used the HTSeq software  recommenThe final step of creating the datasets was adding different level of noise to the reads, as described in the following section.2.6.g. For each read in this subset, we replicate this read gy times, where gy is taken from a negative binomial distribution, that is, gy∼NB. The new noisy number of reads for a particular gene, gX, given that this gene had N original reads, is therefore given as:For the benchmark datasets, to reflect real data, we are adding noise to the RNA-seq data by considering separately reads mapping uniquely to gene, r parameter that is equal to the sum of the individual r parameters. Therefore:It is well-known that a sum of independent negative binomial random variables is itself negative binomially distributed, with a new Given this, along with the properties of expected values and variances of random variables, we have the following:Var(gX) = E(gX):It can be shown that when grc< r for underdispersion or grc>r for overdispersion. Because RNA-Seq data are typically overdispersed (Var(gX) > E(gX))[g, we used the following scheme for obtaining proper g, rgp parameters:(1)gp uniformly from distributions U.Generate (2)gp to generate appropriate cr uniformly, according to distribution U.Use Thus, given this negative binomial noise generation model, we can determine the mean and then choose  > E(Xg)),14,19, t3.3.1.Most of the existing models for RNA-seq analysis employ some form of a parametric distribution of gene counts, typically a gamma-Poisson or a negative binomial, like, e.g., Bioconductor software  suites: N, the total number of mapped reads, n, and the number of molecules in the library corresponding to the gene, i, be im. If the capture process is unbiased (we consider capture bias and its variability later), then the probability of capturing a molecule that corresponds to gene i, which we denote as the probability of success for gene i, is ip = i/Nm. Because by experimental construction the library size is very large compared to the total number of mapped fragments (N ≫ n) and only in very exceptional circumstances is an expressed biological entity represented very few times in the library (im ≫ 1), then for all practical purposes, the probability of success is constant for all capture events. Thus, from the viewpoint of gene i, one can cast this RNA-seq experiment as a series of n trials, where in each trial, there is some probability of success, ip. Now, if we denote by ix the number of reads that map to gene i, given the properties of the RNA-seq experiment as outlined above, the probability of gene i having ix reads follows a binomial distribution [We begin by exploring what an appropriate description of the gene count data may be, and we focus first on the capture step, where members of the library are chosen for sequencing. Let the total number of molecules in a library be ribution .ip ≪ 1. Since the mean and variance of this aforementioned binomial distribution would be, respectively, μ = iNp and σ2 = iNp(1 i− p) [σ2/μ =(1 i− p)) would be approximately one for most genes, similar to that predicted by a Poisson distribution . Therefore, in a situation where the same library is sequenced multiple times with the same total number of mapped reads n, a Poisson distribution should be an adequate representation for most gene counts. This was indeed found to be the case for 99.5% of genes from human tissue samples [Given the large number of genes in any organism, in general, (1 − pi) , the Fan samples .ip, increases, then the Fano factor should decrease, leading to so-called “underdispersion” relative to a Poisson model. However, technical replicate data from sequencing runs using the same library show that the opposite is true; as the mean number of counts increases, so too does the Fano factor  or permutation testing (SAM) and are, therefore, more vulnerable to the small sample size effects.One goal of an RNA-seq experiment is to determine what transcripts (or genes) have significantly different expression levels between various biologically-meaningful conditions. In developing a new method for differential expression analysis, we kept in mind the following three key features of current RNA-seq experiments. First, because count data are not always linearly proportional to the original number of transcripts in the biological sample, converting counts to FPKM measurements for the purposes of differential expression testing is warranted when information sharing across genes is desired, and we wish to remove the size effect. Second, the monetary cost of a single RNA-seq run is high, so having a large number of biological replicates is unlikely. However, differential expression analysis relies on a precise estimate of a transcript’s variance, which is problematic with few replicates and impossible with no replicates. Therefore, information sharing across transcripts to improve variance estimates is necessary. Third, often, one would like to compare more than two biological conditions, so it is preferable to have the ability to do more than pairwise comparisons. Here, we integrate two previously developed software packages, Tophat-Cufflinks ,9 and EBtÂ, in FPKM for each transcript, t, which is given as:tC is a normalizing constant for transcript, t , gX̂ is the number of fragments mapping to gene, g (to which transcript t belongs), and tγ̂ is the fraction of gX̂ attributable to transcript, t (0 ≤ γ̂t ≤ 1). Cufflinks also returns an estimated confidence interval for tÂ, from which, based on the approximately normal behavior of tA .Before formulating an empirical Bayes approach to differential expression analysis, we must first consider how to convert the raw RNA-seq data into FPKM. The Tophat-Cufflinks pipeline is well-suited for this task, taking into account all the forms of bias and variability shown in  At , which specify an associated normal distribution. However, these parameters cannot be used as a direct input to EBArrays, as it expects expression values for various conditions. Therefore, to couple these two pieces of software, for each transcript, we generate M replicates from a normal distribution with mean, tÂ, and variance, V̂At, and, then, use these replicates as the input to EBArrays (see Methods). As indicated already above, we call this combined pipeline R-EBSeq.The output of the Tophat-Cufflinks pipeline for every condition are estimates of the mean and variance of vs. the true positive rate are called receiver operator characteristic (ROC) curves. An ROC curve along the x = y line implies a very poor algorithm that performs no better than random choice, whereas an ROC curve that peaks high above the x = y line, at low x values, implies a very good algorithm. We investigated how three characteristics of a transcript affect the ROC curves: the difference of means between two conditions . Such plots of the false positive rate 3.3.S. cerevisiae [There are several packages that currently exist for performing differential expression analysis on RNA-seq data. One method that is also based on FPKM is Cuffdiff, which is part of the Cufflinks pipeline . We alsorevisiae  at the li.e., those not coming from sequencing reads), we found that increasing the number of “replicate”  used for R-EBSeq always resulted in better performance; with this real data, there is an optimum number of “replicates” . Most of the investigated differentially expressed transcripts have tγ =1 (data not shown). Thus, a potential direction for future work with R-EBSeq is to improve the normal distribution assumption for FPKM data. However, changing the normal assumption may slow down the software significantly, since using the normal distribution allows for a clean analytic solution of the posterior distribution in the EBArray model and, therefore, fast computation. It is not clear whether appropriate changes to the FPKM distribution assumption would also be solvable analytically. In a practical setting, we recommend that one selects the number of replicates based on some self-consistency test, like e.g., the cross-validation Shao [Before comparing the four software packages, we had to select operating parameters for R-EBSeq. Although above idealized data  were treated with etoposide, a DNA-damaging agent, or DMSO (control) and then assayed for gene expression via RNA-seq [A convenient feature of R-EBSeq that was inherited from EBArrays is the ability to easily specify patterns for multiple condition comparison. We illustrate this feature here using RNA-seq datasets for a DNA damage response with or without iewed in ). RKO co RNA-seq . One bio RNA-seq ,27,28,29To determine potential p53- and etoposide-specific genes, we first specified “patterns”. Pattern specification is the aspect of R-EBSeq (and formerly EBArrays) where one defines how many unique population means we expect and which sample we would expect which population mean to correspond to. In our case, we have four samples and three different conditions we are interested in: p53-specific, etoposide-specific and non-specific (null). To specify these conditions, we consider two different population means, 1 and 2, which are distributed as “patterns” among the conditions according to PEREZ_TP53_TARGETS and PUJANA_ATM_PCC_NETWORK. Although it may seem illogical that a list of p53 targets have significant overlap with a list of etoposide-specific genes, this is actually not that surprising and, in fact, expected. There are many proteins that have functional redundancy with p53, so if p53 is not there, it is likely other proteins serve to provide a compensatory effect in the presence of etoposide. Because many p53 targets have been defined as those which respond to DNA damage and many of these targets are likely regulated by compensatory actions, one would expect a large overlap. For the second hit, ATM is a well known DNA-damage related protein, and therefore, its presence is understandably etoposide-related. Some top hits for the p53 pattern are UV response sets (DACOSTA_UV_RESPONSE_VIA_ERCC3_UP and ENK_UV_RESPONSE_KERATINOCYTE_UP) and stem cell related sets (NUYTTEN_EZH2_TARGETS_DN and BENPORATH_NANOG_TARGETS). The presence of the UV sets imply that there may be some distinct pathways that are utilized in response to etoposide-mediated DNA damage vs. UV-mediated DNA damage and that p53 allows response to both types. The presence of the stem cell sets are in line with current thoughts that p53 plays a large role in stem cell function . Thus, 4.Here we have investigated several aspects of RNA-seq differential expression analysis. In the first part of the paper, we have suggested the empirical Bayes method for RNA-seq data analysis based on FPKM measurements , which combines two previously developed and powerful software suits: Cufflinks and EBarrays. R-EBSeq has two important features. First, it shares information across genes (or transcripts) to get better variances estimates when there are few or no replicates. The use of FPKM makes the sharing straightforward and, unlike in some other empirical Bayes RNA-seq software, does not require additional pre-processing. Second, it is capable of doing multiple conditions comparisons in an easy-to-implement manner, as we demonstrate with experimental data. In the second part of the paper, we created the benchmark dataset for RNA-seq differential expression analysis and used it to compare R-EBSeq to Cuffdiff, DESeq and BaySeq software suits developed for RNA-seq analysis. We find that FPKM-based Cuffdiff gives the best performance, followed closely by R-EBSeq. DESeq and BaySeq, which are count-based methods and do not perform as well. Overall, R-EBSeq seems to offer reasonable performance with the flexibility of multiple comparisons and rigorous treatment of information sharing when there are few replicates.


Recently, rapid improvements in technology and decrease in sequencing costs have made RNA-Seq a widely used technique to quantify gene expression levels. Various normalization approaches have been proposed, owing to the importance of normalization in the analysis of RNA-Seq data. A comparison of recently proposed normalization methods is required to generate suitable guidelines for the selection of the most appropriate approach for future experiments.In this paper, we compared eight non-abundance  and two abundance estimation normalization methods (RSEM and Sailfish). The experiments were based on real Illumina high-throughput RNA-Seq of 35- and 76-nucleotide sequences produced in the MAQC project and simulation reads. Reads were mapped with human genome obtained from UCSC Genome Browser Database. For precise evaluation, we investigated Spearman correlation between the normalization results from RNA-Seq and MAQC qRT-PCR values for 996 genes. Based on this work, we showed that out of the eight non-abundance estimation normalization methods, RC, UQ, Med, TMM, DESeq, and Q gave similar normalization results for all data sets. For RNA-Seq of a 35-nucleotide sequence, RPKM showed the highest correlation results, but for RNA-Seq of a 76-nucleotide sequence, least correlation was observed than the other methods. ERPKM did not improve results than RPKM. Between two abundance estimation normalization methods, for RNA-Seq of a 35-nucleotide sequence, higher correlation was obtained with Sailfish than that with RSEM, which was better than without using abundance estimation methods. However, for RNA-Seq of a 76-nucleotide sequence, the results achieved by RSEM were similar to without applying abundance estimation methods, and were much better than with Sailfish. Furthermore, we found that adding a poly-A tail increased alignment numbers, but did not improve normalization results.Spearman correlation analysis revealed that RC, UQ, Med, TMM, DESeq, and Q did not noticeably improve gene expression normalization, regardless of read length. Other normalization methods were more efficient when alignment accuracy was low; Sailfish with RPKM gave the best normalization results. When alignment accuracy was high, RC was sufficient for gene expression calculation. And we suggest ignoring poly-A tail during differential gene expression analysis.The online version of this article (doi:10.1186/s12859-015-0778-7) contains supplementary material, which is available to authorized users. Bioinformatics studies over the past few decades have shed light on topics such as sequence analysis, structural analysis, and network and systems biology , 2. In rIn general, the RNA-Seq pipeline for differential expression analysis contains five steps , 15. FirAs with microarrays , 17, varIn addition to previously introduced traditional normalization methods, two abundance estimation normalization methods have recently been developed for accurate estimation: RNA-Seq by Expectation-Maximization (RSEM)  and SailAmong the various normalization methods, the one that produces the best results must be determined. A comparison of the recently proposed normalization methods will provide clearer guidelines for future analyses. In a previous study , severalIn this study, for gene expression analysis, we first compared eight non-abundance estimation normalization methods  and then compared two abundance estimation normalization methods (RSEM and Sailfish) by combining them with the non-abundance methods. The experiments are based on real Illumina high-throughput RNA-Seq data used in the MicroArray Quality Control (MAQC) project from two RNA samples of brain tissue and a mixture of tissue types, with read lengths of 35 and 76 nucleotides. Simulation data were also obtained using a model derived from RNA-Seq data with a sequence length of 76 nucleotides by RSEM simulator. Reads are aligned with human genome data obtained from the University of California Santa Cruz (UCSC) Genome Browser Database. First, large-scale distribution and detail alignment results were represented. Then, we analyzed Spearman correlation coefficients between the normalization results from RNA-Seq and the results of MAQC TaqMan quantitative reverse transcription polymerase chain reaction (qRT-PCR) of two samples for 996 genes. We have provided a detailed comparison of results among the different normalization methods. In addition, the impact of the poly-A tail was verified by adding 0, 5, 10, 15, 20, or 25 adenine bases to the end of the transcript data. Based on this study, we propose practical recommendations on the appropriate normalization method to use, and determine the effect of adding a poly-A tail to gene expression analyses.For real reads of 35 nucleotides, the range of alignment counts was from 0 to approximately 9,000,000. For approximately 70 % of the genes, the alignment count was between 1000 and 100,000. Only 14 genes had a count number greater than 1,000,000. In one respect, this proves that only a small number of genes can be differentially expressed. For each run, the total number of reads was approximately 11,000,000–16,000,000.The number of reads with at least one alignment (with no poly-A tail) was approximately 500,000–700,000 , and the number with total alignment was approximately 1,500,000–5,000,000.For real reads of 76 nucleotides, the number of reads with at least one alignment (with no poly-A tail) was approximately 300,000–500,000 . The number of alignments decreased as read length increased. This proves that longer reads give more accurate alignment results.The low rate of alignment arose because we only used 996 genes. If we had used full human genes as references, this would have been very different. For example, on real reads of 35 nucleotides, the alignment rate was approximately 55 %, and on real reads of 76 nucleotides, the alignment rate was around 65 %.Detailed alignment results for each accession can be found in Additional file After alignment we removed reference transcripts with no mappings for each accession.We investigated Spearman correlation coefficients between normalization results from RNA-Seq using each normalization method and the values from qRT-PCR.Table All the Spearman correlation coefficient results were larger than 0, indicating that the results from all the RNA-Seq normalization methods were positively correlated with the qRT-PCR values. Furthermore, all p-values were < 2.2e-16 and the null hypothesis was rejected, which proves that all the results of the normalization methods correlated with the qRT-PCR gene expression values. In another aspect, the RNA-Seq data were compatible with the real time PCR for gene expression analysis.As can be seen in Table Table From the comparison results shown above, normalization methods are not necessary for all sequence data. Inter-sample normalization methods, such as TMM, DESeq, and Q, which scale sample size, do not noticeably improve gene expression, regardless of read length. However, RPKM is likely to be more efficient when alignment accuracy is low. Similarly, for read data on lengths of 35 nucleotides, of the two abundance estimation normalization methods, Sailfish with RPKM, which is also quite an efficient combination because it is alignment-free, gave better normalization results than RSEM. However, when alignment accuracy is high, RC seems to be adequate for gene expression calculations in real experiments.For all details of the Spearman correlation coefficient results, please download Additional file To determine whether a poly-A tail can affect the alignment results, we evaluated the results of accession SRX016359 with the addition of various poly-A tail lengths to the end of the reference transcript data. Because the read length was 34 nucleotides, we used poly-A tail lengths of 0, 5, 10, 15, 20, and 25 adenines. Table Table Normalization has proved to be important in the analysis of gene expression using RNA-Seq technology. Recently, various normalization approaches have been developed to accurately identify differentially expressed genes. To provide a guideline for choosing among these methods, we compared eight non-abundance estimation normalization methods  and two abundance estimation normalization methods (RSEM and Sailfish) in this study. The experiments were based on the real Illumina high-throughput RNA-Seq data used in the MAQC project on two RNA samples, brain tissue (HBR) and a mixture of tissue types (UHR), with read lengths of 35 and 76 nucleotides. Simulated data were obtained using an RSEM simulator with parameters derived from real data with a length of 76 nucleotides. Reads were mapped with human genome data using the Bowtie tool. First, we showed a large-scale and detailed distribution of all alignments. We then investigated the Spearman correlation coefficient between the normalization results of each method and the values of MAQC TaqMan qRT-PCR. For accessions with a read length of 35 nucleotides, of the eight non-abundance estimation normalization approaches, RPKM achieved a higher correlation value than RC, UQ, Med, TMM, or Q, proving that consideration of the transcript length in normalization is quite effective. By using effective transcript lengths, ERPKM did not improve the normalization results compared with RPKM. After combining the abundance estimation normalization methods, the normalization results were improved. In particular, Sailfish with RPKM, which we recommend researchers use as a normalization method in future analyses, can almost replace qRT-PCR, as a correlation of nearly 0.8 was observed. Moreover, Sailfish is alignment-free and more time-efficient than RSEM. RSEM also produced good results. However, for data with read lengths of 76 nucleotides, none of the normalization methods improved the correlation results. Therefore, we conclude that when alignment accuracy is high, RC is sufficient for gene expression calculation in real experiments. In addition, the impact of poly-A tail was determined by adding adenines (0–25) to the transcript data. The results showed that by adding short poly-A tails, few relative alignments were required, but longer poly-A tails caused irrelevant alignments to be included. However, our results were not improved. Thus, choosing appropriate poly-A tail lengths may improve differential analysis, but did not appear to have an impact in this study. Therefore, we suggest that researchers do not need to consider ploy-A tails in the normalization step in gene differential expression analysis.http://genome.ucsc.edu/) for use as reference transcript data [Human genome data  were obtained from the UCSC Genome Browser Database  [High-throughput RNA-Seq data were collected from the Sequence Read Archive  . Data wiAs shown in Table We simulated 5-reads data for brain tissue and UHR separately , which consisted of 20 million single-end reads with quality scores each using the RSEM simulator. We derived model parameters when calculating RSEM expression levels from SRX080222 for human brain simulated data, and from SRX080224 for human UHR simulated data. The parameters used in the simulation are given in the following command../rsem-simulate-reads REF/hg uhr_RSEM.stat/uhr_RSEM.model uhr_RSEM.genes.results 0.2 20000000 uhr_readhttp://www.ncbi.nlm.nih.gov/geo/) [MAQC TaqMan qRT-PCR is the benchmark for detecting and quantifying RNA targets . It can ov/geo/) , and comBowtie is widely used in genome sequencing studies for aligning short DNA sequence reads to large genomes because of its fast and memory-efficient alignments . Bowtie Eight non-abundance and two abundance estimation normalization methods were evaluated in this paper, and calculations were performed on each gene. We defined the gene count of one gene as all the mapped reads of each run.RC, UQ, and Med use simple descriptive statistical methods that were also widely used in other areas for capturing data characteristics. TMM, DESeq, and Q are proposed for differential gene expression. They are inter-sample normalization methods that give a scaling factor, which scales sample size for each sample. Here, we treated runs in one accession as samples. RPKM does not just consider sample size, but also read length. Here, we treated one run in one accession as one sample. Therefore, for real data with a length of 76 nucleotides, because they only have one run, only RC, RPKM, and ERPKM were applied. A detailed description of each method is as follows.RC: The raw count for each gene was the sum of gene counts of all runs.UQ: The upper quartile was calculated by applying the upper quartile of 0.75 to the gene counts of all runs.Med: The median was calculated as the median of gene counts of all runs.TMM: The trimmed mean of M-values is a scaling normalization method proposed for differential expression analysis of RNA-Seq data . This noDESeq: DESeq is a differential gene expression analysis method based on a negative binomial distribution model, with variance and mean linked by local regression, and presents an implementation that also gives scale factors . It is wQ: Quantiles has been previously used to normalize single channel or A-value microarray intensities between arrays . The NorRPKM: This approach quantifies gene expression from RNA-Seq data by normalizing for the total transcript length and the number of sequencing reads. RPKM values can easily be calculated using the definition:ERPKM: Since reads have a non-zero length, and the read probabilities depend on an effective length , we calcThus, All calculation of non-abundance estimation normalization methods was carried out using R language. R code is provided in Additional file RSEM and Sailfish are abundance estimation normalization methods. They estimate read count by machine learning methods.Prepare referenceRSEM: RSEM is different from previous normalization methods. It proposes a directed graph model combined with an expectation–maximization algorithm to estimate abundances. RSEM provides a software package for quantifying gene abundances from RNA-Seq data, so we generate a reference index by preparing reference transcript data and calculated RSEM values by inputting RNA-Seq data. The following are the two steps including parameters used in calculating RSEM values:2.Calculate RSEM values./rsem-prepare-reference-no-polyA-no-bowtie reference.fa reference_path./rsem-calculate-expression-sam-p 8 alignmentResults.sam reference_path output_pathPrepare referenceSailfish: Sailfish was introduced to be alignment-free in abundance estimation. It uses the concept of k-mer to index and count RNA-Seq reads. Here, we used the estimated number of k-mers after bias as an estimated count. The following are the two steps including parameters used in computing Sailfish values:2.Calculate Sailfish values./sailfish index−t reference.fa−o reference_path−k 20./sailfish quant−i reference_path−l “T = SE:S = U”−r sequence.fastq−o output_pathBecause abundance estimation normalization methods give count estimations for each alignment, RSEM and Sailfish were evaluated by combining with RC and RPKM.p-values < 0.05, both qRT-PCR values and normalization results were not normally distributed. For the characterization of data, we used Spearman’s rank correlation coefficients to evaluate performance by calculating the similarity between RNA-Seq abundance predictions of each normalization method and the measured qRT-PCR values.The distribution of qRT-PCR values and all normalization results were tested using the Shapiro-Wilk normality test. According to test results with s is calculated as:The Spearman correlation coefficient is widely used because it measures linear dependence between two variables as a non-parameter method . It is ci) is the rank of the ith observed value of X, and R(yi) is the rank of the ith observed value of Y.where The Spearman correlation coefficient will generate a value between +1 and −1, where +1 indicates a total positive correlation, 0 indicates no correlation, and −1 indicates a total negative correlation. Values closest to +1 or −1 indicate the highest correlation and, therefore, the best normalization results. In this study, the Spearman correlation coefficient was executed by R using the cor.test function.