Over the last decade, many analytical methods and tools have been developed for microarray data. The detection of differentially expressed genes (DEGs) among different treatment groups is often a primary purpose of microarray data analysis. In addition, association studies investigating the relationship between genes and a phenotype of interest such as survival time are also popular in microarray data analysis. Phenotype association analysis provides a list of phenotype-associated genes (PAGs). However, it is sometimes necessary to identify genes that are both DEGs and PAGs. We consider the joint identification of DEGs and PAGs in microarray data analyses. The first approach we used was a naïve approach that detects DEGs and PAGs separately and then identifies the genes in an intersection of the list of PAGs and DEGs. The second approach we considered was a hierarchical approach that detects DEGs first and then chooses PAGs from among the DEGs or vice versa. In this study, we propose a new model-based approach for the joint identification of DEGs and PAGs. Unlike the previous two-step approaches, the proposed method identifies genes simultaneously that are DEGs and PAGs. This method uses standard regression models but adopts different null hypothesis from ordinary regression models, which allows us to perform joint identification in one-step. The proposed model-based methods were evaluated using experimental data and simulation studies. The proposed methods were used to analyze a microarray experiment in which the main interest lies in detecting genes that are both DEGs and PAGs, where DEGs are identified between two diet groups and PAGs are associated with four phenotypes reflecting the expression of leptin, adiponectin, insulin-like growth factor 1, and insulin. Model-based approaches provided a larger number of genes, which are both DEGs and PAGs, than other methods. Simulation studies showed that they have more power than other methods. Through analysis of data from experimental microarrays and simulation studies, the proposed model-based approach was shown to provide a more powerful result than the naïve approach and the hierarchical approach. Since our approach is model-based, it is very flexible and can easily handle different types of covariates. The development of new technologies has greatly affected the biological research field. Specifically, the advent of microarray technology provides a crucial turning point in biological research ,2,3,4. MRecently, the improvements in microarray technology have been guiding the development of various platforms. Many studies have tried to integrate several platforms; for example, the MicroArray Quality Control (MAQC) project provided gene expression levels that were measured from seven different platforms. The MAQC study provided a resource representing an important first step toward establishing a framework for the use of microarrays in clinical and regulatory settings . In addiIn general, the primary goal of microarray data analysis is to identify differentially expressed genes (DEGs). Microarray technology allows us to obtain data on the expression of target genes more easily than other technologies. DEGs have become more easily detected by microarray technology than ever before. When applied to experimental data, the causal genes related to diseases can be obtained by discovering DEGs. Over the last decade, numerous statistical methods have been proposed such as t-tests, significance analysis of microarray (SAM) , regressOf these approaches, the t-test is the most popular statistical test for comparing the means between two groups. The t-test is a parametric method that requires a normality assumption. However, microarray data rarely satisfy the normal distribution assumption. Therefore, a permutation test that does not require such assumptions is preferably used to detect DEGs ,14. The HER2 jointly define the subtypes of breast cancer. The triple-negative phenotype  is most commonly used [The application of microarray technology has also led to diverse studies that go beyond identifying DEGs such as a study examining the relationship between phenotype and expression data. Various phenotypes have been used in microarray experiments; for example, the survival time was utilized as a phenotype for analyzing the recurrence of cancer in clinical studies ,17. Sevenly used .Phenotype-associated genes (PAGs) are the genes that are associated with a phenotype of interest. The PAGs can be identified by regression analyses such as linear regression analysis for continuous phenotypes and the Cox regression model for survival time phenotypes . When thIn this article, we focus on the joint identification of DEGs and PAGs in microarray data analyses. Our study was motivated by the need for an analysis of a microarray experiment consisting of high fat diet (HFD) and normal diet (ND) groups. Ten mice were assigned to each group for the microarray experiment. In addition, four phenotypes reflecting the expression levels of leptin, adiponectin, insulin-like growth factor 1 (IGF-1) and insulin were measured in blood samples. Leptin is an adipocyte-secreted hormone with a key role in energy homeostasis . IGF-1 iAlthough many approaches have been proposed for the separate identification of DEGs and PAGs, only a few approaches are available for the joint identification of DEGs and PAGs. The first approach we used for the joint identification of DEGs and PAGs was a naïve approach that detects DEGs and PAGs separately and then identifies the intersecting genes from the lists of PAGs and DEGs. The second approach is a hierarchical approach  that detWe propose a new model-based approach for the simultaneous identification of DEGs and PAGs. Our model-based approach uses a linear regression model. We have use the linear regression model since it is easy to use, flexible in dealing with individual covariates, and easy extendibility . Our method is a one-stage analysis that takes less computing time, makes it easier to control false positive errors, and has greater power than naïve or hierarchical approaches. Through analysis of data from a microarray experiment carried out in mice and from simulation studies, we compare our model-based approach with naïve and hierarchical approaches.All animal experimental procedures were reviewed and approved by the Institutional Animal Care and Use Committee of Sookmyung Women’s University (SMU-IACUC-2011-0401-005).2 inhalation at 13 week of age. At necropsy, blood and tissue samples were collected; serum samples were prepared by centrifugation of whole blood samples at 650 × g for 20 min and stored at -80°C until analysis; colon tissues were rapidly removed, immediately frozen in liquid nitrogen, and stored at -80°C until microarray analysis.Microarray data consisted of data obtained from HFD and ND groups of mice to determine influential genes associated with obesity. Four-week-old male C57BL/6J mice were purchased from SLC Japan . Mice were housed in plastic cages (three to four mice per cage) under a constant temperature 23 ± 2°C) and humidity (50 ± 10%) with a 12-h light/dark cycle. Animals were allowed to acclimatize to the laboratory environment for 1 week before the experiment onset. The composition of experimental diet was based AIN-93G. The fat sources of normal diet  and high-fat diet  were based on corn oil and lard. The reference we have used for such fat percentage definition can be seen in “A high-fat diet impairs neurogenesis: Involvement of lipid peroxidation and brain-derived neurotrophic factor” [ ± 2°C anIllumina MouseRef-8 v1.1 Expression BeadChip was used in our microarray experiment. We observed changes in the gene expression pattern due to HFD-induced obesity. We assigned 10 mice to each ND group and HFD group. Then, three mice from the ND group and six mice from the HFD group were selected via QC for the microarray experiment, and each sample had 45281 probes.Four phenotypes associated with regulating metabolism were extracted using levels of expression in the blood sample including leptin, adiponectin, insulin-like growth factor 1 (IGF-1), and insulin. Serum insulin concentration was measured with an ELISA kit  according to the manufacturer’s instruction. Serum concentrations of IGF-1, leptin  and adiponectin  were also measured with an ELISA kit, according to the manufacturer’s instructions. IGF-1 is similar in molecular structure to insulin and is an important hormone for childhood growth. Adiponectin controls glucose levels as well as fatty acid breakdown, and insulin is one of the most important hormones in the metabolic system of mammals. The expression values are log-transformed. After log-transformation, QQ plots and goodness of fit tests for normal distribution did not provide evidence that the data do not follow normal distribution. We provided Fig A in 0) to common statistics as one of the penalty methods. The variable si is the estimated standard error from gene i, and s0 is calculated as a percentile based on α. Then, the following test statistic is used:First, we detected DEGs by using a two-sample t-test. Secondly, we used significance analysis of microarray (SAM)  for idenIn addition, the SAM method uses a permutation algorithm to control for the false discovery rate (FDR) . Therefoi indicates the expression value for each gene. As mentioned earlier, the phenotypes of interest consist of leptin, adiponectin, IGF-1, and insulin expression. Linear regression analysis is performed for each phenotype. Two linear regression models are applied to identify the linear relationship between genes and phenotypes.i indicates the expression value for each gene. The first model M1 is to identify the effect of expression on the phenotype, whereas the second model M2 is an extension of M1 with an additional Group covariate.Linear regression analysis is utilized to determine PAGs. There are two treatment groups in our microarray data: ND and HFD. Group information is denoted by Group. ExpressionThe significance of the linear relationship between gene and phenotype may be affected by the group effect because some genes may not have marginal effects on the phenotype but may have conditional effects given the group information. M1 is used for detecting the marginal effect, while M2 is used for detecting conditional effects. PAGs may depend on the group effect. For example, the v1rh4 gene is a non-PAG by model M1. However, it is identified as a PAG by model M2 . Model M1 is of the main interest. In model M2, β1 is still of the main interest even though the group effect β2 is added to explain the high fat diet effect between the ND group and the HFD group. The PAGs can be identified by testing the following hypotheses:In model M1, the expression effect βThe naïve approach detects DEGs and PAGs separately, and then identifies the intersection of PAGs and DEGs. It consists of the following two steps:Identifying DEGs set and PAGs set separately for total data.Determining intersection gene sets of DEGs and PAGs.The hierarchical approach  detects t-tests, and SAM.Identifying DEGs by statistical tests such as two-sample Detecting PAGs by linear regression models M1 and M2 for the DEGs selected at Step 1.Both naïve and hierarchical approaches are two-stage analyses that require separate testing of DEGs and PAGs, and as a result, it was not straightforward to control the false positive errors because two tests are not independent. Thus, we propose a new model-based approach for joint identification of DEGs and PAGs. Our model-based approach uses a linear regression model, as explained in the following section.We propose a new test based on the likelihood ratio to determine DEGs and PAGs simultaneously. Consider the following model (6)For finding the DEGs and PAGs, consider the following hypotheses.Under the null hypothesis, the likelihood can be maximized as follows.Thus, the likelihood ratio test statistic Λ is derived as follows:γ1 when γ2 = 0, γ2 when γ1 = 0 and  distribution. Since this likelihood ratio test statistic is smaller than each likelihood ratio test statistic for testing H01: γ1 = 0 or H02: γ2 = 0, the p-value of LRT statistics is larger than that of both likelihood ratio test statistics for testing each beta. In this reason, even we test DEG and PAG at the same time with Chi-square test under 1 degree of freedom, type I error will be conserved. However, we cannot say that our test is most powerful among the tests. Thus, we performed simulation for power comparison between our proposed method and naïve approach.Un the null hypothesis, if For a more systematic comparison, we performed an extensive simulation study with following settings. First, we generated phenotype data from the normal distribution with the same mean and variance as the actual phenotype data. Since adiponectin most closely followed the normal distribution according to the normality test , the meaTfrc, Sprr1a, Cyp4f16, and 9030605I04Rik. Although SAM provided only a few genes that are all up regulated genes, the SAM result is expected to be very reliable because it did not contain any false positives (FDR = 0)[Two-sample t-tests, and the SAM method, were utilized to identify DEGs. The results of t-tests were not significant after controlling for multiple testing at 5% level using Bonferroni correction or FDR. The SAM method provided significant results at the 0% median FDR = 0: (FDR = 0). In TablModels M1 and M2 were employed to detect PAGs for these phenotypes. We applied three joint identification methods to the microarray data from the mice. We first applied the naïve approach by comparing the list of DEGs and the list of PAGs. We then applied the hierarchical approach by detecting DEGs first and then PAGs. Finally, we applied the model based approach using the likelihood ratio test from model M3. For the purpose of a fair comparison, we fixed significance and FDR at the same level. 1 as follows: 0, 0.08, 0.12, 0.16, 0.2, 0.24, 0.28, and let γ2 vary from 0 to 8 by 2. These parameter values represent the effect sizes derived from the real data. Since our type I error needs to be evaluated not at a point but two axes generated by γ1 and γ2, we checked both  and  cases. 1 and γ2 equal 0, the type I error is smaller than 0.05 for all methods. However, as either γ1 or γ2 increases, the type I error becomes closer to 0.05. Therefore, all methods tend to preserve the type I errors.To check the type I error and power, we set the values of γ1 and varied γ2, M1 and M2 showed almost the same power in every case. On the contrary, the model-based method showed better power than other methods. In Figs As the experimental designs using microarrays becomes more complex, a more complicated analysis method needs to be developed. In early microarray studies, either DEGs or PAGs need to be identified. However, recent microarray designs require a more complicated method to detect the genes that are simultaneously DEGs and PAGs.FABP9  is a Protein Coding gene which is associated with sporadic breast cancer. GO annotations related to this gene include transporter activity and lipid binding [SGCE  is a Protein Coding gene which is associated with myoclonic, dystonia-11, or dystonia. It is known that SGCE gene works together with SGCA [SGCA is known that is associated with type 2d [SGCA cannot reach q-value below 0.05, that of p-value for model-based approach is 0.0011. Therefore, these genes would have significant relationships for finding differences of effects between high-fat diet and low-fat diet. LGALS4  is a Protein Coding gene. Diseases associated with LGALS4 include colon adenocarcinoma and measles. Among its related pathways are Cell adhesion_Cell-matrix glycoconjugates. GO annotations related to this gene include carbohydrate binding [Our proposed model provided some interesting lists of genes . Gene ex binding . SGCE . QQ-plots for some genes which are significant in model-based approach . Gene expression profiles for significant genes with leptin detected by model-based approach. Blue dots are ND samples and Red dots are HFD samples. X-axis shows log-normalized gene expression for each gene and Y-axis shows expression of leptin . Gene expression profiles for significant genes with adiponectin detected by model-based approach. Blue dots show ND samples and Red dots show HFD samples. X-axis shows log-normalized gene expression for each gene and Y-axis shows expression of adiponectin . Gene expression profiles for significant genes with insulin detected by model-based approach. Blue dots are ND samples and Red dots are HFD samples. X-axis shows log-normalized gene expression for each gene and Y-axis shows expression of insulin .P-values for most genes are larger than 0.05 which means log-transformed gene expressions follow normal distributions ((PPTX)Click here for additional data file.S2 File(CSV)Click here for additional data file.S3 File(CSV)Click here for additional data file.


If not corrected by suitable numerical algorithms, batch effects may seriously affect the analysis of these datasets. The novel array platform independent software tool BEclear enables researchers to identify those portions of the data that deviate statistically significant from the remaining data and to replace these portions by typical values reconstructed from neighboring data entries based on latent factor models. In contrast to other comparable methods that often use some sort of global normalization of the data, BEclear avoids changing the apparently unaffected parts of the data. We tested the performance of this approach on DNA methylation data for various tumor data sets taken from The Cancer Genome Atlas and compared the results to those obtained with the existing algorithms ComBat, Surrogate Variable Analysis, RUVm and Functional normalization. BEclear constantly performed at par with or better than these methods. BEclear is available as an R package at the Bioconductor project  The well-known batch effect describes a non-biological experimental variation that may result, in the case of genomic datasets, either from manufacturing problems of the diagnostic chips, from imprecisely conducted experiments, or simply from mislabeling of samples. Often it is very hard retrospectively to identify the exact cause of batch effects. In any case, batch effects often severely affect the large-scale automatic processing of genomic datasets . It is tThe most straightforward way to avoid issues resulting from batch effects is to leave out from the analysis all seemingly affected batches of data or genes once they have been detected . Yet, thIn order to allow researchers to overcome problems resulting from batch effects, several algorithms for detecting  and dealS. aureus samples ij approximately equals to Dij for every unaffected entry A. The Gradient descent optimization algorithm was used to minimize the global loss i.e. the difference between [LR] and D. Once it converges, entries that were not batch effected were preserved in the completed matrix Dcomp = [LR] from the original data matrix D, so that the algorithm replaced only the matrix entries for BE-genes in the affected batches. In case if some of the predicted entries lie below 0 or above 1, they were assigned values of 0 and 1, respectively.Based on the BE-scores of all batches, we then identifiedusingtheDixon test from the " package  which ba package , 13. Themdif, p-value and different p-value adjustment methods. For mdif values larger than 0.1, only few genes (from 103 to 1465 BE-genes) were detected as BE-genes. After removing them, the batch effect was still visually observed. On the other hand, when mdif was set to values in the order of 0.01, more than 82% of all genes were detected as BE-genes. Thus, mdif in the order of 0.1 is a reasonable value. Next, we found that different thresholds for the p-value did not have noticeable effects on the results. With a p-value = 0.05, BEclear identified 5990 BE-genes and 5032 BE-genes for p-value = 0.001. Furthermore, three different p-value adjustment methods  yielded similar numbers of BE-genes of around 5500 genes.During the detection of single batch effected genes in adjacent normal BRCA data we tested different values for mdif values for BE-genes We note that due to the usage of the Kolmogorov—Smirnov test, BEclear might not detect batch effect in batches containing fewer than 5 samples unless the batch effect is very strong as in the case of Kidney renal clear cell carcinoma KIRC, where the KS-test was compensated by large The matrix completion method was assessed in terms of overall accuracy and prediction time when applied to DNA methylation data. For testing purposes we used again the BRCA adjacent normal dataset. As a measure of accuracy, we computed the average absolute deviation between known and predicted entries  of the matrix. Due to the fact that BEclear found 5.8% entries to be affected, testing was performed on 6% of additional randomly selected entries. Generally, the time needed to perform LFM prediction grows exponentially with the size of the data. For the BRCA dataset studied here (11213 genes in 96 samples), this task could be infeasible without separating the initial matrix into blocks of data and running LFM independently for every block. This approach also enables parallel execution on multi-core processors, what leads to significant savings in computation time.We also analyzed how the size of the block of data to which LFM was applied affected the prediction accuracy. This parameter was varied from 10 to 250 in increments of 5. In all cases LFM yielded a similar accuracy that differed at most by 0.02 .Note thaWe start by illustrating the performance of BEclear using DNA methylation data for breast cancer (BRCA) samples downloaded from the TCGA portal . For simmdif categories for the difference of median values  in every batch belonging to the respective n values . For exan values  is statin values  what sucmdif of those genes is quite large . ComBat  is a parWe separately corrected adjacent normal  and tumoIn the course of this comparison we noticed the following differences between ComBat and BEclear. As mentioned before, ComBat cannot handle batches that only contain a single sample and assumes as default that the data follow a normal distribution. As is typical for normalization methods, ComBat adjusts all entries in the dataset even though not all of them are affected by batch effect. Especially in the tumor data, which inherently contains more variation, we speculate that the strict adjustment of the data by Combat might diminish biological variation. In contrast, BEclear leaves all unaffected parts of the data as is and only replaces the entries of batch effected genes in certain batches by the predicted entry based on the gene and samples preference. One artifact is that ComBat produces many values above 1 and below 0, whereas β -values must be inside the interval  by definition . For tumsva and fsva R functions [Then, we compared BEclear against Surrogate Variable Analysis (SVA). When applying unctions  to levelFinally, we compared BEclear to the recently presented method Functional normalization that was designed specifically for the 450k methylation array . At firsFor a systematic comparison of batch effect correction methods, we generated synthetic data sets with “known” batch effects as described in . First, When considering all 8000 probes including the 4000 shifted probes,BEclear gave a much smaller total deviation between the methylation values of BEclear-adjusted probes and the original gold standard value. We believe that this is the case because BEclear adjusts only batch affected entries and keeps the other entries at their original values. When focusing on the values of the 4000 probes that were synthetically shifted (batch affected), then the performance of BEclear relative to the other methods depended on the magnitude of the introduced batch effect. For small perturbations (of 1 SD or 2 SD) that are typical magnitudes in real situations, BEclear performed comparably well as Combat and better than SVA and FunNorm . For perFor benchmarking against the RUVm method we considered the identities of differentially methylated genes in breast tumor samples vs. normal samples. As gold standard reference, we used the list of differentially methylated probes identified in the unaffected data using the limma package . Then, wCo-methylation analysis of gene pairs was performed in the same manner as in our previous work  on BRCA NRG4, TUBB, LPL, BRD2, MYB, RAP2C, SIRT7, MAZ, HRAS, TXN, PPM1D, TP53I3, PARK7, TP63 [Finally, differential methylation analysis between tumor and normal samples was carried out by applying KS-test –20 and SK7, TP63 –45. ImpoFinally, we tested our BEclear method on raw adjacent normal data from the HumanMethylation450 chip without initial preprocessing. In this case, it was searching for batch affected CpGs instead of BE-genes and included both types of probes—from the promoter region and other regions of the gene. It was able to detect and adjust batch effect in the same batch 136 with BE-score = 1.47 . Thus we recommend to use BEclear after applying dedicated normalization methods in order to assess batch effect; and in the case of its presence—to adjust it.In this paragraph, we summarize the results from comparing BEclear to the alternative tools SVA, Combat, Functional normalization and RUVm.(1) The first and main advantage of BEclear over all other methods is that BEclear preserves the original measurements to the highest possible extent. Only entries for the batch affected genes in the distorted samples are adjusted by BEclear; all other data points of non-affected genes and also of the batch affected genes in the non-distorted samples are kept at their original values. In our view, this strategy is beneficial whenever the researcher cares about the absolute values of the data entries, not only about relative trends between them. In contrast, all other methods use some sort of normalization and modify the values of essentially all data points. (2) Only ComBat and BEclear can handle data where all samples belong to one class. This helps, for example, in adjusting tumor data that is not accompanied by corresponding adjacent normal samples. Furthermore, BEclear can handle single sample batches whereas this option is not available for ComBat. (3)In contrast to Functional normalization and RUVm, the three methods BEclear, SVA and ComBat are platform independent and can be applied to different levels of data ranging from raw signals to mapped and aggregated β-values. Yet, we showed that SVA and ComBatdid not reach the same level of performance in differential methylation analysis as BEclear and RUVm. (4) RUVm can only correct batch effects in the context of differential methylation analysis. This is not the case for BEclear, SVA and ComBat. (5) BEclear outperformed all other methods on the simulated data in terms of accuracy for batch effects ofless than two standard deviations in magnitude .BEclear applies rigorous statistics to detect whether or not individual batches in the input data are affected by batch effects. The appropriate measure to decide this is the BEscore value computed by the program. Depending on the purpose of the experiment, BEclear can be either used alone or combined with other post processing methods.We have compared BEclear against other well established methods for batch effect adjustment. Depending on the metrics used and the strength of batch effect, BEclear either outperformed other methods or performed comparably well. As the other methods all use some form of normalization, they affect all data entries. Such normalization approaches may be most appropriate to correct for technical variations (or errors) where all probes on an array are affected in a more or less similar way. In contrast, BEclear adjusts only those portions of the data that were identified to differ significantly from the other batches. This strategy may be useful, for example, to process data from diagnostic chips showing some inhomogeneity or ambiguity in certain areas/entries. Thus, we suggest BEclear as a novel method to control batch effects in the data remaining after application of standard normalization techniques.S1 Figmethylumi package when processing level 1 data into level 3 data did not remove the batch effect in the batch 136.This stands for DNA methylation raw signal intensities of probes for each participant's sample. Batch effect is clearly present in batch 136 since the distribution of β-values in these samples significantly deviates from the other samples. This illustrates that the background correction technique applied by the (PNG)Click here for additional data file.S2 FigA. Adjacent normal samples per batch level (13 batches). B. Tumor samples, per batch level (32 batches). C. Subset of tumor samples for batch 136 and surrounding batches, per sample level. All these plots illustrate clearly that batch 136 is affected by batch effect in both tumor and adjacent normal samples.(TIFF)Click here for additional data file.S3 FigA. The heatmap demonstrates that all but two samples from batch 136 form a cluster that splits off from the other samples at the top of the hierarchy. B. Plotting the first two Principle Components and projecting samples on them clearly distinguishes samples from batch 136 from the rest. C. The density plot of every batch shows that the β-values in batch 136 have a different distribution than in the other batches.(TIFF)Click here for additional data file.S4 FigFor this gene, we identified the largest difference of 0.428 between the median of batch 136 and the median of the other batches.(PNG)Click here for additional data file.S5 FigHere, we investigated the impact of the block size on the overall accuracy of LFM matrix completion. Four parameters were computed: mean, median, minimal and maximum difference between actual and predicted β- value entries. The size of the block of the data, to which LFM was applied, was varied from 10 to 250. Larger block sizes increase the frequency of large β-value differences (green curve). Overall, LFM shows good prediction accuracy in a wide range of data block sizes whereby the median of the difference remains in the range of 0.01 and the mean stays around 0.02.(PNG)Click here for additional data file.S6 FigA. Per batch boxplot of corrected adjacent normal data. B. Per batch boxplot of corrected tumor data. C. Density plot and D. PCA plot of corrected adjacent normal data. In the per batch boxplot of corrected normal data  batch 136 does not stand out explicitly anymore. This is also confirmed by the per sample boxplot ((TIFF)Click here for additional data file.S7 FigA. Per sample boxplot. Batch 32, which contains only two samples, has a batch effect score equal to 0.185 signaling that its data should be corrected. B. Number of genes belonging to different categories of median differences (mdif) between genes in the current batch and the same gene in all other batches (as described in section 2.3.3. “Batch effect scoring” in the main text).(TIFF)Click here for additional data file.S8 FigThe previously observed batch effect in batch 136 was corrected both in A. adjacent normal and B. tumor data.(TIFF)Click here for additional data file.S9 FigShown are the number of batch effected genes in single batches from A. BRCA adjacent normal and B. BRCA tumor data.(TIFF)Click here for additional data file.S10 FigNote that BEclear sets these values eventually to 0 and 1. A. Boxplot of entries with values larger than one in the breast cancer tumor data from TCGA adjusted either by ComBat or by BEclear. B. The same as in A showing the number of values below 0. C. Boxplot of values below 0 in adjacent normal data after correction by ComBat.(TIFF)Click here for additional data file.S11 FigA. Per sample boxplot B. Density plot. Functional normalization was able to adjust the batch effect equally well as BEclear since S11A Fig looks very similar to what was obtained after BEclear correction .(TIFF)Click here for additional data file.S12 Figbefore any batch effect adjustment B. after functional normalization C. after batch effect correction with BEclear. The most affected batch is marked in red.(TIFF)Click here for additional data file.S13 FigAs a measure of performance we used the total absolute difference of the β-values between gold standard data and corrected entries for 1000 batch affected probes (out of 8000 probes).(PNG)Click here for additional data file.S14 FigHere, only 1000 probes were perturbed instead of 4000). For all methods the list of differentially methylated genes (DMG) was obtained and then compared to the list of DMG for gold standard data. Here batch affect was introduced to 1000 probes (out of 8000). The x-axis indicates for the magnitude of the introduced batch effect.(PNG)Click here for additional data file.S1 TableCancer types and batches which were identified to have a batch effect are marked in bold. This table contains the description of cancer types, batch identifiers obtained from the TCGA portal and the batch effect score (see (DOCX)Click here for additional data file.S2 TableThe median difference counts the number of genes for which the median DNA methylation in this batch differs from its median in all other batches by a value falling into the respective intervals specified at the top. The BEscore is computed according to (DOCX)Click here for additional data file.S3 Table(DOCX)Click here for additional data file.S4 TableListed are the number of highly co-methylated pairs of genes (Pearson correlation higher than 0.75 or lower than -0.75) for three different types of data after batch effect adjustment with BEclear and before.(DOCX)Click here for additional data file.


Background. A common research goal in transcriptome projects is to find genes that are differentially expressed in different phenotype classes. Biologists might wish to validate such gene candidates experimentally, or use them for downstream systems biology analysis. Producing a coherent differential gene expression analysis from RNA-seq count data requires an understanding of how numerous sources of variation such as the replicate size, the hypothesized biological effect size, and the specific method for making differential expression calls interact. We believe an explicit demonstration of such interactions in real RNA-seq data sets is of practical interest to biologists.Results. Using two large public RNA-seq data sets—one representing strong, and another mild, biological effect size—we simulated different replicate size scenarios, and tested the performance of several commonly-used methods for calling differentially expressed genes in each of them. We found that, when biological effect size was mild, RNA-seq experiments should focus on experimental validation of differentially expressed gene candidates. Importantly, at least triplicates must be used, and the differentially expressed genes should be called using methods with high positive predictive value (PPV), such as NOISeq or GFOLD. In contrast, when biological effect size was strong, differentially expressed genes mined from unreplicated experiments using NOISeq, ASC and GFOLD had between 30 to 50% mean PPV, an increase of more than 30-fold compared to the cases of mild biological effect size. Among methods with good PPV performance, having triplicates or more substantially improved mean PPV to over 90% for GFOLD, 60% for DESeq2, 50% for NOISeq, and 30% for edgeR. At a replicate size of six, we found DESeq2 and edgeR to be reasonable methods for calling differentially expressed genes at systems level analysis, as their PPV and sensitivity trade-off were superior to the other methods’.Conclusion. When biological effect size is weak, systems level investigation is not possible using RNAseq data, and no meaningful result can be obtained in unreplicated experiments. Nonetheless, NOISeq or GFOLD may yield limited numbers of gene candidates with good validation potential, when triplicates or more are available. When biological effect size is strong, NOISeq and GFOLD are effective tools for detecting differentially expressed genes in unreplicated RNA-seq experiments for qPCR validation. When triplicates or more are available, GFOLD is a sharp tool for identifying high confidence differentially expressed genes for targeted qPCR validation; for downstream systems level analysis, combined results from DESeq2 and edgeR are useful. For a long time, biologists have been using microarrays for gene expression studies, and over the years, the collective experience of the community has congealed into a set of best practices for mining microarray data . Hence, t-test  would bet-test , produciparisons . Subsequd change  allows tde novo transcriptome assembly tools , such as edgeR, DESeq and its new version, DESeq2. These are parametric methods that explicitly model the distribution of count data using the negative binomial distribution. Initially designed for standard experiments with biological replicates, these methods were later modified to accommodate analysis of unreplicated experiments as well, but their performance relative to ASC, GFOLD and NOISeq remains unclear. We did not include two methods with high citations per year: Cuffdiff2 and DEGSeq, based on conclusions from recent method comparative analyses. For example, Cuffdiff2 was found to have very low precision when replicate size increased in the analysis of two large RNA-seq data sets from mouse and human . FurtherZ-test for equality of two proportions was included to set upper bounds in the tested performance metrics that are attainable by naive application of a common textbook statistical method. Let ijN be the pooled normalized read counts of the ith gene in the jth phenotype class , jth phenotype class, and ith gene in all phenotype classes. Specifically, the Z-test statistic for the ith gene is given by N is the total number of normalized counts. The inclusion of the popular non-parametric method NOISeq provides a contrast between performance of parametric and non-parametric methods. We included voom, which connects log-transformed read count data to the mature limma analysis pipeline  that hasZ-test, we used a joint filtering criteria (ϕ) and p-value (p) to call DEG. Let y = − log10p and x = log2ϕ. Thus, each gene is associated with a paired score  after differential expression analysis. Following  parameter as 0.4 for the Cheung data set (see details in ‘Benchmarking’), and 0.1 for the Bottomly data set, following recommendations in q = 0.9 for unreplicated experiments, and q = 0.95 for experiments with biological replicates. For ASC, genes that had log2 fold change above 1 or −1, and posterior probability 99% or more were declared to be differentially expressed. For GFOLD, we used the default significant cut-off of 0.01. A gene with GFOLD value of 1 or larger was considered differentially up-regulated, and differentially down-regulated if GFOLD value was −1 or smaller. Except GFOLD, which is written in the C/C+ + language and requires the Linux platform, the other methods were executed in R version 3.1.3  and DBA/2J (n = 11). Both mice strains are known to show large, strain-specific variation in neurological response when subjected to opiate drug treatment , and heterozygotes for the Brd1 gene deletion (n = 8). A total of 115 genes were selected for qPCR validation  or too many (thousands) candidates. Among methods that satisfied this criterion, the one that had relatively higher positive predictive value  would be preferred. Let TPN be the number of true positives, and FPN the number of false positives. Then the number of DEG that lack validation result is U = gN − FPN − TPN. If U is not too large or too small, then the expected number of true positives can be estimated as The validity of voom as a tool for constructing reasonable data set  consistsin silico reference DEG set called using voom for the two test data sets should be independently validated using qPCR, but evidence at such level may not always be available. Where microarray data are available for the same study, a DEG candidate can be considered reliable if it is called in both RNA-seq and microarray analyses, since fold change of DEG from the latter has been found to correlate strongly with fold change from qPCR ; jn are the replicate sizes. Thus, T2 is essentially the square of the t-statistic, which measures the magnitude of squared deviation between mean counts in two different phenotype classes relative to the latters’ variances. By definition, the median values of T2 should be large in a data set that shows strong biological effect size, and vice versa. For the Bottomly data set (strong effect size), median T2 was 27.6; for the Cheung data set (mild effect size), it was 4.6. Both data sets had approximately equal spread of T2 values around the median, the interquartile range being 38.3 and 34.5 for the Bottomly and Cheung data sets, respectively.Once the DEG set had been constructed for the Bottomly and Cheung sets, it became possible to operationally define what we meant by mild or strong biological effect size. For the n = 3, 6, 10), and the Bottomly data set  by random sampling without replacement within each phenotype class.To simulate unreplicated experiments in both data sets, we used all possible sample pairs (11 × 10 = 110) for the Bottomly data set, and 300 random sample pairs for the Cheung data set. Except ASC, which only handles unreplicated experiments, we further examined the behavior of other DEG call methods in cases of low to modest number of replicates. We constructed 100 instances of experiments for each replicate size per phenotype class in the Cheung data set . For each DEG call method, we computed sensitivity as the proportion of reference DEG that were called. PPV was computed as the proportion of DEG called that were members of the reference DEG set. The mean and standard deviation (SD) of these metrics were then reported. Methods that show good PPV are particularly interesting in the context of unreplicated experiments, since DEG obtained from them offer the best potential of being validated. For systems level analysis, DEG should preferably be called using methods with good balance of sensitivity and PPV.The DEG set size and expected PPV of each method in the analysis of the Rajkumar data set are given in We note with interest from n = 3 , and an extremely low sensitivity (0.2 ± 0.6%) as a result of making too few calls. Interestingly, GFOLD attained very high mean PPV at n = 3  for these two replicate sizes, which was caused by its small DEG set size . In unreplicated experiments , mean PPset size .n = 3  compared to GFOLD or NOISeq. Consequently, it is expected that the formers’ sensitivity would increase at the expense of their PPV.The current results suggest that unreplicated RNA-seq experiments, which are still very common among underfunded labs working with non-model organisms, may be a cost-effective way to generate candidate DEG with reasonable likelihood of being validated, provided that the underlying biological effect size is strong. Thus, for unreplicated RNA-seq experiments with phenotype classes such as those associated with pathogenic challenge and physico-chemical stress, we expect DEG called using NOISeq or GFOLD to be good candidates for validation. ASC may also be useful, though it should be noted that it could fail to run for particular combinations of sample pairs, as we found out in the present study. For validation work, GFOLD and NOISeq should be even more efficient once triplicates are available, but further replicate size increase produced only marginal mean PPV gain in the Bottomly data set, suggesting that using more than triplicates is not a cost-effective approach when validation of DEG candidates is the main research goal. When biological effect size is strong, we suggest DESeq2 or edgeR as promising methods to mine DEG for systems biology work, on account of their good PPV and sensitivity balance. However, users should be aware of possibility that shrinkage estimation of dispersion and fold change procedure in DESeq2 may over-correct the initial estimates of these parameters, leading to a DEG set size that is too small, as discovered in the analysis of the Rajkumar data set .n = 10 became too small (below 20).Research programs focusing on the investigation of weak or modest biological effect sizes must have replicates, use NOISeq or GFOLD for DEG calling, and then to restrict the research goal to validation of the DEG candidates. Pursuing a systems biology  direction in such programs is not feasible, since in the Bottomly data set, the DEG set size of both GFOLD and NOISeq at Transcriptome coverage can be another important source of variation for the observed RNA-seq gene counts . AssuminIn the present study, we justified the use of voom for setting the reference DEG set on the basis of its performance in the Rajkumar data set, which has 115 qPCR-validated genes. Ideally, analyses of additional data sets of this nature would help us better understand the variability of method performance. Unfortunately, RNA-seq data sets that are coupled with extensive qPCR validation results remain uncommon.The cost of not constructing the reference gene set using a simulation approach was the lost of one degree of freedom in the number of DEG call methods that could be evaluated, since we had to select one of the methods to determine the reference DEG set. Because of this, it may be possible that voom was actually the ideal method for making DEG calls when sufficient replicates are available. Therefore, in practical situations where systems level analysis is desired, one may wish to consider taking the union of DEG set called using voom with that from DESeq2 or edgeR. If the size of the union set is too large, one may consider taking the intersection set instead to obtain a smaller, but higher confidence DEG set .Many biologists have difficulty publishing results of RNA-seq experiments with no or few biological replicates. Despite including qPCR validation results, these studies are often dismissed by reviewers simply on grounds of ‘not having enough sample size.’ This stand is unnecessarily dogmatic, and does not take into account that some particular combinations in the trinity of replicate size-effect size-call method can potentially yield biologically meaningful results, as shown in the present study.It is gradually being appreciated that RNA-seq analysis is a complex analysis that needs to address the numerous sources of variation from library preparation to bioinformatic processing  to yieldAs more high-quality RNA-seq experimental data continue to accrue in public databases, a better understanding of the anticipated behavior of various DEG calling methods under different biological and replicate size scenarios should gradually emerge from systematic comparison studies such as the current one. A complete dummy’s guide to RNA-seq differential gene expression analysis may not be too far ahead in the future.In RNA-seq experiments, biological effect size is an important determinant of whether a research program at the individual gene or systems level would yield the most biological insight. When it is expected to be mild, RNA-seq experiments should primarily aim at mining DEG for validation purpose, using at least triplicates and either NOISeq or GFOLD for DEG calling. Moreover, systems level analysis remains difficult as none of the methods considered presently showed satisfactory sensitivity and positive predictive value performance. When strong biological effect size is expected, analysis of unreplicated experiments using GFOLD or NOISeq can yield DEG candidates with optimistic validation prospects. The use of triplicates or more not only improves the statistical power of DEG call methods , but als10.7717/peerj.1360/supp-1Table S1Click here for additional data file.10.7717/peerj.1360/supp-2Table S2Click here for additional data file.10.7717/peerj.1360/supp-3Table S3Click here for additional data file.10.7717/peerj.1360/supp-4Figure S1DEG sets called using SAMSeq vary according seed number and number of permutations.Click here for additional data file.10.7717/peerj.1360/supp-5Figure S2Bland-Altman plots for the (A) Bottomly data set and (B) Cheung data set. Differentially expressed genes are indicated in blue. Abbreviations: A for the average of the standard deviation of gene count for the two phenotype classes; M for their difference. The solid line marks M=0, with the dashed lines indicating upper and lower limits that are three standard deviations away.Click here for additional data file.10.7717/peerj.1360/supp-6Figure S3n = 10. The diameter of a circle indicates the size of the DEG set. Color legend: blue(Z-test), pink(DESeq), red(edgeR), brown(DESeq2), purple(GFOLD), green(NOISeq).Scatter plot of PPV against sensitivity for the Cheung data set with Click here for additional data file.10.7717/peerj.1360/supp-7Figure S4n = 10 in the Cheung data set. Color legend: blue(Z-test), pink(DESeq), red(edgeR), brown(DESeq2), purple(GFOLD), green(NOISeq).Box plots of distribution of DEG set size  by method for Click here for additional data file.


Clostridium thermocellum using a machine-learning approach. Our method accurately predicted the genomic boundaries of individual TUs based on two sets of parameters measuring the RNA-seq expression patterns across the genome: expression-level continuity and variance. A total of 2590 distinct TUs are predicted based on the four RNA-seq datasets. Among the predicted TUs, 44% have multiple genes. We assessed our prediction method on an independent set of RNA-seq data with longer reads. The evaluation confirmed the high quality of the predicted TUs. Functional enrichment analyses on a selected subset of the predicted TUs revealed interesting biology. To demonstrate the generality of the prediction method, we have also applied the method to RNA-seq data collected on Escherichia coli and achieved high prediction accuracies. The TU prediction program named SeqTU is publicly available at https://code.google.com/p/seqtu/. We expect that the predicted TUs can serve as the baseline information for studying transcriptional and post-transcriptional regulation in C. thermocellum and other bacteria.Identification of transcription units (TUs) encoded in a bacterial genome is essential to elucidation of transcriptional regulation of the organism. To gain a detailed understanding of the dynamically composed TU structures, we have used four strand-specific RNA-seq (ssRNA-seq) datasets collected under two experimental conditions to derive the genomic TU organization of  Derivation of all transcription units (TUs) expressed under designed conditions is essential to the elucidation of transcription regulation in bacteria as they are the basic functional units . A TU coEscherichia coli have been derived based on tiling array and RNA-seq data  while the control fermentations were done without ethanol treatment. To derive TUs encoded in C. thermocellum ATCC27405, Illumina ssRNA-seq libraries were prepared from the total RNA extracted from one 60-min untreated control fermentation sample and one 60-min ethanol-shock sample obtained from the above study , except that the poly-A selection step was omitted. Briefly, 300 ng of total RNA was fragmented for the directional RNA-seq libraries following by purification and addition of strand-specific adapters. Samples were then reverse-transcribed, amplified and DNA fragments were enriched by a final clean-up step according to the manufacturer instruction (Illumina). Libraries were normalized with duplex specific nuclease  following the instruction in the DSN Normalization Application Note (Illumina). Final libraries were checked for quality control using an Agilent Bioanalyzer  and quantified with a Qubit . Libraries were then diluted and sequencing was done on version-1.5 single-read flow cell with TruSeq chemistry on an Illumina HiSeq 2000 instrument  Sequence Read Archive (SRA) with accession number SRP002548.In addition to the ssRNA-seq data outlined above, non-strand-specific RNA-seq datasets were also generated on the above four samples at multiple time points using the 454 GS FLX instrument , which produced reads with longer lengths, namely 225 bps for the average length versus 50 bps by Illumina sequencers. The 454 cDNA libraries were generated from the total RNA following the manufacturer's instructions, except that the small fragment removal step was not applied and the libraries were then sequenced using the Titanium chemistry . The 454http://www.bioinformatics.babraham.ac.uk/projects/fastqc/) was used to assess the read quality of the four ssRNA-seq datasets. All four datasets passed the check on both per base quality and per sequence quality . The reads were then mapped to the C. thermocellum ATCC 27405 genome (NC_009012.1) using Burrows-Wheeler Aligner (BWA)  denotes the read abundance at location i in the genome. Note that the estimated read depth does not consider intergenic regions.FastQC (er (BWA)  with theE. coli RNA-seq datasets, generated with paired-end and strand-specific Illumina reads, were retrieved from the NCBI SRA database, with SRA accession numbers: SRX315217, SRX315218 and SRX315219, and used to test the generality of our TU prediction method. The three RNA-seq data were collected on triplicates of wild-type E. coli K12 MG1655 that grew anaerobically in glucose minimal media  the gap percentage out of the intergenic region between the two genes is >50%; and (ii) the ratio between the expression levels of the two genes is >10-fold. For the positive training data, we used 454 reads, which have longer reads than the ones by Illumina sequencers, to annotate consecutive gene pairs to be co-transcribed in the same TU based on the following criteria: (i) the intergenic region between a gene pair is fully covered by at least one 454 read; and (ii) the ratio between the expression levels of the two genes is less than and equal to two-fold. Although 454 reads provided some positive training data, they had a limitation in covering TUs whose intergenic regions are >225 bps. To have our positive training data cover TUs with longer intergenic regions, we added additional positive training data, as described in the following section.We constructed a set of consecutive paired coding regions whose expression patterns along with those of the in-between regions resemble the expression patterns of true TUs, to provide additional training data. Each such paired coding region, along with the in-between intergenic region, is termed a constructed TU (cTU). We assume here that all defective RNA molecules due to the early release of the RNA polymerases from the DNA or truncation will be rapidly degraded by ribonucleases  and hencC. thermocellum, whose coding regions each naturally fall into three sections, and resemble those of true TUs as defined in the previous section. To accomplish this, we plotted the length distribution of the annotated intergenic regions of C. thermocellum. Then, for each gene, we probabilistically selected a length for each made-up intergenic region according to this length distribution. Next, we assigned the made-up intergenic region within the gene spanning a region with the lowest GC content, knowing that true intergenic regions tend to have lower GC ratios (0.33) than protein-coding regions (GC ratio = 0.4) in C. thermocellum genome. The process of creating made-up intergenic regions is given as follows.To construct cTUs, we have used genes of C. thermocellum genome (NC_009012.1) as G, which is 3 268 038 bps long and contains 1683 and 1680 genes on the forward and reverse strands, represented as G+ and G−, respectively, where Denote the whole Step 1: Define D(IR+) as the density function of the D(GD+) as the density function of the G+ without genes on the opposite strand in between.ne pairs  in G+ wiStep 2: For each gene Step 2.1: select probabilistically ip and iq from D(IR+) and D(GD+), respectively, according to their density distributions. Set cIR in Step 2.2: determine the start position of the cIR in m denotes the median position of a, b] of We then exclude those predicted cTU that violate the continuity and variance requirements of a TU, which are continuity and variance. For each predicted cTU, we remove it from further consideration if (i) the gap percentage of the intergenic region between the two genes is >50%; (ii) the ratio between the expression levels of the two genes is >10-folds; and (iii) plus the length of each predicted intergenic region being at least 225 bp long.et al.  fold change in expression levels between the consecutive genes and the intergenic region in between and (ii) the variance of the expression levels across the entire candidate cTU region. The cutoffs for these features are determined to give the best cross-validation results.For the ‘continuity’ feature, we used the following statistics to assess the expression gap in an intergenic region in a candidate cTU: a count of nucleotides with RAPSN = 0 in the region and the percentage out of the length of the region. For the We examined the quality of ssRNA-seq data in terms of strand-specificity. The total expression levels on the incorrect (unexpected) strand were 0.55, 0.44, 0.44 and 0.44% of the total expression levels on the expected strand in the four ssRNA-seq datasets, respectively, where an unexpected strand was defined as an expressed DNA that appears on the opposite strand of an annotated gene.Using the definition of whole-genome RAPSN , we checked the read coverage by the ssRNA-seq data. Over 90% of all coding sequences (protein coding regions) were covered by at least one read. The sequence coverage of the intergenic regions varies from 22.06 to 30.72% across the four ssRNA-seq datasets. The estimated depths are listed in Table We first trained TU predictors using only cTU as the positive training data and non-TU gene pairs as the negative training data. Two groups of features were used in our training of the TU predictors: (i) expression-level continuity and (ii) expression-level variance. A support vector machine (SVM) toolkit, named libSVM , was useWe used non-TU gene pairs as the negative training data and verified TUs based on 454 RNA-seq data plus the predicted cTUs as the positive training data for each ssRNA-seq dataset. The same features and training processes as the Illumina-only model were used to train this model. Overall, the four trained TU predictors achieved the accuracy levels at 0.94, 0.95, 0.95 and 0.95 on the four training sets, respectively. The ROC curves for the predictions are presented in Supplementary Figure S8.A total of 2590 distinct TUs were predicted, with 1402, 1398, 1545 and 1604 TUs in datasets 1 through 4, respectively. The detailed TUs were shown in Supplementary file 1. Figure C. thermocellum has 2552 genes on the leading strand and 637 on the lagging strand. In addition, multi-gene TUs account for 81% of all TUs on the leading strand while 19% on the lagging strand. The average gene count per TU is 2.6 on the leading strand and 1.5 on the lagging strand. The detailed results are shown in Supplementary Figure S2.We observed that TUs on the leading-strand have more genes than TUs on the lagging-strand. This is consistent with a previous publication, which found that the leading strands tend to have more multiple-gene TUs than the lagging strands in ∼200 bacterial genomes . We examWe have assessed the accuracy level of the predicted TUs, based on the confirmed TUs by the 454 RNA-seq datasets, where a TU is considered to be confirmed if it is entirely covered by at least one 454 read. Specifically, the prediction sensitivities of the four TU predictors are at 0.90, 0.94, 0.94 and 0.93, respectively, when 396 and 331 confirmed TUs in the control and the treatment datasets are compared against the predicted TUs in the corresponding datasets. We showed a discordant case in Supplementary Figure S3 to explain why our predictions cannot identify TUs suggested by 454 data.C. thermocellum: (i) enrichment of cis-regulatory motifs in the immediate upstream regions of the predicted TUs ransBase , whose dransBase ,35 and SransBase  and SuppThe relationship between sequencing depth and the number of predicted TUs is shown in Figure To further assess the stability of our method when applied to RNA-seq data with different sequencing depths, we have examined the relationship between the sequencing depth and the performance of TU prediction. We used dataset 4 as an example and performed reads resampling (without replacement) to simulate datasets with 40 different levels of sequencing depths ranging from 0.1X to 291X, with an increment of 7.46X. To avoid resampling bias, we performed 25 resampling experiments for each depth. The TU prediction results of resampling experiments were evaluated by the 454 dataset using the same process mentioned in the last section. Figure To elucidate the dynamic nature of TUs under different conditions of ethanol treatment, we performed functional enrichment analysis of differently composed TUs extracted from 1402 and 1604 TUs in dataset 1 (control condition) and dataset 4 (treatment condition), respectively. They share 1007 TUs and dataset 4 consists of 597 differently, where 341 shared the same 5′ genes and 341 shared the same 3′ genes with those in the control set. Among the 597 TUs, 88 have their genes differentially expressed in dataset 4 compared to dataset 1 (with fold change cutoff as 4), which consist of 265 genes; and such genes are considered to be relevant to cellular responses to ethanol treatment. The longest TU among the 88 contains 14 genes (Cthe_2470–2482), which are partially related DNA adenine methylase. The corresponding TU under the control condition consists of the first ten genes and the other four fall into a separate TU.Our function-enrichment analysis of these genes using DAVID  revealedE. coli K12 MG1655 to check the generality of our TU predictors. Specifically, we have built TU predictors based on E. coli data (single-end reads) as done on C. thermocellum data in Illumina-only model. The three trained TU predictors achieved the training accuracy at 0.95, 0.94 and 0.93 on datasets SRX315217, SRX315218 and SRX315219, respectively, with the ROC curves given in Supplementary Figure S9. 1003, 1340 and 1095 annotated TUs based on corresponding paired-end are used to assess the prediction sensitivities on the three datasets, which achieve a sensitivity level at 0.8, 0.9 and 0.92, respectively.We used three sets of RNA-seq data of C. thermocellum and evaluated using a few general genomic features along with a reliable transcriptomic dataset. We observed an association between sequencing depth and the number of predicted TUs, which reveals that a good level of TU prediction requires at least 60X sequencing depth. In addition, the read resampling experiments have shown that TU prediction performance is stable with sequencing depths beyond 60X, while it goes down dramatically when the sequencing depth is below 7.6X. The result suggests 7.6X as the minimum sequencing depth for reliable TU prediction using our method.In this study, we developed a computational method to predict bacterial TUs using RNA-seq data and two features of expression patterns across two consecutive genes and their intergenic region. A total of 2509 distinct TUs were predicted in Lack of large-scale, experimentally verified TUs as training data is a bottleneck in developing computational models for TU identification and evaluation of the prediction results. Some TU prediction studies used mostly predicted operons to evaluate their TU annotations, which is clearly not appropriate knowing the fundamental difference between operons and TUs. Some studies also found new, alternative, extended operons (or expressed transcription units) not identified by operon predictions ,39. Our We fully expect that more transcriptomic data will lead to improved prediction of TUs. For example, 5′ enriched sequencing data should help to improve the prediction of transcription start sites. In addition, more informative RNA-seq libraries such as paired-end reads with longer in-between regions may reveal better continuity information of a transcript. Another area for improvement, with more challenging issues, is to predict overlapping TUs expressed under the same conditions, which we plan to do in the near future.Supplementary Data are available at NAR Online.


One of the most important application spectrums of transcriptomic data is cancer phenotype classification. Many characteristics of transcriptomic data, such as redundant features and technical artifacts, make over-fitting commonplace. Promising classification results often fail to generalize across datasets with different sources, platforms, or preprocessing. Recently a novel differential network rank conservation (DIRAC) algorithm to characterize cancer phenotypes using transcriptomic data. DIRAC is a member of a family of algorithms that have shown useful for disease classification based on the relative expression of genes. Combining the robustness of this family’s simple decision rules with known biological relationships, this systems approach identifies interpretable, yet highly discriminate networks. While DIRAC has been briefly employed for several classification problems in the original paper, the potentials of DIRAC in cancer phenotype classification, and especially robustness against artifacts in transcriptomic data have not been fully characterized yet.In this study we thoroughly investigate the potentials of DIRAC by applying it to multiple datasets, and examine the variations in classification performances when datasets are (i) treated and untreated for batch effect; (ii) preprocessed with different techniques. We also propose the first DIRAC-based classifier to integrate multiple networks. We show that the DIRAC-based classifier is very robust in the examined scenarios. To our surprise, the trained DIRAC-based classifier even translated well to a dataset with different biological characteristics in the presence of substantial batch effects that, as shown here, plagued the standard expression value based classifier. In addition, the DIRAC-based classifier, because of the integrated biological information, also suggests pathways to target in specific subtypes, which may enhance the establishment of personalized therapy in diseases such as pediatric AML. In order to better comprehend the prediction power of the DIRAC-based classifier in general, we also performed classifications using publicly available datasets from breast and lung cancer. Furthermore, multiple well-known classification algorithms were utilized to create an ideal test bed for comparing the DIRAC-based classifier with the standard gene expression value based classifier. We observed that the DIRAC-based classifier greatly outperforms its rival.Based on our experiments with multiple datasets, we propose that DIRAC is a promising solution to the lack of generalizability in classification efforts that uses transcriptomic data. We believe that superior performances presented in this study may motivate other to initiate a new aline of research to explore the untapped power of DIRAC in a broad range of cancer types.The online version of this article (doi:10.1186/s12859-015-0737-3) contains supplementary material, which is available to authorized users. High-throughput genomic technologies have dramatically expanded the breadth of biological information available for the analysis and characterization of disease. One of the most important application fields of transcriptomic data is classification, where a multitude of promising results have been reported in classification literatures with near perfect classification accuracy for given datasets. Yet, despite this progress, the reliability of these results has been proven illusory, as the seemingly promising results often fail in reproducing, inextricable yet largely unachieved goal, in new datasets . This iset al. (2010) . This binary vector is then used to create a most likely binary vector termed the Rank Template. Each element of the Rank Template is 1 if the probability of the gene comparison at that position is greater than 50 % under a given condition, i.e., RTij= [ P(gi<gj|C)>0.5]. The Rank Template for each condition is then used in classification to generate a Rank Matching Score. This is effectively one minus the Hamming distance between an unknown sample’s pairwise gene comparison and the Rank Template, i.e., DIRAC is composed of a set of measures to quantify differential expression variability between conditions using subsets of genes. These subsets generally correspond to predefined gene networks or pathways. The rationale for composing the analysis using these subsets comes from the perspective of systems biology. The contextualization of expression changes in functional units is viewed as more informative than looking at gene-level changes in isolation. Briefly, DIRAC transforms the rank ordering of genes within a network into a binary vector of pairwise gene comparisons Fig. . A more . (2010) . Each elnot a classification algorithm. It falls within the realm of data transformation, i.e. project samples that are originally in the gene space to a much lower dimensional pathway space. The hope is that the difference between groups under investigation will be pronounced in the new space. The transformed data can be combined with any exiting classification algorithm.We find it necessary to point out that the DIRAC is sensitivity×specificity)/(sensitivity+specificity), is a harmonic mean of the former two. It ranges from 0 to 1, with one being the perfect classification.We first perform the pathway and GEV signatures identification for each cytogenetic subtype. Then, the signatures are used in later stage to evaluate their robustness against different preprocessing techniques and reproducibility in independent datasets in presence of batch effect, respectively. Performances of the two classifiers are measured by calculating their sensitivity, specificity and F-score. The F-score, that equals 2: RUNX1T1, POU4F1, CACNA2D2.inv(16): MN1, TM4SF1, MYH11, CLIP3, SPARC, AK5, PTRF, FAM171A1, AK022033, LOC100506388.t: STAB1, FGF13, LGALS12, PTGDS.NUP98.NSD1: VENTX, PRDM16, FAM92A1.CEPBA.dm: LRRC28, CNRIP1, ST8SIA1, MSS51, SLC14A1, LGSN.NPM1: HOXB-AS3, HOXB3, SMC4, SEPP1, LTBP1, CLIC2, HOXB4, TWISTNB, HIST1H2BC, SIAE, PIEZO2, HLF, ELL2, LOC100507520, HOXB2, FOXC1, HOXB6, HOXB5, EMR1, SNCAIP, RPL39L, USP44, BEX1, TTC27, PTPRC, HENMT1, AK027199, COL4A5, NAP1L5, TIAM1, NPDC1, CLEC11A, SCD5, CCL1, FTO, AK093529, ENSG00000184551, CDKN1B, FAM105A, PHLDA1, HOXA-AS5, LOC100506591, GMDS, TOM1L1, IL12A, DMXL2, SDPR, FOXF1.CBPA double mutations, NPM1 mutations are characterized by one or more tightly regulated MAP kinase pathways, P38MAPK, KERATINOCYTE and/or MAPK. This suggests that inhibition of MAP kinase activity within these pathways may be a way to effectively target these cells, which needs further biological validation studies. Another pathway of interest is the BIOPEPTIDES  pathway, which is characterized by the RAS, JAK and STAT signal transduction route. This pathway appears to be tightly regulated in subgroups characterized by t, Inv(16) and NUP98/NSD1. Surprisingly, AML with MLL rearrangements were not classified by any tightly regulated pathway, but rather by three loosely regulated pathways. This may be explained by the relative heterogeneity of this subgroup, which contains several MLL translocation partners, in particular AF6, AF9 and AF10, which have strikingly different clinical characteristics [DIRAC considers a pathway to be tightly regulated under a given condition if, for that condition, the rank ordering of the genes in that pathway are highly correlated; loosely regulated if they are less correlated. We noticed that, the pathway signatures Table  specificThese pathway signatures may be options for understanding specific disease mechanisms and provide a key for the design of new treatment. For example, one may seek network aware intervention to adjust the signature pathways in each subtype to its normal non-leukemic cell behavior and, consequently, to control clinical presentation.et al. (2011) [1 dataset and were used to predict subtype labels of the same dataset that has been preprocessed using six well-known methods. This is meant to be representative of a situation where a researcher intends to combine or compare transcriptomic profiles from two studies where the preprocessing methods were different and the original unprocessed data is not available. Ideally, one would have the original data and use the same preprocessing methods on those data sources, but this is not always an option. The six preprocessing methods are: M1) CEL file transformed to raw expression data using Robust multi-array average expression measure (RMA); M2) Raw data obtained with RMA and normalized with quantile normalization; M3) CEL file transformed to raw expression data using Affymetrix’s MAS 5.0 expression measure (MAS5); M4) Raw data obtained with MAS5 and normalized with scale normalization; M5) CEL file transformed to raw expression data using Robust multi-array average expression measure with help of probe sequence (GCRMA); M6) Raw data obtained with GCRMA and normalized with quantile normalization.Multiple preprocessing methods for gene expression data have been proposed . However. (2011)  reportedFigures Genomic data often contain batch effects, i.e. differences in sample sources and other technical variations. Many algorithms have been proposed to remove these artifacts, but this (often) comes along with the potential cost of removing between-group biological heterogeneity and consequently salient genomic signatures . Note th2 and the adult AML datasets. The GEV-based classifier relies on expression values and reflects commonly used practice when no additional knowledge is available. Both pathway and gene expression classifiers were trained using all samples in the pediatric 1 dataset and used to predict the subtype labels of the two independent datasets. We observed strong batch effects between the pediatric 1 and the two new independent datasets  subtypes.To demonstrate the robustness in translating trained predictor across independent datasets, we analyzed performance characteristics of the DIRAC-based classifier as well as the GEV-based classifier on the pediatric In order to better comprehend the prediction power of the DIRAC-based classifier in general, we conducted performance comparison using publicly available datasets from breast and lung cancer . Then, the rank matrix, instead of the original of gene expression values, was used to construct classifiers. Classification results are shown in Fig. Clearly, the DIRAC-based classifier is robust against most of the major ancillary sources of variation in the data. We attribute this to the robustness of relative expression compared to absolute expression values used  and the According to the no-free-lunch theorem  no classIn this study, we investigate the performance of the DIRAC-based classifier using multiple AML datasets. We believe that its superior performances presented in this study may motivate other to explore the untapped power of DIRAC in different types of cancer or may even open a new line of application.In summary, through this study we have demonstrated the robustness of DIRAC in classification, which has previously been undocumented and underestimated. While this is not the first multi-class DIRAC method proposed , we haveIn this study we investigate the potentials of DIRAC in multiple scenarios frequently appear in analyzing high-dimensional transcriptomic data. The number of conceivable applications of DIRAC is possibly large, hence we do not claim to provide exhaustive list. Instead, the focus is on classification problem that has been the subject of bioinformatics research for long time. Reported experimental results from multiple real-world datasets serve two ends: 1) to illustrate the benefits of the DIRAC-based classifier, 2) to clarify what makes the DIRAC-based classifier superior than exiting approaches. In the following sections we describe the datasets that are considered in this study, and the experimental setups, i.e. how we compare performance of the DIRAC-based classifier with exiting approaches in an unbiased way.1 AML dataset, which was generated in our lab, the raw data (CEL files) were also available [Three publicly available acute myeloid leukemia (AML) datasets Table  were usevailable . Details1 and adult AML datasets, 57 % in the pediatric 2 AML dataset. Note that, for the DIRAC-based classifier only those genes that belong to the retained pathways were used. For the GEV-based classifier the full list of genes, not just the pathway mapped subsets, was used. Note that, the number of genes used for the GEV-based classifier is far more exceeds the number of pathways used for the DIRAC-based classifier. This means, the former has a larger search space to find the discriminative features compared to the latter.In this study, 217 manually curated (comprised of 1279 unique genes) BioCarta pathways from the Molecular Signatures Database  were used to construct the DIRAC-based classifier. For each dataset, genes were grouped according to their presence in these pathways. To reliably represent the pathways, only those pathways that have at least 30 % of associated genes presented in a given dataset were retained. The minimum fraction of genes that are present both in individual pathways and gene expression data is: 70 % in the pediatric 1 AML dataset  [We used the pediatric et Table  to gener. (2011) . SpecifiTo obtain reliable results, we performed 100 outer-cross-validation (OCV) and 100 inner-cross-validation (ICV) on the discovery set Fig. . In eachThe selected features (50 genes/pathways) were processed in following ways to obtain reduced set of GEV and pathway signatures. For each subtype, the GEV (pathways) signatures were ranked (from best to worst) according to their association with the subtype via the global test . Subsequet al. (2011) [At the end of 100 ICV iterations, a signature list of smallest size that renders highest prediction sensitivity was determined. Among the three optimal lists generated via 3-fold CV using the discovery set, the one with the median prediction sensitivity was taken as the optimal signature list for one OCV iteration. The final classifier was trained using the signature list that produced the highest prediction sensitivity in 100 OCV iterations and subsequently used to predict the labels of the validation set. Balgobind . (2011)  argued tEthics approval is not required for this study.Additional file 1Supplementary material. This document includes additional information not included in the paper. (PDF 1239 kb)