Parallelization of qPCR such as by microfluidic Taqman Fluidigm Biomark Platform enables evaluation of multiple transcripts in samples treated under various conditions. Despite advanced technologies, correct evaluation of the measurements remains challenging. Most widely used methods for evaluating or calculating gene expression data include geNorm and ΔΔCt, respectively. They rely on one or several stable reference genes (RGs) for normalization, thus potentially causing biased results. We therefore applied multivariable regression with a tailored error model to overcome the necessity of stable RGs.Gene expression analysis is an essential part of biological and medical investigations. Quantitative real-time PCR (qPCR) is characterized with excellent sensitivity, dynamic range, reproducibility and is still regarded to be the linear error model for parallel qPCR data, called LEMming. It uses the assumption that the mean Ct values within samples of similarly treated groups are equal. Performance of LEMming was evaluated in three data sets with different stability patterns of RGs and compared to the results of geNorm normalization. Data set 1 showed that both methods gave similar results if stable RGs are available. Data set 2 included RGs which are stable according to geNorm criteria, but became differentially expressed in normalized data evaluated by a t-test. geNorm-normalized data showed an effect of a shifted mean per gene per condition whereas LEMming-normalized data did not. Comparing the decrease of standard deviation from raw data to geNorm and to LEMming, the latter was superior. In data set 3 according to geNorm calculated average expression stability and pairwise variation, stable RGs were available, but t-tests of raw data contradicted this. Normalization with RGs resulted in distorted data contradicting literature, while LEMming normalized data did not.We developed a RG independent data normalization approach based on a tailored If RGs are coexpressed but are not independent of the experimental conditions the stability criteria based on inter- and intragroup variation fail. The linear error model developed, LEMming, overcomes the dependency of using RGs for parallel qPCR measurements, besides resolving biases of both technical and biological nature in qPCR. However, to distinguish systematic errors per treated group from a global treatment effect an additional measurement is needed. Quantification of total cDNA content per sample helps to identify systematic errors. Fluorescence-based quantitative real-time PCR (qPCR) is the commonly accepted gold standard to quantitate the amount of mRNA transcripts in biological samples. Benefits of this procedure over conventional methods for measuring RNA include its sensitivity, large dynamic range and its potential for high throughput as well as accurate quantification . To achigeNorm [NormFinder [BestKeeper [Depending on the number of measured gene transcripts and samples the available normalization strategies can be roughly divided into three groups, (i) knowledge-driven approaches, (ii) data-driven approaches and (iii) modeling approaches. Knowledge-driven approaches are usually applied to small data sets that measure only few gene transcripts in a limited number of samples. In such cases, preselection of internal standards is used for data normalization. Usually, such an internal standard is represented by a number of reference genes (RGs) known to be stably expressed under the different experimental conditions. While a single RG is sufficient for data normalization under ideal assumptions, according to MIQE guidlines  it is begeNorm , NormFinrmFinder  and BeststKeeper . While tThe second group of approaches encloses the data-driven normalization methods such as quantile normalization and rank-invariant set normalization. These methods where initially developed for the normalization of high-throughput gene expression data such as microarrays. In their study, Mar et al.  proposedThe third group of methods is formed by modeling approaches, i.e., approaches that model the gene expression values as a composition of the true gene expression and various effects of technical and biological nature. For example Steibel et al.  use lineCt method described by Livak and Schmittgen [Ct method relies on a single RG. Current tools like +qBase [DAG Expression [SASqPCR [geNorm allows the most appropriate RGs to be chosen. geNorm calculates gene expression stability values (M) values for each gene, being values below 1.5 the most stable RGs. Additionally, it estimates the normalization factor (NFn) using the geometric mean of expression levels of n best RGs, given by the pairwise variation (V) analysis, by selecting a cut-off (0.15) below which n RGs should be used for normalizing.The use of RGs in a knowledge-driven approach is widely spread for normalization and is implemented in many software tools for qPCR data analysis. Software tools are reviewed by Pabinger et al . A basichmittgen  and Pfafhmittgen . The ΔΔCe qBase+ , DAG Exppression  and SASq[SASqPCR  use mult[SASqPCR , who sugBas et al  and Tricalbumin, actins, Gapdh, tubulins, cyclophilin as well as 18S or 28S rRNA, may vary under experimental conditions. Furthermore, the difficulty of finding reliable RGs increases with the number of experimental conditions used in one experimental setup. The usage of multiple RGs can reduce the impact of single outliers in a RG measurement on the whole sample. However, in case of responsiveness of RGs towards experimental conditions, the number and the choice of RGs will alter the results. A distortion of results can be identified if the mean of raw values compared to processed values is shifted under an experimental condition.Numerous studies –19 have LEMming (linear error model) and uses linear mixed models. LEMming allows the exclusion of technical errors, the retention of the biological variation and the exclusion of systematic errors, if they are identified by external measurements (e.g. cDNA quantification). LEMming reduces the variance in measurements while exerting no influence on the mean value of gene measurements under an experimental condition. Here we use LEMming to analyze three data sets with different stability patterns of RGs. Furthermore, we use LEMming to assess the contribution of several sources of variation in qPCR data. LEMming is implemented in the freely available language R and can be applied by customization of an R-script to a data set.To solve these problems we introduce a new modeling approach based on multivariable regression, which is specialized for the normalization of parallel qPCR data. Spurgeon et al.  introducWe selected three data sets with different stability patterns of RGs reflecting different experimental situations:Data set 1 (DS1): The first experiment was designed for precise quantification of various technical errors, as well as for the evaluation of reverse transcription (RT) reaction variability. In detail, HepG2 cells were treated with peroxisome proliferator activating receptor alpha (PPARa) agonist (N = 4). HepG2 cells, passage 11, were plated out in 12-well plates at 5 × 105 cells/well density in high-glucose DMEM medium , containing Glutamine and 10% FCS. 24h later, 100 μM WY14,643—4-Chloro-6--2-pyrimidinylthioacetic acid  or solvent control, DMSO—Dimethyl sulfoxide  were added to 4 wells each . After 48h of treatment, all wells were lysed using 300 μL of RLT buffer from RNeasy kit  and RNA was immediately isolated following manufacture’s instructions. Next, three independent RT reactions have been performed from each sample, resulting in 3 cDNAs for each sample . Finally, 24 generated cDNAs were run in technical duplicates on a 48×48 microfluidic dynamic array. To sum up, DS1 has four biological replicates, three technical replicates for each cDNA conversion step and two replicates for the qPCR step per condition. DS1 is available in Data set 2 (DS2): DS2 was selected because of its technical replicates and its seemingly stable RGs in raw data. Male mice (C56Bl6N)  were employed in this study. The animal protocol was reviewed and approved by the “Thüringer Landesamt für Lebensmittelsicherheit und Verbraucherschutz—Dez.22/Fachgebiet Tierschutz/Tierarzneimittel” Germany . Liver tissue samples, generated in a vivo experiment in mice, were investigated under the influence of starvation on the kinetic expression level with 7 housekeeping genes . The mice were denied access to food for a period of 0, 24, 48 or 72 hours (n = 6 each group). The mice were housed under standard animal care conditions and access to water over the whole time. After observation times liver tissue samples were taken. The mRNA was extracted from the frozen tissues using the Qiagen RNeasy Mini Kit . RNA quantity was measured using Nanodrop . RNA integrity number (RIN) was checked by Agilent 2100 Bioanalyzer  and was above 8.5 for all samples. cDNA synthesis was performed with 2 μL of 50 ng/μL total RNA, 1 μL of 10 × TaqMan RT Buffer, 2.2 μL 25mM MgCl2, 2 μL of 2.5 nM dNTP-Mix, 0.5 μL of 50 μM random hexamers, 0.2 μL of RNase Inhibitor, 0.25 μL of 50 U/μL Multiscribe reverse transcriptase and 1.85 μL RNase-free water. All reagents were purchased from Applied Biosystems (TaqMan Reverse Transcription Reagents: N808-0234). The reaction mixtures were mixed with RNA and incubated by 25°C for 10 min, 48°C for 30 min and then 95°C for 5 min. All RNA samples were transcribed twice to detect systematic errors during the cDNA synthesis. In the end, 48 generated cDNAs were put on in technical duplicates on a 96×96 microfluidic dynamic array. Likewise, all 46 primers were put in twice on the array as technical replicates. The DS2 is available in Data set 3 (DS3): DS3 was selected as a complex in vivo experiment where no stable RGs were identified in raw data. Male inbred Lewis rats (Lewis/HanTMHsd)  were employed in this study. The animal protocol was reviewed and approved by the “Thüringer Landesamt für Lebensmittelsicherheit und Verbraucherschutz—Dez.22/Fachgebiet Tierschutz/Tierarzneimittel” Germany .in vivo experiment in rats, were used to investigate the influence of different experimental conditions  on the kinetic expression level of 9 commonly used RGs .Liver tissue samples, generated in a complex The experiment was originally dedicated to explore the effect of vaso-active drugs  on the recovery from focal outflow obstruction in rats with portal hypertension due to liver resection (named: ligation/PH). Focal outflow obstruction was induced by ligation of the right median hepatic vein. Animals subjected to ligation only (named: ligation) and untreated rats (named: untreated) were used as controls. Samples were obtained after observation times of either 0h, 24h, 48h and 7d from 3 different locations of the right median lobe: the obstructed zone, the border zone and the normal zone. Further details regarding the experimental design and procedures are published in Hai et al. .The mRNAs were extracted from the frozen tissues based on manufacturer’s protocols from Qiagen RNeasy Mini Kit . RNA quantity was determined using Nanodrop  and the quality was assessed by Agilent 2100 Bioanalyzer . RNA integrity number (RIN) was above 8.5 for all samples. cDNA synthesis was performed as described in DS2 with reagents by Applied Biosystems (TaqMan Reverse Transcription Reagents: N808-0234).Pre-designed validated Taqman Gene expression assays were purchased from Life Technologies  for the detection of human DS1), mouse DS2) and rat DS3) transcripts. Gene expression assays are listed in the according data set table , TBP—TATA box binding protein (Hs00427620_m1), POLR2A—polymerase (RNA) II (DNA directed) polypeptide A (Hs00172187_m1), GAPDH—Glyceraldehyde-3-phosphate dehydrogenase (Hs02758991_g1), HMBS—hydroxymethylbilane synthase (Hs00609296_g1) and RPLP0—ribosomal protein, large, P0 (Hs02992885_s1).For the analysis of DS1, following genes were used as potential reference genes: Actb—Actin beta (Rn00667869 m1), Gapdh—Glyceraldehyde-3-phosphate dehydrogenase (Rn01775763 g1), Hprt1—Hypoxanthine phosphoribosyltransferase 1 (Rn01527840 m1), Mrps18a—Mitochondrial ribosomal protein S18A (Rn01511938 m1), Rpl27—Ribosomal protein L27 (Rn00821099 g1), Rpl32—Ribosomal protein L32 (Rn00820748 g1), Rps13—Ribosomal protein S13 (Rn02606812 g1), Tfrc—Transferrin receptor (Rn01474701 m1) and Ywhaz—Tyrosine 3-monooxygenase/tryptophan 5-monooxygenase activation protein, zeta (Rn00755072 m1).The assays of 9 reference genes used for DS2 and DS3 analysis are: qPCR was carried out using the 48×48 (for the DS1) and 96×96 (for the DS2 and DS3) Dynamic Array  with minor modifications to the manufacturer’s protocol found in Spurgeon et al. . BrieflyA dynamic array for Fluidigm Biomark measurements has 48 or 96 probe slots and 48 or 96 sample slots. The center of the chip is an integrated fluidic circuit (IFC), which is a network of fluid lines, NanoFlex valves and reaction chambers . The flum0, the cDNA is roughly doubled (PCR efficiency E is around 2) in every PCR cycle. The cDNA generates a fluorescence signal. When the fluorescence signal exceeds a certain threshold T the number of cycles Ct are registered.Ct value indicates a lower mRNA starting concentration m0.A protocol for quantification of mRNA by real-time RT-qPCR (reverse transcription (RT) followed by polymerase chain reaction (PCR)) is given by Nolan et al. . The tarRT-qPCR measurements have technical and biological sources of variation. Sources of technical variation are pipetting errors of probes and samples, as well as errors within the steps of RNA extraction, reverse transcription (RT) and qPCR. The most critical step is the RT, which contributes most to the variation of mRNA measurements . The errCt methodThe ΔΔ is described by Livak and Schmittgen [TG) in comparison to a reference gene (RG). The relative expression ratio of a TG is calculated based on PCR efficiency E and Ct deviation of a sample (s) versus a control (c) [geNorm is a method for the identification of the most stable RGs given a list of potential candidate RGs [NF). For each sample, NF itself is defined as the geometric mean of their expression values. The computation of expression stability and normalization factor as well as the calculation of geNorm normalized values that are comparable with raw and LEMming processed data is described in detail in Ct method for normalization as well as identification of stable RGs using geNorm require complete data sets without missing values. Therefore, imputing of missing values was performed using K-nearest neighbor imputing [hmittgen  and Pfafhmittgen . It detetrol (c) :ratio= is the effect or mean of all 48/96 sample measurements of a gene on an array. Pipetting of the probe and the channel for probe transportation on the array influence ϵP:A. The sample error (ϵS) is the effect or mean of all 48/96 gene measurements of a sample. Sample preparation steps which are exclusive to a sample like pipetting, the channel for sample transportation and the RNA-extraction influence ϵS. Both technical errors are systematic influences which are present in 48/96 measurements at a time. They can be excluded in order to reduce variance of all measurements.We distinguish two types of technical errors: The probe error per array  and the treatment effect per gene (ΔT:G). Biological variance and non-systematic technical errors are described by the variable ϵ.Other systematic effects like influence of treatment and biological variance are part of the measurement and should be retained for visualization. The treatment effect splits up into two parts, the global effect .Estimate the probe error per array (Estimate systematic errors/batch effects (T).Estimate the treatment/tissue effect (ΔϵS).Estimate the sample error (T:G).Estimate the treatment effect per gene .The fold change values are 2ebayes with option robust from the limma package [ package .We recommend to use a proper experimental design (e.g. latin square). Based on that systematic errors can be identified and excluded by using LEMming.geNorm  calculated from the genes GAPDH, HMBS, RPLP0 and TBP show a low pairwise variation compared with the NF that includes POLR2A (pairwise variation 0.15). Thus, there is no need to include POLR2A into the calculation of the NF. Consequently, we used the four most stable RGs for normalization with geNorm. Standard deviations (σ’s) per gene and condition were computed from raw data, from ΔΔCt, geNorm and LEMming normalized data. The comparison of σ’s between raw data, ΔΔCt and LEMming normalized data is presented in H0-hypothesis of equality of σ’s with the Wilcoxon signed-rank test. Std. devs of LEMming residuals per gene and condition are significantly smaller than std. devs of raw data (p ≤ 10−11) and std. devs of ΔΔCt normalized data (p ≤ 10−6). Additionally, we present a proof in Ct with a single RG is greater than of std. devs LEMming processed data. The boxplots per gene and condition comparing raw data, geNorm and LEMming processed data are presented in geNorm, but not with LEMming. A comparison between std. devs of geNorm and LEMming processed data per gene is shown in geNorm and 23.7% ± 3.8 with LEMming compared to raw data. The std. devs of WY14,643 treated samples are reduced on average by 1.5% ± 9.7 with geNorm and 6.5% ± 2.8 with LEMming compared to raw data. We calculated the average expression stability and the pairwise variation of common RGs in DS1 according to geNorm are presented in Eef1a1, Hprt and Ppia were chosen for geNorm normalization . We applied LEMming to DS2  which is present for all genes measured in DS2 . In contrast, t-tests with geNorm normalized data have significant results for all three RGs . LEMming normalized data shows also significant results for the three RGs (see LEM_diff).T-tests on the three RGs in raw data have no significant results comparing the 24h, 48h and 72h condition to the 0h condition with a Bonferroni corrected geNorm and LEMming processed data are compared in geNorm (sign test p ≤ 10−11). The mean reduction of std. dev. per gene and condition is 49.4% for geNorm and 50.5% for LEMming. LEMming excludes systematic effects that are responsible for 83.1% of variance of the median gene in this experiment.Std. devs of geNorm are presented in geNorm and LEMming processed data of RGs in DS3 is available as geNorm and LEMming processed values and the according variance-mean plot is part of geNorm and LEMming shift the mean values per gene and treatment, whereby the distance to the mean of raw data is smaller with LEMming. In LEMming this effect appears, because the data set is measured on three different arrays. The exclusion of batch effects with LEMming in DS3 is discussed in The average expression stability and the pairwise variation of common RGs in DS3 according to geNorm (and NormFinder) Rpl27, Rps13 and Rpl32 are the most stable RGs which were selected for calculation of the NF .We analyzed how variance relates to the absolute ncreases . The samncreases . Here thϵS and the residuals of LEMming ϵ  to the sample error EMming ϵ  can be rϵ is explained by 24% (median) biological variance, 36% (median) cDNA conversion error and 38% (median) qPCR error in the control group  to the overall variance, while biological variance only accounts for 23% and qPCR error for 15% of the overall variance see . This isϵS into a cDNA conversion error and a sample pipetting error. Here the cDNA conversion is the dominant variance source in the raw data. Due to the multiple technical replicates in the experiment, this effect is nearly completely removed by the LEMming approach.The same analysis was performed with DS2 see . With DSϵ of LEMming can be described by a Student-t distribution. Distribution tests ) for DS1 accepted the H0 hypothesis (Kolmogorov-Smirnov test p = 0.228 and Kuiper test p = 0.096). H0 hypothesis  was accepted by a Kolmogorov-Smirnov test (p = 0.723) and a Kuiper test (p = 0.397). The QQ-Plot The residuals Here we generated and validated a method for normalization of parallel qPCR measurements, called LEMming, which is based on a linear error model to exclude technical and systematic errors. LEMming takes advantage of the experimental design of qPCR studies conducted on microfluidic arrays, which is a high throughput platform for qPCR. Our LEMming tool allows, therefore, the analysis of such data without usage of reference genes (RGs). We applied LEMming to three data sets with different stability patterns of common RGs.geNorm and LEMming resulted in nearly the same results. The slight shift in the mean value of 5 genes in geNorm processed data could be neglected since the RGs themselves were affected. We concluded that in DS1 geNorm and LEMming are on a par in reduction of standard deviation per gene and treatment group. Both methods were superior to the use of a single RG with the ΔΔCt method.DS1 represents a data set with stable RGs available. geNorm criteria. However, after geNorm and LEMming normalization none of them was accepted as stably expressed by t-tests. Here the question appears, whether such RGs are appropriate for normalization. geNorm-normalized data finally showed an effect of a shifted mean per gene per condition compared to the raw data and LEMming-normalized data. Comparing the decrease of standard deviation from raw data to geNorm and to LEMming, the latter was superior. LEMming compensated for errors that are responsible for 83.1% of variance in the raw data for the median gene  stable RGs existed, while t-tests between treatment conditions in raw data suggested that this is not the case. Even after exclusion of systematic errors by LEMming no stable RGs existed following this t-test criteria. Applying RGs for normalization shifted the mean value per gene under most treatment conditions, which distorts the data analysis. The case of the marker gene Pcna, which is upregulated after 24 h and 48 h of partial hepatectomy (PH), demonstrated that raw data and LEMming are in accordance with microarray data. geNorm makes the opposite claim of down regulation. Interestingly, Assy et al [DS3 evaluated rats after partial hepatectomy under vaso-active drugs in different zones of the liver after various observation times with a control of untreated and ligation of the right median hepatic vein. Thus, DS3 presents a very complex sy et al  found thRpl27, Rps13 and Rpl32 showed similar expression patterns  is the dominant technical error. Treatment conditions had a positive effect on the biological variance compared to the untreated condition. LEMming was able to remove large proportions of technical errors and retained the biological variation. We showed theoretically and practically that applying LEMming results in reduced gene wise variances per treatment group compared to normalization with a single RG. The reduction of these variances is based on the removal of systematic errors which are part of a linear mixed model estimated from the data. It is important to estimate the effects of this model in a particular order, otherwise effects like the global treatment effect would be removed as a sample error. Usually the residuals of linear mixed models are assumed to be normally distributed. If this is not the case, estimated parameters might be biased. We observed a Student t-distribution of residuals of the linear mixed model. This distribution is symmetric like a normal distribution, but has heavier tails. Thus, the estimated effects are not biased, but the standard error of the estimated effects might be inaccurate. To not overestimate the significances of differential expressions, we recommend to use robust tests to analyze them. We used the function ge limma  in orderCt values within the samples of similarly treated groups are equal. Since the genes are selected by the criterion to see a difference between conditions, a global treatment effect ΔT can be shown in most data sets. However, a systematic sample error per treated group T are indistinguishable by LEMming. The use of RGs which are provably independent of the treatment would automatically compensate this. If such RGs are available, we strongly recommend to use them because of their capability to automatically remove systematic batch effects. However, with a growing number of experimental conditions the chance of finding such RGs decreases.LEMming uses the assumption that the means of T) is not distorted.Thus, we recommend to use quantification of total cDNA content per sample as a second independent measurement in order to identify systematic sample errors per treated group Click here for additional data file.S2 File(PDF)Click here for additional data file.S3 File(R)Click here for additional data file.S4 File(PDF)Click here for additional data file.S1 Table(XLSX)Click here for additional data file.S2 Table(XLSX)Click here for additional data file.S3 Table(XLSX)Click here for additional data file.


Adenocarcinoma (ADC) and squamous cell carcinoma (SCC) are the most prevalent histological types among lung cancers. Distinguishing between these subtypes is critically important because they have different implications for prognosis and treatment. Normally, histopathological analyses are used to distinguish between the two, where the tissue samples are collected based on small endoscopic samples or needle aspirations. However, the lack of cell architecture in these small tissue samples hampers the process of distinguishing between the two subtypes.Molecular profiling can also be used to discriminate between the two lung cancer subtypes, on condition that the biopsy is composed of at least 50 % of tumor cells. However, for some cases, the tissue composition of a biopsy might be a mix of tumor and tumor-adjacent histologically normal tissue (TAHN). When this happens, a new biopsy is required, with associated cost, risks and discomfort to the patient. To avoid this problem, we hypothesize that a computational method can distinguish between lung cancer subtypes given tumor and TAHN tissue.Using publicly available datasets for gene expression and DNA methylation, we applied four classification tasks, depending on the possible combinations of tumor and TAHN tissue. First, we used a feature selector (ReliefF/Limma) to select relevant variables, which were then used to build a simple naïve Bayes classification model. Then, we evaluated the classification performance of our models by measuring the area under the receiver operating characteristic curve (AUC). Finally, we analyzed the relevance of the selected genes using hierarchical clustering and IPA® software for gene functional analysis.All Bayesian models achieved high classification performance (AUC > 0.94), which were confirmed by hierarchical cluster analysis. From the genes selected, 25 (93 %) were found to be related to cancer (19 were associated with ADC or SCC), confirming the biological relevance of our method.The results from this study confirm that computational methods using tumor and TAHN tissue can serve as a prognostic tool for lung cancer subtype classification. Our study complements results from other studies where TAHN tissue has been used as prognostic tool for prostate cancer. The clinical implications of this finding could greatly benefit lung cancer patients.The online version of this article (doi:10.1186/s12885-016-2223-3) contains supplementary material, which is available to authorized users. Lung cancer is the leading cause of human cancer death in both sexes in the United States. In 2014, there was an estimate of 224,210 new cases, while 159,260 patients were estimated to have died from the disease , other bAdenocarcinoma (ADC) and squamous cell carcinoma (SCC) are the most common histological subtypes among all lung cancers. Both of them are a form of cancer that develops in the epithelial cells (carcinoma), and belong to the category of non-small cell lung cancer. Lung ADC develops in the glands that secrete products into the bloodstream or some other cavity in the body –the mucus secreting glands in the lungs. Most lung ADC arise in the outer, or peripheral, areas of the lung . In contThe diagnosis of early stage lung cancer involves the use of imaging techniques, followed by a biopsy for pathology analysis . InitialSeveral studies have shown that molecular profiling of lung carcinoma is a viable tool for disease diagnosis , and proThe molecular mechanisms of ADC and SCC are considerably different. The standard molecular testing for lung cancer is to check for mutations of two molecules: epidermal growth factor receptor (EGFR) and rearrangement of anaplastic lymphoma kinase (ALK). Each protein has mutations that lead to the development of lung cancer. However, EGFR is found to be mutated only in around 10 % of tumors . SimilarDNA methylation is an emerging diagnostic technology to measure the epigenetic changes in the DNA, characterized by the addition of a methyl group in regions of the DNA known by having CpG islands. Traditionally, gene expression has been used as a prognostic biomarker for lung carcinoma, and differentially expressed genes between lung cancer subtypes have been found . HoweverComputational modeling methods, such as Bayesian classifiers, have been used successfully to model the complexity of genomic data. A study by Chang and Ramoni , yieldedA major challenge of our study is the lack of tissue availability in public datasets. Typically, a biopsy tissue represents a very small portion of the lung. In spite of ultrasound guidance, it is easy to miss a small focal malignancy, and end up retrieving tumor-adjacent histologically-normal tissue (TAHN) along with Tumor tissue. In those cases, the biopsy is discarded if it cannot retrieve more than 50 % of tumor tissue . The patOur goal in this work was to test whether computational modeling can be a viable approach to accurately differentiate between lung cancer subtypes, given molecular profiles of tumor tissue using DNA methylation data. Specifically, we tested the hypothesis that “Bayesian modeling is sufficient to classify lung cancer subtypes, regardless of the tissue sample being tumor or tumor-adjacent.” In this paper, we evaluated the ability of a Bayesian classifier to accurately differentiate lung cancer subtypes using real lung cancer molecular profiling data sets that are also publicly available.ADC vs. TumorADC in gene expression), Additional file SCC vs. TumorSCC in gene expression), and Additional file ADC vs. TumorADC in methylation). The annotations from TCGA to identify these samples are provided in Additional file To test our hypothesis, we extracted datasets containing gene expression and DNA methylation beta values from the Cancer Genome Atlas (TCGA) data portal for lung adenocarcinoma (LUAD ) and lunWe followed a supervised classification process on 10-fold cross-validation. That is, for each fold we partitioned the dataset into training and test, where the former contains 90 % of the samples, while the latter contains the remaining 10 %. We ensured that each partition maintains the same class distribution as the whole dataset (stratified). In each fold, we analyzed the datasets using the experimental design as illustrated in Fig. High-throughput platforms, such as gene expression and methylation microarrays, generate high-dimensional data that is typically very complex for analysis. Feature selection is a machine learning pre-processing step that tries to find a subset of the original variables  that are highly associated with the target class variable . We used the ReliefF algorithm  to rank Limma can be seen as a feature selection method (or ranked list). Similarly to the ReliefF selection, we selected the top 30 most DE genes and DM probe sites (based on log2-fold change) to build a classifier for comparison with ReliefF. The output of the resulting classifiers was evaluated using the area under the receiver operating characteristic curve (AUC) performance metric in the test datasets.Similarly, we also selected the differentially expressed (DE) genes and differentially methylated (DM) probe sites from each dataset using Limma, which is an R-language package for the analysis of microarray data . Limma uMost ‘omic’ data such as gene expression and methylation are represented with continuous values. However, many machine learning algorithms are designed to only handle discrete  data, using nominal variables, while real-world applications, like ‘omic’ data analysis, typically involves continuous-valued variables. Discretization, the process of transforming continuous values into discrete ones, has been shown to improve the performance of machine learning classifiers . To discFor continuous methylation values ranging from 0 to 1, three possible strategies for discretization can occur. The first strategy is when a fixed cut-point is determined arbitrarily for all variables . The second strategy, when an expert-based discretization is made for all variables . The thIn computational genomics, heatmaps are used to graphically show the level of expression that a selected group of genes have in a cohort of patient samples. A heatmap can also be built with methylation intensity values. We build heatmaps from the genes selected by Limma and ReliefF to further validate the results obtained with these feature selection methods. The clusters are a visual representation of the class discrimination ability of the genes selected.The order in which genes (rows) and samples (columns) are ordered in the heatmap matrix is often based on an agglomerative hierarchical clustering. We used the Minkowski measure to calculate the pairwise distances between elements, and then aggregated the closest elements in clusters using the Ward linkage calculation of distances between clusters. This combination of Minkowski distance and Ward linkage has been shown to perform well in biomedical and synthetic datasets .www.ingenuity.com) to gain insight into the biological role of the genes selected by our framework. First, all gene symbols selected were used as input for the IPA platform, which will search for correlations between these genes and functions or pathways in their curated literature. A p-value is computed using Fisher’s right-tailed exact test for the gene list to a function/pathway it may be associated with. The p-values indicate the likelihood of association between the gene set (as selected by ReliefF) and a specific function (set of genes associated with a function) to have occurred due to random chance alone. A p-value of less than 0.05 is considered to be significantly better than random chance. Methylation probe sites were mapped into their corresponding gene symbols that they methylate.We also performed Gene Functional Analysis using QIAGEN’s Ingenuity® Pathway Analysis tool  states of disease. Bayesian networks  are partWe evaluated the NB classifiers using the area under the receiver operating characteristic (AUC), which is a measurement of the area created by plotting the performance of a classifier for the true positive rate versus the false positive rate. When presented with a test dataset, the Bayesian network calculates a posterior probability for every case, and a threshold is used to assign the class for the new cases. The curve is constructed by varying the threshold to which the probability is considered for class determination. Also, the 95 % confidence interval (C.I.) of the AUC was calculated using DeLong’s method for variance estimation .AUC  is a useful measurement of the ability of models to discriminate between two (or more) classes . CalibraTAHNADC vs. TumorADC, and TAHNSCC vs TumorSCC, searches for molecular differences between tumor tissue and TAHN tissue. These tasks are only applied to one lung cancer subtype at a time, either adenocarcinoma or squamous cell carcinoma patients;TumorADC vs. TumorSCC, which searches for molecular differences between subtypes using only Tumor tissue;TAHNADC vs. TAHNSCC, which searches for molecular differences between subtypes using only TAHN tissue; andTAHN-TumorADC vs. TAHN-TumorSCC, which searches for molecular differences between subtypes using both TAHN and Tumor tissue.We investigated four classification tasks depending on the tissue type. These tasks test our hypothesis that the TAHN tissue has distinct genomic signatures that can differentiate among non-small cell lung cancer subtypes. We describe the classification tasks as follows:The classification performance for every naïve Bayes classifier was calculated by averaging the AUCs over all folds from the experimental design illustrated in Fig. All classification tasks achieved high predictive performances with AUC values higher than 0.8. For these datasets, the classification performance was similar between the NB classifiers created after applying ReliefF and Limma as feature selection methods. Limma is a popular method, among the genomics community, for the selection of differentially expressed genes, but it is not used as a feature selection method by the machine learning community. In contrast, ReliefF is a popular method among machine learning studies but not widely used in genomic studies. Figure ADC vs TAHN-TumorSCC using IPA®. The results of the IPA® core analysis show a significant association between ReliefF-selected genes and the following diseases: cancer (25 out of 27) connective tissue disorder (13 out of 27), dermatological diseases and conditions (13 out of 27). Interestingly, the ReliefF-selected genes (19 out of 27) are associated with either adenocarcinoma (16 genes), squamous-cell carcinoma (4 genes) or carcinoma of the lung (4 genes). The list of genes and their associations can be seen in Table We analyzed the genes found by ReliefF in the classification task of TAHN-TumorThe classification performance for all models is high , and when a mix of tissue was used (TAHN-TumorADC vs. TAHN-TumorSCC). The high AUC results are an indication of the diagnostic potential of this technology.Lung cancer patients could benefit with a potentially novel approach for subtyping. The diagnosis of adenocarcinoma vs. squamous cell carcinoma is routinely accomplished using histology supplemented by immunohistochemistry (TTF-1 and p63/p40). It is therefore not likely that our approach would change this practice, which is well established, quick and inexpensive. Rather, we suggest that the use of epigenomic changes could help in the small number of tumors which remain difficult to classify. However, the primary importance of our work may be in providing additional understanding of the origins of squamous cell and adenocarcinomas, which suggest that these phenotypes are associated with, or perhaps even derived from, different epigenomic phenotypes. Epigenomic alterations, in the form of DNA methylation, prevent the binding of transcription machinery, resulting in gene silencing . MoreoveOur study had some limitations, which include the following: 1) There were a limited number of tumor-adjacent histologically normal tissue samples used. However, the homogeneity of these normal tissues we observed suggests that additional normal tissues would not improve the classifier. 2) The resulting classifiers were not validated in another dataset outside of TCGA lung samples. 3) Each ‘omic’ classifier is independent of one another. In the future, we would like to explore data integration models in a multi-omic approach. 4) The classification problem of discriminating cancer subtypes of adenocarcinoma and squamous cell carcinoma could also be explored in a pan-cancer analysis, to validate the same finding seen in our study of lung cancer subtypes. 5) Due to the challenge of data availability, in this study we did not analyze biopsies with varying percentages of tumor and TAHN tissue (mixed biopsies). Instead, we took relatively ‘pure’ biopsies of either tumor or TAHN to classify between lung cancer subtypes. A future study could consider the molecular classification or discovery of cancer given a mixture of tumor and TAHN tissue. For example, an analysis of ‘omic’ data from cancerous and non-cancerous tumor tissues, as well as TAHN tissue for both types of tumors, might be performed in the same way as presented in this manuscript.In this paper, we addressed the issue of lung cancer subtyping using DNA methylation data from TAHN tissue, which is a novel strategy for classification of non-small cell lung cancer samples. This study demonstrated that using computational Bayesian modeling, it is possible to discover the molecular differences between tumor and tumor-adjacent tissue of lung cancer patients. This discovery will allow clinicians to use the available biopsy material without worrying about its tissue composition, yielding in less invasive diagnostic procedures for the patient. We hope that our results will encourage researchers to also make use of TAHN tissue samples generated in their laboratories for predictive modeling and make this data available for public use. As more data becomes available, our models can be further improved, and future discoveries could be made in other cancers.https://tcga-data.nci.nih.gov/tcga/) in datasets LUAD and LUSC; and also from the Gene Expression Omnibus (http://www.ncbi.nlm.nih.gov/geo/), accession number GDS3257. The formatted datasets used in this study, along with sample IDs, are provided in Additional file ADC vs. TumorADC in gene expression), Additional file SCC vs. TumorSCC in gene expression), and Additional file ADC vs. TumorADC in methylation). The annotations from TCGA to identify these samples are provided in Additional file The datasets used in this study are publicly available from The Cancer Genome Atlas (


We benchmarked results relative to sets of DEGs identified through either quantitative RT-PCR or microarray. We observed that edgeR performs slightly better than DESeq and Cuffdiff2 in terms of the ability to uncover true positives. Overall, DESeq or taking the intersection of DEGs from two or more tools is recommended if the number of false positives is a major concern in the study. In other circumstances, edgeR is slightly preferable for differential expression analysis at the expense of potentially introducing more false positives.Recent advances in next-generation sequencing technology allow high-throughput cDNA sequencing (RNA-Seq) to be widely applied in transcriptomic studies, in particular for detecting differentially expressed genes between groups. Many software packages have been developed for the identification of differentially expressed genes (DEGs) between treatment groups based on RNA-Seq data. However, there is a lack of consensus on how to approach an optimal study design and choice of suitable software for the analysis. In this comparative study we evaluate the performance of three of the most frequently used software tools: Cufflinks-Cuffdiff2, DESeq and edgeR. A number of important parameters of RNA-Seq technology were taken into consideration, including the number of replicates, sequencing depth, and balanced  Compared to microarray techniques, RNA-Seq can be performed without prior knowledge of reference sequences and enables a wide range of applications including transcriptome de novo assembly strategies for organisms without reference sequences. The number of mapped reads is calculated based on the outcome of the alignment to estimate the relative expression level of genes and subsequently statistical methods are applied to test the significance of differences between groups. The general workflow for the analysis of differential expression is illustrated in In RNA-Seq experiments, the primary interest of biologists in many studies is differential expression analysis between groups. To quantify gene expression, RNA-Seq reads need to be aligned to the reference genome for model organisms  or to the transcriptome sequences reconstructed using Although initially it was claimed that RNA-Seq applications could produce unbiased, ready-to-analyze gene expression data Several attempts have been made to address the aforementioned difficulties (e.g. ref.16–18). In early RNA-Seq studies lacking biological replicates, the distribution of feature counts across technical replicates was reported to fit well to a Poisson distribution where the variance is equal to the mean A number of tools have been developed that use the NB distribution for differential expression analysis based on RNA-Seq data  vs. unbalanced sequencing depth within and between groups. The results of this study will not only guide the choice of analysis tools, but more importantly will help researchers design optimal and efficient RNA-Seq experiments for their future studies.Here, we evaluate the performance of Cufflinks-Cuffdiff2 as well as two widely used count matrices based tools - DESeq and edgeR, for DEG analysis of RNA-Seq data using both real RNA-Seq datasets and simulated datasets. In addition, we consider some important parameters of RNA-Seq technology that impact experimental design, including the number of biological replicates, sequencing depth, and balanced All animal treatments and experimental procedures were carried out in accordance with the Australian Code of Practice for the Care and Use of Animals for Scientific Purposes. The experimental procedures were approved by The University of Queensland Animal Ethics Committee (QBI/293/13/NHMRC).The performance of Cufflinks-Cuffdiff2 vs. uhr.MicroArray Quality Control Project (MAQC) data in vitro experiments conducted in this study. All mice were housed in groups and maintained on a 12-hour light/dark cycle with ad libitum access to food and water. After mice were sacrificed by cervical dislocation, the neural precursor activity was examined as described in detail in a previous study The K_N RNA-Seq dataset was generated in-house at Queensland Brain Institute from mouse neurosphere cells treated by potassium chloride (KCl) and norepinephrine. Briefly, adult (8–12 week old) male C57BL/6J mice were used for all In order to investigate the impact of sequencing depth on the performance of three software tools, we randomly subsampled four subsets of data with different sequencing depths from the “K_N_full” dataset. Each subset contains the balanced dataset around 30 M, 20 M, 10 M, 5M for each individual sample, and these subsets are named as “K_N_30M”, “K_N_20M”, “K_N_10M” and “K_N_5M”, respectively. In order to study the effects of unbalanced sequencing depth between treatment groups, another two subsets were generated. One subset includes 30 M of data in the “K” group but 5 M in the “N” group, and this subset is denoted as “unbalanced between groups 1”. The other unbalanced subset includes all data in the “K” group but 20 M in the “N” group and it is denoted as “unbalanced between groups 2”.The lymphoblastoid cell lines (LCL) RNA-Seq datasets were generated by Pickrell et al. In this study, we used samples with more than 8 M reads per individual from those sequenced at Yale to gain enough sequencing depth for each sample, which resulted in 40 samples in total . From these 40 individuals, we randomly selected and assigned N  samples to each of the two hypothetical treatment groups. N is the number of biological replicates in each hypothetical treatment group. We expect no differentially expressed genes between the two hypothetical groups because samples were randomly selected from a same population. This dataset (hereafter referred to as the LCL1 dataset) is used to estimate false positive rates for the three software tools.iδj} where two different groups are indexed by i = 1, 2 and δj follows a two component normal distribution with parameter μ =  and σ = . We employed this simulation design based on the LCL1 dataset. According to the simulated count values, bam files were generated by randomly adding or removing aligned reads using custom PERL scripts and SAMtools Comparisons of methods based on real RNA-Seq data are limited because a complete list of true DEGs and non-DEGs is not known. Therefore, we also used simulated data with known DEGs to estimate the true positive rate and false positive rate. Specifically, we simulated DEGs in two hypothetical groups using a strategy similar to that described in In addition, five randomly subsampled subsets (hereafter we referred as the LCL3 dataset) were generated based on the LCL2 dataset to test performance of each of three software tools using RNA-Seq datasets with different sequencing depths or with unbalanced sized data between/within groups, named as “S1_8M_balanced”, “S2_5M_balanced”, “S3_1M_balanced”, “S4_5M_1M_btw” and “S5_5M_1M_within”. Of them, “S1_8M_balanced”, “S2_5M_balanced” and “S3_1M_balanced” are subsets with around 8 M, 5 M and 1 M number of reads, respectively, for each sample in both conditions. “S4_5M_1M_btw” is the subset with numbers of reads being much higher in one condition (5 M) than that in the other condition (1 M). “S5_5M_1M_within” is the subset that the data size is unbalanced within each condition (i.e. numbers of reads for some of the samples in condition 1 and condition 2 are around 5 M while numbers of reads of other samples are around 1 M). See details in Table S3 in 2FC) of the gene expression value between hbr and uhr was calculated according to Bullard et al.'s study 2FC|>2  are considered as differentially expressed (the positive set). In contrast, genes with |log2FC|<0.2 are considered as the negative set. Among the 1044 genes, 410 genes are in the positive set and 86 genes are in the negative set.The results of quantitative RT-PCR (qRT-PCR) data from NCBI GEO (series ID GSE5350) was used as gold standard to evaluate the performance of three tools based on MAQC RNA-Seq datasets. We downloaded the four technical replicates of the uhr sample from GSM129638 to GSM129641, and the four technical replicates of the brain sample from GSM129642 to GSM129645, with a total of 1044 genes in the list. Among these genes, only those genes with the unique matched Refseq (hg19) gene ID were kept and genes with zero read counts in all samples of qRT-PCR were filtered out. The log2 ratio of fold change  and converted to cDNA using Applause WT-Amp ST kit (NuGEN). Fragmentation and biotin labelling of the cDNA was carried out using Encore Biotin module (NuGEN) for Affymetrix GeneChip Mouse Gene 1.0 ST arrays. Labelled cDNA was hybridised to arrays in an oven at 45°C. Hybridised arrays were scanned using the Affymetrix GeneChip Scanner and the scanned data (.CEL files) was transferred to the Partek Genomics Suite and data analysis was carried out as described in Narayanan et al.'s study Our basic analyses followed the general workflow depicted in The area under the receiver operating characteristic (ROC) curve (AUC) is used to evaluate the performance of different techniques for the detection of DEGs as suggested in The relationship between TPR and FPR is depicted by the ROC. As pointed out by The K_N RNA-Seq dataset has been deposited in the NCBI SRA with accession ID SRX516577. The microarray data for the neurosphere samples has been deposited in the NCBI Gene Expression Omnibus with accession ID GSE57440.https://github.com/Qiongyi/RNA-Seq-comparison/.Analysis pipelines with detailed descriptions for subsampling datasets and generating BAM files based on simulated count values are listed in File S1 in We first investigated the impact of biological and technical replicates on the performance of differential expression analysis for each of three tools. The MAQC RNA-Seq datasets include two technical replicates that were independently generated in two different laboratories. The K_N RNA-Seq dataset as well as its subsets include 4 biological replicates. The LCL RNA-Seq dataset as well as its subsets include 20 biological replicates in each group. All datasets have been processed through the analysis pipeline shown in As expected, analyses from all three software tools with biological replicates or technical replicates outperform those without replicates as judged by both AUC1 and AUC2 . InteresFor the K_N dataset, the incremental improvement became smaller as judged by the AUC as the number of biological replicates increased . HoweverGenerally, although each tool adopts different underlying statistical models to identify DEGs from RNA-Seq data, they all need replicates to separate the real differences between treatment groups from the random variation within groups. In the LCL dataset, when the amount of biological variation in the samples is large, all three tools take advantage of biological replicates to gain better performance. Recent studies To determine how sequencing depth affects the performance of differential expression analysis for the three software tools, we randomly subsampled seven subsets of data with different sequencing depths from the K_N and LCL datasets. The results of experiments from both the K_N and LCL datasets show that Cuffdiff2 is the most sensitive to sequencing depth while the performance of DESeq and edgeR are stable under different sequencing depths . It suggIn order to understand the impact of unbalanced sequencing depth on the performance of each tool for differential expression analysis, we made comparisons based on subsets of data from the K_N and LCL datasets with balanced sequencing depth among all samples, unbalanced sequencing depth within groups and unbalanced sequencing depth between groups. The sequencing depth of the K_N_full dataset is unbalanced within groups, as shown in Table S2 in We observed from both K_N and LCL datasets that DESeq is  most sensitive to unbalanced sequencing depth between groups . AUC1 ofSince budgetary constraints are common with sequencing, optimal experimental designs need to balance the sequencing depth for each sample with the number of replicates for each group. As our results show  and 4 anOur results suggest that more biological replicates are required to identify DEGs using RNA-Seq when studying human cell lines from unrelated individuals in the same ethnic group than equivalent studies based on neurosphere cells from inbred mice. This suggests that the amount of biological variation in the samples to be sequenced should be a major consideration when determining the optimal number of biological replicates required for a given RNA-Seq study. It is likely that more biological replicates will be required in animal/human tissue samples compared to human cell lines or cells from inbred lab strain models. However, more gold standard datasets for benchmarking and more comprehensive evaluations based on these datasets are required to guide future RNA-Seq study designs in terms of the optimal number of biological replicates relative to the amount of biological variation in the samples.For all three datasets, we find that edgeR performs slightly better than the other two tools as judged by the two AUC statistics . A recenIn this study, although most of the DEGs identified by each of three tools overlapped, edgeR always detected more DEGs than the other tools, with a number of DEGs identified exclusively by edgeR . HoweverIt is interesting to note that, although both DESeq and edgeR are negative binomial distribution model-based algorithms using the same count matrices as input, many genes with high fold change were picked up by edgeR but not by DESeq  for differential expression analysis of RNA-Seq data while considering a number of important parameters of RNA-Seq experiments, including number of replicates, sequencing depth, and the unbalanced data within or between groups. By using the results from qRT-PCR and microarrays as benchmarks, we observed that edgeR performs better than DESeq and Cuffdiff2 in terms of the ability to uncover true positives with the default FDR setting (FDR<0.05). However, this reflects that edgeR could always detect more DEGs than the other two tools , which may also introduce more false positives. All three tools perform much better when there are biological or technical replicates available. Consistent with previous studies it suggests that biological replicates are a key factor for differential expression analysis in RNA-Seq datasets Our results show that Cuffdiff2 is most sensitive and DESeq is least sensitive to sequencing depth, but the overall impact of sequencing depth is not as critical as the number of biological replicates, which is in agreement with previous studies No single method is clearly superior for differential expression analysis, since each has particular strengths that may be suitable for specific RNA-Seq datasets. Considering the overall performance based on three independent datasets in this study, Cuffdiff2 is not recommended for differential expression analysis at gene-level resolution, particularly if sequencing depth is low . DESeq or the intersection of DEGs from two or more tools is recommended if the number of false positives is a major concern in the study. DESeq is not recommended if sequencing depth is unbalanced between groups, although down-sampling of the libraries to a common size before processing with DESeq may ameliorate this problem - an approach used by NBPSeq Supporting Information S1vs. edgeR (top panel), DESeq vs. Cuffdiff2 (middle panel) and edgeR vs. Cuffdiff2 (bottom panel). File S1, Analysis pipelines, methods and examples of commands for differential expression analysis, subsampling fastq files and generating SAM/BAM files based on simulated count values. File S2, The raw count values for genes with high fold changes were picked up by edgeR but not by DESeq. Genes with high fold changes  identified as DEGs by edgeR but not by DESeq are listed in the file. The gene ID, the log2 fold changes (logFC) and FDR from DESeq, the logFC and FDR from edgeR, the raw count values for the four replicates of sample K (K1–K4) and sample N (N1–N4) are shown in each of the columns. Table S1, Numbers of reads for the human hbr and uhr samples from the MAQC dataset. Table S2, Numbers of reads for the mouse neurosphere samples for treatment groups of K and N (the K_N dataset). Table S3, The number of reads for each individual sample of the LCL3 dataset. Table S4, The definition for TP, FP, TN, FN, TPR and FPR. Table S5, The false positive rate for Cuffdiff2, DESeq and edgeR based on the LCL1 dataset.Figure S1, Venn diagram showing the number of differentially expressed genes identified by two versions of Cuffdiff2. Figure S2, The effects of biological replicates on the differential expression analysis for Cuffdiff v2.0.2. Figure S3, The detected fold changes of all the differentially expressed genes identified by three tools were compared and shown, including DESeq (ZIP)Click here for additional data file.


Meta-analysis has become a popular approach for high-throughput genomic data analysis because it often can significantly increase power to detect biological signals or patterns in datasets. However, when using public-available databases for meta-analysis, duplication of samples is an often encountered problem, especially for gene expression data. Not removing duplicates could lead false positive finding, misleading clustering pattern or model over-fitting issue, etc in the subsequent data analysis.We developed a Bioconductor package Dupchecker that efficiently identifies duplicated samples by generating MD5 fingerprints for raw data. A real data example was demonstrated to show the usage and output of the package.Researchers may not pay enough attention to checking and removing duplicated samples, and then data contamination could make the results or conclusions from meta-analysis questionable. We suggest applying DupChecker to examine all gene expression data sets before any data analysis step.The online version of this article (doi:10.1186/1471-2105-15-323) contains supplementary material, which is available to authorized users. Publicly available high-throughput genomic data, especially gene expression data, have greatly changed the way genomic research is conducted recently. The major online databases such as the Gene Expression Omnibus (GEO)  ArrayExpIt is very easy to ignore removal of the duplicated gene expression samples in curated high-throughput data, and the consequences could be obtaining false positive findings or misleading cluster patterns, etc. If the duplicated samples were in both training and testing cohorts for gene signature validation study, it would lead over-fitting of the classifier.Nevertheless, the identification of duplicated samples could be complicated and labor intensive. For microarray gene expression data, if the preprocessing, normalization and transformation procedures used are different, the normalized datasets for duplicated samples may not be identical. We developed a bioconductor package DupChecker that can efficiently check sample redundancy based on the raw data files of high-throughput genomic data.The method that we implemented in DupChecker is to examine MD5 hash for each raw data file. MD5 is a message-digest algorithm that can be utilized to check data integrity by producing 128-bit fingerprint of the data input . The duplicated gene expression samples can be identified by checking to see if they have identical MD5 fingerprints.geoDownload and arrayExpressDownload to download multiple gene expression data sets from GEO or ArrayExpress databases and deposit the files under the specified directory. The functions buildFileTable and validateFile will go through each raw data files under the directory to calculate MD5 fingerprint and return a table listing all duplicated samples.For users’ convenience, we also developed the functions validateFile function, we found there were 64, 231 and 167 duplicated samples in each data set compared with the other two data sets. Table We applied the DupChecker package to three colon cancer Affymetrix gene expression data sets - GSE13067, GSE14333 and GSE17538 from GEO with 74, 290 and 244 samples respectively. Both GSE13067 and GSE14333 were contributed from the same laboratory in Australia, and GSE17538 was from an institute in the US. The raw data in Affymetrix CEL file format were needed for DupChecker analysis. From the final summary table generated by the The DupChecker package is computationally efficient. After downloading CEL files for these three data sets, it took less than one minute to calculate MD5 fingerprints for all 608 files and to deliver a summary table using a 2.7GHz Intel Core i7 CPU.We also tested DupChecker package on 2 ArrayExpres and 22 GEO breast cancer datasets containing 5203 affymetrix CEL files. Among those files, 696 CEL files were actually duplicated from 348 CEL files. The full summary table and the R code can be found in Additional files Gene expression meta-analysis has become increasingly popular for high-throughput genomic data analysis. Due to the large amount of publicly available gene expression data contributed by different researchers, it is almost inevitable to include duplicated samples in the data sets collected for meta-analysis. In the example we showed in Section 3, among the three GEO data sets, there were 463 out of 608 samples with at least one duplicate. It was difficult to tell by IDs since all duplicate samples had different GEO accession numbers. It is effective to identify duplications using MD5 fingerprint generated from raw data since it would not involve any gene expression data processing procedures.Researchers may not pay enough attention to checking and removing duplicated samples, and then data contamination could make the results or conclusions from meta-analysis questionable. We suggest applying DupChecker to examine all gene expression data sets before any data analysis step. We also want to point out that the data files must be identical to be identified as duplicates. Specimens or RNA samples profiled twice, whether on the sample platform or different platforms, will not be identified using DupChecker.In this application note, we illustrated the application using gene expression data, but DupChecker package can also be applied to other types of high-throughput genomic data including next-generation sequencing data.Project name: DupCheckerProject home page: http://www.bioconductor.org/packages/devel/bioc/html/DupChecker.htmlProgramming languages: ROperating system(s): Platform independentOther requirement: Bioconductor 3.0 or higherLicense: GPLThe full result table generated by DupChecker for the colon cancer data.(CSV 16 KB)Additional file 1:The full result table generated by DupChecker for the breast cancer data.(CSV 28 KB)Additional file 2:The R code for breast cancer example.(PDF 24 KB)Additional file 3:


The advent of high throughput RNA-seq at the single-cell level has opened up new opportunities to elucidate the heterogeneity of gene expression. One of the most widespread applications of RNA-seq is to identify genes which are differentially expressed between two experimental conditions.3E), a novel algorithm specifically designed for single-cell RNA-seq data. We use synthetic data to evaluate D3E, demonstrating that it can detect changes in expression, even when the mean level remains unchanged. Since D3E is based on an analytically tractable stochastic model, it provides additional biological insights by quantifying biologically meaningful properties, such as the average burst size and frequency. We use D3E to investigate experimental data, and with the help of the underlying model, we directly test hypotheses about the driving mechanism behind changes in gene expression.We present a discrete, distributional method for differential gene expression  contains supplementary material, which is available to authorized users. Over the last two decades, several methods for global quantitative profiling of gene expression have been developed , 27, 36.5 cells. Consequently, only information about the mean expression of each gene in a sample could be extracted. However, it has been known since the 1950s , where X is a sample of read counts.for an i’th exponential moment: α, β and γ: The parameters of a Poisson-Beta distribution can also be estimated by a Bayesian inference method . The Baywhere x, was drawn from a Poisson-Beta distribution: The number of read counts, Parameter estimation was performed by a collapsed Gibbs sampler, using Slice sampling . ConditiThe likelihood ratio test provides a parametric test for differential expression. One of the two conditions is designated as the control and the other is designated as the test conditions. For each gene, the log-likelihood of the data from the test condition is calculated using the parameters estimated for both the control and the test. We then test the null hypothesis that the ratio of the likelihoods calculated using the two parameters is not significantly different from 1.N randomly generated samples. N can be set by the user as a flag and the default value is 1,000.Calculating the likelihood using the Poisson-Beta distribution numerically is very challenging due to the presence of the confluent hypergeometric function . The metc from Beta distribution with parameters α and β: c∼Beta and then drawing from a Poisson distribution with parameter λ=cγ: x∼Poisson(cγ).Synthetic data was produced by sampling from a Poisson-Beta distribution, i.e. first drawing an auxiliary variable 3E is capable of detecting changes in different regimes of the parameter space, we systematically varied the three parameters of the Poisson-Beta model across the range of values representative of the biological data, α∈,β∈, and γ∈. We fixed a pair of parameters and varied the third in 10 steps over its range, recording the p-value for one of the tests. For each of the 100 different combinations, it was assumed that the sample consisted of 50 cells from each condition was generated. Close to the diagonal, the changes in the parameters are small, and we expect a high p-value in these positions. To summarize the matrix of p-values, we calculate a composite score as To evaluate how well DI is the indicator function. The score S ranges between 0 and 1, and 1 is obtained when all of the p-values on the diagonal are greater than.05 while the off-diagonal elements are smaller than.05. The score is mapped to a color and reported in Fig. where μ is the mean expression and b is a parameter [To evaluate the effect of experimental noise, we considered the possibility of transcript dropout events. Dropouts are likely to occur either as a consequence of failure to isolate the transcript when lyzing the cell, failure of the RT reaction, or failure of the amplification. The probability of failure is poorly understood, but it has been suggested that the dropout probability can be modeled as arameter .b=10,100,200.For our dropout experiments, we randomly generated 1,000 parameter triplets based on the fitted data from Islam et al. . For eacxij represent the raw number of reads for i=1,2..N and j=1,2..M, where N is the number of genes, and M is the total number of cells in the experiment. Then, the size factor sj is found as The normalization of the raw read counts was performed by the same method used by DESeq2 . Let xijThe corrected read counts are then calculated as p-value threshold for D3E, we first take the sample which will be used as the control group , and split it into two non-overlapping subsets. Next, one of the tests is applied to the split sample, and the lowest p-value observed, p∗, is recorded. When comparing the case and the control sets, 0.1∗p∗ is used as a threshold, and only genes with a p-value lower than 0.1∗p∗ are considered significant.To determine the z-score which we transform to a p-value using the formula p=2Φ(−|z|), where Φ(x) is the cumulative density of the standard normal distribution. When choosing the threshold for SCDE, we used the same strategy as for D3E.SCDE reports a p-value reported by the algorithm, and we required it to be <.1 to be significant.For DESeq2 we used the adjusted S(ω), of the mRNAs for the transcriptional bursting model is given by [The power spectral density, given by  (23)\dod=α/(α+β). By definition, the auto-correlation, R(t), is given by the inverse Fourier transform of S(ω), where τc=S(0)/2R(0).The characteristic time of the auto-correlation is defined as


RNA-Sequencing (RNA-Seq) provides valuable information for characterizing the molecular nature of the cells, in particular, identification of differentially expressed transcripts on a genome-wide scale. Unfortunately, cost and limited specimen availability often lead to studies with small sample sizes, and hypothesis testing on differential expression between classes with a small number of samples is generally limited. The problem is especially challenging when only one sample per each class exists. In this case, only a few methods among many that have been developed are applicable for identifying differentially expressed transcripts. Thus, the aim of this study was to develop a method able to accurately test differential expression with a limited number of samples, in particular non-replicated samples. We propose a local-pooled-error method for RNA-Seq data (LPEseq) to account for non-replicated samples in the analysis of differential expression. Our LPEseq method extends the existing LPE method, which was proposed for microarray data, to allow examination of non-replicated RNA-Seq experiments. We demonstrated the validity of the LPEseq method using both real and simulated datasets. By comparing the results obtained using the LPEseq method with those obtained from other methods, we found that the LPEseq method outperformed the others for non-replicated datasets, and showed a similar performance with replicated samples; LPEseq consistently showed high true discovery rate while not increasing the rate of false positives regardless of the number of samples. Our proposed LPEseq method can be effectively used to conduct differential expression analysis as a preliminary design step or for investigation of a rare specimen, for which a limited number of samples is available. High-throughput sequencing of cDNA derived from an RNA sample, known as RNA-Seq, has recently been developed and applied to various studies depending on the scientific interests such as detecting fusion genes, transcribed single nucleotide polymorphisms (SNPs), and differential expression  , 2. In pThe main purpose of DE analysis is to identify transcripts that changed significantly in abundance across experimental conditions. This goal has been achieved using different statistical methods for data from array-based technologies. Compared to microarray, however, RNA-Seq has different characteristics such as high dynamic range and low background expression level. In order to address those properties, various methods have been proposed using Poisson and negative binomial distributions –5.The software edgeR assumes that the mean and variance are related with a single proportionality constant that is the same throughout the experiments; thus, only one parameter needs to be estimated for each transcript . InsteadComprehensive reviews on pre-existing methods for DE analysis with both real and simulation datasets has been recently published. By examining 11 methods including edgeR , DESeq , and NBPDespite the decreasing sequencing costs, RNA-Seq experiments remain expensive to allow extensive biological replications. Moreover, limited specimen availability often leads to studies with a small number of (sometimes none of) replicates. Unfortunately, many of the existing methods have been largely unsuccessful in DE analysis with a small number of samples, and few researches have correctly addressed the problem arisen from non-replicated samples per each class. Therefore, there remains the need for an effective method that could be applicable to DE analysis with small samples or with one sample per each class.local-pooled-error with RNA-Sequencing data (LPEseq), which proposes to analyze DE detection of RNA-Seq experiments with a small number of or non-replicated samples in each class. Based on the LPE method  usually assume a well-behaving distribution, and test if the target extreme observation is an outlier of that distribution. It is obvious that these tests largely depend on the type of observations and distribution assumed .Calculate the mean intensity of transcript Calculate quantiles (percentiles by default) with the mean intensities of the whole transcripts evaluated at step I, and define “intensity bins” using adjacent quantile values. Thus the width of the bin depends on these quantile values and each bins have different widths but the same number of transcripts. Note that 100 bins are generated by default and the number of bins can be adjusted manually.Place all the transcripts in the bin to which their mean intensities belong.Label the transcript as an outlier if the difference between classes is larger than the threshold : Sultan et al.  performeet al.  of the estimates for the dispersion parameter Case IV. Different number of replicates per each class: : We repeated the analyses of case I–III with different numbers of replicates per each class, focusing on small sample analysis.All the cases were repeated 100 times.http://bibs.snu.ac.kr/software/LPEseq) or Bioconductor. LPEseq uses any kind of count table format as the input file. The package was designed to perform the analysis with only a single command, allowing researchers who are unfamiliar with the R language to use it easily. All the analyses in this work were performed with the package LPEseq. In addition, the whole analysis code used in this work and the LPEseq manual can be found in our website.We implemented the method as a package for the R environment and named it “LPEseq.” It is freely downloadable from our website .We investigated the performance of LPEseq regarding the true DE detection ahead of false discovery under 72 combinatorial cases of four different parameters, The primary purpose of developing LPEseq was to apply it to data with non-replicated samples in each class. In such a case, among the methods under consideration, edgeR and DESeq are applicable. Thus, we compared the results obtained from LPEseq, edgeR and DESeq. The overall performances of the three methods with one replicate in each class are reported in In this study, we sought to establish a method applicable to a small number of replicates, not just one in each class. Here, we considered testing between two classes in RNA-Seq experiments assuming that transcripts (or genes) are independent from each other, which might be unlikely in many real datasets. We also have investigated a situation including correlated transcripts with simulated datasets and the results did not differ much in DE analysis . Since tWe used LPEseq to analyze four different RNA-Seq data sets: non-replicates (Hammer’s dataset), small replicates (Sultan’s dataset), large replicates (MAQC dataset), and large normal sample with unequal group sizes (Montgomery’s dataset). The primary purpose of the analysis was to discover DE transcripts and to compare the results obtained using competing methods.et al. reported that about 2,000 transcripts were DE and, among them, validated 755 transcripts using quantitative polymerase chain reaction (qPCR) [et al., while edgeR and DESeq detected 1,305 and 74 transcripts as DE, respectively  . Interesectively .As shown in The Sultan’s dataset consists of two different cell types, a kidney and a B-cell, and it would be expected that for such a comparison a large number of genes are differentially expressed. As can been seen in To observe characteristics of transcripts uniquely found by each method, we plotted the group mean differences in raw counts using DE transcripts found in each dataset  and the similar observations were made. Except edgeR, most methods identified uniquely found by each method and those uniquely identified genes with very small mean difference between conditions showed no enriched term by GSA. Only genes identified by LPEseq were enriched in transcriptional regulation function See , which iTo make inferences based upon large as well as small samples through the LPEseq, we analyzed Montgomery’s dataset , which cWe proposed a method for DE analysis with a small number of replicates, especially when a single replicate in each class is available. By extending the LPE method, the proposed method is applicable to the RNA-Seq experiments either with or without replicates per each class. Even though the proposed LPEseq followed the idea of the original LPE method, it is different from that in two aspects: (i) estimating local-pooled variance from different classes assuming them as replicates and (ii) removing outliers derived from the replicates assumption between classes. These two differences make DE analyses with non-replicated datasets feasible. By adding an auxiliary step, our proposed method performed more robustly compared to existing methods regardless of the number of replicates in each class, even when only one replicate in each class was available.In general, the performance of DE test for RNA-Seq data can be influenced by sequencing depth and the number of replications. By increasing the sequencing depth, it is expected that the accuracy of expression counts will increase. With higher accuracy, we expect that LPEseq could better estimate the variances, resulting in improved performance. However, it was reported that the number of sample replicates is a more significant factor in accurate identification of DE genes in comparison to the sequencing depth , 26. A fWe considered the reproducibility score which represents how much the result made with replicated data is reproduced by the analysis with non-replicated or a smaller number of replicated data in our model development. Using this score, the threshold value for detecting outliers was suggested. By comparing the reproducibility of five methods, LPEseq showed the largest number of DEGs and the highest overlaps between the analyses with subset of samples and total samples, indicating the robust reproducibility regardless of the number of samples  Figs.It is worth noting that, even though LPEseq takes the reproducibility into account to the statistical testing without replicates, the scope of any conclusions drawn from it may be limited and need to be interpreted with extra cautions. For example, the outliers flagged in LPEseq may not really be outliers. They can be treated as outliers due to the mean difference between two groups, and this might affect the DE result. We have investigated this effect with MAQC dataset as follows. For MAQC data, there are 7 replicates in each condition. We analyzed all possible combinations of choosing 2 replicates from 7 in the same condition. We performed DE tests as if the two replicates had been obtained from different conditions. The average number of DE found by LPEseq was 5.095 (SD was 5.991) from 52580 transcripts. This analysis shows that the LPEseq method controls the false discovery rate when there are no outliers. Through the analysis with non-replicated data, we believe that the proposed method can be useful to obtain results that are comparable with those obtained from data containing replicates.Note that the log-transformed count data such as negative binomial data would be an approximation to normal distributed data . AlthougThe proposed idea can be easily extended to more than two comparison problems. For example, more complicated models such as analysis of variance (ANOVA), the “local-pooled-error” and replicates assumption between classes can be applied to the ANOVA problems, with the exceptions that the residual mean sum of squares and residual degrees of freedom are evaluated using observations without outliers in a local bin. The method can also be applicable to test differential expression in the context of high-throughput experiments such as measuring protein expression or DNA methylation.S1 FigSix different datasets were used to suggest optimal threshold value used in LPEseq method. Reproducibility score versus D values is plotted in blue line with 95% confidence interval colored in red. The D value giving the highest reproducibility score is shown in the center of each plot. The key characteristics of the data appear below each plot.(TIF)Click here for additional data file.S2 FigThe number of DEGs is plotted with different number of bins (from 50 to 150 bins).(TIF)Click here for additional data file.S3 FigThe effect of correlated genes in DE analysis with LPEseq is shown in boxplot for FDR (left) and TPR (right). The different proportions of correlated genes (blue) and the difference correlation coefficient between correlated genes (pink) were denoted in each plot. The analysis was repeated 100 times.(TIF)Click here for additional data file.S4 FigVenn diagram of DEGs is shown for MAQC dataset. Five different methods, i.e., LPEseq (brown), edgeR (sky blue), DESeq (green), DESeq2 (violet) and NBPSeq (red) were used. A density plot of the mean difference between classes of uniquely found DE transcripts in each method was indicated. X- and Y-axis represent group mean difference and density. The number in parentheses indicates the total number of DE transcripts found. The criterion used to call DE was Benjamini-Hochberg corrected p-value less than 0.05 for all methods. The enriched terms gene set analysis was performed by DAVID web-tool.(TIF)Click here for additional data file.S5 FigThe overlapped proportion of DEGs with varying number of technical replicates (left) and biological replicates (right) are shown. The overlap proportion indicates the number of DEGs identified both with subset of samples and with total samples divided by the number of DEGs identified with total samples.(TIF)Click here for additional data file.S6 FigThe plot shows the variance curve estimated with different numbers of samples. The X-axis represents log-transformed intensity and the Y-axis does the variance estimates. The solid blue line indicates the ‘true’ variance curve  and all other dashed lines the variance curve estimates using different numbers of samples. None of p-values by a two-sample KS test using the solid blue line and the dashed grey lines were less than 0.05.(TIF)Click here for additional data file.S1 Table(DOCX)Click here for additional data file.S1 FileA file describing how optimal threshold value ‘D’ was defined and evaluated for LPEseq.(DOCX)Click here for additional data file.S2 File(XLSX)Click here for additional data file.


Analysis of complex metatranscriptome datasets of these communities poses a considerable bioinformatic challenge since they are non-restricted with a varying number of participating strains and species. For RNA-Seq data a standard approach is to align the generated reads to a set of closely related reference genomes. This only works well for microbial communities for which a near complete catalogue of reference genomes is available at a small evolutionary distance. In this study, we focus on the design of a validated de novo assembly metatranscriptome pipeline combined rRNA removal, IDBA-UD assembler, functional annotation and taxonomic classification. Different assemblers were tested and validated using RNA-Seq data from an in silico generated mock community and in vivo RNA-Seq data from a restricted microbial community taken from a mouse model colonized with Altered Schaedler Flora (ASF). Precision and recall of resulting gene expression, functional and taxonomic profiles were compared to those obtained with a standard alignment method. The validated pipeline was subsequently used to generate expression profiles from non-restricted cecal communities of four C57BL/6J mice fed on a high-fat high-protein diet spiked with an RNA-Seq data set from a well-characterized human sample. The spike in control was used to estimate precision and recall at assembly, functional and taxonomic level of non-restricted communities.The here developed de novo assembly pipeline for metatranscriptome data analysis was designed for microbial ecosystems, which can be applied for microbial metatranscriptome analysis in any chosen niche.A generic  High throughput metagenomics have revolutionized our knowledge of microbial communities such as those that populate the human gastrointestinal (GI) tract. Complementing 16S ribosomal RNA gene-based compositional analyses, metagenome sequencing of these communities provided a broad description of the genetic content and relative abundance of individual members . Metagende novo assembly method can be used. De novo assembly of RNA-Seq reads into contigs increases the information content and therefore grants a more reliable annotation of the expressed genetic content of an unknown microbial community [de novo assembly of Illumina sequencing data [de novo metagenome assemblies’ will not reliably work for RNA-Seq derived contigs. Consequently additional verification strategies, such as PCR, are necessary as confirmation of the genetic context predicted by assembled contigs.To overcome these problems a ommunity ,17. Subsing data ,19. Mosting data ,17. Sequde novo assembly pipeline for single-end Illumina sequencing using existing assemblers. The method provides better gene assignment results then alignment methods, and we evaluated sensitivity, reliability and validity of the method for the function analysis of complex metatranscriptome data. The generic de novo assembly method developed was validated by PCR and using metatranscriptome datasets of community-restricted samples and a spike-in of a human control sample, and enabled a reliable functional profiling and taxonomic binning of samples obtained from the mouse GI tract.The microbiome of the GI tract of healthy human individuals fulfils a variety of beneficial functions for human health . NumerouA generalized metatranscriptome assembly and analysis pipeline was designed . Brieflyin silico generated mock community composed of eight strains, was used for a primary evaluation of the entire workflow. The second RNA-Seq dataset used to further validate the workflow resulted from an in vivo study of a restricted intestinal microbial community of limited complexity obtained by the colonization of inbred non-obese diabetic (NOD) mice with Altered Schaedler Flora (ASF) [de novo assemblies obtained from this complex non-restricted mouse community, RNA-Seq data from a well-characterized sample from the human small intestine [In order to validate the proposed pipeline, two single-end RNA-Seq metatranscriptome datasets were used. The first RNA-Seq dataset, obtained from an ra (ASF) . The datra (ASF) . To estintestine  was usedin silico mock metatranscriptome was built by merging RNA-Seq data of eight single species transcriptome profiling experiments downloaded from public repositories  for the mock community as a whole as well as for each of the individual members  , and appEven though there were many contaminating reads in these samples that could interfere with the assembly process only a few false positive reads assignments were detected. The precision and recall for functional annotation was 0.97 and 0.58 respectively. The precision was therefore comparable to what was observed for the mock community, whereas the recall is much lower, which is probably due to a much lower sequencing depth of ASF samples .de novo assembled sequences. For a taxonomic classification a two-step procedure was developed. First, from the set of de novo assembled sequences from the ASF microbiome a subset of 103 proteins derived from essential single copy genes (ESCG) [Additionally we explored the performance of a taxonomic classification of s (ESCG)  was idens (ESCG)    . Relativ family) .de novo assembly of 24077 contigs and allowed for the prediction of 36012 partial and full length ORFs within these contigs. Of these ORFs, 25897 were exclusively derived from the mouse datasets, whereas 9707 were assembled exclusively from the human dataset. A total of 407 hybrid ORFs were assembled consisting of reads obtained from both data sources. Virtually identical results were obtained for the mouse-derived RNA-Seq data when the mouse reads were assembled separately (results not shown). This suggests that a substantial increase in microbial complexity essentially did not influence the outcome of the mouse assembly. Of the RNA-Seq mRNA reads derived from the human small intestine sample, 85% was captured in an assembly whereas this fraction was 61–71% for the mRNA reads derived from the four individual mouse cecum samples. The fraction of the reads that was assigned to an ORF was 80% and 45–55% for the human small intestine sample and mouse cecum samples, respectively   was usedAssemblies were performed using the assemblers’ default setting –52. WhenS1 FileAssembler Performance: Testing of various assemblers and output performance; Filtering: Ribosomal RNA, adapter and quality filtering results. Mock_TNF_hybrids: Number of shared assemblies and tetra nucleotide frequency occurrence regression coefficient between mock members. Coverage_Assembly: Distribution of assembled proteins based on transcriptome coverage. Mock_assembly ouput: Precision and recall for functional assignments. NOD_mouse_taxonomy_output: Precision and recall for taxonomy assignments(XLSX)Click here for additional data file.S2 File(XLSX)Click here for additional data file.S3 FileTaxonomy profiles for all mouse and human samples using blastx, megablast and assembly strategies.(PNG)Click here for additional data file.S4 File(PNG)Click here for additional data file.S5 File(PDF)Click here for additional data file.