Drosophila Genetic Reference Panel in order to quantify differences in gene expression among single flies. One of our experimental goals was to identify the optimal analysis approach for the detection of differential gene expression among the factors we varied in the experiment: genotype, environment, sex, and their interactions. Here we evaluate three different filtering strategies, eight normalization methods, and two statistical approaches using our data set. We assessed differential gene expression among factors and performed a statistical power analysis using the eight biological replicates per genotype, environment, and sex in our data set.A generally accepted approach to the analysis of RNA-Seq read count data does not yet exist. We sequenced the mRNA of 726 individuals from the We found that the most critical considerations for the analysis of RNA-Seq read count data were the normalization method, underlying data distribution assumption, and numbers of biological replicates, an observation consistent with previous RNA-Seq and microarray analysis comparisons. Some common normalization methods, such as Total Count, Quantile, and RPKM normalization, did not align the data across samples. Furthermore, analyses using the Median, Quantile, and Trimmed Mean of M-values normalization methods were sensitive to the removal of low-expressed genes from the data set. Although it is robust in many types of analysis, the normal data distribution assumption produced results vastly different than the negative binomial distribution. In addition, at least three biological replicates per condition were required in order to have sufficient statistical power to detect expression differences among the three-way interaction of genotype, environment, and sex.DESeq method and apply a generalized linear model assuming a negative binomial distribution using either edgeR or DESeq software. Genes having very low read counts were removed after normalizing the data and fitting it to the negative binomial distribution. We describe the results of this evaluation and include recommended analysis strategies for RNA-Seq read count data.The best analysis approach to our data was to normalize the read counts using the The online version of this article (doi:10.1186/s12864-015-2353-z) contains supplementary material, which is available to authorized users. The genotype of each fly was assigned to the DGRP line having an Rij = 1. We used the 5 % level of the distribution of rij across genotypes, 0.10, as the threshold for the number of acceptable mismatched SNPs. We included all fly sequences that could be assigned to their expected DGRP line in subsequent analyses.We used the raw sequence data to verify the sequence pool index, genotype, and sex labeling of each fly, to define a threshold for low (undetectable) gene expression, and to assess the technical variance between library preparations. We searched the raw sequence data for all 24 indices used in the experiment in order to confirm the expected index and to identify any contaminating indices. We retained all samples with 95 % or greater of the expected index in the analysis. The DGRP lines are fully sequenced , 35; thuDrosophila gene expression [Some genes present on the Y chromosome have duplicates in other regions of the genome, and the Y chromosome is gene-poor; thus, mapping sequences to the Y chromosome is not a reliable indicator of sex. We used the well-known high levels of sex dimorphism in pression –46 to veth percentile of the intergenic distribution as the low threshold level of gene expression [We defined a gene expression threshold based on a comparison of the distribution of read counts in annotated gene regions (Flybase annotation 5.57)  to read pression . We remoxi. ≠ 0. xik and xir are the read counts for gene i of sample k and the reference sample respectively, while Nk and Nr are the sample sizes of sample k and a reference sample respectively. We used the default reference sample, which is the sample having the upper quartile most similar to the mean upper quartile across all samples. The genes in the middle 40 % of the M-values and the middle 90 % of the A-values are recorded. The list of genes overlapping in these two groups of genes is designated as G*. Note that the user may choose M-value and A-value percentages other than 40 % and 90 %, respectively; these are the default percentages. In the second step, a normalization factor is calculated as the weighted mean of M-values for each sample, which is:w is the weight calculated as the inverse of approximate asymptotic variance as given by the following expression:We examined the impact that read count normalization methods have on the identification of differentially expressed genes. We considered the effect of seven popular normalization methods used in RNA-sequencing data analysis as well as the un-normalized number of reads that mapped uniquely to each gene. We applied total count normalization (TC) , 21, uppDESeq program, we also divided the normalized read counts by the mean of the normalized library size [To obtain the TMM-normalized read counts when using the ary size .Like the TMM normalization, DESeq normalization requires a reference sample to calculate the scaling factor for normalization . DESeq cRUVSeq R package [In addition to the seven normalization methods applied above, we also considered a recently published normalization method called remove unwanted variation with negative control genes (RUVg) . RUVg no package  to normaNB) distribution, [i in sample k can be modeled byμik is the mean, and σik2 is the variance. The mean isi determines the extent to which the variance exceeds the mean. We used the DESeq and edgeR packages to estimate the dispersion parameter [We modeled the count data as both a negative binomial distribution and as a normal distribution. To model the count data as a negative binomial  for each gene S is sex, G represents the DGRP genotype, and E is the environmental condition. To test the significance of all factors in the model, we fitted the following series of models:where Model 1 as the full model, and calculated the likelihood ratio between Model 1 and Model 1 with each of the main effects removed in turn, which we term the reduced Model 1. The likelihood ratio statistic comparing these two models is simply the difference between the deviances of the full model and the reduced modelTo test each term of the main effects, we used G × E, G × S, and E × S, we used the same approach; we added each term to be tested in turn, defining it as the full model, and compared it to the previous reduced version of the model. For example, Model 2(b) and 2(a) were used to find genes with a significant G × S interaction; Model 2(b) was the full model, while Model 2(a) was the reduced model. To test the significance of the three-way interaction term S × G × E, we used the same approach, where Model 2 was the reduced model. Inspection of the Model 2, 2(a), and 2(b) equations above suggests that differential expression detected for each first-order interaction term is dependent upon its ordering in the equation. We therefore compared this analysis with two other ways of detecting differential gene expression for first-order interaction terms. In the second approach, we used the Model 1 as the reduced model and then added each first-order interaction term in turn to Model 1 to test the significance of each first-order interaction term. In the third approach, we assessed the contribution of each first-order interaction term by using Model 2 as the full model and Model 2 without each of the first-order interaction terms in turn as the reduced model.To test the two-way interaction terms SAS (version 9.3) [In addition to using the GLM with negative binomial distribution to model the count data, we also evaluated the ln-transformation of the normalized count data combined with analysis of variance (ANOVA), which we called the ln&ANOVA method. We ln-transformed the normalized read counts and then fitted the ANOVA model below using ion 9.3) :\documenS, G, and E are as defined above, and β0 is the intercept, while ε is error.where P-values obtained from the analysis. Genes having P-values with an FDR threshold of < 0.05 were designated as differentially expressed . Given an ANOVA model with three fixed factors [αβγ)ijk is the difference between the condition mean and the value that would be expected if main effects and two-way interaction terms are sufficient to account for all factor effects. By introducing a new parameter μijk refers to the mean of the three-way interaction condition for the first factor at the ith level, the second factor at the jth level and the third factor at the kth level. For our data, μijk is the mean of ln-transformed normalized counts under the condition of ith genotype, jth environmental condition and kth sex; D is called the fold-change. Hence we can calculate a conservative power estimate using the ln-transformed normalized counts, the desired significance level, sample size (1–8 flies), and variance σ2 (as estimated by the mean sum of squares).For a fixed-effect multi-factor ANOVA model, the test statistic has an pothesis . The tesis false . Thus, t factors , 53, theμijkσ=Dσ , it can Additional file Drosophila melanogaster, did not require approval from an ethics committee.The research performed in this study on the fruit fly, All RNA-seq data from this study are available from the NCBI Gene Expression Omnibus (GEO) under the accession number GSE60314.


Personalized medicine has become a priority in breast cancer patient management. In addition to the routinely used clinicopathological characteristics, clinicians will have to face an increasing amount of data derived from tumor molecular profiling. The aims of this study were to develop a new gene selection method based on a fuzzy logic selection and classification algorithm, and to validate the gene signatures obtained on breast cancer patient cohorts.We analyzed data from four published gene expression datasets for breast carcinomas. We identified the best discriminating genes by comparing molecular expression profiles between histologic grade 1 and 3 tumors for each of the training datasets. The most pertinent probes were selected and used to define fuzzy molecular grade 1-like (good prognosis) and fuzzy molecular grade 3-like (poor prognosis) profiles. To evaluate the prognostic performance of the fuzzy grade signatures in breast cancer tumors, a Kaplan-Meier analysis was conducted to compare the relapse-free survival deduced from histologic grade and fuzzy molecular grade classification.We applied the fuzzy logic selection on breast cancer databases and obtained four new gene signatures. Analysis in the training public sets showed good performance of these gene signatures for grade . To validate these gene signatures, we designed probes on custom microarrays and tested them on 150 invasive breast carcinomas. Good performance was obtained with an error rate of less than 10%. For one gene signature, among 74 histologic grade 3 and 18 grade 1 tumors, 88 cases (96%) were correctly assigned. Interestingly histologic grade 2 tumors (n = 58) were split in these two molecular grade categories.We confirmed the use of fuzzy logic selection as a new tool to identify gene signatures with good reliability and increased classification power. This method based on artificial intelligence algorithms was successfully applied to breast cancers molecular grade classification allowing histologic grade 2 classification into grade 1 and grade 2 like to improve patients prognosis. It opens the way to further development for identification of new biomarker combinations in other applications such as prediction of treatment response.The online version of this article (doi:10.1186/s12920-015-0077-1) contains supplementary material, which is available to authorized users. Breast cancer, the most common invasive cancer in women, is an heterogeneous and complex disease. Currently, the management of breast cancer patients is based on clinicopathological characteristics such as age, menopausal status, tumor size, lymph node status, histologic grade  and on thttps://www.adjuvantonline.com/index.jsp), for treatment decision making. The combination of these parameters provides assessments of the benefit of a systemic endocrine or chemo-therapy . This linear re-scaling of the variable into the interval  was performed according to equation (1) where This standardization, based only on histologic grade 1 and grade 3 status, was necessary since gene expression values in different platforms are scaled by unknown parameters and because grade compositions diverge between datasets.fuzzified using the training data based on the appropriate learning process (LAMDA). The resulting fuzzy sets represent the probe’s (feature) membership to each of the two existing classes. Probe fuzzification was performed according to two membership functions proposed by Aguado and Aguilar in [Next, each gene probe value was uilar in . Both fufuzzified, the fuzzy feature selection algorithm MEMBAS was performed to rank probes in descending order according to their resulting fuzzy weight (wf). Then, iteratively, the classification performance in terms of overall classification error, sensitivity (percentage of histologic grade 3 tumors correctly classified), and specificity (percentage of histologic grade 1 tumors correctly classified) was calculated using LAMDA classification algorithm. Hence, for each iteration, the number of probes was incremented, following the ranked order, until the whole probe set was tested. Since most cohort datasets are scarce and small (in terms of patient number), classification performance was evaluated with a leave-one-out cross-validation (LOOCV) to estimate the optimal classification parameters as proposed by Wessels et al. [α). The resulting model was employed to classify the test sample. This was carried out on all samples so that each of them was used once for testing. This procedure is commonly used to evaluate the generalization performance of a classifier, since it reduces bias from the training set, i.e. data for a sample (tumor) are never used to estimate the classifier parameters (class profile) for its own classification. The resulting probe set, providing the higher classification performance, characterizes the fuzzy Molecular Grade 3-like (fMG3) and fuzzy Molecular Grade 1-like classes.Once all gene expression values from probes had been s et al. . This meFuzzy Molecular Grade 3-like (fMG3) class. Then, if the score was greater than or equal 0.5 the patient is most likely to have a Grade 3-like gene expression profile (fMG3) and if the score was less than 0.5 the patient experiences a Grade 1-like gene expression profile (fMG1).Given that LAMDA classification provides a global adequacy degree (GAD) for each sample to both classes Figure , a molecfMG1-like or fMG3-like according to their resulting score (GAD).Once the optimal probe subset had been identified for a given cohort, the histologic grade 2 and unknown grade patients were classified into either For each breast cancer cohort we identified the most relevant genes using MEMBAS/LAMDA algorithms. The procedure was as follows:Gene signatures obtained were validated on an additional cohort. In order to focus the attention to the most informative genes, custom NimbleGen Roche 4x72K gene expression microarray kit  was designed by NimbleGen support, based on genes included in gene signatures provided by the proposed fuzzy logic selection strategy (2030 accession numbers). We added controls of microarray experiment and housekeeping genes for normalization data. A mean of 9 probes by sequence were designed.fGS), 150 frozen breast cancer tumors from the tumor bank of the Claudius Regaud Institute  were selected. This cohort consisted of consecutive invasive breast carcinoma patients treated at Claudius Regaud Institute between 2009 and 2011. All patients included in this cohort signed an informed consent. Clinico-pathological characteristics of the series were similar to those observed in routine clinical practice . Clinical–pathological data of our samples are available in Additional file In order to validate the different generated gene signatures (http://www.bioconductor.org/) and R software (version 2.14.1). All data were Robust Multichip Average (RMA) background corrected, log2-transformed, summarized by a robust mean for each probe. Then, housekeeping normalization was done on the experimental data. This method involved subtracting the average from a subset of 20 selected reference genes. As numerous studies reported that commonly used reference genes are not constantly expressed under different experimental conditions, we selected the 20 most suitable reference genes among 62 candidate genes previously reported as ‘housekeeping’ genes, based on their expression stability and correlation. Correlation coefficient between fluorescence and quantity of controls were comprised from 0.96 to 1, providing high confidence in the experimental data. The microarray data have been deposited in the Gene Expression Omnibus (GEO) public database. The GSE53958 study can be found at: http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE53958.Total RNA was extracted from 12 μm-thickcryostat sections of with Qiazol and RNeasy Lipid Tissue Kit . The concentration of total RNA obtained from each tumor sample was determined using a Nanodrop® spectrophotometer  and the integrity of the RNA was assessed using a 2100 Bioanalyzer® . The percentage of tumor cells was evaluated on frozen a section stained with hematoxylin-eosin. Double-stranded cDNA (ds-cDNA) was synthesized from 2 μg of total RNA using SuperScript One cycle kit  with random primers and Oligo(dT) primers, then cleaned with MinElute PCR Purification Kit . ERCC RNA Spike-In Control Mix , a set of external RNA positive controls, was added to total RNA at the beginning of the experiment to assess accuracy of measurements of gene expression. One μg ds-cDNA was Cy3 labeled using One-Color DNA labeling kit . Hybridization on our Roche NimbleGen 4x72K custom microarray, washing slides and scan were performed according to the manufacturer’s protocol . DEVA software v1.2  was used to extract the raw intensity values. Spike in control analyses, normalization and statistical analyses of the data were performed with Bioconductor packages  or last date of follow-up . SurvivaAll P-values reported were two-sided. For all statistical tests, differences were considered significant at the 5% level. Statistical analyses were performed using R software.


Gene expression microarray has been the primary biomarker platform ubiquitously applied in biomedical research, resulting in enormous data, predictive models, and biomarkers accrued. Recently, RNA-seq has looked likely to replace microarrays, but there will be a period where both technologies co-exist. This raises two important questions: Can microarray-based models and biomarkers be directly applied to RNA-seq data? Can future RNA-seq-based predictive models and biomarkers be applied to microarray data to leverage past investment?We systematically evaluated the transferability of predictive models and signature genes between microarray and RNA-seq using two large clinical data sets. The complexity of cross-platform sequence correspondence was considered in the analysis and examined using three human and two rat data sets, and three levels of mapping complexity were revealed. Three algorithms representing different modeling complexity were applied to the three levels of mappings for each of the eight binary endpoints and Cox regression was used to model survival times with expression data. In total, 240,096 predictive models were examined.Signature genes of predictive models are reciprocally transferable between microarray and RNA-seq data for model development, and microarray-based models can accurately predict RNA-seq-profiled samples; while RNA-seq-based models are less accurate in predicting microarray-profiled samples and are affected both by the choice of modeling algorithm and the gene mapping complexity. The results suggest continued usefulness of legacy microarray data and established microarray biomarkers and predictive models in the forthcoming RNA-seq era.The online version of this article (doi:10.1186/s13059-014-0523-y) contains supplementary material, which is available to authorized users. Microarray-based gene expression profiling represents a mature, high-throughput, transcriptomic analysis approach that has been extensively applied in biomedical and clinical research as the major biomarker tool for almost two decades. An important outcome is a number of large-scale microarray data sets for public reference, for example, the Connectivity Map  database ,2, ChemiDuring more recent years, next-generation sequencing technologies (NGS) have emerged as a powerful alternative to microarrays, particularly for whole transcriptome analysis with RNA-Seq -15. BesiAs a part of the FDA-led community wide Sequencing Quality Control (SEQC) project , we broa2 intensity/counts data and per sample z-scored data; and group C mappings were excluded from modeling since their ambiguous mapping relationships were not suitable for cross-platform prediction. Thus, we conducted 216 model level assessment processes  and thereby generated 96,024 predictive models (500 trained models per process for binary endpoint prediction and one model per process for Cox modeling). Figure Examining the reciprocal transferability of predictive models and signature genes between microarray and RNA-Seq data requires understanding the sequence correspondence in gene mappings between the two platforms. Consequently, we first performed cross-platform mappings of microarray probes or probe sets to RNA-Seq genes and explored the consistency of gene expression measurements between microarray and RNA-Seq data using two rat and three human data sets having both microarray and RNA-Seq data. The two rat data sets were from the NCTR rat toxicogenomics study ,21 and tGene mapping is an essential step to assess the transferability of gene expression-based predictive models between microarray and RNA-Seq. Cross-platform gene mapping is complicated by disparity between array annotations and RNA-Seq gene models. Moreover, given the complexity of human transcriptomes and the lack of complete genome functional annotation , gene anWe first mapped Affymetrix Rat_230_2 arrays to Illumina GA II RNA-Seq using the method depicted in Figure We then mapped Affymetrix Rat_230_2 arrays to six RNA-Seq gene sets generated from the same raw data of 62 rat liver samples using different bioinformatics pipelines and references, representing a diversity of approaches popularly used in RNA-Seq data analysis . Array pTo assess the transferability of signature genes from microarray to RNA-Seq data, we first applied the signature level assessment process . The comparability is the same in terms of AUC. Per sample agreement beyond chance between the two predictions from each pair of microarray and RNA-Seq models was evaluated with Kappa statistic . Thus, the gene mapping complexity did not affect the transferability of signature genes from microarray to RNA-Seq data.We first mapped Agilent array probes to RNA-Seq genes and then sorted them into groups A, B, C, and D in accordance with gene mapping complexity. As indicated above, the probes in groups A, B, and C exhibited different degrees of consistency in gene expression measurements between microarray and RNA-Seq Table c. For eac Figure b. For thTo confirm these findings, we also applied nearest shrunken centroids (NSC) and support vector machine (SVM) modeling algorithms. Using the same comparison workflow, the results from NSC and SVM are similar to those from k-NN . Thus, signature genes identified from RNA-Seq can also be directly transferred back to microarray data for model development without significant loss of predictive accuracy. Again, the cross-platform gene mapping complexity did not affect the transferability of RNA-Seq signature genes back to microarray data.To assess the transferability of signature genes of models developed from RNA-Seq data to microarray data . The per sample prediction agreement assessed with the Kappa statistic is inversely correlated to the complexity of the clinical endpoints . The results from NSC algorithm are similar to those from k-NN . Therefore, the transferability of microarray models to predict RNA-Seq data is dependent on the choice of modeling algorithms. For algorithms that are not too sensitive to data values such as k-NN and NSC, microarray-based models can directly be applied for prediction of RNA-Seq-profiled samples; while for data-value-sensitive algorithms such as SVM, such a direct application is challenging.When SVM algorithm was applied, the difference between the model performances in predicting microarray and RNA-Seq data was quite large . The accuracies of the models in cross-platform prediction dropped dramatically for most endpoints compared to that in predicting the samples profiled with the same platform as used for model development  for the prediction of patient’s survival status. Because survival times were not considered during modeling process, the analyses may not be extrapolated to the models in which survival times were directly modeled. To examine the transferability of such models and associated signature genes, we applied Cox proportional hazards survival regression to model survival times with gene expression data for the SEQC NB data set and compared the performance of Cox models in terms of concordance index and survcomp . The consurvcomp .P values (<0.01) calculated with the function of concordance.index in R package survcomp [To train a Cox’s model, the genes in a training set were first filtered with their median intensities  and survcomp  and rankP value indicating whether the concordance index is significantly different from 0.5. The two concordance indices were finally compared with the function of cindex.comp to test whether the first concordance index is significantly greater than the second. As shown in Additional file To assess gene level transferability, a Cox model was first trained from a training data set from one platform through the leave-one-out cross-validation process and then used to predict the corresponding validation samples profiled with the same platform. The signature genes were then used with the training data set from the other platform to build a Cox model to predict the corresponding validation samples. The performance of each Cox model was assessed with a concordance index with a To examine model level transferability, we first did per-sample z-score transformation for data and then trained a Cox model using a training set from one platform and then applied it to separately predict the validation samples profiled with the two platforms. The performance of the model in predicting both validation samples was separately measured with the concordance index and then the two concordance indices were compared to each other with cindex.comp. The transferability of Cox models between microarray and RNA-Seq data sets also shows a similar pattern to those from the previous binary endpoint prediction analyses  study that the prediction performance of microarray gene expression profile-based models is mainly dependent on endpoints . The resThe analyses demonstrated that microarray models can directly predict RNA-Seq-profiled samples if the gene-expression data were z-score preprocessed before modeling and prediction and that the signature genes of a model developed from one platform can be directly transferred to the other platform for model development. However, it is difficult to directly use the models developed from RNA-Seq data to predict microarray-profiled samples. Our study offers a viable option for the proper use of legacy microarray data, microarray-based predictive models and biomarkers in the RNA-Seq era and demonstrated a means to utilize RNA-Seq-based signature genes in microarray data analysis.The FDA SEQC NB data set contains 498 NB samples that were independently profiled with Agilent customized 4 × 44 K oligonucleotide microarrays and Illumina HiSeq 2000 RNA-Seq. In the FDA SEQC NB study, six binary clinically relevant endpoints and four continuous survival times were defined among the 498 NB samples Table . For eacThe TCGA AML  data setWe used patients’ sex and cytogenetic risk as two binary endpoints for binary endpoint prediction analysis and used event-free survival (EFS) and overall survival (OS) times as two continuous responses with Cox proportional hazards regression to predict patients’ potential survival risk based on gene expression data. The training and validation sets were constructed as follow: (1) since the original cytogenetic risk includes good, intermediate, and poor three levels, we combined the intermediate and poor levels together to form a new ‘poor’ level and used with the original good level for binary endpoint prediction analysis; (2) Then randomly split the patients in the two cytogenetic risk groups into a training set (17 good +51 intermediate +21 poor) and a validation set (16 good +50 intermediate +20 poor). The same splitting was also used for the endpoint sex. The training set includes 43 female and 46 male patients, while the validation set contains 39 female and 47 male patients.The NCTR rat toxicogenomics data set includes eight microarray and eight RNA-Seq assays. The microarray and RNA-Seq data were generated from exactly the same set of RNA samples isolated from the kidneys of four aristolochic acid-treated and four control rats . The micMicroarray data came from the FDA MAQC main study  and consde novo assembly). Details about RNA-Seq reads generation, alignment, and gene counting can be found elsewhere [The SEQC rat toxicogenomics data contains 62 rat liver RNA samples. Each individual RNA sample was separately assayed with Affymetrix Rat_230_2 arrays and Illumina HiScanSQ/HiSeq 2000 RNA-Seq. The microarray data were generated and normalized in the National Toxicology Program DrugMatrix Database. Details about the data generation and normalization can be found elsewhere . Here, wlsewhere . The micde novo assembly or used by RNA-Seq pipelines for quantification of gene expression.The method used for cross-platform gene mapping between microarray probes/probe sets and RNA-Seq genes is depicted in Figure To map by one of the four types of public gene IDs, each array probe/probe set was examined by comparing the gene ID or ID list in the corresponding array type annotation file to all RNA-Seq genes. In order to map with a genome location mapping approach, the coordinates in the ‘Alignments’ column of array type annotation files were used to calculate the overlap between each microarray probe set and all exons of each RNA-Seq gene. A microarray probe set was considered to be mapped to a RNA-Seq gene if the length of the overlap between the coordinates specified in the annotation file and an exon of an RNA-Seq gene was greater than 40 base pairs, or if it overlapped with at least two exons of the RNA-Seq gene. After mapping, both microarray probes/probe sets and RNA-Seq genes were separately classified into four different groups: A, B, C, and D Table .The Agilent customized 4 × 44 K oligonucleotide microarray annotation file was obtained from the GEO database with series accession number GSE49710.The microarray probe set annotation files for Rat230_2 and HG-U133_Plus_2 were downloaded from the Affymetrix web site . Both fiWe used the T-index  score toTA is a metric for estimating the transferability of the models developed from platform A to the models for platform B. N is the total number of models (500 in this study). A and B, respectively. s.d. is the standard deviation of (where Table 2RPM used in this study can be downloaded from the GEO database with series accession number GSE49710 and GSE62564, respectively. The TCGA AML clinical information, microarray and RNA-Seq data can be separately downloaded using links https://tcga-data.nci.nih.gov/docs/publications/laml_2012/clinical_patient_laml.tsv, https://tcga-data.nci.nih.gov/docs/publications/laml_2012/HG-U133_Plus_2.Level_2.tgz and https://tcga-data.nci.nih.gov/docs/publications/laml_2012/laml.rnaseq.179_v1.0_gaf2.0_rpkm_matrix.txt.tcgaID.txt.gz. The NCTR rat toxicogenomics microarray and RNA-Seq data can be obtained with GEO series accession numbers GSE5350 and GSE21210, respectively. The MAQC-I main study and the SEQC main study data can be downloaded from GEO database with series accession numbers GSE5350 and GSE47774, respectively. The SEQC rat toxicogenomics microarray and RNA-Seq data have been deposited in GEO database under series accession numbers GSE47875 and GSE55347, respectively.The SEQC NB microarray gene expression data and RNA-Seq log


Despite an overall decrease in incidence of and mortality from cancer, about 40% of Americans will be diagnosed with the disease in their lifetime, and around 20% will die of it. Current approaches to test carcinogenic chemicals adopt the 2-year rodent bioassay, which is costly and time-consuming. As a result, fewer than 2% of the chemicals on the market have actually been tested. However, evidence accumulated to date suggests that gene expression profiles from model organisms exposed to chemical compounds reflect underlying mechanisms of action, and that these toxicogenomic models could be used in the prediction of chemical carcinogenicity.In this study, we used a rat-based microarray dataset from the NTP DrugMatrix Database to test the ability of toxicogenomics to model carcinogenicity. We analyzed 1,221 gene-expression profiles obtained from rats treated with 127 well-characterized compounds, including genotoxic and non-genotoxic carcinogens. We built a classifier that predicts a chemical's carcinogenic potential with an AUC of 0.78, and validated it on an independent dataset from the Japanese Toxicogenomics Project consisting of 2,065 profiles from 72 compounds. Finally, we identified differentially expressed genes associated with chemical carcinogenesis, and developed novel data-driven approaches for the molecular characterization of the response to chemical stressors.Here, we validate a toxicogenomic approach to predict carcinogenicity and provide strong evidence that, with a larger set of compounds, we should be able to improve the sensitivity and specificity of the predictions. We found that the prediction of carcinogenicity is tissue-dependent and that the results also confirm and expand upon previous studies implicating DNA damage, the peroxisome proliferator-activated receptor, the aryl hydrocarbon receptor, and regenerative pathology in the response to carcinogen exposure. Weinberg Despite an overall decrease in mortality from cancer, about 41% of Americans will be diagnosed with the disease and about 21% will die from it High-throughput genomic approaches have been successfully applied toward the elucidation of the molecular mechanisms of cancer initiation and progression, to the identification of novel therapeutic targets, and to the development of diagnostic and prognostic biomarkers, resulting in thousands of publications. However, their application to the study of the environmental causes of cancer has not received as much attention.de facto “gold-standard”. The 2YRB requires, for each compound, the use of more than 800 rodents and for each rodent a histopathological analysis of more than 40 tissues, with a cost per compound in the $2–4 million range depending on route of administration, number of doses to be examined, and chemical being evaluated. As a result, only approximately ∼1,500 of the ∼84,000 chemicals in commercial use have been tested Standard approaches to carcinogen testing have adopted the 2-year rodent bioassay (2YRB) as the In this study, we present the results of our analysis of two large cohorts of rat-based expression profiles from animals exposed to hundreds of well-annotated chemicals with varying carcinogenicity and genotoxicity  was performed to identify the major sources of variation in the DrugMatrix dataset. A plot of the first two principal components shows that the data are stratified by tissue type , with heNext, we sought to rigorously define the transcriptional response to chemical carcinogens in terms of the genes and signaling pathways significantly associated with chemical perturbations, and differentially expressed between carcinogens and non-carcinogens, as well as between sub-types of carcinogens. To this end, we carried out within- and across-compound differential and pathway enrichment analyses of the DrugMatrix liver samples.perturbational transcriptome – defined as the set union of the genes that significantly respond to chemical perturbation by any compound – and to evaluate whether the perturbation patterns are significantly associated with the carcinogenicity of the compounds. To this end, we identified for each compound the transcripts significantly up- or down-regulated with respect to the matched controls, across multiple durations of exposures. In total, 2,745 (∼24%) transcripts showed significant  ≤0.01, fold-change≥1.5) up-/down-regulation for at least 5 compounds relative to their matched controls (Table S28 in −6), and with a significantly higher number of up/down-regulated genes  sigely both .In summary, our analysis shows that carcinogenic compounds  induce a more pervasive (more genes) and marked (significant) transcriptional response than non-carcinogens, a response that is consistent across multiple compounds, and that manifests itself more often as an up-regulation of expression than a down-regulation. Furthermore, this heightened response is mainly driven by non-genotoxic mechanisms, since no significant enrichment for genotoxicity is observed in either cluster.ZDHHC2, AQP7, IL33), centrilobular hepatic eosinophilia, peroxisome proliferation , hepatocellular hypertrophy  and hepatic lipid accumulation  were differentially regulated. The identification of pathology-associated biomarkers is consistent with the observation that pathological manifestations in short-term studies are associated with cancer outcomes in rodents, and that pathology such as Cirrhosis in humans is a risk factor for hepatocellular carcinoma JAM3, BTG2, MDM2, PLN, NHEJ1, CCNG1, MGMT) appear to be significantly up-regulated in response to carcinogen exposure.Next, we carried out differential analysis aimed at comparing a gene's expression between carcinogens and non-carcinogens , irrespective of their level in the controls. The main purpose of this analysis was not the selection of features for predictions, but rather the investigation of the exposure-induced transcriptional changes toward the elucidation of mechanisms of response. Rigorous statistical testing based on a moderated t-test see , yieldedWithin the list of carcinogenic compounds, comparison of genotoxic carcinogens vs. non-genotoxic carcinogens yields a list of 191  DEGs with a FDR≤0.01, but only 86 of these genes have a FC≥1.35  , further suggesting that genomic outcomes of short-term exposure truly reflect the longer-term process of malignant transformation. A detailed list of all DEGs, including hyper-enrichment analyses of the top genes using DAVID vs. non-carcinogen’ signature , protein turnover , and enrichment of metabolic pathways . Further analysis of the ‘genotoxic vs. non-genotoxic carcinogen’ signatures  of the ‘carcinogen repertoire of transcripts and associated biological pathways previously implicated in the response to carcinogen exposure, thus confirming the quality of the expression data analyzed and their adequacy for our predictive model building effort, to which we now turn.In summary, our supervised analysis of the DrugMatrix data recapitulates and refines the known tissue-agnostic classifiers, were not fruitful  algorithm Predictive models of genotoxicity based on a 500-gene Random Forest classifier were built from the DrugMatrix liver samples. The random resampling-based estimation of classification performance yielded an AUC of 75.1%.tissue-specific labels of carcinogenicity to train a set of predictive models. The resulting carcinogenicity classifier achieved a prediction performance as measured by AUC of 76.7% in liver tissue . To this end, we performed Random Forest classification of all compounds in the DrugMatrix using the structural features, instead of gene expression, as predictors. Evaluation by random resampling yielded an AUC of 70.9% when predicting genotoxicity, and 59.9% when predicting hepato-carcinogenicity  The Random Forest classifier was distinct compounds tested was comparatively small . To assess whether we had reached the maximally achievable predictive accuracy, we analyzed learning curves for both carcinogenicity and genotoxicity based on down-sampling, whereby AUCs were estimated for classifiers built on training sets of progressively larger size , the number of size see . As showIn summary, our Random Forest-based classifier trained on the gene expression data from the DrugMatrix was capable of predicting carcinogenicity with a random resampling AUC of 77.6%, and significantly outperformed other state-of-the-art classifiers , thus making us confident that our modeling approach would generalize well to new untested chemicals.validation set, the TG-GATEs . To this end, a final 500-gene random forest classifier of liver carcinogenicity was trained on all of the available compounds in the DrugMatrix (n = 127) using the tissue-specific carcinogenicity labels. The top 50 markers as ranked by variable importance are shown in The performance of our classification model was next evaluated on an independent The prediction performance of the models trained on DrugMatrix and tested on TG-GATEs provides supporting evidence of the validity of our approach since significant classification accuracy was achieved across datasets despite the difference in experimental conditions (dose and time) of the two datasets, and the known dataset-to-dataset bias inherent in the Affymetrix microarray platform With the predictive model established and validated on an independent dataset, we next tested the impact of exposure time and dose on the effect of a chemical compound. The repeat samples  from TG-GATEs correspond to systematic tests of chemical compounds at four different exposure times between 4 and 29 days and at three doses, with three replicates for each condition. Predictive accuracy for each time-dose combination was assessed based on the random resampling scheme, and the corresponding AUCs and 95% confidence intervals are shown in In summary, validation of our carcinogenicity classifier on an independent dataset confirmed the predictive accuracy obtained in the discovery set, thus proving the robustness and generalization capability of our modeling approach. Furthermore, the increased accuracy we achieved by training and testing within the same validation dataset, while taking advantage of dose-dependent labels, further emphasizes the critical role played by across-dataset bias, and the importance of using accurate (dose-dependent) phenotypic labels.dH-score, and a manually derived score, mH-score , and used these scores as gold-standard proxies of the carcinogenic potential of a given compound-time-dose instance against which to test our classifier's accuracy.The availability of the short-term histopathology reviews for the samples profiled in TG-GATEs allowed us to preliminarily assess our ability to predict the carcinogenicity of chemicals not included in the CPDB, and thus begin to address our ultimate goal of predicting the carcinogenicity of as-yet untested chemicals. To this end, we derived two binary scores from the histopathology findings included in the TG-GATEs, a fully data driven score, mH-score; however, the false positive instances mostly correspond to compounds whose multiple replicates disagree with respect to their mH-score, that is, the false positive instance was predicted as positive by our classifier, but was mH-score negative, while the additional replicates of the same compound were both predicted and mH-score positive . We expect that with a longer time of exposure the pathology report would also show evidence for carcinogenicity.Since this evaluation required the time-consuming manual review of histopathology findings, the analysis was limited to a subset of the available samples. In particular, repeat samples from rats exposed at maximum dose and maximum time (29 days) were selected. Next, a 500-gene Random Forest classifier was trained on the samples with the same exposure time and dose level for which hepatocarcinogenicity status was available (n = 108). This classifier was applied to the prediction of all unknown compounds (n = 252), and only samples with prediction probability above 0.66 (carcinogenic) or below 0.33 (non-carcinogenic) were selected, yielding a final set of 124 samples for which manual (and blind) review of the histopathology findings was available. The comparison of the classifier's predictions with the pathology-derived scores is summarized in gene set projection approach, whereby the data are mapped from single genes to gene sets representing well-annotated biological pathways and processes  onto the top 50 gene sets as ranked by variable importance. We then performed hierarchical clustering to identify modules of coordinated gene sets likely to reflect distinct mechanisms of action. The resulting heatmap is shown in Even though there are only 41 distinct carcinogenic compounds tested in the dataset, the gene set projection-based clustering results highlight the considerable heterogeneity in the response to carcinogen exposure, likely reflecting distinct mechanisms of cancer induction, and point to a promising approach to their data-driven categorization. A notable example is represented by the seven genotoxic compounds clustered under the orange color bar on top of the heatmap . GenotoxA third cluster of compounds exclusively captures lipid lowering compounds , which all show a significant up-regulation of lipid metabolism pathways . Lipid-lowering drugs have been under suspicion as potential carcinogens for more than a decade Finally, more than two thirds of the carcinogenic compounds show an up-regulation of the proteasome pathway. This is interesting since a large body of scientific literature e.g.  identifiTaken together, these results suggest that gene set projection is a helpful approach for controlling for batch-to-batch and cross-dataset variability, while increasing a classifier's interpretability by making explicit the biological pathways that contribute to prediction.Through our computational analysis of two large rat-based gene expression datasets, we conclusively validated the hypothesis that expression profiles of short-term exposure are highly predictive of the long-term carcinogenicity of (exposure to) chemicals as measured by the 2-year rodent bioassay. Additionally, we extensively evaluated the capability of gene expression profiling to model the transcriptional effects of exposure to chemical perturbations, and showed that the integration of data-driven analysis and pathway-centered annotation best captures the biological processes and pathways that this exposure affects.Analysis of expression data from multiple tissues  showed that the most effective approach to carcinogenicity prediction necessitates the definition of tissue-specific classifiers. Consequently, we focused our classification effort on data from liver, since this tissue had the largest number of profiles and compounds evaluated, as well as the most thorough compound annotation.Our classifiers based on the Random Forest, and on as few as 500 genes as predictors (selected by variance filtering), yielded predictive accuracy as measured by AUC ranging from 76.7 (DrugMatrix) to 82.7 (TG-GATEs), with the sensitivity/specificity trade-off depending on the cost function adopted . The preWe were also interested in comparing our predictive model to two published gene signatures: the Ellinger-Ziegelbauer et al. 2008 - 512-gene carcinogenicity signature Supervised analysis of the DrugMatrix  shows that the “exposure to carcinogens” phenotype is not adequately modeled as a simple dichotomy, especially when we consider the non-genotoxic carcinogens. This is reflected in the results of the differential analysis . Additionally, a disproportionate number of compounds analyzed in the DrugMatrix act through the peroxisome-proliferating receptor (PPAR) pathway, hence compounds acting through other mechanisms of action might not be adequately represented. Our down-sampling simulation analysis aimed at evaluating sample size adequacy shows that the classification learning curve see  does notProjection of the expression data matrix into gene set space, and subsequent classification using the gene sets as predictors, had the dual advantage of increasing the interpretability of the model (by identifying pathways and processes relevant to cancer induction) and of making it more robust across datasets (by correcting for batch-to-batch bias). However, it adversely impacted the predictive accuracy modestly within datasets see . ConsequThe availability of data from a whole-transcriptome array allowed us to evaluate the dependency of a classifier's performance on the number and identity of the genes used as predictors, and to determine what would be a sufficient number of gene markers to include in a custom array designed to model chemical carcinogenicity. As noted, the selection of the top 500 genes as ranked by variance (rigorously carried out within the training set of each training-/test-set split) was sufficient to train a Random Forest classifier with highest predictive accuracy. Increasing the number of genes to 1000 or more, or replacing the variance ranking with a t-score ranking (with respect to the phenotype to be predicted) did not measurably affect the predictive accuracy . This is particularly intriguing since it emphasizes the potential role played by loss of normal tissue function in carcinogenesis. Equally of notice was the identification of a module reflecting up-regulation of the proteasome in response to carcinogens. The proteasome is closely tied to ribosome function, which is in turn linked to cell proliferation.Even though we have a large number of profiles at our disposal (2195 liver samples in the Drugmatrix), there are only 127 well-annotated tested compounds and only 41 of these are known hepatocarcinogens in rodents. Furthermore, there are various (>5) mechanisms of action, as shown in in vitro human models amenable to high-throughput screening Toxicogenomic short-term exposure studies based on in-vivo (rat) models remain expensive and time consuming and therefore limit the number of chemical compounds that can be tested. Furthermore, as noted, animal models make for an imperfect proxy to test human carcinogenicity. To address both these shortcomings, the next generation of toxicogenomics tests is poised to rely on tissue-agnostic carcinogenicity label, set to be positive if the compound was found to cause cancer in any tissue type, negative otherwise. Additionally, tissue-specific carcinogenicity labels were also defined for liver.The Carcinogenic Potency Database (CPDB)discovery set is based on the DrugMatrix and genotoxicity annotation were profiled in liver, yielding a set of 1,221 profiles available for model building.The validation set is based on the Toxicogenomics Project-Genomics Assisted Toxicity Evaluation system (TG-GATEs) single group, rats were exposed at a single time point, and mRNA was extracted after 3 to 24 hours, in the repeat group, rats were exposed daily for 4 to 29 days, and mRNA was extracted at each of four end points , and at each of three doses . For this study we used only the repeat group of TG-GATEs. Of the 72 compounds tested in TG-GATEs, 25 were also tested in the DrugMatrix, leaving 47 unique compounds for validation  for which either carcinogenicity or genotoxicity information was available, were considered for this analysis. Only the genes with FDR-corrected q-value≤0.01 and fold-change≥1.5 in at least five compounds were included. Hierarchical clustering of both the compounds and the genes based on the t-scores' matrix was performed, and the results visualized in a heatmap with the color-coding based on the t-test's q-values  and the compound status  across compounds, and the nominal p-values were corrected for multiple hypothesis testing by the FDR procedure  implements an ensemble classification approach combined with bagging, whereby multiple decision trees are inferred from random subsets of the training data, and the class predictions of the component trees are combined by majority voting. After evaluation of multiple sizes, a Random Forest based on 500 trees (the package's default) was selected as the size that yielded the best trade-off between accuracy and computational efficiency. In addition to the performance measurements, we also report the variable importance for each gene. This measurement reflects the increase of the error rate across all trees, if the value of the tested gene is randomly permuted when testing.The Random Forest algorithm . The area under the ROC curve (AUC) was chosen as our primary evaluation criterion since this measure is independent of the threshold chosen to call a compound carcinogenic (genotoxic). The choice of the appropriate threshold depends on the relative costs assigned to false negatives and false positives, and these in turn depend on the primary purpose for which the classifier is used, an assessment that is beyond the scope of this study. For completeness, accuracy, sensitivity, specificity, positive and negative predictive values, and false discovery rate are also reported for every classification task  Gene set projection was used to map the original data from gene space to gene set space. In particular, each treated sample was compared with the set of corresponding control samples, and a weighted Kolmogorov-Smirnov Learning curves relating classification AUC to compound's sample size were built based on a variation of the standard random resampling scheme. Starting from a training set consisting of 70 compounds, up to the total number of compounds in increments of 10, AUC means and standard deviations were estimated based on 200 random resampling iterations. The estimated AUCs and their corresponding number of compounds are shown in http://toxico.nibio.go.jp/). The histopathology findings are coded into 133 categorical covariates, each taking values in the range 0–4  and includes items such as liver microgranuloma and liver hypertrophy centrilobular. To summarize these findings and relate them to carcinogenicity, we defined two binary scores (negative/positive) to label each of the compound-dose-time instances. The first score (dH-score) is data-driven and represents the logic OR of all the covariates, denoting an instance as positive if any of the covariates for that instance has a value greater than zero .TG-GATEs provides the results of histopathology exams of tissues from the profiled animals, including high-resolution whole slide digital images of their liver and kidney on the TG-GATEs portal (mH-score) results from the manual review of a compound-dose-time instance by a board certified toxicologist with experience designing and interpreting subchronic and chronic toxicity/carcinogenicity studies. Factors taken into account when scoring the samples included the degree of adversity associated with specific pathologies , the historical association between the pathological manifestation and subsequent liver cancer outcomes in a 2-year bioassay, the severity of the pathology observed, and the multiplicity of pathology types. Due to the time-consuming nature of the manual review, only a subset of compound-dose-time instances were annotated, corresponding to the repeat samples from rats exposed at the maximum dose for 29 days (maximum time). The manual review and annotation of the instances was blinded, that is, the carcinogenicity status predicted by the classifier was withheld at the time of the instances' annotation. The resulting scores were used as a proxy measure of carcinogenicity to evaluate the prediction performance of our classifiers on compounds for which no CPDB annotation was available.The second score Click here for additional data file.File S2Supporting Information Tables. Contains Tables S3–4, Table S19 and Tables S24–29. Table S3: GSEA report for carcinogens - GSEA results of carcinogenic versus non-carcinogenic compounds in liver - Gene Set Enrichment Analysis comparing the gene expression profiles from rats exposed to carcinogenic and non-carcinogenic compounds. Only gene sets enriched in the carcinogenic compounds with a false discovery rate ≤0.05 are shown. Table S4a: GSEA report for genotoxic carcinogens - GSEA results of genotoxic versus non-genotoxic carcinogens in liver - Gene Set Enrichment Analysis comparing the gene expression profiles of rats exposed to genotoxic and non-genotoxic carcinogens. Only gene sets enriched in either the genotoxic or non-genotoxic carcinogens with a false discovery rate ≤0.05 are shown. Table S4b: GSEA report for non-genotoxic carcinogens - GSEA results of genotoxic versus non-genotoxic carcinogens in liver - Gene Set Enrichment Analysis comparing the gene expression profiles of rats exposed to genotoxic and non-genotoxic carcinogens. Only gene sets enriched in either the genotoxic or non-genotoxic carcinogens with a false discovery rate ≤0.05 are shown. Table S19 Variable importance ranking of the top 150 predictive pathways - Variable importance was calculated by training 200 Random Forest models on the DrugMatrix dataset using a 70–30% random resampling scheme. The variable importance of all 200 models was averaged. Table S23: Chemicals tested in the DrugMatrix - Chemicals with known hepatocarcinogenicity status that were tested in liver. Also included are exposure durations, compound doses and genotoxicity information based on Ames tests if available. Table S24: Chemicals tested in TG-GATEs - Chemicals with known hepatocarcinogenicity status that were tested in liver. Also included are exposure durations and compound doses. Table S25: DrugMatrix cross-validation predictions - Hepatocarcinogenicity predictions in the DrugMatrix Random forest cross-validation results for carcinogenicity in liver using different numbers of features, based on a variance ranking. The prediction results for the 3 replicates for each compound were collapsed. Each value represents the mean and 95% confidence interval over 200 iterations of a 70%/30% train/test dataset split. Table S26: TG-GATEs predictions – Hepatocarcinogenicity predictions in TG-GATEs based on a model trained on the DrugMatrix. The prediction results for the 3 replicates for each compound were collapsed. Table S27: Hepatocarcinogenicity predictions in TG-GATEs. Random forest cross-validation results for carcinogenicity in liver using different numbers of features, based on a variance ranking. The prediction results for the 3 replicates for each compound were collapsed. Each value represents the mean and 95% confidence interval over 200 iterations of a 70%/30% train/test dataset split. Table S28: Ranked gene list from differential analysis - Genes ranked according to the number of compounds inducing their significant up-/down-regulation between carcinogenic and non-carcinogenic compounds. Table S29 – Complete list of compounds: List of compounds that are exclusively in the DrugMatrix and TG-GATEs or present in both sets, with additional information on which compounds were used for training and testing.(ZIP)Click here for additional data file.


New data sources for the analysis of cancer data are rapidly supplementing the large number of gene-expression markers used for current methods of analysis. Significant among these new sources are copy number variation (CNV) datasets, which typically enumerate several hundred thousand CNVs distributed throughout the genome. Several useful algorithms allow systems-level analyses of such datasets. However, these rich data sources have not yet been analyzed as deeply as gene-expression data. To address this issue, the extensive toolsets used for analyzing expression data in cancerous and noncancerous tissue  could be redirected to extract a great deal of predictive information from CNV data, in particular those derived from cancers. Here we present a software package capable of preprocessing standard Agilent copy number datasets into a form to which essentially all expression analysis tools can be applied. We illustrate the use of this toolset in predicting the survival time of patients with ovarian cancer or glioblastoma multiforme and also provide an analysis of gene- and pathway-level deletions in these two types of cancer. Copy number variations (CNVs) are promising DNA-level biomarkers of cancer subtype. CNVs can influence the phenotypes of cancer by disrupting  or duplicating  copies of a gene . Althoughttp://cnv-tools.sourceforge.net/) [Previous studies describe several software packages useful for analysis of CNVs –8, primage.net/)  and its ge.net/) . These aIn the past decade, a number of useful and well-organized toolsets were developed for analyzing cancer phenotypes or genotypes. However, most of these software packages are commonly used to analyze gene-expression datasets. For example, unsupervised clustering methods and supervised classification methods are applied to machine learning algorithms in order to classify cancer phenotype, survival time, cancer metastasis, and so forth. Such algorithms create distinctions based on tissue RNA signatures for their predictive and classification tasks. Certain tissue samples contain macroscopic DNA variations which extend RNA variations based on CNVs, especially in cancer.Particularly in cancer tissue, distinctions, classifications, and predictions based on such DNA variations may be useful. Indeed, DNA variation in a tumor changes more slowly than RNA variation and thus may be considered less noisy. Therefore, it is worthwhile to consider DNA-based algorithms.If DNA copy number datasets  could be reprocessed into formats that parallel RNA expression signatures obtained from microarrays, it would be possible to construct DNA-based algorithms that parallel existing algorithms based on RNA. To the extent that a primary contributor to the expression level of a gene in cancerous tissue is the corresponding gene copy number, this type of analysis, using DNA copy number microarrays, could be considered as a proxy for RNA microarrays. Given that RNA signatures are more time-dependent than cancer-cell DNA signatures, the latter provide a more stable set of biomarkers for use in prediction of survival time, chemotherapy response, and other outcomes.Here we present an algorithm which reprocesses Agilent DNA copy number signatures into a format that parallels the microarray signatures used in standard software packages for prediction and classification based on microarrays. Because copy number datasets are provided with probe IDs, we first need to download gene information to convert Agilent probe positions to gene regions. The following describes the steps for generation of CNARs corresponding to known genes.We downloaded 216 ovarian cancer (OV) and 215 glioblastoma multiforme (GBM) CNA datasets, produced by Harvard Medical School using the Agilent Human Genome Comparative Genomic Hybridization Microarray 244A platform (HG-CGH-244A), updated in 2012 from the TCGA data portal. The portal also provided probe IDs with probe positions on the chromosome covered by 60 base pairs (bp). We also selected two classes of survival datasets for classification. From the GBM datasets, we extracted 23 samples from patients whose survival time was less than 100 days (short class) and 23 samples from patients with survival time greater than 1500 days (long class). From the OV datasets, we extracted 18 samples from patients whose survival time was less than 1 year (short class) and 18 samples from patients whose survival time was greater than 5 years (long class). The survival times distinguishing the long and short classes differed between GBM and OV because GBM is a more fatal disease.http://www.biomart.org/biomart/martview), which provides chromosome location for each HGNC_ID gene according to the HUGO gene nomenclature committee (http://www.genenames.org/) [We downloaded Agilent genome region from BioMart (es.org/) . Out of We extended each gene region by including 2000 bp for the promoter region and 100,000 bp on each side, the so-called pseudogene. Each chromosome includes approximately 20 million base pairs, and the number of genes in each chromosome is about 1000. Therefore, an expansion of 100,000 bp on each side is reasonable. In addition, the region involved in CNV is commonly much larger than the size of a gene. We note that the range of CNV is commonly 1 kb or larger . The numCij of the base matrix reports the existence of a probe ID, ith, corresponding to a pseudogene jth by matching chromosome regions obtained from Steps k is the total number of the probe IDs in pseudogene j: pi is the ith probe and gj represents the extended jth gene region.First, we generated a sparse matrix called the base matrix, with dimensions 21,856 (genes) by 227,612 (probe IDs). Each component We combined all CNA datasets from These comprehensive datasets allow a large class of gene-expression software to be utilized to study DNA rather than RNA signatures. In μA − μB)2/(σA2 + σB2), representing the square of the difference in means of two classes  divided by the sum of the square of their variances.To test our newly generated CNAR datasets, we implemented well-known machine learning algorithms including unsupervised cluster methods and supervised classification methods: consensus clustering, silhouette clustering, and the support vector machine (SVM). The Fisher criterion method  was adopWe applied the newly generated CNAR datasets to SVM with feature selection for classification of survival in patients with OV (18 short and 18 long samples) and GBM (23 short and 23 long samples). To avoid any prior bias, the training of the classifier, the choice of the number of features, and feature selection were done strictly in test datasets. For evaluation, we used standard leave-one-out cross validation (LOOCV). In OV, the best classification accuracy of long versus short survival was 83.33%, using four features. For GBM, the best classification accuracy of long versus short survival was 82.61%, using 20 selected features. The accuracies obtained using CNAR datasets were higher than the accuracies obtained using gene-expression microarray datasets .In OV datasets, since four-feature selection performed best results, we collected all selected four features from all train datasets as performing LOOCV. STOML3, including eight CN probes from chr13:39482884 to chr13:39554832, was selected in all training samples. The boxplots of STOML3 show the comparison of short survival patients to long survival patients . In geneWe applied the CNAR datasets to the unsupervised consensus clustering algorithm  providedP = 0.09; data not shown).We also subjected OV CNAR datasets to the same procedure. In that case, we downloaded 216 samples and selected 175 genes with variance of 0.4, and the results are shown in y-axis represents CNAR level; the x-axis represents genes in chromosome 7. In addition, we selected the gene with the largest gain in copy number, EGFR, using standard deviation from the GBM CNAR dataset. The distribution of CNAR in chromosome 7 is provided in We subjected CNAR datasets derived from OV and GBM samples to gene set enrichment analysis (GSEA) . The OV In this study, we generated new datasets that represented DNA variation in formats that paralleled variation in RNA expression and subjected the data to established machine learning algorithms. Most previous studies identified changes in DNA using CN alteration and then interpreted the relationship with genes by looking up the chromosome region. These new CN alteration-based datasets, which we call CN array (CNAR) datasets, could be used directly for identification of genetic variation at the RNA level. In addition, the new CNAR datasets enable straightforward visualization of gain or loss of pseudogenes along the chromosome. In our analyses, we used several existing methods including clustering, classification, and GSEA. In addition, we applied CNAR datasets to two clustering methods, consensus clustering and silhouette clustering. The clustering performances were superior to those obtained using RNA expression. We also applied CNAR to the SVM classification algorithm for GBM and OV cancer patients stratified for short and long survival times; the accuracies were 83.33% and 82.61%, respectively. When we subjected CNAR datasets to GSEA, we found two enriched metabolic pathways in both OV and GBM with a FDR cutoff of zero. These new datasets enable many applications, including clustering of cancer subtypes, prediction of survival times, and classification of cancer metastasis by analyzing DNA alterations using tools developed for RNA-level analysis. Such analyses may provide novel insights into the biological mechanisms underlying cancer. The major limitation of this analysis is the fixed extension of the gene region. Up to date there is no information for extending gene region for copy number alteration. Therefore, we need further study for extension method for each gene region.


Currently, quantitative RNA-seq methods are pushed to work with increasingly small starting amounts of RNA that require amplification. However, it is unclear how much noise or bias amplification introduces and how this affects precision and accuracy of RNA quantification. To assess the effects of amplification, reads that originated from the same RNA molecule (PCR-duplicates) need to be identified. Computationally, read duplicates are defined by their mapping position, which does not distinguish PCR- from natural duplicates and hence it is unclear how to treat duplicated reads. Here, we generate and analyse RNA-seq data sets prepared using three different protocols . We find that a large fraction of computationally identified read duplicates are not PCR duplicates and can be explained by sampling and fragmentation bias. Consequently, the computational removal of duplicates does improve neither accuracy nor precision and can actually worsen the power and the False Discovery Rate (FDR) for differential gene expression. Even when duplicates are experimentally identified by unique molecular identifiers (UMIs), power and FDR are only mildly improved. However, the pooling of samples as made possible by the early barcoding of the UMI-protocol leads to an appreciable increase in the power to detect differentially expressed genes. High throughput RNA sequencing methods (RNA-seq) are currently replacing microarrays as the method of choice for gene expression quantification12347910RNA-seq library preparation methods are designed with different goals in mind. TruSeq is a method of choice, if there is sufficient starting material, while the Smart-Seq protocol is better suited for low starting amounts137That said, it is unclear whether removing read duplicates computationally improves accuracy and precision by reducing PCR bias and noise or whether it decreases accuracy and precision by removing genuine information. Here, we investigate the impact of PCR amplification on RNA-seq by analyzing datasets prepared with Smart-Seq, TruSeq and UMI-seq as well as different amounts of amplification. We investigate the source of read duplicates by analysing PCR bias and fragmentation bias, assess the accuracy using ERCCs - spike-in mRNAs of known concentrations17et al.We analyse five different datasets that represent three popular RNA-seq library preparation methods. We started with two benchmarking datasets from the literatureAll datasets contain ERCC-spike-ins, which allows us to compare the accuracy of the quantification of RNA-levels. Furthermore, all datasets except the UHRR-UMI-seq have paired-end sequencing, which should provide more information for the computational identification of PCR duplicates.The number of computationally identified paired-end read duplicates (PE-duplicates) varies between 6% and 19% for the bulk data and 1% and 59% for the single cell data. Since single-end data is commonly used for gene expression quantification, we also consider the mapping of the first read of every pair. The resulting fractions of computationally identified duplicates from single-end reads (SE-duplicates) are much higher. For the bulk data, it ranges from 36–74% and for the single cell data from 6–94% , Fig. 2aGenerally, the fraction of read duplicates is expected to depend on library complexity, fragmentation method and sequencing depth. Sequencing depth is the factor that gives us the most straight-forward predictions and in the case of SE-duplicates they are by in large independent of other parameters such as the fragment size distribution. As expected, we observe a positive correlation between the number of reads that were sequenced and the fraction of SE-duplicates . In ordeIn contrast to this simple sampling expectation for SE-duplicates, fragments produced during PCR-amplification after adapter ligation, will necessarily produce fragments with the same 5′ and 3′ end and consequently will have identical mapping for both ends. If the sampling was shallow enough so that we would not expect to draw the same 5′ end twice by chance, the 3′ end position should also be identical and no reads with only one matching 5′ end are expected. If same 5′ ends are more frequent due to biased fragmentation, we expect a higher ratio of SE- to PE-duplicates. Thus, the relationship between PE- and SE-duplicates contains information about the relative amounts of duplicates produced by fragmentation as compared to amplification. More specifically, we expect that the fragmentation component of the PE- vs. SE-duplicates should be captured by a quadratic fit with an intercept of zero .The only dataset for which the quadratic term is not significant is the UHRR-TruSeq dataset. This could be seen as an indication of a higher proportion of PCR-duplicates, but it is more likely due to the low sample size of only 5 replicates. More importantly, the quadratic term is significant and positive for the HBRR TruSeq, the UHRR Smart-Seq and the scHCT116 datasets, supporting the notion that at least for those datasets library PCR amplification is not the dominant source of duplicates. This is also consistent with our finding that most observed SE-duplicates are simply due to sampling .R2s of 0.77 and 0.85 for the Smart-Seq and TruSeq protocols, respectively. Note, that this high R2 holds for samples that were prepared in different labs: for example the R2 between the Smart-Seq samples prepared in our lab and the single cell data from the Quake lab ranges between 0.56–0.90. In contrast, if the R2 is calculated for the comparison between one TruSeq and one Smart-Seq library, it drops to 0.0012 (R2 =  0.08) than with the TruSeq data , and none in the Smart-Seq data  of the SE-duplicates unexplained. For the UHRR-Smart-Seq data, a ~38.5-fold reduction in the effective length is needed and leaves only 3% (0.6–5.1%) of the duplicates unexplained. For the single cell data, the fragmentation bias that gives overall the best fit is a ~8-fold reduction, however the fit is worse since the fraction of unexplained duplicates is still at ~7% and varies between 0.3% and 61% . In summR2 values of a log-linear fit of transcripts per million (TPM) versus ERCC concentration to quantify how well TPM estimates molecular concentrations and compare the fit among the different duplicate treatments. In no instance does removing read duplicates improve the fit, but in most cases the fit gets significantly worse  except for the computational PE-duplicate removal of the UHRR-Smart-Seq and the duplicate removal using UMIs  genes between groups of samples. The extra noise from the PCR-amplification has the potential to create false positives as well as to obscure truly DE genes. In order to assess the impact of duplicates on the power and the false discovery rate (FDR) to detect DE genes, we simulated data based on the estimated gene expression distributions of the five datasets. For comparability, we first equalized the sampling depth by reducing the number of mapped reads to 3 million and 1 million for bulk and single cell data, respectively. Next, we estimated gene-wise base mean expression and dispersion using DESeq2N There are no big differences in the distributions of mean baseline expression and dispersion estimates from the different duplicate treatments for the two Smart-Seq datasets, whereas there is a shift towards lower means and higher dispersions, when removing SE-duplicates for the TruSeq datasets. Dispersions shift only to lower values if we exclude duplicates based on identification by UMIs . The empα =  5% is exceeded: the means vary between 5.4% and 10.1%, whereas the HBRR TruSeq has the lowest and the scHCT116 Smart-Seq data the highest FDR (p <  1 ×  10−15).Except for the UHRR-UMI-seq dataset, the nominal FDR that we set to hest FDR . Computahest FDR . The comThe differences in the power are more striking. As for the FDR, the major differences are not between duplicate treatments, but between the datasets. For the TruSeq and the UHRR-UMI-seq datasets, the average power to detect a log2-fold change of 0.5 is ~80% . For thoThe large differences in power between the datasets are unlikely to be ameliorated by increasing the number of replicates per group. In addition to the 6 and 45 replicates for which the results are reported above, we also conducted simulations for 12 and 90 replicates for bulk and the single cell data, respectively. This doubling in replicate number increases the power for the UHRR-Smart-Seq dataset only from 52 to 63% and for the single cell dataset from 27 to 34% .RNA-seq has become a standard method for gene expression quantification and in most cases the sequencing library preparation involves amplification steps. Ideally, we would like to count the number of RNA molecules in the sample and thus would want to keep only one read per molecule. A common strategy applied for amplification correction in SNP-calling and ChIP-Seq protocols23Since computational methods cannot distinguish between fragmentation and PCR duplicates, the removal of read duplicates could introduce a bias rather than removing it. Using the ERCC-spike-ins, we can indeed show that removing duplicates computationally does not improve a fit to the known concentrations, but rather makes it worse, especially if only single-end reads are available . This is2526Moreover, the Smart-Seq protocol, which was designed for small starting amounts, involves PCR amplification before the final fragmentation of the sequencing library. Thus in the case of Smart-Seq, computational methods cannot identify PCR duplicates that occur during the pre-amplification step. When we use unique molecular identifiers (UMIs), we find that 66% of the reads are PCR duplicates and only 34% originate from independent mRNA molecules. In contrast, when using paired-end mapping for a comparable Smart-Seq library, we identify 13% as duplicates and 87% as unique. This might in part be due to the fact that in UMI-Seq we sequence mainly 3′ ends of transcripts, thus decreasing the complexity of the library, which in turn increases the potential for PCR duplicates for a given sequencing depth . HoweverAll in all, computational methods are limited when it comes to removing PCR-duplicates, but how much noise or bias do PCR duplicates introduce? In other words, we want to know how PCR-duplicates impact the power and the false discovery rate for the detection of differentially expressed genes. Both, power and FDR, are determined by the gene-wise mean expression and dispersion. Based on simulated differential expression using the empirically determined mean and dispersion distributions, we find that computational removal of duplicates has either a negligible or a negative impact on FDR and power, and we therefore recommend not to remove read duplicates. In contrast, if PCR duplicates are removed using UMIs, both FDR and power improve. Even though the effects in the bulk data analysed here are relatively small: FDR is improved by 4% and the power by 2%, UMIs will become more important when using smaller amounts of starting material as it is the case for single-cell RNA-seq6The major differences in power are between the datasets with the TruSeq and the UMI-seq data achieving a power of around 80%, the UHRR-Smart-Seq 52% and the single cell Smart-Seq data (scHCT116) only 27%. Note that this apparently bad performance of the single cell Smart-Seq data is at least in part due to an unfair comparison. While all the other datasets were produced using commercially available mRNA and thus represent true technical replicates, the single cell data necessarily represent biological replicates and thus are expected to have a larger inherent variance and thus lower power.However, also the UHRR Smart-Seq bulk data achieves with 52% a much lower power than the other bulk datasets. One possible explanation for the differences in power is the total number of PCR-cycles involved in the library preparation. With every PCR-cycle the power to detect a log 2-fold change of 0.5 appears to drop by 2.4% . The onlWe conclude that computational removal of duplicates is not recommendable for differential expression analysis and if sufficient starting material is available so that only few PCR-cycles are necessary, the loss in power due to PCR duplicates is negligible. However, if more amplification is needed, power would be improved if all samples are pooled early on, and for really low amounts as for single cell data also the gain in power that is achieved by removing PCR-duplicates using UMIs will become important.μl). The recommended ERCC spike-in amounts result in 5–107 ERCC RNA molecules in the cDNA synthesis reaction.We used six datasets representing the TruSeq, Smart-Seq and UMI-seq protocols and varying amounts of starting material from bulk RNA or single cell RNA. All analysed datasets contain the ERCCs spike-in RNAs. This is a set of 92 artifical poly-adenylated RNAs designed to match the characteristics of naturally occurring RNAs with respect to their length (273–2022 bp), their GC-content (31–53%) and concentrations of the ERCCs  for the two datasets produced for this study. We downloaded UHRR- and HBRR-TruSeq data from SEQC/MAQC-IIIet al.et al.For the Smart-Seq libraries, 250 ng of Universal Human Reference RNA  and ERCC spike-in control mix I (Life Technologies) were used and cDNA was synthesized as described in the Smart-Seq2 protocol from Picelli et al.For the UMI-seq libraries, we started with 10 ng of UHRR-RNA to synthesise cDNA as described in Soumillon Library pools were sequenced on an Illumina HiSeq1500. The Smart-Seq libraries were sequenced using 50 cycles of paired-end sequencing on a High-Output flow-cell. The UMI-seq libraries were sequenced on a rapid flow-cell with paired-end layout, where the first read contains the sequences of the sample barcode and the UMI sequence using 17 cycles. The second read contains the actual cDNA fragment with 46 cycles.For Smart-Seq and TruSeq libraries, the sequenced reads were mapped to the human genome (hg19) and the splice site information from the ensembl annotation (GRCh37.75) using STAR(version:2.4.0.1)et al.For UMI-seq data, cDNA reads were mapped to the transcriptome as recommended in Soumillon We defined single-end (SE) read duplicates as reads that map to the same 5′ position, have the same strand and the same CIGAR value. Because we cannot determine the exact mapping position for 5′ soft clipped reads, we discard them. To flag paired-end duplicates (PE), we used the same requirements as for the SE-duplicates, those requirements had just to be fulfilled for both reads of a pair.rG) and the effective length of the gene (LeG =  L −  2 ×  read-length). Then the expected number of unique reads can be estiamted asWe obtain an expectation for the number of reads if duplicates are identified via their 5′ position and only one read per 5′ end position is kept. The only input parameters are the observed number of reads per gene (P (X =  k) can be calculated using a positive Poisson distribution with λG =  rG/LeG and s is a scaling factor whereas LeG by a factor f ×  LeG.In order to estimate the level of fragmentation bias, we simply modified the effective length To compare fragmentation sites across libraries, we counted 5′ read starts per position for the ERCCs across all datasets using samtools and in house perl scripts. To avoid edge effects in later analyses, we excluded the first and last 100 bases of each ERCC, whereas 100 bases is the maximum read length of datasets analysed here.We generated a Position Weight Matrix (PWM) for the transposase (Tn5) motif by simply stacking up the 30 bases of the putative Transposase binding sites from all UHRR-Smart-Seq reads. Those 30 bases are identified as 6 bases upstream of the 5′ read end and the 24 downstreamet al.N . We generated 100 simulations per original input data-set and analysed them using DESeq2. Next, we calculated the power to detect a log2-fold change of at least 0.5 and the according FDR using α =  0.05.For power analysis, we estimated the mean baseline expression and dispersion for all datasets after downsampling them to 3 and 1 million reads for bulk and single cell data, respectively. This was done for all three duplicate treatments  using DESeq2Accession codes: RNA-seq data generated for this study is submitted to GEO under the accession code: GSE75823.How to cite this article: Parekh, S. et al. The impact of amplification on differential expression analyses by RNA-seq. Sci. Rep.6, 25533; doi: 10.1038/srep25533 (2016).