Breast cancer, the second leading cause of cancer death of women worldwide, is a heterogenous disease with multiple different subtypes. These subtypes carry important implications for prognosis and therapy. Interestingly, it is known that these different subtypes not only have different biological behaviors, but also have distinct gene expression profiles. However, it has not been rigorously explored whether particular transcriptional isoforms are also differentially expressed among breast cancer subtypes, or whether transcript isoforms from the same sets of genes can be used to differentiate subtypes. To address these questions, we analyzed the patterns of transcript isoform expression using a small set of RNA-sequencing data for eleven Estrogen Receptor positive (ER+) subtype and fourteen triple negative (TN) subtype tumors. We identified specific sets of isoforms that distinguish these tumor subtypes with higher fidelity than standard mRNA expression profiles. We found that alternate promoter usage, alternative splicing, and alternate 3’UTR usage are differentially regulated in breast cancer subtypes. Profiling of isoform expression in a second, independent cohort of 68 tumors confirmed that expression of splice isoforms differentiates breast cancer subtypes. Furthermore, analysis of RNAseq data from 594 cases from the TCGA cohort confirmed the ability of isoform usage to distinguish breast cancer subtypes. Also using our expression data, we identified several RNA processing factors that were differentially expressed between tumor subtypes and/or regulated by estrogen receptor, including YBX1, YBX2, MAGOH, MAGOHB, and PCBP2. RNAi knock-down of these RNA processing factors in MCF7 cells altered isoform expression. These results indicate that global dysregulation of splicing in breast cancer occurs in a subtype-specific and reproducible manner and is driven by specific differentially expressed RNA processing factors. Breast cancer, the second leading cause of cancer death of women worldwide, is a heterogenous disease. Different subtypes of breast cancer display very different expression programs, and these expression programs are associated with different patient outcomes and with different treatment protocols. However, little is known about what drives these subtype differences. By sequencing RNA in a discovery cohort of breast cancer patients, we demonstrate that different subtypes of breast cancer can be distinguished by simply using differential transcript isoform expression. We confirmed our findings using two additional patient cohorts. We also demonstrate that differential expression of RNA processing factors between subtypes can affect differences in isoform usage. Using RNAi we knock down differentially expressed RNA processing factors including YBX1, YBX2, MAGOH, MAGOHB, and PCBP2, and show that this knock-down results in differential isoform expression of the genes identified in our disease subtype panel. Taken together, our results indicate that global dysregulation of splicing occurs in a subtype-specific and reproducible manner in breast cancer, and is driven by specific differentially expressed RNA processing factors. Breast cancer is the most common carcinoma in women world-wide and is the second leading cause of cancer death in American women . The vasStudies establishing the molecular subtypes of breast cancer demonstrated the global dysregulation of the breast cancer transcriptome. However, microarrays do not accurately capture variation in isoform usage. Dysregulation of differential splicing of the transcriptome has emerged as an important phenomenon in tumorigenesis ,1011][1[111]12]12]. IndeRNA sequencing allows for efficient and accurate assessment of isoform usage ,18. To dRNA-sequencing allows accurate assessment of transcript abundance, identification and quantification of isoform usage and efficient discovery of fusion-genes ,18. To eAs microarray based differential expression measurements may be confounded by isoform specific microarray probes, we explored the replication rate between microarray data and RNAseq data for single isoform and multi-isoform Refseq genes. As expected, we observed a higher replication rate for single isoform genes as opposed to multi-isoform genes . We nextWe also performed DAVID analysis of differentially expressed genes, and this approach demonstrated enrichment for regulatory processes that affect both gene and protein expression levels and function, such as phosphoproteins, 3’-UTR mediated translational regulation and gene expression . The mosTo replicate subtype-specific isoform usage, we used data generated by The Cancer Genome Atlas (TCGA).We downloaded and aligned 594 breast cancer RNAseq data sets from TCGA, following the same data processing pipeline used for our internal dataset . As in oTo further validate our results, we designed isoform-specific Nanostring probes for 212 isoform pairs that showed differential subtype expression in the original 25 samples . To asseTo determine if the ability of differential isoform expression to segregate ER+ and TN subtype was biologically reproducible, we measured isoform expression via Nanostring in an independent set of 68 breast cancer specimens obtained from FFPE blocks from the University of Chicago pathology core (44 ER+ and 24 TN). To assess the ability of our isoform expression signature to distinguish ER+ and TN breast cancer subtypes in this independent cohort, we used the Nanostring data from the discovery cohort to build a logistic regression-based subtype classifier and applied that classifier to the replication cohort. Subtype prediction based on isoform expression was consistent using this replication cohort and an independent method for RNA measurement  were then compared to determine whether they differed in 5’UTR, exons, and/or 3’UTR. 967 isoform pairs were found to be differentially expressed. The majority of differentially expressed pairs were alternatively spliced (63.5%), but differential promoter usage (24.3%) and differential 3’UTR usage (12.2%) were also common . These fWe next examined the role that differential 5’UTR and 3’UTR usage played in differentiating ER+ and TN subtypes. Alternate UTR usage could result in either change of coding sequence or of only non-coding sequence. For each pair of differentially expressed isoforms that differed in UTR sequence, we determined whether the resulting differences also resulted in coding sequence differences. 50% of all 5’UTR differences resulted in coding sequence differences between isoforms, while the vast majority of 3’UTR differences were accompanied by changes in coding sequence We hypothesized that differential expression of RNA processing factors generates these differences in isoform expression. Using a list of human RNA processing factors derived from Uniprot , we founWe next asked whether splicing targets of these factors were also known ER and PR targets, using previously published ChIP binding data. InteresA better understanding of the biological mechanisms that underlie gene expression differences between ER+ and TN breast cancer subtypes promises to yield new insights into subtype-specific therapeutic targets. Here we have used paired-end, whole transcriptome sequencing to identify a set of isoforms that distinguish ER+ and TN tumors. Recently published studies have strongly implicated alternative splicing as an emerging factor in tumorigenesis ,1011,12,1211,12.Our finding that differential transcript isoform usage can differentiate breast cancer subtypes is strongly reproducible. Using only ~200 isoforms, we demonstrate this finding in three independent cohorts, University of Chicago Frozen, University of Chicago FFPE, and TCGA. Thus, breast cancer subtypes could be distinguished based on the expression of different versions of the same genes. These results suggest that subtype-specific, differential regulation of promoter usage, alternative splicing, and 3’UTR usage contributes to differences in gene expression and thus differences in biological behavior between subtypes. We find that many RNA processing factors are differentially expressed between ER+ and TN breast cancer subtypes. Furthermore, we have identified several RNA processing factors, including YBX1, YBX2, MAGOH, MAGOHB, and PCBP2, that appear to contribute to subtype-specific splicing. We have shown that, not only are these RNA processing factors differentially expressed between subtypes, but also that when they are knocked down, splicing changes in the direction predicted. In other words, knockdown of factors expressed in ER+ BRCA led to a more TN-like splicing pattern, and vice versa. Thus, global dysregulation of splicing events is a fundamental component of the breast cancer transcriptome that occurs in a subtype-specific and reproducible manner. Differential expression of splicing factors between ER+ and TN breast cancer subtypes alters the transcriptome in predictable ways that likely contribute to differences in biology between breast cancer subtypes. Addition of splice usage to breast cancer transcriptome studies may further refine our ability to distinguish breast cancer subtypes, as well as perhaps identify fine details within subtypes. Of note, we also attempted to knock-down these factors in a MBA-MD-468 and measure isoform usage, but the knock-down was lethal to these cells.Additionally, further study of the splicesome may lead to improved understanding of breast cancer biology and suggest several ways in which altered splicing may affect phenotype in breast cancer subtypes. For example, TN tumors collectively express a short, anti-apoptotic isoform of caspase 2 that was not expressed in ER+ tumors,29. ThusThus, in this work, we have used RNA-seq to mine the transcriptome of two common invasive breast cancer subtypes; ER+ and triple negative breast cancer. We demonstrate that, in addition to transcript abundance, isoform abundance is sufficient to distinguish breast cancer subtypes, and implicate differences in isoform usage as underlying some of the differences seen in the biology of these two tumors. We have identified differentially expressed RNA processing factors, and demonstrate that they are responsible for splicing of subtype-specific isoforms.Twenty-five frozen breast carcinoma samples (11 ER positive and 14 triple negative) were obtained from the de-identified tissue bank in the Human Tissue Resource Center in the Department of Pathology at the University of Chicago Medical Center. A de-identified pathology report was supplied with each case, which contained grade, stage, lymph node status, age +/- 5 years and self-reported race . Additio36, 37). Briefly, frozen section slides cut from each block were used to select areas with high tumor cell concentration. These areas of the frozen block were then punched using a sharpened 1mm tissue microarray punch. OCT was removed from the ends of this punch via a scalpel, and the frozen core was ground to a fine powder under liquid nitrogen. Total RNA and gDNA was then extracted from frozen tissue samples using Qiagen QiaShredder/RNeasy and DNeasy Blood and Tissue kits. RNA quality was confirmed using an Agilent Bioanalyzer, and no RNA with a RIN < 7.5 was used. Poly-adenylated RNA was purified with the micropolyA purist kit (Ambion), ethanol precipitated, and resuspended in 10 uL RNAse free water. First strand reverse transcription was primed with random nonomers, using SuperscriptII RT (Invitrogen). Second strand synthesis was performed with RNAse H and DNA Polymerase I. Double stranded cDNA was repaired and polished with T4 DNA polymerase, Klenow, and T4 polynucleotide kinase. Adenosine was added to 3' ends with Klenow and paired end Illumina adapters ligated with T4 DNA ligase. Ligation reactions were size selected for 400–500 bp fragments after agarose gel electrophoresis and PCR amplified for 15 cycles using Illumina PCR primers and Pfx polymerase. Paired end Illumina sequencing was performed according to the manufacturer's instructions. RNA from MCF7 knockdowns was isolated using Qiagen RNeasy kits, and RNAseq libraries were built using ScriptSeq kits from Epicentre, following manufacturer’s protocols.cDNA libraries were prepared for Illumina sequencing with minor modifications to published protocols(39) and further alignment manipulation was performed with SAMTools, Picard (http://picard.sourceforge.net), and custom perl scripts. Read alignments with mapping qualities less than 10 were removed. Alignments generated from more than one sequencing lane were merged with Picard MergeSamFiles. We downloaded TCGA consortium aligned RNA-seq bams for 657 samples from cgHub  as a member of the TCGA working group. TCGA's Breast RNA-seq libraries are generated using the Illumina TrueSeq library preparation protocol and sequenced on the Illumina HiSeq 2000 machines. A median of 76.4 million 2x50bp reads were generated for each sample. Using in-house scripts, we generated raw fastq sequence files from the bams for re-analysis.Illumina sequence files were converted to Sanger fastq format . Paired end reads were aligned to the human reference genome (build 36) using TopHat. Tophat was run with an estimated mean and standard deviation insert size of 200 and 50 bases, respectively. Sequencing artifacts occasionally produced lanes with fewer than 20% aligned reads. Raw data QC revealed that many such cases were caused by precipitous drops in sequence quality at late run cycles. In such cases, reads were trimmed from the 3' end such that, on average across all reads in a lane, the Phred quality score based estimate of the probability of a sequencing error within the remaining read was less than 95%. TopHat alignments were generated in SAM format at the 3' ends of the reads using a trimming algorithm implemented in BWA. Tophat requires the parameters for insert size distribution of the RNA-seq library. We determined for each sample this by sub-sampling 1 million reads and re-aligning them to the spliced transcriptome using bowtie-2.0.0-beta. Mean and standard deviation are determined from the reads that align concordantly to the transcriptome using the CollectInsertSizeMetrics module in Picard suite of tools. We next aligned all reads to hg18 reference genome using tophat-1.3.3 (—no-coverage-search—segment-length 25—segment-mismatches 2). We also provided the gene models to the aligner at this step for faster and accurate resolution of known splice junctions. Transcript quantification on the aligned bams is performed using cufflinks-2.0.0 .http://cufflinks.cbcb.umd.edu/) for transcript assembly and quantitation, with RefSeq gene models as a reference annotation set. FPKM distributions were quantile normalized across lanes. For principal component analysis (performed via pcaMethods package in R), FPKMs were mean centered. Subtype-specific differential gene expression was calculated in two ways 1) via gene-by–gene linear model, of the form y ~ m0 + subtype + E and 2) by Wilcoxon rank sum test, comparing the two subtypes. Subtype-specific isoform expression was calculated via gene gene-by–gene linear model, of the form y ~ m0 + subtype + isoform + subtype*isoform + E, where the interaction between isoform and subtype identified differentially expressed genes. All effects were treated as fixed effects, and multiple hypothesis testing was controlled via Storey’s q-value FDR.RNASeq: Transcript expression levels were estimated directly from each lane of RNAseq data, as Fragments Per Kilobase per Million mapped reads (FPKM) using cufflinks , of the form y ~ m0 + subtype + E. The intensities of probes targeting the same gene were averaged. Tumor subtype was treated as a fixed effect, significance of the tumor subtype effect was assessed by t-test, and multiple hypothesis testing was controlled via Storey’s q-value FDR. Estrogen time course data was analyzed as in Hua et. al.TM Digital Analyzer by counting the individual specific fluorescent barcodes and quantification of target RNA molecules in each sample.Isoforms with significant subtype*isoform interactions (FDR = 0.05) were submitted to Nanostring for custom design. Two hundred and twelve isoforms, representing 106 genes, could be designed with high quality nanostring probes unique for each isoform. A codeset specific to the targets was designed using a 3’ biotinylated capture probe and a 5’ reporter probe tagged with a specific fluorescent barcode; creating two sequence-specific probes for each target transcript. Probes were hybridized to 100 ng of total RNA for 19 hours at 65°C. Following incubation, samples were applied to the NanoString Preparation Station for automated removal of excess probe and immobilization of probe-transcript complexes on a streptavidin-coated cartridge. All manipulations were performed using the NanoString Preparation Station robotic fluids handling platform. Data were collected using the nCounterAll normalization and analysis of the Nanostring data was performed in R. Nanostring data were normalized by fitting a negative binomial model with terms for assay, well, median negative control, median house keeping gene, and transcript. The resulting residuals were used for subsequent analysis. To evaluate concordance between RNAseq and nanostring expression levels, the ratio of each isoform pair was calculated for all samples in which RNAseq and nanostring data was obtained. To determine the ability of isoform expression to differentiate subtypes, the nanostring data from the 26 samples in our discovery cohort was used to train a logisitic regression classifier. The weights from this regression classifier were then extracted and used to classify each of the 68 FFPE samples into either ER+ or ER- cases. This classification was compared to the pathologic diagnosis, and the performance of the classifier assessed by receiver-operator characteristics.AStalavista was used to extract splicing events from refseq gene models. For all gene symbols with multiple refseq isoforms, subtype-specific isoform expression for all possible isoform pairs was calculated via gene gene-by–gene linear model, of the form y ~ m0 + subtype + isoform + subtype*isoform + E, where the interaction between isoform and subtype identified differentially expressed genes. By comparing isoforms in a pairwise fashion, we could determine whether subtype-specific isoforms differ in 5’UTR, 3’UTR or coding sequence. For differentially expressed isoforms that differ in 5’UTR sequence, 1000bp upstream of each different 5’UTR sequence was extracted. Beta coefficients from the linear model were used to determine which isoform was expressed in each subtype to allow separation of ER+ 5’UTR and TNBC 5’UTRs. These sequences were supplied to MEME 2.6 to identify enrichment of transcription factor binding sites.Illumina reads were treated as above, and aligned to hg18 with Tophat. Cufflinks with refseq gene models was used to assemble transcripts. Cuffdiff was used to identify differentially expressed isoforms, using refseq genes models as a reference.MCF7 were cultured in DMEM/10% FBS + penicillin/streptomycin. Dharmacon ON-Targetplus SMARTpools were ordered for each of the splicing factors, plus negative controls. siRNA transfection used DharamaFECT1 and followed manufacturer’s protocols. Briefly, cells were trypsanized, counted and diluted to 1x10^5 cells/ml, and 3 mls were plated/well in 6-well plates. Cells were incubated at 37C overnight. Cells were transfected the next day with 2uM siRNA using DharmaFECT1. RNA was harvested using Qiagen RNeasy, following manufacturer’s protocols, 72 hours after transfection. Experiments were performed as biological duplicates. Knockdown was confirmed using qrtPCR, using one step RT-PCR SYBR Green master mix from BioRad, following manufacturer’s instructions. qPCR was performed in an ABI StepOne real time PCR machines. GAPDH primers were used as a control. All primer sequences are shown in S1 FigPlot of the beta coefficient from the linear model used to identify subtype-specific expressed genes, with RNAseq data plotted on the x-axis and microarray data plotted on the y-axis. RNAseq is highly concordant with microarray data. ESR1, FOXA1, and XBP1 (in red), which are known to be highly differentially expressed between breast cancer subtypes, are highly differentially expressed in both RNAseq and microarray data sets.(TIF)Click here for additional data file.S2 FigTop two gene sets from Gene Set Enrichment Analysis of normalized RNAseq data are genes up-regulated and down-regulated in ER+ breast cancer.(TIF)Click here for additional data file.S3 FigA) Top 10 enriched pathways from DAVID analysis on differentially expressed isoforms between subtypes . Genes in apoptotic categories are enriched. B) Top gene sets from Gene Set Enrichment Analysis of Isoform FPKM data show enrichment for apoptotic gene modules.(TIF)Click here for additional data file.S4 FigFirst two principal components derived from RefSeq gene RNAseq FPKM expression levels for single isoform genes only. TCGA samples are segregated by breast cancer subtype.(TIF)Click here for additional data file.S5 FigEach panel is represents one of the initial discovery samples, with the ratio of two isoform expression levels from RNAseq data plotted on the x-axis and the ratio of two isoform expression levels from nanostring data plotted on the y-axis.(TIF)Click here for additional data file.S6 FigThere is no significant difference between subtypes.(TIF)Click here for additional data file.S7 FigLeft panel shows FPKM expression level, by sample, for MAGOH (top) and YBX1 (bottom). Right panel show the gene model for MAGOH (top) and YBX1 (bottom), with genomic position on the x-axis, exons represented as thick lines, and introns represented as thin lines.(TIF)Click here for additional data file.S8 FigBoxplots of log2 FPKM values for 6 splicing factors in the discovery data set by subtype.(TIF)Click here for additional data file.S9 FigFollowing transfection of siRNA, expression levels of splicing factors were quantified by qrt-PCR, normalized using TUBB, and compared to scrambled control siRNA.(TIF)Click here for additional data file.S10 Fig(TIF)Click here for additional data file.S11 Fig(TIF)Click here for additional data file.S12 Fig(TIF)Click here for additional data file.S1 Table(XLS)Click here for additional data file.S2 Table(XLS)Click here for additional data file.S3 Table(TXT)Click here for additional data file.S4 Table(XLS)Click here for additional data file.S5 Table(TXT)Click here for additional data file.S6 Table(XLS)Click here for additional data file.


Next generation sequencing provides a count of RNA molecules in the form of short reads, yielding discrete, often highly non-normally distributed gene expression measurements. Although Negative Binomial (NB) regression has been generally accepted in the analysis of RNA sequencing (RNA-Seq) data, its appropriateness has not been exhaustively evaluated. We explore logistic regression as an alternative method for RNA-Seq studies designed to compare cases and controls, where disease status is modeled as a function of RNA-Seq reads using simulated and Huntington disease data. We evaluate the effect of adjusting for covariates that have an unknown relationship with gene expression. Finally, we incorporate the data adaptive method in order to compare false positive rates.When the sample size is small or the expression levels of a gene are highly dispersed, the NB regression shows inflated Type-I error rates but the Classical logistic and Bayes logistic (BL) regressions are conservative. Firth’s logistic (FL) regression performs well or is slightly conservative. Large sample size and low dispersion generally make Type-I error rates of all methods close to nominal alpha levels of 0.05 and 0.01. However, Type-I error rates are controlled after applying the data adaptive method. The NB, BL, and FL regressions gain increased power with large sample size, large log2 fold-change, and low dispersion. The FL regression has comparable power to NB regression.We conclude that implementing the data adaptive method appropriately controls Type-I error rates in RNA-Seq analysis. Firth’s logistic regression provides a concise statistical inference process and reduces spurious associations from inaccurately estimated dispersion parameters in the negative binomial framework.The online version of this article (doi:10.1186/s12859-017-1498-y) contains supplementary material, which is available to authorized users. These RNA sequencing (RNA-Seq) methods produce data that can be transformed into numerical values that are proportional to the abundance of RNA molecules and reflect the expression and turnover of those molecules. Identifying differentially expressed (DE) genes is an important step to understanding the molecular mechanism of disease. As with any statistical analysis, the underlying structure of the data dictates appropriate methodology. The NGS methods provide a count of RNA molecules that do not follow a normal distribution. The Negative Binomial (NB) distribution appropriately models the biological dispersion of a gene, and NB regression has been used to analyze RNA-Seq data. When Y, a random variable, follows a NB distribution with mean  of each gene is challenging with the small number of observations typically available in RNA-Seq studies. An overestimated dispersion may result in loss of power to detect DE genes and an underestimated dispersion parameter may increase false discoveries. Two of the most sophisticated and widely used software packages for identifying DE genes are DESeq2 and edgeR [Estimation of the dispersion parameter  NB regression where gene expression values are the outcome variable and case–control status is the predictor variable and 2) logistic regression where case–control status is a function of gene expression values. If disease-associated covariates are not associated with gene expression, then these covariates are non-predictive (NP) covariates in models with RNA-Seq as the outcome. The effect of adjusting for covariates, when the relationship between covariates and gene expression is not assessed through statistical tests or prior studies, has not been extensively evaluated in RNA-Seq studies using the NB framework. If we alternatively consider a logistic model, the NP covariates in the NB model become non-confounding predictive (NCP) covariates in the logistic model, because the covariates are not associated with the independent variable (gene expression) but are associated with the dependent variable (disease status).We use both simulated data sets and an application to a real Huntington’s disease (HD) RNA-Seq data set . The resThis study focuses on a gene as a unit; hence various gene-based scenarios are considered.glm) function in the R-package “MASS” and utilized either the estimated dispersion from ML, Quasi-likelihood (QL) or the true dispersion value from the simulation scenario. ML and QL methods are described in Additional file NB regression uses the Maximum-Likelihood (ML) fitting process . The gen.binomial/ϕ) functglm function in R. In the RNA-Seq setting CL regression may be limited by small sample size and complete separation. If the expression values of a gene are completely or nearly completely separated between case and control groups, which may occur when the effect size is large, the ML estimation from CL regression may fail to converge. Because complete separation may be an indicator of differential expression, we implemented Bayes and Firth’s logistic regressions, which overcome complete separation bias.We conducted GLM in a logistic regression framework using the logit link function using the bayesglm function in the R-package “arm”.Gelman et al.  proposedlogistf function in the R-package “logistf” was used.The ML estimators may be biased due to small sample size and small total Fisher information. Firth proposed a method that eliminates first-order bias in ML estimation by introducing a bias term in the likelihood function . Heinze The performance of statistical models was evaluated through Type-I error and power scenarios using combinations of the parameter values in Table yi) were sampled from the NB distribution with mean (μ) and dispersion (ϕ) as specified in Table For each scenario, the read counts (i) data with their case (D = 1) and control (D = 0) statuses determined by the study design shown in Table g) expression value for each sample (yig) was generated following the NB distribution conditioning on the disease status. The log2fc determined the mean expression value for cases (ygD=1) in power scenarios. When simulating under the null hypothesis (Type-I error scenarios) log2fc = 0 and the mean expression value (μDg) was equal for cases and controls. We considered only up-regulation of genes, and assumed the dispersion was the same for cases and controls. The simulation model for the RNA-Seq count data is:μgD≥0, ϕg≥0, D is the binary case–control status of a sample (i), μg is mean expression value of a gene g. μgD=1 is the mean expression value for cases for gene g and is calculated as 2log2fc × μgD=0.First, we generated simulated sample (case D =  and contX) were simulated to follow a binomial distribution conditioning on case–control status of each sample. The conditional probability was calculated based on the CovOR.D is disease status , ND is sample size of D, PD=0 = 0.5, and PD=1 = CovOR/(CovOR + 1). When the number of cases is 10, CovOR of 10 is not considered. For every 10 replications among 10,000 replications in a scenario, a new covariate set was generated to incorporate between and within variance of covariates. All covariates in a model were independent and had the same CovOR.The binary covariates  levels 0.05 and 0.01, were calculated based on replicates with converged results. For power studies, because different Type-I error rates were observed among the distinct regression methods, comparing the power of different regression methods using the same threshold is not appropriate. For fair comparison, an empirical threshold for each regression method was calculated. Then, we computed the empirical power of each regression method using their empirical thresholds. Although only positive log2fcs were simulated in those scenarios in which power was evaluated, we consistently used two-sided tests for both Type-I error and power studies. The equations for Type-I error rate and empirical power are shown in Additional file The DA method re-estimates a distribution of test statistics under the null hypothesis of no association . The DA https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE64810). This data set contains 20 HD cases and 49 neurologically normal controls. Details of the HD data set are provided in Additional file A real RNA-Seq data set was analyzed using the DESeq2 R-package, which implements a NB GLM. The data set was also analyzed utilizing R (v3.0.0) to implement CL, BL, and FL regressions. The logistic regressions modeled case–control status as a function of normalized counts of a gene and covariates. The publicly available HD data set  was downThe original HD study used RNA Integrity Number (RIN) and Age at Death (AAD) as covariates. RIN was included due to the potential confounding effect between HD and the abundance of RNAs. To remove the effect of RIN in our permutations, at first, samples were divided by RIN categories. Then, each gene was resampled within each category of RIN. Because AAD was included in the regression model due to its association with HD, the relationship between HD and AAD was preserved during the permutation process. We generated 10,000 Monte-Carlo permutations.For the original HD data and each permutated data set, DE genes were identified using the NB model (Model A) as implemented in DESeq2. We also implemented the CL, BL, and FL regressions (Model B) to compare statistical models. In logistic models, we used normalized counts from DESeq2 as an independent variable. For these analyses, we adjusted for AAD and RIN.p-values [p-values of each gene.The Type-I error rates at our alpha levels using supplementary equation (2.1), and the exact p-values  were calp-values  to measup-values, exact p-values using 10,000 permutation results, and DA using 1000 p-values were calculated and corrected for multiple testing by imposing a false discovery rate (FDR) of 0.05. To assess the model adequacy, QQ-plots of original, exact and DA p-values were generated, and genomic inflation factors, λgc, were calculated. This λgc is the ratio of the median of the observed test statistics divided by the expected median of an asymptotic distribution. If a λgc is greater than 1, this may suggest inflation in the test statistics and may indicate the presence of systemic bias such as hidden population structures, latent covariates, technical artifacts, etc. [For HD data analysis, asymptotic ts, etc. .C = 1, 2, 3, 5, or 10). The change of λgc with the addition of a varying number of simulated covariates in a model was evaluated.To evaluate the effect of covariates in a model, the same method for generating covariates in our simulation study was applied to the HD data set to create simulated covariates. In this real data application, we focused on a moderate covariate effect on HD status (CovOR = 1.2). The HD data were analyzed using the NB GLM in DESeq2 with Model A and using the FL regression with Model B with 1000 replicated sets of simulated covariates , , and . However, genes in the , and  categories exhibit decreasing Type-I error rates , , and  produce increasingly conservative Type-I error rates at both alpha levels, as presented in Fig. The DA method controls Type-I error rates well for the DESeq2 results  in the CL, BL and FL analyses, the 10 most significant (FDR < 0.05) from DESeq2 are shown in Additional file We analyze the HD data using NB GLM in the DESeq2 R-package, and using CL, BL, and FL regressions with the R-functions described in the Methods section. All regression results are corrected with the DA method using 1000 permutation results, and are adjusted for multiple testing using an FDR of 0.05. The Q-Q plots and Simulation results of NB vs. Logistic regressions are also presented in the simulation results with various covariate models.The properties of the NB regression that are shown in Type-I error rates from the simulated data to evaluate the inclusion of covariates in the models are presented in Table The same covariates that are non-predictive (NP) covariates in a NB model are non-confounding predictive (NCP) covariates in logistic models. Unlike the NB regression, even with small sample size Table , when thIn all scenarios, the DA method controls Type-I error rates well in the NB and FL regressions at both alpha levels. The newly approximated distribution of test statistics diminishes deviated Type-I error rates that were not controlled when many covariates were included in the NB and FL models.In addition to Type-I error rates, the bias is also calculated for each scenario. As shown in Additional file The results of the power simulations to evaluate the inclusion of covariates in the models are summarized in Fig. In our simulation, the power of NB and FL regression is affected by three factors 1) Dispersion, 2) CovOR, and 3) Number of NP/NCP covariates in a model, when sample size is fixed. Large dispersion, large CovOR, and increasing number of NP/NCP covariates in a model decrease power. NB regression is less sensitive to an increase of NP covariates with small dispersion. As shown Fig. λgc is presented in Table λgc. The standard deviations of the λgc are increased with the increase of NP/NCP covariates.The HD data were analyzed using DESeq2 and FL regression with additional simulated covariates. A summary of 2fold-change); transcriptome-wide data were not simulated.In this study, we propose using a logistic regression framework as an alternative to NB regression to analyze RNA-Seq data for case–control studies. We have shown in our simulations that FL regression performs well in terms of controlling Type-I error rates and shows comparable empirical power. The dispersion is not estimated in the logistic framework, thus avoiding potential false association resulting from incorrectly estimated dispersion. The simulations presented focused on single genes varying relevant parameters  did not show evidence of association with HD when using DESeq2, but was highly significant when using FL regression, as shown in Table SLC1A6, which is highly expressed in the cerebellum of human brain compared to other regions [PCP4), also known as PEP-19, had dramatic reduction in HD. This gene was not significantly associated with HD status when using DESeq2  but showed strong association when using FL regression . We also found that some genes expressed highly in both cases and controls may not be detected in the NB framework, because it utilizes the ratio of mean expressions of cases and controls. For instance, the normalized mean expression value of SPOCK2 is 12,370 in cases and 15,649 in controls. Although the difference of the means is large, the gene might not be statistically significant due to the small effect size (log2fold-change = −0.34) in the NB framework. However, this gene is strongly associated with HD in our logistic framework (Table SPOCK2, also known as SPARC/osteonectin and Testican-2, plays an important role in the central nervous system [Although it is unknown which genes are truly DE in the HD data set, we compared DE genes identified in the HD data by different statistical approaches. We found that  regions , showed  regions . In addi regions  showed ts system . As a mes system . Severals system .S100A11, have low average counts as shown in Additional file The top genes that showed associations exclusively in NB GLM, except for gene The effect of including covariates has not been investigated for RNA-Seq studies. Identifying relationships with covariates for all genes is computationally demanding and existing software do not allow for defining gene-wise models for all genes, which makes this approach challenging. Therefore, RNA-Seq studies that include covariates in a single model applied to all genes will likely result in some gene expression models that include unassociated covariates. Hence, it is important to investigate the effect of NP covariates in RNA-Seq analysis.Simulations that included NP covariates in the NB model showed inflated Type-I error rates and a loss of power. With large dispersion, this inflation and loss of power becomes severe. The Type-I error in the FL regression is not notably affected by the increment of number of NCP covariates when the CovOR is small. With large CovOR and increased number of NCP covariates, conservative Type-I error rates are observed. The DA method effectively controls the increase of Type-I error rates even with larger CovOR and high number of NP/NCP covariates. Our empirical power results show that the FL regression is more greatly influenced by the increase of covariates than the NB regression, when CovOR is large.λgc(Table λgc, and hence an increased in the median of p-values. In other words, many p-values in a set are generally increased. Based on our simulation results with NP covariates, Type-I error rates of largely dispersed genes are likely to be inflated and power of differentially expressed genes are likely to be decreased. Therefore, this slightly decreased median may indicate that the loss of power is greater than the gain of Type-I error rates.Our HD analyses with simulated NP/NCP covariates demonstrated that an increase in the number of NP/NCP covariates results in a less stable gcTable . Adding λgc in FL regression. This decreased median might be caused by the loss of power, and this loss may occur from NCP covariates in a model according to our simulation results. Under a moderate CovOR, the number of NCP covariates in a model does not affect the Type-I error rates.Adding more NCP covariates in a model also slightly decreases the median of the λgc with additional covariates is larger in FL regression than in NB regression because the FL regression results are solely affected by the loss of power. NB regression results are influenced by both increased Type-I error and decreased power. The standard deviation of λgc is increased with adding NP/NCP covariates in a model. This means that the results generated from a model that includes many covariates is not likely to be reliable, even if these covariates are associated with case–control status but not gene expression.The change in the median In conclusion, unlike NB, CL and BL regressions, FL regression controls Type-I error rates well and maintains comparable power even with small sample size. Firth’s logistic regression is an excellent alternative to NB regression for analysis of RNA-Seq data in case–control studies. We recommend implementing the DA method in analysis of RNA-Seq data to appropriately control Type-I error rates. If computational burden of permutations required for the DA method precludes using this approach, FL regression is the best option for controlling Type-I errors with comparable power to NB regression. However, a parsimonious model is necessary to obtain robust results in the FL regression setting. This approach can be extended in multiple classes of disease status using a multinomial logistic regression method.


NCBI’s Gene Expression Omnibus (GEO) is a rich community resource containing millions of gene expression experiments from human, mouse, rat, and other model organisms. However, information about each experiment (metadata) is in the format of an open-ended, non-standardized textual description provided by the depositor. Thus, classification of experiments for meta-analysis by factors such as gender, age of the sample donor, and tissue of origin is not feasible without assigning labels to the experiments. Automated approaches are preferable for this, primarily because of the size and volume of the data to be processed, but also because it ensures standardization and consistency. While some of these labels can be extracted directly from the textual metadata, many of the data available do not contain explicit text informing the researcher about the age and gender of the subjects with the study. To bridge this gap, machine-learning methods can be trained to use the gene expression patterns associated with the text-derived labels to refine label-prediction confidence.Our analysis shows only 26% of metadata text contains information about gender and 21% about age. In order to ameliorate the lack of available labels for these data sets, we first extract labels from the textual metadata for each GEO RNA dataset and evaluate the performance against a gold standard of manually curated labels. We then use machine-learning methods to predict labels, based upon gene expression of the samples and compare this to the text-based method.Here we present an automated method to extract labels for age, gender, and tissue from textual metadata and GEO data using both a heuristic approach as well as machine learning. We show the two methods together improve accuracy of label assignment to GEO samples. The NCBI Gene Expression Omnibus  is a larFor each archived experimental sample, GEO provides both the data itself, consisting of a vector of counts or probe intensities, as well as the metadata associated with the sample. The metadata usually contains valuable information regarding the nature of the sample, for example the species of origin, array or sequencing platform, as well as the age, gender, tissue of origin, and experimental perturbation(s), mutations, or disease state applied to the sample. Unfortunately, most of this metadata is not provided in a standardized format directly amenable to larger-scale analyses. Rather, it is provided as free-text descriptions which are optionally provided by the investigator. Therefore, these textual fields have a potential to be missing, misspelled, or described in a variety of ways with various synonyms and identifiers. For example, the binary variable representing “gender” can be described in a variety of ways, e.g., “M”, “male”, “1” (in 0-1 coding), and so on. For other fields which may contain a larger variety of values, such as tissue type, the heterogeneity is even larger. This heterogeneity in data documentation in the GEO metadata causes difficulty or completely disables the ability to query the GEO database effectively for large scale comparisons of this large amount of biological data.Furthermore, in compiling our gold-standard for annotation of GEO records, we found 86% of metadata descriptors contained tissue information provided by the investigator, while only 26 and 21% contained gender and age information respectively. This suggests that a machine-learning approach to predicting age and gender labels is needed in addition to text-based classification, because there is otherwise no way of obtaining this data apart from time-consuming measures such as contacting the original investigator on a case-by-case basis. Therefore, we present a system which enables researchers to more easily obtain labels and compare datasets within GEO via similar groupings such as age, tissue, and sex.The need for automated metadata structuring and error correcting in large biological databases has been acknowledged within the field, and there have been attempts to ameliorate these problems with several tools in the past few years. Various methods have been developed to infer labels from GEO for downstream meta-analysis or other large-scale uses of GEO data whereby the sheer volume of samples makes it infeasible to manually curate labels for all samples. Crowdsourcing is one means of doing this cheaply, but will require continued effort as new data comes out .infer the labels from the gene expression data itself. Lee et al. developed URSA (Unveiling RNA Sample Annotation) as an automated method, which utilized one-vs-all or one-vs-rest (OVR) support vector machines (SVMs) on gene expression data in order to infer labels from the gene expression data [Methods have also been developed to not only extract labels from text, but to ion data . They thBuckberry et al. developed a method to infer sex from gene expression data by clustering the expression data, and inferring the labels from the expression of Y chromosomes. Unfortunately, this assumes that the data consists of samples from both sexes, and that the Y chromosome expression is one of the main data features for which the data will cluster . Other wIn addition to label extraction from GEO, a recent study has provided a tool for label extraction from the Sequence Read Archive (SRA) metadata as well . The datA graphical overview of our algorithmic process is shown in Fig. Human gene expression data  were downloaded from GEO and values log transformed . Probes were collapsed to gene-level (Entrez Gene ID) by choosing the probe with the highest mean expression per gene, and normalized between arrays by quantile normalization. Imputation of missing values was done using k-nearest neighbors with k = 5. Metadata text for the downloaded GEO data was obtained from the GEOmetadb package [In terms of problem difficulty, we hypothesized that gender would be the simplest label to extract using text-mining methods, and would therefore yield the highest performance, because of the limited number of possible values and limited number of ways it can be described. Similarly, in GEO metadata, age is generally clearly signaled by an “Age:” prefix, but there is additional difficulty because a variety of units can be used , sometimes it is implicit in the wording ”), and occasionally units of age are misspelled. Tissue extraction should be the most difficult problem, because there are thousands of potential tissue types or cell lines, with a variety of synonyms.In order to extract labels from the unstructured metadata text, two broad approaches are used. For sex and age extraction, simple regular-expression based approaches are used, because these label types consist of a small vocabulary and the presentation of this type of data is relatively consistent within the metadata. For tissue extraction, a string-matching approach is used to map metadata to ontology term names and synonyms. For all problem types, the parsed “Characteristics” fields are searched first, because they contain attributes the experimenter has explicitly labeled as such, and if a match is found for that label-sample combination in the “Characteristics”, the search is terminated. Otherwise, it continues to the other fields in the order “Description”, “Source Name”, “Title”, and finally, all other fields.Heuristic extraction of gender was performed using regular expressions searching for patterns such as “gender: male”, “sex: M”, or “sex: 1” for numerical, abbreviations, or complete text encodings.Similarly, extraction of age was also performed using regular expressions; however, in this case, the regular expression also attempts to extract the units in text such as “age: 29 y”, “age (mo): 520”, etc. Where an age number was extracted, but not a unit, a default unit was assigned depending on the species. For humans, this is years, but the software can also extract labels for nonhuman species, although the current paper does not address this topic. In the case of rodents, for example, the default is months.To extract tissue types, the metadata was searched for term names or synonyms from the BRENDA Tissue Ontology (BTO)  using thIn addition to the extraction of labels from text, we perform inference of labels from gene expression. Because the problem types are different, we use different classifier/regression algorithms for each label type. The input data to the classifier is gene expression data normalized as described above, and the most informative 100 genes are selected to reduce dimensionality. To predict gender and tissue type, we used one-vs-rest (OVR) logistic regression.The choice between multi-class (one-to-one mapping) vs multi-label (one-to-many mapping) is important, as some samples may be comprised of multiple tissues, and therefore will be more suited to a multi-label approach. However, the benefit of using the BRENDA Tissue Ontology is that it contains nodes for tissue/cell types, such as blood, that are actually composed of multiple cell types, yet one node may imply many labels that are associated with the parent node within the ontology structure. This helps reduce the problem to a multi-class problem, wherein the single parent node may be used to retrieve any tissue belonging to this tree structure. Therefore, we utilize this ontological structure to reduce the problem to that of a multiclass problem and consider the tissue prediction in a multiclass context.VarianceThreshold function to exclude genes with the low expression variance, then f_classif was utilized to pick the top 100 F-scores among the remaining sites for downstream usage. Lastly, LogisticRegression was used to predict the probability a sample belonged to any given feature (such as tissue).Expression matrices were pre-processed using two feature selection methods, first using scitkitlearn’s To extract metadata labels from GEO, our system uses both the textual metadata provided by the investigator with each sample, and the sample’s expression data itself. It first applies pattern-matching algorithms to the investigator-supplied textual metadata to attempt to extract each label type . If this fails, either because the pattern-matching was insufficiently robust or because the investigator did not provide the label, the system uses a machine-learning classifier, trained on manually curated labels or labels extracted from pattern-matching, to predict the label from that particular sample’s gene expression vector itself, similar to ontology-mapping OVR SVM approaches used in previous works . At the We manually labeled 38,188 samples from GEO from human gene expression experiments with gender, age, and tissue (using the BRENDA Tissue Ontology ). This dThe evaluation of performance for each method was determined using manually curated gold standard labels, as described in the methods. To train the machine-learning classifiers, we used the labels extracted from the heuristic approach which were not part of the manually curated set, and tested the results against the manually curated labels, using 10-fold cross-validation. A summary of the labels extracted with the heuristic approach is given in Table The results demonstrating the performance of each label category are summarized in Fig. micro and macro averaged tissue to be similar, while the recall diverges. Micro-average precision and recall over the 108 tissue types which had at least 10 samples was 0.77 and 0.54, whereas within the top 25 most frequent gold-standard tissues, it was 0.85 and 0.75. Figure Tissue label extraction performance in Fig. The ML approach was able to predict gender from gene expression data with a high macro precision (0.915) and recall (0.917). Tissue was not as accurate, but during cross-validation on the top 25 most frequently occurring tissues, we achieved a macro precision of 0.70 and macro recall of 0.67. Micro-precision was 0.73 and micro-recall was 0.77. The macro AUC for ML classification was 0.83, contrasted with a baseline 0.49 for a naive “dummy classifier”, which makes predictions based purely on the distribution of training class labels (without using expression data).The extraction of labels from the textual metadata was performed using pattern matching, which is appropriate for tuning performance metrics towards high precision at the expense of recall. This choice allows for more certainty in the results, which are used as training data provided to the GEO expression machine learning algorithm. While the precision and recall within gender and tissue are within ~20%, the recall for age is noticeably lower than gender. We attribute this to a greater variation in the patterns used to describe age, and to the fact that the variable placement of the age unit often interferes with heuristic extraction.micro- and macro- precision and recall, whereby micro-averaging will be the overall precision/recall across all predictions, not taking into account label imbalance  annotated with the label type under query (gender or tissue), and b) expression data was available. Additionally, for tissue, evaluation was performed only on the top 10 most frequently occurring tissue types, as assessed during text-based label extraction. In total, cross-validation was performed on 10,129 samples for gender and 13,427 samples for tissue.The choice to evaluate tissue label predictions for the most frequently occurring tissue types is due to lower micro-precision and micro-recall (0.73-0.77) received for these performance metrics. The lower relative cross-validation performance on tissue likely reflects several issues. First, a multiclass problem with more classes is an inherently more difficult classification task. Secondly, our feature selection approach of selecting the most informative 100 genes for downstream training and classification may be more applicable to extraction of gender, which is reflected largely by expression of X and Y chromosome genes, than the case of tissue, as tissue-specific or tissue-predictive genes are more heterogeneous and scattered throughout the genome, therefore 100 genes may not be sufficient for this task. Finally, errors during text-based label extraction will be propagated to some extent to the classification stage. This last issue highlights the importance of attaining maximum accuracy during the initial text extraction phase to maximize performance on the downstream ML classification stage, and we would expect precision improvements in the text extraction method to result in improved ML classification performance.Although the current study is limited to a relatively constrained set of three types of labels, it lays the groundwork for a broader range of label type. For example, many other label types of interest, such as disease state, drug application, or diet, are specified in other ontologies, and our ontology-based approach could be applied with minimal modification to novel label types. Although we have not assessed performance on other label types, the code library for label extraction provides generic methods for extracting labels from any ontology in Open Biomedical Ontology (OBO) format.Similarly, although the current study is limited in scope to expression data from humans, the text-based approaches in particular may be suitable for future extraction of labels from other species and data types, because the format of metadata is relatively independent of the underlying type of high-throughput data in GEO or the species. However, in the process of developing this tool, we did anecdotally notice minor differences between metadata between species, which would require additional work to make the tool fully applicable to model organisms such as mouse. For example, while human ages are usually expressed in terms of years, there is much more variety in terms of the time unit used to express mouse age: days, weeks, and months are all common, making the accurate detection and conversion of time unit difficult but necessary to properly expand our tool to model organisms. We also observed that the distribution of tissue type is somewhat different in humans compared to model organisms, in the sense that tissue samples derived from non-invasive procedures (such as blood samples from a blood draw) are relatively more common in humans compared to tissues, such as brain, that can only be obtained from invasive procedures. These observations imply that evaluation metrics and extraction methods would likely require additional tuning to apply fully to model organisms.Previous work by other authors, e.g. , has levWe have presented a tool for the extraction of gender, tissue, and age labels for GEO data from the associated metadata, as well as a label prediction tool to probabilistically assign missing labels based on gene expression data. Broadly, we found that relatively simple heuristic text-extraction approaches based on regular expression and string matching can identify labels, especially those consisting of small vocabularies, such as age and gender, with high precision and moderate recall. The labels from heuristic extraction can then be used to provide larger training sets to train ML models on expression data to expand the range of samples that can be categorized, in case heuristic extraction fails, or the sample metadata does not contain the required information. Thus, by combining the two approaches, we retrieve accurate labels without sacrificing too many samples , greatly enhancing the ease by which large scale analysis can be performed using GEO data spanning across many studies. Another advantage of this two-pronged approach is that ML models can be trained on newly released transcriptional profiling platforms without the need for manual annotation of labels for each individual platform.


Stromal content heavily impacts the transcriptional classification of colorectal cancer (CRC), with clinical and biological implications. Lineage-dependent stromal transcriptional components could therefore dominate over more subtle expression traits inherent to cancer cells. Since in patient-derived xenografts (PDXs) stromal cells of the human tumour are substituted by murine counterparts, here we deploy human-specific expression profiling of CRC PDXs to assess cancer-cell intrinsic transcriptional features. Through this approach, we identify five CRC intrinsic subtypes (CRIS) endowed with distinctive molecular, functional and phenotypic peculiarities: (i) CRIS-A: mucinous, glycolytic, enriched for microsatellite instability or KRAS mutations; (ii) CRIS-B: TGF-β pathway activity, epithelial–mesenchymal transition, poor prognosis; (iii) CRIS-C: elevated EGFR signalling, sensitivity to EGFR inhibitors; (iv) CRIS-D: WNT activation, IGF2 gene overexpression and amplification; and (v) CRIS-E: Paneth cell-like phenotype, TP53 mutations. CRIS subtypes successfully categorize independent sets of primary and metastatic CRCs, with limited overlap on existing transcriptional classes and unprecedented predictive and prognostic performances. Stromal cells contribute to the gene expression profiles based on which colorectal cancer (CRC) molecular subtypes are classified. Here, patient-derived xenografts enable the authors to obtain cancer cell-specific transcriptomes by excluding transcripts from murine stromal cells, based on which they define CRC intrinsic subtypes (CRIS) and evaluate their prognostic and predictive potential. A number of classification systems based on gene expression have been proposed that stratify colorectal cancer (CRC) in subgroups with distinct molecular and clinical features12345689Interestingly, we and others independently reported that a large portion of the genes sustaining the SSM subtype (CMS4 within the CMS) are of stromal origin, and that the presence of stromal cells, mainly cancer-associated fibroblasts (CAFs), is a strong indicator of tumour aggressiveness8n=515 samples from 244 patients) of patient-derived xenografts (PDXs)13To tackle this issue, we exploited a large collection (n=450)12n=185) of CRC liver metastases (CRC-LM) (GSE73255) using the nearest template prediction (NTP) algorithmWe and others have recently reported that CRC classification based on published transcriptional signatures is heavily affected by the tumour stromal content8χ2-test assuming as null hypothesis that the fraction of cases classified as SSM in PDXs is not different from that obtained in the CRC-LM data set; P<5 × 10−10, χ2=41.084, for CCS3; P<1 × 10−4, χ2=15.688, for CRCA5; P<5 × 10−09, χ2=41.679, for CCMS4). SSM classes also demonstrated the lowest validation rate when comparing the classification of CRC-LM with that of their corresponding PDXs 8ing PDXs . We reasP<5 × 10−16, In line with previous genetic and phenotypic evidence12The tumour stroma certainly contributes to tumour biology but at the same time is not a direct expression of the transformed phenotype of neoplastic cells. Furthermore, classical histopathological studies have shown that many solid tumours (including CRC) feature a desmoplastic reaction whereby dense connective tissue, produced by activated fibroblasts, tends to prevail quantitatively over areas occupied by cancer cells18By applying non-negative matrix factorization (NMF)Because CRIS was derived from metastatic samples grown as xenografts in mice, we wished to verify whether CRIS signatures could be also applied to CRC gene expression profiles obtained in different conditions and by means of diverse technologies. To this aim, we initially tested the CRIS classifier against two independent gene expression data sets of primary CRCs, the previously mentioned 450-sample TCGA RNAseq data set811P<5 × 10−10, odds ratio 17.309, confidence interval (CI)=5.670–71.189; P<5 × 10−3, odds ratio=2.701, CI=1.459–5.038 for right-colon location; P<5 × 10−10, odds ratio=16.207, CI=5.751–56.814 for mucinous histology; P<1 × 10−11, odds ratio=27.254, CI=7.811–147.338 for CIMP; P<5 × 10−7, odds ratio=16.094, CI=4.388–89.516 for hypermutator phenotype; P<5 × 10−6, odds ratio=23.231, CI=5.318–144.640 for mucinous histology and P<5 × 10−3, odds ratio=10.417, CI=2.031–69.611 for CIMP; Using NTP-calculated centroid distances in the TCGA data set , CRIS clP<5 × 10−08; MYC proto-oncogene  and two to CRIS-B. Within MSS samples, KRAS was frequently mutated in CRIS-A  and, within the CRIS C–D–E subfamily, specifically in CRIS-E  against the ‘Hallmark’ gene sets, a panel of signatures representing an extensive spectrum of key biological functions, from the MSigDB ref.  Fig. 4e. On the KRAS-mutated tumours30IGF2 amplification observed in the TCGA data set) and FGFR activity =2.961, CI=1.615–5.43; P<5 × 10−4; χ2, P<0.05, HR=2.674, CI=1.045–6.845; 8χ2, P<0.005, HR=3.042, CI=1.366–6.777; To explore CRIS prognostic impact we examined the association of CRIS subtypes with disease-free survival (DFS) of 290 patients included in a publicly available and clinically annotated CRC data setχ2, P<5 × 10−3, HR=2.707, CI=1.384–5.295; χ2, P<5 × 10−6, HR=5.216, CI=2.486–10.95; χ2, P<5 × 10−5, HR=3.248, CI=1.843–5.724; χ2, P<5 × 10−5, HR=5.736, CI=2.334–14.1; To evaluate possible prognostic interactions between the CRIS-B subtype and high levels of CAF infiltration, we stratified tumours with high or low CAF content, as previously estimated in this data setAmong those individuals who did not receive adjuvant treatment based on clinical and pathological parameters, the integrative deployment of CRIS classes and CAF score was able to discriminate a relatively large subset (30%) of poor-prognosis patients who relapsed in 5 years in more than 40% of cases (sensitivity of 0.68 and specificity of 0.75 for 5-year DFS). The identification of patients with tumours that, in spite of displaying ‘benign’ clinical and pathological features, appear to have an aggressive biological behaviour, may prove useful to inform a more proactive follow up in such high-risk subpopulation.χ2, P<1 × 10−7, HR=1.777, CI=1.433–2.204; Importantly, the association between CRIS-B and poor prognosis was confirmed in an independent cohort of 1,261 samples, which was assembled by combining data from five independent data sets . Also inAs an initial attempt to translate the CRIS taxonomy into a diagnostic tool amenable to clinical applications, we developed a single-sample classifier based on the top scoring pair algorithm (TSP)4243K=0.7384). CRIS-TSP was then applied to eight independent data sets, for a total of 2,024 samples . We also found limited concordance when comparing CRIS-TSP classification of surgical resections of liver metastases versus their corresponding PDXs (K=0.4053), overall suggesting that the performance of the CRIS-TSP classifier was less robust when applied outside the training set. To assess whether such reduced classification coherence was due to low classification efficacy of the selected TSP genes, we reconstituted a NTP-based classifier using the same TSP transcripts . Consistently, the classification coherence between original tumours and PDXs was also increased (K=0.5577) and similar to that obtained with the full-size NTP. CRIS-NTP80 also confirmed the prognostic significance of CRIS-B  and detection of genetic and functional peculiarities with a potential to instruct novel diagnostic and therapeutic approaches.BRAF-mutated MSI tumours and KRAS-mutated MSS tumours, for which no targeted treatment options are currently available. It is worth noting that such tumours typically exhibit strong glycolytic/hypoxic signatures. Although further studies based on preclinical experimentation and prospective trials in patients are needed to support this assumption, CRIS-A might pinpoint tumour subgroups potentially responsive to anti-metabolic therapiesIGF2 overexpression, which has been recently implicated in desensitization to EGFR blockade in patients with KRAS wild-type tumoursIGF2 levels attenuate dependency on the EGFR pathway underscores the functional relevance of this alteration, which is also a candidate target for alternative treatment protocolsKRAS-mutated, Paneth cell-like CIN tumours that are refractory to treatment with anti-EGFR antibodies. High WNT pathway activity was more generally observed in the CRIS-C–D–E subfamily, thus defining a subset of tumours for which pharmacologic inhibitors of this pathway48CRIS-A is highly enriched for Although the analysis of cancer-cell intrinsic traits can provide relevant information for CRC management, the contribution of the stromal compartment should not be overlooked. We and others have reported that the extent of stromal infiltration predicts poor outcome, resistance to radiotherapy and—possibly—sensitivity to chemotherapy8One potential limitation of our study is that some CRIS features could in fact emerge as a consequence of tumour xenotransplantation. In principle, the PDX approach might exert a number of distortive effects on the transcriptome of cancer cells, including selection drifts related to engraftment and propagation, limited cross-species reactivity between human and mouse cytokines with consequent perturbation of paracrine signals, and lack of proper immune components in recipient animals. However, the likelihood of a strong impact of such biases on the CRIS taxonomy is reduced by the observation that CRIS efficiently classified several data sets from bulk CRC patient tumours, regardless of their source of origin (primary or metastatic). A way to conclusively cope with this issue would be to exploit alternative methods to gather pure cancer cell transcriptional profiles from patient tumours, and test whether the key CRIS features remain valid. Such kind of approaches—mainly based on cell sorting of dissociated tumours or microdissection of histological specimens—have been already applied in small-scale efforts505152In conclusion, our data advocate the integrative exploitation of independently assessed cancer-cell intrinsic and stromal CRC transcriptional components as a key opportunity to improve patients’ management in the context of precision medicine approaches. Similar findings are beginning to emerge also in other tumour types, using different methodologiesA total of 244 tumour samples and matched normal samples were obtained from patients who had undergone surgical resection of liver metastases at the Candiolo Cancer Institute, the Mauriziano Umberto I Hospital and the San Giovanni Battista Hospital . All patients provided informed consent and study approval was obtained from the review boards of the three institutions .In vivo experiments and related biobanking data were stored in the Laboratory Assistant Suite, a web-based, in-house developed data management system for automated data trackingEach collected sample was fragmented and either frozen or prepared for implantation subcutis as previously described1212RNA was extracted using the miRNeasy Mini Kit (Qiagen), according to the manufacturer’s protocol. Synthesis of cDNA and biotinylated cRNA  was performed using the IlluminaTotalPrep RNA Amplification Kit (Ambion), according to the manufacturer’s protocol. Quality assessment and quantitation of total RNA and cRNAs were performed with Agilent RNA kits on a Bioanalyzer 2100 (Agilent). Hybridization of cRNAs (750 ng) was carried out using Illumina Human 48 k gene chips (Human HT-12 V4 BeadChip). Array washing was performed by Illumina High Temp Wash Buffer for 10 min at 55 °C, followed by staining using streptavidin-Cy3 dyes (Amersham Biosciences). Hybridized arrays were stained and scanned in a Beadstation 500 (Illumina).For GSE76402, probe intensity data were extracted using the Illumina Genome Studio software (Genome Studio V2011.1) and subjected to Loess normalization using the Lumi R package57P value=0) in at least 10% of the samples. For each of such genes, only the probe with the highest variance of signal was selected. Similar analyses were carried out for the panel of liver metastatic CRCs composing GSE73255. The panel included a total of 185 samples corresponding to 167 unique patients.For the remaining 515 samples—derived from 244 unique patients—probes were filtered to select those that showed detectable signal  data available from the TCGA data portal. In particular the READ and COAD data sets were mined. All the somatic mutations were included in the calculation and normalized assuming an approximate exome size of 30 megabases. The analysis was carried out with the SomaticSignatures R packagehttp://pandas.pydata.org/; http://docs.scipy.org/doc/scipy/reference/).To calculate the copy number variation load we took advantage of the segmented data provided by the Broad Institute and available from the TCGA data portal (Genome-Wide Human SNP Array 6.0). Only data unambiguously referred to unique tumour samples were considered. For each sample, all the regions with an absolute segmented value greater than 0.3 were categorized as altered. This threshold was chosen based on previous work, in which standard methods for calling a copy number alteration for a segment in GISTIC analysis of a single sample were definedTo represent gains and losses of whole chromosome arms, we employed the GISTIChttp://gdac.broadinstitute.org)  .Affymetrix gene expression profiles of 80 metastaticet al.1N . NMF was performed with the predetermined number of clusters (K) varying from 2 to 6. Quality of the clustering was evaluated by the cophenetic coefficient, with K=5 gathering the highest value and K=2, K=3 displaying similarly high coefficients (K=5 was selected because it (i) demonstrated higher subtyping resolution, splitting K=2 and K=3 in smaller and more homogeneous subclusters clusters .n=90; To identify gene signatures able to discriminate CRIS subtypes, we first focused on the samples that best represented each subtype, using silhouette width—as previously published1For implementation of an NTP-based classifier, we selected genes positively and specifically associated to each of the subtypes. Indeed, the PAM score represents the extent and sign of association of each gene to each class. Starting from the 903 genes selected as specified above, 102 genes did not have a positive PAM score for any of the classes (as a consequence of the centroid shrinkage procedure) and could not be used for NTP. Then, we deployed our published methodologyet al.et al.NTP-based classificationTo develop a simplified classification system for CRIS, we used the TSP approach, a rank-based, parameter-free binary predictor relying on the relative ordering of two features , and its extension, the k-TSP classifier, which aggregates the votes of multiple TSPs and can be used for multiclass problems414243To this end we first identified candidate genes for classifier development starting from 526 CRIS genes out of 565) in common across three distinct data sets obtained from different platforms: 254 PDXs analysed on Illumina microarrays, 104 RNA-seq samples from TCGA and 266 samples analysed on Affymetrix microarrays available from the public domain , for a total of 624 samples . We then in commoThe TSP algorithm assigns a sample to a specific phenotype if gene A is larger than gene B, or to the other phenotype otherwise. Since there are five CRIS classes, there are ten possible pair-wise comparisons among the CRIS classes: (1) A versus B; (2) A versus C; (3) A versus D; (4) A versus E; (5) B versus C; (6) B versus D; (7) B versus E; (8) C versus D; (9) C versus E; and (10) D versus E.There are 138,075 possible TSPs that can be formed using all combinations of 526 genes. To avoid over-fitting, however, we limited the search space in the training phase by filtering out all genes that proved to be irreproducible across the three analytical platforms considered . To this end, we used the MergeMaid R-package to calculate a gene reproducibility index called ICOR65There are still 35,778 possible TSPs that can be formed using all combinations of the 268 most reproducible genes. Hence, to further avoid over-fitting, we limited the search space in the training phase by selecting only the top 50 most differentially expressed genes for each class (if available) using a Wilcoxon rank-sum test, and then by constraining pairing only between genes belonging to the CRIS classes being considered .We then identified k-TSPs for each of the ten pair-wise comparison between the CRIS classes. To select disjoint TSPs for each class comparison, the genes used to form pairs were omitted from the search in subsequent comparisons. In selecting the most discriminative TSPs we started from the comparisons between CRIS-B and the other classes, since this class showed prognostic value in our previous analyses. We then proceeded to compare CRIS-C to the other classes, since this class showed association with EGFR signalling and cetuximab sensitivity in our previous analyses. We then proceeded with the remaining class comparisons according to the total number of available genes to form the pairs, in increasing order. The order of the search was therefore as follows: CRIS-B versus E, versus D, versus C and versus A; CRIS-C versus E, versus D and versus A; CRIS-E versus D and versus A; CRIS-D versus A. For each of the ten pair-wise comparisons we selected from 1 to 5 TSPs, for a total of 10, 20, 30, 40 and 50 disjoint TSPs, using a total of 20, 40, 60, 80 and 100 non-overlapping genes, respectively.The classification rule to assign a new sample to a CRIS class was constructed as follows: (i) classify each sample according to each individual TSP included in the k-TSP classifier and available for analysis ; (ii) count the proportion of votes for each CRIS class ; (iii) assign each sample to the class with the highest proportion of votes; (iv) in case of ties between three classes or more, do not classify sample (‘Unassigned’); (v) in case of ties between two classes, assign samples to the class with the highest proportion of votes in the pair-wise comparisons between these two classes; do not classify sample otherwise (‘Unassigned’).k=1), then incremented the number of TSPs until reaching the maximum agreement with NTP. Hence, we developed our kTSP classifier using 10, 20, 30, 40 and 50 non-overlapping TSPs for a total of 20, 40, 60, 80 and 100 genes, respectively. We then compared the agreement of the aggregated k-TSP classifier with NTP classification in the training set and internal testing set and selected the one showing the highest agreement in the internal testing set , as shown in In the training phase on 2/3 of the data, we evaluated the concordance between the classification obtained using the k-TSP classifier and the reference NTP classification based on the complete 526 ‘intrinsic’ gene set. We assessed the agreement starting from the minimum set of ten TSPs  to independent tumour data sets are provided with the CRISclassifier R package, available as a http://www.broadinstitute.org/cancer/software/genepattern/).The similarity between transcriptional traits underlying subtypes across different data sets was evaluated using the SubMap R packagehttp://scikit-learn.org/stable/). Enrichment analyses were conducted on the first principal component  was employed to identify the main sources of variation in the TCGA gene expression data set. In particular, we exploited the PCA class from Scikit-learn python library  in the PDX data set were used to calculate Pearson’s correlations for any possible CRC-LM/PDX permutation. Correlations of matched versus unmatched CRC-LM/PDX pairs were compared by two-tailed Student’s http://www.broadinstitute.org/gsea/index.jsp. The significance of enrichment was estimated using default settings and 1,000 gene permutations28The GSEA software was downloaded from the For SSEA of curated functional signatures, a score was calculated for each signature and each sample using median-centered Log2 ratios of gene expression values, as follows:j.where ModuleUP is the average expression value of the genes positively correlated with the phenotype and ModuleDOWN is the average expression value of the transcripts anti-correlated with the phenotype in sample We then evaluated the enrichment in class assignment for each CRIS class by performing GSEA preranked analysis using as ranked lists the samples ordered by the score of interest, and as sets the lists of sample membership to the different CRIS subtypes. Calculations were done with 1,000 permutations.http://dip.doe-mbi.ucla.edu/dip/DLRP.cgi). Then, each set of genes encoding the receptor and its ligands was used to rank samples, based on median-centered Log2 ratios of gene expression values, as follows:For SSEA of autocrine loops, a total of 472 receptor/ligand interactions were downloaded from the Database of Ligand Receptor Partners  and GSE73255 .http://bioconductor.org/packages/release/data/experiment/html/TCGAcrcmRNA.html); from the GEO database  and from the Synapse data portal from the Colorectal Cancer Molecular Subtyping Consortium .Other gene-expression data that support the findings of this study are available through the TCGA data portal .Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.Supplementary figures and supplementary tables.TCGA datasetPrincipal component analysis on the TCGA datasetClassification of the CRC-LM dataset according to three public transcriptional classifiersCRC PDX datasetClassification of the CRC PDX dataset according to three public transcriptional classifiersCorrelation between PDX and CRC-LM gene expression profilesIdentification of CRC subtypes in the PDX dataset and NTP-based CRIS classificationCRIS ClassifierClassification of the CRC-LM dataset according to CRISGSE14333, a clinically annotated dataset of primary CRC tumorsCRIS classification of public CRC gene expression datasetsGSEA of hallmark gene sets in CRIS classesSample set enrichment analysis (SSEA) of curated signatures' expression across CRIS classesSample set enrichment analysis of ligands/receptor pairs' expression in CRIS classes80 CRC liver metastases annotated for clinical response to cetuximab monotherapyClinical annotation of public gene expression datasets of CRC primary tumorsCRIS-TSP and CRIS-NTP80 classification on CRC samplesGene pairs for CRIS-TSP and CRIS-NTP80 classifiersThe CRISclassifier, an R-Bioconductor package to classify independent gene expression datasets according to either CRIS-NTP or CRIS-TSP algorithms