RATs, an R package that identifies DTU transcriptome-wide directly from transcript abundance estimates.RATs is unique in applying bootstrapping to estimate the reliability of detected DTU events and shows good performance at all replication levels . We compareRATs to two existing DTU tools,DRIM-Seq &SUPPA2, using two publicly available simulated RNA-seq datasets and a published human RNA-seq dataset, in which 248 genes have been previously identified as displaying significant DTU. RATs with default threshold values on the simulated Human data has a sensitivity of 0.55, a Matthews correlation coefficient of 0.71 and a false discovery rate (FDR) of 0.04, outperforming both other tools. Applying the same thresholds forSUPPA2 results in a higher sensitivity (0.61) but poorer FDR performance (0.33). RATs and DRIM-seq use different methods for measuring DTU effect-sizes complicating the comparison of results between these tools, however, for a likelihood-ratio threshold of 30,DRIM-Seq has similar FDR performance toRATs (0.06), but worse sensitivity (0.47). These differences persist for the simulated drosophila dataset. On the published human RNA-seq dataset the greatest agreement between the tools tested is 53%, observed betweenRATs andSUPPA2. The bootstrapping quality filter inRATs is responsible for removing the majority of DTU events called bySUPPA2 that are not reported byRATs. All methods, including the previously published qRT-PCR of three of the 248 detected DTU events, were found to be sensitive to annotation differences between Ensembl v60 and v87.The biological importance of changes in RNA expression is reflected by the wide variety of tools available to characterise these changes from RNA-seq data. Several tools exist for detecting differential transcript isoform usage (DTU) from aligned or assembled RNA-seq data, but few exist for DTU detection from alignment-free RNA-seq quantifications. We present the However, in higher eukaryotes, alternative splicing of multi-exon genes and/or alternative transcript start and end sites leads to multiple transcript isoforms originating from each gene. Since transcripts represent the executive form of genetic information, analysis of differential transcript expression (DTE) is preferable to DGE. Unfortunately, isoform-level transcriptome analysis is more complex and expensive since, in order to achieve similar statistical power in a DTE study, higher sequencing depth is required to compensate for the expression of each gene being split among its component isoforms. In addition, isoforms of a gene share high sequence similarity and this complicates the attribution of reads among them. Despite these challenges, several studies have shown that isoforms have distinct functionsIt is possible to find significant DTE among the isoforms of a gene, even when the gene shows no significant DGE. This introduces the concept of differential transcript usage (DTU), where the abundances of individual isoforms of a gene can change relative to one another, with the most pronounced examples resulting in a change of the dominant isoform (isoform switching). The definitions of DGE, DTE and DTU are illustrated inCufflinks8,DEXSeq9,LeafCutter10) rely on reads that either span splice-junctions or align to unique exons. However, with the newest generation of transcript quantification tools , reads are aligned to neither the transcriptome nor the genome. Instead, these tools combine a pseudo-mapping of the k-mers present within each read to the k-mer distributions from the transcriptome annotation with an expectation maximization algorithm, to infer the expression of each transcript model directly. Such alignment-free methods are much faster than the traditional alignment-based methods  or assembly-based methods , making it feasible to repeat the process many times on iterative subsets of the read data and, thus, quantify the technical variance in the transcript abundance estimates. However, the lack of alignments prevents these new methods from being compatible with differential expression methods such asCufflinks,DEXSeq andLeafcutter. Instead,Sleuth19 is a tool that handles DTE analysis from alignment-free transcript quantifications. DTU analysis is currently less straight-forward.SwitchSeq20 focuses on a particular subset of DTU analysis from alignment-free data, namely isoform switching, whereasiso-kTSP6 identifies both DTU and isoform switching, but focuses on the highest-ranking pair of change-exhibiting isoforms per gene.SUPPA22, on the other hand, primarily identifies differential splicing events at the junction level, with recent developmental versions having added isoform-level capability. Finally,DRIM-Seq23 identifies DTU directly from quantification data, but defines the effect size as a fold change which may not be the most appropriate way to compare proportions.To quantify the isoforms and assess changes in their abundance, most existing tools for DTE and DTU analysis (e.g.RATs (Relative Abundance of Transcripts), an R package for identifying DTU directly from isoform quantifications. It is designed to use alignment-free abundance data and is the only tool that exploits bootstrapping to assess the robustness of the DTU calls.RATs provides raw, summary and graphical results, allowing for ease of use as well as for advanced custom queries, and the R language is the environment of choice for many widely-used DGE and DTE tools, allowing for easy integration ofRATs in existing workflows. We assess the accuracy ofRATs in comparison toSUPPA2 andDRIM-Seq and findRATs to perform at as well as or better than its competitors. Finally, we demonstrate that the results of both RNA-seq based and qRT-PCR based analyses are sensitive to the annotation used for transcript quantification and primer design, respectively.In this paper, we presentRATs identifies DTU independently at both the gene and transcript levels using an efficient implementation of the G-test of independence24, without continuity corrections. The criteriaRATs uses to identify DTU are described in detail below.RATs first filters the input isoform abundance data to reduce both the number of low quality calls and the number of tests carried out. Specifically: (i) isoform ratio changes can only be defined for genes that are expressed in both conditions, with at least two isoforms detected, and (ii) transcript abundances must exceed an optional minimum abundance threshold. Transcripts with abundances below the threshold are considered as not detected.Prior to statistical testing by either method,RATs compares the set of each gene’s isoform abundances between the two conditions to identify if the abundance ratios have changed. At the transcript level,RATs compares the abundance of each individual transcript against the pooled abundance of its sibling isoforms to identify changes in the proportion of the gene’s expression attributable to that specific transcript. Both methods include the Benjamini-Hochberg false discovery rate correction for multiple testing25. These tests are performed on the summed abundance of each isoform across the replicates.Significant changes in relative transcript abundance are detected using two separate approaches: one at the gene level and the other at the transcript level. At the gene level,Transcripts whose absolute difference in isoform proportion is below a set threshold are rejected, even if the difference is statistically significant.RATs provides the option to use the bootstrapped abundance estimates obtainable from alignment-free quantification tools to apply a reproducibility constraint on the DTU calls, by randomly selecting individual quantification iterations from each replicate and measuring the fraction of these iterations that result in a positive DTU classification. Typically, each sample is represented by the mean abundance of each transcript, calculated across the quantification iterations. However, this loses the variance information of the quantification. By referring back to the quantification iterations,RATs highlights cases where the quantification was unreliable due to high variability and therefore the DTU result should also be considered unreliable. Similarly,RATs optionally also measures the reproducibility of the DTU results relative to the inter-replicate variation by iteratively sub-setting the samples pool.RATs is implemented inR26 and has been freely distributed throughGithub as an R source package since August 2016.RATs accepts as input either a set of R tables with abundances (with or without bootstrap information), or a set ofSalmon14 orKallisto11 output files. An annotation table mapping the correspondence between transcript and gene identifiers is also required, either provided directly or inferred from a GTF file. Results are returned in the form of Rdata.table objects27. Along with the DTU calls per transcript and gene, the tables record the full provenance of the results. Convenience functions are provided for summary tallies of DTU and isoform-switching results, for ID retrieval, and for visualization of the results viaggplot2 (v2.2.1)28. Details on these are available through the user manual of the package. Once created, all plots produced byRATs remain customisable via standard ggplot2 operations.RATs (v0.6.2) for detection of DTU between two groups relative to the level of experimental replication was measured on groups generated by random selection without replacement from a pool of 16 high-quality wild-type Colombia-0Arabidopsis thaliana replicates291. This was iterated 100 times for each replication level in the range 3 ≤ n ≤ 8. As the two groups are drawn from the same condition, any positive DTU calls must be considered to be false positives. For each iteration, we recorded the fraction of genes and transcripts that were reported as DTU, relative to the total number of genes or transcripts tested in that iteration. The commands and scripts used are from theRATs Github repository.The performance was assessed in two ways. Firstly, the false positives (FP) performance of30 were used to benchmark the sensitivity , false discovery rate  and Matthews correlation coefficient (MCC) ofRATs,SUPPA2 andDRIM-Seq. The datasets were made of simulated RNA-seq reads based on the transcriptome annotation and to match realistic RNA-seq transcript expression values. To create the second condition, the abundance values of the two most abundant transcript isoforms originating from a gene locus were swapped for 1000 well-expressed coding gene loci. The transcriptome annotation used for both Human and fly comprised only annotated protein coding genes  leaving a number of other classifications of gene unaccounted for . These genes constitute a convenient negative set for simulation and should have no expression, save for any reads misallocated to them by the quantification tools. The simulated datasets were obtained from ArrayExpress2 and quantified with bothKallisto . The sensitivity, FDR and MCC were measured for a range of comparable parameters betweenRATs (v0.6.4),SUPPA2 (v2.3) andDRIM-Seq . No transcript abundance pre-filter was imposed on any of the three DTU tools, and the significance level was set to 0.05 for all runs. ForRATs andSUPPA2, three thresholds for the effect size (difference in proportion) were tested; theRATs’ current default of 0.2, and more permissive values 0.1 & 0.05. ForDRIM-Seq, threshold values of the likelihood ratio were explored from 0-30. Finally,RATs reproducibility thresholds were explored in the range of 0.8-0.95 for the quantification reproducibility and 0.55-0.85 for the inter-replicate reproducibility. The tool performance was measured using annotations comprised of all annotated genes and only protein coding genes.Secondly, two simulated datasetsRATs to identify known instances of DTU, we compared it against validated instances of DTU from publicly available RNA-seq data. We took read data from Denget al.  to quantify the isoform abundances. DTU was identified byRATs v0.6.2. For comparison, we repeated the quantification and DTU analysis of the data with the same tool versions and parameters, but using the annotation and assembly from Ensembl v87, the current version at the time of this study.To test the ability ofl. 2013,, who ideSUPPA2, in itspsiPerIsoform mode, and toDRIM-Seq. For a fair comparison, we tried to minimize variability in the parameters and data type used by the three tools. AsSUPPA2 offered no abundance pre-filtering,RATs andDRIM-Seq were run with abundance threshold values of 0. The p-value cut-off was set at 0.05 for all three tools, using the corrected p-values where available. For the difference in isoform proportion (SUPPA2 andRATs) the threshold was set at 0.20. No threshold was set for the fold-changes inDRIM-Seq.SUPPA2 required and was provided with TPM abundances. For consistency in the use of abundances normalised for transcript length,RATs andDRIM-Seq were also provided with TPM, but the values were scaled up to the average library size of 25M reads, as their testing methods expect counts and would be under-powered if used directly with TPMs. Again, the commands and scripts used are available from theRATs Github repository.We also submitted the quantification data toRATs achieved a median FP fraction <0.05 on ourA. thaliana dataset, even with only three replicates per condition  result in a sensitivity ofs = 0.55, MCC = 0.71 and FDR = 0.04, outperforming both other tools. With the same thresholds,SUPPA2 has a higher sensitivity (s = 0.61) but poorer FDR performance (FDR = 0.33). Direct comparison withDRIM-Seq is complicated by different methods for measuring DTU effect-sizes between the tools, however for a likelihood-ratio threshold of 30,DRIM-Seq has similar FDR performance toRATs (FDR = 0.06), but worse sensitivity (s = 0.47). These differences persist for the simulated drosophila dataset.DRIM-Seq consistently shows the lowest sensitivity (≤0.65), while maintaining a FDR ≤0.2 in any of the tried parameter sets.SUPPA2 is the most sensitive of the three tools (0.6 ≤s ≤ 0.9), but also has the highest FDR .RATs can match the sensitivity ofSUPPA2 while maintaining a lower FDR thanSUPPA2 by relaxing its quantification reproducibility (Qrep) and inter-replicate reproducibility (Rrep) thresholds. At the highest effect-size thresholds (DpropRATs = 0.2 andDRIM-Seqlr = 0.3)DRIM-Seq has a comparable FDR to that ofRATs. Surprisingly, the sensitivity, MCC and FDR ofDRIM-Seq is not strongly sensitive to variations in the likelihood ratio effect-size threshold. Consequentially, RATs has worse FDR performance, but better sensitivity thanDRIM-Seq at lower effect-size thresholds. Across all the simulated dataset and parameter combinations the gene-level test implemented inRATs shows higher sensitivity and higher FDR compared with the results from the transcript-level test. Extending the test to isoforms from the full set of annotated genes, rather than only those from protein coding genes, adds a considerable number of additional true negatives  resulting in a small increase of FDR and slight reduction of MCC for all tools in both datasets (Extended data 233). Similarly, usingKallisto isoform expression quantifications in place of the quantifications fromSalmon does not strongly affect the results (Extended data 233). The performance results of RATs on these simulated datasets are in good agreement with those presented in Loveet al. , which also demonstrates that the performance of RATs is similar to, or exceeds, the performace of other DTU tools , including DRIM-seq, SUPPA2 or DEX-Seq.The sensitivity, FDR and MCC performance ofet al. , CMTM4 (ENSG00000183723), and PEX11B (ENSG00000131779).RATs,SUPPA2 andDRIM-Seq, as well as each tool’s verdict on each of the three validated genes. The genes reported as DTU byRATs are listed in Extended data 3 & 433 respectively, based on the Ensembl v60 and v87 human annotations.After pre-filtering, Dengl. 2013, tested 3DRIM-Seq possibly due to a tendency to over-predict (see next section). Of the three validated genes, only CMTM4 is reported by all methods, and onlySUPPA2 reports all three genes. Although the rejection of TOM1L1 and PEX11B byDRIM-Seq was due to poor statistical significance,RATs reported that the changes found were both statistically significant and of sufficient effect size. Instead,RATs rejected the genes on the grounds of poor reproducibility .None of the three tools recapitulated the reported 248 genes well, with the highest fraction of 26% achieved byet al. . Of the three validated genes, TOM1L1 was unanimously rejected by all methods, CMTM4 remained unanimously reported as DTU, and PEX11B was reported as DTU byRATs andSUPPA2, but not byDRIM-Seq.There have been extensive changes in the human transcriptome annotation since Ensembl v60. We hypothesized that these changes could have a significant impact on the set of genes identified in Dengl. 2013,.Table 231. Indeed, when the reported qRT-PCR primers were aligned to the Ensembl v87 sequence and annotation (see Extended data 633), only the primers for PEX11B yielded the same conclusion as with Ensembl v60. For TOM1L1, the primers intended for ENST00000445275 no longer matched that isoform, but matched two other isoforms instead (ENST00000570371 and ENST00000575882). Additionally, the primers intended to quantify the gene as a whole failed to match half of the gene’s new isoforms, and the two sets of captured isoforms did not overlap completely and were thus incomparable in any meaningful way. As a consequence, the qRT-PCR intensities measured in the original study are actually impossible to interpret in the context of the updated annotation and the originally reported conclusion is likely wrong. For CMTM4 the primers reported matched multiple but not all isoforms, casting doubt on the interpretation of the qRT-PCR measurements for this gene as well. Only for PEX11B did the primers target the isoforms in a way that would give interpretable results and indeed lead to the same conclusion as originally reported31.The isoform abundances inRATs,SUPPA2 andDRIM-Seq for the Denget al. , is responsible for rejecting 43% of theSUPPA2 DTU transcripts and 28% of theDRIM-Seq DTU genes that pass the significance and effect size filtering criteria. 53% of theDRIM-Seq results and, perplexingly, 18% of theSUPPA2 results are rejected due to the effect size filter , despite all the tools operating on the same input isoform quantifications.RATs’ runtime and memory consumption depend on the size of the annotation and the number of bootstraps iterations. Where multiple processing cores are available,RATs can be instructed to take advantage of them. The runtime and maximum memory usage for the two simulated datasets from our benchmarks, running on a high-specification laptop, are shown inRATs is the first differential isoform usage tool to include the reproducibility of the upstream isoform expression quantifications to refine its DTU identifications, directly addressing the accuracy of the upstream isoform expression quantifications. Leveraging the bootstrapped isoform expression quantifications from fast modern alignment-free isoform expression quantification tools  allowsRATs to reject those cases of DTU that are based on highly uncertain isoform quantifications. Existing tools rely on the mean isoform abundances, which can hide a large degree of variability, and are thus insensitive to this reproducibility criterion. We recommend runningRATs, and the underlying alignment-free isoform expression quantification tools that generate the data it operates on, with at least 100 bootstrap iterations.Reliable identification of differential isoform usage depends critically on i) the accuracy of the upstream isoform expression quantifications, and ii) on the accuracy of the annotation they use.RATs on both simulated data and on a high-quality experimental dataset from Denget al. . Comparing the DTU classifications of the three tools against the instances of DTU identified in the Denget al. . RunningRATs,SUPPA2, andDRIM-Seq on the Denget al.  show that pre-filtering annotations can improve quantification performance and this approach may also be helpful in qRT-PCR primer design.We evaluatedl. 2013, and showl. 2013, dataset,l. 2013, datasetsl. 2013, study inPacBio sequencing orOxford Nanopore Direct RNA-seq. An advantage of this approach is that it would better define the transcriptome for the specific experiment41. This may be of importance for experiments focusing on specific tissues or developmental stages of an organism, where the active transcriptome is likely to be only a subset of the global reference transcriptome of the organism.In the future, experiment-specific transcriptome annotations could be obtained by including a parallel set of full-length isoform RNA-seq data in the experimental design, such as viaArabidopsis thaliana RNA-sequencing data used in this study is available from ArrayExpress under the studyE-MTAB-5446. The simulatedHomo sapiens andDrosophila Melanogaster datasets are available from ArrayExpress under the studyE-MTAB-3766. The Denget al.  The methods are not sufficiently described, I have the following questions:What is the input to RATs? Is it TPM or counts or scaledTPM? Should the library size differences be removed prior to providing to RATs or does RATs take care of library size differences internally?Can the methods described all analyze the same type of experiment, are they all restricted to two-group analyses? Can any of them control for batch effects?What are the default pre-filtering and post-filtering settings? What is the default minimum abundance threshold or proportion threshold for an isoform to be considered expressed? What is the default effect size cutoff, and how is it implemented per isoform, per gene? What is the default fraction for determining that evidence of DTU is not substantiated across inferential replicates? Likewise, what default fraction for biological replicate variation?2) I didn't understand why abundance thresholds were not used, as described here, "No transcript abundance pre-filter was imposed on any of the three DTU tools," and also "As SUPPA2 offered no abundance pre-filtering, RATs and DRIM-Seq were run with abundance threshold values of 0." et al. (20161) and Loveet al. (20182), performance of a number of DTU methods is greatly improved by filtering out lowly expressed transcripts. It can be inferred from the title of the former paper: "Isoform prefiltering improves performance of count-based methods for analysis of differential transcript usage".  As shown in Sonesonet al. (20182): SUPPA2 does have an abundance pre-filtering option, which was used in LoveWe enabled a filter to remove transcripts with less than 1 TPM. TPM filtering is a command-line option available during the diffSplice step of SUPPA2 and this greatly improved the running time without loss of sensitivity". "-th | --tpm-threshold: Minimum expression  to be included in the analysis. (Default: 0)." From the SUPPA2 manual: " Given that all the methods have abundance and/or proportion filters available, that filters are recommended by at least two of the three methods in their documentation (DRIMSeq and RATs), and at least two independent review papers (not introducing methods) have shown that abundance and/or proportion filtering improves performance of methods, I can't see why the choice was made to not use filters.3) It is mentioned in the false positive analysis that the median FP fraction was less than 0.05 and a horizontal line is drawn on Fig 2A and B. This is misleading, as the adjusted p-values are being thresholded at 0.05 (I assume), and in a null comparison the rate of false positives from an adjusted p-values should be 0, not 0.05. Drawing or mentioning a 0.05 cutoff would be relevant for the p-values (uncorrected), but has no bearing on the adjusted p-values. This may confuse readers.4) The authors repeatedly refer to the reported effect size in DRIMSeq being an issue for comparison across methods, e.g. "Direct comparison with DRIM-Seq is complicated by different methods for measuring DTU effect-sizes between the tools", but this is only an issue to the extent that the authors wish to perform post-hoc filtering on effect size. It is not an issue for null hypothesis testing without post-hoc filtering, because all methods are testing against the null that the underlying proportion of expression across isoforms has the same distribution for control samples and treated samples. However, I agree that for post-hoc filtering, one may want to filter the methods in a similar manner. It should be easy to filter the DRIMSeq results directly on absolute difference in isoform proportion, for example in Loveet al. (20182) we performed post-hoc filtering for DRIMSeq on the SD of proportions across all samples using a 6-line R function. As the likelihood ratio statistic should be 1-1 and monotonic with the p-value for DRIMSeq (if the degrees of freedom is constant across genes or transcripts), then I would not compare effect size filtering with likelihood ratio filtering, as the latter is simply filtering the p-value at a lower threshold.Minor comments:there is little justification for choosing DGE over DTE in the study of complex transcriptomes". The authors imply that gene-level and transcript-level analysis are mutually exclusive analyses, when they are not, and so I would suggest to reword or reconsider this statement. I and others have encouraged assessment of total changes in gene expression (DGE) as well as changes in isoform proportion (DTU), as both may be present in an experiment and both may be of biological importance to the system being studied. DGE has the property that the majority of inferential uncertainty which exists in an RNA-seq sample is removed (because it occurs across isoforms within genes), leaving inferential uncertainty from reads mapping across gene loci, but this property of reduced uncertainty does not preclude a transcript-level analysis. While DTE has advantages, the above sentence claiming that DGE has none overstates a more complex situation in my opinion.In the Introduction, the authors state "Throughout the paper, the authors refer to "DRIMSeq" as "DRIM-Seq" which is minor but different than the software and publication.et al. (20161) and Loveet al. (20182), and so such an approach has some evidence of working well for detection of isoform changes within a gene.For what it's worth, the transcript-level test is similar conceptually to the current implementation of testForDEU in DEXSeq which compares the expression of each feature to the sum of expression from all other features of the gene . Running DEXSeq on transcript estimated counts with testForDEU was tested on simulated data in Sonesonthe tables record the full provenance of the results".I didn't understand what was meant by the following: "The performance results of RATs on these simulated datasets are in good agreement with those presented in Love et al (2018)". However, this seems to be not clearly the case, which may be due to differences in the simulated data in the two articles, or some other reason. In the present article, DRIMSeq is reported as having lower sensitivity with lower achieved FDR than other methods, SUPPA2 has higher sensitivity and higher FDR, and RATs with various filter thresholds falls in between. In Loveet al. (20182), DRIMSeq had the opposite performance: higher sensitivity but higher FDR relative to SUPPA2 and RATs run with default filters. However interpretation is made difficult by all the filtering options in Figure 3. It would be easier to compare perhaps if an additional supplementary plot to Figure 3 was made with only the default filter thresholds instead of the filter threshold ranges for all methods. The main commonality across the two benchmarks seems to be that RATs can achieve higher sensitivity than SUPPA2 while maintaining the same precision, for the 5% nominal FDR threshold.It is stated that, "As a consequence, the qRT-PCR intensities measured in the original study are actually impossible to interpret in the context of the updated annotation and the originally reported conclusion is likely wrong." Specifically which conclusion is likely wrong? From the analysis, it seemed like there is not a problem with the original qRT-PCR intensities and interpretation for at least one of the three genes.This sentence needs to be made more specific, or else it could be misleading: "18% of the SUPPA2 results are rejected due to the effect size filter". I didn't follow the authors in that statement.Why is it perplexing that "Existing tools rely on the mean isoform abundances...". This implies that the mean of inferential replicates is used for statistical testing. It's perhaps subtly different, other methods are typically using the maximum likelihood estimate, which may be different than the mean of the bootstrap distribution, and different than the mean of the Gibbs sampling distribution. I would just say that other tools do not make use of inferential replication.It is stated, "I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. ‘RATs’ is addressing an important problem: how to quantify changes in transcript isoform usage. Other tools, like ‘Sleuth’, address a related problem, which is differential transcript expression. Both RATs and Sleuth take advantage of the bootstrapping data that tools like ‘kallisto’ and ‘Salmon’ generate when quantifying transcript isoform abundance. By taking advantage of bootstrapping, such tools can estimate the technical variation within the data, to better look for differential changes. Transcript isoform usage is often linked to changes in alternative splicing or isoform specific decay rates (e.g. from NMD). Therefore, having a tool to accurately find changes in isoform usage is vital to our ability to address a range of biological problems. et al., 20181). A bonus of RATs is that several figures can be generated from the data within the tool. This was simple to do, but allowed for you to visualize your data in a straightforward but powerful way. This is one of the rare tools that just works and was relatively intuitive and well-documented.  I have tried a version of RATs. I found that it was easy to install and easy to use. Being able to install bioinformatics software is no guarantee  or does it help get closer to the true rate of biological variation, thus increasing the true positive rate?  In the methods, it would be good to see more explanation on how RATs does its pre-filtering. For example, if a transcript has zero expression in one treatment but a modest to high expression in the other treatment, would RATs keep this transcript or discard it? This would be of interest to people working on RNA decay pathways, such as NMD. et al., 20182). This tool appeared to also perform well in the publication of this approach, where they used sim data. Therefore I think a comparison to RATs here, using sim and real data (human) would be appropriate. One thing that I would love to see is a comparison of RATs to the DEXSeq/DRIMSeq approach used to address differentiation transcript isoform usage  With regards to the datasets tested, the published human data set which is shown here is not directly confirming the accuracy of RATs, as the authors show that the qPCR validation within the original study may be inaccurate, and underlying issues are present due to the reference genome version. Although the dataset is being used to compare RATs to SUPPA and DRIM-Seq, it is not validating the tool. I think that this manuscript would benefit from comparison of the three tools using another "real-life" dataset, which has been validated in some way, to support that RATs is detecting known DTU. A. thaliana dataset. The authors should include details on how the transcript abundances were produced for this . All datasets used are publicly available with accession numbers given. Additional data is provided within published links; however, these would benefit from a simple readme file, which explains the contents of each extended data file so the reader doesn't need to search through them.  Methods for tool development and testing are clearly described, apart from with false positive testing with Within the results, it would be nice to see more discussion on the impact of the bootstrapping information used by RATs. I think that this is a really beneficial part of this tool and this has not been demonstrated enough. It should also be made clearer if this bootstrapping information is obtained solely from Salmon/Kallisto or if RATs implements it's own bootstrapping.  Although the testing of the simulated datasets does show that RATs outperforms DRIM-Seq and SUPPA2, I don't feel that as it stands you can conclude that the analysis of the published dataset shows this. When comparing to the published findings, SUPPA2 shows better results with confirmation of the qPCR results. As I've mentioned above, I would find another "real-life" dataset for comparison, or simply re-word the conclusion so that this isn't overstated.I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.


The response to treatment for juvenile idiopathic arthritis (JIA) can be staged using clinical features. However, objective laboratory biomarkers of remission are still lacking. In this study, we used machine learning to predict JIA activity from transcriptomes from peripheral blood mononuclear cells (PBMCs). We included samples from children with Native American ancestry to determine whether the model maintained validity in an ethnically heterogeneous population.Our dataset consisted of 50 samples, 23 from children in remission and 27 from children with an active disease on therapy. Nine of these samples were from children with mixed European/Native American ancestry. We used 4 different machine learning methods to create predictive models in 2 populations: the whole dataset and then the samples from children with exclusively European ancestry.In both populations, models were able to predict JIA status well, with training accuracies > 74% and testing accuracies > 78%. Performance was better in the whole dataset model. We note a high degree of overlap between genes identified in both populations. Using ingenuity pathway analysis, genes from the whole dataset associated with cell-to-cell signaling and interactions, cell morphology, organismal injury and abnormalities, and protein synthesis.This study demonstrates it is feasible to use machine learning in conjunction with RNA sequencing of PBMCs to predict JIA stage. Thus, developing objective biomarkers from easy to obtain clinical samples remains an achievable goal. One of the underappreciated advances in the field of pediatric rheumatology has been the recognition that therapeutic response during the course of treated juvenile idiopathic arthritis (JIA) can be staged based on specific clinical features . Using hOver the past 10 years, significant advances have been made in machine learning approaches to complex datasets. Augmented computational power and new methods have facilitated successful classification even using heterogeneous populations, such as PBMCs. For instance, Showe et al. used a support vector machine model of 29 genes expressed in PBMCs to distinguish between non-small cell lung cancer and non-malignant lung disease with 86% accuracy . SerranoIn the present study, we renewed our efforts to determine the feasibility of developing biomarkers for disease activity in JIA from PBMCs, which are easy and relatively inexpensive to obtain and prepare from clinical blood samples. We also sought to determine whether such biomarkers might be detected in an ethnically heterogeneous population that included Native American children, as the optimal biomarkers will be those that are useful across the age range and diverse ethnicities of the JIA population. Although patient-to-patient variability in gene expression continues to be a challenge, we demonstrate that machine learning approaches can overcome this and successfully classify patients by disease activity.This project was reviewed and approved by the IRBs of the University of Oklahoma Health Sciences Center, the University at Buffalo, and Seattle Children’s Hospital. All research was conducted according to the IRB-approved protocol. Written informed consent documents were executed for each subject, and, where appropriate, children also executed consent documents.Samples for this study were obtained from children with polyarticular JIA as determined from standard criteria and included three children who were rheumatoid factor positive , as, oncThe first dataset included 12 samples from patients who were involved in a previous study . Of thesFor this study, we also performed RNAseq on 38 additional samples from children with polyarticular JIA. This study included samples from 15 patients (4 boys and 11 girls) who had achieved clinical remission on medication (CRM). Of these 15 patients, 10 achieved CRM status on methotrexate (MTX) alone, while 5 achieved CRM on MTX plus a TNF inhibitor . The remaining 23 (6 boys and 17 girls) were classified as having an active disease on therapy (ADT). Among the patients with ADT, 18 were on MTX alone, while 5 were on TNF inhibitors . There were 7 individuals from whom we collected longitudinal data; therefore, these subjects have both ADT and CRM samples. Nine samples were from individuals with mixed Native American/European ancestry recruited from the pediatric rheumatology clinic at the University of Oklahoma. Only 1 of these patients achieved CRM within a year of diagnosis, which may be a reflection of the severity of disease in the Native population, which we have previously documented [Whole blood was drawn into 10-mL citrated CPT tubes . Cell separation procedures were started within 1 h from the time the specimens were drawn. Peripheral blood mononuclear cells were separated from granulocytes and red blood cells by density gradient centrifugation. Cells were then immediately placed in TRIzol® reagent  and stored at − 80 °C.RNA isolation and sequencing were carried out exactly as described in . In brieRNA sequencing data analysis was carried out using Galaxy suite of tools. Sequenced paired-end reads were aligned to the human reference genome hg19 with Bowtie2 (version 2.3.4.2) using default settings. Counts were assigned to aligned reads using htseq-count (version 0.9.1). Counts were then converted to transcripts per million (TPM) to facilitate a comparison between the samples. As data was collected during 2 different time points, we considered the 12 samples previously analyzed to be batch 1 and the 38 new samples to be batch 2. We used ComBat under default settings in R to perform batch effect correction.We performed principal component analysis (PCA) using raw counts of all 50 samples in R using prcomp function to visually assess the presence of outliers or batch effects in our dataset.K-nearest neighbors (KNN) [K-nearest neighbors: The K-nearest neighbor method using a Euclidean metric was employed. The number of neighbors, K, was set as 10. By calculating the distance to each training sample, the testing sample was classified as the class that was most common among its K-nearest neighbors.Random forest: The random forest method was employed to build a prediction model. The number of trees was set at 1000. The random forest was built by contrasting a multitude of decision trees based on subsets of the training data generated by random sampling with replacement. The resulting model classifies testing samples by the majority vote of the decision trees.Support vector machine: We trained prediction models using support vector machine with two different kernels: cubic and Gaussian. To separate the binary-labeled samples, the SVM transforms them into a multidimensional space using the kernel, followed by a hyper-plane that maximizes the margin separating samples of either class. The resulting model classifies testing samples by transforming them into the higher dimensional space with the corresponding kernel and making decisions based on their signed distance to the hyper-plane.In order to develop and test prediction models, we randomly divided the whole cohort into training and testing cohorts. In this way, the features of the model are identified in the training cohort and then tested in an independent cohort to avoid bias. We used a 70-30 split to ensure that the testing set encompassed sufficient sample heterogeneity. In training cohort data, we employed a cutoff of average TPM > 1 and restricted our analysis to protein-coding genes to increase the likelihood that genes identified are significantly expressed. In this subset of transcripts, we performed a supervised feature selection using Hilbert-Schmidt independence criterion least absolute shrinkage and selection operator (HSIC LASSO)  to identrs (KNN) , random rs (KNN) , supportrs (KNN) , and SVMrs (KNN) . We brieModels were firstly evaluated using tenfold cross-validation in the training cohort. Predicted disease stage was compared to clinical diagnosis in order to determine the accuracy, sensitivity, and specificity of each model, which were averaged across the tenfold cross-validation for each model. We also implemented a receiver operating characteristic (ROC) curve analysis to determine the area under the curve (AUC) for each model as an additional performance metric. Trained models were then evaluated in an independent testing cohort. We calculated the accuracy, sensitivity, specificity, and AUC for each model based on the model predictions of testing samples.We executed this pipeline in two different populations. First, we developed models for JIA disease stage prediction in the whole dataset. Then, since clinical experience has demonstrated that patients with Native American ancestry have more difficulty reaching remission (possibly reflecting different genetic or epigenetic influences), we sought to determine whether including this diverse patient population affected model performance. Therefore, we developed models only using samples of European descent and compared their performance to models generated using all samples.t test p values  were used as input. We considered disease, biological functions, and upstream regulators to be significant if the absolute value of their activation z-score is > 2. The z-score reflects the match between the observed gene expression and the predicted relationship direction. It is used to infer activation states of called functions and regulators. Networks were deemed significant if their score was greater than 15. Network score uses a hypergeometric distribution and calculated with right-tailed Fisher’s exact test in this equation: p-score = −log10. This score reflects the approximate fit between the generated networks and eligible molecules from the input list.We used ingenuity pathway analysis (IPA) to investigate the biological pathways and networks associated with model genes using the whole dataset. Model genes with their log2-fold change of the ADT group relative to the CRM group and n = 50) using PCA, it became clear there were 3 outliers . These samples were removed prior to any model development, making our dataset 47 samples in total. We then used Combat in R to correct for batch effects present in this 47 sample dataset.For the newly sequenced samples, we achieved, on average, 73% of reads aligned concordantly exactly 1 time or more than 1 time using Bowtie2. Raw counts with a sum > 0 across all samples were used to visualize all samples by PCA. Upon visualizing the entire dataset . Approximately 70% of ADT and approximately 70% of CRM samples were randomly assigned to the training cohort for a total of 33 samples . The remaining 30% of both ADT and CRM samples were used to form an independent testing cohort of 14 samples . Cohort assignment is reported in Additional file The other dataset we used to develop predictive models for JIA stage consisted of all European samples. There were 38 samples in total, 17 ADT and 21 CRM. As in the whole dataset model, approximately 70% of ADT samples and approximately 70% of CRM were randomly assigned to the training cohort, which consisted of 27 samples . The remaining samples comprised the testing cohort of 11 samples . Cohort assignment is reported in Additional file Genes in each model are reported in Table We note that multiple genes within the two models are associated with biological functions that are relevant to JIA. For example, CD97 is an early activation marker on T cells and interacts with CD55 to serve as a co-stimulator . CD97 alz-scores 2.7, 2.9). No upstream regulators were found to be significant in our data. Three networks had significant scores. The first network, which scored 46, is associated with cancer, dermatological diseases and conditions, and hematological disease. The first network, shown in Fig. Due to the high degree of overlap between the model genes identified using the whole dataset or only European samples and the fact that the whole dataset model uses both more genes and more patient samples, we conducted an IPA analysis on the genes selected in the whole dataset model. Organismal death and morbidity or mortality were predicted to be increased in the ADT group , using PBMC expression data and machine learning methods. Furthermore, we demonstrate that the models performed equally well whether we included only non-Hispanic, European-descended subjects or a larger cohort that included enrolled members of Native American tribes from Oklahoma. Indeed, the models performed better when the Native American patients were added, although this may simply have been due to the fact that adding these patients increased the number of subjects and thus the accuracy and power of the predictive models. We expect that we will further improve the predictive power by further increasing the sample size. Implementing other machine learning methods, such as neural networks , may furOur results were corroborated by functional analysis of the genes that were used to discriminate active disease and remission. The discriminating transcripts included genes plausibly linked to the biology of therapeutic response (a process about which we still understand very little). Furthermore, the functional associations with MAP kinase signaling, intercellular communication, and cell adhesion are consistent with the findings from our previous gene expression studies , 3, 5.We were unable to discriminate between children who achieved remission on MTX alone vs. those who achieved remission on MTX plus a biological agent. These findings are consistent with our earlier paper showing the CRM, as defined by the Wallace criteria, represents a distinct immunologic/transcriptional state that differs a little (at least in PBMC) whether that state was achieved with MTX or MTX plus a biological agent . These fDespite these promising results, we are still faced with the challenge of using PBMC expression data from individual patients to assign “active disease” or “CRM” status, which will be required if such information is to be used to inform clinical care. Achieving this goal will require several additional steps, which will include  refiningAnother step that will need to be taken before expression-based assessments of CRM come into clinical use will be the demonstration that this approach is superior than the method currently in use: application of the clinical and laboratory findings that make up the Wallace criteria (1). There is an ongoing concern in the field that physical examination alone is insensitive to detect subtle, smoldering synovitis, for example . In the PBMCs remain a promising source for the development of expression-based biomarkers, provided that the proper analytic tools are used to develop classification algorithms. The ease with which these cells can be obtained and the continued improvement and affordability of sequencing techniques make this a fruitful line of research despite the inherent challenges of cell and patient heterogeneity.Additional file 1: Table S1. Sample information.Additional file 2: Table S2. Gene names and functions.


Transcript assembly from RNA-seq reads is a critical step in gene expression and subsequent functional analyses. Here we present PsiCLASS, an accurate and efficient transcript assembler based on an approach that simultaneously analyzes multiple RNA-seq samples. PsiCLASS combines mixture statistical models for exonic feature selection across multiple samples with splice graph based dynamic programming algorithms and a weighted voting scheme for transcript selection. PsiCLASS achieves significantly better sensitivity-precision tradeoff, and renders precision up to 2-3 fold higher than the StringTie system and Scallop plus TACO, the two best current approaches. PsiCLASS is efficient and scalable, assembling 667 GEUVADIS samples in 9 h, and has robust accuracy with large numbers of samples. Transcript assembly is an important step in analysis of RNA-seq data whose accuracy influences downstream quantification, detection and characterization of alternative splice variants. Here, the authors develop PsiCLASS, a transcript assembler leveraging simultaneous analysis of multiple RNA-seq samples. A crucial step in virtually all RNA-seq data analyses is assembling the reads into full-length transcripts. The accuracy of transcript reconstruction is critical for quantification, detection, and characterization of alternative splice variants, and the identification of differences in gene expression and splicing patterns between tissues, developmental stages, and physiological or disease states.RNA sequencing (RNA-seq) has become the de facto standard in surveying the transcriptome of a cell, organism, or species, to determine the expressed genes and transcripts and their expression levels, and to enable differential and functional analyses3, which is used as reference for downstream quantification and differential analyses. Most single-sample assemblers including Cufflinks, isoCEM, Scripture, Traph, CLASS, iReckon, CIDANE, FlipFlop, CLASS2, StringTie, Scallop, and TransComb15 build a graph structure from read alignments on the genome, then traverse the graph to select an optimized set of transcripts, represented as paths. Recent second-generation transcript assembly methods including StringTie, CLASS2, and Scallop have taken great strides toward increasing the accuracy and efficiency of assembly at single-sample level, and meta-assemblers such as StringTie(ST)-merge and TACO3 have led to more robust collections of meta-annotations. Despite these efforts precision remains low, with <40% of the predicted transcripts in a single-sample and <30% of transcripts in meta-annotations representing complete and accurate reconstructions16. Concomitantly, a handful of efforts have focused on designing methods that simultaneously assemble transcripts across multiple RNA-seq samples, notably CLIIQ, ISP, MiTie, and FlipFlop19. However, none of these programs has comparable performance or is equipped to handle more than a limited number of samples.  across all samples to create a unified set of meta-annotations3, producing a set of meta-annotations obtained by combining the individual samples’ outputs. Second, unlike traditional single-sample assemblers, it uses information from all samples to produce transcripts for individual samples and for the unified annotation set. PsiCLASS starts by selecting a set of high-confidence introns and subexons at each locus, using statistical models of introns and intronic read levels. In doing so, it first generates a set of features  for each sample, and then corroborates information across all samples to select a high-confidence subset. Next, PsiCLASS builds a unified subexon splice graph that is used within a dynamic programming optimization procedure to select a representative set of transcripts in each sample , medium , and high . While, StringTie showed the best overall performance on the single-transcript genes, PsiCLASS had the highest sensitivity and precision on the medium and high complexity classes, in particular a 16.8–24.5% gain in sensitivity and 21.0–31.8% in precision on highly expressed transcripts . The diagrams in Fig. Most RNA-seq analyses are aimed at determining gene expression or splicing differences between two biological conditions. To explore the robustness of PsiCLASS when combining two-condition samples, we applied it and the other methods to RNA-seq samples from hippocampi of normal mice (24 samples) and mice with induced epileptic seizures (20 samples) . We then aggregated the results in each collection in three ways, using PsiCLASS voting, TACO, and TACO applied to the samples assembled separately by tissue. In all cases, PsiCLASS with its default voting algorithm had better performance, with both higher sensitivity, and higher precision  and meta-assemblers  at multiple levels. Notably, voting significantly increased transcript level precision for all programs, albeit PsiCLASS maintained a clear advantage in all but one case (GEUVADIS-25), where Scallop’s performance was comparable  number of samples, with benefits most evident for data sets of three or more samples. It also ranked comparable with StringTie and Scallop on single-sample data sets in our control experiments, and similarly to StringTie on the three samples for which the two reference programs were optimized. Therefore, our program can be effectively used for small RNA-seq collections and within the conventional assemble-and-merge approach.As the emerging landscape of RNA-seq foresees increasingly larger data sets from large patient cohorts and population variation studies, we aimed to assess the suitability of the multi-sample approach as the data set grows. We evaluated program performance on increasingly larger subsets of RNA-seq samples from the GEUVADIS population variation project, up to the full set of 667 samples Fig. . All metDetermining the set of expressed genes and transcripts in an RNA-seq experiment is critical for subsequent quantification and differential expression and splicing analyses. The conventional approach to process each sample separately and then merge the sets of transcripts to create a unified set of annotations has limitations, in particular low precision. We present PsiCLASS, a transcript assembler and meta-assembler that simultaneously analyzes all samples in an RNA-seq experiment. Algorithmic underpinnings of PsiCLASS include a global subexon graph, statistical methods for cross-sample feature selection, and dynamic programming optimization and voting algorithms for transcript selection at the level of individual samples and over the entire collection of data, which collectively lead to more complete, more accurate, and more consistent sets of annotations. In particular, our study suggests that voting-based approaches may have advantages over the current assembly-based aggregation methods in some cases, albeit their broader applicability needs to be further investigated.24 or TCGA25. Nevertheless, our limited tests on collections of samples from 2 to 6 GTEx tissues reflect that PsiCLASS has better performance than TACO, and therefore show promise for exploring capabilities for handling more heterogeneous data sets in the future.PsiCLASS is designed to leverage the gene information across samples in a one or two condition experiments, to improve the accuracy of gene and transcript reconstruction for downstream differential gene expression and differential splicing studies, and may not be ideally suited to multi-condition experiments, such as annotation of genes and alternative splicing in large data sets such as GTExhttps://github.com/splicebox/PsiCLASS.PsiCLASS had significantly higher precision at similar sensitivity when compared with current best methods, and showed consistently high precision (>50%) when tested on data from a variety of experimental conditions. PsiCLASS is scalable, efficient, and robust with large numbers of RNA-seq samples, thus providing a highly effective paradigm for large-scale analyses of collections of hundreds and thousands of samples. PsiCLASS is available free of charge from PsiCLASS builds a global subexon graph of a gene and its splice variants from genome-aligned RNA-seq reads in all input samples. It then traverses the graph to select a subset of the encoded transcripts in each sample. Lastly, it combines the predicted transcript sets across all samples, using a voting procedure to select a final set of meta-annotations.Building per sample subexon graphs. PsiCLASS builds a subexon graph for each sample, then combines graphs across all samples to create a global subexon graph. In each sample, PsiCLASS uses candidate introns extracted from spliced read alignments to divide the genome into regions and subexons. A region denotes a maximal contiguous portion of the genome covered by reads. A subexon is a portion of a region delimited by two consecutive splice junctions and/or the end(s) of the region. A subexon graph has subexons as vertices, and two subexons are connected by an edge if they are adjacent in the same region or connected by an intron. Candidate splice variants are encoded as maximal paths in the subexon graph.To build the sample-level subexon graph, PsiCLASS clusters read alignments along the genome that are colocated and on the same strand. Introns are extracted from spliced alignments and used to divide the region into subexons. A major confounding factor in determining subexons from RNA-seq data is the presence of intronic unprocessed RNA (‘noise’). To differentiate between intronic noise and signal, such as retained introns or alternative 5′ and 3′ gene ends, PsiCLASS assigns each subexon a score that reflects the probability that it is noise. In contrast to current single-sample methods, which simply discard a subexon if it fails sample-wide cutoffs, PsiCLASS then combines sample-level scores across all samples to determine a final label for the subexon and its inclusion in the global subexon graph.ci be the average read coverage of (intronic) subexon i. In the coverage ratio model, PsiCLASS calculates a score that is equal to the coverage ratio of this subexon versus its flanking subexons: π, (1 − π) are the prior probabilities that an intronic subexon is noise or signal, respectively, and i is noise according to the Bayes formula: More specifically, PsiCLASS computes the probability that a subexon is due to intronic noise using two models: (i) the exon–intron coverage ratio, and (ii) the intronic read coverage. Let The coverage ratio model above is insufficient when the overall gene coverage is low. Hence, the second model establishes a similar formula for coverage levels, Building the global subexon graph. PsiCLASS removes likely artifactual introns and intronic noise subexons by evaluating evidence across all sample, and builds a global subexon graph by combining individual samples’ graphs that share at least one intron. Multi-sample intron selection: To select a highly accurate set of introns, PsiCLASS assesses each candidate intron’s read support across all samples. Assume the experiment contains M samples, and denote each intron by its coordinates in the genome, e.g., . Let S  denote the total number of read alignments supporting  over all samples. Then the total number of alignments supporting its splice sites: a, b) iff: (i) a, b) appears in at least M0 samples, where b − a| ≥ 100,000 (long intron). Condition (ii) is intended to filter out long intron-type alignment artifacts due to gene families and repeats, or from sequencing errors, which can lead to merged genes and transcripts. Multi-sample subexon selection: To determine a global set of subexons, PsiCLASS combines the subexon sets of individual samples with some modifications. Where multiple 3′ or 5′-end  candidate subexons occur with the same endpoint and potentially different lengths among the samples, PsiCLASS creates a unique subexon with the median length. Further, to determine intronic subexons, PsiCLASS calculates a final score by combining all sample scores with a Bayesian formula. More specifically, let s = 1,…, M denotes the sample. Here, i in sample s, and coverage model is employed. Similarly, Transcript selection. Candidate transcript models are represented as maximal paths in the global subexon graph, from a node with no incoming edges (source) to a node with no outgoing edges (sink). Since the graph generally encodes a much larger number of transcripts than is biologically possible, PsiCLASS identifies and selects a subset of transcripts that can explain all contiguity constraints from spliced reads and paired reads. PsiCLASS first predicts a set of transcripts for each sample, using a graph-based dynamic programming algorithm with the global subexon graph and the sample specific alignment data, then combines the individual samples’ transcript sets and selects a subset of meta-annotations by voting.12, adapted for subexon graphs. Namely, we define a constraint as a cluster of read alignments with the same subexon pattern. Like CLASS2, PsiCLASS uses constraints to decrease the memory usage while preserving the structural and contiguity information contained in the full set of reads. For a given graph G, let C = {c1, …, cm} denote the set of constraints and T = {t1, …, tn} the set of candidate transcripts, encoded in the graph. Given a constraint ci, its abundance ai = a(ci) defined as the number of supporting reads (or read pairs) normalized by the number of possible start positions of the reads within the constraint’s subexons. To reduce the transcript selection problem to SET_COVER, we view each candidate transcript tj as the set of constraints that are compatible with its exon-intron structure: C(tj) = {c1, …,cnj}, where ci ~ tj. Constraint ci is compatible with the transcript tj’s exon-intron structure, denoted ci ~ tj, if its list of subexon intervals is contained in that of the transcript’s and all the exon–exon junctions correspond to introns in the transcript.To predict a set of transcripts for each sample, PsiCLASS employs a SET_COVER framework and dynamic programming algorithmsAj = min { ai| ci ~ tj }. The goal then becomes to determine a subset of transcripts that most closely explain the constraints and their abundance levels. PsiCLASS uses a greedy approximation framework to address this problem, iteratively selecting the transcript that covers the largest number of constraints weighted by the constraints’ abundance, then adjusting the constraints’ abundance levels before the next iteration:In the simplest formulation, the goal then is to select a minimal (parsimonious) subset of transcripts that satisfies all constraints. More realistically, to account for the different abundance of constraints, we define a transcript’s abundance as the minimum abundance among its set of constraints: While  or a variation of the splice-graph dynamic programming algorithm in Song et al.12 that considers all reads single-end, for fast processing. Once the candidate transcript set T is determined, PsiCLASS applies the greedy SET_COVER approximation algorithm above.PsiCLASS implements the procedure above in two steps. First, it determines the candidate set of transcripts L, and recursively calculates the maximum number of constraints f(L) for substranscripts starting with subpath L: f(L) = maxL’ { f(L′) + c, if L′ exists; c(L), if L′does not exist }, where: (i) L′ is a subpath immediately following L so that all constraints compatible with L end before or within L′; (ii) c is the number of constraints starting in and  compatible with L and L′, and compatible with the concatenated subpath L, L′; and (iii) c(L) is the number of constraints covered by subtranscript L. To take into account the abundance levels in the optimization process, at each sweep of the graph the algorithm excludes subpaths that cover constraints with abundance below a fixed value x; hence, the dynamic programming algorithm will return the best transcript with abundance greater than x (x-abundance transcript). With this modification, at each graph sweep the selection process selects an x-abundance transcript, starting with x0 = 0 (thus guaranteeing that such a transcript exists), and each selected transcript’s abundance value used as lower bound for the selection process at the following step: 0 = x0 < x1 < x2 < ⋯xm, until no transcript can be found. The optimal transcript then is among those selected by the sweeps. More details, along with proof of correctness for the algorithm, can be found in Song et al.12.For completeness, we include a brief description of the dynamic programming optimization procedure. The algorithm considers all subpaths Selecting a global set of transcripts. PsiCLASS selects a set of meta-annotations from the individual samples’ sets of transcripts by voting. Each transcript is assigned a score equal to the transcript’s average estimated abundance across the samples . As different cutoff parameters values might work best for data with specific characteristics, the user can readily adjust or recalibrate the voting parameters postassembly, starting from the already computed sets of transcripts for the individual samples.20 with the default gene and transcript distribution models and randomly sampling 10% of the transcripts  from the human GENCODE v.2726 gene annotations. Reads were aligned to the reference genome hg38 separately with Hisat2 v.2.0.521 and STAR v.2.5.3a22. Chromosome 2 alignments were extracted and used in the assembly and evaluations. Human liver samples were obtained from the Stanley Medical Research Institute (SMRI) and previously sequenced by Dr. Sabunciyan’s lab. All ethical standards set forth by the Johns Hopkins University and local laws were complied with. The liver tissue was collected postmortem by the SMRI after obtaining consent from the family of the deceased. A detailed description of the consent procedure and the collection was published27. The project was reviewed by the Johns Hopkins University Institutional Review Board (IRB) and was given exempt status since only postmortem tissue was involved in the research. For sequencing, total RNA was isolated using the Qiagen RNeasy kit and libraries were constructed using the Illumina TruSeq Stranded Total RNA kit for Human/Mouse/Rat following the manufacturers recommended protocol. The resulting stranded, rRNA depleted liver libraries were sequenced on an Illumina HiSeq 2000 instrument. 667 RNA-seq samples from human lymphoblastoid cell lines part of the GEUVADIS population variation project were publicly available from ArrayExpress (accession: E-GEUV-6), and mouse hippocampus RNA-seq data were those reported in ref. 23 and available from GenBank (ProjectID: PRJEB18790). Lastly, RNA-seq samples from six human tissues  used in the heterogeneity study were downloaded from the dbGAP GTEx24 repository. The lists of samples for each evaluation are included in the Source Data file.We generated 25 RNA-seq samples, with ~85 million 100 bp paired-end reads, using the software Polyester5. Performance metrics at the transcript, exon, and intron levels were calculated with the tool grader included in the PsiCLASS package, and at the gene level with the tool Cuffcompare (v.2.2.1). For gene content analyses, only transcripts that were a full match, contained or splice variant transcript  of a reference gene were considered.Once the reads were mapped to the genome, we used StringTie v.1.3.3.b and Scallop v.0.10.2 to assemble them into transcripts, for each individual sample. Transcript sets for all samples in an experiment were then merged with StringTie(ST)-merge and TACO v.0.7.3. For PsiCLASS v.1.0.1, reads were assembled simultaneously across all samples. To evaluate the accuracy of transcript assembly, we employed standard sensitivity (Sn) and precision (Pr) measures and evaluation criteria to assess the accuracy of transcript reconstructions by comparison to a gold reference, namely the set of simulated transcripts and the human GENCODE v.27 and mouse RefSeq gene annotations. A predicted transcript is deemed a true positive (TP) iff its intron chain fully matches that of a gold reference transcript. If N is the number of predicted transcripts, M be the number of ground truth transcripts, then Sn = TP/M and Pr = TP/N Further information on research design is available in the Supplementary InformationPeer Review FileReporting SummarySource Data


Harmonization techniques make different gene expression profiles and their sets compatible and ready for comparisons. Here we present a new bioinformatic tool termed Shambhala for harmonization of multiple human gene expression datasets obtained using different experimental methods and platforms of microarray hybridization and RNA sequencing.Unlike previously published methods enabling good quality data harmonization for only two datasets, Shambhala allows conversion of multiple datasets into the universal form suitable for further comparisons. Shambhala harmonization is based on the calibration of gene expression profiles using the auxiliary standardization dataset. Each profile is transformed to make it similar to the output of microarray hybridization platform Affymetrix Human Gene. This platform was chosen because it has the biggest number of human gene expression profiles deposited in public databases. We evaluated Shambhala ability to retain biologically important features after harmonization. The same four biological samples taken in multiple replicates were profiled independently using three and four different experimental platforms, respectively, then Shambhala-harmonized and investigated by hierarchical clustering.Our results showed that unlike other frequently used methods: quantile normalization and DESeq/DESeq2 normalization, Shambhala harmonization was the only method supporting sample-specific and platform-independent biologically meaningful clustering for the data obtained from multiple experimental platforms.The online version of this article (10.1186/s12859-019-2641-8) contains supplementary material, which is available to authorized users. Public repositories of gene expression data cover a rich spectrum of normal and pathological conditions, including all known human diseases and developmental features –4. The mThis non-comparability of gene expression data hampers further levels of data analysis for the different datasets, e.g. finding differentially expressed genes and assessing activation of molecular pathways , 12.To solve this problem, the data must be either normalized  or harmonized (when different platforms were used) . For theThe normalization methods include quantile normalization (QN) , frozen However, to our knowledge all previously published harmonization methods have a substantial limitation that they are capable of performing harmonization for only two expression datasets . Thus, oShambhala that may be considered a more universal tool compared to the existing approaches. Unlike previous harmonizers, Shambhala is independent on (i) number of harmonized datasets and/or experimental platforms, and (ii) number of samples in every dataset. The Shambhala harmonization protocol includes several specific features such as the auxiliary calibration dataset that helps to initially transform the data, and the reference definitive dataset that defines the universal shape of the output harmonized gene expression profile. Next, we investigated the performance of Shambhala to harmonize the gene expression data from multiple experimental platforms obtained from the Microarray Quality Control (MAQC) [Here, we present a new method for cross-platform data harmonization termed l (MAQC)  and Sequl (MAQC) . Our datreference definitive dataset (Q). The process of profile transformation involves multiple iteration steps, when the dataset P, which contains profiles under harmonization, is altered, whereas the dataset Q remains unchanged. Consequently, the output of such transform has gene expression profiles like those obtained using the same experimental platform, as for the dataset Q.We developed Shambhala method for cross-platform comparisons of multiple datasets. In its present form, the method was tailored for the comparison of human gene expression data, and its application for other organism data requires further specific data search. Let us look at the problem of cross-platform harmonization in more detail. Imagine an arbitrary set of experimental platforms that has produced a set gene expression profiles. Our goal is to make them all comparable. To do so, we may make them similar to a pre-defined reference. This reference may be taken from a set of profiles that has been obtained at a widely used experimental platform; we can term this set the Q-shape, we should equalize it using another pre-defined dataset called auxiliary calibration dataset (P0). In other words, it means that each individual gene expression profile under harmonization, say i, is transformed into the Q-shape not within the original dataset of unharmonized profiles from certain experimental platform, but rather being taken alone, and then merged with P0. Namely, we quantile-normalize [i with the dataset P0, which produces the dataset P for further transformation. This dataset P is then transformed into the shape of the dataset Q, thus producing the dataset P1. From this dataset P1, only the transformed single profile i is taken for further analysis. This procedure is then applied to all other gene expression profiles which need to be harmonized  to increase stability of the results, Shambhala uses spherical (cosine-based) [P and Q datasets.Some features of this pipeline, which are used for transformation of dataset N method  that shoN method . Such fee-based) , 29 rathP0 and Q, the latter acting as the reference for gene expression profiles after harmonization, and the former serving for preliminary calibration of expression level ranges. As the dataset Q for this application, we used the mRNA expression profiles taken from the Genotype Tissue Expression (GTEx) project [Q dataset because it is frequently considered the golden standard for microarray hybridization of human tissues [Q-set obtained using the Illumina HiSeq 2000 platform.Importantly, the Shambhala pipeline depends on two datasets,  project , namely  tissues , 32, whiP0) for Shambhala implementation, we found that our previous experimental dataset including 39 human gene expression profiles obtained using CustomArray microchip platform  showed the best performance in clustering tests compared to more than twenty other datasets of the comparable size (data not shown). Interestingly, our attempts to use the GTEx dataset for both P0 and Q, have failed to produce good sample clustering.When selecting the optimal auxiliary calibration dataset , Affymetrix Human Genome U133 Plus 2.0 Array (GPL570) and Illumina Sentrix Human-6 Expression Beadchip (GPL2507). In the SEQC project, the microarray expression profiles for the same biosamples were compared with the RNA sequencing data obtained using Illumina HiSeq 2000 platform (GPL11154), see Table To assess quality of data harmonization, we tested whether hierarchical clustering of the harmonized genes expression profiles will be biologically meaningful or rather dependent on the experimental platforms used. For the clustering, Euclidean distance was used as a metric of proximity. An ideal method for data harmonization would allow grouping of output expression profiles according to the type of biosamples (A to D), but not according to a platform used. Similar types of biosamples  were expected to show more tight clustering. In contrast, the platform-based clustering independent on the biological similarities of biosamples could be considered bad result.Quantile normalization, QN .estimateSizeFactors module. To make the microarray data formally suitable for DESeq/DESeq2 normalization, we took an integer part of all microarray-measured expression level values for each gene and each sample. The intensity values for microarray-measured signal were taken as they were deposited in GEO repository, i.e. after device-dependent primary background correction or equilibration but before any cross-platform transformation or harmonization. Although the DESeq/DESeq2 method was designed for normalization of NGS data and assumes that the count data follow a negative binomial distribution, there were several examples when DESeq/DESeq2 was formally applied to rounded microarray data, both in model investigations based on microarray profiles [Differential expression analysis for sequence count data, DESeq /DESeq2  using thprofiles  and for profiles , 36. MorShambhala harmonization with two different GTEx definitive datasets (obtained using either microarray Affymetrix or NGS Illumina HiSeq 2000 platforms). Shambhala method was compared with other above normalization techniques  because they are popular tools used for merging data from multiple datasets. The standard harmonization methods such as XPN  are not To test Shambhala, we took data from three experimental platforms for MAQC dataset and from four platforms for SEQC. All gene expression profiles were harmonized using three alternative methods:We tested Shambhala, QN and DESeq/DESeq2 methods for their abilities to simultaneously harmonize data from three experimental microarray platforms  from the MAQC project.Q-set and QN/DESeq2 normalization  project , Table 1The results obtained Fig.  suggest Q-dataset .Although attempts to develop universal cross-platform transcriptome harmonization technique are known for more than a decade, the acceptable performance was shown before only for harmonization of up to two expression datasets , 21. In In this study, we tried two sets of expression profiles (obtained using microarray Affymetrix and Illumina HiSeq 2000 platforms) from the GTEx project  as the rP0-dataset) were to provide the best merging of biologically relevant profiles after harmonization. During the training stage, we selected the P0-dataset, which could ensure the good-quality harmonization of the MAQC dataset, namely for the profiles obtained using the Affymetrix and Agilent microarray platforms. Importantly, we did not observe good clustering quality when trying the same GTEx dataset as both P0 and Q, so we had to select another dataset (originated from the CustomArray platfrom) as P0.The criteria for selecting the auxiliary calibration dataset Additional file 2:Q) and auxiliary calibration (P0) datasets for the Shambhala method. (XLSX 42198 kb)Definitive Additional file 4:Harmonized SEQC gene expression profiles. (XLSX 219245 kb)Additional file 5:A detailed view of hierarchical clustering for gene expression levels for MAQC project data after application different harmonization methods. (PPTX 857 kb)Additional file 6:A detailed view of hierarchical clustering for gene expression levels for SEQC project data after application different harmonization methods. (PPTX 647 kb)


Medical image biomarkers of cancer promise improvements in patient care through advances in precision medicine. Compared to genomic biomarkers, image biomarkers provide the advantages of being non-invasive, and characterizing a heterogeneous tumor in its entirety, as opposed to limited tissue available via biopsy. We developed a unique radiogenomic dataset from a Non-Small Cell Lung Cancer (NSCLC) cohort of 211 subjects. The dataset comprises Computed Tomography (CT), Positron Emission Tomography (PET)/CT images, semantic annotations of the tumors as observed on the medical images using a controlled vocabulary, and segmentation maps of tumors in the CT scans. Imaging data are also paired with results of gene mutation analyses, gene expression microarrays and RNA sequencing data from samples of surgically excised tumor tissue, and clinical data, including survival outcomes. This dataset was created to facilitate the discovery of the underlying relationship between tumor molecular and medical image features, as well as the development and evaluation of prognostic medical image biomarkers. Moreover, in comparison to molecular profiling, radiomic characterization provides a more comprehensive representation of the tumor. Since molecular profiling is restricted to the region of the biopsy, it results in an incomplete representation of the heterogeneous tissue of the tumor. On the other hand, molecular technologies allow profiling of genes expressed in the tissue sample. This complementary relationship suggests that combining the use of molecular and imaging biomarkers has the potential to improve patient care and to provide insight into how molecular mechanisms give rise to imaging phenotypes.Advances in high-throughput molecular technologies hold great promise for the development of genomic biomarkers that enable precision medicine tailored to specific patients. These molecular biomarkers deliver powerful diagnostic information, as well as high prognostic significance. Similarly, medical imaging technologies provide tools for measuring the structural, functional and physiologic properties of tissue. Identifying image-based properties of tumors through medical images is a standard part of diagnosis, clinical staging, and treatment planning. Because image interpretation can be subjective, for medical imaging to have a role in personalized medicine, the development of robust, standardized image features that can be used to predict molecular properties, prognosis and/or treatment response, is required. These standardized features can be in the form of semantic annotations acquired from human observers, or radiomic features, i.e. quantitative image features computed from the image pixels. Quantitative image features include tumor size and shape, image intensity distributions, and image texture. While the adoption of molecular technologies can be limited by cost and the invasiveness of the procedure, medical imaging is, more commonly, part of the standard of care2–20. An important challenge in such radiogenomic studies is the scarcity of large data sets containing medical images, extracted image features, gene expression profiles, and clinical data with survival outcomes. Specifically, for NSCLC, which is the leading cause of cancer death21, there is a dearth of available datasets that contain medical images, molecular features, and associated clinical data. In NSCLC, CT and PET/CT are the investigation tools of choice for diagnosis, staging and monitoring of response to treatment. From these scans, one can compute a large number of quantitative image features for associations with tumor molecular features and clinical outcomes. Molecular profiles of tumors can be obtained through needle biopsies or samples of surgically-excised tumors. Clinical data and outcomes can be obtained from standard medical follow-up. While large molecular datasets with clinical data are readily available22–25, there are fewer public medical imaging datasets combined with clinical and molecular data. For example, while five independent NSCLC datasets containing collectively 788 subjects were used in a radiogenomics study7, only 89 subjects had imaging, molecular and clinical data. Moreover, that dataset included CT scans but did not contain PET/CT data. It is important to continue to create large integrated databases available for discovery and validation of biomarkers, and so we created this dataset to allow researchers to investigate the relationships between image features, tumor molecular phenotype, and survival outcomes.The prognostic power of medical image features and their link to molecular properties has only been recently investigated for certain cancer typesBetween 2008 and 2012, we collected clinical and imaging data for 211 subjects referred for surgical treatment and obtained tissue samples from the excised tumors, where available. Tissue samples were analyzed to produce molecular phenotypes using gene microarrays, RNA sequencing technology, or both, in addition to standard-of-care NSCLC mutational testing. We also collected clinical data, such as: age, gender, weight, ethnicity, smoking status, TNM stage, histopathological grade. In addition, we included 3D tumor segmentations of the CT studies that were used for computation of 3D quantitative image features. Not all data are available for all subjects due to limitations in resources; out of the 211 subjects, 116 have all data types expect for micro-array , 130 have clinical, imaging (CT and PET/CT), and molecular (RNA-Seq) as detailed in th, 2008 and September 15th, 2012. Subjects signed written consent forms according to the guidelines of institutions’ IRBs. The subjects were selected from a pool of early stage NSCLC patients, referred for surgical treatment with preoperative CT and PET/CT performed prior to surgical procedures. Samples of excised tissues were later used to obtain mutation data and gene expression data using gene expression microarrays, or RNA sequencing, or both. Identifiers for this set of 162 subjects are in the format R01-XXXXXX. (2) The AMC cohort, consisting of 49 additional subjects , was retrospectively collected from Stanford University School of Medicine based on the same criteria in addition to the availability of the following clinical mutational test results: Epidermal Growth Factor Receptor (EGFR), Kirsten Rat Sarcoma viral oncogene homolog (KRAS), and Anaplastic Lymphoma Kinase (ALK). Identifiers for this set of 49 subjects are in the format AMC-XXXXXX. For both cohorts, clinical data included, where available, smoking history (211), survival (211), recurrence status (210), histology (211), histopathological grading (162) and Pathological TNM staging (161). There were 172 adenocarcinomas and 35 squamous cell carcinomas and 4 not otherwise specified with grades ranging from poorly to well-differentiated. Clinical date features (e.g. recurrence date and scan dates) are shifted for anonymization purposes and are chronologically ordered relative to each other. With approval of our respective Institutional Review Boards (IRB), we recruited a total of 211 subjects for the following two cohorts: (1) The R01 cohort consisted of 162 NSCLC subjects  from Stanford University School of Medicine (69) and Palo Alto Veterans Affairs Healthcare System (93). Subjects were recruited between April 7Subjects received preoperative CT and PET/CT scans at Stanford University Medical Center and Palo Alto Veterans Affairs Healthcare System prior to surgical treatment as part of their care. Different scanners were used depending on the institution and physician choice and scanning protocols also varied.All imaging data were de-identified prior to analysis at Stanford. For subjects from Stanford, we de-identified the imaging data using the Medical Imaging Resource Center (MIRC) Clinical Trial Processor (CTP) . The MIRC CTP is a software tool designed to Anonymize DICOM objects to remove protected health information. Medical image data from VA subjects were de-identified using PACSGEAR .26, we performed a second round of de-identification using CTP, further assuring complete removal of any identifying information. TCIA complies with HIPAA de-identification standards using the Safe Harbor Method as defined in section 164.514(b)(2) of the HIPPA Privacy Rule. This is done by incorporating the “Basic Application Confidentiality Profile” which is amended by inclusion of the following profile options: Clean Pixel Data Option, Clean Descriptors Option, Retain Longitudinal with Modified Dates Option, Retain Patient Characteristics Option, Retain Device Identity Option, and Retain Safe Private Option. The de-identification rules applied to each object are recorded by TCIA in the DICOM sequence Method Code Sequence  by entering the Code Value, Coding Scheme Designator, and Code Meaning for each profile and option that were applied to the DICOM object during de-identification27.Prior to making the data available on The Cancer Imaging Archive (TCIA)28 are available from 211 subjects. Since this is a retrospectively collected dataset, different subjects were scanned using different scanners, scanning protocols and scanning parameters: slice thickness of 0.625–3 mm (median: 1.5 mm) and an X-ray tube current of 124–699 mA (mean 220 mA) at 80–140 kVp (mean 120 kVp). Detailed scanning parameters, including scanner make and model are specified in the DICOM headers. Scans were acquired with subjects in supine position with arms at sides, from the apex of the lung to the adrenal gland within a single breath-hold. CT images in DICOM format18F-FDG PET/CT data are available for 201 subjects. A GE Discovery D690 PET/CT was used for PET/CT scanning at Stanford University Medical Center, while the Palo Alto VA employed a GE Discovery PET/CT scanner. (The exact model of PET/CT scanners are specified DICOM image headers.) FDG Dose and uptake time were 138.90–572.25 MBq (mean 309.26 MBq) and 23.08–128.90 min (mean 66.58 minutes), respectively. PET images were generated at both sites using a similar protocol. Specifically, CT-based attenuation correction was utilized with iterative Ordered Subset Expectation Maximization (OSEM) reconstruction. Image acquisition included routine coverage of base-of-skull to mid-thigh with additional spot views where necessary. Each bed position was 1–5-minute acquisition, dependent on su weight. 29.Fasting Fluorodeoxyglucose 30–38. However, we note that the imaging datasets reported here were acquired over several years and from several institutions, and not as part of a prospective trial. For these reasons there was no attempt to harmonize the acquisition and reconstruction protocols.It has been recognized that the results of quantitative analyses  of images will vary as a function of image acquisition and reconstruction protocolSemantic annotations are available for axial CT series of 190 subjects. The template of semantic terms was developed in consensus by two academic thoracic radiologists (A.N.C.L. and D.A.) with expertise and interest in lung cancer imaging. The template was developed for nodules as they are the most common manifestation of lung cancer. As a result, we provide no semantic annotations for cancers of other manifestations, e.g., central obstructive tumors or "pneumonic tumors”. The template contains 28 nodule analysis features and parenchymal features comprising conventional and newly developed features used for diagnosis and staging using the CT images. Nodule features describe anatomy location, geometry, internal features and other associated findings of the nodules.39. 41. One radiologist (A.N.L.) with more than 20 years of experience ascribed the semantic annotations for all subjects’ CT scans using ePAD, an open-source and freely available web-based quantitative imaging informatics platform41. While we acknowledge that semantic annotations are subjective and subject to intra-and inter-reader variability, these were used in several studies, e.g., to predict EGFR and KRAS mutation status42, and to create a radiogenomic map linking semantic features to gene expression profiles generated by RNA sequencing13.Parenchymal features characterize lung emphysema, bronchi and lumen. The selected terms are in common usage in radiology clinical practice and are derived from descriptions in the radiology literature; definitions of some of these, such as “nodule” are found in the Fleischner Society: Glossary of Terms for Thoracic Imaging28.Initial segmentations for 144 subjects were obtained from an axial CT image series using an unpublished automatic segmentation algorithm. All of these segmentations were viewed by a thoracic radiologist (M.K.) with more than 5 years of experience and edited as necessary using ePAD. Final segmentations were reviewed by an additional thoracic radiologist (A.N.L.); disagreements in tumor boundaries were discussed and edited as appropriate, with final approval by A.N.L. All segmentations are stored as DICOM Segmentation ObjectsAll tumor samples were collected from treatment-naïve subjects during surgical procedure. Following excision, the surgeon cut a 3–5-mm-thick slice along the longest axis of the excised tissue, which was frozen within 30 minutes of excision. It was later retrieved for RNA extraction. Molecular data are available from EGFR, KRAS, ALK mutational testing, gene expression microarrays, and RNA sequencing. Tumors from 17 subjects were analyzed using both gene expression microarrays and RNA sequencing.EGFR, KRAS and ALK mutation status are available from clinical records in 206, 205, and 196 subjects, respectively. Single nucleotide mutation detection was performed using SNaPshot technology based on dideoxy single-base extension of oligonucleotide primers after multiplex polymerase chain reaction (PCR). Exons 18, 19, 20 and 21 were tested for EGFR mutations. Exon 2 Positions 12 and 13 were tested for missense KRAS mutations with amino acid substitution. Mutation results were a combination of mutation at any location of the tested exons. For ALK, EML4-ALK translocation detection test was performed using fluorescence in situ hybridization (FISH).29.Gene expression microarray data was collected for the subset of 26 subjects, who underwent surgical treatment between April 7, 2008 and May 21, 2010. RNA was processed at the Stanford Functional Genomics Facility using Illumina Whole Genome Bead Chips . These data were preprocessed as follows: First, we filtered the microarray probes on the basis of a significant detection call in at least 60% of the samples. Next, we log transformed the microarray data and used quantile normalization to normalize between arrays. These data, along with the corresponding CT images, were used to describe associations between image features, gene expression, and survivalBased on availability and quality of available tissue, RNA sequencing was performed on samples from 130 subjects (17 of which intersect with the gene expression microarray dataset described in the previous section). We excluded RNASeq for tissue samples with RNA integrity number (RIN) below 2.5. Total RNA was extracted from the tissue samples and converted into a library for paired-end sequencing on Illumina Hiseq according to the protocol for the Illumina TruSeq Sample preparation kit . Briefly, total RNA quality and quantity were measured by BioAnalyzer (Agilent). For library preparation, the TruSeq Total Stranded RNA with Ribo-Zero Reduction (Illumina) was used following manufacturer’s instructions. This method includes a Ribo-Zero rRNA depletion step, followed by fragmentation and cDNA synthesis using SuperScript II (Life Technologies). The cDNA was A-tailed, ligated and amplified using the materials in the TruSeq Total Stranded RNA with Ribo-Zero Reduction kit. Quality was confirmed using the BioAnalyzer and finally the concentration evaluated by KAPA qPCR (KAPA Biosystems). Prior to sequencing, samples were diluted to 4 nmol and pooled. Pooled libraries were clustered via the cBOT and sequenced on the HiSeq 2500 (illumina) following manufacturer’s instructions. The set of 130 tissue samples was sequenced in three batches of sizes 16, 66, 48.43 version 2.3 with 91 bases of splice junction overhangs. Next, Cufflinks version 2.0.244 was used to determine the expression calls in each sample using Fragments Per Kilobase of transcript per Million mapped reads (FPKM).Data processing was performed by Centrillion Biosciences as follows: reads were aligned to the human genome (hg19) using the alignment algorithm STARA unique identifier for each subject is identical in all four public data records in this dataset. Subject ID’s are 6-digit numbers in the form of R01-XXXXXX or AMC-XXXXXX.45. CT and PET/CT Images are in DICOM format. Where available, segmentations are provided as DICOM Segmentation Objects.Clinical, image, semantic data for all subjects are stored in The Cancer Imaging Archive (TCIA) (Data Citation 1). One comma-delimited file contains clinical data for all subjects with unique subject identifiers. Semantic features for each subject are stored in Annotation and Image Markup (AIM) filesImage data of 26 subjects had been previously deposited in the TCIA repository (Data Citation 2). These images were given new subject names in the form R01-XXXXXX as part of the new dataset described in this work.46 (Data Citation 3). The subject identifiers are identical to subject names in Data Record 2. Processed gene clusters were deposited in tab-delimited files with column values corresponding to microarray ID, log2 transformed quantile normalized and probe selection detection-p-value, respectively. This data record also contains raw expression data, as well as matrix data obtained prior to normalization.Gene expression microarray data, available for 26 subjects, were deposited in National Center for Biotechnology Information Gene Expression Omnibus (NCBI GEO)Raw and processed sequencing data obtained from RNASeq for 130 subjects are available at NCBI GEO (Data Citation 4). The subject IDs are identical to subject names in Data Record 1.All CT and PET/CT data were collected as part of patient care and therefore all quality assurance was performed by the institution that collected the data.All data are freely available to browse, download, and use for commercial, scientific and educational purposes as outlined in the Creative Commons Attribution 3.0 Unported License. Users should properly cite this source for any work based on this dataset.How to cite this article: Bakr, S. et al. A radiogenomic dataset of non-small cell lung cancer. Sci. Data. 5:180202 doi: 10.1038/sdata.2018.202 (2018).Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.