Renal carcinoma is the 20th most common cancer worldwide. Clear cell renal cell carcinoma is the most frequent type of renal cancer. Even in patients diagnosed at an early stage, characteristics of disease progression remain heterogeneous. Up-to-date molecular classifications stratify the ccRCC samples into two clusters. We analyzed gene expression in 23 T1 or T3 ccRCC samples. Unsupervised clustering divided this group into three clusters, two of them contained pure T1 or T3 samples while one contained a mixed group. We defined a group of 36 genes that discriminate the mixed cluster. This gene set could be associated with tumor classification into a higher stage and it contained significant number of genes coding for molecular transporters, channel and transmembrane proteins. External data from TCGA used to test our findings confirmed that the expression levels of those 36 genes varied significantly between T1 and T3 tumors. In conclusion, we found a clustering pattern of gene expression, informative for heterogeneity among T1 and T3 tumors of clear cell renal cell carcinoma. Renal tumors are classified as the 20th most common malignancy worldwide, both based on incidence and death rates . Clear cThere are several classification systems used to describe renal tumors. Grading is performed by Fuhrman system, based on the nuclear and nucleolar features, and recently modernized by International Society of Urologic Pathology . The mosIn the up-to-date molecular classifications, ccRCC samples are classified into two groups . The aut23 ccRCC tumor samples were collected during radical nephrectomy at the Department of Urology, JUMC. Samples were fixed with formalin and embedded in paraffin at the Department of Pathology for microscopic evaluation and transferred to the Center for Medical Genomics OMICRON for gene expression studies. The study was approved by the Bioethics Committee of the Jagiellonian University.All patients signed written informed consent forms. Experiments conform to the provisions of the Declaration of Helsinki in 1995 (as revised in Edinburgh 2000). Patient tumors were classified into T1 (13) or T3 (10) stages by a pathologist and independently re-evaluated. Selection of T1 and T3 tumors, as a basis of sample collection for our study, gave prospect to investigate clinically most frequent specimens. In addition, each study group remains homogeneous and sample selection parallels kidney restriction of the tumors in T1 group and extension beyond the kidney in T3 group. Additional clinical data were collected, along with immunohistochemical information summarized in RNA was isolated from 10 x 5 μm slides from Formaldehyde Fixed-Paraffin Embedded (FFPE) block, using Maxwell 16 FFPE Tissue LEV DNA Purification Kit (Promega). Briefly, 300 μl of Mineral Oil and 250 μl of lysis master mix were added per sample and incubated in 56°C for 15 min and subsequently at 80°C for 1 hour. DNA was degraded by DNase I treatment . The aqueous phase was transferred to Maxwell FFPE Cartridge and RNA was isolated according to Promega RNA—FFPE protocol. 50 μl of Nuclease-Free Water was used for RNA elution. The RNA quantity was measured using NanoDrop 1000 (Thermo Scientific) device and quality was assessed on 2200 TapeStation System , according to manufacturer instructions. DV200 parameter, describing percentage of RNA fragments longer than 200 bases was used for sample classification . SamplesThe Illumina Whole Genome-DASL assay was performed using 200 ng of RNA following the manufacturer's instructions. Briefly, RNA was reverse transcribed to cDNA using biotinylated primers, followed by immobilization to streptavidin-conjugated paramagnetic particles. Biotinylated cDNAs were then simultaneously annealed to a set of assay-specific oligonucleotides. Extension and ligation of the annealed oligonucleotides generated PCR templates that were amplified using Titanium Taq DNA Polymerase (Clontech). Labeled PCR products were washed and denatured to yield single-stranded fluorescent molecules, which were hybridized to the HumanHT12 v4.0 Whole Genome Gene Expression BeadChips for 16 h at 58°C. The Illumina HiScan was used to scan the arrays.Microarray data in *.IDAT format were uploaded and pre-processed in R environment. The ‘beadarray’ package was used to upload the data and ‘lumi’ for normalization and filtration of the data.The differentially expressed probes were detected via the Generalized Linear Model framework implemented in the package 'limma'.For the comparison between T1 and T3 groups as well as the groups defined via hierarchical clustering  the functions 'contrast.fit' and 'eBayes' were used. For analysis of differential expression in the TCGA cohort the framework implemented in the package 'edgeR' was utilized. Gene counts were normalized with the default options and subsequently filtered to reduce the number of hypotheses tested. After estimating the dispersion parameter, the Generalized Linear Model was fitted and tests for coefficients were performed. Since this was used as a replication cohort, we have only recorded the number of genes differentially expressed between the two study groups with the standard level of statistical significance 0.05.The 23 T1 and T3 samples were clustered based on expression of 543 probes. To this aim the function 'hclust' with complete linkage as implemented in the 'heatmap.2' procedure was used. The noticeable pattern where the dendrogram is divided into three main groups was further confirmed with the use of the 'cutreeDynamic' function in the 'dynamicTreeCut' package. The faithfulness of clustering was evaluated using the cophenetic correlation coefficient.Both the T1 and the T3 samples were clustered based on normalized gene expression values (pseudocounts) generated with 'edgeR' package. To overcome the issue of the Euclidean metric being driven by highly expressed genes, the Renyi divergence function was used as the measure of similarity. Renyi divergence was previously used by the authors of  in the cThe t-SNE algorithm was used as implemented in R-package 'Rtsne, with all default parameters except for 'perplexity' where 7 was chosen as the value that is expected to produce the least number of 'groupings' for the sample size of 23.The ROC analysis was based on logistic models with the indicator of the event that the sample is T3 used as the dependent variable. For each of the probes used for hierarchical clustering 300 random training and testing sets were selected (each time the testing set was of size 7) and ROC as well as AUC was calculated as implemented in packages 'ROCR' and “OptimalCutpoints'. Subsequently, for each probe the median AUC was calculated for each sample . For each sample, the 'goodness' of classification was quantified as the median of these median AUC values over all probes.http://www.cytoscape.org/, [Pathway Enrichment analysis was performed using ‘ClueGO’ plugin for Cytoscape 3.3.0 . For alWe analyzed 23 ccRCC samples on a microarray platform. Our samples belonged to T1 and T3 stages, as the T2 and T4 stages are rarely diagnosed (only 69 (13%) T2 and 11 (2%) T4 samples in TCGA database). Our main interests were to (1) test the hypothesis if gene expression reflects the histological classification of the JUMC samples  and (2) whether we will be able to find molecular features that reflect the observed diversity of disease progression.Differential gene expression comparing T3 vs. T1 samples resulted in 481 genes  was narrowed down to those in level 3 in the Genome Ontology (GO) hierarchy. This returned a list of enriched terms, presented in Unsupervised hierarchical clustering, based on expression of 481 genes, divided 23 T1 and T3 samples into three distinct clusters: A1, A2, and A3. Two of the clusters contain populations of T1 (A1) or T3 (A3) samples only, whereas the third cluster includes samples from both groups (A2). This three-cluster pattern (two 'pure' and one mixed) is not present when all (~34K) probes are used for analysis. Therefore, it is unlikely that it is due to a batch effects.As the distances between the clusters suggests that the A2 cluster is more closely related with A3, despite containing samples from both T1 and T3, we aimed to investigate which expression profiles characterize the A2 group. A heatmap presenting relative gene expressions is shown in To further test whether the pre-selection of features  allows for faithful sample classification between T1 and T3, an additional machine-learning approach has been adapted. Three sets of probes were used in this analysis: (1) the probes used for hierarchical clustering ; (2) top 40 differentially expressed probes, and (3) all 34476 probes. Subsequently, using these sets of features, samples were projected, using the t-SNE algorithm see  section,To further test whether there are indeed samples more difficult to correctly classify as T1 or T3 (i.e. samples in the 'mixed cluster'), a ROC-based analysis was performed. For each of the probes aligned to 481 genes, the AUC was calculated for 300 random test subsets of size 7 for a (logistic) model fitted on the remaining 16 training samples. Subsequently, the median for each sample/probe was calculated and the median of these 500 number was assigned to each sample as a measure of 'goodness' of classification. In the current study, we use a relaxed threshold (p<0.1) in the process of selection of probes for sample clustering. We wish to support this choice by demonstrating that probes, which are differentially variable between the study groups are more informative about the clustering of samples than the ones with similar variances. To this aim we first compare the variances between T1 and T3 tumors using Levene's test and detect six probes (ILMN_1762410 (SLC22A2), ILMN_1716246 (FRZB), ILMN_1677851 (RARRES1), ILMN_1746128 (ACSM2B), ILMN_3311035 (miR-1251) and ILMN_1793309 (BEND4)) with FDR below the standard 0.05 significance threshold. Secondly, we compare the cophenetic correlation coefficient for two different clusterings: (1) based on differentially expressed probes (p<0.1) with p-value in the Levene's test above the median, and (2) based on differentially expressed probes (p<0.1) with p-value in the Levene's test below the median. We note that for the first set of probes the coefficient equals 0.72 and the second 0.87. Note, that in both of the above clusterings, we use the same number of probes for analysis.First the A2 cluster was compared to A1. In total 13 genes with adj. p-value < 0.05 were found, with the largest log-fold-change = -1.98 achieved by interleukin 6.Secondly, A2 and A3 clusters were compared. 22 differentially expressed genes  were identified and the top 15 (with |log-fold-change|>1.5) of them are presented in A list of all differentially expressed probes between A2 and A3 is presented in Differential expression analysis of A3 and A1 clusters revealed a larger set of differentially expressed genes than A1 vs A2 and A3 vs A2. A list of 58 down-regulated and 101 up-regulated probes with |logFoldChange| > |1.5| is presented in Our sample size was relatively small, therefore we used TCGA RNA-seq data as a larger replication cohort. Of 481 genes, differentially expressed between T1 and T3 groups, 394 had expression levels available in the TCGA database. Illumina Probe IDs were converted to ENSG# using BioMart. A Gene was considered for further analysis if it was expressed in at least 80% of samples and the median read count exceeded 10.Validation of our primary analysis revealed that almost 67% of differentially expressed genes  were also differentially expressed in the TCGA RNA-seq data. We additionally note that the correlation coefficient for the logFC’s between the two cohorts equals 0.78, as presented in the The TCGA cohort was further used to test the observations made with the use of unsupervised classification. T1 and T3 samples from TCGA were clustered based on each of the 394 differentially expressed genes in the UJ CM cohort. These 394 clusters were then evaluated, using Renyi Divergence measures, for heterogeneity with the expectation that those genes driving the clustering observed in the UJ CM cohort will show evidence of heterogeneity. To this aim, differential expression analysis was performed . The results of this analysis were compared to the set of 36 genes which discriminate between A2 from A1 and A2 from A3.Using above-described procedure, T1 samples were divided into 8 clusters (C1-C8), where C1 was the largest and was further used as reference. Results of this analysis are presented in T3 samples were also divided into 8 clusters. Clusters 5 to 8 were excluded from further analysis based on sample size. Results of the analysis are presented in Clear cell renal cell carcinoma is the most frequent kidney neoplasm in adults, comprising 80–90% cases of renal tumors . A charaAmong the 36 differentially expressed genes identified between T3 and T1, several have known associations with ccRCC: TRPM3, AQP1, FBP1, ITPKA, LOX, TUBB3, IGFBP1, ALDOB –25, otheOne of the main goals of the study was to emphasize heterogeneity of expression patterns in the context of discrimination between study groups. Therefore, as noted in the Results section  we choose to relax the statistical significance threshold (from 0.05 to 0.1) to include in further analysis genes which have more heterogeneous expression profiles in our cohort and thus higher chance of falling above the standard significance level.Pathway enrichment analysis emphasized the role of copper metabolism, which is an important process in renal tissue in general, and has a role in cancer development. However, presented genes do not take direct part in pathways regarding those issues.Other differentially expressed genes include molecular transporters , (trans)membrane proteins  and other channel proteins  involved in regulation and signal transduction in cell metabolism and response to external stimuli. This suggests that dysregulation of signal transduction maybe important in defining the observed diversity of ccRCC outcomes.Clustering of 23 samples, based on all significantly differentially expressed genes, revealed partition of T1 and T3 samples into 3 distinct clusters . Two of Comparison of a A2 with A1 cluster revealed a role for IL-6. Overexpression of IL-6 is associated with enhanced invasiveness and epithelial–mesenchymal transition (EMT) and IL-6 is involved in a JAK/STAT signaling pathway . AlthougComparison of a mixed cluster with pure T3 cluster resulted in 15 genes with |logFoldChange| > 1.5. The 13 overexpressed genes were reported to play a role in RCC: PAX2, NAT8, GBA3, SLC22A2 –43 otherThe differences shown here lead us to postulate that the isolated intermediate cluster reflects the tumors that are less metastatic prone/aggressive. Several statistically significantly disturbed genes  show contradictory expression change trend to expression changes described in the literature and associated with tumor progression and metastasis ,38,43.The 36 genes obtained from A2 vs A1 and A2 vs A3 comparisons code for proteins associated with intracellular signaling and metabolic processes, but lack driver genes or commonly known cancer master regulators, yet these modulators account for the observed sample heterogeneity. This is in line with the previous results of T3 vs T1 comparison and underlies the significance of regulatory/modulatory genes in the progression of the disease.Use of TCGA ccRCC cohort confirmed almost 70% of our results. We tested whether genes differentially expressed between A1/A3 and A2 can be used to measure heterogeneity in a larger dataset. For that purpose, we clustered TCGA ccRCC samples separately T1 and T3 and used the A2-specific genes as an input for differential expression. We found that expression changes in gradual fashion for T3 clusters  suggestiIn conclusion we propose that expression of certain RNAs can be used to study the molecular basics of the heterogeneity of ccRCC.We have found a clustering pattern reflecting heterogeneity of samples. Furthermore, we detected genes associated with diversity of ccRCC samples. We postulate that genes associated with regulatory or signal transduction modulation roles are related to diverse representations of ccRCC occurring regardless of the histological classifications. Further functional research is needed to test these observations.S1 TableT, N, M–classification of samples, T_—expanded T classification, diameter–measured in the widest dimension, Grade–ISUP modified Fuhrman grade, survival time–calculated as the number of days between collection date and date of death , mdm2 –result of histochemical staining of mdm2 protein, p53—result of histochemical staining of p53 protein, procedure–name of the procedure at which the sample was obtained, necrosis–was the tumor tissue necrotic, DV 200 –Illumina proposed parameter for description of quality of FFPE derived RNA samples .(DOCX)Click here for additional data file.S2 TableILMN ID–Illumina Probe ID, logFC–logFoldChange of probe expression, AveExpr–average expression of the given probe, P.Value–p value, adj.P.Val–p value adjusted for multiple testing.(DOCX)Click here for additional data file.S3 TableAll probes that reached adj. p. value < 0.05 cut-off value. ILMN ID–Illumina probe ID, logFC–log Fold Change, AveExpr–average probe expression value, P.Value–p value, adj.P.Val–p value adjusted for multiple testing.(DOCX)Click here for additional data file.S4 TableAll probes that reached adj. p. value < 0.05 and logFC > 1.5 cut-off values. ILMN ID–Illumina probe ID, logFC–log Fold Change, AveExpr–average probe expression value, P.Value–p value, adj.P.Val–p value adjusted for multiple testing.(DOCX)Click here for additional data file.S1 FigCut-off p-value 0.05. Blue–underexpressed, red–overexpressed genes. Based on the expression pattern the samples were divided into three clusters. Colour bar indicates what cluster the sample was assigned to: red–A1 (pure T1), green–A3 (pure T3), blue–A2 (mixed).(TIFF)Click here for additional data file.S2 FigAdjusted p-value < 0.1, 481 genes. Narrowed down to genes in level 3 in the Genome Ontology (GO) hierarchy.(TIF)Click here for additional data file.S3 FigThree sets of probes were used in this analysis: (1) the probes used for hierarchical clustering ; (2) top 40 differentially expressed probes, and (3) all 34476 probes. S were projected on a 3-dimensional space. For the unbiased case  no association between tumor size and the three components is present. Interestingly, for the two sets of pre-selected features, not only do we see a separation between T1 and T3 samples in the 3D space, but also a separation between the three clusters defined in the previous section.(TIF)Click here for additional data file.S4 FigEntire dataset (~33000 probes) was projected onto a 10-dimensional space. Three dimensions for which the projection has the strongest association with the clinical diagnosis (T1 vs T3) was selected and projected data visualized. Interestingly, even in this agnostic approach (with features not being pre-selected) we see a further support for the ‘intermediate cluster’ to appear.(ZIP)Click here for additional data file.


We study the problem of identifying differentially mutated subnetworks of a large gene–gene interaction network, that is, subnetworks that display a significant difference in mutation frequency in two sets of cancer samples. We formally define the associated computational problem and show that the problem is NP-hard.We propose a novel and efficient algorithm, called DAMOKLE, to identify differentially mutated subnetworks given genome-wide mutation data for two sets of cancer samples. We prove that DAMOKLE identifies subnetworks with statistically significant difference in mutation frequency when the data comes from a reasonable generative model, provided enough samples are available.We test DAMOKLE on simulated and real data, showing that DAMOKLE does indeed find subnetworks with significant differences in mutation frequency and that it provides novel insights into the molecular mechanisms of the disease not revealed by standard methods. For exadirectly identify sets of interacting genes that are significantly more mutated in a set of samples compared to another. The task of finding such sets is more complex than the identification of subnetworks significantly mutated in a set of samples, since subnetworks that have a significant difference in mutations in two sets may display relatively modest frequency of mutation in both set of samples, whose difference can be assessed as significant only by the joint analysis of both sets of samples. Pathways that are significantly mutated only in a specific cancer type may not be identified by analyzing one cancer type at the time or all samples together Fig. , but, inSeveral methods have been designed to analyze different aspects of somatic mutations in a large cohort of cancer samples in the context of networks. Some methods analyze mutations in the context of known pathways to identify the ones significantly enriched in mutations . Other Few methods studied the problem of identifying subnetworks with significant differences in two sets of cancer samples using data other than mutations.  studied In this work we study the problem of finding subnetworks with frequency of mutation that is significantly different in two sets of samples. In particular, our contributions are fourfold. First, we propose a combinatorial formulation for the problem of finding subnetworks significantly more mutated in one set of samples than in another and prove that such problem is NP-hard. Second, we propose DifferentiAlly Mutated subnetwOrKs anaLysis in cancEr (DAMOKLE), a simple and efficient algorithm for the identification of subnetworks with a significant difference of mutation in two sets of samples, and analyze DAMOKLE proving that it identifies subnetworks significantly more mutated in one of two sets of samples under reasonable assumptions for the data. Third, we test DAMOKLE on simulated data, verifying experimental that DAMOKLE correctly identifies subnetworks significantly more mutated in a set of samples when enough samples are provided in input. Fourth, we test DAMOKLE on large cancer datasets comprising two cancer types, and show that DAMOKLE identifies subnetworks significantly associated with one of the two types which cannot be identified by state-of-the-art methods designed for the analysis of one set of samples.This section presents the problem we study, the algorithm we propose for its solution, and the analysis of our algorithm. In particular, "m genes C and D, of dimension i is mutated in the j-th sample of i interacts with gene j .We are given measurements on mutations in S is mutated in sample coverage  of S in S is mutated in the sample, that iscoverage  of S in Given a set of genes S, with G and displaying a significant difference in coverage between differential coverageWe are interested in identifying sets of genes In particular, we study the following computational problem.S of G of size The differentially mutated subnetworks discovery problem: given a value S in the solution may contain genes that are not mutated in S.Note that by finding sets that maximize We have the following.The differentially mutated subnetworks discovery problem is NP-hard.G defined on a set n vertices, a family I , an algorithm to solve the differentially mutated subnetworks discovery problem. DAMOKLE takes in input mutation matrices etSolutions, described in Algorithm 2, is a recursive algorithm that, give a current subgraph S, identifies all maximal connected subgraphs S and with S one edge at the time and stopping when the number of vertices in the current solution is k or when the addition of no vertex leads to an increase in differential coverage S. In Algorithm 2, N(S) refers to the set of edges with exactly one vertex in the set S. GThe motivation for design choices of DAMOKLE are provided by the results in the next section.The design and analysis of DAMOKLE are based on the following generative model for the underlying biological process.i. Let k genes that is differentially mutated in samples of F a mutation in gene i is observed with probability F but we condition on the event H is mutated in the sample”.For each gene H, all mutations come from distribution F. For genes in H, in a perfect experiment with no noise we would assume that samples in F. However, to model realistic, noisy data we assume that with some probability q the “true” signal for a sample is lost, that is the sample from F. In particular, samples in q from F.For genes not in p be the probability that H has at least one mutation in samples from the control model F, Let i is mutated in a sample from E(H)  in a sample from E(H) in a sample from If we focus on individual genes, the probability gene The efficiency of DAMOKLE is based on two fundamental results. First we show that it is sufficient to start the search only in edges with relatively high differential coverage.Ifthen, in the above generating model, with high probability (asymptotic inand)there exist an edgesuch thatfor anyz. Clearly, if for all S, thenFor a set of genes S is connected, there is an edge S in a sample from Since the set of genes entclass1pt{minimaThe second result motivates the choice, in Algorithm 2, of adding only edges that increase the score of the current solution (and to stop if there is no such edge).If subgraphScan be partitioned asandthen with high probability (asymptotic in)S then We first observe that if each sample in S, we need to correct for the fraction of samples in j and To remove the assumption that a sample has no more than one mutation in To compute a threshold that guarantees statistical confidence of our finding, we first compute a bound on the gap in a non significant set.Assume thatSis not a significant set, i.e.,andhave the same distribution onS,thenS, otherwise S, otherwise Let S is not significant Clearly To apply Hoeffding bound (Theorem 4.14 in ), we notFWER: if we want to find just the subnetwork with significant maximum differential coverage, to bound the FWER of our method by FDR: if we want to find several significant subnetworks with high differential coverage, to bound the FDR by Let entclass1pt{minimaWhile Theorem We consider two permutation tests to assess the association of mutations in the subnetwork with the highest differential coverage found by DAMOKLE. The first test assesses whether the observed differential coverage can be obtained under the independence of mutations in genes by considering the null distribution in which each gene is mutated in a random subset  of all samples, independently of all other events. The second test assesses whether, under the observed marginal distributions for mutations in sets of genes, the observed differential coverage of a subnetwork can be obtained under the independence between mutations and samples’ memberships . For both tests we estimate the p-value as follow:N (permuted) datasets from the null distribution;generate N permuted datasets;run DAMOKLE  on each of the x be the number of permuted datasets in which DAMOKLE reports a solution with differential coverage p-value of S is let Let G the HINT+HI2012 networkWe implemented DAMOKLE in PythonS of k genes with differential coverage g in G (including genes in S) is mutated with probability g with probability S, and for each such sample we mutated, with probability c, one gene of S, chosen uniformly at random. In this way c is the expectation of the differential coverage We tested DAMOKLE on simulated data generated as follows. We assume there is a subnetwork cer data . We consS or part of it. In particular, we studied the impact of three parameters: the differential coverage S; the number k of genes in S; and the number n of samples in each class. To evaluate the impact of such parameters, for each combination of parameters in our experiments we generated 10 simulated datasets and run DAMOKLE on each dataset with S as the solution with the highest differential coverage, andthe fraction of times that DAMOKLE reported S that are in the solution with highest differential coverage found by DAMOKLE.the fraction of genes of The goal of our investigation using simulated data is to evaluate the impact of various parameters on ability of DAMOKLE to recover S, for values of c, with S as the best solution found and only a small fraction of the genes in S are part of the solution reported by DAMOKLE. However, as soon as the differential coverage is S most of the times, and even when the best solution does not entirely corresponds to S, more than S are reported in the best solution. For values of always reports the whole subnetwork S as the best solution. Given that many recent large cancer sequencing studies consider at least 200 samples, DAMOKLE will be useful to identify differentially mutated subnetworks in such studies. We first investigated the impact of the differential coverage entclass1pt{minimak in S. We tested the ability of DAMOKLE to identify a subnetwork S with differential coverage k of genes in S varies as S increases, the fraction of times S is the best solution as well as the fraction of genes reported in the best solution by S decreases, and for S only S are reported in the best solution by DAMOKLE. Therefore DAMOKLE can be used to identify relatively large subnetworks mutated in a significantly different number of samples even when the number of samples is relatively low.We then tested the performance of DAMOKLE as a function of the number of genes entclass1pt{miniman in each set S of n increases. We analyzed simulated datasets for S as the best solution and only a small fraction of all genes in S are reported in the solution. However, for S as the best solution only S are reported in the best solution. More interestingly, already for always reports S as the best solution. These results show that DAMOKLE can reliably identify relatively large differentially mutated subnetworks from currently available datasets of large cancer sequencing studies.Finally, we tested the performance of DAMOKLE as the number of samples entclass1pt{minimap-values with the permutation tests described in "We use DAMOKLE to analyze somatic mutations from The Cancer Genome Atlas. We first compared two similar cancer types and two very different cancer types to test whether DAMOKLE behaves as expected on these types. We then analyzed two pairs of cancer types where differences in alterations are unclear. In all cases we run DAMOKLE with We used DAMOKLE to analyze 188 samples of lung squamous cell carcinoma (LUSC) and 183 samples of lung adenocarcinoma (LUAD). We only considered single nucleotide variants (SNVs)pression  but are pression .We used DAMOKLE to analyze 456 samples of colorectal adenocarcinoma (COADREAD) and 496 samples of ovarian serous cystadenocarcinoma (OV) using only SNVs.l cancer , but altl cancer . The TFGCOADREAD . The higCOADREAD .p = 0.16). We first considered single genes, identifying TP53 with high . DAMOKLE identifies the significant  and copy number aberrations (CNAs) in 509 samples of lower grade glioma (LGG) and in 303 samples of glioblastoma multiforme (GBM).pathways . Interespathways  did not comparative analysis of mutations in two sets of samples. We tested DAMOKLE on simulated and real data. The results on simulated data show that DAMOKLE identifies significant subnetworks with currently available sample sizes. The results on two large cancer datasets, each comprising genome-wide measurements of DNA mutations in two cancer subtypes, shows that DAMOKLE identifies subnetworks that are not found by methods not designed for the comparative analysis of mutations in two sets of samples.In this work we study the problem of finding subnetworks of a large interaction network with significant difference in mutation frequency in two sets of cancer samples. This problem is extremely important to identify mutated mechanisms that are specific to a cancer (sub)type as well as for the identification of mechanisms related to clinical features . We provide a formal definition of the problem and show that the associated computational problem is NP-hard. We design, analyze, implement, and test a simple and efficient algorithm, DAMOKLE, which we prove identifies significant subnetworks when enough data from a reasonable generative model for cancer mutations is provided. Our results also show that the subnetworks identified by DAMOKLE cannot be identified by methods not designed for the While we provide a first method for the differential analysis of cohorts of cancer samples, several research directions remain. First, differences in the frequency of mutation of a subnetwork in two sets of cancer cohorts may be due to external (or hidden) variables, as for example the mutation rate of each cohort. While at the moment we ensure before running the analysis that no significant difference in mutation rate is present between the two sets, performing the analysis while correcting for possible differences in such confounding variable or in others would greatly expand the applicability of our method. Second, for some interaction networks  that are relatively more dense than the protein–protein interaction network we consider, requiring a minimum connectivity  in the subnetwork may be beneficial, and the design of efficient algorithms considering such requirement is an interesting direction of research. Third, different types of mutation patterns  among two set of samples could be explored . Fourth


Background: RNA sequencing has enabled high-throughput and fine-grained quantitative analyses of the transcriptome. While differential gene expression is the most widely used application of this technology, RNA-seq data also has the resolution to infer differential transcript usage (DTU), which can elucidate the role of different transcript isoforms between experimental conditions, cell types or tissues. DTU has typically been inferred from exon-count data, which has issues with assigning reads unambiguously to counting bins, and requires alignment of reads to the genome. Recently, approaches have emerged that use transcript quantification estimates directly for DTU. Transcript counts can be inferred from 'pseudo' or lightweight aligners, which are significantly faster than traditional genome alignment. However, recent evaluations show lower sensitivity in DTU analysis compared to exon-level analysis. Transcript abundances are estimated from equivalence classes (ECs), which determine the transcripts that any given read is compatible with. Recent work has proposed performing a variety of RNA-seq analysis directly on equivalence class counts (ECCs).Methods: Here we demonstrate that ECCs can be used effectively with existing count-based methods for detecting DTU. We evaluate this approach on simulated human and drosophila data, as well as on a real dataset through subset testing.Results: We find that ECCs have similar sensitivity and false discovery rates as exon-level counts but can be generated in a fraction of the time through the use of pseudo-aligners.Conclusions: We posit that equivalence class read counts are a natural unit on which to perform differential transcript usage analysis. Therefore, one important analysis task is to look for differential transcript usage (DTU) between samples.RNA sequencing with short-read sequencing technologies (RNA-seq) has been used for over a decade for exploring the transcriptome. While differential gene expression is one of the most widely used applications of this data, significantly higher resolution can be achieved by using the data to explore the multiple transcripts expressed from each gene locus. In particular, it has been shown that each gene can have multiple isoforms, sometimes with distinct functions, and the dominant transcript can be different across samplescufflinks/cuffdiff)2 deconvolve transcript read distributions and test differences in inferred transcript abundances. Other methods consider reads supporting particular isoforms or junctions (such as MISO3 orleafCutter4). Alternatively, DTU can be inferred through differences in exon usage, where the proportions of RNA-Seq reads aligning to each exon change relative to each other between biological groups. Anderset al.5 showed that exon read counts could be used to test for differential exon usage with a generalized linear model that accounts for biological variability. However, counting fragments across exons is not ideal because many fragments will align across multiple exons, making their assignment to an individual exon ambiguous. Moreover, individual exons often need to be partitioned into multiple disjoint counting bins when exon lengths differ between transcripts. Genomes of complex organisms typically contain more exons per gene than transcripts per gene. Supplementary Table 2 shows the human reference example, with an average of 3.5 transcripts per gene and 11.9 exons per gene. Spreading information over a larger number of bins, such as exons that are always transcribed together, results in lower power in statistical tests for testing for differences between samples.Several approaches already exist to characterise DTU. Transcript-assembly based approaches , domination (high abundance isoform(s) that mask transcript-level differences) and collapsing 9. This methodology utilises aggregation of p-values to identify differential transcript expression. Isoform switching, however, cannot be distinguished using this method if gene expression is also changing between conditions. Here we focus on detecting differences in isoform expression irrespective of gene expression differences, using methods originally designed for testing exon read counts. We evaluate the appropriateness of equivalence class read counts as an alternative choice for quantification compared to exon- and transcript-level quantification. We propose that DTU can be more accurately detected using equivalence class counts directly, rather than using these counts to first estimate individual transcript abundances before performing DTU. Sonesonet al. applied a conceptually similar method with MISO3 by defining counting bins as combinations isoforms and counting according to isoform compatibility6. In our scenario, count-based DTU testing procedures such as DEXSeq are applied directly to equivalence class counts generated from fast lightweight aligners, such as Salmon and Kallisto. DTU testing on equivalence class counts is not only fast but also bypasses inherent uncertainty in directly estimating transcript abundances before statistical testing.Recently, equivalence class counts have been used for clustering single-cellsWe evaluate the performance of DTU testing on equivalence class read counts using real and simulated data, and show that the approach yields higher sensitivity and lower false discovery rates than estimating counts from transcript abundances, and performs faster with accuracies similar or better than counting across exons.13.The method we propose is to first perform alignment with a lightweight aligner and extract equivalence class (EC or transcript compatibility) counts. These ECCs are assigned to genes using the annotation of the transcripts matching to the EC. Next, each gene is tested for DTU between conditions using a count based statistical testing method where exon counts are replaced with EC counts . Signifiet al.6, simulated human data from Loveet al.13 and biological data from Bottomlyet al.14. Each of the Soneson datasets consists of two sample groups, each with three replicates, where 1000 genes were randomly selected to have DTU such that the expression levels of the two most abundant transcripts were switched. The Loveet al. data contains genes defined with differential transcript expression and DTU across two groups with 12 replicates each. The Bottomly dataset contains 10 and 11 replicates each from two mouse strains that were used to call truth and then were subsampled to three replicates in the testing scenarios.We evaluated performance on both simulated and real datasets, using simulated data from human and drosophila from SonesonThe number of counting bins used for DTU detection has an impact on sensitivity. More bins leads to lower average counts per bin and therefore lower statistical power per bin and more multiple testing correction. We therefore examined the number of ECs, transcripts and exons present in each dataset. Although the theoretical number of ECs from a set of transcripts can be calculated from the annotation and has the potential to be large, not all combinations of transcripts exist or are expressed. The number of equivalence classes calculated from pseudo-alignment depends on the experimental data as only ECs with reads assigned to them are reported. We compared the number of transcripts and exon bins in the three datasets (with at least one read) to the number of reported ECs. In both the simulated human and drosophila datasets, as well as in the Bottomly mouse data, the number of ECs is greater than the number of transcripts, but substantially fewer than the number of exons, indicating that there might be more power for testing DTU using ECCs, compared to exon counts .2 variance to mean ratio of -2.249 and -1.519 in exons and ECs respectively, compared to 0.115 in transcripts. The simulated data followed a similar pattern. Supplementary Figure 115 shows the dispersion-mean trends, again demonstrating lower dispersion in ECs compared to transcript abundance estimates. We hypothesise that the greater dispersion observed for transcript data arises from the abundance estimation step used by pseudo-aligners to infer transcript counts. Due to the lower dispersion, we anticipate that analysis of ECCs yield greater power for DTU compared to transcript abundance estimates.In addition, we found that the variability of counts across replicates calculated from ECs was lower than that from estimated transcript abundances across all three data sets . Count vet al.6; DEXSeq’s default counting pipeline and featureCounts were shown to perform best. We recalculated exon counts using DEXSeq’s counting pipeline  and ran Salmon8 to obtain both transcript abundance estimates and equivalence class counts. All other comparison results were obtained from Sonesonet al.6. We also included MISO’s results as the method was implemented in a conceptually-similar way to our proposed EC method. For the simulated datasets, we found that ECs had the highest sensitivity in both the drosophila and human datasets  for DTU using DEXseq on counts generated from transcript abundance estimates, exons and ECs. To calculate the FDR, we considered the set of 'true' DTU genes to be the union of all genes called significant (FDR < 0.05) across the three methods. To calculate the TPR, the intersect of genes called by all methods was used. Supplementary Figure 215 shows the number of significant genes and overlap between all three methods. Exons called the highest number of genes with significant DTU . In contrast, transcripts called the fewest number of significant genes (391). Similar to the FDR experiments described in Pimentelet al.16, we randomly selected three samples per condition and performed DTU using all three methods and repeated this for 20 iterations.15. The ECC-based method had consistently lower number of false positives, which is also illustrated by the rank-order plot , showing the number of false positives present in the top 500 FDR-ranked genes. The range of FDR results across different subsets of the data was generally higher for ECCs and transcript counts, which may be the result of substructure in the data. However, FDR was similar to exon-counts in each random selection and consistently lower than in transcripts . In terms of the TPR, ECs performed better than transcripts, but worse than exons when using the union of all methods as the truth set. In the Bottomly analysis, Salmon was used as a representative method for transcript abundance estimation. We also performed the analysis with Kallisto7, which gave results consistent with Salmon .We next tested the performance of the EC method on a biological dataset from Bottomlyet al.13 data. Here we found that ECC and transcript count performance was similar, with exon count analysis faring significantly worse. The simulations were based on baseline abundances derived using Salmon, which may have favoured Salmon-derived transcript quantifications in the downstream analysis. There results, taken together with the Soneson simulation and the Bottomly date, indicate that ECCs perform as well as the best method regardless of the assumptions and biases in the datasets. To evaluate the performance of ECC testing on a different simulation scheme, we ran the ECC, transcript count  and exon count-based (using DEXSeq-count) methods on the LoveWhile the performance of EC counts in term of sensitivity and FDR are only slightly better than exons level counts in the Soneson and Bottomly data sets, another advantage of using ECs for analysis is the speed of alignment. The process can be broken down into workflow components that include alignment of sequenced reads, quantification and testing.15), and alignment was found to use the most RAM. Overall, methods utilising pseudo alignment required significantly lower memory compared with traditional alignment. For the most RAM intensive dataset, the human simulation, exon counting required 29 GB compared to 10 GB for ECs and 5 GB for estimated transcript abundances.We also considered peak RAM usage .17. Although, we do not consider probability-based equivalence classes in this work, incorporating this information for DTU deserves exploration in future work. In addition, ECCs may be calculated from full read alignment rather than pseudo-alignment19, which has the potential to improve accuracy further. In this work, we limited our investigation to comparing the best counting metric preceding DTU statistical testing, using DEXSeq as a representative method. Evaluation of statistical testing methods for DTU is outside the scope of this manuscript and would require further work.The ECs used in our evaluation are defined using only the set of transcripts for which reads are compatible. Extensions to this model have been proposed that incorporate read-level information, such as fragment length, to more accurately calculate the probability of a read arising from a given transcript20 or Sashimi plots21, which are methods for visualising the transcriptome, could be used for interpretation. Alternatively, transcript abundances, which are generated together with ECCs, can still be used to provide insight into the isoform switching.One limitation to consider is transcriptome reference completeness. Pseudo-alignment is dependent on the reference, and therefore unannotated, or poorly annotated transcripts may influence downstream results. Additionally, interpretation of the results may be more difficult with equivalence classes compared with exon and transcript-based approaches. Although we can detect DTU at the gene-level, it is not simple to determine which isoforms have changed abundance without further work. We propose that superTranscripts10, gene-level differential expression12, single-cell transcriptomics11 and fusion detection22. We foresee that equivalence classes could serve as a base unit of measurement in many other types of analyses.Finally, in this work, we have focused on differential transcript usage, but ECCs have the potential to be useful in a range of other expression analysis. ECCs have already been applied to areas such as clustering and dimensionality reductionFastQC v0.11.4, and therefore performed trimming usingTrimmomatic23 0.35, using recommended parameters  MINLEN:36 LEADING:3 TRAILING:3 SLIDINGWINDOW:4:15). The simulated Soneson data was not trimmed.We detected sequence content bias in the Bottomly RNA-seq data usingSalmon8 v0.13.0 (development version) was run on the drosophila, human, Love and Bottomly datasets in quant mode using default parameters. Transcript-level count estimates was obtained using tximport24 using the ‘scaledTPM’ scaling option. To obtain ECCs, the--dumpEq argument was used, as well as the--skipQuant to skip the quantification step.Kallisto7 0.43.0 was run inpseudo mode with the--batch argument to run all samples simultaneously. Fragment length and standard deviation were estimated from the read lengths of a single sample (SRR099223) from the Bottomly data . Equivalence classes were then matched between samples and compiled into a matrix using the python scripts , available onGitHub and archived onZenodo15. Equivalence classes mapping to more than a single gene were removed. No other filtering was performed on any of the data types.To obtain transcript abundance counts,STAR25 v2.5.2a with default parameters, then the DEXSeq-count annotation was prepared excluding overlapping exon-parts, from different genes, on the same strand (--aggregate=‘no’). DEXSeq-count was then run using default parameters to obtain read counts for the counting bins specified in the GTF reference. Filtering tests on the Soneson data were performed using DRIMSeq26 with min_samps_feature_expr = 3, min_feature_expr = 10, min_samps_gene_expression = 6 and min_gene_expr = 10. The same genome and transcriptome references for drosophila and human were used as in Sonesonet al.6, with only protein-coding transcripts considered for the Salmon index (as the simulations only considered protein-coding genes). For the Bottomly data, we used the NCBIM37 mm9 mouse genome and Ensembl release 67 transcriptome. Non-protein-coding transcripts were filtered out, as with the Soneson transcriptome reference in order to keep the references comparable. The samples used in the Bottomly iteration experiments were checked to ensure each sample was used in at least one iteration. We used the Gencode v28 transcriptome reference, and hg38 for the genome reference for the Love data.DEXSeq v1.26 was used to run all DTU analyses, and theperGeneQValue function was used to obtain gene-level FDR values from features. Cross-replicate log2(var / mean) calculations were performed on count-per-million-transformed data with light filtering (at least one sample had to have a CPM >= of 1 per feature). Compute times and RAM usage were calculated per process using /usr/bin/time -v.To perform the exon-based counts, raw reads were first aligned usinghttps://doi.org/10.1101/501106).An earlier version of this article can be found on bioRxiv  os       CentOS release 6.7    system   x86_64, linux-gnu            ui       RStudio                      language (EN)                         collate  en_US.UTF-8                  ctype    en_US.UTF-8                  tz       Australia/Melbourne          date     2019-04-16                  ─ Packages ────────────────────────────────────────────────────────────────────────────────────────────────────────────── package              * version   date       lib source                             acepack                1.4.1     2016-10-29 [2] CRAN (R 3.5.0)                     annotate               1.58.0    2018-05-15 [2] Bioconductor                       AnnotationDbi        * 1.42.1    2018-05-15 [2] Bioconductor                       assertthat             0.2.1     2019-03-21 [1] CRAN (R 3.5.0)                     backports              1.1.2     2017-12-13 [2] CRAN (R 3.5.0)                     base64enc              0.1-3     2015-07-28 [2] CRAN (R 3.5.0)                     bindr                  0.1.1     2018-03-13 [2] CRAN (R 3.5.0)                     bindrcpp               0.2.2     2018-03-29 [2] CRAN (R 3.5.0)                     Biobase              * 2.40.0    2018-05-15 [2] Bioconductor                       BiocGenerics         * 0.26.0    2018-05-15 [2] Bioconductor                       BiocParallel         * 1.14.2    2018-07-08 [2] Bioconductor                       biomaRt                2.36.1    2018-06-05 [2] Bioconductor                       Biostrings             2.48.0    2018-05-15 [2] Bioconductor                       bit                    1.1-14    2018-05-29 [2] CRAN (R 3.5.0)                     bit64                  0.9-7     2017-05-08 [2] CRAN (R 3.5.0)                     bitops                 1.0-6     2013-08-17 [2] CRAN (R 3.5.0)                     blob                   1.1.1     2018-03-25 [2] CRAN (R 3.5.0)                     checkmate              1.8.5     2017-10-24 [2] CRAN (R 3.5.0)                     cli                    1.1.0     2019-03-19 [1] CRAN (R 3.5.0)                     cluster                2.0.7-1   2018-04-13 [2] CRAN (R 3.5.0)                     colorspace             1.3-2     2016-12-14 [2] CRAN (R 3.5.0)                     crayon                 1.3.4     2017-09-16 [2] CRAN (R 3.5.0)                     data.table           * 1.11.4    2018-05-27 [2] CRAN (R 3.5.0)                     DBI                    1.0.0     2018-05-02 [2] CRAN (R 3.5.0)                     DelayedArray         * 0.6.5     2018-08-15 [2] Bioconductor                       DESeq2               * 1.20.0    2018-06-18 [2] Bioconductor                       DEXSeq               * 1.26.0    2018-07-19 [1] Bioconductor                       digest                 0.6.16    2018-08-22 [2] CRAN (R 3.5.0)                     dplyr                * 0.7.6     2018-06-29 [2] CRAN (R 3.5.0)                     DRIMSeq              * 1.8.0     2018-10-30 [1] Bioconductor                       edgeR                * 3.22.3    2018-06-21 [2] Bioconductor                       evaluate               0.11      2018-07-17 [2] CRAN (R 3.5.0)                     foreign                0.8-71    2018-07-20 [2] CRAN (R 3.5.0)                     formatR                1.5       2017-04-25 [2] CRAN (R 3.5.0)                     Formula                1.2-3     2018-05-03 [2] CRAN (R 3.5.0)                     futile.logger        * 1.4.3     2016-07-10 [2] CRAN (R 3.5.0)                     futile.options         1.0.1     2018-04-20 [2] CRAN (R 3.5.0)                     genefilter             1.62.0    2018-06-18 [2] Bioconductor                       geneplotter            1.58.0    2018-05-15 [2] Bioconductor                       GenomeInfoDb         * 1.16.0    2018-05-15 [2] Bioconductor                       GenomeInfoDbData       1.1.0     2018-05-15 [2] Bioconductor                       GenomicRanges        * 1.32.6    2018-07-20 [2] Bioconductor                       ggplot2              * 3.0.0     2018-07-03 [2] CRAN (R 3.5.0)                     glue                   1.3.0     2018-07-17 [2] CRAN (R 3.5.0)                     gridExtra            * 2.3       2017-09-09 [2] CRAN (R 3.5.0)                     gtable                 0.2.0     2016-02-26 [2] CRAN (R 3.5.0)                     Hmisc                  4.1-1     2018-01-03 [2] CRAN (R 3.5.0)                     hms                    0.4.2     2018-03-10 [2] CRAN (R 3.5.0)                     htmlTable              1.12      2018-05-26 [2] CRAN (R 3.5.0)                     htmltools              0.3.6     2017-04-28 [2] CRAN (R 3.5.0)                     htmlwidgets            1.2       2018-04-19 [2] CRAN (R 3.5.0)                     httr                   1.4.0     2018-12-11 [1] CRAN (R 3.5.0)                     hwriter                1.3.2     2014-09-10 [2] CRAN (R 3.5.0)                     IRanges              * 2.14.11   2018-08-24 [2] Bioconductor                       knitr                  1.20      2018-02-20 [2] CRAN (R 3.5.0)                     lambda.r               1.2.3     2018-05-17 [2] CRAN (R 3.5.0)                     lattice                0.20-35   2017-03-25 [2] CRAN (R 3.5.0)                     latticeExtra           0.6-28    2016-02-09 [2] CRAN (R 3.5.0)                     lazyeval               0.2.1     2017-10-29 [2] CRAN (R 3.5.0)                     limma                * 3.36.2    2018-06-21 [2] Bioconductor                       locfit                 1.5-9.1   2013-04-20 [2] CRAN (R 3.5.0)                     magrittr               1.5       2014-11-22 [2] CRAN (R 3.5.0)                     Matrix                 1.2-14    2018-04-09 [2] CRAN (R 3.5.0)                     matrixStats          * 0.54.0    2018-07-23 [2] CRAN (R 3.5.0)                     memoise                1.1.0     2017-04-21 [2] CRAN (R 3.5.0)                     munsell                0.5.0     2018-06-12 [2] CRAN (R 3.5.0)                     nnet                   7.3-12    2016-02-02 [2] CRAN (R 3.5.0)                     pillar                 1.3.0     2018-07-14 [2] CRAN (R 3.5.0)                     pkgconfig              2.0.2     2018-08-16 [2] CRAN (R 3.5.0)                     plyr                   1.8.4     2016-06-08 [2] CRAN (R 3.5.0)                     prettyunits            1.0.2     2015-07-13 [2] CRAN (R 3.5.0)                     progress               1.2.0     2018-06-14 [2] CRAN (R 3.5.0)                     purrr                  0.2.5     2018-05-29 [2] CRAN (R 3.5.0)                     R6                     2.4.0     2019-02-14 [1] CRAN (R 3.5.0)                     RColorBrewer         * 1.1-2     2014-12-07 [2] CRAN (R 3.5.0)                     Rcpp                   0.12.18   2018-07-23 [2] CRAN (R 3.5.0)                     RCurl                  1.95-4.11 2018-07-15 [2] CRAN (R 3.5.0)                     readr                * 1.1.1     2017-05-16 [2] CRAN (R 3.5.0)                     reshape2               1.4.3     2017-12-11 [2] CRAN (R 3.5.0)                     rlang                  0.2.2     2018-08-16 [2] CRAN (R 3.5.0)                     rmarkdown              1.10.2    2018-06-18 [2] Github (rstudio/rmarkdown@18207b9) rpart                  4.1-13    2018-02-23 [2] CRAN (R 3.5.0)                     rprojroot              1.3-2     2018-01-03 [2] CRAN (R 3.5.0)                     Rsamtools              1.32.3    2018-08-22 [2] Bioconductor                       RSQLite                2.1.1     2018-05-06 [2] CRAN (R 3.5.0)                     rstudioapi             0.10      2019-03-19 [1] CRAN (R 3.5.0)                     S4Vectors            * 0.18.3    2018-06-18 [2] Bioconductor                       scales                 1.0.0     2018-08-09 [2] CRAN (R 3.5.0)                     sessioninfo            1.1.1     2018-11-05 [1] CRAN (R 3.5.0)                     statmod                1.4.30    2017-06-18 [2] CRAN (R 3.5.0)                     stringi                1.2.4     2018-07-20 [2] CRAN (R 3.5.0)                     stringr                1.3.1     2018-05-10 [2] CRAN (R 3.5.0)                     SummarizedExperiment * 1.10.1    2018-05-15 [2] Bioconductor                       survival               2.42-6    2018-07-13 [2] CRAN (R 3.5.0)                     tibble                 1.4.2     2018-01-22 [2] CRAN (R 3.5.0)                     tidyselect             0.2.4     2018-02-26 [2] CRAN (R 3.5.0)                     tximport             * 1.8.0     2018-05-15 [2] Bioconductor                       VennDiagram          * 1.6.20    2018-03-28 [2] CRAN (R 3.5.0)                     withr                  2.1.2     2018-03-15 [2] CRAN (R 3.5.0)                     XML                    3.98-1.16 2018-08-19 [2] CRAN (R 3.5.0)                     xtable                 1.8-3     2018-08-29 [2] CRAN (R 3.5.0)                     XVector                0.20.0    2018-05-15 [2] Bioconductor                       yaml                   2.2.0     2018-07-25 [2] CRAN (R 3.5.0)                     zlibbioc               1.26.0    2018-05-15 [2] Bioconductor                      [1] /home/marek.cmero/R/x86_64-pc-linux-gnu-library/3.5[2] /usr/local/installed/R/3.5.0/lib64/R/library                                    et al.6 drosophila and human simulation data was obtained from ArrayExpress repository, accession numberE-MTAB-3766.The Sonesonhttp://imlspenticton.uzh.ch/robinson_lab/splicing_comparison/, files diff_splicing_comparison_drosophila.zip anddiff_splicing_comparison_human.zip.Truth data was obtained fromet al.14 dataset was obtained from the NCBI Sequence Read Archive, accession numberSRP004777.The Bottomlyet al. dataset was obtained from:The Lovehttps://doi.org/10.5281/zenodo.129137527https://doi.org/10.5281/zenodo.129140428https://doi.org/10.5281/zenodo.129144329et al.15 data feature counts are available from:The Lovehttps://doi.org/10.5281/zenodo.26447233031.All other feature count data is available in the ec-dtu-paper repositoryhttps://doi.org/10.5281/zenodo.264464915. The following extended data are available:Zenodo: Supplementary Material for "Fast and accurate differential transcript usage by testing equivalence class counts".et al., the red line shows the fitted dispersion-mean trend, the blue dots indicate the shrunken dispersion estimates, and the blue circles indicate outliers not shrunk towards the prior.Supplementary Figure 1: Shows the dispersion versus mean normalised counts for all features across the three data sets, generated using DEXSeq’s ‘plotDispEsts’ function. As described in Loveet al. data set for each feature.Supplementary Figure 2: Shows the significant genes (FDR < 0.05) shared between the methods, obtained from DEXSeq run on the full Bottomlyet al. data using random subsets of 3 vs. 3 samples across 20 iterations using FDR < 0.05 (FDR = 0.05 is indicated by the dotted line). The lines between the plots join data points from the same iteration. Each row uses a different ‘truth’ set: union is the set of genes called significant by any method, intersect is the set of genes called significant by all methods, and individual is the set of genes called significant by that method only.Supplementary Figure 3: Shows the ability of the three methods to recreate the results of a full comparison (10 vs. 11) of the BottomlySupplementary Figure 4: The number of false positives versus each gene’s rank (by FDR) for one iteration (3 vs. 3) of the Bottomly subset tests for the top 500 genes. The union of significant genes across all methods was used as the truth set.Supplementary Figure 5: Kallisto versus Salmon’s performance on the Bottomly subset testing experiments, using each method’s significant genes from the full (10 vs. 11) run as the truth set for calculating both metrics.Supplementary Figure 6: Performance of ECC, transcript count  and exon count (using DEXSeq-count) methods on the Soneson data with and without DRIMSeq filtering (see paper methods for full filtering criteria).EC DTU vignette for how to run the plotting code.Supplementary Figure 7: An example EC usage plot for a single gene from the hsapiens DECU results. Log EC usage is shown across each equivalence class for both conditions. Yellow blocks indicate significant ECs. As ECs do not correspond easily to genomic locations, no special ordering is applied the the ECs. In the given example, transcripts mapping to each significant EC can be observed in the data. The significant ec142580 corresponds to a single transcript (ENST00000443443), indicating that interpretation can be straight-forward if the EC is associated with a single transcript. See Section 6.4 of theet al. data (12 vs. 12 samples). The points show nominal FDR cutoffs of 0.01, 0.05 and 0.1.Supplementary Figure 8: Performance of ECC, transcript count  and exon count (using DEXSeq-count) methods on the LoveSupplementary Table 1: Maximum RAM usage for each job in GB. Each task was run as specified in the compute times table in the main paper .Extended data are available under the terms of thePipeline used to reproduce the quantification data generated in this paper:https://github.com/Oshlack/ec-dtu-pipe.Archived source code at time of publication:https://doi.org/10.5281/zenodo.264472531.Source code to run the analyses and generate the paper figures:https://github.com/Oshlack/ec-dtu-paper.Vignette for running DTU analyses using ECCs:https://github.com/Oshlack/ec-dtu-paper/wiki/VignetteArchived source code at time of publication:https://doi.org/10.5281/zenodo.264472432.License:MIT license. In the revised version of the manuscript, the authors have addressed all the concerns I raised in the previous version. The conclusions are well-supported by data and scientifically sound.I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. vignette provided in response to another reviewer1. Given this, I'm happy to approve their article for indexing.In version 2 of their paper, the authors thoroughly and comprehensively addressed my minor points from version 1. I appreciate the effort the authors did to improve the reproducibility of their work thanks to the R session information and their My only minor comment is that it would have been ideal to mention in the amendments from Version 1 that featureCounts and kallisto were removed from Figure 3. This change initially confused me, given my version 1 comment on the color issue between `featurecounts_flat` and `salmon`. I have checked that this change in Figure 3 does not affect the rest of the main text.2. Anyhow, for other readers: this change in Figure 3 was requested by one of the reviewers and the authors complied in their responseI have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Summaryet al. compares a TCC based DTU workflow against at transcript based and an exon based workflow using both simulated and real data reaching the conclusion that a TCC based workflow is superior – a novel and important finding. The manuscript is overall well presented and the approach is state-of-the art. Unfortunately the actual analysis suffers from major technical problems which undermines the findings presented. Importantly these problems were NOT fixed in the second edition! In this manuscript CmeroMajor comments:1). The lack of expression filtering will affect all analysis presented since many lowly or zero expressed features will be included thereby skewing the global comparison due to the difference in the proportion of low/zero in the different datasets compared  all the current analysis with an analysis based on dataset which have been pre-filtered for expression.1,2, the edgeR::filterByExpr function or use the classical 1 TPM cutoff.For inspiration of expression filtering seeNaturally the 3 dataset should be also filtered to be comparable .Except for Figure S6 all analysis presented is still performed on unfiltered data which makes the analysis untrustworthy. Therefore please note:et al. dataset as the fragment length in the Salmon quantification the entire analysis of the Bottomlyet al. dataset is not trustworthy since read-length != fragment-length (!). Using wrong fragment length will disproportionally affect the the Salmon based DTU analysis (not the exon and less the ECC based approaches since they do not rely on the TPM estimation affected by the fragment length). The authors really need to elaborate on this and justify the quantification approach!If, as indicated in the methods, the authors actually used the read length identified directly from the Bottomlyet al. 2018 data is now also included – but it seems strange strangely selective that it is not included in the main figures and the there is no analysis similar to Fig 2. Furthermore, the authors should  include an analysis of Loveet al. data using only 3 replicates – else the results are not comparable to the Bottomly and Sonneson results.I really appreciate that the LoveMinor comments:3.The authors should use “transcript compatibility counts” (TCC)  and derivations thereof) since TCC is the terminology used in the field when ECs are used for quantification IntroductionIn addition to exon and transcript based analysis approaches the authors also need to mention analysis of individual splice events (via tools such as SUPPA2 and rMATS) as well as the types of analysis which groups multiple features together (such as Leafcutter and MAJIQ) to make clear that there are 4 different approaches (with TCC based approach being a fifth (or a deviation of transcript based)). I do not require the authors to also compared the TCC based approach to the two omitted workflows – but they should be mentioned in the introduction for completeness.3 for the term “transcript compatibility count”.The authors should cite Resultset al. data as interpreting these measures together is necessary for a fair comparison.Please also show the average TPR vs FDR plot  for the BottomlyThe authors should discuss the much larger variance in FDR for TCC and exon based approaches. Methodset al. data as supplementary files to facilitate reproducibility.Please provide the unfiltered salmon quantification results (the “quant.sf” files) from the Bottomly Generally:I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. In this manuscript the authors Marek Cmero, Nadia M. Davidson and Alicia Oshlack describe in detail their proposed approach for identifying genes with differential transcript usage  using equivalence classes obtained through pseudo-alignment methods such as Salmon and Kallisto. By doing so, the authors leverage the computational advantages of pseudo-alignment methods, particularly speed and RAM requirements, together with statistical methods initially developed for differential exon usage (mainly DEXSeq) to identify genes with DTU events at a comparable (or even lower) error rates than exon based analyses which are more precise than transcript-level analyses. That is, their proposed method is fast, has low computational requirements (measured by RAM usage), and has error rates comparable if not better than state of the art alternatives. If time and computational resources are not limiting factors, the method the authors propose still gains an advantage over exon based methods by taking advantage of the nature of the human and mouse transcriptomes where genes can have more exons than transcripts, thus leading to power gains by their method. However, as presented their method also relies on a correct annotation of the transcriptome since un-annotated isoforms that involve new exons or new exon boundaries could potentially affect the results.https://github.com/Oshlack/ec-dtu-paper has all the foundations for making an R/Bioconductor package. Nevertheless, I think that it should be possible to apply their method in combination with others in order to minimize this issue. Overall the authors of this manuscript did an excellent job explaining their new method, comparing against earlier work, and explaining the different implications of their work. I look forward to their future software for applying this method asMinor pointsFigures 2 and 3 are missing labels for each sub-panel. For example, the legend for Figure 2 talks about (a) and (b) and while one can assume that the top panel is (a), it's best to be explicit about this type of information.here for some code by Rafael Irizarry orhere for longer code examples that I wrote. If it looks like a bell-shaped distribution, then I think that it could be okay to simply mention that in the text (in the case that the figure has many points and you prefer not to include it). From Figure 3 bottom panel, I can see that you already plotted the points in that case.Figure 2 top panel. Maybe show the data points in case the boxplots mask some information about the distribution. SeeThis also has some code for showing density plots with little bars in the bottom for the observed points.Figure 2 bottom panel.Page 5, bottom left. "Count variability of ECs was on average closer to the exon count variability distribution than ECs." is incorrect. I believe that it should read "Count variability of ECs was on average closer to the exon count variability distribution than transcripts".Figure 3, top panel. I can't distinguish the colors between `featurecounts_flat` and `salmon`. Figure 3, top panel. I don't know what the dotted lines represent: maybe FDR 0.01, 0.05 and 0.1?Figure 3, top panel. You might want to consider annotating with text the highest TPR point for each dataset which is quoted in the text in page 5 right side.I appreciate that Figure 3 (top panel) shows the full range, but maybe it would be useful to have a zoomed-in version in the supplementary material in order to see the differences more clearly. Maybe have a ylim from 0.5 to 0.8, and an xlim from 0 to 0.6 (or something like that).Page 6, bottom left. "ECs called the highest number of genes with significant DTU ." That sentence is incorrect based on Supplementary Figure 2. The numbers for genes with significant DTU match for the transcript and exon based methods, but they don't for the EC based method since 228 + 204 + 147 + 96 = 675. This numerical change affects the conclusions drawn from Supplementary Figure 2. Page 6, bottom right. Were all samples from the full bottomly dataset used in any of the 20 iterations? Or were there some samples that were used in many of the replications? With 20 iterations I guess that there's a small chance that some samples were under-represented or over-represented in the iterations.Figure 3, bottom panel. I really liked the lines you show in Supplementary Figure 3 to identify the different comparable replicates. The lines helped me visualize that the ranks were consistent across replicates since the lines rarely intersect each other. I suggest mentioning those lines in the caption for Figure 3 where you refer to Supplementary Figure 3, or maybe even swapping the panels from Figure 3 (bottom) for the equivalent ones from Supplementary Figure 3 .Page 7, right side. Typo "psuedo-" instead of "pseudo-".Page 8, left side. "We also found the analysis was quick to run and we provide code to convert [...]". I highly recommend including the URL here for the code or mention in a parenthesis in which section of the paper can one find the link to the code.Page 8, right side. I recommend also citing the Bottomly et al paper when you mention that the data was downloaded from SRR099223. You already cite the paper in other parts of your manuscript.tweet saying "this type of stuff is what the field needs".From the link to the bioRxiv pre-print I was able to find tweets citing the pre-print and have to agree with this Here, I didn't find the actual versions of the packages used. I suggest including the output of "options(width = 120); sessioninfo::session_info" somewhere in that repository.here. Normally R takes care of it.I think that you don't need to call gc manually in your function callshere that 8 cores were used for your method, I'm curious now looking at Supplementary Table 1 if the RAM presented there is by thread (core) or by process, and if so, how many cores were used for the other steps.Since I seeI have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Thank you for taking the time to review our paper and for the helpful suggestions.We have fixed the issues with Figures 2 and 3, and have added a description of the dotted lines in Figure 3a.We now use more distinguishable colour-palettes in all cases where colours identify data points.Supplementary Figure 6 has been added (showing filtered vs. unfiltered results), and uses zoomed-in axes. This plot contains the original results for ECC, exon and transcript counts, which should now be easier to distinguish.et al.[1] Supplementary Figure 3 for an example. We have therefore opted to retain boxplots. We note that the source code for generating these plots is available in the ec-dtu-paper github repository should readers want to inspect the raw data.We agree with the reviewer that boxplots can mask information. However, due to the discrete nature of the data, combined with the log-scale, results in a stepwise artefact. Please see SonesonWe also like the suggestion of the density ridges, however, due to the high number of data points (>1 million), this did not add any additional information to the visualisation.Instead of annotating TPR points for clarity on the plots in Figure 3a, we have added Supplementary Figure 6, which contains the same data points for ECs, transcripts and exons, as well as their respective performance using filtered features.We inspected the random samples selected for the Bottomly analyses (we have provided random seeds in the R markdown notebook) and noted that all samples were used at least one time. We have added code to the paper R markdown notebook to show sample usage across iterations. As is apparent in Supplementary Figure 3, the usage of particular samples is less important relative to the performance ranks observed of the method types across the iterations.The number of significant genes found, reflected in Supplementary Figure 2, has been corrected in the main text.We now mention the lines between replications in Figure 3’s caption.We have added all suggested links, references and fixed the typos pointed out in the paper.Session info has been added to the main paper, and we have removed the gc statements from the code.Supplementary Table 1 lists RAM by process; this has been clarified in the caption.  Minor points:References:Genome Biology,17(1), 1–15. https://doi.org/10.1186/s13059-015-0862-3[1] Soneson, C., Matthes, K. L., Nowicka, M., Law, C. W., & Robinson, M. D. (2016). Isoform prefiltering improves performance of count-based methods for analysis of differential transcript usage. Cmero, Davidson and Oshlack propose a novel approach to use RNA-seq data to test for differences in transcript usage between conditions. Instead of using exon-level or transcript-level counts, the authors propose using equivalence class counts (ECCs) resulting from pseudo-aligning/quasi-mapping to reference transcriptomes as input to existing methods to test for differences in exon usage. Using both simulated and real datasets, the authors show that using ECCs is comparable to using exon-level counts in terms of false discovery rates and true positive rates. They show that the ECC approach is computationally more efficient, although its results are more difficult to interpret. The analyses are reproducible and available through Github. The manuscript is well written and easy to follow. The whole idea is straightforward and very clever.Although some python scripts are available, they need better documentation and examples with toy datasets. From the code in the Github repository, it is not clear what steps one should follow to use the ECC approach for DTU. I would suggest writing a Bioconductor-like vignette that explains how to run kallisto/salmon with the parameters to get equivalence classes, how to use the python scripts to generate the equivalence class matrices, and how to transform these matrices into objects from the DEXSeq, DRIMseq and similar packages.As the authors acknowledge, a strong limitation of the ECC approach is result interpretation, which could be improved by visualizing the ECC equivalence classes. The interpretation of the ECC approach would be much easier if the authors provide code to plot transcripts and ECC classes of a gene  linked with the counts of each equivalence class for each sample. Minor points:1, I would suggest to describe what each point represents, what each axis is and how the metrics shown were defined.It would be helpful for the reader if the authors improved figure labels and figure legends. For example, in Figure 3a, rather than just referring to the paper by Soneson et al.In the introduction, the authors say “Typically, there will be more counting bins than transcripts, resulting in lower power to detect differences between samples.” Could the authors either explain further this statement or cite a reference that explains it?2).I understand the logic behind defining a “truth set” of genes with DTU in the analysis of the real data. However, the real number of true positives is likely larger and thus the resulting metric is not strictly a true positive rate. Perhaps it would be more accurate to call it differently  or require a junction. As ECs are determined by kmers, creating a direct mapping between ECs and the genome is challenging. Given the complexity of ECs, even a clean mapping between EC and genome position may be difficult to interpret. Given these limitations, we have instead opted to include a simple visualisation option, similar to DEXSeq, plotting EC names and their relative log counts across conditions, per gene. Such a visualisation example can be found in Supplementary Figure 7 (note the large number of ECs present in this gene). The function to create these plots (plot_ec_usage) is found in the ec-dtu-paper repository (and is referenced in the vignette) will also print all significant ECs of the gene, and their associated transcripts. In the example, one of the significant ECs has a single associated transcript, making DTU inference relatively straight-forward. Minor comments:We have added explanatory text in Figure 3a to explain the FDR/TPR plots and their respective FDR cutoffs. We have also added (a) and (b) labels for Figures 1-3.We have further explained the sentence about how the number of counting bins affects power. We have also added Supplementary Table 2 to illustrate this point, which shows the average number of exons and transcripts per gene for the Ensembl human gene reference.We have rename the TPR to ‘Fraction recalled’  to indicate that the metric does not strictly measure false positive rate.Major comments: Summaryet al suggest to use the ability of modern lightweight RNA-seq aligners to produce transcript compatibility counts (TCC) in combination with standard tools designed for differential transcript usage (DTU). Although the idea, as described in the introduction of the article, have been partly touched on by previous publications from the Pachter Lab, the approach used in this manuscript is novel since it describes a direct DTU analysis whereas the previous publications only inferred DTU indirectly. In this manuscript Cmeroet al compares a TCC based DTU workflow against at transcript based and an exon based workflow using both simulated and real data reaching the conclusion that a TCC based workflow is superior – a novel and important finding. The manuscript is overall well presented and the analysis approach is state-of-the art. Unfortunately the analysis is not quite extensive enough and it suffers from a few major technical problems which together with a general lack of clarity in the writing means the manuscript requires major revisions. In the manuscript “Fast and accurate differential transcript usage by testing equivalence class counts” by CmeroMajor comments:et al 20181 to test the effect of:et al data).A different simulation scheme  an analysis based on dataset which have been pre-filtered for expression. For inspiration of expression filtering1,2, the edgeR::filterByExpr function or use the classical 1 TPM cutoff. Naturally the 3 dataset should be also filtered to be comparable .All analysis presented is performed on unfiltered data which is problematic. Firstly it does not reflect typical RNA-seq analysis workflows which always include a step which filters out lowly expressed features before continued analysis. Furthermore, and more problematically, the lack of expression filtering will affect all analysis presented since many lowly or zero expressed features will be analyzed thereby skewing the global comparison due to the difference in the proportion of low/zero in the different datasets/pipelinesTo ensure correct quantification and to make the genome based (STAR) and lightweight based  analysis comparable the Salmon index should be build from all transcripts and subsequently (after quantification) the data should be reduced to only protein coding genes. This is necessary to ensure that reads mapping to both protein coding genes and lncRNAs are correctly quantified (and are quantified in a manner comparable to the genome based approach).The manuscript is in general not concise enough. Throughout, the manuscript is very hard to follow which workflow is referred to and the order in which workflows they are presented is not logical . Figures contain data never mention or used. Especially the discussion falls short of the mark as major parts are either repetitive non-informative.Minor comments:3.The authors should use “transcript compatibility counts” (TCC)  and derivations thereof) since TCC is the terminology used in the field when ECs are used for quantificationGenerally:The title seems to lack a word after such as “analysis” or “testing” after “differential transcript usage”Title:In the sentence “However, recent evaluations show lower sensitivity in DTU analysis” I guess the authors mean compared to exon-level analysis but this needs to be specified.The conclusion is to broad. The authors investigate DTU but conclude about “many” analysis. Such a sentence should probably be saved for a review paper.Abstract:In addition to exon and transcript based analysis approaches the authors also need to mention analysis of individual splice events (via tools such as SUPPA2 and rMATS) as well as the types of analysis which groups multiple features together (such as Leafcutter and MAJIQ) to make clear that there are 4 different approaches (with TCC based approach being a fifth (or a deviation of transcript based)). I do not require the authors to also compared the TCC based approach to the two omitted workflows – but they should be mentioned in the introduction for completeness.I think it could be beneficial to refer more the lower part of Figure 1 in the Introduction since it very clearly present the 3 different workflows in question?The drawbacks of pseudo/quasi alignment should be mentioned/discussed either in the introduction or discussion.In the sentence “Depending on its sequence, a read can align to all three transcripts, only two of the transcripts or just one transcript. These different combinations result in four possible equivalence classes, containing read counts, for this gene” the last statement is wrong. There are 6 possible (the authors omit uniquely t2 and uniquely t3). This should either be mention or it should be highlighted the example reads in Figure 1 give rise to 4 possibilities.3 for the term “transcript compatibility count”.The authors should provide a reference et al 20194 in the discussion of the Yi et al 2018 paper. Specifically the “catch-it-all” and “any transcript-level phenomena” part of the sentence in Cmeroet al: “Yi and colleagues have recently introduced direct differential testing on equivalence classes in a catch-all method to identify genes that display any transcript-level phenomena” needs to be changed as aggregation of DTE p-values cannot identify isoform switches if the gene expression is also changing  – hence the need for methods specifically designed for DTU detection and thereby also the need for the workflow presented by the authors Cmeroet al.The authors should also discuss the ideas presented in NtranosFor Figure 1: Could it be beneficial to divide Figure 1 into A and B referring to respectively TCC and analysis pipelines?Introduction:It needs to be described in detail how the fragment length and standard deviation were estimated from the Bottomly data since it is single end data. The actual values should also be reported for reproducibility.Is there a particular reason why Salmon/Kallisto was not run with the bias correction algorithms?Since the authors have to rerun salmon anyway (see major comments) it might be beneficial to update to Salmon v0.13.1 and also use the “--validateMappings” option.Please state the parameters used with Trimmomatic for reproducibility.Please provide info on how the transcript-level counts was obtained .Please also indicate how the exon/transcript level analysis was summarized to gene-level for each of the 3 workflows.et al data as supplementary files to facilitate reproducibility.Please provide the unfiltered salmon quantification results (the “quant.sf” files) from the BottomlyPlease provide details of how the STAR mapped data was converted to DEXSeq ready counts (currently only implied in the result section).Methods:From the first paragraph in results it is not clear that you are actually doing all 3 types of analysis and comparing them. And starting with mentioning the “alternative approach” is not reader friendly.1.For references to previous DTU benchmarking please also cite Love et al 2018For the “Fewer equivalence classes are expressed than exons” analysis it is unclear whether it is the number of exons or disjointed exon bins necessary for a standard DEXSeq workflow  which are quantified.Figure 2: Was any pseudocounts or transformation used to calculate the normalized cross replicate variance (Log2(var / mean))?Figure 3: Please add “A” and “B” to the figure in accordance with the figure legend.What is visualized is not explained in figure legend.It is currently not possible to distinguish between the different methods on the plot. Please provide zoom in versions of the plot to enhance visual comparison.From the point of this paper  it is very strange that multiple exon-based workflows as well as the result of a MISO based workflow (which is never discussed) are also shown. Would it not make more sense to only show exon-based workflow used and omit the MISO based workflow? Furthermore since the supplementary figures show that Salmon and Kallisto produce the same results why not only show one of them?Figure 3A:Please report which FDR cutoff was used to call significance.Please also report the result analysis on the feature level (transcript/exon) and not just for the gene-level.The authors should discuss the much larger variance in FDR for TCC and exon based approaches as well as the generally large FDR values Figure 3B:Results:“We propose that equivalence classes are the optimal unit for performing count based differential testing” is to broad a claim since this article is about DTU analysis. Save it for a review :-).sashimi plots in addition to superTranscripts – they had the visualization idea first.Please refer toDiscussion:I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Thank you for taking the time to review our paper and for the helpful suggestions.et al.[1]. EC-based results can be seen to be on par with transcript-based results . As we note in the revised paper, the simulations were based on baseline abundances derived using Salmon, which may have favoured Salmon-derived transcript quantifications in the downstream analysis. Together with the Soneson simulation, this indicates that ECCs perform as well as the best method regardless of the assumptions and biases in the simulated datasets. We ran our EC-based method, as well as the transcript and exon-based methods, through DTU analysis on the simulated data from LoveRegarding filtering, we note that DEXSeq automatically filters out zero-count features and low-count data. In order to show the effects of basic filtering on the EC, transcript and exon-based approaches, we present Supplementary Figure 6. This figure shows that filtering performs slightly better in controlling FDR per approach. Importantly, ECC-based DTU still out-performs transcript-based DTU in both the drosophila and human data.We agree with the reviewer’s comment that the pseudo-alignment index should be built on all transcripts and only subsequently filtered. The Soneson simulation data, however, is restricted to protein-coding genes only. All downstream results obtained from the Soneson data were run on references containing protein-coding genes only, therefore we opted to keep references consistent with the EC-based approach for optimal fairness. As we also compared the Bottomly data with the Soneson data in Figure 2, we opted to take the same approach with the Bottomly data. We have noted this decision in the methods. We used the whole transcript index without gene filtering for the Love data.We have updated the manuscript for conciseness, and updated labels figures and captions to improve clarity. Minor comments:We have opted to retain the use of ‘equivalence class counts’, noting that both ‘transcript compatibility counts’ and ‘equivalence class counts’ are used in the literature.We have updated the manuscript title for clarity.We have addressed the points regarding the abstract.General, title and abstract:We have now cited transcript-assembly and spliced-in DTU approaches.We now discuss some of the limitations of pseudo-alignment in the Discussion.We opted to show four equivalence classes in Figure 1 for simplicity. We have noted the possibility of ECs containing solely t2 and t3 in the main text.We have added a reference for the term ‘transcript compatibility counts’.et al.[2]We have corrected the discussion on ideas presented in NtranosWe have added (a) and (b) labels for Figures 1-3Introduction:Fragment lengths and standard deviations were estimated directly from the read lengths (as these varied between reads due to trimming). The length and standard deviation values have been added to the methods section.Salmon/Kallisto were run with default arguments  in order to run the software more-or-less ‘out of the box’ without parameter tuning, which may take focus away from the conceptual advance of using equivalence classes. Additionally, --validateMappings can be seen as a further optimisation to EC-derivation.Trimmomatic parameters have been added to the methods section. STAR was run with default parameters, which has also been added to the methods section.Tximport with ‘scaledTPM’ scaling was used to obtain transcript abundances from Salmon. This is now reflected in the methods.perGeneQValue function was used to obtain gene-level significance values. This is now reflected in the methods.DEXSeq’sSalmon’s “quant.sf” files are available in the ec-dtu-paper github repository. This is now reflected in the methods.We have clarified how exon counts are obtained from STAR counts.Methods:We have revised the first paragraph for clarity.et al.[1] in reference to DTU method benchmarking.We now cite LoveThe “Fewer equivalence classes are expressed than exons” analysis considers exon counting bins. This has now been clarified in the main text.Cross-replicate log2(var / mean) calculations were performed on CPM-transformed and lightly filtered data. This is now reflected in the methods section.We have described the figure in greater detail in the caption.Supplementary Figure 6 has been added, which uses zoomed-in axes and shows results for ECC, exon and transcript counts.et al. is conceptually similar and have removed feaureCounts and kallisto results to remove clutter.We show MISO as the way the method was used in SonesonFDR cutoff is now stated in the figure legend.Reporting the results on the feature level is not feasible as truth data is not available at the feature level. Additionally, equivalence classes do not map cleanly to features, which would make it difficult to asses the truth of features even if exon and transcript-level truth were available.For the Bottomly replication data, we now note the FDR variance of the ECC and exon-count based methods, indicating that this may be the result of substructure in the data. Importantly, FDR is lower in all iterations but one for ECCs compared with transcript counts.Figure 3Results:We have addressed the suggestions for the discussion.Discussion:  Major comments:References:F1000Research,7, 952.https://doi.org/10.12688/f1000research.15398.1[1] Love, M. I., Soneson, C., & Patro, R. (2018). Swimming downstream: statistical analysis of differential transcript usage following Salmon quantification.Nature Methods,16(February), 1.https://doi.org/10.1038/s41592-018-0303-9[2] Ntranos, V., Yi, L., Melsted, P., & Pachter, L. (2019). A discriminative learning approach to differential expression analysis for single-cell RNA-seq.