Tumour mutation burden and other exome-wide biomarkers are used to determine which patients will benefit from immunotherapy. However, the cost of whole exome sequencing limits the widespread use of such biomarkers. Here, we introduce a data-driven framework for the design of targeted gene panels for estimating a broad class of biomarkers including tumour mutation burden and tumour indel burden. Our first goal is to develop a generative model for the profile of mutation across the exome, which allows for gene- and variant type-dependent mutation rates. Based on this model, we then propose a procedure for constructing biomarker estimators. Our approach allows the practitioner to select a targeted gene panel of prespecified size and construct an estimator that only depends on the selected genes. Alternatively, our method may be applied to make predictions based on an existing gene panel, or to augment a gene panel to a given size. We demonstrate the excellent performance of our proposal using data from three non small-cell lung cancer studies, as well as data from six other cancer types. A data-driven procedure to estimate immunotherapy biomarkers is shown to perform well on data from three Non-Small Cell Lung Cancer studies as well as data from six other cancer types. The method is based on a generative model of how mutations arise in the tumour exome. Since the discovery of immune checkpoint blockade (ICB)4, there has been an explosion of interest in cancer therapies targeting immune response and ICB therapy is now widely used in clinical practice5. ICB therapy works by targeting natural mechanisms (or checkpoints) that disengage the immune system, for example, the proteins cytotoxic T-lymphocyte-associated protein 4 (CTLA-4) and programmed death ligand 1 (PD-L1)6. Inhibition of these checkpoints can promote a more aggressive anti-tumour immune response7, and in some patients, this leads to long-term remission8. However, ICB therapy is not always effective9 and may have adverse side-effects, so determining which patients will benefit in advance of treatment is vital.It has been understood for a long time that cancer, a disease occurring in many distinct tissues of the body and giving rise to a wide range of presentations, is initiated and driven by the accumulation of mutations in a subset of a person’s cells11. TMB is defined as the total number of non-synonymous mutations occurring throughout the tumour exome, and can be thought of as a proxy for how easily a tumour cell can be recognised as foreign by immune cells12. However, the cost of measuring TMB using whole exome sequencing (WES)13 currently prohibits its widespread use as standard-of-care. Sequencing costs, both financial and in terms of the time taken for results to be returned, are especially problematic in situations where high-depth sequencing is required, such as when utilising blood-based circulating tumour DNA (ctDNA) from liquid biopsy samples14. The same issues are encountered when measuring more recently proposed biomarkers such as tumour indel burden (TIB)16, which counts the number of frameshift insertion and deletion mutations. There is, therefore, demand for cost-effective approaches to estimate these biomarkers18.Exome-wide prognostic biomarkers for immunotherapy are now well-established—in particular, tumour mutation burden (TMB) is used to predict response to immunotherapy19 which implements the methodology and reproduces the experimental results in the paper.In this paper, we propose a data-driven method for biomarker estimation, based on a generative model of how mutations arise in the tumour exome. More precisely, we model mutation counts as independent Poisson variables, where the mean number of mutations depends on the gene of origin and variant type, as well as the background mutation rate (BMR) of the tumour. Due to the ultrahigh-dimensional nature of sequencing data and the fact that in many genes mutations arise purely according to the BMR, we use a regularisation penalty when estimating the parameters of the model. In addition, this identifies a subset of genes that are mutated above or below the background rate. Our model facilitates the construction of an estimator of TMB, based on a weighted linear combination of the number of mutations in each gene. The vector of weights is chosen to be sparse , so that our estimator of TMB may be calculated using only the mutation counts of a subset of genes. In particular, this allows for accurate estimation of TMB from a targeted gene panel, where the panel size (and therefore the cost) may be determined by the user. We also provide an R package ICBioMark20, and include a comparison with existing state-of-the-art approaches for estimating TMB. We further validate these results by testing the performance on data from two more NSCLC studies22. Moreover, since our model allows variant type-dependent mutation rates, it can be adapted easily to predict other biomarkers, such as TIB. Our method may also be used in combination with an existing targeted gene panel. In particular, we can estimate a biomarker directly from the panel, or first augment the panel and then construct an estimator. Finally, in order to further investigate the utility of our proposal across a range of mutation profiles, we use it to select targeted gene panels and estimate TMB in six other cancer types.We demonstrate the excellent practical performance of our framework using a non-small cell lung cancer (NSCLC) dataset23. investigate how the accuracy of predictions made in this way are affected by the size of the targeted region, where mutations are assumed to occur at a uniform rate throughout the genome. More recently Yao et al.24 modelled mutations as following a negative binomial distribution while allowing for gene-dependent rates, which are inferred by comparing non-synonymous and synonymous mutation counts. In contrast, our method does not require data including synonymous mutations. Where they are included, we do not assume that synonymous mutations occur at a uniform rate throughout the genome, giving us the flexibility to account for location-specific effects on synonymous mutation rates such as chromatin configuration25 and transcription-dependent repair mechanisms26. Linear regression models have been used for both panel selection27 and for biomarker prediction28. A review of some of the issues arising when dealing with targeted panel-based predictions of TMB biomarkers is given by Wu et al.29. Finally, we are unaware of any methods for estimating TIB from targeted gene panels.Due to its emergence as a biomarker for immunotherapy in recent years, a variety of groups have considered methods for estimating TMB. A simple and common way to estimate TMB is via the proportion of mutated codons in a targeted region. Budczies et al.20. Our main focus is the prediction of TMB, and we show that our method outperforms state-of-the-art approaches. We also analyse the suitability of our generative model, include a panel augmentation case study with the TST-170 gene panel, and consider the task of predicting the recently proposed biomarker TIB. Finally, in this section, we test our method’s generality and robustness by applying it to data from two further NSCLC datasets and then six further cancer types.In this section, we demonstrate in detail the practical performance of our proposal using the dataset from Campbell et al.20, which contains data from 1144 patient-derived tumours. For each sample in this dataset, we have the genomic locations and variant types of all mutations identified. At the time of the study, the patients had a variety of prognoses and smoking histories, were aged between 39 and 90, 41% were female and 59% were male; see Fig. 30. For simplicity, we only consider seven non-synonymous variant types: missense mutations (which are the most abundant), nonsense mutations, frameshift insertions/deletions, splice site mutations, in-frame insertions/deletions, nonstop mutations and translation start site mutations. We present the frequencies of these mutation types in Fig. Our methodology can be applied to any annotated mutation dataset obtained by WES. To demonstrate our proposal we make use of the NSCLC dataset produced by Campbell et al.G denotes the collection of genes that make up the exome. For a gene g ∈ G, let ℓg be the length of g in nucleotide bases, defined by maximum coding sequence as collected from the Ensembl database31. A gene panel is a subset P ⊆ G, and we write ℓP ≔ ∑g∈Pℓg for its total length. We let S denote the set of variant types in our data . Now, for i = 0, 1, …, n, let Migs denote the count of mutations in gene g ∈ G of type s ∈ S in the ith sample. Here the index i = 0 is used to refer to an unseen test sample for which we would like to make a prediction, while the indices i = 1, …, n enumerate the samples in our training dataset. In order to define the exome-wide biomarker of particular interest, we specify a subset of mutation types i = 0, …, n. For example, including all non-synonymous mutation types in i, whereas letting It is useful at this point to introduce the notation used throughout the paper. The set Mgs0: g ∈ P, s ∈ S}, where the panel P ⊆ G has length ℓP satisfying some upper bound. When it is clear from context that we are referring to the test sample and a specific choice of biomarker , amongst reformed smokers is 5.26 (0.84), and among lifelong non-smokers is 4.04 (1.12). This suggests that smokers may have higher BMRs, as would be expected.The first step in our analysis is to fit our generative model using only the training dataset. In particular, we obtain estimates of the model parameters using equation , where tentclass1pt{minimaentclass1pt{minimaWe now demonstrate the practical performance of our procedure for estimating TMB. First, it is shown that our method can indeed select gene panels of the size specified by the practitioner and those good predictions can be made even with small panel sizes (i.e. ≤1 Mb). We then compare the performance of our proposal with state-of-the-art estimation procedures based on a number of widely used gene panels.R2 score on the validation data as follows: given predictions of TMB, In order to evaluate the predictive performance of an estimator we calculate the 32 and Wu et al.29. Here we also report the estimated area under the precision-recall curve (AUPRC) for a classifier based on our estimator. We define the classifier as follows: first, in line with major clinical studies34 the true class membership of a tumour is defined according to whether it has t* ≔ 300 or more exome mutations (~10 Mut/Mb). In the validation set, this gives 47(27.5%) tumours with high TMB and 124(72.5%) with low TMB. Now, for a cutoff t ≥ 0, we can define a classifier by assigning a tumour to the high TMB class if its estimated TMB value is greater than or equal to t. For such a classifier, we have precision and recall  given byr(t), p(t)): t ∈ [0, ∞)}. Note that a perfect classifier achieves a AUPRC of 1, whereas a random guess, in this case, would have an average AUPRC of 0.275 .Other works have aimed to classify tumours into two groups ; see, for example, Buttner et al.R2 and AUPRC for the first-fit and refitted estimators  as the 27. The three next-generation sequencing panels that we consider are chosen for their relevance to TMB. These are TST-17035, Foundation One36 and MSK-IMPACT37. Further, the panel selected by the approach in Lyu et al.27 consists of the genes that are mutated more than 10% of the time, that are less than 0.015 Mb in length and for which the presence of a mutation in the gene is significantly associated with higher TMB values. For each panel P ⊆ G, we use four different methods to predict TMB:(i)P: we estimate TMB using WP is defined in (Our refitted estimator applied to the panel fined in .(ii)24.Estimation and classification of tumour mutation burden (ecTMB): the procedure proposed by Yao et al.(iii)P.A count estimator: TMB is estimated by (iv)A linear model: we estimate TMB via ordinary least-squares linear regression of TMB against The latter three comprise existing methods for estimating TMB available to practitioners. The second (ecTMB), which is based on a negative binomial model, is state-of-the-art. The third is a standard practical procedure for the estimation of TMB from targeted gene panels. The fourth is the approach proposed by Lyu et al.27. The refitted estimator applied to panel P is also included here, in order to demonstrate the utility of our approach even with a prespecified panel.We now compare our method with state-of-the-art estimators applied to commonly used gene panels, as well as a panel selected by the proposal of Lyu et al.R2 with a selected panel of half the size (with 0.2 Mb we obtain an R2 of 0.78). The best available existing method based on the TST-170 panel, in this case, the linear estimator, has an R2 of 0.74. Moreover, data-driven selection of panels considerably increases the classification performance for the whole range of panel sizes considered. In particular, even for the smallest panel size shown in Fig. 27, which involves applying the linear regression estimate to the panel selected as described above, also performs well here.We present the results of these comparisons in Fig. R2 value (on the test data) of 0.85. The other methods have R2 values of 0.67 (ecTMB), −36 (count) and 0.64 (linear regression). The count-based estimator here gives predictions which are reasonably well correlated to the true values of TMB but are positively biased. This is because our selection procedure tends to favour genes with higher overall mutation rates and thus a count estimator based on the highly mutated genes will overestimate the total number of mutations. We also include a red shaded region comprising all points for which heuristic 90% prediction intervals  include the true TMB value. We find in this case that 93.6% of the observations in the test set fall within this region, giving valid empirical coverage.Finally, in this subsection, we demonstrate the practical performance of our method using the test set, which until this point has been held out. Based on the validation results above, we take the panel of size 0.6 Mb selected by our procedure and use our refitted estimator on that panel to predict TMB for the 173 samples in the test set. For comparison, we also present predictions from ecTMB, the count-based estimator and the linear regression estimator applied to the same panel. In Fig. We may wish to include genes from a given panel, but use our methodology to augment the panel to include additional genes with the goal of obtaining more accurate predictions of TMB (or other biomarkers). In this section we demonstrate how this can be done starting with the TST-170 panel (~0.4 Mb) and augmenting to 0.6 Mb in length, demonstrating impressive gains in predictive performance.P0 taken to be the set of TST-170 genes and Q0 to be empty. The genes added to the panel are determined by the first-fit estimator in equation  of panel sizes and find that we are able to accurately predict TIB with larger panels. Our results also demonstrate that accurate classification of TIB status is possible even with small gene panels.In this section, we demonstrate how our method can be used to estimate TIB. This is more challenging than estimating TMB due to the low abundance of indel mutations relative to other variant types see Fig. d, as welSindel be the set of all frameshift insertion and deletion mutations, and apply our method introduced in the methods section with R2 and AUPRC, respectively, where in this case tumours are separated into two classes: high TIB (ten or more indel mutations) and low TIB (otherwise). In the validation dataset, this gives 57(33.3%) tumours in the high TIB class.We let R2 values for our first-fit and refitted estimators are much lower than what we achieved in estimating TMB. The refitted approach improves for larger panel sizes, while the first-fit estimator continues to perform relatively poorly. On the other hand, we see that the classification performance is impressive, with AUPRC values of above 0.8 for panels of less than 1 Mb in size.The results are presented in Fig. R2 = 0.35); the count estimator (R2 = −44); and the linear regression estimator (R2 = −0.15). We also include (shaded in red) the set of points for which 90% prediction intervals contain the true value. In this case, we find that 97.7% of test set points fall within this region.We now assess the performance on the test set of our refitted estimator of TIB applied to a selected panel of size 0.6 Mb, and we compare with a count-based estimator and linear regression estimator. We do not compare with ecTMB here, since it is designed to estimate TMB as opposed to TIB. The count-based estimator, in this case, scales the total number of non-synonymous mutations across the panel by the ratio of the length of the panel to that of the entire exome, and also by the relative frequency of indel mutations versus all non-synonymous mutations in the training dataset:21, which contains 75 samples with an average TMB of 261; and Rizvi et al.22, which contains 34 samples with an average TMB of 258.The aim of this section is to further test our proposed estimator of TMB by making use of two external NSCLC datasets for which the response to immunotherapy is available: Hellmann et al.R2 = 0.70 across the two datasets, with a joint AUPRC for classifying tumours to high or low TMB classes of 0.91.We first use our refitted estimator trained on the same data as in the section above on predicting TMB to predict TMB for the samples in the new datasets using the selected panel of size 0.6 Mb. The predictions are given in Fig. 21 study, 37 were identified as having a Durable Clinical Benefit (Class 1) in response to immunotherapy (PD-L1+CTLA-4 blockade), and the remaining 38 were deemed to have No Benefit (Class 0). Of the 34 samples in the Rizvi et al.22 studies, 14 were identified as having a Durable clinical benefit beyond 6 months (Class 1) in response to immunotherapy (Pembrolizumab), while the remaining 20 were deemed not to have such benefit (Class 0). Since the treatment and outcome definitions differ between studies, we separate them for analysis of response. We construct two simple classifiers for comparison, the first assigning a sample to Class 1 if the true TMB value is greater than some threshold t, and the second using our estimated value of TMB in the same way. In Fig. t varies). The area under the ROC curve is 0.68 for the Hellman et al.21 dataset when using the true TMB value and is 0.64 when using the estimated TMB value. Rizvi et al.22 have an area under the ROC curve of 0.79 using true TMB values and 0.76 using estimated TMB values. We see that, in both cases, very little is lost in terms of predicting response to immunotherapy when using our estimated value of TMB.These datasets also allow us to assess the practical utility of using our estimated TMB values to predict response to immunotherapy. Of the 75 samples in the Hellman et al.The aim of this section is to further demonstrate the performance of our proposed framework in a number of other cancer types. We apply our method for estimating TMB in six more cancer types, namely bladder cancer, breast cancer, colorectal cancer, melanoma, prostate cancer and renal cell cancer. For each cancer, data from two studies are used. Data from the first study is (randomly) split into a training and validation set; the training data is used to construct our estimator for a range of panel sizes, we then evaluate the predictive performance on the validation set . Further, in order to test the robustness of our approach to study effects, for each cancer type, we will also apply our fitted estimator (trained using data from the first study) to predict TMB values for tumours from the second study.47 used are detailed in Supplementary Table The twelve datasetsR2 values obtained on the internal validation set from the first study for the six cancer types as the panel size varies from 0.25 Mb to 1.25 Mb. The blue lines show the R2 values obtained when predicting TMB for tumours in the external test set from the second study. We see that the performance on the internal validation set is very good and broadly in line with the performance we obtained for the NSCLC dataset . The main factor affecting the performance appears to be the overall mutation rate; our method performs very well in cancer types with large mutation rates , but less well in the cancers with lower overall mutation rates . The performance on the renal cell dataset is particularly poor due to the combination of low sample size and the low average mutation rate.In Fig. The results on the external test datasets are more mixed; there is a drop-off in performance in comparison with the internal validation results for breast cancer and melanoma, but apparent improvement for prostate cancer. This highlights that study effects, such as differences in patient demographics and clinical profiles, as well as variations in sequencing technologies need to be considered carefully. In practice, one should ensure that the patients in the training data used to fit the model have similar characteristics to the intended test cohort.20, Hellman et al.21 and Rizvi et al.22, we have demonstrated the excellent predictive performance of our proposal for estimating tumour mutation burden and tumour indel burden, and shown that it outperforms the state-of-the-art procedures. We further tested the applicability and robustness of our method, by applying it to datasets on several other cancer types. Our framework can be applied to any tumour dataset containing annotated mutations, and we provide an R package19, which implements the methodology.We have introduced a data-driven framework for designing targeted gene panels which allows for cost-effective estimation of exome-wide biomarkers. Using the non-small cell lung cancer datasets from Campbell et al.11, it is of course desirable to improve the predictive performance by including other factors. For instance, these may include cancer type (and subtype), specific mutational signatures, aneuploidy and tumor histology, as well as other variables, such as gender, age and exogenous factors. Indeed, Litchfield et al.48 show that, by including markers of T-cell infiltration and other factors, a multivariate predictor of response to immunotherapy significantly improves the classification performance in comparison to using TMB alone. Nevertheless, one would certainly like to include TMB (or a closely related measure) as a factor in any classifier of response.The main use of TMB is often to help identify patients that are more likely to respond to immunotherapy. While TMB is a good single predictor of responseTP53, CDKN2A) are known tumour suppressors50 and KRAS is an oncogene51. Furthermore, indel mutations in KRAS are known to be deleterious for tumour cells52—in our work, the KRAS gene has a large negative indel-specific parameter ; we may seek to predict other features ; or we may wish to extend the method to incorporate multiple potentially incomplete data sources.We now describe the main statistical model that underpins our methodology. In order to account for selective pressures and other factors within the tumour, we allow the rate at which mutations occur to depend on the gene and type of mutation. Our model also includes a sample-dependent parameter to account for the differing levels of mutagenic exposure of tumours, which may occur due to exogenous  or endogenous  factors.Migs as independent Poisson random variables with mutation rates ϕigs > 0. More precisely, for i = 0, 1, …, n, g ∈ G and s ∈ S, we haveMigs and s1 ∈ S and all g ∈ G.We model the mutation counts μi corresponds to the BMR of the ith sample. The offset λg corresponds to an increased or decreased mutation rate relative to the BMR. The parameters νs and ηgs account for differences in frequency between mutation types for each gene.The terms in our model can be interpreted as follows. First, the parameter i = 0) has n + ∣S∣ + ∣G∣∣S∣ free parameters and we have n∣G∣∣S∣ independent observations in the training dataset. In principle, we could attempt to fit our model directly using maximum likelihood estimation. However, we wish to exploit the fact that most genes do not play an active role in the development of a tumour, and will be mutated approximately according to the BMR. This corresponds to the parameters λg and ηgs being zero for many g ∈ G. We, therefore, include an ℓ1-penalisation term applied to the parameters λg and ηgs when fitting our model. We do not penalise the parameters νs or μi since we expect that different mutation types occur at different rates and that the BMR is different in each sample.The model in  and 3)  3) , λ ≔ (λg: g ∈ G), ν ≔ (νs: s ∈ S) and η ≔ , and given training observations Migs = migs, we letκ1 ≥ 0 is a tuning parameter that controls the number of non-zero components in Writing entclass1pt{minimaified by . We thenT that only depends on the mutation counts in a gene panel P ⊂ G, subject to a constraint on ℓP. To that end, we consider estimators of the formS of variant types, rather than just those in w are chosen to minimise the expected squared error of T(w) based on the generative model described in the previous section.We now attend to our main goal of estimating a given exome-wide biomarker for the unseen test sample. Fix wgs = 1 for g ∈ G and wgs = 0 otherwise) will give T(w) = T. However, our aim is to make predictions based on a concise gene panel. If, for a given g ∈ G, we have wgs = 0 for all s ∈ S, then T(w) does not depend on the mutations in g and therefore the gene does not need to be included in the panel. In order to produce a suitable gene panel (i.e. with many wgs = 0), we penalise non-zero components of w when minimising the expected squared error. We define our final estimator via a refitting procedure, which improves the predictive performance by reducing the bias, and is also helpful when applying our procedure to panels with predetermined genes.Of course, setting T(w) isμ0, λg, νs and ηgs, the latter three of which are replaced by their estimates given in .To construct our estimator, note that under our model in  we have given in . It is af only depends on μ0 via the K(μ0) term, which can be interpreted as a penalty factor controlling the bias of our estimator. For example, we may insist that the squared bias term K(μ0) = ∞. In practice, we propose to choose the penalty K based on the training data.Note that f(w) is minimised by choosing w to be such that wgs = 1 for all wgs = 0 otherwise. As mentioned above, in order to form a concise panel while optimising predictive performance, we impose a constraint on the cost of sequencing the genes used in the estimation. More precisely, for a given w, an appropriate cost isAt this point L, our goal is to minimise f(w) such that ∥w∥G,0 ≤ L. In practice, this problem is non-convex and so computationally infeasible. As is common in high-dimensional optimisation problems, we consider a convex relaxation as follows: let ∥w∥G,1 ≔ ∑g∈Gℓg∥wg∥2, where g ∈ G, and ∥ ⋅ ∥2 is the Euclidean norm. Defineκ2 ≥ 0 is chosen to determine the size of the panel selected.Now, given a cost restriction P ⊆ G, letT using The final form of our estimator is obtained by a refitting procedure. First, for In practice, when designing gene panels a variety of factors contribute to the choice of genes included. For example, a gene may be included due to its relevance to immune response or its known association with a particular cancer type. If this is the case, measurements for these genes will be made regardless of their utility for predicting exome-wide biomarkers. When implementing our methodology, therefore, there is no additional cost to incorporate observations from these genes into our prediction if they will be helpful. Conversely, researchers may wish to exclude genes from a panel, or at least from actively contributing to the estimation of a biomarker, for instance, due to technical difficulties in sequencing a particular gene.P0, Q0 ⊆ G to be included and excluded from our panel, respectively. In this case, we replace P0 from the penalty term means that P0, while restricting our optimisation to Q0 by definition. This has the effect of augmenting the predetermined panel P0 with additional genes selected to improve predictive performance. We then perform refitting as described above. We demonstrate this procedure by augmenting the TST-170 gene panel.We can accommodate these restrictions by altering the structure of our regularisation penalty in . Supposeκ1 in , we will use tenfold cross-validation. To highlight one important aspect of our cross-validation procedure, recall that we consider the observations Migs as independent across the sample index i ∈ {1, …, n}, the gene g ∈ G and the mutation type s ∈ S. Our approach, therefore, involves splitting the entire set {: i = 1, …, n, g ∈ G, s ∈ S} of size n∣G∣∣S∣  into tenfolds uniformly at random. We then apply the estimation method in  and κ2. As mentioned above, we could set K(μ0) = ∞ to give an unbiased estimator, however, in practice, we found that a finite choice of K(μ0) leads to improved predictive performance. Our recommendation is to use 55) for μi, so that the penalisation is broadly in proportion with the largest values of μi in the training dataset. The tuning parameter κ2 controls the size of the gene panel selected in , which has a computational complexity of O(N∣G∣2∣S∣2) per iteration. We fit the model ten times, one for each fold in our cross-validation procedure. This is the most computationally demanding part of our proposal—in our experiments below, it takes approximately an hour to solve on a laptop—but it only needs to be carried out once for a given dataset. The convex optimisation problem in , we aim to find an interval α)-prediction interval for T. Of course, the mean squared error λ, η, ν and μ0, which are unknown. Our approach is to utilise the estimates μ0 with α)-prediction interval for T, we see in our experimental results that in practice this approach provides intervals with valid empirical coverage.Finally, we describe a heuristic procedure for producing prediction intervals around our point estimates. In particular, for a given confidence level fined in  depends ,ν^ (see ) and repFurther information on research design is available in the Peer Review FileSupplementary MaterialReporting Summary


RNA-seq data are increasingly used to derive prognostic signatures for cancer outcome prediction. A limitation of current predictors is their reliance on reference gene annotations, which amounts to ignoring large numbers of non-canonical RNAs produced in disease tissues. A recently introduced kind of transcriptome classifier operates entirely in a reference-free manner, relying on k-mers extracted from patient RNA-seq data.In this paper, we set out to compare conventional and reference-free signatures in risk and relapse prediction of prostate cancer. To compare the two approaches as fairly as possible, we set up a common procedure that takes as input either a k-mer count matrix or a gene expression matrix, extracts a signature and evaluates this signature in an independent dataset.We find that both gene-based and k-mer based classifiers had similarly high performances for risk prediction and a markedly lower performance for relapse prediction. Interestingly, the reference-free signatures included a set of sequences mapping to novel lncRNAs or variable regions of cancer driver genes that were not part of gene-based signatures.Reference-free classifiers are thus a promising strategy for the identification of novel prognostic RNA biomarkers.The online version contains supplementary material available at (10.1186/s12885-021-08021-1). The outcome of human cancer can be predicted in part through gene expression profiling –3. OutcoGene expression profiling of prostate biopsies is performed either using DNA microarrays –16 or hiOur group , 20 and Although attractive in principle, k-mer derived prognostic signatures pose two major challenges. First, a single RNA-seq dataset commonly contains tens to hundreds of millions distinct k-mers. Therefore false positive and replicability issues encountered with gene expression profiles –25 are eOur main objective here was to compare the characteristics and performances of reference-based and reference-free classifiers for PCa risk and relapse prediction. We built both types of classifiers using the same discovery dataset and assessed their performances in independent datasets using equivalent pipelines and parameters. For the reference-free approach, this required special developments to reduce the number of variables and to transfer expression measures between datasets. We present below a detailed analysis of the relative performances and sequence contents of the different classifiers and discuss possible future developments to improve performances of models.We used tumor samples from the TCGA-PRAD data collection  (N=505) TCGA-PRAD RNA-seq data were retrieved from dbGAP accession phs000178.v9.p8 with permission. ICGC-PRAD-CA RNA-seq data (EGAD00001004424) were downloaded from the European Genome-Phenome Archive (EGA) with permission. The RNA-seq files from the “Porto” cohort  were retWe built predictors for risk and relapse using two-class prediction models. To achieve a clear separation between the two classes, we only focused on high risk (HR) samples versus low risk (LR) samples, ignoring the medium risk, and we focused on relapse prior to a given year and non-relapse after a given year. For this reason, only a fraction of samples could be labelled for a given class in each set. Risk information was not available in the Stelloo dataset and relapse labelling on the ICGC dataset led to a small validation set (only 7 relapse samples).We classified tumor specimens into low-risk and high-risk groups using an adaptation of d’Amico’s classification which does not take into account the PSA rate but only the anatomo-pathological data on the basis of Gleason and TNM features as performed previously . Tumors For relapse analysis, we distinguished patients with biochemical relapse (BCR) and time to BCR <2yr and patients with no BCR after 5 years or longer, except for Stelloo et al. where only precomputed relapse data was available with cutoffs at 5yr and 10yr, respectively  normalization, while for contigs, we used Cb=109, or count per billion (CPB). For contigs, normalization is applied on the contig count table produced after contig extension and for genes it is applied on the recurrence filtered gene expression matrix.To account for differences in sequencing depth among samples, we applied a normalization step on feature counts (genes or contigs) in discovery and validation datasets. Each feature count in a sample is divided by the sum of all feature counts in this sample, then multiplied by a constant base number: f1 score by 5-fold cross validation (TP/(TP+FP) and recall=TP/(TP+FN) and FP,TP,FN are respectively the False Positive, True Positive and False Negative). In cases where 5-fold cross-validation returned an undefined value, f1 score was set to 0 (the worst). The average f1 score was used to rank features. The Bayes classifier implementation was taken from the MLPack library [Given the limited number of samples, it was necessary to reduce the number of features (genes or contigs) in the dataset. We discarded irrelevant features to focus on a subset of 500 top candidates for subsequent feature selection. To rank features, we selected a Bayes classifier because the C++ implementation of this classifier was the fastest to run among several available feature ranking tools. We did not try to optimize this part to avoid biasing the comparison towards gene-based or gene-free methods. In detail, we performed prediction of status (risk/relapse) using a Bayes classifier on each independent feature, after log transformation of the normalized counts . To assess the quality of the prediction, we computed the average  library .vs. Gencode V34 transcripts. HGNC symbols for signature genes were obtained from the Ensembl EnsDb.Hsapiens.v79 R package [To select a subset of non-correlated features (genes or contigs) among the top 500 candidates, we performed penalized logistic regression using the implementation from the glmnet R package . We impl package .To measure contig expression in the validation cohort we implemented the procedure schematized in Fig. We first applied the gene-free and gene-based signature discovery procedures detailed above to infer PCa risk signatures. The k-mer table for 374 TCGA-PRAD risk-labelled samples Fig. ” sectionThe 5.2M contig matrix and the 38k gene expression matrix were submitted to screening using univariate Bayes classification and the top scoring 500 features were retained for feature selection and model fitting. Interestingly, the 500 top scoring contigs were significantly longer than prior to selection , especially when considering that no gene information was used to infer our signature. The gene and contig signatures involved five shared genes: MYBPC1, ASPN, SLC22A3, SRD5A2 and CD38 , a gene also mapped by an exonic contig, the other an intron of LDLRAD4 (ctg_23). Contig ctg_23 corresponds to a 1.29 kb spliced transcript located between exons 4 and 5 of LDLRAD4 and is strongly upregulated in HR samples, as displayed in the Integrative Genomics Viewer (IGV)  in AdditTo assess the replicability of risk signatures, we evaluated their performance in the ICGC-PRAD independent dataset. To this aim, we developed a specific procedure to estimate the expression of an arbitrary sequence contig across datasets using matched k-mers see “” sectionFor relapse prediction, we distinguished patients with biochemical relapse within less than 2 years and patients with no BCR after 5 years or longer. Application of the gene-free and gene-based signature discovery procedures to relapse prediction produced a 14-contig reference-free signature and a 10-gene reference-based signature . The KLK3 contig is located in a distal part of the 3’ UTR region present only in longer isoforms of KLK3. Its lower expression in relapsed patients was unexpected as low expression of PSA is usually associated to a lower risk. It is possible though that only this longer isoform is depleted in relapsing samples. The expression boxplot shows the KLK2 contig occurs only in a few outlier patients while the AR and KLK3 contigs are common Fig. c. The coThe contig matching lncRNA AC069228.1 also raised our attention since AC069228.1 is the only gene mapped by contigs in both relapse and risk signatures. The AC069228.1 lncRNA is antisense of PPFIA2, a protein tyrosine phosphatase that is itself an alleged urine biomarker of PCa . The conOf note, the 10 genes in the reference-based signature were also clearly PCa-related: one was the major PCa biomarker PCA3  and 5 otContrary to the risk signatures, relapse signatures showed little overlap with each other and with published PCa signatures (Additional file As in the risk model, both reference-based and reference-free signatures had excellent cross-validation performance on the discovery set (AUC of 0.84 and 0.93 respectively, Fig. Low replicability of the relapse model may be caused in part by weaknesses in validation datasets: the ICGC dataset had only 7 samples labelled for relapse Fig. a and theWe evaluated here a method for building transcriptome classifiers that are totally reference-free, i.e. that do not require prior knowledge of genes or genome. The major interest of this approach lies in its ability to discover and incorporate in models previously unknown RNA biomarkers. Multiple examples exist of such disease-specific RNAs produced by genome alterations or deficient RNA processing and we hypothetized their inclusion in predictive models would be beneficial . ApplyinTo our knowledge the only other software using a reference-free approach for inferring predictive signatures is Gecko . Gecko uTransferring a reference-free model to a new dataset is challenging. This requires that important features, such as SNPs, are precisely evaluated in the independent dataset. To this aim, we transferred signatures between datasets based on exact k-mer matches. As k-mer contents vary a lot between library preparation protocols, we expected this strategy to show poor sensitivity when discovery and validation datasets differed substantially. Indeed, transfer of signatures trained on the TCGA-PRAD dataset to the low coverage Stelloo dataset caused the loss of a majority of contigs. However, in this particular case, the remaining contigs were sufficient to maintain a prediction performance at the same level as that of the gene-based signature.To compare the reference-free and reference-based strategies, a common evaluation framework was adopted. For both risk and relapse predictions, performances of the reference-free classifiers were on a par with that of reference-based classifiers. However while risk signatures showed satisfying reproducibility, relapse signatures performed poorly in independent datasets.A possible reason for the low performance of relapse models is our grouping of patients in discrete relapse and non relapse categories as done in other studies , 15, 16.A more general concern with relapse analysis is related to difficulty of predicting an outcome occurring several years after a sample is biopsied and analyzed. There might just be too little information available in the training data to infer a reliable classifier, a problem that is independent of the use of contigs or genes. However, both gene-level and contig-level signatures were highly enriched in PCa driver genes, which suggests information about tumor progression was indeed present in the primary tumor biopsy. The key problem with relapse analysis was more likely related to sample heterogeneity. The diversity of relapse mechanisms was not properly represented in a training set of 100 patients as we used here. Patient stratification have been proposed to deal with sample heterogeneity in omics data , 60. AdaFor prediction of PCa risk and relapse, reference-free classifiers did not significantly outperform reference-based classifiers, however they incorporated a distinct set of RNA sequences including unannotated RNAs and novel variants of annotated RNAs. It is likely that with other diseases and datasets, novel biomarkers will be identified with an even greater impact on prediction performance. The reference-free approach will be of particular interest in problems where unknown RNAs are expected to play an important role, such as when studying rare diseases, poorly studied tissue types or when analysing dual human-pathogen RNA-seq samples. Our strategy also permits to infer efficient transcriptome classifiers in species lacking an accurate genome or transcriptome reference.Additional file 1 Contig sequences and mapping locations in the risk and relapse signatures.Additional file 2 Published PCa risk and relapse signatures. Genes in common between published and this publication’s signatures.Additional file 3 Contents and expression characteristics of all signatures in the discovery and validation datasets.Additional file 4 Supplementary figures and tables.


Integrating multigenomic data to recognize cancer subtype is an important task in bioinformatics. In recent years, some multiview clustering algorithms have been proposed and applied to identify cancer subtype. However, these clustering algorithms ignore that each data contributes differently to the clustering results during the fusion process, and they require additional clustering steps to generate the final labels. In this paper, a new one-step method for cancer subtype recognition based on graph learning framework is designed, called Laplacian Rank Constrained Multiview Clustering (LRCMC). LRCMC first forms a graph for a single biological data to reveal the relationship between data points and uses affinity matrix to encode the graph structure. Then, it adds weights to measure the contribution of each graph and finally merges these individual graphs into a consensus graph. In addition, LRCMC constructs the adaptive neighbors to adjust the similarity of sample points, and it uses the rank constraint on the Laplacian matrix to ensure that each graph structure has the same connected components. Experiments on several benchmark datasets and The Cancer Genome Atlas (TCGA) datasets have demonstrated the effectiveness of the proposed algorithm comparing to the state-of-the-art methods. Tumor is a malignant heterogeneous disease caused by changes in cellular components at the levels of expression, epigenetics, transcription and proteomics. The heterogeneity will be reflected in that the same cancer will produce the subtypes with different phenotypes, which will affect the clinical treatment and prognosis ,2. With In order to implement the task of clustering, scholars initially focus on dimensionality reduction, matrix decomposition and linear regression technologies. They all use different strategies to project high-dimensional data into low-dimensional feature space, and then achieve clustering by k-means ,7,8,9,10However, most existing graph-based multiview clustering methods separate the data clustering process from graph learning process ,19. In sp value obtained from survival analysis on the TCGA dataset can all show that the proposed LRCMC approach achieves considerable improvement over the state-of-the-art baseline methods. In the analysis of the Glioblastoma Multiforme (GBM) subtypes, we found these clusters have biological significance, e.g., the Proneural subtype granted by G-CIMP phenotype has a better survival advantage. The source code and datasets can be found in the To sum up, we designed a graph-based multiview clustering algorithm, called Laplacian Rank Constrained Multiview Clustering (LRCMC). Firstly, the Laplacian Rank Constraint (LRC) algorithm  is used m views Z. Simultaneously, each view is automatically assigned a different weight Z during the fusion process. Finally, the learned consistent graph matrix Z is used to optimize the affinity graph matrix for each view. The LRC method is also imposed to constrain that the number of connected components in the Z is equal to the required number of clusters c by constructing the fusion embedded matrix U. Our LRCMC improves the affinity matrix of each view, builds a fused consistent graph matrix and obtains clustering results simultaneously.The overall flow of LRCMC is shown in v-th view data with vd features, where n is the number of data points. c categories, the obtained c connected components. Based on the following Theorem 1, Given a single biological data entation , Gaussiaentation , etc. FoTheorem 1.Multiplicity c of zero eigenvalues of Laplacian matrixis equal to the number of connected components of its similarity matrixWhen all the elements in property ,31.i-th smallest eigenvalue of c categories [c eigenvectors of c smallest eigenvalues. Theorem 1 means if tegories , Laplaci theorem , we can jth column of However, the solution to mn of Sv . FinallyA set of the affinity graph matrices Z with noise representation and outlier interference. (2) Z should be given greater weight to reduce the perturbation of the low-quality graphs on the fusion graph. In this way, Z can accurately capture the true similarity hidden in the multiview data. Therefore, we employed the proposed method of Nie et al. [Z as follows:Z and setting the derivative to zero, we can obtain:Integrating these basic graphs to form the fused affinity graph e et al.  to optimZ are obtained from Equation (3). In order to make the learned Z also have c connected components for clustering, the LRC term is added to Equation (3) according to Theorem 1 and Ky Fan’s Theorem. The objective function is as follows:c eigenvectors of c smallest eigenvalues, i.e., the embedded matrix corresponding to Z. Here, a set of weights Z. Therefore, we can combine Equations (2) and (7) into a final objective function, i.e., the proposed Laplacian Rank Constrained Multiview Clustering (LRCMC). It is represented as:As described in Z can help each other embedded in a joint coupling problem. The objective function Equation (8) enjoys the following properties:c connected components, instead of most multiview clustering methods requiring predefined graphs;Our method can effectively learn a set of affinity graph matrices with Z, rather than simply superimposing then together;In the graph fusion process, we assign the weight to each view to represent their contribution to the consistent graph Z, and at the same time complete the task of clustering.We use LRC to constantly adjust the structures of Here, we complete the tasks of graph construction, graph fusion and clustering in one step through the integrated model. In this way, the learning of Z, U. The specific solution process is as follows:1.Z and U, solve Fix Obviously, since the variables in Equation (8) are coupled to each other, we use alternating iterative method and Augmented Lagrange Multiplier (ALM) scheme to solve The Equation (8) becomes:i-th and j-th column of i, j)th element of Due to Denote Then, the Equation (11) can be written as:Therefore, the Lagrangian function of Equation (12) combined with its constraints can be defined as:ondition , the optk nonzero adaptive neighbors for The study in  found thMoreover, according to Equation (15) and the constraint condition Therefore, according to Equations (15) and (16), the range of Then, the parameter 2.Z and U, solve Fix Finally, according to Equations (15), (16) and (18), the optimal solution of Updating 3.Z and U, solve Fix Updating 4.U, solve Z.;Fix U, i, j)th element of Z, Equation (7) yields:Updating Z in Equation (8) is converted to Equation (7). Due to Denote Based on the Karush–Kuhn–Tucker (KKT) condition , the cloEquation (22) can be solved by an efficient optimization method proposed in .5.Z, solve U.Fix U is obtained as follows:According to the method of finding U is the c eigenvectors of c smallest eigenvalues.The final solution of In order to verify effectiveness of LRCMC in cancer subtype recognition, LRCMC was compared with four state-of-the-art clustering algorithms, i.e., ANF , SNF 1111, PFA  and MVCM3-source : It contCalt-7 : The objMSRC : The sceWebKB : It collThe benchmark datasets are described as follows:n, m, and c describe the number of samples, views, and classes for each dataset, respectively, vd denotes the i-th feature of these datasets.k required by ANF, SNF, MVCMO and LRCMC was set within the range of Z is greater than c, we will shrink c, we will increase Z. Algorithm 1. LRCMC algorithmInput: Original data m views, the number of clusters c, the number of neighbors k, the regularization parameter Output: The learned consensus matrix Z.Initialize the affinity matrices Initialize the embedded matrices Initialize the weights Initialize Z by connecting Initialize the fused embedded matrix U by using Equation (23);RepeatFix Z and U, update Fix Z and U, update Fix Z and U, update Fix U, update Z by using Equation (22);Fix Z. update U by using Equation (23);Until Satisfy Theorem 1 or the maximum iteration reached.The learned consensus matrix Z with exact c connected components, which are the final clusters.Three commonly used evaluation metrics, i.e., Accuracy (ACC), Normalized Mutual Information (NMI) and Purity, are used to quantitatively measure the clustering performance of the algorithms. The metrics compare the resulting labels with the real labels provided by the dataset. The larger the value obtained, the better the clustering results. To ensure the fairness of the comparison experiments, each algorithm was run 10 times to reduce the impact of randomness. The mean and standard deviation of the obtained metrics were calculated. In addition, the neighbor e et al. . β is serange of . If the To demonstrate the effectiveness of LRCMC in identifying cancer subtype, the designed LRCMC was applied to four cancer omics datasets, i.e., GBM, BIC, LSCC, and COAD. Each cancer subtype contains three types of expression data from different platforms, i.e., mRNA expression data, DNA methylation data and miRNA expression data. p values based on Cox log-rank model were used to evaluate the clustering results of these algorithms in survival analysis [p value is smaller, the survival rate between different groups is more significant and the difference is greater, which means the cluster is considered to have different characteristics of the underlying cancer subtypes. Cancer survival curves can also represent heterogeneity between different subtypes. As shown in p value in BIC, GBM, KRCCC and COAD. Other algorithms also had good results in specific datasets, but they were all lower than our algorithm. Therefore, we believe that LRCMC is significantly advantageous in the topic of cancer subtype recognition. To ensure that the identified cancer labels conform to the true clinical diagnosis, we specified that the number of samples in each cluster should be at least 3. We used the number of subtypes of GBM, BIC, LSCC, and COAD specified by Wang et al., which were 3, 5, 4 and 3, respectively. Then, the analysis . If the GBM is the most malignant glioma among astrocytomas. It has been studied and analyzed at the genetic level by many scholars, and specific subtypes and treatment protocols have been proposed. For example, according to the mRNA expression data, Verhaak et al.  reportedhttp://www.cbioportal.org/ accessed on 15 December 2020). The age profiles of the three clusters (IDH1) and CNVs  are significantly different in the three identified clusters. In particular, IDH1 mutation only occurs in cluster 2, while EGFR amplification is 0. Then, we divided the patients within the three clusters into two groups: patients treated with TMZ and those not treated with TMZ, then we compared the drug response. TMZ is a drug that is commonly used to treat GBM, but only responds well to a subset of patients. The p values of survival analysis in Cox log-rank model of the three cluster comparison experiments are 2.0 × 10−6, 0.76 and 0.01, respectively, which indicates that TMZ treatment has no effect on the patients in cluster 2. Therefore, in summary, we can infer that the subtype belonging to G-CLMP subtype and Proneural subtype might be a potentially new subtype. This also verified by the fact that that the Proneural subtype granted by the G-CIMP phenotype proposed by Canmenron et al. has unique properties [To further analyze the identified clusters, we downloaded clinical data, somatic mutation data and CNV data for all patients from the cBio Cancer Genomis Portal database . The gene differences in miRNA expression data were not significant enough  and were omitted from consideration. PRKAA1 overexpressed in Cluster 3, also known as AMPK, induces antitumor activity in GBM cells and has become a possible tumor control target [MUC1 overexpressed in cluster 1 is a pathogenic gene that induces GBM and can be used as a target for cellular immunotherapy [In addition, mRNA expression data and DNA methylation data were used to compare the differentially expressed genes in cluster 1 and 3 to look for the heterogeneity between them. We compared the genes in the two clusters using ANOVA (the lower the l target . MUC1 ovotherapy .https://toppgene.cchmc.org/enrichment.jsp accessed on 20 December 2020). From Finally, we compared the three clusters with normal samples and screened for differentially expressed genes using ANOVA. We did Gene Ontology (GO: BP), KEGG pathway and Disease Ontology (DO) enrichment analysis using the top 100 differential genes in ToppGene Suite database  instead of simply treating each view equally, a wealth of heterogeneous information is taken into account to provide appropriate weight for each view; (2) the tasks of constructing the affinity matrix of each view, learning the fused matrix and clustering are completed simultaneously in a system. Furthermore, LRCMC has the following two advantages in algorithm running: (1) there is no need to spend a lot of time choosing the appropriate parameters; (2) the final consensus graph has been assigned to the given categories without adding additional base clustering algorithms. We demonstrated the power of LRCMC using four benchmark datasets and four cancer datasets. The experiments show that LRCMC has a good clustering evaluation. The cancer subtype recognition results on GBM data show that LRCMC can effectively capture cancer subtypes with specific biological characteristics based on omics data.In addition, we must admit that LRCMC also has shortcomings and limitations. It is not suitable for binary data (somatic mutation) or categorical data , and has only limited application to continuous data (mRNA expression) to identify cancer subtype. It also does not have the ability to find the gene modules that affect differences in each subtype. Therefore, we will continue our efforts to improve and extend the LRCMC algorithm to explore cancer heterogeneity.


These improve statistical power by: (i) reducing the total features tested and (ii) relaxing the requirement of within-cohort uniformity at the transcript level. We propose Inter-N-of-1, a novel method, to identify meaningful differences between very small cohorts by using the effect size of ‘single-subject-study’-derived responsive biological mechanisms.Identifying altered transcripts between very small human cohorts is particularly challenging and is compounded by the low accrual rate of human subjects in rare diseases or sub-stratified common disorders. Yet, single-subject studies (SInter-N-of-1 requires applying previously published S3-type N-of-1-pathways MixEnrich to two paired samples (e.g. diseased versus unaffected tissues) for determining patient-specific enriched genes sets: Odds Ratios (S3-OR) and S3-variance using Gene Ontology Biological Processes. To evaluate small cohorts, we calculated the precision and recall of Inter-N-of-1 and that of a control method (GLM+EGS) when comparing two cohorts of decreasing sizes (from 20 versus 20 to 2 versus 2) in a comprehensive six-parameter simulation and in a proof-of-concept clinical dataset. In simulations, the Inter-N-of-1 median precision and recall are > 90% and >75% in cohorts of 3 versus 3 distinct subjects , whereas conventional methods outperform Inter-N-of-1 at sample sizes 9 versus 9 and larger. Similar results were obtained in the clinical proof-of-concept dataset.In each subject, R software is available at Lussierlab.net/BSSD. Empirical evidence unveils a methodological gap when comparing transcriptomic differences in biological mechanisms within very small human cohorts due to variations in heterogenicity, uncontrolled biology  and diversity of environmental factors  . Even in3). While transcript-level differences between two-sample remain inaccurate (gene set-level (pathway/biosystem) S3 have been shown to accurately discover altered biomechanisms from paired transcriptome samples drawn from the same patient under two conditions  . To address the latter, we reported that determining single cohort-level significance by combining gene set signal (e.g. pathways) from S3 analyses can be more accurate than conventional DEG analyses (e.g. GLMs) followed by gene set enrichment analysis (GSEA)  can yield differentially expressed genes (DEGs) between two small samples. We and others have applied the power of the isogenic framework through the comparison of two sample transcriptomes from one subject in single-subject studies - the Inter-N-of-1 method outperforms traditional methods, which are based on generalized linear modeling (GLM) followed by common gene set enrichment or overlap analysis. We then apply this novel method to the effect sizes of two different single-subject analyses to illustrate the flexibility and utility of the proposed method for a variety of inputs.To address the methodological gap, we therefore hypothesized that single-subject transcriptomic studies of gene sets increase the transcriptomic signal-to-noise ratio within subject and lead to an improved signal between small patient cohorts, as small as 3vs3 subjects per group. While technically different from the analysis of the standard two factor interactions in conventional cohort statistics, the proposed framework is conceptually related to a statistical interaction in that a within-single-subject analysis (subject-specific transcriptome dynamics) is followed by within-group agreement for characterizing Factor 1  and between group comparisons . The strategy improves the statistical power by: (i) reducing the total features tested (gene set-level rather than transcript-level), (ii) relaxing the requirement of within-cohort uniformity at the transcript level as the coordination is conducted at the gene set-level and (iii) reducing confounding factors through the paired sample design of SInter-N-of-1). To motivate the development of transcriptome analytics between very small human samples, by nature heterogenicity, we first demonstrate the limitation of a Generalized Linear Model to DEGs between 23 TP53 and 19 PIK3CA breast cancer samples. Next, we describe two new methods Inter-N-of-1 (MixEnrich) and Inter-N-of-1 (NOISeq) that work by combining single subject study results using contingency tables to obtain cohort-level estimates for enrichment of GO terms which are then contrasted to discern differences in enrichment between the two cohorts. We then compare these two methods to a Generalized Linear Model (implemented in LIMMA) (i) in simulation studies with parameters estimated from empirical analyses of real datasets and (ii) in a proof-of-concept study of TCGA breast cancer cohorts. Also, the evaluation of the proposed new methods is conservative as it is conducted against a reference standard built with a distinct Generalized Linear Model (edgeR) using all samples.ene sets from Gene Ontology (We obtained 5179 gHuman breast cancer cohort consists of RNA-seq expression profiles samples of 224 paired breast cancer (BC) tumor and unaffected breast tissue normal (Factor 1) from the same subjects (n = 112) from The Cancer Genome Atlas (TCGA) Breast Invasive Carcinoma data collection (proof-of-concept application of the proposed methods (TP53 (n = 23) or PIK3CA (n = 19) mutations Factor 2) in expression patterns (Data access and preparation: (A) within each sample pair of a single-subject study, (i) we removed all transcripts with mean expression ≤5 counts, (ii) found the union of all genes remaining amongst all pairs, (iii) excluded all genes not included in the union of these two steps  and (iv) added ‘1’ to expression counts to eliminate ‘zeros’. (B) For GLM analyses, we eliminated all the transcripts with 0 counts for each subject and calculated each transcript’s coefficient of variation (CV). We retained the top 70% of transcripts ranked by CV .The  methods  and 4 pepatterns , (ii) caSimulated datasets production is described in Section 2.4.MCF7 breast cancer cells, for estimating simulation parameters, consist of 7 estrogen-stimulated and 7 unstimulated cells sample replicates  for each gene across the two paired transcriptomes being studied and uses a probabilistic Gaussian mixture to assign a posterior probability that the gene is differentially expressed between tumor and normal conditions. Within the simulation, prioritized transcripts were defined as those with a posterior probability of being differentially expressed higher than 0.99. Within the TCGA breast cancer cohort, said definition included having both a posterior probability of being differentially expressed higher than 0.99 and an absolute-valued log2FC higher than log2(1.2), which was determined as optimal for downstream GO terms enrichment in this dataset. Genes were assigned to gene sets using the Gene Ontology (3-OR) as the effect size.Ontology  with a F2FC| and the | Δ(gene expression) | between conditions. The estimated noise distribution provides the probability of a DEG. For these applications of NOISeq with no replicates, the ‘pnr’ and ‘v’ parameters were set to 0.0002 and 0.00002 to prevent the method from producing any errors related to setting the size of the inherent multinomial distributions to an integer too large for R to handle. The criteria for identifying genes as differentially expressed for NOISeq were the same as those used for N-of-1-MixEnrich. As shown in Inter-N-of-1. This process generated two different applications of Inter-N-of-1, N-of-1-MixEnrich and NOISeq, to conduct the single-subject analyses preceding the cohort comparison.We also applied NOISeq-sim to each of the tumor-normal pairs  for a specific gene set We average the P-value for gene set gs isP-values to 1.0 if both cohort means of the log odds ratios are negative. This step ensures interpretable results since impoverished GO terms with significantly fewer-than-expected DEGs are not well understood from a biological context.When the null hypothesis g method  is then :Generalized Linear Model (GLM) Designs For the cohort analyses, we applied a generalized linear model as implemented in limma  method (We used the three different designs described in ) method .edgeR generalized linear model followed by gene set enrichment:Reference standard construction within TCGA human breast cancer cohort data using  We chose to construct reference standards using all samples of the human breast cancer cohort to estimate accuracies of analyses of smaller sample size. After pre-processing for cohort analyses, we applied generalized linear models as implemented in the R software package edgeR . For each of the three methods, FDR < 5% adjustment of the P-values was done with respect to all 5179 GO terms tested.limma. The performance of these transcript-level applications of limma was assessed and illustrated in For random subsamples of size Accuracy measures within TCGA human breast cancer cohorts: For each method, we calculated the precision and recall using the following functions. When a method produced no positive predictions for the gene sets, we assigned values of zero to the precision and recall of the given method. Otherwise, we calculated the precision and recall using Powers' calculations with adjustments of adding 0.5 to numerators and 1.0 to denominators to avoid divisions by zero .2 Fold Changes (log2FC) and then we set the gene’s expression as the product of the gene’s paired normal expression and 2 raised to the exponent of the log2FC value. To generate the expression value for an altered transcript in a tumor sample, we randomly sampled a log2FC from a gamma distribution with parameters described in 2FC value. We generated only positive log2FCs for the DEGs to improve the GLM's ability to detect them as differentially expressed cross subjects. We specified a gamma distribution for these positive log2FCs since all the absolute-valued log2FC distributions we examined possessed significant right-skew.For each sampled normal breast tissue sample, we generated transcript expression for a paired breast cancer sample of that subject rather than sampling the corresponding breast cancer sample from the TCGA data. To produce a paired tumor expression value for a non-differentially expressed gene, we first followed the steps outlined in limma assumes the presence of coordination of gene expression across subjects. Thus, we varied the expected proportion of subjects with shared DEGs within cohort A  using quartiles estimated from real datasets along with the sizes of the two cohorts  while holding all other parameters constant. These two varying parameters make up the core of the simulation since these determine how GLM+EGS performs relative to Inter-N-of-1, while all other parameters provide a background for operation. We generated 30 datasets for each parameter combination leading to a total of 450 datasets for our downstream simulations.We chose to evaluate the methods using the four GO terms described in Data preprocessing within simulation: (A) For the GLM analyses, we preprocessed the simulated data by removing all genes with mean expression values less than 30 across all the simulated transcripts and subsequently added 1 to each of the expression counts. (B) For the single-subject analyses, we applied a three-stage pre-processing method in which we (i) removed all the transcripts with mean expression less than 30 within each sample-pair and (ii) found the union across all pairs of genes remaining and eliminated any genes not contained within. (iii) The remaining genes for the single-subject analyses then had 1 added to their expression counts to eliminate any remaining zeroes.Application of methods to simulated data: The GLM+EGS and the two versions of the Inter-N-of-1 method were applied to each of the generated datasets as described previously. The Benjamini–Hochberg False Discovery Rate (FDR)   adjustmeAccuracy measures within the simulation: To estimate the overall performance of each method within the simulation, we calculated the number of true positives, true negatives, false positives and false negatives occurring within the 2 enriched and 2 control GO terms across all 30 resampling of each combination of parameters. When any of the methods made no positive predictions for the gene sets, we artificially assigned values of 0 to the precision and recall of the given method. Otherwise, we calculated the precision and recall through the use of their traditional formulae  did not perform as well. Both cohort size and the expected proportion of subjects within groups with coordinated DEGs affect the performance of the GLM+EGS method. Increasing either of these parameters significantly improves the performance of the GLM+EGS method, with the single exception of the 2vs2 cohort size where GLM+EGS produces 0 precision and recall for all specifications of the proportion of subjects within group with coordinated DEGs. At the anti-conservative levels for these parameters, the GLM+EGS method matches the performance of the two versions of the Inter-N-of-1 method. However, decreasing either parameter quickly leads the GLM+EGS method to underperform. For cohort sizes of 10vs10 and lower, the GLM+EGS method fails to match the performance of the two versions of the Inter-N-of-1 method and so supports the superiority of Inter-N-of-1 in such small sample sizes for breast cancer-like data.The simulations indicates that the proposed 3  of human breast cancer cohorts  and diversity of environmental factors . Yet, even in isogenic conditions, two studies have recommended at least six biological replicates for applying generalized linear models . Examini cohorts  exemplif3 have been shown to be effective in viral response  The simulation does present some inconsistencies with observations made within the TCGA human breast cancer subsets. Methods within the simulation attain performances which appear to be highly inflated compared to observations made within the small human cohorts. Part of this points to the need to alter simulation parameters to provide more realistic conditions. Part of these discrepancies can probably also be explained by the fact that the breast cancer analyses used a reference standard that favored GLM+EGS over Inter-N-of-1 methods by design. (iii) We explored only one type of difference within gene set response between cohorts in the simulations: a cohort responsive versus unresponsive. We are thus undertaking the complementary analysis to compare the more general paradigm of gene sets more responsive in one cohort than in the other. (iv) Although the developed methods allow for a more accurate testing of interactions in datasets with small sample sizes, the importance of balancing confounders between the two cohorts cannot be overstated. The small samples used within these analyses prevent randomization from balancing key covariates and confounders between cohorts. Future studies could model unbalanced covariates through data or knowledge fusion with external datasets. (v) Transcript independence assumptions in the calculation of the single-subject odds ratio and its variance (Inter-N-of-1 methods) may be transgressed. However, many such assumptions are routinely overlooked in related analyses, such as BH-FDR  This study focuses on parameters related to cancers, where substantial differences exist between paired normal and cancer tissues. While Sresponse  or respos BH-FDR  with sims BH-FDR . Other mInter-N-of-1 evaluate different phenomena. The GLM+EGS method discovers GO terms enriched for transcripts and requires the coordination of signals at the transcript-level across subjects belonging to similar classes before the enrichment. The Inter-N-of-1, on the other hand, assesses whether the proportion of responsive transcripts within a given GO term measured in each subject significantly differs across cohorts. In other words, in the Inter-N-of-1, the transcripts contribution to the gene set signal may be different between subjects, while in the GLM+EGS methods a transcript-level coordination is required. The Inter-N-of-1 favors clinical applications where gene set mechanisms are causal to the disease. Cancer is one such condition where numerous genetic and transcriptomic root causes may differ between subjects and yet converge to comparable cellular and clinical phenotypes.Another consideration concerns the fact that GLM+EGS and 3-anchored Inter-N-of-1-methods demonstrate the utility of within-subject paired sample designs for better controlling the heterogenicity between subjects in a manner reminiscent of experimental isogenic models (e.g. cell lines or mice models). These results motivate further studies of new experimental designs, where paired within-subject samples allow analyses of datasets previously considered too small. The new design not only presents opportunities in terms of performance within small cohorts, but also in terms of utility. By examining the single-subject results one can directly see the degree of concordance and discordance amongst subjects and answer questions pertaining to whether specific subjects possess the overall observed signal. Thus, the Inter-N-of-1 presented here represents not just a new method that performs better within small sample sizes, but also an example for how to simultaneously conduct analyses on patient variability within and across cohorts. In addition, precision therapies designed for increasingly sub-stratified common disorders can benefit from the proposed methods. The strategies and methods presented here open a new frontier that may greatly enrich our understanding of the genetic foundations of rare diseases.In conclusion, the proposed S


Plasmodium falciparum, confounds the accurate detection of specific transcriptional differences, e.g. as provoked by the development of drug resistance. In lab-based studies, P. falciparum cultures are synchronized to remove this confounding factor, but the rapid detection of emerging resistance to artemisinin therapies requires rapid analysis of transcriptomes extracted directly from clinical samples. Here we propose the use of cyclical regression covariates (CRC) to eliminate the major confounding effect of developmentally driven transcriptional changes in clinical samples. We show that elimination of this confounding factor reduces both Type I and Type II errors and demonstrate the effectiveness of this approach using a published dataset of 1043 transcriptomes extracted directly from patient blood samples with different patient clearance times after treatment with artemisinin.The cyclical nature of gene expression in the intraerythrocytic development cycle (IDC) of the malaria parasite, both false positives and false negatives with the largest effect on datasets from samples with greater variance in developmental stage.We apply this method to two publicly available datasets and demonstrate its ability to reduce the confounding of differences in transcript levels due to misaligned intraerythrocytic development time. Adjusting the clinical 1043 transcriptomes dataset with CRC results in detection of fewer functional categories than previously reported from the same data set adjusted using other methods. We also detect mostly the same functional categories, but observe fewer genes within these categories. Finally, the CRC method identifies genes in a functional category that was absent from the results when the dataset was adjusted using other methods. Analysis of differential gene expression in the clinical data samples that vary broadly for developmental stage resulted in the detection of far fewer transcripts in fewer functional categories while, at the same time, identifying genes in two functional categories not present in the unadjusted data analysis. These differences are consistent with the expectation that CRC reduces in vitro experiments.Cyclical regression covariates have immediate application to parasite transcriptome sequencing directly from clinical blood samples and to cost-constrained The online version contains supplementary material available at 10.1186/s12864-021-08281-y. Plasmodium falciparum causes the most lethal form of malaria, a vector-borne disease that killed an estimated 405,000 people in 2018, 272,000 of them children under the age of five [The parasite (ex vivo) samples and therein lies an important problem. Unless all transcriptomes are precisely aligned at identical stages of development, changes in gene expression due to drug response are significantly confounded with gene expression due to developmental stage.Most antimalarial drugs, including artemisinin-based combination therapies (ACT), target the parasite in the intraerythrocytic development cycle (IDC), when the parasite invades red blood cells . HoweverP. falciparum transcripts is expressed in a single sinusoidal pattern smoothly extending in a continuous cascade across a morphological progression through three asexual stages: ring, trophozoite, and schizont [Plasmodium species and another apicomplexan parasite, Toxoplasma gondii [P. falciparum development through the three asexual stages of IDC [The vast majority of schizont –15. The schizont . Importaschizont . This paa gondii . Accurats of IDC .P. falciparum cultures in vitro and infections in vivo in a single individual progress in a largely synchronous manner, the determinants of synchrony are not yet fully understood, influenced by both external factors and an intrinsic circadian-like regulatory mechanism [in vitro studies, it is possible to impose synchrony experimentally, albeit imperfectly. The point at which half of an experimentally well-synchronized culture has converted from mature schizonts to new rings, the first stage of progression through the IDC, is the 0 h post-invasion (hpi) benchmark [ex vivo studies, the parasites taken straight from patients are not laboratory cultured and thus these samples cannot be experimentally synchronized and aligned relative to each other. Simple morphological examination of samples based on microscopy is not accurate enough to ensure that ex vivo transcriptomes are precisely aligned [ex vivo sample  depends heavily on minimizing the confounding effect using an alignment method that can account for the cyclical pattern of stage progression.While single echanism . For in enchmark –21. Dete aligned . The preex vivo samples became feasible [ex vivo blood samples taken from patients for whom drug clearance half-lives, the standard clinical measure of artemisinin resistance, were also available. While P. falciparum exhibits synchronous behavior within a single person, blood samples taken from different people will necessarily be at some degree of different IDC developmental stages. In a study of 1043 ex vivo whole transcriptome samples designed to detect the mechanisms for artemisinin resistance, Mok et al. relied on k-means clustering as a transcriptional staging method, identifying three subgroups of samples based on their developmental profiles [The emergence of resistance to artemisinin therapies, the last line of defense in geographic regions where multiple drug resistance has arisen, occurred during the time when rapid whole transcriptome sequencing of parasites in feasible , 24. Thiprofiles . The autIDC progression differences within these clusters could still lead to misattribution of stage-based transcriptional differences to the variable under study . Another study applied a novel correction method to the 2015 Mok et al. data by removing the effect of a linear and polynomial covariate of IDC time point before differential expression analysis . BecauseP. falciparum parasites. We demonstrate that small differences in developmental stage can generate both false positive and false negative associations between transcript levels and the clinical phenotype for artemisinin resistance and show how these errors can be significantly reduced by using CRC to align developmental progression in two different strains. We then apply this method as a linear model correction with cyclical regression covariates to the Mok et al. set of ex vivo P. falciparum transcriptional samples previously assessed by transcriptional staging against Dd2 and found fewer differences in the number of transcripts identified as differentially expressed in clinically artemisinin resistant parasites[Cyclical regression covariates have been applied to a range of biological data with cyclical covariance, including diarrheal severity, transmission of malaria, and seasonal changes in human gene expression –32. Hereparasites. The genP. falciparum strains with identical gene expression profiles. Comparison of a transcriptional sample at the same point in developmental progression reveals no differential expression  in the eal data , 33. TheP. falciparum dataset containing transcriptional profiles of parasite strain PB58 collected at three time points  with three biological replicates for each time point [We tested the effect of cyclical regression covariates on a published me point . BecauseWe next generated a differential expression analysis using a linear model including the covariates (Methods). Briefly, if we consider IDC progression as a circle, we can calculate any sample’s location on that circle using its IDC time period and include those coordinates as covariates in a linear model. After accounting for stage progression using cyclical regression covariates, our analysis finds no differentially expressed genes. Table  The methex vivo samples could reveal the genetic basis for known drug resistance and reveal new resistance mechanisms; however, the transcriptomes of parasites from these clinical samples will necessarily represent a range of different stages of progression through the IDC. A confounding factor, in this case progression through the IDC, is a variable that influences the values of another variable, in this case response to artemisinin. If the approach used to identify Group A has resulted in a relatively accurate alignment of developmental stage across samples   Fig. . The comet) Fig. C. The coet) Fig. . The A+BP. falciparum developmental stages improves the accuracy of detection of transcriptional changes in the parasite due to specific perturbations. The challenge of accurately aligning stage can be partly assuaged when working in cultured (in vitro) parasites using synchronization methods and careful monitoring of stage composition, however these steps are time and labor consuming and some stage variation will remain. For studies using clinically derived (ex vivo) blood samples it is not possible to experimentally control for the stage composition. Studies that aspire to efficiently identify emerging drug resistance mechanisms using transcript readouts will benefit from a method that can specifically correct for diversity in developmental stages among clinical samples. A specific (known) confounding factor cannot be removed post hoc by simply changing the False Discovery Rate to zero or lowering the value used to decide the cutoff for significance. Such a strategy is based on the fundamental assumption that the investigator has already accounted for serious confounding in the experimental design or has validated a method of correction prior to analysis. Here we demonstrated how misalignment of IDC between samples generates both Type I and Type II errors. We further demonstrated the strong impact that even a small degree of misalignment has on the detection of differential gene expression associated with drug resistance.Precise alignment of We expected to see an impact of CRC on our reanalysis of the Mok et al. 2015 data because of the ideal opportunity it provided to directly assess our method on samples where stage could not be experimentally controlled. While we detected small differences even in the comparatively well-staged Group A, we observed much larger differences between CRC-corrected and CRC-uncorrected in Group B. Importantly, the Gene Ontology enrichments of the differentially expressed genes we detected in Group B contain many terms associated with cellular metabolism, a finding consistent with the working theory that Kelch13 propeller mutations influence rates of endocytosis . The lacin vitro time series data, investigations of mechanisms controlling the cascade, and the regulation of transcription as pertains to artemisinin resistance and the connection of cell metabolism to transcriptional signatures of stage progression at high resolution. With the exception of single cell RNAseq studies, any statement about a P. falciparum sample’s location in the IDC is, in reality, an estimate of the midpoint of the IDC progression of all parasites in an ex vivo sample. The general assumption is that in vivo, within a single human host, parasites are highly synchronized. This assumption rests on years of clinical data that show that the recurrent fever occurs when millions of parasites simultaneously burst out of circulating red cells. Novel resistance phenotypes may involve a degree of asynchrony with the result that some parasites escape drug treatment targeted to the early ring stage. Reports of patients with two parasite broods shifted by 24 h, instead of one brood synchronized to a 48 h cycle indicate that the parasite is capable of altering the period of the cycle [Most transcribed genes are strongly influenced by developmental stage. Their expression across the cycle takes the form of a cascade that cannot be accurately parsed using visual assignment of morphological stages , 16, 33.he cycle . Differehe cycle . Predicthe cycle . Our methe cycle . The strof P. falciparum by leveraging the unusual periodicity of transcription due to developmental progression through the IDC. Cyclical regression covariates have immediate application to studies where in vitro synchronization of all samples to the same developmental time point is not feasible, primarily parasite transcriptome sequencing direct from clinical blood samples, a widely used approach to frontline detection of emerging drug resistance.Here we have demonstrated the use of cyclical regression covariates to reduce the major confounding factor in studies focused on the detection of expression difference between strains P. falciparum time course data, we obtained multiple time point transcription data from PlasmoDB.org for two strains, 3D7 and HB3. [In order to demonstrate the effect of shifts in IDC timing on gene expression, two model data sets were used. The model datasets (Genes A and B) are sinusoidal curves with a period of 48 h. To demonstrate the effect of shifts in IDC timing in real and HB3. , 33 The in vitro 3D7 IDC) were determined [stagingByTranscription in the R package PFExpTools.For determination of the actual point in IDC progression for a given sample, Pearson Correlation Coefficients (PCC) between each sample’s whole transcriptome profile and the profile of each time point in the 3D7 reference IDC time course  was used in the following formula:CRCgeneration in the R package PFExpTools.The covariates were generated using the function lm in R; for the uncorrected analysis our model was Timepoint ~ Gene Expression for each gene, and for the corrected analysis the model was Timepoint ~ Gene Expression + xcovariate + ycovariate. Covariates were generated as described previously. Genes were considered differentially expressed if the p-value of fit was < 0.01 and FDR < 0.05.For testing the initial proof of concept, we looked for a dataset containing multiple replicated time points from a single parasite strain; this allowed us to associate any observed differences between time points as progression driven. The gene expression data used to demonstrate the efficacy of the cyclical regression covariate method were obtained from the Gene Expression Omnibus; the data for PB58 are listed under Accession Number GSE119514. We used ex vivo dataset, 1043 clinical samples taken at various points in IDC progression, to examine the effect of cyclical regression covariates on the results of the gene expression analysis. We retrieved the normalized uncurated Mok data from the Gene Expression Omnibus, Accession Number GSE59097, and updated the curation to the current transcriptome assembly. We aligned the sequence for each probe with Version 46 of the P. falciparum transcriptome. Probes that had an alignment with a bitscore > 130 and no secondary alignment scored over 60 were named to the gene they aligned with [We used the Mok et al. 2015 ned with . In the CRCgeneration in the R package PFExpTools; the IDC progression value provided in the associated metadata was used. We compared the gene expression analysis uncorrected and then corrected using CRC for groups A and B separately.Data analysis and figure generation were performed in R Version 3.3.1. Venn diagrams were created using the R package VennDiagram [Gene expression analysis for two groups identified by Mok et al. 2015 (Group A+B) was performed as follows. The relationship of the expression of each gene with patient clearance half-life was evaluated using a simple linear regression model in R. For each gene in the corrected dataset, the simple linear model included the paired cyclical regression coefficients. Each gene in the uncorrected dataset was evaluated using a simple linear regression model without the paired cyclical regression coefficients. For the uncorrected analysis our model was Phenotype ~ Gene Expression for each gene, and for the corrected analysis the model was Phenotype ~ Gene Expression + xcovariate + ycovariate. The cutoff value of p < 0.05 and FDR < 0.25 were the criteria for a decision of differential gene expression for each gene. The covariates were generated using the function nDiagram .https://plasmodb.org/plasmo/app/). GO Slim provides a less detailed annotation by grouping features into broader categories of biological functions. This is useful when the investigators wish to focus on a given broad biological function or, as is the case with this investigation, to determine if a broad category of biological functions was missed if the CRC correction is not used. Additional files contain the Go Slim annotations (Additional File We used the GO and GO Slim applications implemented in PlasmoDB beta release 50 (PFExpTools is freely available for download and use at https://github.com/foster-gabe/PFExpTools, including all documentation and source code under the GPL-3 license. This manuscript was prepared using Version 1.0 of PFExpTools.The HB3 and 3D7 time course data is available for download at PlasmoDB.org. The PB58 data is available at the Gene Expression Omnibus under Accession Number GSE119514. The Mok et al. dataset is available at the Gene Expression Omnibus under Accession Number GSE59097. The updated curation of the Mok et al., 2015 dataset is provided as Additional File Additional file 1. The result of our re-curation of the Mok et al. data originally sourced at the Gene Expression Omnibus at Accession Number GSE59097. The dataset was curated as described in Methods using the PFExpTools R package. This file is a matrix of expression values, with the features in rows and samples in columns in a comma delimited file format.Additional file 2. Provides quality control comparisons between our curation and analysis of the Mok et al. data and the originally published results.Additional file 3. Contains the Gene Ontology Go Slim category results for all the comparisons.Additional file 4. Contains the complete Geno Ontology category results for all the comparisons.