The analysis of mRNA transcript abundance with RNA-Seq is a central tool in molecular biology research, but often analyses fail to account for the uncertainty in these estimates, which can be significant, especially when trying to disentangle isoforms or duplicated genes. Preserving uncertainty necessitates a full probabilistic model of the all the sequencing reads, which quickly becomes intractable, as experiments can consist of billions of reads. To overcome these limitations, we propose a new method of approximating the likelihood function of a sparse mixture model, using a technique we call the Pólya tree transformation. We demonstrate that substituting this approximation for the real thing achieves most of the benefits with a fraction of the computational costs, leading to more accurate detection of differential transcript expression and transcript coexpression. The past decade has seen RNA-Seq become a central tool in molecular biology research. Along the way there have been numerous methods developed to analyze this data. We will propose an entirely new methodology based on likelihood approximation, which enables inference on full probabilistic models that would otherwise quickly grow intractable. In preparation, we will first give a brief overview of notable approaches to RNA-Seq transcript and gene quantification, to give some sense of where this new method fits in.de novo assembly . As Mortazavi et al.  notes, requivalence classes, which are sets of transcripts that do not share any reads. Similarly, Yanagi . More explicitly, given a set r of m reads, we define a probability function pj over possible reads, for every transcript j ∈ {1, …, n}. The likelihood for a relative expression vector x ∈ Δn − 1 (here Δn − 1 is the open unit (n − 1)-simplex, i.e. the set of all vectors of length n, with entries in  summing to 1) isThe common probabilistic approach to the transcript quantification problem is to treat transcripts as inducing distinct probability distributions over reads (or read pairs). Starting with the assumption that we have a known transcriptome consisting of pj. RNA-Seq protocols involve many steps like fragmentation, reverse transcription, amplification, and fragment size-selection which each influence the observed distribution of reads. Assuming reads to be uniformly distributed across a transcript, subject to some fragment length distribution, is the most straightforward model, but some success has been had in building more accurate models that capture positional and sequence-specific biases (see e.g. (There are many issues surrounding the question of how best to model each transcript read distribution see e.g. ). Thoughsee e.g. , we willk samples consisting of sets of reads r(1), …, rk), then consider the posterior distribution P(θ|r(1), …, rk)∝P(r(1), …, rk)(|θ)P(θ).Relative transcript expression for an individual RNA-Seq sample is not typically interesting on its own. RNA-Seq experiments are nearly always concerned with detecting transcriptional changes between groups of samples. From a Bayesian perspective, we would like to build a model of transcriptional changes among ri), …, xk). This translates to hundreds of megabytes to several gigabytes per sample. Estimating the posterior for moderately large experiments requires either a great deal of memory or cycling read data in and out of memory, as is done in stochastic gradient methods.This is all very straightforward but presents some practical problems. To evaluate the likelihood functions, some information about each unique read must be stored  for every transcript j are treated as equivalent and combined in Equation . For a large experiment, all of the reads r will not fit in memory, but this problem is amenable to stochastic variational inference, or SVI (xi)( for each sample i), before updating ‘global’ latent parameters (θ).The obvious alternative for tractable inference is simply to use variational inference on the posterior we are actually interested in. That is, if we have a model of, say, differential expression, with parameters θ, we want to approximate the intractable posterior , or SVI . In SVI,Because SVI algorithms must cycle many gigabytes of sequencing data in and out of memory to repeatedly compute stochastic gradients, they are likely much less efficient than likelihood approximation. Two additional considerations further increase the latter’s desirability.First is reusability. Large RNA-Seq experiments can be rich with insight, and lend themselves to multiple analyses. Differential expression, differential splicing, clustering, and dimensionality reduction are separate tasks that might be carried out on the same data, each with its own model, each representing a separate inference task. For these tasks, likelihood approximations must be made only once, after which they can be reused over an over. Amortized over every iteration of every analysis typically run on a dataset, likelihood approximation is far more efficient than other approaches to tractable inference.Second, likelihood approximation can obviate some of the cumbersome data transfer and storage issues with RNA-Seq. High throughput sequencing produces huge datasets, which must be stored and transferred to collaborators, which can mean waiting on long downloads or exchanging hard drives. Our approach to approximated likelihood, on the other hand, summarizes all expression information for a sample using only a few megabytes per sample. The 1443 brain samples produced by the GTEx project , are redp, and a family of distributions q parameterized by ϕ, we fit q to p, given some data y, by choosing ϕ to minimize the Kullback-Leibler (KL) divergence,Variational inference is usually presented as a means of estimating an otherwise intractable posterior distribution. Given a distribution function q does not depend directly on y, so that once ϕ is optimized, the approximate probability can be evaluated without the data, only retaining the typically much smaller parameter vector ϕ.A key feature of this method of inference is that q for each sample’s likelihood function P(r|x), we can capture the likelihood with some fidelity without having to keep the RNA-Seq reads in memory. This would allow us to build a very large model, encompassing hundreds or thousands of samples, that can be run on laptops or meager servers.This suggests a solution to the issue of building large joint models. Instead of using variational inference to approximate a posterior distribution, we can use it to separately approximate the likelihood of each sample. By substituting an approximation x but over reads r, so the KL divergence is not well-defined here. But in models making use of the likelihood function, multiplicative constants are generally irrelevant, so our approximation only has to be proportional to the likelihood. To bring the machinery of variational inference to bear, instead of approximating likelihood directly, we approximate a normalized likelihood function, mathematically equivalent to a posterior distribution under a uniform prior. We will denote with Of course, the likelihood is not a distribution over expression vectors entclass1pt{minimauity see , but is uity see . Minimizuity see .q must be defined over a simplex Δn − 1. Without going into the mathematical details .T: n − 1 → Δn − 1), with an efficiently computable Jacobian determinant.The Dirichlet distribution tends to be the default choice distribution over the unit simplex, but besides being insufficiently expressive for our goals , then the distribution induced by alr−1(y) is sometimes referred to as a multivariate logit-normal distribution.The alr is typically defined, as it is here, with the divisor multiplicative log-ratio transformation (The second useful transformation is the ormation (5)\documn pieces, and x1, …, xn give the size of each piece, in proportion to the whole, then mlr(x) gives the log-ratio between each piece and the remaining length of the stick, if the stick were broken one piece at a time. We will return to this stick-breaking metaphor shortly.This transformation can be best understood using a sequential stick-breaking metaphor. If a stick is broken into The probabilistic programming language Stan  implemenisometric log-ratio (ilr) transformation  represent the proportion of the remaining stick to break off on the ith break. Then x ∈ Δn − 1 can be produced from y ∈ n − 1 withi − 1 breaks.Let nduction , or by nyi is uniformly distributed in  using the motivation of distributing a pile of gold dust to a countably infinite sequence of beggars. The possibility of yi being drawn independently from arbitrary distributions is also briefly considered. In recent literature, stick breaking occurs most commonly in descriptions of the Dirichlet process, which can be formulated as an infinite stick breaking procedure in which the breaks yi are Beta distributed variables  = Beta distributed. The resulting stick sizes are then used to weight draws from a base distribution. When n → ∞ in Equations  distributed, for some θ, the resulting distribution family is commonly denoted GEM(θ), after Griffiths, Engen, and McCloskey (Stick-breaking metaphors like the one used to describe the mlr transformation have a long history. An early example  exploresariables . Random quations  and 7),,yi is uncCloskey . This nocCloskey .i and the n − i categories that follow it in the process, so a particular ordering may fit the data poorly if no such boundary naturally exists. Zhang et al. .The second key insight that is sometimes neglected is that there are other ways of breaking a stick. Rather than breaking pieces off the stick and setting them aside, we might keep and recursively break both of the resulting pieces. This can be thought of as n! permutations of the stick breaking process, but also the n − 1)st Catalan number) possible full binary trees with n leaves, resulting in a family of With these two insights, we have a space of possible stick-breaking transformations along the lines of Aitchison’s mlr, but considering not just the Pólya tree transformations. One special case of distribution families induced by Pólya tree transformations is the Hyper-Dirichlet Type 1 distribution described by Dennis  at each step. Transcripts, or sets of transcripts, that share a large proportion of their compatible reads have a higher Jaccard index, and thus their common ancestor is placed lower in the tree. In an ideal scenario, this constructs a tree that encodes a distribution family that has a similar independence structure to that of the real likelihood function.Aitchison  discusseIn cases of complete subcompositional independence, where no reads are shared between transcripts, the likelihood function in Equation  is propon − 1 → Δn − 1 transformation. Most of the compositional data analysis transforms take the form n − 1. To minimize the parameter space, each element will be considered independent .Given a transformation onto the simplex, we now need to choose the distribution that will be transformed. The Pólya tree transform has been described here as a We are limited to distributions that lend themselves to the reparameterization trick. The two distributions we will consider are the normal  distribution and the Kumaraswamy distribution , which iZ ∼ Normal, we useskew-normal distribution, though other distribution families also go by this name pplement .n − 1 nodes, n of which are leaves. To simplify notation somewhat, assume these are assigned indices so that the root node is labeled 1, internal nodes 1, …, n − 1, and leaf nodes n, …, 2n − 1. Additionally, we assume internal nodes are numbered so that no node has a smaller index that any of its ancestors, for example, according to a pre-order traversal. Intuitively, we can think of the ith node as representing the ith break in a stick-breaking process.The Pólya tree transformation is best represented as a full binary tree, with 2The tree can then be defined by functions giving an internal node’s left and right children respectively.i): index of node i’s left childleft(i): index of node i’s right childrightn − 1↦Δn − 1. Figure 4 to Δ4.Often we are operating on the unit simplex, in which case ui values are computed by traversing the tree with any top-down traversal from the root. In the stick-breaking metaphor, intermediate values can be thought of as sizes of intermediate sticks after some number of breaks are performed. The inverse transformation can be computed by traversing the tree up from its leaves, as is done in a post-order traversal.When implemented, The Pólya tree transformation has a few interesting properties, which we summarize here with proofs and further exposition left to the supplement.O(n). It is simply the product of the internal node intermediate values.First, importantly for the transformation to be tractable, the Jacobian determinant has a simple form that can be computed in time Theorem.LetandJTbe the Jacobian matrix ofT). Further, letui for i = 2, …, 2n − 1 be the intermediate values as defined in Equations (and (. Thenquations  and 9)..Theorem.The choice of base distribution induces different families of distributions on the simplex under the transformation. If we use Beta distributions as our base and parameterize them by assigning a vector of intensities α to the leaves, then choose Beta parameters to be the sums of the left and right subtree intensities, respectively, we have a class of distributions we call HierarchicalBeta. We go on to show this family is equivalent to the Dirichlet distribution, regardless of the tree topology defining the transformation.Theorem.For any Pólya tree transformationT: n − 1↦Δn − 1wheren ≥ 2and anyThis equivalency opens up the possibility of using Beta distributions as the base distribution, which gives an exact fit in cases of complete subcompositional independence, while generalizing to capture uncertainty information in ambiguous features. This relationship could also be exploited to speed up inference in models with Dirichlet priors.To demonstrate the usefulness of our approach to likelihood approximation, we undertook five analyses. In the first two subsections we show that the approximation is a good fit to the likelihood function of actual RNA-Seq datasets, and that it can be efficiently fit and evaluated. Then we focus on demonstrating that it can improve the detection of differentially expressed transcripts, using an existing simulation benchmark, and then separately using real data from the GTEx project. Finally, we show that using the approximate likelihood produces more reliable estimates of of pairwise correlation.We explored the fit of a number of possible approximations to the RNA-Seq likelihood function produced transforming standard probability distributions. We used a three bijective transformations from the compositional data analysis literature: additive log ratios , multiplicative log ratios (mlr),  and isomP-value.We first evaluated the fit of transcript marginal densities using Wilcoxon signed-rank tests. For every transcript in a set of annotations, 1000 samples were drawn from a Gibbs sampler (representing the exact likelihood function) and the same number of samples were drawn from the approximated likelihood. The signed-rank test was run for each transcript, producing a The Gibbs sampler used 8 randomly initialized chains. Each was burned-in for 2000 iterations, then 25 000 samples were generated and every 25th was saved for analysis, producing a total of 1000 samples. We found these settings sufficient for marginal distributions to have converged for a vast majority of transcripts, where convergence was measured by comparing within-chain and between-chain variance .P-values. Imperfect approximations will yield P-value distributions that are increasingly skewed towards smaller numbers. The greater the tendency towards small P-values, the worse the overall fit. Figure P-value and fit, a number of examples with low P-values are plotted in Figure If an approximation is a perfect fit, we would expect to see a uniform distribution of We see that the approximating distribution family matters a great deal. Distributions based on the Pólya tree transformation (labeled ‘PTT’) offer a dramatically better fit than the traditional compositional data analysis transforms alr and ilr, and we see that the heuristic tree construction is a sharp improvement over a random tree topology. The common sequential stick-breaking approach  is seen to be the worst approach to stick-breaking, perhaps in part because such a long chain of dependent breaks is numerically less stable and thus more difficult to fit to the target likelihood. As no attempt was made to optimize over permutations, it is also possible sequential stick breaking could be improved with the right permutation heuristic.et al.  distribution, and apply the transform, which is O(n) where n is the number of annotated transcripts. Because samples are so cheap to generate, for a sufficiently large numbers of samples, it outperforms not only MCMC approaches, but also the extremely fast bootstrap approach implemented in Kallisto , the amortized cost of sampling becomes less for Polee than Kallisto .Approximate likelihood functions can be evaluated directly, so there is not necessarily a need to sample, but sampling is useful in some applications. For example, Polee can be used as a drop-in replacement for Kallisto when using Sleuth  to call et al. . These results suggest that samples drawn in proportion to the likelihood are more informative than bootstrap samples for this task, and that most informative of all is actually including the likelihood function, or its approximation, in the model. When detecting gene-level differential expression, Polee very slightly trails Sleuth, which exceeds the performance of all the other methods. Oddly, using likelihood approximation samples with gene-level Sleuth analysis . These tests were run with 10 different random subsets to draw the aggregate FDR/recall curves in Figure The results show broadly that the full approximate likelihood model outperforms point estimates, as well as bootstrap and Gibbs sampler estimates. Estimating variance with a Gibbs sampler is the most competitive alternative, outperforming bootstrap variance estimates. This agrees closely with with the results for ‘polee/sleuth’ we saw in the simulated differential expression benchmark, where we improved the baseline performance of Sleuth by using approximate likelihood samples rather that bootstrap samples. Posterior mean estimates offer an improvement over maximum likelihood estimates, and appear to begin to catch up to the approximate likelihood approach when a large enough number of samples is used. An important caveat is that producing posterior mean point estimates either involves MCMC, or a variational inference, so in generating the estimates, there is little or no performance advantage to using posterior mean estimates instead of approximate likelihood. In the pairwise tests, bootstrap estimates perform similarly to using posterior mean point estimates, while underperforming when using all samples.The amygdala versus hippocampus test shows very poor performance for all the involved methods, as the differences that do exist between these tissues are small or inconsistent, so they are only reliably detectable with a large number of samples. Nonetheless, approximate likelihood does make better use of the limited data, showing a clear improvement.Large amounts of available sequencing data have led to an increasing emphasis on deciphering the functional relationships between genes. Co-expression networks are often a preliminary step towards inferring regulatory networks . Construet al. (Co-expression in the the GTEx data was examined by Saha et al. . To contThis filtering procedure reduces false-positives, but at the cost of potentially introducing false-negatives. A model affording a more principled accounting of estimation uncertainty would obviate the need for much of this ad hoc filtering.To explore this idea, we used a more simplistic analysis of co-expression, computing pairwise Spearman correlation matrices across all annotated transcripts, with no filtering whatsoever. To see how the choice of estimate affected the results, we did this with maximum likelihood, posterior mean, bootstrap, and approximate likelihood. To avoid division by zero, and to otherwise slightly moderate the effects of zeros, we added a pseudocount of 0.1 TPM to all estimates in the maximum likelihood and bootstrap samples. The uncertainty information provided by bootstrap and approximate likelihood was incorporated by computing the average Spearman correlation across 20 samples.As with differential expression, there are no plausible gold standard estimates to compare to, so we resorted to using consistency as a proxy for accuracy. We selected one tissue from the GTEx data, cortex, consisting of 118 samples, and computed the correlation matrices. Treating this as ground truth for each respective estimate, we recomputed the matrices using random subsamples of 12 of the 118 samples, and measured the difference between each element in the matrix. This was repeated 10 times for different random subsamples. The aggregate results are plotted in Figure Looking first at transcript expression (Figure Here we have described how a full likelihood model of RNA-Seq transcript expression can be made tractable by approximating the likelihood function. Because including the likelihood, rather than relying on point estimates, better accounts for estimation uncertainty, differential expression calls are more reliable. The method we developed to do so, the Pólya tree transformation, is a general purpose approach to approximating sparse mixture models. Though we have confined ourselves here to showing its benefits on a few analyses, there are other possible applications. Other RNA-Seq analyses, like classification, clustering, and dimensionality reduction could benefit from the same approach. It also presents an opportunity of performing isoform level analysis of single-cell RNA-Seq, accounting for the high estimation uncertainty where there are relatively few reads per cell.This rethinking of RNA-Seq analysis also poses questions about how best to construct models of transcript expression. In our purposely simplistic model of pairwise transcript correlation we made a point of doing no filtering or preprocessing. Sticking to this principle when developing a more sophisticated model of coregulation, while keeping inference tractable, is not trivial. There is also also additional work needed to scale the method to run inference on tens or hundreds of thousands of cells. Though we were able to run a regression model on the GTEx brain data consisting of 1443 samples and over 39 billion reads, this took over two days to run. Scaling the method to tens or hundreds of thousands of samples or cells will require additional engineering, but with out of core methods and faster evaluation of the approximation, it is a goal that is well within reach.Gelman  describehttps://github.com/dcjones/poleeThe method is implemented in a Julia package available from The mouse body map data used for goodness of fit tests is available at accession PRJNA375882. Accession numbers for the data shown in Table lqab046_Supplemental_FileClick here for additional data file.


P = 8.11 × 10-5). As an independent prognostic factor relative to clinical information, these two subtypes have some significant genomic differences. Remarkably, the subtype of poor prognosis had significant higher frequency of chromosome 3p deletion. Immune decomposition analysis results showed that these two MIBC subtypes had different immune components including macrophages M1, resting NK cells, regulatory T cells, plasma cells, and naïve B cells. Hallmark gene set enrichment analysis was performed to investigate the functional character difference between these two MIBC subtypes, which revealed that activated IL-6/JAK/STAT3 signaling, interferon-alpha response, reactive oxygen species pathway, and unfolded protein response were significantly enriched in upregulated genes of high-risk subtype. We constructed MIBC subtyping models based on multi-omics data and single omics data, respectively, and internal and external validation datasets showed the robustness of the prediction model as well as its ability of prognosis . Finally, through bioinformatics analysis and immunohistochemistry experiments, we found that KRT7 can be used as a biomarker reflecting MIBC risk.Muscle-invasive bladder cancer (MIBC) is the most common urinary system carcinoma associated with poor outcomes. It is necessary to develop a robust classification system for prognostic prediction of MIBC. Recently, increasing omics data at different levels of MIBC were produced, but few integration methods were used to classify MIBC that reflects the patient’s prognosis. In this study, we constructed an autoencoder based deep learning framework to integrate multi-omics data of MIBC and clustered samples into two different subgroups with significant overall survival difference ( Bladder urothelial carcinoma (BLCA) is one of the most common cancer types in human , while mIn recent years, many studies have characterized the molecular features at different omics levels and reported subclassification of bladder cancer into distinct subtypes based on unique molecular signatures –11. For Nevertheless, the previous studies investigated the molecular subtypes of bladder cancer only based on single omics level, and did not connect with the survival information during the process of defining subtypes. Thus, a subtyping method that could reflect different survival profiles is valuable for the clinical application in guiding the treatment of MIBC patients.Here, we employed a multi-omics-based utilized deep learning (DL) computational framework to stratify the MIBC patients into two subgroups concerning different risks of overall survival (OS) (https://xenabrowser.net/).The multi-omics data of TCGA-BLCA, including gene-level copy number variation (CNV) profile, mRNA and miRNA expression profile revealed by RNA-seq and miRNA-seq, and DNA methylation data profiled by Illumina Infinium HumanMethylation450 platform, were downloaded from the University of California Santa Cruz (UCSC) Xena database (https://www.ncbi.nlm.nih.gov/geo/), serving as out-group validation datasets. For these datasets, only samples with prognostic information were taken into consideration for downstream analysis.Only samples with tumor stage II/III/IV (MIBC) remained for downstream analysis. These TCGA-MIBC datasets were used in two ways: 1) All samples were used to perform subgroup stratification based on deep learning and clustering algorithm; 2) samples were randomly split by 4:1, including a training dataset to train the classification model and an in-group testing dataset to validate the prediction accuracy. Three gene microarray matrices containing 43 MIBC patients (GSE19915), 62 MIBC patients (GSE48277-GPL14951), and 73 MIBC patients (GSE48277-GPL6947) were downloaded from Gene Expression Omnibus (GEO) database (https://github.com/fchollet/keras).The autoencoder framework was chosen as the implementation of deep learning for integrating the results derived from multi-omics data. The CNV, gene expression, miRNA expression, and methylation data extracted from TCGA-MIBC dataset served as an input for the autoencoders framework. The autoencoder was a dimensionality reduction method based on an unsupervised feed-forward, non-recurrent neural network, which is implemented in python with package Keras  into the same dimension output layer y through an activation function tanh (a hidden layer between x and y). In this study, we used the four preprocessed data matrices of different level of omics data (features × samples) and stacked all features together into a merged big matrix. In total, 350,631 features were used for downstream analysis. All of the features except CNV features were scaled so that all values are within a similar distribution range. This step could be expressed as:For a given input layer, the objective of an autoencoder reconstructed the input layer ib is an intercept vector of size p and i·x = Σj Wi,j·xjW, in which jx is the value of a single feature of x. When the autoencoder framework has k layers,where where iW minimizing a specific objective function. We chose binary crossentropy as the objective function, which measures the error between the input x and the output y:To train an autoencoder, the objective is to find the different weight vectors wα and aα for both weight vector iW and node activities F1→k (x):We added two regularization penalty wα and aα were set as 0. 1 and 1 × 10-7, respectively. Finally, the autoencoder was trained by the gradient descent algorithm with 10 epochs and a batch size of 64.We set the three hidden layers in the autoencoder, which included 500, 100, and 500 nodes, respectively. The bottleneck layer of the autoencoder was adopted to generate novel characteristics from the four-level omics data. The penal values P < 0.05). Subsequently, we used these selected features to cluster the MIBC samples into groups based on the K-means clustering algorithm. The hazard ratio and the p-value derived from log-rank test were used to evaluate the prognostic differences.One hundred novel features were derived from the omics data based on the deep learning algorithm. For each of these transformed features, we performed the univariate Cox proportional-hazards regression analysis to find out the OS-related features , respectively, and MIBC samples were extracted for downstream analysis. The mutation data was converted into “maf” format and visualized by Maftools (Somatic mutation data of TCGA BLCA and copy number segment data were downloaded from UCSC Xena database (Maftools . The segMaftools . The freMaftools .http://software.broadinstitute.org/gsea/msigdb/), and gene set enrichment analysis (GSEA) was performed using the R package “clusterProfiler”  of TCGA data were detected by DESeq2 , and DEGrofiler” .To test for differentially methylated CpG sites (DMS), we use the limma package. CpG site was defined as a DMS that |log2(fold-change)| of Beta value was more than 1 and adjusted p-value was less than 0.05. DMS located genes were extracted, and over-represent enrichment analysis was performed using the R package “clusterProfiler”.All TCGA MIBC samples were randomly separated into training/testing datasets following a 4:1 split. Then, we build a supervised classification model using random forest, Naïve Bayes, k-Nearest Neighbor, and Adaboost algorithms. For the training dataset, we normalized each omics layer and calculated the p-value (Wilcox test) of each feature between these two prognostic subgroups. Then, we selected top features  that are most correlated with subgroup labels based on the p-values. Then, we conducted 10-fold cross-validation with 10-time repeat to evaluate the predictive ability of the selected features.During each repetition, different algorithms were applied (mentioned above), and receiver operating characteristic (ROC) curves were executed. The area under the curve (AUC) in all the repeats would provide us the predictive value of the classification. Once the AUC value was less than 0.7, the whole dataset would be re-split and the analysis would be re-started till the satisfying results were obtained. Finally, we select the best classification model with the highest AUC.We selected the same features of each omics data in the testing dataset and predicted the label of each sample based on the classification model. The univariate Cox proportional-hazards regression analysis was performed to test the survival risk difference between the predicted groups.For the out-group validation dataset, which only has a gene expression profile, we just use the overlapped features with the 100 mRNAs mentioned above to fit the classification model. The same tests were performed on TCGA testing dataset.Twenty-two MIBC samples were selected from Sun Yat-sen University Cancer Center, Guangzhou, China, between January 2015 and December 2015. Only samples with overall survival less than 1.5 years or over 5 years were taken into consideration in this study. IHC staining was performed using BenchMark ULTRA automatic immunostaining device according to the manufacturer’s instructions to analyze the KRT7 expression. In brief, the paraffin-embedded MIBC samples were sectioned and deparaffinized using EZ prep solution . The endogenous peroxidase activity was inhibited, and the sections were subjected to antigen retrieval in a cell-conditioning solution maintained at 95°C for 30 min. The sections with the primary antibody mouse anti-CK7  were incubated at 37°C for 1 h after adding Liquid crystal solution . A secondary antibody was then added at 37°C for 15 min, and signals were detected using the chromogen 3,3’-diaminobenzidine (DAB). The sections were counterstained with hematoxylin and then dehydrated and mounted on a coverslip. Staining proportion (0–100%) and staining strength (- to 4+) were measured for each sample, and an IHC score was calculated as follows:proS stands for the score of staining proportion  and strS stands for the score of staining strength . The IHC score was used to measure the expression level of KRT7.where Materials and methods). As a result, 100 new features were extracted from the bottleneck hidden layer, which represented the features of omics. We performed univariate Cox proportional-hazards regression analysis on these features and identified 98 features that were highly correlated with patients’ OS , which contains some important tumor suppressor genes (TSGs) including FANCD2, VHL, RPARG, XPC, TGFBR2, MLH1, SETD2, and RHOA. Interestingly, TGF-Beta receptors were significantly altered in S2 at both SNV and CNV levels. Considering that transforming growth factor (TGF)-b is a key executor of immune homeostasis and tolerance, which can inhibit the expansion and function of many components of the immune system, we next performed immune decomposition for each sample and investigated the differences in immune components between the two molecular subtypes using CIBERSORT .In order to analyze the molecular characteristics of the two molecular subtypes, we firstly compared the differences in mutation and CNA levels between the two groups. There is no significant difference between the two subtypes in terms of mutation burden . The same features were extracted from the internal validation cohort, and samples were classified into two different groups according to the prediction model. Considering the previous subtype labels of samples from internal validation set, we construct the ROC curve to evaluate the robustness of the supervised classification model  TCGA-MIBC cases as the training set and the other 81 (20%) MIBC cases as an internal validation set  and two downregulated genes (GGTLC1 and KRT7) are overlapped among these three datasets, respectively. We selected KRT7  as a candidate marker to distinguish high-risk and low-risk MIBC. As expected, the expression level of KRT7 was negatively correlated with risk-score of MIBC  and found that KRT7 protein was highly expressed in the low-grade bladder cancer cells but medially or lowly expressed in high-grade bladder cancer cells . As shown in Different levels of omics data could present diverse tumor landscape from different angles. It is required to integrate multi-omics data to describe the relations between clinical outcomes and molecular characteristics, then get a comprehensively understanding of cancer. In the present study, we construct an autoencoder-based deep learning framework to integrate CNV, gene expression, miRNA expression, as well as CpG methylation results to classify MIBC into two prognostic subtypes. The subtype S2 shows a significantly higher risk on overall survival and some specific genetic characters compared with the other subtype. We construct a robustness MIBC subtyping model depending on different omics layers and assessed the prognostic value in both internal and external validation datasets. We also detected KRT7 as a biomarker to reflect the risk of MIBC.TP53, VHL, MLH1, TGFBR2, THRB, RARB, and FHIT. Loss of one copy of chromosome 3p is one of the most frequent and early events in human cancer, found in 96% of lung tumors and 78% of lung preneoplastic lesions (NFE2L2 (the most significant gene that mutated in 16% of S2 but 4% in S1) has been reported in types of cancers. NFE2L2 has long been considered a cytoprotective transcription factor, which is essential for the defense against oxidative stress, and activation of the NFE2L2 pathway has been proposed as potential preventive strategy against carcinogenesis due to its function as a master regulator of the expression of antioxidant and detoxifying enzymes , Natural Science Basic Research Program of Shaanxi (2020JC-01), Medical Scientific Research Foundation of Guangdong Province, China (Grant No. A2019401), and National Key Research and Development program (No. 2017YFA0105900).The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


Due to the high mortality of many cancers and their related diseases, the prediction and prognosis techniques of cancers are being extensively studied to assist doctors in making diagnoses. Many machine-learning-based cancer predictors have been put forward, but many of them have failed to become widely utilised due to some crucial problems. For example, most methods require too much training data, which is not always applicable to institutes, and the complicated genetic mutual effects of cancers are generally ignored in many proposed methods. Moreover, a majority of these assist models are actually not safe to use, as they are generally built on black-box machine learners that lack references from related field knowledge. We observe that few machine-learning-based cancer predictors are capable of employing prior knowledge (PrK) to mitigate these issues. Therefore, in this paper, we propose a generalisable informed machine learning architecture named the Informed Attentive Predictor (IAP) to make PrK available to the predictor’s decision-making phases and apply it to the field of cancer prediction. Specifically, we make several implementations of the IAP and evaluate its performance on six TCGA datasets to demonstrate the effectiveness of our architecture as an assist system framework for actual clinical usage. The experimental results show a noticeable improvement in IAP models on accuracies, f1-scores and recall rates compared to their non-IAP counterparts . Cancer is a major cause of human death, as it often leads to a series of related fatal diseases that are complicated to control . As a coDue to its complexity, the clinical treatment of cancer is in high demand, the goal of which is more practical and real-life reasonable. The early detection and prediction techniques of cancers, which facilitate successive medical treatment of patients, are being extensively studied  in orderWith the improvement of device performance and accuracy, DNA sequencing techniques are now able to collect genomic profiles of tumour samples, such as Gene Expression (GE), miRNA Expression (ME) and DNA Methylation (DM), without extensive physical or mental exertion. While it is demanding to excavate possible patterns of tumour-causing gene expression signatures from clinical data for even well-experienced physicians, many data-driven approaches can be used to assist doctors in the early detection of cancers. Because of their ability to discover feature patterns from large-scale data, machine learning algorithms such as Support Vector Machines (SVMs), Bayesian Networks (BNs) and Decision Trees (DTs) have been well employed for modelling the development and treatment of cancer conditions ,7,8. TheMany machine learning methods use gene expression signatures as inputs, which indicates a distinctive, species- or tissue-specific expression pattern of genes in one cell . While iThere are a series of deep learning models that achieve elevated performance on certain cancer prediction tasks ,13,14. HRecently, the notion of IML has gained a growing interest, which is especially considered a potential solution for dealing with insufficient training data . Unlike According to our knowledge, few methods in relevant fields are already able to integrate the novel concept of IML into the cancer prediction task, and therefore, few approaches are capable of leveraging PrK to assist cancer diagnosis. Inspired by this, we designed and proposed a conceptual architecture, the Informed Attentive Predictor (IAP), which aims at the cancer prediction task, as well as its extended field application, namely assisted diagnosis of cancers. Moreover, the IAP embodies the idea of informed machine learning (IML) and is designed to be generalisable, i.e., it describes a method of knowledge integration where the human experience straightforwardly takes part in the decision-making process, which is classifier-independent. Several IAP implementations were constructed and evaluated to demonstrate their effectiveness on either full- or reduced-size datasets.We propose an end-to-end IML architecture (IAP) for the assisted diagnosis of cancers, which describes a method through which the underlying classifier can fuse human experience into its whole process, even if the training data have a limited size. Because the PrK integration module is dependent on the underlying classifier, the IAP is totally generalisable. To the best of our knowledge, the IAP is the first attempt to incorporate IML with cancer prediction.Based on some previously proposed classifiers targeted at the same task, we developed and evaluated several IAP implementations. Specifically, we compared the performance between each IAP implementation and its basic counterpart. Experiments show that by applying the IAP architecture, model performance is improved by a noticeable margin on either full- or reduced-size datasets.The comparative analyses amongst multiple experiments show that our proposed architecture and models achieve higher accuracies, F1-scores and recall on the tested TCGA datasets. Thus, the validity and performance of our methodologies are verified.To summarise, our main contributions are threefold:In this section, we exemplify related work targeting the cancer prediction task. Specifically, we focus on methods with machine learning and/or deep learning techniques, as well as some recent applications of IML in relevant fields.Machine Learning (ML) is a branch of Artificial Intelligence (AI) research that utilises a variety of statistical and probabilistic techniques to automatically learn patterns of features from past examples (train) and then use the trained model to classify new data .Traditional Artificial Neural Networks (ANNs) , SupportWith the development of computational devices, Deep Learning (DL) methods started to play an important role in the construction of classifiers. A DL model has a branch of multiple processing layers, through which representations of data can be learnt in a way that contains different levels of abstraction .A majority of DL methods utilise the Convolution Neural Network (CNN), a feed-forward network that allows for effective feature extraction by applying convolution on embeddings. For instance, in , the autHowever,  mentioneAs research into cancers has progressed, multiple large-scale datasets have been made available. However, the heterogeneity of these datasets makes it difficult for researchers to construct cancer predictors that learn from multiple data sources. In addition, DL models are becoming much less explainable, as multiple layers have to be stuck to abstract the representations of numerous features.As a source of information, PrK has been used to enhance the performance of many ML and DL classifiers. The ML techniques that learn from a synthesis of multiple information sources that consist of data and PrK can be summarised by a recently proposed term, i.e., IML . By emplIML has been used in a wide range of research fields. In , the autFurthermore, PrK has also been made use of in many biological and medical fields. In , a prediAs to cancer-related methodologies, the authors of  incorporHowever, we can find a few additional articles on methods that integrate the concept of IML into the cancer prediction task. Therefore, in this study, we propose the IAP that describes an end-to-end method to incorporate the IML concept with basic classifiers to predict cancers based on gene expression signatures. Unlike the aforementioned PrK-based models, we fuse the knowledge in a different way that aims at enhancing prediction instead of features, which is more generalisable because the underlying classifier can be altered or specifically designed for the targeted task.In this section, we describe the formulations of our architecture and the associated task. We initially describe the cancer prediction task that is considered in this work; then, we describe an introduction to the preprocessing of data. Finally, we put forward the structure of our IAP and give a detailed explanation of the methodologies.The cancer prediction task considered in this work is essentially the classification of samples, which takes features  of samples as input and gives predictions  accordingly. The aim of this task is to build an errorless and robust classifier by which a high classification accuracy can, even on new samples, be obtained.The symbols used in this paper are listed in We follow up several preprocessing steps to extract PrK from its data source. In addition, the original dataset is standardised into graph-structured data to meet the needs of graph-based IAP models.As one of our major contributions, we employ the idea of IML in our model by integrating PrK from another data source. The fusing process varies according to different types of PrK and the supporting task. In this work, we use PrK as an aid for the training process in a 4-step manner: (1) PrK extraction, (2) data reduction, (3) PrK encoding and (4) PrK-based difference amplification. The initial two steps, viz. PrK extraction and data reduction, take place in the preprocessing period. The latter two steps are both taken in the training period. We propose their description in later sections.1. PrK ExtractionThe types of available PrK and the corresponding obtaining methods vary for different research contexts, and they require human effort to design the specific procedure for PrK extraction and encoding. For instance, if the training data are served in forms such as WHI images, it would be convenient to employ pathological image patterns of confirmed cases as PrK to enhance the convoluted representations of images. In this work, based on the type of dataset used , it is natural to find relevant gene-related knowledge, be it gene mutation rates of the cancers released by a recognized authority, relative expression levels of different genes recorded in an NIH database, etc.Through careful investigation, we eventually selected mutation rate as the source of PrK, because it fits well to the main procedure of knowledge integration  and is n2. Data Reductioni. g-th reported gene. The ‘reduced’ feature vectors of all samples were produced and saved for later utilisation.Once all Equation (1)ei˜=eThe matrices and vectors produced in the preprocessing period, i.e., adjacency matrix We standardised the original datasets into undirected and unweighted graphs, as some graph-based classifiers  take graph-structured data as their input. For simplicity, we constructed the graph according to the vector distances between the original samples. Specifically, we assumed two samples to be ‘correlated’ if their vector distance equalled or was smaller than the threshold. We used cosine similarity as the metric of the distance between entities, as described in Equation .(2)lij=i. We used the mean distance of all neighbours as the threshold, i.e.,The adjacency matrix of the graph was then constructed according to Equation (3)Aij=1The IAP architecture can be divided into modules, viz. (1) Underlying Classifier Module (UCM) and (2) Knowledge Integration Module (KIM). They are trained in an end-to-end manner. The overall structure of IAP is shown in The subsequent part will follow up the division in The UCM aims at giving a basic prediction result based on the original dataset. The prediction vectors are then fed into the KIM and enhanced therein. The enhancements will be formalised in detail in To be precise, As a key feature of IML, the KIM serves as an extension to the UCM. It learns an effective way to fuse PrK into the prediction process e˜ and the normal sample representative, which is described as Equation .(6)di=ei and normal samples are dissimilar in terms of gene expression, and so does the gene mutation rate. Inspired by this, a value of how likely this sample can be categorised as tumour type c can be obtained by calculating an element-wise similarity between likelihood of sample i being diagnosed as tumour type c with only a reference to PrK. A prediction vector can thereby be constructed according to Equation sic=sEquation (8)pipi=soAs an example, In this example, the domain data, i.e., gene signature of cancer/normal samples, are preprocessed to a graph-like structure. The graph is then fed into the first GGAT layer of the UIM, followed by two subsequent processes performed by the latter two GGAT layers. Furthermore, as in KIM, the reduced feature vectors are squashed into one, which is then used by the encoder to give predictions based on knowledge and features. The prediction vectors produced, i.e., We implemented four IAP models based on two traditional machine learning classifiers that were previously utilised in similar cancer prediction tasks ,54, i.e.To evaluate the effectiveness of the IAP, we carried out cancer prediction experiments on several TCGA datasets that are available to the public. All of these datasets were initially preprocessed  into grahttps://www.cancer.gov/about-nci/organization/ccg/research/structural-genomics/tcga (accessed on 28 September 2021). The datasets can be downloaded from the Genomic Data Commons (GDC) Data Portal ) of the U.S. National Institutes of Health (NIH).The datasets on which we carried out experiments were released by The Cancer Genome Atlas (TCGA), a cancer genomics programme of the U.S. National Cancer Institute and the U.S. National Human Genome Research Institute, each containing gene expression values (gene signatures) of samples from either a specific type of tumour or normal tissues. Detailed information on TCGA can be found in In this work, we chose six datasets (tumour types) for evaluation, because they are covered by both TCGA and the original research article of the PrK: Urothelial Bladder Carcinoma (BLCA), Breast-Invasive Carcinoma (BRCA), Head and Neck Squamous Cell Carcinoma (HNSC), Kidney Renal Clear Cell Carcinoma (KIRC), Lung Adenocarcinoma (LUAD) and Lung Squamous Cell Carcinoma (LUSC). Statistics of these datasets are available in Here, we utilise t-SNE  to visuaWe observed an imbalance between the two tumour types in terms of the number of samples, i.e., the number of tumour samples is much more than that of the normal samples in any dataset. This disparity can cause a variety of problems, such as overclassifying the majority group  and slowIt is not reasonable to mitigate the number of any sample types, as the datasets used are all small in size (no more than 1215 samples). In this case, to minimise the effect of data imbalance, we only consider methods that increase the number of minority samples. Advanced upsampling techniques such as ADASYN and SMOTE are generative . However, the generation of these artificial samples does not involve specialised knowledge. They simply use Euclidean distance and random bias to sample new data. Furthermore, according to the t-SNE visualisation of the datasets , it can In our experiments, the parameters of our model are initialised using Xavier initialization  and are To mitigate the occurrence of possible exceptions, we employed k-fold cross-validation in our experiments, where the datasets are randomly divided into five parts, and each is then utilised to test the models. Eventually, the average performance of the five test sets is reported. Considering the size of the datasets, we set k = 5 to ensure that the test sets were not too small whilst enabling adequate repetitive experiments.Here, we plot the changes of accuracies and loss values during GGAT’s and IAP-GGAT’s training process in Noticeably, for IAP-GGAT, the learning curve has a highly remarkable smoothness, whereas there is observable volatility of GGAT when reaching its plateau. Under extreme circumstances, as in KIRC, for example, where a sudden expansion of loss occurs at approximately the 980th epoch, we can hardly observe even marginal changes in either loss or accuracy of IAP-GGAT at the corresponding moment. Furthermore, the time needed for IAP-GGAT to converge  is visibly shorter than that for GGAT (between 50 and 400 epochs). These differences highlight two strengths of IAP models: (1) less volatile training curve and (2) less time required for model convergence.bold texts;The best results are in A number marked ‘+’ indicates a higher result in the corresponding pair of models;A number marked ‘=’ indicates that the results of corresponding pair of models are equal to each other;The values in the rightmost column are the count of ‘+’s that the corresponding models achieve on the datasets listed.The experimental results in terms of accuracy, recall and F1-score are shown in The experimental results show that all of our IAP implementations, on average, achieved better performance on a majority of datasets compared to their non-IAP counterparts. Only a marginal difference can be observed where the IAP models ranked second. Furthermore, in most cases, the IAP models achieve more ‘+’s than their basic version, with the exception of recall on (IAP-)SVM, where small differences can be discovered in the experimental results on BRCA and KIRC datasets. We believe the occurrence of this is occasional and can be amended by refining the model’s initial settings . Nevertheless, the effectiveness of our PrK integration technique is clearly highlighted.Additionally, for ablation study purposes, we introduce a control group of experiments performed on a naïve classification model that makes decisions based only on PrK (without other underlying models), whose performance in terms of accuracy is shown as ‘PrK-ONLY’ in As discussed in We performed a series of additional experiments using the same settings as those in The charts point out that the IAP-GGAT achieves the highest scores in the three reported metrics. Furthermore, the IAP models  all achieve higher scores than the base models  before and after the training data reduction, which reveals the effectiveness of the KIM. The most significant finding in this additional study is that despite tiny effects of reduced datasets can be observed, IAP models can often mitigate the influence, resulting in acceptable scores in certain circumstances where the training data have a limited size.We plotted bar charts to compare the training time of the IAP and normal models, as shown in From the bar chats, we can observe a marginal difference between the average training time of a normal model and its IAP implementation (shorter than 1 s). Despite the small increase in training time, the IAPs achieve significantly better performance than the non-IAPs. Furthermore, the increments in training time on different datasets and of different models are roughly the same  into the learning process, thereby providing sufficient information to enhance our implemented IAP model’s performance to a greater extent. Moreover, we will attempt to verify the interpretability of our architecture according to certain specifications so as to ensure its safety of usage under actual production circumstances.


Background: Cancer cell lines are frequently used in research as in-vitro tumor models. Genomic data and large-scale drug screening have accelerated the right drug selection for cancer patients. Accuracy in drug response prediction is crucial for success. Due to data-type diversity and big data volume, few methods can integrative and efficiently find the principal low-dimensional manifold of the high-dimensional cancer multi-omics data to predict drug response in precision medicine. Method: A novelty k-means Ensemble Support Vector Regression (kESVR) is developed to predict each drug response values for single patient based on cell-line gene expression data. The kESVR is a blend of supervised and unsupervised learning methods and is entirely data driven. It utilizes embedded clustering  and local regression (Support Vector Regression) to predict drug response and obtain the global pattern while overcoming missing data and outliers’ noise. Results: We compared the efficiency and accuracy of kESVR to 4 standard machine learning regression models: (1) simple linear regression, (2) support vector regression (3) random forest (quantile regression forest) and (4) back propagation neural network. Our results, which based on drug response across 610 cancer cells from Cancer Cell Line Encyclopedia (CCLE) and Cancer Therapeutics Response Portal (CTRP v2), proved to have the highest accuracy  measure). We next compared kESVR with existing 17 drug response prediction models based a varied range of methods such as regression, Bayesian inference, matrix factorization and deep learning. After ranking the 18 models based on their accuracy of prediction, kESVR ranks first (best performing) in majority (74%) of the time. As for the remaining (26%) cases, kESVR still ranked in the top five performing models. Conclusion: In this paper we introduce a novel model (kESVR) for drug response prediction using high dimensional cell-line gene expression data. This model outperforms current existing prediction models in terms of prediction accuracy and speed and overcomes overfitting. This can be used in future to develop a robust drug response prediction system for cancer patients using the cancer cell-lines guidance and multi-omics data. Emerging Next Generation Sequencing (NGS) techniques and large-scale cancer screening data helps in achieving this goal ,2. Databne 2019) ,4 quantine 2019) .Accuracy in drug response prediction is of utmost importance in this regard. Over the years various models have been developed for this purpose ,7,8,9. CThese models, however, only focus on a single model  trained However, it is hard to identify the homology features which one contribution to drug response prediction for increasingly heterogeneous datasets comprised of multi-omics data collected from overlapping latent low-dimensional subpopulations. Principal Component Analysis (PCA) can generate statistically uncorrelated principal components (PC) while retaining as much as possible of the variation present in the original data set. PCA has been used previously to delineate homogenous regions by PCs regression and applying in all kinds of fields such as temperature , hydroloMachine learning nonlinear regression Support Vector Regression (SVR) was first introduced by Vapnik  and has To address these challenges, a novelty k-means Ensemble Support Vector Regression (kESVR) is developed to predict each drug response values for single patient based on cell-line gene expression data. kESVR’s origin stems from previous work interval SVR , where wThe kESVR is a blend of supervised and unsupervised learning methods while being an entirely data driven model. It utilizes embedded clustering (PCA and k-means clustering) and local regression (Support Vector Regression) to predict drug response and obtain the global optimal value with the smallest mean squared error (MSE) while overcoming missing data and outliers’ noise. In contrast to classical 17 machine learning models  that esthttps://portals.broadinstitute.org/ccle, accessed on 1 June 2019) [v2, 2015; http://portals.broadinstitute.org/ctrp, accessed on 1 June 2019) [Molecular mRNA data from database Cancer Cell Line Encyclopedia  ,25,26 whne 2019) ,4 which k-means Ensemble Support Vector Regression (kESVR) model consists of 4 distinct steps. In this section we provide a brief overview of the different steps comprising our model. Dimensional reduction: In the first step, we convert high dimensional gene expression data to low dimensional data for better handling and visualization in the subsequent steps.Embedded Clustering: In the second step, we split the lower dimensional data into distinct clusters based on the their labeled/given drug response value. This is the done so that data points (cell-lines) that have similar or close drug response values are grouped together.Local regression and ensemble value selection: In the third step, we train different instances of a machine learning (ML) model on each of the different clusters of data points obtained in the previous step. If there are Optimal drug response value prediction: In the final step, we optimize the number of clusters We define the problem of drug response prediction using cell-lines gene expression as follows: Given gene expression data Let pth Principal Component of p in such a manner that results in minimum loss of information (variation of the data) and use that We use PCA  to perfoUsing the We create a set of labeled data Instead of the full set of genes (ns (SVR) . We traiustering  on Ψ. out of the 481 drugs from CTRP. We compare kESVR with 4 other machine learning models: (i) Linear regression (LR), (ii) Back Propagation Neural network (BPNN), (iii) Support Vector Regression (SVR) and (iv) Quantile Regression Forest (QRF). For each model the labeled dataset We also compare the model setup time across all the models for each drug. We next compare the performance of kESVR against the same 4 models for 5 drugs that have the highest variance in drug AUC values amongst the 481 drugs. These 5 drugs are SB743921, paclitaxel, daporinad, neopeltolide and docetaxel. The idea is to see how well kESVR handles large variance of AUC value for a particular drug across the 610 cell-lines. We use the same metric for evaluation. We further compare the goodness of fit of our kESVR model with the 4 models using the R-squared metric. For each drug, we treat the entire data of that drug as our test dataset and use the trained 5 models to generate predicted values for the test data. We then calculate the R-squared value  select the prediction value whose data-point has the maximum average Spearman correlation value with all cell-lines in the training set of its parent SVR. (ii) return a weighted average of the (iii) select the prediction value whose data-point has the maximum number of neighbors (using (iv) return a weighted average of the Our experimental comparisons show that the maximum Lastly feature selection plays an important role in the performance of our model kESVR. Intuitively, features indicate those genes that play a crucial role in the functioning of a drug. Expression (high/low) of these feature genes, affects how a cell-line/patient responds to the drug. We use genes that are known to be target genes for the drugs (available from public databases) as our feature genes. This contributes to the improved performance of kESVR over other drug response prediction methods.In this paper we introduce a novel computational model for drug sensitivity prediction. We show that our model kESVR consistently outperforms existing traditional ML tools and at least 12 recently developed drug response prediction methods in terms of prediction accuracy. Even among the 4 good methods deemed best performing by Chen and Zhang  it is thAs of now kESVR only uses gene expression data. However, as future work we plan to extend our kESVR model, to incorporate other kinds of omics data such as mutation, copy number variation and methylation data. Currently kESVR does not allow for feature selection. This is something that we will be working on, so that future users can use kESVR to get bio-markers for the drugs as well as their response prediction results. Finally, we plan to use kESVR to create a precision medicine drug recommendation system for patients wherein we use cancer lines and other omics data to perform drug response prediction and drug ranking recommendation efficiently.


The sparse data were obtained by reducing broadband Fourier transform infrared attenuated total reflectance spectra of bovine and human cartilage, as well as of simulated spectral data, comprising several thousand spectral variables into datasets comprising only seven spectral variables. Different preprocessing approaches were compared, including simple baseline correction and normalization procedures, and model-based preprocessing, such as multiplicative signal correction (MSC). The optimal preprocessing was selected based on the quality of classification models established by partial least squares discriminant analysis for discriminating healthy and damaged cartilage samples. The best results for the sparse data were obtained by preprocessing using a baseline offset correction at 1800 cm Infrared spectroscopy is an emerging technique in biomedical applications that has already demonstrated potential for diagnostics of various pathological conditions, such as cancer, osteoarthritis, and infectious diseases ,2,3. TheAs one might guess, sparse data have their challenges when it comes to analyses and modelling. One of the challenges in dealing with sparse data is preprocessing. Preprocessing involves removal of unwanted variation in the data that is due to white noise and other instrumental effects, as well as to detrimental interaction of infrared radiation and the sample. This type of detrimental interaction, such as scattering and multiple reflections of the infrared radiation, often hinders analysis of spectral data. Preprocessing of broadband spectral data is known to improve data modelling results, at least when classical machine learning methods are used ,15,16,17In this study we compare different preprocessing methods for the sparse data obtained from the raw broadband Fourier Transform Infrared Attenuated Total Reflectance (FTIR-ATR) spectra. ATR-IR spectra are recorded by placing a sample in contact with internal reflection elements, such as diamond or zinc selenide crystals. In general, ATR-IR data is devoid of many unwanted spectral variations that are commonly encountered in transmittance IR data, such as scattering of the infrared radiation at a sample surface or interior (for example scattering artifacts in microparticulate samples), and multiple reflections of the infrared radiation within a thin sample (for example interference fringes in film samples) ,25,26. TOptimal preprocessing of the spectral data was selected based on the performance of classification of samples into healthy and damaged cartilage groups. In addition to the binary classification into healthy and damaged samples, a multiclass classification model was established for the bovine data. Classification models were built using Partial Least Squares Discriminant Analysis (PLSDA) ,28,29.Binary classification of cartilage samples of human, bovine and simulated data was done in this study. Cartilage damage was graded using the Osteoarthritis Research Society International (OARSI) grading system. Balance between groups is an important aspect of successful classification. When the classes are heavily imbalanced, linear models such as PLSDA tend to become biased towards the larger class. Therefore, to balance the groups of healthy and damaged samples, the cut-off oups see . It is iWhen multiclass classification into treatment groups G1–G5 and binary classification into treatment and control groups were done for the bovine data, oversampling was not needed because the treatment groups G1–G5 as well as treatment and control groups were balanced a priori. In Prior to preprocessing, spectral quality was tested to remove spectra with too little cartilage signal and water spectra. Such spectra might be obtained during measurements when the contact between the cartilage samples and the ATR crystal is suboptimal or, in the case of excessive cartilage damage, when the spectra show signals of synovial and phosphate buffered saline (PBS) fluids surrounding the sample. In both cases, the obtained spectra are not useful for cartilage degeneration diagnostics, and such spectra can be considered as low-quality spectra and removed. Such preclassification resulted in nine spectra of the bovine dataset removed, which left 171 spectra in the treatment group and 180 in control for further analysis.Some technical replicates of human data were removed due to technical issues. Then a preclassification method was applied to the datasets to remove water spectra. Twenty-one spectra of dataset 1 and six of dataset 2 were removed, which left 781 spectra in dataset 1 and 791 in dataset 2 for further analysis. The resulting spectra of dataset 2 after preclassification are presented in In addition to the measured spectra of the cartilage samples, we simulated cartilage spectra. Spectral simulation was done by two independent PCA models for healthy and damaged group, respectively, where the threshold was set to To find optimal preprocessing of the broadband spectra, two types of measures of the preprocessing success were used: (i) visual inspection of the spectra, and (ii) classification results. The latter was based on the performance of binary models to classify healthy and damaged samples established using preprocessed broadband spectra and sparse spectra generated from the broadband spectra. Thus, eight datasets were used for that task: four sets of broadband spectra  and four sets of sparse spectra obtained from each set of the broadband spectra.−1, followed by peak normalization at 850 cm−1;  MSC with and without weighting;  EMSC1 with and without weighting; and  EMSC2 with and without weighting. Broadband spectra preprocessed by EMSC1 with weights showed visually better results where all baseline variations were removed. The classification results for the weighted EMSC1 preprocessed broadband spectra were among the best results for all eight datasets (results not shown). Thus, the optimal preprocessing for the broadband spectra was identified to be weighted EMSC1 method. The seven different preprocessing options were compared: (1) simple preprocessing which includes baseline offset correction at 1800 cmSparse data were collected from the benchmark broadband data for each dataset. Different preprocessing strategies for the sparse raw data were then compared to the results of classification using sparse data of the benchmark.−1 with peak normalization at 850 cm−1) with MCR, FNR and FPR around 40%. Other approaches gave worse performances: MSC and EMSC1 gave almost 50% misclassification, while no preprocessing (raw data) gave very biased classification towards the healthy group (FNR raised to around 50% while FPR was close to 20%). Interestingly, sparse data of the benchmark spectra, i.e., spectra preprocessed before variables were selected, also gave quite unsatisfactory results with strong bias towards the healthy group.Different preprocessing strategies were applied to the sparse spectra obtained from the raw broadband spectra of human datasets 1 and 2, bovine dataset, and the simulated data. Binary classification models differentiating samples into healthy and damaged groups based on OARSI grades were built by PLSDA using preprocessed sparse data. The results are presented in The classification results for the human 1 dataset were quite similar irrespective of preprocessing strategy selected, with MCR around 30% see . When usResults of classification into OARSI groups of healthy and damaged samples obtained for the bovine data are presented in The multiclass classification results for the classification into the five bovine treatment groups are presented in When looking at the groups G1–G5, we can see that MSC preprocessing in four cases  produced lowest FNR among preprocessing approaches for the sparse data followed by simple correction. In cases of classes G2, G4 and G4, EMSC1 produced as good results as MSC preprocessing or even better results.Binary classification into treatment and control groups for the bovine data is shown in Binary classification for the simulated data worked well both for the broadband data and sparse data with MCR, FNR and FPR around 22% and 30%, respectively see . The bes−1 and 850 cm−1 which were selected for the preprocessing purposes, and therefore were not supposed to be important for the discrimination of the cartilage quality.To support the selection of the seven wavelengths as important for the classification of the cartilage diagnostics, regression coefficients of the PLSDA models are presented in −1 spectral resolution, digital spacing of 0.2411 cm−1, and averaging 64 scans over the range from 4000 cm−1 to 400 cm−1. The background spectrum of air was measured for each sample separately. Measurements were controlled by OMNIC software . Thus, the dataset of treatment samples consists of 180 spectra in total with additional 180 spectra of controls.Bovine cartilage samples are commonly used as a model system for human cartilage samples . The sam−1 to 600 cm−1 at a spectral resolution of 2 cm−1, and digital spacing of 1.0292 cm−1. All samples were measured in triplicates, resulting in 807 and 801 spectra in dataset 1 and dataset 2, respectively.Samples of nine human cadavers were obtained for this study . During After the experiments, the sample plugs were fixed in formalin, decalcified in ethylenediaminetetraacetic acid, and embedded in paraffin. Subsequently, 3-µm-thick histological sections were cut and stained using Safranin-O for qualitative evaluation of the cartilage damage.One of the most used assessment systems for cartilage damage is the Osteoarthritis Research Society International (OARSI) histopathological grading . It is a−1 were simulated with the digital spacing of 1.0292 cm−1.In addition to experimental data, simulated broadband spectral data were also used in this study. The simulations were based on principal component analysis (PCA) using measured data of human samples of dataset 2. Full range spectra 4000–600 cm−1 by applying a window function based on Tukey [PCA decomposition of matrix A.Calculation of mean New scores The first set of simulated data were obtained by The resulting dataset Group specific EMSC1 variations were added to simulated pure spectra using parameters w were also added by randomly drawing from a uniform distribution The spectra were merged into one dataset and white noise vectors To obtain the simulated data, we first corrected the measured spectral data by the extended multiplicative signal correction (EMSC) model with a linear term  and the method as described below. To capture potential differences in parameters of healthy (OARSI grade 0–2) and damaged samples (OARSI grade > 2), the EMSC1 parameters for the samples of these groups were stored separately on Tukey   of collagen; 1560 cm−1, corresponding to the C-N-H stretching and bending vibration (amide II) of collagen; 1210 cm−1, corresponding to the O=C-N-H stretching and bending vibration (amide III) of collagen, and 1080 cm−1 corresponding to the C-O stretching vibration of carbohydrate residues in collagen and proteoglycans [−1, is a band related to librations of water in the cartilage and synovial fluid. This wavenumber was selected as a reference variable since it is assumed that the chemical absorbance of water is mainly invariant in all the samples, and therefore it could be used for normalization of all measured variables.The first wavenumber at 1800 cmoglycans ,38. The A preprocessing approach that is widely used in infrared spectroscopy in Extended Multiplicative Signal Correction (EMSC) and Multiplicative Signal Correction (MSC). The MSC correction model is given byresidual ,21,39. ASC model ,38. EMSCWe used the following convention to refer to EMSC models with different orders of polynomials. EMSC1 is the model obtained when MSC is extended by linear term −1 and 900–800 cm−1. The weights are presented in In a weighted EMSC we applied weighting of spectral regions when estimating the EMSC parameters. This strategy was followed when we needed to up-weight or down-weight some of the regions in the spectra when estimating the EMSC parameters. In this paper, we up-weighted regions 1800–1780 cm−1, followed by peak normalization at 850 cm−1;  MSC with and without weighting;  EMSC1 with and without weighting, and  EMSC2 with and without weighting. The broadband data preprocessed by weighted EMSC1 are referred to as the benchmark data in this study.The following preprocessing strategies for the broadband spectra were tested: (1) simple preprocessing which includes baseline offset correction at 1800 cm−1, followed by peak normalization at 850 cm−1; (2) MSC, and (3) EMSC1. The EMSC2 approach was not used in this study for two reasons. First, EMSC with its four parameters to be estimated for the sparse spectra containing seven channels has too few degrees of freedom to be implemented. Second, the spectra in this study were ATR-FTIR spectra and are thus devoid of complex spectral variations caused by radiation-sample interactions (such as scattering artifacts and interference fringes), that would have required correction by nonlinear EMSC terms. Simple baseline effects that can occur in ATR spectra can be corrected by MSC or EMSC1. To compare the performance of the preprocessing approaches for the sparse data, classification models were built, and the results were compared to the classification results using sparse data of the benchmark data - broadband spectra preprocessed by EMSC1 with weights.Different preprocessing strategies were evaluated for the sparse data in this study. The sparse data were collected from raw broadband spectra of human dataset 1 and 2, the bovine dataset and the simulated dataset. The sparse raw data were preprocessed by three approaches: (1) simple preprocessing which includes baseline offset correction at 1800 cmThe MSC method was also used in this study for preclassification of spectral data. The preclassification method was proposed in  and is bthod see .OptA) corresponded to the maximum accuracy of the group with the lowest classification accuracy. Three different models were used in this study. First, binary classification using OARSI groups of the cartilage samples for human, bovine and simulated data. Second, multiclass classification into treatment groups of bovine samples: five groups G1–G5. Third, binary classification into treatment and control groups for bovine samples.To establish classification models, the partial least squares discriminant analysis (PLSDA) method was used. In PLSDA, a classifier is established by regressing the matrix of indicator variables (dummy matrix representing each sample class belongingness) Y onto the matrix of FTIR spectra X ,29,41. TOptA was obtained by Thus, in the case of binary classification Due to imbalance in the OARSI groups of bovine data, an oversampling technique was used . OversamThe study was done in connection with the Horizon 2020 Research and Innovation Programme (H2020-ICT-2016-2017) project MIRACLE, in which the primary goal was to build a system based on seven QCL lasers in mid-IR region and an ATR probe to assess cartilage quality. In this study we evaluated different preprocessing strategies for such sparse data using spectral data of FTIR-ATR.−1 and peak normalization by the water band at 850 cm−1. In some cases, preprocessing of sparse data by MSC showed good results as well. However, in most cases when extended by more terms, as in EMSC1 by a linear term, the performance of the classification dropped. Since the EMSC1 estimates three parameters from the seven selected wavenumbers in the sparse data, relevant chemical or physical information related to cartilage quality, may be modeled and removed from the data.The study shows that when an optimal preprocessing of the spectra is selected, the classification success using sparse data does not drop strongly compared to models where the broadband spectral data were used. The best preprocessing for the sparse data in this study seemed to be simple baseline correction by a baseline offset at 1800 cmThe data used in this paper were obtained from FTIR-ATR measurements, which are highly reproducible, and mostly devoid of undesirable spectral variations caused by radiation-sample interactions, such as scattering artifacts and interference fringes. Therefore, advanced spectral preprocessing approaches are normally not needed in FTIR-ATR spectroscopy. Since the MIRACLE fiber-optic probe will be based on a similar ATR measurement setup, it is expected that the sparse data will have a low level of undesirable spectral variations. When data are obtained by other techniques, such as transmission, transflection and specular reflection measurements, EMSC preprocessing may be of advantage. However, more of the spectral variables would be needed to apply an EMSC model with four parameters to have enough degrees of freedom for further analysis.The spectral simulation algorithm proposed in the study was shown to create cartilage spectra with distinct patterns between healthy and damaged cartilage. The classification of these spectra worked very well both for the broadband spectra and the sparse data.The results of the study were obtained using cartilage spectra; however, they can be generalized and applicable to different type of samples, and even other types of spectral data. In the era of rapid development of cheap photonic solutions, the preprocessing approaches for sparse spectral data will show their potential.