Independent component analysis (ICA) is a matrix factorization approach where the signals captured by each individual matrix factors are optimized to become as mutually independent as possible. Initially suggested for solving source blind separation problems in various fields, ICA was shown to be successful in analyzing functional magnetic resonance imaging (fMRI) and other types of biomedical data. In the last twenty years, ICA became a part of the standard machine learning toolbox, together with other matrix factorization methods such as principal component analysis (PCA) and non-negative matrix factorization (NMF). Here, we review a number of recent works where ICA was shown to be a useful tool for unraveling the complexity of cancer biology from the analysis of different types of omics data, mainly collected for tumoral samples. Such works highlight the use of ICA in dimensionality reduction, deconvolution, data pre-processing, meta-analysis, and others applied to different data types . We particularly focus on the technical aspects of ICA application in omics studies such as using different protocols, determining the optimal number of components, assessing and improving reproducibility of the ICA results, and comparison with other popular matrix factorization techniques. We discuss the emerging ICA applications to the integrative analysis of multi-level omics datasets and introduce a conceptual view on ICA as a tool for defining functional subsystems of a complex biological system and their interactions under various conditions. Our review is accompanied by a Jupyter notebook which illustrates the discussed concepts and provides a practical tool for applying ICA to the analysis of cancer omics datasets. Cancer research is one of the most important providers of large-scale molecular profiling data, which help in understanding not only the state of human cells in disease but also shed light on the normal physiological processes measurable and detectable in various kinds of omics datasets. Determining robust and biologically meaningful ways of quantifying cellular and organismal and normal and pathological physiology using high-throughput molecular data remains a major challenge (making biology a quantitative science). Different kinds of biological processes leave characteristic traces at different levels of genome-wide measurements depending on their nature and timescales: some significantly affect transcriptomes, some rather modify DNA methylation programs or mutational spectrum, others are measurable only at the level of proteome and phosphoproteome. In order to reliably quantify some of these biological mechanisms, one will need to design multi-omics signatures spanning several levels of molecular data descriptions. On top of this, various technical factors interplay with biological ones, frequently in a way which makes it difficult to clearly distinguish both.linear mixture of signals plays a pivotal role in the modern molecular data analysis. In this framework, one assumes that a measurable elementary quantity such as expression of a single gene is a result of weighted summation of some latent, and not always directly observable, factor activities which should have associated numerical values. The nature, the number of factors and the way they are represented numerically can be known or unknown in advance. A toolbox of existing mathematical approaches provides concrete scenarios in which the additive factors can be determined and quantified, under acceptance of certain assumptions about the statistical properties of their numerical values or the weights connecting them to the measurements.Rarely does molecular data “speak for themselves”: they need to be properly pre-processed, analyzed in the light of mathematical modeling, statistical assumptions, and prior biological knowledge and, finally, should be represented at some pre-defined level of abstraction. In this sense, one of the simplest paradigms of One of the standard methods in such a toolbox is independent component analysis (ICA) having a long standing history of application to biological data, including the analysis of molecular profiles . Formally, ICA belongs to a family of methods called matrix factorizations , the mosThe first applications of ICA in biology contrasted it to PCA and standard clustering methods and found that the factors determined through ICA are easier to interpret biologically ,2. This Independent component analysis and matrix factorization approaches are standard methods in the rapidly growing arsenal of machine learning methods applied to the molecular biology and medical data. At the same time, remarkable success has recently been achieved in applying deep learning techniques in certain fields of cancer biology such as clinical imaging of various kinds ,6,7,8,9.Over the last decade, significant experience in applying ICA to different kinds of omics data for addressing various problems has been obtained, including data pre-processing, task of cell type deconvolution, and meta-analysis of multiple omics datasets c. In thihttps://github.com/sysbio-curie/ICA-in-Cancer-research-review-materials.This review is accompanied by interactive Jupyter notebook located at N, and with number of observed features, m) as an input and approximates it as a sum of products of p pairs of vectors of size N and m. The fundamental equation for all matrix factorization methods states (note that the product of ka and ks vectors gives a one-rank matrix of the same dimension as X):ka and ks such that: Independent component analysis belongs to a family of matrix factorization methods. Each of these methods takes a rectangular matrix ka and ks will be called a component throughout this review. Therefore, a component is represented by a vector ks of size m containing weights of omics variables . At the same time a component is associated to a vector ka of size N, containing contributions of the component to measured samples. We will use these notations and meaning of ka and ks vectors throughout the whole review.Each vector pair ka and ks. For example, the terms “loadings”, “activations”, “factor strength” or “sample-associated weights” have been used to denote the elements of ka vectors. The matrix composed from the ka vectors is sometimes called the “mixing matrix” and denoted as A. The elements of ks vectors have been called “weights of the component” or “signals” and the matrix composed of them (denoted as S) is sometimes called the “signal matrix”. Moreover, ks vectors themselves are frequently referred to as “components” or “factors”. In the matrix factorization literature, various terms are used to denote the elements of the vectors ks vector is frequently named a metagene [ks even when describing application of ICA to various data types. Similarly, the ka vectors are sometimes called metasamples, and we will adopt this term in the text , see In the context of transcriptomic data analysis, the metagene ; in the p metagenes. Each metagene abstractly represents a molecular program  by assigning a numerical weight to each gene of the organismal genome. The activity of metagenes in a sample is combined additively, and each metagene acts on a sample with a sample-specific strength or activity. Activities of the same metagene over all measured samples is called a metasample. A metasample is the profile of the corresponding metagene activity similarly to a gene expression profile across samples.Intuitively, a transcriptome of a biological sample is described as a combined action of X matrix is known; the ka and ks vectors are unknown. As such, the problem of matrix factorization (**) is heavily underdetermined, and additional constraints need to be introduced on ka and ks vectors in order to find its solution. First of all, it can be required that the all ka vectors would have length one. In the equation (*), only the ka vectors:  = 0, for i ≠ j and that the solution of (**) should give the same result for different orders of matrix decomposition p, i.e., ka and ks vectors computed for the order p = p’ would be the same as for the decomposition of order p’’ > p’. In this case, solving (**) is equivalent to computing the singular value decomposition (SVD) of X and gives a set of principal components. There exist several ways to introduce PCA, as reviewed in Reference [Furthermore, one can require orthogonality of the eference .ka and ks vectors would be non-negative. This constrains the problem (**) and leads to NMF. The simplest approach to solve (**) with these constraints is to repetitively apply the non-negative least squares regression method, considering ka as unknown at one iteration and ks as unknown at the next iteration, until convergence to a local minimum.Alternatively, one can require that all elements of as mutually independent as possible. More precisely, the elements of vectors ks  have to represent maximally mutually independent distributions, for different k. The perfect independence would mean that the joint probability distribution P can be factorized as ks are i.i.d. samples of the underlying probability distributions kP(ks).When computing ICA, the resulting components are required to be ka vectors and the components cannot be naturally ranked. The NMF components contain only non-negative elements, which makes the intuitive picture of the additive action of metagenes simpler to interpret, while in PCA and ICA some metagenes can cancel the action of other metagenes if they are summed up with different signs.From the different nature of the constraints follow different properties of matrix factorization algorithms . It can be shown that maximizing entropy of joint distributions of pairs of ks leads to minimizing their mutual information. One of the historically first and still popular practical algorithms for solving ICA problem is based on the general Infomax (or maximum entropy) principle . Indeed,ks distributions [It appeared also that under some assumptions, minimizing the mutual information is equivalent to maximizing the non-Gaussianity of the individual ibutions . Quantifal works ,20. ks or ka vectors is always finite in real-life applications, one needs to introduce the way to effectively approximate it from the finite samples. For this purpose, various surrogate functions  have been proposed, one the most popular of which involving the kurtosis. Empirically, kurtosis was found to be an appropriate choice of non-linearity in the analysis of transcriptomic data. Other types of non-linearity functions have been suggested; however, the appropriate choice of non-linearity for applying ICA to different kinds of omics measurements remains an open question. The two most popular ICA algorithms based on non-Gaussianity maximization are fastICA [Since the length of the  fastICA  and join fastICA . Most of fastICA ,23.A typical preprocessing step used before application of ICA algorithms is the so-called data whitening or sphering see b. Whitenka and ks vector elements. This allows combining the nice properties of non-negative mixture problem and the requirement for mutual independence of the components. A kernel version of ICA have been developed [Various flavors of ICA have been suggested and some of them were tried on omics data. Bayesian ICA with prior constraints have been suggested and tried on the metabolomics data . The prieveloped  and spareveloped , but boteveloped ). Finalleveloped ,29 and rSome flavors of ICA seems interesting to explore more in biological applications, in the view of the concept of the integration of functional subsystems see , such asBesides the choice of ICA algorithm (which is frequently fastICA), there are several choices to be made when ICA is applied to omics datasets. The first evident but non-trivial choice concerns a necessity for data log-transformation, which is especially important in the case of gene expression and protein expression data. On one hand, it is strongly desirable in the case of, for example, RNA-Seq data, since empirically they are found to be characterized by log-normal distribution. When ICA is applied to non-transformed data, the resulting components are frequently dominated by single genes or single samples , which contradicts the initial concept of linear mixture . Simple log-transformation usually fixes this issue. However, log-transformation makes the direct interpretation of the ICA model difficult, since, formally speaking, one deals with a multiplicative rather than an additive model of signal mixture. This is particularly important for the applications of ICA in the field of cell type deconvolution where the linearity assumption is explicitly made for mixing transcriptomes of different cell types (see Reference  which ciks) or metasamples (vectors ka). Technically, the first case corresponds to the application of ICA algorithm to the initial matrix X containing samples as rows and omics variables as columns, and the second case corresponds to the application of ICA to the transposed matrix X. Surprisingly, both ways of applying ICA to omics data are wide-spread, and sometimes it makes an effort to figure out in which way ICA was applied. Some studies aim at maximizing the non-Gaussianity of metagenes [Another choice in applying ICA to a matrix of omics measurements is the choice between what distribution independence (or non-Gaussianity) is maximized . One canetagenes ,33,34,35etagenes ,37. Empietagenes . This chicasso method, introduced by the creators of fastICA [Furthermore, in several studies it was found that stabilized or consensus independent components have better characteristics in terms of generalization and interpretation ,39,40,41ka and ks can be inverted simultaneously without changing the definition of the component. Some methods (such as BIODICA or DeconICA) avoid this ambiguity by assuming that the heaviest tail of the ks distribution should correspond to positive values, which usually gives satisfactory results. In Reference [ka and ks. In this case, the orientation was chosen based on the values of ka vectors. For example, in a disease study, one can require that any component would be oriented towards aggravation of the disease condition . This approach was recently used for quantifying disease comorbidity using ICA [Lastly, in some applications of ICA , it is desirable to fix the orientation of the independent components. We remind that in PCA and ICA, the signs of the elements in the vectors eference , each ICsing ICA .In several recent studies, ICA was systematically compared with the other most used matrix factorization methods such as PCA and NMF, using large collections of cancer omics measurements. In Reference , it was Three major matrix factorization approaches were systematically discussed in a recent review for their ability to discover functional subsystems or tissue-type specific signals . The maiall computed components. In the case of ICA, which geometrically only rotates the PCA axes, choosing the number of independent components can rely either on the methods for determining the number of relevant principal components or it can use some features of the independent components themselves in order to determine the optimal decomposition order.The most important parameter in the application of any matrix factorization method is the number of components to determine. This question is less crucial in the case of PCA due to the orthogonality constraint and that computing higher-order components does not affect the definition of the lower-order ones. However, this is not the case with ICA and NMF: choosing the order of decomposition affects the definition of In the first case, the effective global dimensionality of the data can be determined through the standard Kaiser rule, use of broken stick distribution, Horn’s parallel analysis or estimating the conditional number of the covariance matrix . One canHowever, the second case appears to be more consistent in applications, even being computationally more challenging. Thus, in Reference , Bayesiaks distributions are characterized by the presence of one or few weights with exceptionally large values, separated by a significant gap from the other values. Secondly, it was shown that a certain level of over-decomposition of transcriptomic datasets, i.e., choosing the number of independent components several times larger than MSTD, does not drastically change the definition of most of the components within the MSTD range. At the same time, it was observed that increasing the number of independent components over the MSTD value sometimes leads to biologically meaningful splitting of the components. For example, a component within the MSTD range which was associated with the total level of immune infiltrate in tumoral microenvironment splits into three components in higher-order decompositions which can be associated with the presence of T-cells, B-cells, and macrophages [In Reference  stabilitrophages ,42.In Reference , a rangeks vectors  to the set of most contributing genes, or Gene Set Enrichment Analysis to the whole ranking defined by ks), using large-scale collections of reference gene sets. The distribution of gene weights from ks vectors can be projected on top of genome-wide biological network reconstructions where the network edges represent different types of interactions or regulations between genes and/or proteins. This can be further used for various types of network-based analyses, leading to the determination of biological network “hotspot” areas and eliminating the need of having a reference gene set collection [ks vectors  can be projected onto genome and be a subject of peak-calling analysis, which can sometimes lead to associating a component to genomic alterations [Assigning a meaning to the extracted independent factors remains a major problem in exploiting ICA in biological research. Standard practice consists of applying various kinds of functional enrichment analyses to llection . The sk erations . ka are used to associate components to sample annotations such as clinical data . Metasamples can be also associated with some clinically relevant molecular data, such as mutations in known cancer drivers for a particular cancer type. Metasamples can be also associated with known labels for molecular tumor subtype.Metasample weights ks vector can be projected on cancer-specific biological network maps such as the Atlas of Cancer Signaling Network (ACSN) using user-friendly Google Maps-based online platforms such as NaviCell and MINERVA [In parallel to rigorous statistical testing, insightful visualizations of the results of ICA application can be of great help. For example, gene weights from  MINERVA ,52. Func MINERVA . https://github.com/LabBandSB/BIODICA), a set of tools is provided for performing hypergeometric tests of the metagenes, automated feeding of Gene Set Enrichment Analysis with ICA metagenes, projecting metagenes onto biological network maps, correlating computed metagenes with a reference database of previously annotated metagenes, associating components with categorical and numerical sample annotations, and tools for meta-analysis of ICA decompositions. There exist integrated solutions allowing the computation of ICA components for omics datasets and containing a built-in set of tools for their interpretation. For example, in the BIODICA package . The simplest idea is to select the variables  bypassing the threshold in p standard deviations, with some choice for p . A combined set of the most contributing to different ICs genes can be used to define a subset of data for further analysis.Frequently, each one-dimensional ka vector to the dates of sample preparation. Another component frequently identified in the analysis of transcriptomic data is related to GC-content, which might reflect the influence of GC-content on the RNA amplification step common for both microarray-based and sequencing-based methodologies. In Reference [Interestingly, ICA decomposition can be used to identify and disregard technical biases among omics datasets produced by different platforms. For example, in the study of 198 bladder cancers in Reference , one of eference , a smallInterestingly, ICA-based analysis sometimes can lead to identification of the factors whose origin is intermediate between technical and biological. For example, in Reference  one of tSome components identified through ICA could describe various cell populations present in the sample in addition to cells of direct interest. Typically, this was the case for the stroma-related signals in the ICA-based analysis of tumor bulk samples see . ICA canAnother frequently employed idea is to use the results of ICA decomposition in order to define a set of variables for further application of various machine learning methods. Zhang et al.  were amoRecently ICA was used to engineer features for further use in cancer-related classification tasks, using naïve Bayes classifier . In RefeAny matrix factorization method can be used for dimensionality reduction. The specifics of ICA are in that it is usually performed in an already reduced space and only defines a new coordinate basis in the principal linear manifold. Therefore, ICA itself does not reduce the data dimension more that it is done by PCA. Nevertheless, it is a frequent practice to consider the coordinate basis defined by few independent components as a subspace to further application of various data analyses. For example, this approach is used for a standard pipeline of single cell RNA-Seq data analysis . SimilarOne of the strongest motivations behind applications of ICA to omics data is in that it can help identifying functional subsystems  which are the building blocks determining response to perturbation of a tumoral cell or a whole cellular ecosystem such as tumor microenvironment (TME) composed of different cell types. The underlying principle is that genes or proteins do not react to an external stimulus individually but always integrate into a (sub-)system with more or less defined limits. Importantly, it is biologically feasible to assume the phenomenon of plurifunctionality, i.e., potential participation of an elementary entity (such as gene or protein) into several functional subsystems. ks vector (weights associated with the omics variables) or metagene. The level of activation (or inhibition) of an identified functional subsystem ks can be read in the corresponding metasample vector ka. The same is relevant for an independent component associated with a technical factor intensity.The composition of a functional subsystem is defined by a matrix factorization method in the form of the ks) to the definition of the subsystem, which can be positive or negative. However, those variables having close to zero weights can be neglected from the subsystem definition. An important characteristic of a metagene is the set of the most contributing genes (see discussion in the previous section). The most contributing genes are useful to characterize the functional subsystem and to identify if this subsystem corresponds to an existing known one. After determination of the sets of the most contributing genes per each metagene , one can check if a gene is associated with the subsystem exclusively or contributes to several ones. This analysis can be used to identify potential coupling between the subsystems and their concrete mechanisms (see further discussion). Sometimes it is convenient to distinguish two gene sets per metagene, having the largest and the smallest set of weights, from the positive and the negative sides of the ks distribution. If no explicit sparsity constraint is imposed when computing the vectors, then each omics variable  has a non-zero contribution  ,64. If aIdentification of functional subsystems (modules) from cancer omics data was first historically approached with hierarchical clustering of genes . Matrix Functional subsystems identified by ICA can reveal an important coupling of several known biological mechanisms and relate it to the biological phenotype such as cancer patient outcome. In the case of breast cancer, this phenomenon was described in Reference  through ka is the unimodal or bi- or multi-modal character of the distribution. In the case of well-defined bimodality of a metasample, one can stratify the distribution of samples into two groups, with respect to the nature of the functional subsystem identified. A typical example of this kind is the identification of the functional subsystem of proliferation in single-cell RNA-Seq data, where the corresponding metasamples frequently have two modes, corresponding to proliferative and non-proliferative cell states.One important characteristics of the weight distribution composing Functional subsystems have been systematically identified using ICA from a large cohort of transcriptomic profiles in Reference , where 2ks vectors were matched with each other through correlation in order to reveal the functional modules implicated in colon cancer tumor cells’ and the variability of tumoral microenvironment.Identification of the functional subsystems and distinguishing them from potentially technology driven factors can be strongly improved by the application of ICA analysis to multiple similar datasets independently (without merging them). In this scenario, the ICA results from several datasets were compared with each other in terms of the correlation or other suitable similarity measure among metagenes . In the In the case of a very good match between ICA-based metagene definitions from several independent datasets, one can define a consensus metagene definition from the meta-analysis (a meta-metagene). An exemplary set of such reference metagenes was built in Reference  and usedIn cancer biology, bulk omics data  represent heterogeneous samples such as peripheral blood mononuclear cell (PBMC) and tumor biopsies, in which the expression profiles of distinct cell types are mixed in each sample at a priori unknown proportions. The tumor microenvironment is composed of many different cells including a plethora of immune cells, stromal cells, and blood and lymphatic vessels . The quaTherefore, one of the major challenges for computational analysis of bulk samples is evaluating the proportions and the properties of individual cell types composing the sample, frequently called deconvolution problem in this context . In geneDeconvolution of cancer bulk transcriptomes gained a lot of popularity in the last several years due to the abundance of data sources. Several methods were proposed to estimate the abundance of immune infiltration in cancers at different levels of granularity ,68,69,70In response to this problem, reference-free  approaches propose a more data-driven way of performing the deconvolution. This group of approaches is able to discover the cell types and their markers as well as approximate profiles of those cell types (perform “complete deconvolution”). Different types of matrix factorization are suitable for solving this problem. Even though these deconvolution methods are called reference-free, known reference profiles are used to interpret and select the cell type-related components. Different possible benefits of reference-free approaches can be listed as (a) flexibility—discovering the context-dependent cell-type markers, (b) discovery of new cell types or cell types that are specific to a certain context, (c) determining deconvoluted profiles of cell types that can be used to remove the immune-related signal or to better understand the cell type features, and (d) ability to characterize biological processes (such as cell cycle activity) simultaneously with the cell types.The reference-free approaches were already applied for deconvolution of cell types in blood using semi-supervised NMF . They wehttps://github.com/UrszulaCzerwinska/DeconICA) was developed with the objective to apply ICA to the task of cell type deconvolution. It was shown that ICA is able to efficiently estimate the cell type proportions with better accuracy than leading supervised algorithms even though it can identify less cell sub-types than most of the published solutions. It suggests that ICA-based deconvolution is less prone to overfitting and enables discovery and quantification of strong and stable signals (not necessary the most abundant but rather the most specific). DeconICA was applied to a big corpus of data containing more than 100 transcriptomic datasets composed of over 28,000 samples of 40 tumor types generated by different technologies and processed independently. In addition, the ICA-derived metagenes were used as context-specific signatures in order to study the characteristics of immune cells in different tumor types. The analysis revealed a large diversity and plasticity of immune cells dependent and independent on tumor type. Some conclusions of the study can be helpful in the identification of new drug targets or biomarkers for immunotherapy of cancer. In Reference  icasso-sCell-type composition can be also computed from DNA methylation data. In the EWAS (Epigenome Wide Association Studies), the variation origination from cell types is considered as an important confounding factor that should be removed before comparing cases and controls and defining Differentially Methylated Positions (DMPs). For example, in Reference  ten toolStatistical properties of ICA seem to be very attractive to justify its application to the emerging wealth of single-cell omics data profiles. ICA can serve here to improve the data analysis regarding dimensionality reduction, removing technical biases, integrating datasets. ICA also looks promising and represents an alternative to the standard dimensionality reduction followed by clustering methodology for identifying cell types or states, suggesting a more continuous way of considering them, with a possibility of existence of intermediate or mixed cell populations.Similar to bulk RNA-seq, technical biases and batch effects are limiting factors for single-cell RNA-Seq and should be either removed or taken into account. One example of ICA application to normalize batch effect was recently reported by Dirkse et al. . The autThe ICA-based dimensionality reduction is a standard step in the most popular packages for analyzing single cell RNA-Seq data. In MONOCLE ,81, ICA In Reference , ICA wasICA served as a principal machine learning method for discovering functional subsystems involved in the response of Ewing sarcoma cells to the induction of the chimeric oncogenic transcription factor EWSR-FLI1 . In thisIn principle, ICA is the methodology able to exploit strong non-Gaussianity in the multidimensional distributions formed by single cells in the space of omics profiles. However, in order to optimally use this potential, one probably needs to identify the most suitable non-linearity functions, for each particular type of single cell measurements, and take into account the nature of the multivariate distribution of points in data space. Recently, a matrix factorization-based method ZINB-WaVE was adapted to the single cell RNA-Seq measurements, using the model of zero-inflated negative binomial distribution (ZINB) . In prinThe majority of published works on applying ICA in cancer research deals with transcriptomic data. This is connected in part to the relatively high abundance of such data type from collections of bulk tumors, and in part to the availability of bioinformatics tools helping to interpret the obtained components . Yet another aspect is that transcriptomic data are better connected so far to the clinical questions such as defining molecular subtypes of tumors. However, applying ICA should not be limited to only one level of omics profiling, and there is a lot of potential in applying it to several levels of molecular description. The multi-level datasets become increasingly available in the cancer biology. The levels of molecular description can be gene copy number profiles, binary mutation profiles, measured mutational signatures, measured total expression of genes or spliced mRNA isoforms or non-coding genes such as microRNAs, DNA methylation or histone mark modification profiling, protein or protein phosphoforms relative abundances or some other less frequently used omics types. Identification of functional subsystems can be facilitated through the use of several data types, since the adaptation process is frequently expected to span several levels. As a good example of such a multi-omics dataset, one can cite recent work on comprehensive characterization of medulloblastoma .ka vectors (metasamples). Such an approach was recently applied in Reference [Ideally, several levels of omics profiling should be collected for the same and sufficiently large set of samples. Independent components can be then computed for each data type separately and then the identified components can be compared by computing correlations between the corresponding eference  to a seteference , 77 breaAn alternative and somewhat more powerful idea consists in stacking several matrices corresponding to the different levels of omics profiling into a tensor , in order to apply the tensorial version of ICA. In this case, ICA will be able to learn and jointly optimize the signals which can involve variables from several levels of molecular description. This requires making at least two dimensions of the data common, while the third matrix dimension indicates the data type. Typically, all molecular measurements are mapped onto the genes through application of procedures that can be non-trivial . ijkX has dimensions “number of samples × number of genes × number of data types”. For example, Xi = 4, j = 5, k = 2 element in the tensor can indicate DNA methylation level of the promoter of the gene 5 in the sample 4. Xi = 4, j = 5, k = 1 could indicate expression of the same gene in the same sample. In the case of tensor factorization, the resulting components represent matrices rather than vectors having dimensions “number of genes versus number of data types” (for metagenes) and “number of data types versus number of samples” (for metasamples). The existence of correlations among different data types within the same matrix-component indicates coupling among several levels of molecular descriptions captured by tensorial ICA.The resulting three-dimensional measurement tensor Tensorial ICA was recently applied in Reference  to colonApplications of ICA to data types other than transcriptomic or to several data types simultaneously remain limited; however, first applications of this approach in cancer biology are rather promising ,92. MultFunctional subsystems identified through ICA and fixed in the form of metagenes can be studied for their statistical relationships within a dataset, among datasets of the same kind, or among datasets that are not closely related in terms of the nature of the biological samples profiled. ka vectors. This analysis confirmed previously identified comorbidity patterns based on the analysis of individual gene expression profiles  and suggested new molecular mechanisms of comorbidity between lung cancer and Alzheimer’s disease such as estrogen receptor signaling pathway or the involvement of cadherins. In the latter case, one can use ICA for studying disease–disease relationships. An example of such a relation is the phenomenon of inverse comorbidity between cancer and some other diseases, in terms of the anti-correlated activation pattern of the common functional subsystems. For example, the ICA method was exploited in Reference  in orderindependent component splitting and merging might depend on the order of ICA decomposition , on theThe theoretical principles of functional subsystems integration have been developed ,64. HoweIn recent decades, independent component analysis has become a standard tool for the analysis of tables of omics measurements in cancer biology. In certain applications, it was shown to have advantages, especially in terms of reproducibility or generalizability and biological interpretability, compared to other popular matrix factorization methods. Despite ICA being shown to be a useful tool, it seems to be under-appreciated partly due to the fact of historical reasons and partly due to the presence of existing confusions in the underlying assumptions and/or interpretation of the resulting matrix decompositions. For example, it is frequently commented that biological processes are not perfectly independent and that they are expected to be correlated in some conditions. Even though this is true, ICA can distinguish signals coupled to some extent by making the corresponding components as independent as possible. In this review, we made a comprehensive effort to mention most of the recent studies in cancer research where ICA served as the essential data analysis tool. We classified them into several common topics: data preprocessing, data dimension reduction and visualization, identification of functional subsystems and their correlations or interactions, deconvolution of cell types, data integration and meta-analysis. We also reviewed the methodological works aimed at defining the best practices of applying ICA to concrete types of omics data. Compared to the early times of applying ICA to omics datasets, today there exists a variety of implementations and improved methodologies allowing us to use the valuable idea behind ICA  in the best possible way. Certain progress has been made in clarifying such important questions such as determining the optimal number of components to retain or establishing the biological significance of the extracted components.Most of the existing applications of ICA have been done so far for transcriptomic data even if the interpretation of the components used other types of molecular data . Since recently, ICA started to be applied to other types of omics profiles, including methylation profiles and proteomics datasets. It seems interesting to determine, using ICA, independent sources of variance in newly emerging omics data types, such as systematic Chip-Seq datasets mapping the state of histone modifications or mutational signature profiles. More experience, standardization and assessment are required to use ICA in the most optimal way for the analysis of single-cell and multi-omics datasets. Moreover, in this review we did not even mention other fields of ICA application in cancer biology, including the analysis of imaging data , clinicMatrix factorization represents an alternative approach to the standard clustering methods, being more flexible in terms of taking into account gene plurifunctionality and ability for unsupervised deconvolution of factors whose activity can be correlated. It is worth noticing that some ICA algorithms (such as fastICA) are computationally performant when properly implemented and potentially able to deal with large amounts of molecular measurements. In this sense, ICA remains competitive vis-a-vis many other approaches .We believe there are interesting directions to further explore and more deeply use the concepts behind ICA in the context of cancer biology data analyses. It would be interesting to reconsider the roots of independent component analysis into artificial neural network methodology, suggesting novel scalable autoencoding-based techniques in order to solve the problem of blind source separation adapted to the nature of the biological data. Assessing the value of supervised learning of the features extracted by ICA from omics and other data types and comparing them to “hand-crafted” or convolutional neural network-based features can lead to designing performant hybrid learning approaches, as in Reference . It appeTo conclude, as a team of authors all having extensive experience in applying independent component analysis as a tool in computational cancer biology, we advocate for its wider use in making sense of the growing amount of omics data in this and other fields.


C7, MMP3, KRT14, LOC642587, CASP7, S100A7 and miRNAs hsa-mir-205 and hsa-mir-203b as the key genomic features that may substantially contribute to the oncogenesis of melanoma. Our study also proposes genes ESM1, NFATC3, C7orf4, CDK14, ZNF827, and ZSWIM7 as novel putative markers for cutaneous melanoma metastasis. The major prediction models and analysis modules to predict metastatic and primary tumor samples of SKCM are available from a webserver, CancerSPP (http://webs.iiitd.edu.in/raghava/cancerspp/).The metastatic Skin Cutaneous Melanoma (SKCM) has been associated with diminished survival rates and high mortality rates worldwide. Thus, segregating metastatic melanoma from the primary tumors is crucial to employ an optimal therapeutic strategy for the prolonged survival of patients. The SKCM mRNA, miRNA and methylation data of TCGA is comprehensively analysed to recognize key genomic features that can segregate metastatic and primary tumors. Further, machine learning models have been developed using selected features to distinguish the same. The Support Vector Classification with Weight (SVC-W) model developed using the expression of 17 mRNAs achieved Area under the Receiver Operating Characteristic (AUROC) curve of 0.95 and an accuracy of 89.47% on an independent validation dataset. This study reveals the genes  As per the American Cancer Society, there is an estimation of 96,480 melanoma related new cases and 7,230 deaths in 2019 in the US. Melanoma is more prominent in males as compared to females2. The malignant transformation of normal human epithelial melanocytes, located within the basement membrane of the skin results in melanoma development. There are several genetic3 and environmental factors such as excessive exposure of UV radiations, indoor tanning devices and contacts with certain chemicals like arsenic and hydrocarbons, etc. that contribute to melanoma carcinogenesis4.Cancer is one of the major causes of mortality worldwide since the last few decades. According to GLOBOCAN, 2018, 18.1 million new cancer cases and 9.6 million deaths have been estimated worldwide. The melanoma contributes 1.6% of the new cancer cases and 0.6% of deaths due to cancer worldwide5, which can be explored for the identification of diagnostic and prognostic cancer biomarkers6. The Cancer Genome Atlas (TCGA) is one of the prominent and inclusive repository containing genomic, transcriptomic, epigenetic, proteomics and clinical information of 33 types of cancer7. The core study on SKCM done by TCGA has revealed four subtypes of cancer, which include mutant BRAF, mutant RAS, mutant NF1, triple WT (wild-type) based on mutant genes. The triple WT SKCM subtype mainly exhibits KIT mutations, focal amplifications and structural rearrangements. Further, it has been observed that the mutational rate of these genes is much higher in melanoma patients than other cancer types of TCGA9. Interestingly, over 50% of melanoma patients have BRAF kinase  mutations10. In addition, various studies have demonstrated that the SKCM arises from the anomalies in transcriptomic and epigenetic factors such as expression of mRNAs, miRNAs, the aberration in methylation patterns of CpG islands of genes and histone modifications, which paves the way for the development of potential molecular biomarkers in melanoma23. In the past, several reports have revealed the potential role of miRNA expression as prognostic biomarkers in cutaneous melanoma. For instance, miR205 and miR29c both act as tumor suppressors and down-regulate the expression of E2F1, E2F524 and DNMT315 genes, respectively. Besides miRNAs, histone methyltransferases also act as crucial players in the progression of melanoma by enhancing the expression of enhancer of zeste homolog 2 (EZH2)25.Recently, with the advancement of genomic technologies, there is a huge increment in the generation of big multi-omics data, particularly in the field of cancer32. The metastasis mechanism involves several pathways including epithelial-mesenchymal transition (EMT), angiogenesis and invasion. Furthermore, the aggressive stage of melanoma can metastasize to lymph nodes, distinct tissues and organs33. Although the survival of cutaneous melanoma patients is affected by various factors, the disease’s early stage diagnosis is one of the most vital parameters with the greatest impact on survival. Different studies have shown the metastasis-free malignant melanoma patients, i.e. patients with primary tumor have significantly prolonged survival35. Evidently, the five-year relative survival rate of melanoma patients is 23%, 64% and 98% for distinct, regional stage and localized tumors, respectively2. Hence, the detection of tumor at a localized stage, i.e. primary tumor is crucial for patient management and implementation of an appropriate therapeutic strategy for prolonged survival of patients. The genomic and epigenomic biomarkers that can detect primary tumor with high precision might prove to be a boon in this regard and can eventually result in the better outcome of the patients with personalized treatment.Earlier studies have scrutinized the distinctions between primary melanoma and metastatic melanoma37. Recently, one study has predicted the metastatic progression score for the assignment of metastatic and primary melanoma based on key miRNA and mRNA expression based putative biomarkers38. Although, all the metastatic samples were correctly assigned to a metastatic category based on metastatic progression score. But, the lack of gold standard performance measures like sensitivity, specificity and AUROC, absence of the performance on independent validation dataset and the unavailability of any web-service to analyse new data based on those identified markers are the major lacunae. Hence, the current study is designated to overcome these inadequacies.Previously, several stochastic stage wise prediction and classification methods have been developed for diverse cancer typesIn this analysis, we have made an effort to understand the cutaneous skin melanoma progression based on multi-omics layers of data in TCGA that comprises of RNAseq, miRNAseq and methylation expression. Through state-of-the-art machine learning-based feature selection techniques, we have identified genomic signatures that can categorize both primary and metastatic samples with high accuracy. Subsequently, prediction models were developed based on these key identified genomic features using several supervised machine learning techniques that can segregate primary and metastasized SKCM patients.40, Support vector machines with L1 regularization (SVC-L1)42 and Principal Component Analysis (PCA)43. These methods have been previously used in various studies49. Subsequently, prediction models have been developed implementing several machine learning techniques like ExtraTrees50, KNN, Random forest51, Logistic Regression (LR)52, Ridge classifier53 and SVC - RBF kernel with class weight factor employing scikit package54 (described in Methods). The pipeline depicting the workflow of this study is shown in Fig. In the current study, we have analysed the RNAseq, miRNAseq, methylation-seq data of SKCM from TCGA for 466, 444, 466 patients, respectively. To mine important genomic and epigenomic features which can discriminate various degree of metastatic tumors P2, M1 and M2) from the primary tumors (P1), we used well established feature selection methods like WEKA-FCBF and M2 fWith an aim to classify the metastatic and primary tumor samples with high precision, first the RNAseq expression data of 466 patients consisting of 20,502 genes was used to select the relevant features using three feature selection methods; SVC-L1, WEKA-FCBF and PCA. Primarily, we obtained nearly 150 and 17 features using WEKA-FCBF and SVC-L1, respectively. Further, we applied six different machine learning algorithms on the selected features obtained using the above three methods. As shown in Table Interestingly, a model based on 150 features selected using WEKA-FCBF also attained almost similar performance , while KRT17 and KRT14 are part of intermediate filament component . It has been shown that metastatic cancer cells use actin bundles to disrupt from a primary tumour and invade the surrounding tissue. After travelling in the vasculature or lymphatic system, they exit into a new niche and form a new tumour55.The enrichment analysis of the 17 features shows the biological role of the mRNA signature in melanoma carcinogenesis. Out of 17 genes, As we analysed the new tumour event (NTE) clinical file of SKCM patients, we observed that 16 patients with primary tumor have been shown to be in distant metastasis with new tumour events. Therefore, we removed these 16 samples from the dataset and again developed the classification model. There was a marginal increase of MCC from 0.73 to 0.77 and alike AUROC on validation dataset  implemented using the RtsneThe primary tumors (P1) are localised lesions and P2 includes the samples with-in-transit metastasis and satellite metastasis which represent intra-lymphatic tumour. At this stage, tumor has not still spread to lymphatic nodes. We selected 10 features using SVC-L1  and the results of classification models on these 10 features show that it is difficult to classify the samples with intra-lymphatic tumour (P2) from the primary tumour (P1). The KNN-based model correctly identified 84.75% (Sensitivity) of metastatic and 93.83% (Specificity) of primary tumor patients of training data with MCC of 0.79 and AUROC of 0.96. On validation data, this model identified 73.33% (Sensitivity) of metastatic and 85.71% (Specificity) primary patients correctly with MCC 0.60 and AUROC of 0.84  from the primary tumors (P1). Our analysis shows that these tumors can be classified with high precision. The SVC-W based model using mRNA expression of 12 genes  from primary tumors (P1). Surprisingly the classification of these two groups of samples is not as good as lymphatic node v/s primary on 5 features  and regional tumors, we combined primary (P1) and in transit and satellite tumors (P2). The LR model using 14 features  from the tumors which were localized (P1 and P2). The Logistic Regression (LR) model based on 15 features  metastatic samples and 90.79% (specificity) primary samples of training dataset and 90.14% metastatic samples and 78.95% primary samples of validation dataset were correctly predicted Table .Table 3P58) such as ZEB2, ZEB1, ERBB3, PRKC, ERBB2, E2F1, BCL2, ITGA5, VEGFA, AR, SMAD4, EGLN2, LAMC1, VEGFA, SMAD1, SRC, VEGFA, DDX5 and YES1, etc. Gene enrichment analysis have shown that these genes are significantly enriched  in various cell growth promoting and oncogenesis associated pathways including transcriptional misregulation, TGF-beta signaling, wnt signaling, PDGF signaling, EGFR signaling, PI3K signaling, p53 signaling, ErbB signaling, VEGF signaling, cell cycle, hypoxia and angiogenesis, apoptosis processes, etc. This analysis signifies the role of hsa-mir-205 as tumor suppressor in melanoma development as it gets downregulated with the progression of metastatic melanoma.Among the miRNA signatures, hsa-mir-205 targets various genes . We used SVC-L1 method here as this feature selection method has shown reasonably higher performance with a minimum number of features in comparison to WEKA-FCBF and PCA in the previous analyses. The 20 features  value of the feature (if feature is upregulated in metastatic) is higher than a threshold value, otherwise, it is classified as a primary sample. In these models, the threshold is varied incrementally from minimum to maximum RSEM value. Finally, that threshold is selected, which have the maximum AUROC in classifying metastatic and primary tumor samples. Consequently all the mRNA and miRNA sites are ranked on the basis of maximum AUROC and MCC with the minimum difference in sensitivity and specificity to assess the ability of each feature to classify metastatic and primary samples , respectively. Among them, C7 is upregulated in metastatic samples, while rest of the observed genes are downregulated in metastatic samples.The hsa-mir-205 and hsa-mir-203b are the top 2 miRNAs that can classify the metastatic and primary tumor samples with AUROC 0.83 and 0.75 at thresholds 4.3 and 1 , respectively. Both of these miRNAs are downregulated in metastatic samples, which indicates their potential role as tumor suppressors. The We further subdivided the heterogenous P1 subgroup according to the SKCM tumor stage. Of the total 103 samples of primary SKCM, the tumor stage information is available for 98 patients with gene expression data and for 96 patients with miRNA expression data.We have used both WEKA-FCBF and SVC-L1 based feature selection method (described in methods) to extract the important gene expression based features which could discriminate the early stage and late stage primary tumors. The WEKA-FCBF based method resulted in fewer features and better performance. Due to availability of a lesser number of samples (less than 100) we have used leave-one-out cross-validation technique to develop the prediction model using selected 37  and has also exhibited to be an important therapeutic target in melanoma62. Other miRNAs such as has-let-7f-1 has been implicated in lung cancer and renal cancer64, while hsa-mir-219a-1 in renal cancer65.Next, miRNA expression was explored to distinguish between the early stage and late stage primary SKCM samples. Using 32 miRNA . CancerSPP is designed for the prediction and analysis of metastatic and primary tumor of SKCM from RNAseq, miRNA and methylation expression data. The web server has two modules; Prediction module and Data analysis module.i.e. Intra-lymphatic tumors ((P2) v/s Primary tumor (P1), Lymphatic tumors (M1) v/s Primary tumors (P1), Distant Metastatic tumors (M2) v/s Primary tumors (P1), Regional (P1P2) v/s Lymphatic tumors (M1) and Metastatic tumors (M1M2) v/s Regional tumor (P1P2) utilizing RSEM expression quantification values of signature genes. The user needs to submit the RSEM value of signature genes for every melanoma patient. In the input file, the number of patients represents the number of columns in the file. The output file contains the prediction outcome with a score. Greater the score, higher is the probability of correct prediction.This module permits the users to predict different states of metastatic samples and primary tumor samples This module is used to evaluate the role of each gene in various melanoma states such as regional metastatic, lymph node metastatic and distinct metastatic vs primary tumors based on mRNA and miRNA expression profiles. Moreover, it also incorporates threshold-based MCC of each feature and mean expression values for the RNAseq expression data in the primary and metastatic state of SKCM.67. Previous studies concerning SKCM have focussed on determining its sub types9 and survival31. Li et al. made an attempt to predict metastatic progression of melanoma tumor samples and predicted metastatic progression scores using mRNA and miRNA expressions individually; based on which they assigned primary and metastatic samples to primary and metastatic groups. Further, they also found a correlation between clinical characteristics of samples, i.e. Clark’s level and lymph node status with metastatic progression score. Although all of metastatic samples were correctly assigned to the metastatic group; but, many of primary samples were incorrectly assigned to the metastatic group based on the metastatic progression score. They reported that the proportion of runs where a primary tumor specimen was classified as metastatic among the 10,000 runs was also highly non-uniform38. But, they did not report the performance of their method in terms of standard parameters  as well as standard cross-validation techniques have not been implemented . Additionally, the classification models were not available to the public. Thus, the current study is an attempt to overcome these inadequacies.There is an emergence of synergized clinical and molecular profiles of cancer samples that can aid in predictive modelling for early tumor detection and progression. This prediction helps the physicians in making a suitable decision about the treatment courseThe present study is an effort for the identification of genomic signatures that can classify both metastatic and primary samples with high precision based on mRNA, miRNA and methylation data. Further, our aim is to validate the performance of our prediction model on an independent dataset in addition to 10-fold internal cross validation. Additionally, we have also identified signatures that can further categorize different types of metastatic states, i.e. intra-lymphatic tumor, lymphatic tumor and distant metastatic tumor samples from the primary tumor samples. Furthermore, we also developed a web server to predict and analyse new data based on those identified markers.C7 gene has shown to be a potential tumor silencer gene and its expression is highly downregulated in various carcinomas such as ovarian cancer and non-small cell lung cancer (NSCLC)68. In current study, this gene alone can correctly predict 83.79% metastatic samples with MCC of 0.56 and AUROC of 0.81. Another gene MMP3 has been reported to acts as melanoma suppressor gene69 and is observed repeatedly in different signatures that distinguish various states of metastatic tumors from primary tumors in our analysis. Further, KRT14, a keratin gene, has shown to be downregulated in case of skin cancer70. In the present study, it classified metastatic and primary samples with high sensitivity of 94.14% and low specificity of 56.79% with overall the MCC of 0.56. Notably, the role of 11 out of 17 mRNA features have been previously reported in literature for cutaneous melanoma; while 6 genes including ESM1, NFATC3, C7orf4, CDK14, ZNF827, and ZSWIM7 have been described for other cancers and other melanoma types like uveal melanoma but have not been specifically described for cutaneous melanoma72. Thus, the current study revealed the potential role of these six genes in the classification of the metastatic and primary tumor samples of SKCM for the first time. Earlier, the role of 8 out of 11 genes that include C7, MMP3, KRT14, KRT17, MASP1, S100A7A, MUC21, and DNAJC5B was previously reported in the metastatic progression of SKCM patients by Li et al.38. Martins et al. observed that one of the genes from our 17-gene signature, i.e. C10orf12 gets upregulated with the loss of ColVII in squamous cell carcinoma model73. CLIC5 which get upregulated in metastatic samples, has been previously shown to be methylated in one of 13 melanoma cell line74, while another study elucidated FILIP1L as a potential antivascular target for cancer therapy in melanoma model75.In this study, we have identified discriminative genomic features using different feature selection methods and their classification prediction potential elucidated implementing various machine learning algorithms in the segregation of metastatic from primary tumor samples. Our analysis shows that the mRNA expression profile is the strongest predictor of metastasis as compared to miRNA expression and methylation profile. In the current study, the SVC-W model based on the expression of 17 mRNAs is the best performer in discriminating metastatic from primary tumors with overall accuracy of 89.47%, MCC of 0.73 and AUROC 0.95 on independent validation datasets  of 4.3. The expression of hsa-mir-205 among the miRNAs has shown to be downregulated in various solid tumorsWe also developed models combining different omic layers at the feature level (Combo model) and at the model level (ensemble model). Their performance was either less or comparable to the model based on 17 mRNA expression features. Further, we subdivided the heterogeneous group of primary tumors as per the tumor stage and segregated early stage and late stage samples. Based on 37 features, the random forest model segregated these samples with 95.52% sensitivity and 83.87% specificity with MCC of 0.81 and AUROC of 0.96  IDs of patients/subjects. Finally, we obtained 466 patients  for mRNA and methylation data, whereas 444 samples were available  for miRNA expression data. We referred primary tumors, Regional Cutaneous or Subcutaneous Tissue (includes satellite and in-transit metastasis), Regional Lymph Node and Distant Metastasis samples as P1, P2, M1 and M2, respectively. The clinical characteristics of these patients displayed in Supplementary Fig. The RNAseq, miRNAseq and methylation profiling data for SKCM was retrieved from TCGA project using TCGA - Assembler 2 version83. Here, the methylation value for each gene represents the average of methylation beta values of all CpG sites located on each individual gene.In the present study, we have used mRNA and miRNA expression profiles in terms of RSEM values for 20,502 genes and 1,870 miRNAs, respectively. We downloaded the methylation profiles for 20,879 genes . Notable, we downloaded all CpG sites for each gene, (DNase hypersensitive and non-DNase hypersensitive). The data is in the form of Beta values, a quantitative measure of DNAFurther to study the Primary tumors stage-wise analysis using gene expression data, we segregated 67 stage-1 and stage-2 primary SKCM tumors as early stage and 31 stage-3 and stage-4 primary SKCM tumors as late stage SKCM tumors. In case of miRNA, 61 early stage and 30 late stage primary SKCM tumors are available for the analysis.84, followed by z-score normalization of data. Thus, log2-transformed RSEM values for each mRNA and miRNA were centred and scaled by employing caret package in R. Following equations were used for computing the transformation and normalization:Z_score is the normalized score, x is the log-transformed expression, sd is the standard deviation of expression.Z-score Scaling: It has been observed that there is a wide range of variation in RSEM values of mRNAs and miRNAs. Thus, we transformed these values using log2 after addition of 1.0 as a constant number to each of RSEM value. Further, features with low variance were excluded from the data using caret package in RMin-Max normalization: When we combined all the features from the three omics layers, the RNAseq expression, miRNA expression and methylation profiling data for feature selection in combo model, the Min-Max normalization method in R was employed using range option of preProcess from caret package. This ensured that all three types of features were in the same range of 0 and 1. The validation dataset was transformed in accordance with the training data using predict function in caret.It was observed that in some of the patients, mRNA expression available for both tissue and blood samples. Here we took the average of both the samples for each patient.49. In this study, we implemented three techniques, i.e. SVC with L1 penalty employing Scikit package54, ‘SymmetricalUncertAttributeSetEval’ with search method of ‘FCBFSearch’ of WEKA software package85 and PCA in R. We filtered genes (mRNA expression), methylation pattern of genes and miRNA expression as features that can distinguish metastatic samples from primary tumor samples using these techniques. The FCBF (Fast Correlation-Based Feature) algorithm employed correlation to identify relevant features in high-dimensional datasets in small feature space39. SVC-L1 method selects the non-zero coefficients and then applies L1 penalty to select relevant features to reduce dimensions of the data. For feature selection using PCA, we selected those Principal Components that represented at least 1% variance of the data.One of the challenges in developing the prediction model is to extract important features from the large dimension of features. Although, there are a number of methods for feature selection, we have used only those methods, which are well established and previously implemented in similar types of studies86 for each subset and finally calculated the overall stability index. For all the signatures the average stability index is nearly in the range of 0.40 to 0.43.To select the robust features, first, the data was split into the ratio of 80:20 for 10 times followed by features selection using SVC-L1 or WEKA-FCBF on each occasion from the training dataset. From this resampling process, we obtained 10 sub-sets of features. We have selected the subset having the highest performance. To check the robustness of features, we computed the average stability index (Jaccard index) using OmicsMarkeR packagei.e. Intra-lymphatic tumors ((P2) v/s primary tumor (P1), lymphatic tumors (M1) v/s primary tumors (P1), distant metastatic tumors (M2) v/s primary tumors (P1), regional (P1P2) v/s lymphatic tumors (M1) and metastatic tumors (M1M2) v/s regional tumor (P1P2).Firstly, we have developed the prediction models to categorize primary tumor and metastatic samples based on selected genomic features using various classifiers implementing Scikit package. These classifiers include ExtraTrees, KNN, Random forest, Logistic Regression (LR), Ridge classifier and SVC - RBF kernel with class weight factor were implemented employing scikit package. In addition, to understand the progression of skin cancer from primary tumor to metastasis, we also analyse and develop prediction models using various machine learning classifiers of scikit package based on genomic features (mRNA expression) to classify the sub-categories of metastatic samples from primary tumor samples The optimization of the parameters for the various classifiers was done by using a grid search with area under PR  curve as scoring performance measure for selecting the best parameter as our data was imbalanced.87.After applying supervised learning to classify samples, we visualised the distribution of samples based on selected features on reduced dimensions using t-SNE methods implementing the two R Packages; Rtsne and scatterplot3d packages. t-SNE is a non-linear dimensionality reduction algorithm employed to analyze the high-dimensional data. It converts multi-dimensional data to two or more dimensionsHere, we employed AUROC and MCC based feature selection technique to identify important features and developed single feature-based prediction models to distinguish metastatic samples from primary tumor samples. Single feature based models are also called threshold based models in which feature having a score below a specific threshold is assigned to metastatic tumor if it is downregulated in metastatic tumor samples otherwise it as primary tumor sample and vice versa. We computed performance of each given feature and identified features having the highest performance in terms of AUROC, MCC with minimum difference in sensitivity and specificity. Additionally, we have also computed their mean difference, log fold change, Bonferroni adjusted p-value using Wilcoxon test in R.89. Therefore, we implemented this standard protocol and subdivided our dataset into two subsets, i.e. training dataset and independent validation dataset in ratio of 80:20. We used 80% of the main dataset for training and remaining 20% for independent validation. First, the training dataset is used for developing model and for performing ten-fold cross-validation as internal validation. In this ten-fold-cross validation technique, training dataset is randomly split into ten sets; of which nine out of ten sets are used as training sets and the remaining tenth set as testing dataset. This process is repeated ten times in such a way that each set is exploited once for testing. The final performance of the trained model is the mean performance of all the ten sets.In the present study, both internal and independent validation techniques were employed to evaluate the performance of models. Previously, different studies employed the 80:20 ratio for the partitioning of a dataset into training and validation dataset90.In order to avoid the over-optimization of parameters in ten-fold cross-validation, we have also implemented independent validation. In the case of independent validation, we evaluated our model on an independent dataset, which was kept aside and remained unseen during feature selection and training or development of the modelIn order to measure the performance of models, we used standard parameters. Both threshold-dependent and threshold-independent parameters were employed to measure the performance. In case of threshold-dependent parameters, we measured sensitivity, specificity, accuracy and Matthews correlation coefficient (MCC) using the following equations.While, for threshold-independent measures, we used standard parameter AUROC. The AUROC curve is generated by plotting sensitivity or true positive rate against the false positive rate (1-specificity) at various thresholds. Finally, the area under ROC curve was calculated to compute a single parameter called AUROC. Notably, we have obtained classification performance in terms of sensitivity, specificity, accuracy, MCC, AUROC on various thresholds of prediction score for each of prediction models. We have selected only those threshold dependent measures based on the threshold of prediction score that gives maximum accuracy along with the minimum difference between sensitivity and specificity.91. Enrichr executes the Fisher exact test to identify enrichment score. It provides Z-score and the adjusted p-value which is derived by applying correction on the Fisher Exact test. Further, to understand the biological impact of miRNAs in metastatic melanoma development, we employed miRTarBase to identify target genes of signature miRNA.In order to discern the biological relevance of the signature genes, enrichment analysis was performed using EnrichrSupplementary Information


We developed an unsupervised deep learning method, stacked denoising autoencoders (SdA), by integrating multi-platform genomic data for subtyping ccRCC with the goal of assisting diagnosis, personalized treatments and prognosis. We successfully found two subtypes of ccRCC using five genomics datasets for Kidney Renal Clear Cell Carcinoma (KIRC) from The Cancer Genome Atlas (TCGA). Correlation analysis between the last reconstructed input and the original input data showed that all the five types of genomic data positively contribute to the identification of the subtypes. The first subtype of patients had significantly lower survival probability, higher grade on neoplasm histology and higher stage on pathology than the other subtype of patients. Furthermore, we identified a set of genes, proteins and miRNAs that were differential expressed (DE) between the two subtypes. The function annotation of the DE genes from pathway analysis matches the clinical features. Importantly, we applied the model learned from KIRC as a pre-trained model to two independent datasets from TCGA, Lung Adenocarcinoma (LUAD) dataset and Low Grade Glioma (LGG), and the model stratified the LUAD and LGG patients into clinical associated subtypes. The successful application of our method to independent groups of patients supports that the SdA method and the model learned from KIRC are effective on subtyping cancer patients and most likely can be used on other similar tasks. We supplied the source code and the models to assist similar studies at  Tailoring treatments to specific groups of patients becomes very popular in recent years due to the confirmation that the fundamental biology of each patient is unique, such as DNA, RNA, protein and methylation. Recent studies have shown that tumor is a highly heterogeneous tissue. With progression, the tumor becomes more and more divergent, which is a leading challenge to the treatments. Therefore, stratifying the patient is an initial and important step for developing personalized treatments.1. There are three major histologic subtypes of RCC from a clinical viewpoint2: clear cell renal cell carcinoma, papillary RCC, and chromophobic RCC. ccRCC is not only the most common subtype, account for 80–90% of all RCC cases2, but also the major cause of the deaths from kidney cancer1. Recent studies demonstrated that ccRCC is highly heterogeneous either intra-tumor or inter-tumor4. ccRCC is classified into four grades (1–4)5 according to Fuhrman grading system. Grade 1 and 2 can be classified as low grade, and grade 3 and grade 4 are high grades but behaves differently on 5-year cancer-specific survival5. ccRCC also has four stages (1–4) according to American Joint Committee on Cancer staging6. The tumor at Stage I and II is still local. At stage III, the tumor extends to major veins or spreads to the perinephric tissues but is not beyond Gerota’s fascia. At Stage IV, the tumor migrates to distant metastasis. According to the staging system, different treatments were suggested for different stages of cancer. For localized cancer, partial nephrectomy approach is generally higher recommended than radical Nephrectomy2. When tumor migrates to other locations, systematic therapy is recommended2. In recent years, increased genomic and molecular studies on ccRCC greatly aided the diagnosis and treatments. Such as, the target therapy, mTOR pathway inhibitor, is benefit from genetics studies8. However, most of these studies focused on genetics variants discovery or identifying biomarkers for subtypes of RCC. In our study, we developed a new method that can integrate multi-platform genomic datasets to subtype ccRCC.RCC is one of the top 10 most common cancers in the world9. Except the COC method, Bayesian method10 and non-negative matrix factorization method11 have also been developed to integrate datasets from multi-platform to do clustering. However, few of them are designed to handle very complicated multi-modality datasets from both intra-platform and inter-platform. In recent years, several deep learning methods have been developed to solve the difficulties on analyzing the complicated multi-platform datasets13. All of them are based on stacked Restricted Boltzmann Machines (sRBM), a generative algorithm, which uses RBM to capture the features of the data from each platform first and then join the latent variables from each platform to generate the common features of the cross-platform datasets. Authors showed that the sRBM successfully grouped the patients into clinically relevant subtypes. The successful application of the sRBM method on cancer patient classification inspired us to develop another deep learning method, autoencoder, a discriminative model, for cancer patient classification. Since a generative algorithm is unlikely to accurately model complicated datasets, a discriminative algorithm may be better on classification for complicated datasets.With the fast increase of huge amount of multi-platform datasets, more and more computational methods have been developed to identify the characteristics of each cancer patient. However, most methods are applied on one type of data or study each type of data separately. K-means clustering and Principal Component Analysis (PCA) are commonly used to classify the genomic data. Although both methods work well on one type of datasets, they have not been applied directly to cross-platform datasets. The TCGA network developed an algorithm called Cluster of Cluster (COC), re-clustering the clusters by converting the initial clusters computed from each platform into binary vectorshttps://github.com/tjgu/cancer_subtyping.In this work, we developed a stacked denoising autoencoder (SdA) to integrate gene expression, protein expression, microRNA (miRNA) expression, methylation and CNA datasets to subdivide the KIRC patients. We successfully divided the patients into two groups, in one of which had significantly higher survival probability than the other. We further identified a set of genes, miRNAs and proteins from differential analysis between the two groups, which can be potential biomarkers for diagnosis and developing new proper treatments. In comparison with K-means clustering method, the results showed that the SdA method produced a much clearer separation of the patients based on the clinical output. We also compared with stacked of Restricted Boltzmann Machines (sRBM) method and obtained consistent results. To further validate our model and show its generalization, we applied the learned KIRC model as a pre-trained model to independent datasets, LUAD and LGG datasets, and classified LUAD and LGG into clinical associated subtypes. Using the learned model as a pre-trained model can save the computing time on selecting the best model parameters and can improve the model from learning directly from the original dataset. We supplied both the learned models and the source codes for usages on similar tasks at There are much less miRNAs 1870) and proteins (189) than genes (20530), methylations (25062) and CNAs (9176). Therefore, we used one hidden layer with 50 units for miRNA and protein datasets, but two hidden layers with 500 and 50 units separately for gene expression, methylation and CNA datasets. We named the last hidden layer for each dataset as the Last Individual Hidden Layer (LIHL) Fig. . We comb870 and pWe did Pearson correlation analysis between the reconstructed input from the LIHL with each input dataset. The training datasets and the majority of the test datasets have correlations higher than 0.5 although the training datasets had higher correlations than the test datasets Fig. , demonstWe also did Pearson correlation analysis between the reconstructed input from the LJHL with each original input. All the correlations were positive with the lowest at 0.029 for miRNAs in the test dataset, the highest at 0.281 for methylations in the training dataset Fig. . The resFinally, similar correlation analysis was done on the whole dataset (combining the training and test datasets), more consistent results across the five datasets were observed both with LJHL and LIHL Fig. , indicatIn our work, we cannot assess the classification results based on the labels since we do not have the labels for the patients. Nevertheless, the goal of classifying patients is to identify the genomic features to assist the diagnosis and personalized treatments. Therefore, the better we can classify the patients into clearer clinical associated groups, the better the method works. We used this criterion to evaluate different methods on classification. And three clinical associated features were used in our study: the patient survival time, neoplasm histologic grade and pathologic stage. As described in the introduction, there are generally four neoplasm histologic grades and four pathologic stages for tumors, and the lower grade and the lower stage of the tumor is closer to a normal tissue. Grade 4 (G4) is the highest grade that the tumor has the fastest growth and spreading. The tumor at Stage II is larger than Stage I, but is still local. At Stage III, tumor has grown into the vein and may be lymph node. Stage IV is the highest stage that the tumor spreads into distant parts of the body.The probability of survival for the two KIRC group patients is significantly different  Fig. . We obse14. The significantly DE genes were called at thresholds of FDR 0.05 and fold change >2. A total of 634 DE genes were identified and most of them  were down regulated in the KIRC_SdA_G2 patients (Supplementary Table https://reactome.org/). Twenty terms were found at threshold of FDR less than 0.1 18. Several MMPs were in the DE gene set: MMP1, MMP7, MMP9 and MMP17. Two of them, MMP7 and MMP9, were reported that higher expression of the two genes with poor prognosis of ccRCC20, supporting our results that higher expression of the two genes with poor survival probability, higher neoplasm histologic grade and pathologic stage. The results from the pathway analysis were consistent with the clinical features of the two groups. In summary, our results implied that gene expression contributes to stratify the patients and the DE genes are potential biomarkers for separating the patients and are potential targets for developing appropriate treatments.Differential expression analysis was conducted between the two groups of patients using edgeR14 and 24 DE miRNAs were found at thresholds of FDR at 0.05 and fold change >2 . SERPINE1 was more than two-fold higher expressed in the KIRC_SdA_G1 patients, which had lower probability of survival and higher neoplasm histologic grade and pathologic stage. Our results were supported by the data in Human Protein Atlas database that higher expression of SERPINE1 is associated with lower probability of survival in multiple cancers: urothelial cancer, stomach cancer, lung cancer, head and neck cancer, colorectal cancer and cervical cancer, demonstrating that SERPINE1 is a high confident protein biomarker (https://www.proteinatlas.org/ENSG00000106366-SERPINE1/pathology).T-test was employed for the protein DE analysis and the significantly DE proteins were called at thresholds of FDR at 0.05 and fold change >2. One DE protein was found, SERPINE1, which is a member of the serine protease inhibitor sup-family that negatively regulates fibrinolysis and impairs the dissolution of clots  and KIRC_sRBM_G1 patients had much less probability of survival, similar to the results from the SdA method Fig. . There wTo further validate the SdA method and also show its general application, we applied the KIRC model as a pre-trained model to classify LUAD and LGG patients, two independent datasets. We first downloaded the miRNA, protein and gene expression, methylation and CNA datasets from TCGA for LUAD and LGG patients and processed the data the same way as the KIRC dataset. Since the dataset and the goal of the work were similar between KIRC, LUAD and LGG, transferring learning probably works and produces better results than learning directly from LUAD and LGG datasets separately. We transferred the hidden variable values from KIRC model to LUAD and LGG models and then fine-tuned the variables using LUAD and LGG datasets separately with a lower learning rate (0.0005). The methods split the patients into two clinical relevant groups for LUAD and LGG: the LUAD_SdA_G2 group has significantly lower survival probability than the other group Fig. , similarIn this work, we developed a SdA model to classify the kidney clear cell cancer patients based on multi-platform genomic datasets. The model classified the patients into two groups. The KIRC_SdA_G1 group had significantly lower survival probability, higher grade of neoplasm histology and pathology than the KIRC_SdA_G2 group. We compared the SdA model with K-means clustering method. SdA produced a much clearer classification of the KIRC patients according to the clinical information, indicating SdA is a more promising method for cancer subtype discovery. We did limited tests using RBM based DBN model and obtained similar results as the SdA method. Further, we identified a group of DE genes, miRNAs and one protein that may be potential biomarkers for developing novel diagnosis method and personalized treatments. In the end, we demonstrated using two independent datasets that the SdA method and the learned models can apply to other similar tasks.In our current study, we did not consider the biological connections between each dataset, but treated each dataset independently in the integration analysis. Then we performed the differential analysis for gene, miRNA and protein expression separately between the two subgroups. However, it is widely accepted that miRNAs and methylation typically repress gene expression both at mRNA and protein level. It may improve the accuracy for biomarker discovery by including the biological connections between the datasets and give a better explanation on the role of biomarker in cancer development.et al.)23, in which RCC was classified into nine subtypes based on the same types of genomic datasets, including three subtypes  of ccRCC that have better (CC-e.2), worse (CC-e.3), and intermediate (CC-e.1) patient survival probablity. All the 441 ccRCC patients used in our study were included in the Chen et al.23 study. Of these 441 patients, ~94% were mapped to CC-e.1, CC-e.2 and CC-e.3. In our study, KIRC_SdA_G1 contained 174 patients and was the group with worse survival probability. The majority of the KIRC_SdA_G1 patients were in the subtypes that defined as worse and intermediate patient survival categories (CC-e.3: ~50% and CC-e.1: ~28%) in the Chen et al. study. (Supplementary Table 5) Similarly, most of the KIRC_SdA_G2 patients were in the subtype of CC-e.2 (~68%), indicating the consistency between our study and the Chen et al. study.We compared our subtyping results with a previous study from TCGA consortium 25 but in special cases, SA is better or comparable to SdA26. A nice property of dA is that it naturally handles missing values or noising values or multi-modal values because of being trained using corrupted input. RBM is also a stochastic model, similar as dA. Although dA and RBM are different in training process and formula format, they function in a similar way (mapping the inputs to hidden representations)  Fig. . SdA and31, SdA and sRBM behaves better than K-means on classifying using complicated datasets. In our study, K-means was able to separate the patients into two groups, but the groups were not significantly different on survival time. Because K-means is sensitive to the initial assignment of the centroids, we tested several initializations of the centroids, but still obtained similar results. Another study13 also reported that sRBM works better than K-means. These results suggest that comparing with the simple method, K-means, the multi-layer deep learning method is much more robust on dealing with multi-platform and complicated datasets.Although in comparison with shadow deep learning method, K-means can produce comparable or even better resultsIt is common that deep learning methods consume more time to train the model than the commonly used methods, such as K-means and PCA. However, as shown in our study, the SdA method performed apparently better than K-means. To reduce the computing time on generating optimal parameters for SdA models and learning from scratch, we tested whether we can use the learned model as a pre-trained model. As expected, the KIRC model worked efficiently on classifying the LUAD and LGG patients by taking it as a pre-train model. The limitation of transfer learning is that the model probably cannot be used directly on tasks with less similarity. When it occurs, a new model will be needed to learn from scratch.32 for KIRC patients. TCGA-assembler processed each type of data into a matrix that makes easier for downstream analysis. Some data from the same patient were duplicated either because multiple samples were collected from the same patient, or different portions of the sample were measured or sequenced in different orders or different plates. The average value of the duplicated data points was used in the downstream analysis.miRNA, protein and gene expression, methylation and CNA were downloaded and re-formatted using TCGA-assemblerFor the gene expression and miRNA expression, both the counts and FPKM (Fragments Per Kilobase of transcript per Million mapped reads) values were downloaded. The FPKM values were used for classification and the counts were used for differential expression analysis. The normalized protein expression was downloaded and used both for the classification and differential expression analysis.32. During the merge process, the data were normalized using quantile method.The Infinium Human Methylation 27 BeadChip and Infinium Human Methylation 450 BeadChip were downloaded, processed and merged by TCGA-assembler32. A data matrix was produced with rows corresponding to CNAs and columns corresponding to samples. The process procedure performed by TCGA-assembler mapped the CNA locations to genes. Some genes are within the same CNA region, therefore, duplicated rows in the data matrix were produced. We removed the duplicated rows from downstream analysis.The CNA data without the probes frequently containing germline CNVs were downloaded and processed using TCGA-assemblerx is the input value, xmin is minimum value across all the samples, xmax is the maximum value across all the samples and a is a small number to avoid potential errors in the calculation.We filtered the patients with missing values for any of the five datasets. In the end, we obtained a total of 441 patients for each dataset. Before fit into the deep learning methods, all the input values were normalized across samples to a value between 0 and 1 using the formula ofThe miRNA, protein and gene expression, methylation and CNA datasets for LUAD and LGG patients were downloaded and processed in the same way as the KIRC dataset. The sample size for LUAD is much smaller, comparing to KIRC dataset, and a total of 354 patients were remained. The LGG dataset do not have methylation data from Infinium Human Methylation 27 BeadChip, therefore, a similar number of methylated loci from Infinium Human Methylation 450 BeadChip was used. All the data were normalized to a value between 0 and 1 in the same way as done for the KIRC dataset (formula 1).x, is randomly altered into 0, called xT. The hidden layer (y) was built based on formula 2, where W is the weight matrix, b is the bias term, and s is the sigmoid function. The reconstructed input layer (z) was built based on the formula 3, where W′ is a tied weight of W and b′ is the bias term. Cross-entropy (formula 4) was used to measure the difference between the input data and the reconstructed input data. The training process is to find the W, W′, b, b′ that can minimize the cross-entropy.A denoising autoencoder (dA) is an extension of an autoencoder. An autoencoder is a variant of neural networks that commonly used for feature extraction. It maps the input data into a hidden representation through a deterministic function. When the hidden units are less than the input units, the input is compressed. The training objects for an autoencoder is to build the compact features that can reconstruct the input data. By minimizing the difference between the reconstructed input and the original input, the intrinsic features of the data are learned. Therefore, an autoencoder is an un-supervised method that is independent of the data labels. Denoising autoencoders improve the basic autoencoders by corrupting part of the input data. By learning from the corrupted input, dA could not only capture the essential features of the input robustly, but also avoid the problem of reconstructing the same input from an un-corrupted input. Since dA reconstructs the input from the corrupted input that involved in sampling, dA becomes a stochastic model. Normally, a simple dA contains three layers: an input data layer, a hidden layer and a reconstructed input layer. And the input data is corrupted. Here we used binomial noise to corrupt the input, that is part of the input data, 33 for the SdA analysis.In our study, we have five types of genomic data, miRNA expression, protein expression, gene expression, methylation and CNA. To subgroup the patients based on the five genomic datasets, we developed stacked layers of dA (SdA). Each dataset has its independent hidden layers and then the hidden layers from the five datasets were combined together Fig. . The sizTo train the model, we tested a wide range for the following parameters: the learning rate at 0.005, 0.01, 0.05, 0.1, 0.15, 0.2, 0.3 and 0.5; the corruption level at 0.01, 0.05, 0.1, 0.2 and 0.5; the batch size at 10, 20, 50 and 100; and the epoch size at 1000, 2000 and 5000. We did two-fold cross validation and calculated the correlations between the reconstructed input from the last joint hidden layer and the last individual hidden layer with the original input from both the training and test datasets Fig. . We summ13. We used the same hidden layer structures to perform the analysis  normally refers to stacked of RBMs. RBM, unlike the deterministic model of the autoencoder method, is a generative model, using stochastic process to map the input to hidden variables, thus similar to dA. Several papers showed DBN works on cancer subtype discoverysis Fig. . The alg14 was used for the DE gene and miRNA analysis between the KIRC_SdA_G1 and KIRC_SdA_G2 patients. Upperquartile from edgeR was used to normalize the raw counts across samples. The genes and miRNAs with expression less than 1 Counts Per Million (CPM) in at least half of the SdA_G1 samples were removed from the analysis. The final set of DE genes and miRNAs were called at thresholds of FDR at 0.05 and fold change >2. The protein differential expression analysis was performed using two-tailed t-test and the significantly DE proteins were called at thresholds of FDR at 0.05 and fold change >2.edgeR34. To do transferring learning, we initialized the variables of the LUAD and LGG models with the learned variable values from KIRC model, and then fine-tuned the variables using the LUAD and LGG datasets separately with a lower learning rate at 0.0005, 20 fold less than the KIRC model as suggested in the study of35.We downloaded the miRNA, protein and gene expression, methylation and CNA datasets from TCGA for LUAD and LGG patients and processed the same way as KIRC. To save the time on selecting the optimal model parameters, we used the KIRC model as a pre-trained model and do fine-tuning using the LUAD and LGG datasets . Since the three datasets  have similar data structure and goal of subtyping, transferring learning most likely can produce better results than learning from LUAD and LGG directlySupplementary InformationSupplementary Tables 1-5