Clustering of gene expression data is widely used to identify novel subtypes of cancer. Plenty of clustering approaches have been proposed, but there is a lack of knowledge regarding their relative merits and how data characteristics influence the performance. We evaluate how cluster analysis choices affect the performance by studying four publicly available human cancer data sets: breast, brain, kidney and stomach cancer. In particular, we focus on how the sample size, distribution of subtypes and sample heterogeneity affect the performance.n = 40 as for n = 330. The relative distribution of the subtypes had a noticeable effect on the ability to identify the disease subtypes and data with disproportionate cluster sizes turned out to be difficult to cluster. Both the choice of clustering method and selection method affected the ability to identify the subtypes, but the relative performance varied between data sets, making it difficult to rank the approaches. For some data sets, the performance was substantially higher when the clustering was based on data from only one sex compared to data from a mixed population. This suggests that homogeneous data are easier to cluster than heterogeneous data and that clustering males and females individually may be beneficial and increase the chance to detect novel subtypes. It was also observed that the performance often differed substantially between females and males.In general, increasing the sample size had limited effect on the clustering performance, e.g. for the breast cancer data similar performance was obtained for The number of samples seems to have a limited effect on the performance while the heterogeneity, at least with respect to sex, is important for the performance. Hence, by analyzing the genders separately, the possible loss caused by having fewer samples could be outweighed by the benefit of a more homogeneous data. Diseases like cancer can arise from a multitude of genetic and epigenetic changes. Studying gene expression profiles from tumor samples in cancer patients can reveal information about novel cancer subtypes , 2. HoweProfiling of gene expression is a way of analyzing the activity of genes, and technologies like RNA- sequencing and microarrays have made it possible to look at gene expressions for thousands of genes simultaneously. Signatures in gene expression are widely used in medical research, e.g. in the assessment of breast cancer , 4 and fA popular approach for detecting novel subtypes of a disease is to use cluster analysis, which is an unsupervised approach for finding groups with similar patterns . It has The high dimensionality of gene expression data makes detection of novel subtypes a difficult task, which is also complicated by the noisy character of the data . It is tThe clustering algorithms use different strategies to define groups and can therefore cluster samples in rather dissimilar partitions. Consequently, the relative performance of different clustering methods can be expected to vary among different data sets. Understanding which factors affect the clustering performance is essential in order to select a suitable clustering approach for a specific problem. However, for problems aiming at detecting novel disease subtypes using gene expression data, there are few studies that have evaluated the performance of different clustering approaches.Jaskowiak et al.  comparedA challenge in detecting novel disease subtypes using cluster analysis is that the individuals in the study are often heterogeneous and that they can be grouped with respect to several factors, e.g. disease subtype, age, and gender. The fact that several of the factors may have a pronounced effect on the gene expression reduces the chance that the resulting cluster partition will agree with the partition defined by the disease subtype. We argue that the more heterogeneous a patient group is, the more difficult it will be to achieve a partition that corresponds to the disease subtype.One solution would be to base the clustering on a relatively homogeneous subset of patients. A possible drawback with this approach may be that including fewer patients will in itself have a negative effect on clustering performance.In this paper we compare the performance of 30 clustering approaches, defined by which clustering algorithm is used and how the variable selection and feature extraction are performed. The approaches are used to analyze gene expression data (RNA-seq data) from four human cancer tumor types, each with two known subtypes. In particular, we study how the clustering is affected by the characteristics of the data, including the sample size, the distribution of the subtypes and the homogeneity of the individuals.We included four human cancer data sets from The Cancer Genome Atlas (TCGA) network, where the gene expression data were obtained using Illumina HiSeq 2000 RNA Sequencing Version 2. Clinical data together with Level 3 mRNA-seq RSEM data  were dowOnly samples from primary solid tumors were included in the study. For all data sets, the individuals belong to one of two predefined subclasses, see The Brain data consists of normalized gene expression measurements in tumor samples from patients with lower grade glioma (denoted as LGG by TCGA). The patients were classified into three prognostically significant subtypes, IDHcodel—gliomas with IDH mutation and 1p/19q co-deletion, IDHnocodel-gliomas with an IDH mutation and no 1p/19q co-deletion, and gliomas with wild-type IDH -IDHwt. . GliomasThe Breast data consists of normalized gene expression measurements in tumor samples from patients with breast invasive carcinoma (denoted as BRCA by TCGA). Subtypes were defined by the Estrogen Receptor (ER) status, either positive or negative . The ER The Kidney data consists of normalized gene expression measurements from tumors in patients with kidney renal papillary cell carcinoma (denoted as KIRP by TCGA). Two main subtypes, type 1 and 2, have been histologically determined . A totalThe Stomach data consists of normalized gene expression measurements from tumors in patients with stomach adenocarcinoma (denoted as STAD by TCGA). Four molecular subgroups have been identified: Epstein-Barr Virus (EBV)–positive, Genomically Stable (GS), MicroSatellite Instability (MSI) and Chromosomal INstability (CIN) . Here wehttp://cancergenome.nih.gov/abouttcga/policies/informedconsent.The data included in this study was accessed from TCGA according to open access guidelines and comes from previously publishes articles where written informed consents were obtained in accordance with the local institutional review boards , 29–31. th percentile) were removed prior to the downstream analysis.The analyses were based on normalized level 3 mRNA-seq gene expression data from TCGA. Prior to the analyses the data were log2-transformed. Furthermore, genes that either showed no variation between samples  or were expressed at very low levels  and let the analysis be based on the M first principal components (PC). The number of components M was set to 5 or 30 and N was set at three levels: 100, 1000, and all genes. Hence five selections methods were considered: top 100 genes, top 1000 genes, all genes, first 5 PC and first 30 PC. The principal components were obtained using the R-function “prcomp”.The first step in the downstream analysis was to determine which variables or features should be included in the cluster analysis. This was done either by selecting the The final step in the analysis was to cluster tumor samples based on their gene expression profiles with the objective to detect cancer subtypes. Clustering algorithms often demand that the user specifies the number of resulting clusters. Here, the analyses were performed in such way that the clustering resulted in two clusters. The main aim here was not to study the relative performance of different clustering approaches, but to study how data characteristics like sample size, distribution of subtypes and sample heterogeneity influence the clustering performance. Therefore we decided to include commonly used clustering algorithms and commonly used methods for variable selection and feature extraction.Self-organizing map (SOM) is arguably the most known unsupervised neural network and was therefore included in the study. For low dimensional problems, i.e. few features, k-means and SOM can be expected to perform similar, but this may not hold true for high-dimensional problems. Affinity propagation (AP) may not be widely used for clustering cancer samples, but were included since it did not resemble any of the other included algorithms. Finally, we included an ensemble approach since it arguably offers a robust clustering alternative. The included algorithms are described below.Hierarchal clustering is frequently used and an example of a connectivity model. We choose to include a hierarchical clustering using the Manhattan distance and Ward´s linkage due to its relative good performance . A rathed = 1 –| ρ |, where ρ is Pearson correlation) were used as dissimilarity measures and Ward´s linkage was used to calculate the distance between clusters [There are two strategies in hierarchical clustering; agglomerative and divisive. Here the agglomerative clustering was used. This bottom-up approach starts by treating the individual samples as clusters and then recursively joins them until only one single cluster remains. Hierarchical clustering requires that the user specifies a dissimilarity measure and a method for calculating distances between clusters. In this paper both the Manhattan distance and the absolute Pearson correlation distance  partitions entroids .The analyses were made using the R-function “kmeans”, which used 10 random starts (nstart = 10).(SOM) is a type of artificial neural network that can be used for clustering. A SOM consists of two layers; one input and one output layer. The output layer typically consists of two-dimensional arrays of nodes that are ordered as a rectangular or hexagonal grid. The input layer is fully connected to the output layer, and each node has a vector of weights of the same dimension as the input vector. The weights are updated during the training process and when the weights are fully trained, the distance from each node to the input data is calculated and the sample falls into the cluster (node) that is closest [Self-Organizing Map  closest .The clusters were obtained using the function “som” in the R-package “kohonen” , with dex is to serve as exemplar to a data point y, and one indicating how suitable it is for y to choose x as it´s exemplar. The algorithm updates the messages until convergence [The Affinity Propagation (AP) algorithm tries to identify data points (exemplars) that can represent clusters. It considers every data point as a potential exemplar and tries to maximize the total similarity between data points and their exemplars. If we consider data points as nodes in a network, then the algorithm operates by sending two types of messages along the edges: one that represent how well suited a data point vergence .k was set to 2 and the function “negDistMat” was used for calculating dissimilarities.The R-function “apclusterK” in the package “apcluster”  with parThe resulting clusters from the above mentioned methods were used to create a consensus cluster using Cluster Ensemble (CE). This was achieved using the R-function “cl_consensus” in package “clue” , with meAside from the fact that the data sets in this study come from different cancer diseases, there are a number of other things that characterize them as well, e.g. the sample size, the relative distribution of the two subtypes and underlying factors like gender and age distribution.In order to study how the data characteristics affected the clustering performance, we constructed sub-data sets by sampling the data as described below. Unless otherwise stated the sampling was made without replacement and repeated 10 times to reduce sampling effects. We sampled a maximum of 70% of the available samples. All cluster analysis approaches were applied to each of the sub-data sets.The number of samples in the data sets varied between 163 and 1031. To evaluate how sensitive the clustering performance was to sample size, we compared the performance by considering sub-data sets with different number of samples. The sample size was varied (with a step of 10) from 40–110, 40–330, 40–100 and 40–80 for Brain, Breast, Kidney and Stomach, respectively. Each sampling was made so that the fraction of observations belonging to each of the two subtypes was fixed at 50% . In addition, the analysis were repeated for a case where 20% of the samples were in the smallest group. The maximum number of samples were then 120, 270, 70 and 120 for Brain, Breast, Kidney and Stomach respectively.Here the fraction of samples belonging to the first subtype was varied while keeping the sample size fixed. The percentage of samples belonging to the first subtype was altered at the levels 10%, 20%, 30%, 40% and 50%, with 108, 100, 65 and 83 samples for Brain, Breast, Kidney and Stomach, respectively.Homogeneous data sets with respect to gender  were constructed such that the distribution of the subtypes was symmetric and the sample size was the same for both genders. As a control, we constructed a heterogeneous data set with 50% female and 50% male, having the same sample size and relative distribution of the subtypes as the two homogeneous data sets. Here the sample sizes were 55, 28 and 36 for Brain, Kidney and Stomach, respectively. The ARIs for the heterogeneous data were calculated separately for males and females.The considered sub-classes for the data sets were regarded as gold standards. Arguably, other divisions of the data could have been considered. A minimum criterion on a gold standard is that there should be a genetic signal that can be detected using supervised techniques. To assess how well the gene expression data separated the cancer subtypes, we used PCA and plotted the first two components with the patients colored according to subtypes. In addition, the supervised classifier random forest was applied to the data sets with the gold standard partition as response . The fraThe performance of a clustering approach was determined by how well the resulting partition matched the gold standard partition. Several clustering similarity measures have been proposed –42. We un objects in a set S. Let A = {A1, A2, …, lA} and B = {B1, B2, …, mB} be two partitions of S (one is the gold standard partition and one is the result from clustering). Then the overlap of A and B can be described by a contingency table, see ijn denotes the count of common objects in jA and iB.ARI is calculated as follows. Suppose we have The Adjusted Rand Index (ARI) is then obtained asThe ARI takes value 1 if the two partitions are identical and has expected value 0 in the case of a completely random partition. It can yield negative values if the agreement is worse than expected by chance. The R-function “RRand” in package “phyclust”  was usedThe same ARI-value can be obtained by very different partitions of the samples. Hence, the fact that two cluster outputs yield the same ARI-values compared to the gold standard does not necessarily imply that the resulting clusters are similar. In addition to quantifying the agreement with the gold standard, ARI was used to quantify the similarity between different clustering approaches, by considering the average of the pairwise ARI (apARI).The agreement between partitions was also evaluated using the distance metric variation of information . Since tn = 40 versus n = 100) we used the one-sample Wilcoxon signed rank test based on the values from the 30 clustering approaches given in the tables (clustering approaches using 30 PC were omitted for comparisons with sample size lower than 30). The Wilcoxon rank sum test was used in the case when we compared apARI for different sample sizes. For each test we report the median of the ARI-differences (i.e. delta ARI) and the corresponding p-value (p). Note that the figures were obtained in a slightly different way than the tables. The figures show the result for the 10 replicates, where each value was obtained by averaging over all 30 clustering approaches. All analyses were performed using the R programming language version 3.4.3 [In the cases where we have subsampled data, the tables contain mean values of the 10 replicates. For comparisons between different settings  were applied to the four data sets, where the performance was quantified using ARI. The clustering performance varied considerably between the data sets. Breast was relatively easy to cluster while the other data sets were harder to cluster. The median ARI  for Brain, Breast, Kidney and Stomach were 0.10, 0.55, 0.17 and 0.15 respectively. Interestingly, the overall clustering performance for Brain was low while the obtained accuracy using supervised classification was relatively high. This shows that a simple classification problem is not necessarily an easy clustering problem.The choice of clustering method had a large impact on the clustering performance, but the methods’ relative performance varied between the data sets making it difficult to rank the methods, see The relative performance of the five selection methods also varied between the data sets, making it hard to draw general conclusions, see n = 40 and n = 110). For Stomach we found no significant increase in performance  between the smallest (n = 40) and largest (n = 80) sample sizes. This may partially be explained by the fact that the performance was very low also when the sample size was relatively high, see n was increased from 40 to 100 , but there was no stable trend, see n = 40 and n = 330, . The trends presented were observed independently on which clustering algorithm was considered, see Next we investigated how the sample size affected the performance of the clustering, while keeping the distribution of subtypes fixed at 50%. To our surprise, only limited gain in performance was observed when the sample size was increased, see In addition to consider the balanced distribution with 50% of each of the subtypes, we also considered a case where 20% of the observations were in the smallest group. Again, it was observed that the sample size had a limited effect on the performance, see n = 40) to modest (n = 100), see n = 40 than at n = 330 . For all data sets, the agreement between the approaches was higher than the agreement with the gold standards. Arguably, this indicates that there are relatively strong signals in the data sets that are not in absolute agreement with the considered cancer subtypes, see Figs Interestingly, although increased sample size had limited effect on the performance it was evident that the 30 approaches clustered the individuals more similar when the sample size increased from relatively small n = 0 to modeWe investigated how the performance was affected when the relative distribution of the two subtypes was altered. Here the sample size was fixed while the proportion of the smallest subtype was altered from 10% (disproportionate data) to 50% . It should be stressed that the relative distribution of the classes in practice are unknown, and cannot be estimated in advance. For Brain, Breast, Kidney and Stomach, the performance was significantly higher for the balanced data compared to the disproportionate data, , ,  and  respectively, see Breast was investigated further by considering subtype fractions between 10–90%, see Investigating the clustering methods with respect to the distribution of the subtypes revealed that all clustering methods were affected, but for hclust(cor) the differences were less visible due to the overall low performance, see It is well known that there are widespread gender differences in gene expression , 47. HenArguably, a relatively high ARI for Females  compared to Mixed females  suggests that a homogeneous data w.r.t. sex has a positive effect on the clustering performance. The ARI-values for Females and Males were used to compare the strength of the subtype specific signals.The clustering performance was higher when the clustering was performed on homogeneous data compared to heterogeneous data in all cases but one, see The overall aim with this work was to identify and quantify sources that affect the performance of the clustering. Arguably, unsupervised classification, where the objective is to discover novel subgroups, is among the most difficult statistical problems. The two major difficulties being that we are unable to train our model since the true partitions of the samples is not known and that the data generally are affected by several factors and not just the factor of interest, e.g. the factor defining the novel cancer subtypes. For example, patients are diverse with respect to gender, age, ethnicity, etc. and all factors may contribute to differences in gene expression. The lack of a true answer makes it hard to validate and evaluate clustering approaches, which is a serious problem since applying different clustering approaches often yield very different results. In addition, feature selection is a challenge since class labels are unknown making it relatively harder to represent a cluster analysis problem in a lower dimension compared to a supervised classification problem.In this work the gold standard partitions were defined by two known subtypes. Although our ambition was to select as relevant subtypes as possible it is clear that there are several alternative partitions that could have been considered for validation. Here, the focus was to examine the relative effect of applying different clustering approaches and to study the impact sample size, disproportionate cluster size and data heterogeneity had on the performance. Arguably, this makes the definition of the gold standard less important although the choice of the golden standard may influence the relative performance of different methods. The gold standards were chosen based on knowledge, availability within TCGA and having a strong “genetic signal”, i.e. high accuracy when applying supervised classification. An alternative approach would be to define the gold standard by several factors, e.g. cancer type, gender and age (young and old), resulting in eight classes. Challenges with this approach is that not all underlying factors are known, that misclassification with respect to all factors will be penalized equally, and that a large number of classes are likely to result in low ARI values.n > 20) were considered.Supervised classification suggested that there was a strong genetic signal (i.e. the subtypes could be predicted) in all four data sets. Despite this, the clustering resulted in relatively low performance for Brain, Kidney and Stomach, although higher than expected by chance. This may suggest that the data are strongly affected by factors that are not necessarily related to the considered subtypes. Arguably, clustering homogeneous data should be easier than clustering heterogeneous data. This was confirmed when the genders were analyzed separately, where the performance was substantially higher when the clustering was based on only one sex compared to a mixed population. Splitting the samples into more homogeneous sets inevitably leads to fewer samples, which may have a negative effect on the clustering performance. To our initial surprise, increasing the sample size had little influence on the performance. We did get similar performances independently if we used 40 or 300 samples. However, the performances of the clustering approaches became more similar when the sample size increased. Arguably, this suggest that sample size has some importance, but that increasing the sample size per se may not solve the clustering problem. These findings were also observed in a simulation, where heterogeneous data in comparison with homogeneous data showed very little gain in performance when adding samples, see Even though there are recognized gender differences in both cancer survival and treatment response, it is not common that the clustering of subtypes are made separately for males and females . Based oAlthough the primary aim was not to evaluate clustering algorithms, but to investigate other sources contributing to the variations in performance, we made some findings. In particular, hclust(cor) often yielded lower performance than the other methods, especially when the selection method involved principal components. We included a set of principally different clustering methods that are commonly used, but the included methods only cover a fraction of the available algorithms. Further studies are needed to ensure that our findings are valid also for other methods.The clustering performance was sensitive to the relative distribution of the subtypes, where data with heavily disproportionate distributions turned out to be difficult to cluster. The distribution of the subtypes is not known in advance, but if the subtypes are believed to be associated with difference in survival (or some other variable), it may be possible to use survival data to get an idea of the subtype distribution.Clustering high-dimensional gene expression data is a challenge. Even with a strong genetic signal, cluster analysis approaches may fail to identify the partition of interest. One important reason for this is that gene expression data are influenced by several factors, which may or may not be known to the researcher. Furthermore, the optimal clustering approach depends on the data and general recommendations are therefore difficult to give. However, the results suggest data characteristics may influence the clustering performance. Interestingly, increasing the sample size may not enhance the performance of the clustering although it makes the clustering approaches more similar. If the distribution of the subtypes is disproportionate, clustering can be very difficult. Finally, the result shows that homogeneous data are easier to cluster than heterogeneous data. Together this suggests that it may be beneficial to analyze the genders separately. The gain of obtaining a more homogeneous data outweighs the possible drawback of having fewer samples.S1 TableAdjusted Rand index for different data sets and 30 clustering approaches (combination of clustering method and selection method).(XLSX)Click here for additional data file.S2 Tablen) a 50/50% distribution of subtypes. The table shows the mean values based on 10 random sub-sampled data sets.Average adjusted Rand for different sample sizes ((XLSX)Click here for additional data file.S3 Tablen) and a 50/50% distribution of subtypes. The table shows the mean values based on 10 random sub-sampled data sets.Average adjusted Rand for different sample sizes ((XLSX)Click here for additional data file.S4 Tablen) a 50/50% distribution of subtypes. The table shows the mean values based on 10 random sub-sampled data sets.Average adjusted Rand for different sample sizes ((XLSX)Click here for additional data file.S5 Tablen) a 50/50% distribution of subtypes. The table shows the mean values based on 10 random sub-sampled data sets.Average adjusted Rand for different sample sizes ((XLSX)Click here for additional data file.S6 Tablen) for a 20/80% distribution of subtypes. The table shows the mean values based on 10 random sub-sampled data sets.Average adjusted Rand for different sample sizes ((XLSX)Click here for additional data file.S7 Tablen) for a 20/80% distribution of subtypes. The table shows the mean values based on 10 random sub-sampled data sets.Average adjusted Rand for different sample sizes ((XLSX)Click here for additional data file.S8 Tablen) for a 20/80% distribution of subtypes. The table shows the mean values based on 10 random sub-sampled data sets.Average adjusted Rand for different sample sizes ((XLSX)Click here for additional data file.S9 Tablen) for a 20/80% distribution of subtypes. The table shows the mean values based on 10 random sub-sampled data sets.Average adjusted Rand for different sample sizes ((XLSX)Click here for additional data file.S10 TableAverage adjusted Rand index for 10 random samplings for different proportions of subtypes in the Brain data set.(XLSX)Click here for additional data file.S11 TableAverage adjusted Rand index for 10 random samplings for different proportions of subtypes in the Breast data set.(XLSX)Click here for additional data file.S12 TableAverage adjusted Rand index for 10 random samplings for different proportions of subtypes in the Kidney data set.(XLSX)Click here for additional data file.S13 TableAverage adjusted Rand index for 10 random samplings for different proportions of subtypes in the Stomach data set.(XLSX)Click here for additional data file.S14 TableAverage adjusted Rand index for 10 random samplings when keeping the subtypes fixed at 50%.(XLSX)Click here for additional data file.S1 FigAdjusted Rand index for 10 random samplings and 5 gene selection methods.(TIF)Click here for additional data file.S2 Fign patients , where the patients were categorized with respect to gender, age (old or young) and cancer type (A or B). Fifty percent of the patients were male, 50% were young and 50% had type A such that the samples were divided in eight equally sized homogeneous groups. Twenty of the genes were affected by gender, 20 by age and 20 by cancer type and the remaining 940 genes were not affected by any factor. The expression values from the non-affected genes were simulated from a normal distribution with mean zero and standard deviation one ). The expression values of the affected genes were simulated from N if male/young/type A and N if female/old/type B. Three hundred simulations were made for each considered sample size. Hierarchical clustering using the Manhattan distance and complete linkage was applied to each simulated sample. The resulting dendrogram was cut so that the samples were clustered into two groups and the results were compared to the partition defined by cancer type (i.e. the gold standard) using the adjusted Rand index. The clustering was made using all genes (All), excluding genes affected by either gender or age (Without) and including only the 20 genes affected by cancer type (Only). Here “All” relates to the problem when the data are affected by three factors, “Without” to the situation when the data are affected by a single factor, while “Only” relates to the ideal situation when only the cancer genes affect the clustering. Here the performance improvement when increasing the sample size was considerably lower for the complex problem (All) compared to the easier problem (Without).Thousand features (genes) were simulated for (TIF)Click here for additional data file.


Multiplexed single-cell RNA-seq analysis of multiple samples using pooling is a promising experimental design, offering increased throughput while allowing to overcome batch variation. To reconstruct the sample identify of each cell, genetic variants that segregate between the samples in the pool have been proposed as natural barcode for cell demultiplexing. Existing demultiplexing strategies rely on availability of complete genotype data from the pooled samples, which limits the applicability of such methods, in particular when genetic variation is not the primary object of study. To address this, we here present Vireo, a computationally efficient Bayesian model to demultiplex single-cell data from pooled experimental designs. Uniquely, our model can be applied in settings when only partial or no genotype information is available. Using pools based on synthetic mixtures and results on real data, we demonstrate the robustness of Vireo and illustrate the utility of multiplexed experimental designs for common expression analyses. Single-cell RNA-seq (scRNA-seq) is a rapidly evolving technology. Robust protocols and reduced costs have fostered applications in biomedicine, for example to identify biomarkers in disease , 2, or tAcross these use cases, multiplexed experimental designs that combine multiple samples in a single experiment have critical statistical advantages compared to the serial analysis of samples in independent experimental batches , 6. In pThe aforementioned advantages have motivated a series of barcoding strategies to demultiplex samples from pooled experiments. In addition to simplified experimental logistics and reduced batch variation, pooled designs can also facilitate the identification of doublet cells. Existing barcoding strategies include molecular labelling prior to analysis –12 as weMultiplexed designs with genetic barcoding are particularly applicable in biomedical research, where the analysis of larger cohorts of genetically distinct individuals is particularly relevant . HoweverTo address this, we here present Vireo , a principled Bayesian method to demultiplex arbitrary pooled designs that combine genetically distinct individuals. Uniquely, Vireo models the genotypes of each individual as latent variables, which are inferred from the observed scRNA-seq reads. The model can also leverage partial genotype information, e.g., when genotype data are available for a subset of individuals, and hence can be applied to a wide range of experimental settings.K individuals and estimates the genotypic state of these individuals at known polymorphic loci. Specifically, the model takes a set of common genetic variants as input (for example derived from the 1000 Genomes Project  and y=, we define the expected genotype for the doublet sample as follows,To detect doublets, we construct the genotype of each pair of samples and expand the t=0.5 and t=1.5 respectively for combinations of genotype 0 & 1 and 1 & 2 in the doublet sample. For convenience, we consider the binomial parameters for the alternative allelic reads and assume that the binomial parameters θ0.5 and θ1.5 also follow beta distributions. We approximate the hyper-parameters of the beta distribution empirically by respectively taking the ratio and shapes with the arithmetic and geometric means from the two ordinary genotypes. The resulting distribution of θ0.5 can be expressed as followswhere we introduce two pseudo-genotypes θ1.5.Similarly, we define the distribution of G and θ, consisting of K biological and (K−1)∗K/2 doublet samples. In this model we can calculate the probability that a cell originates from one of the doublet samples using Eq. In this augmented model, we have the full distribution for the extended genotype reference η to define the a priori belief of observing a doublet. Specifically, the prior binomial distribution π is constructed as followsAs an additional refinement, we specify a non-uniform prior on K2=(K−1)×K/2 as number of the combined sample pairs. The prior probability for doublet cells is low in most assays, e.g., η=0.05. In case of the 10x Chromium platform, the prior value can be estimated as a function of the number of loaded cells M, e.g., η=M/100,000 following [where ollowing , which bj coming from any of the K input samples or K2 combined sample pairs . We use the highest assignment probability of the K input samples, prob_max, as the confidence score for singlet assignment and use the summarized probability of all K2 sample pairs as the confidence score of a doublet, namely prob_doublet.Therefore, we can obtain the posterior of each cell’s sample identity, i.e., the probability of cell Vireo implicitly estimates the genotypes for the subset of variants with sufficient coverage =0Initialize the variant set IG=H(T)−H(T|v)Rank variants by the information gain S, T, and H(T)Select the variant with highest information gain and update H(T)= log2(K), return S and T, otherwise go to step 3.If Given a set of variants for which estimated genotypes are available, the Vireo software implements a heuristic to define a minimal and informative set of discriminatory variants. This set of variants can be used to perform qPCR-based genotyping or for other targeted genoytping methods. Briefly, the algorithm implemented in Vireo prioritizes variants with largest information gain in splitting samples, as follows.Additionally, variants with homozygous alternative alleles in the pooled samples can also be filtered out before hand if needed. Examples of discriminatory variant sets for the six-sample pool from HipSci project are shown in Additional file Differential expression analysis was performed with edgeR  between y∼cdr+condition+sample, where y is the expression count for a specific gene, which is regressed on three covariates: cdr, the cell detection rate , stimulation condition and the sample identity.Method 1: y∼cdr+condition, where we ignore the sample identity of each cell in the pool (Method 2). This same model can also be used in a pseudo-bulk manner where we aggregate the counts for all cells of the same type in a sample (Method 3). Alternatively, we can always apply this model at single-cell level for each sample separately  extracted from the 1000 Genome Project, phase 3 [A and D, for UMI counts of alternative allele and the total counts respectively, which are used as input for the Vireo model.In this study, we considered two existing multiplexed scRNA-seq datasets that consist of a total of five batches . Raw.bam phase 3  were use phase 3  for thispipw, jejf, qehq, juuy, uilk, and toco [http://www.hipsci.org). Both scRNA-seq and bulk RNA-seq data sets were downloaded in.bam files and genotyped on 7.4 millions common bi-allelic variants  extracted from the 1000 Genome Project with the cellSNP package. For single-cell data, we only keep variants with minor allele frequency ≥0.1 and ≥20 UMIs. For each bulk RNA-seq sample, we also only keep variants with minor allele frequency ≥0.1 but require ≥100 read counts. Then the genotypes of each bulk RNA-seq sample can be used to align to the samples that are demultiplexed from scRNA-seq data.In order to link the inferred samples to other ’omics data, we used one scRNA-seq pool for iPSC differentiation in the HipSci project  with six samples: and toco , and theand toco  (http://We obtained raw 3’ scRNA-seq data based on 10x Genomics platform (v2 kit) for 16 genetically distinct samples from the Human Cell Atlas (Census of Immune Cells) . These dN/100,000 where N is the total number of cells in the pool.By only keeping cells with >500 genes and >1000 UMIs, we had in total 66,410 cells across 16 samples, with each sample having 2495 to 4909 cells. On average, there are 4000 UMIs per cell (median 2700 UMIs). In the synthetic mixture, we pooled reads for a subset of cells from each sample  and generated multiplexed scRNA-seq data . The script to generate these synthetic data is provided in Vireo’s GitHub repository. Doublets were added into the pooled data by adding proportional extra cells and combining them with another cells randomly. The doublet rate is By default, we pooled 1000 cells from each of 8 samples with doublet rate of 8%. This simulator also allows setting different size of input samples, for example by setting one sample with fewer cells ranging from 50 to 500 (Additional file All these simulations were randomly repeated for five times to account for the variability in the simulation.Additional file 1 Supplementary methods and supplementary figures S1–S13.Additional file 2 Review history.


More than 90% of neuroblastoma patients are cured in the low-risk group while only less than 50% for those with high-risk disease can be cured. Since the high-risk patients still have poor outcomes, we need more accurate stratification to establish an individualized precise treatment plan for the patients to improve the long-term survival rate.We focus on extracting features and providing a workflow to improve survival prediction for neuroblastoma patients. With a workflow for gene co-expression network (GCN) mining in microarray and RNA-Seq datasets, we extracted molecular features from each co-expressed module and summarized them into eigengenes. Then we adopted the lasso-regularized Cox proportional hazards model to select the most informative eigengene features regarding association to the risk of metastasis. Nine eigengenes were selected which show strong association with patient survival prognosis. All of the nine corresponding gene modules also have highly enriched biological functions or cytoband locations. Three of them are unique modules to RNA-Seq data, which complement the modules from microarray data in terms of survival prognosis. We then merged all eigengenes from these unique modules and used an integrative method called Similarity Network Fusion to test the prognostic power of these eigengenes for prognosis. The prognostic accuracies are significantly improved as compared to using all eigengenes, and a subgroup of patients with very poor survival rate was identified.We first compared GCNs mined from microarray and RNA-seq data. We discovered that each data modality yields unique GCNs, which are enriched with clear biological functions. Then we do module unique analysis and use lasso-cox model to select survival-associated eigengenes. Integration of unique and survival-associated eigengenes from both data types provides complementary information that leads to more accurate survival prognosis.Reviewed by Susmita Datta, Marco Chierici and Dimitar Vassilev.The online version of this article (10.1186/s13062-018-0229-2) contains supplementary material, which is available to authorized users. Neuroblastoma (NB) is one of the most common cancers in children. The patients of high-risk (HR) subtype usually have the poorer prognosis . Better n = 1), Germany (n = 420), Israel (n = 11), Italy (n = 5), Spain (n = 14), United Kingdom (n = 5), and United States (n = 42). The patients’ age at diagnosis varied from 0 to 295.5 months .The data used in this study is obtained from the Neuroblastoma Data Integration Challenge of CAMDA 2017. It contains tumor samples of 498 neuroblastoma patients from seven countries: Belgium (n = 121), stage 2 (n = 78), stage 3 (n = 63), stage 4 (n = 183), and stage 4S (n = 53). 176 patients are labeled as high-risk, which are the patients with stage 4 disease more than 18 months at diagnosis and patients of any age and stage with MYCN-amplified tumors [Transcriptome datasets from both microarray (Agilent 44 K oligomicroarray) and RNA-seq are obtained for the 498 patients with known clinical endpoints. The RNA-seq includes 60,788 transcripts and Agilent microarray data for 45,198 probesets, both from 498 primary neuroblastomas. Tumor stage was classified according to the International Neuroblastoma Staging System (INSS): stage 1  to cluster genes into co-expressed modules and summarize each module into an “eigengene”. This approach not only substantially improves statistical power but also allows us to focus more on important biological processes or genetic variations associated with the co-expressed gene modules, making the results more interpretable. We applied our recently developed weighted network mining algorithm local maximum Quasi-Clique Merging (lmQCM) for GCN mining . Unlike ustering , which hustering , 16, we After using lmQCM reduced data dimension, we want to find more important survival-associated modules as features of subsequent integration algorithms. Thus, we built a lasso-regularized Cox proportional hazards (lasso-Cox) model to compute the risk index of each patient, using the eigengenes generated from GCN . Lasso pWith the lasso-regularized Cox proportional hazards model, we can obtain eigengenes that are strongly associated with survival times. However, if only consider selected features of one dataset were used to predict the endpoint of patients, it will result in the lack of information as this lasso Cox model is based on the median of risk indices of the training examples. To obtain a more reasonable classification result, a more effective way might be to make full use of all the information, but we know that in essence microarray and RNA-Seq data are the same, it calls for extra caution to incorporate these datasets.We use two steps to address this problem: First, based on the unique module analysis, we can identify the unique modules in survival-associated features (co-expression modules) selected by Lasso-cox model of each gene dataset. Secondly, respectively building patients similarity network based on about unique modules in each gene dataset, then integrate these two networks. The Similarity Network Fusion(SNF) is a state-of-the art network integrative method and is adopted here.p-value greater than 0.05 as the metrics to determine the uniqueness of co-expression modules between the Microarray and RNA-Seq data  each timet is the number of Iterations. We found that by setting the three parameters to 30, 0.8, and 20, respectively, it can obtain the best classification result.SNF  construcThe key step of SNF is to iteratively update similarity weight matrix corresponding to each of the data types as follows:Where D(i) represent a set of xi’s neighbors including xi in G. Given a graph, G, we use K nearest neighbors (KNN) to measure local affinity. So S(m) is defined as:Let (m) only encodes the similarity to the K most similar patients for each patient. This procedure updates the weight matrices each time generating two parallel interchanging diffusion processes. After t steps, the overall weight matrix is computed.That http://toppgene.cchmc.org) developed by Cincinnati Children’s Hospital Medical Center [The online gene list enrichment tool ToppGene , RNA-Seq data analysis pipelines, and feature levels (gene vs. transcript vs. exon-junction level) do not significantly affect performances of the models . But thep-values being 1.05e-70 and 3.88e-78). M10 and M37 are highly enriched with immune response genes, M7 is highly enriched with extracellular matrix organization genes . All of these agree with the previous pan-cancer study that the top three most common GCN in cancer are cell cycle, immune response and extracellular matrix organization genes [Nine survival-associated eigengenes were selected by using (lasso-Cox) model. Among them, five are the survival-associated eigengenes from microarray data , and four from RNA-seq . Especially, R7, R17, R21 are from RNA-seq only modules, these modules are not present in Microarray data. Most of the nine modules are highly enriched with biological functions: M2 (127 genes) and R2 (268 genes) are highly enriched with cell cycle genes  and RNA-seq data (four eigengenes) that eight of them is unique. To test the power of the combination of these eight unique eigengenes as prognostic biomarkers, we use spectral clustering as well as SNF for survival prognosis, these eight eigengenes significantly improved the survival prognosis by several magnitudes in terms of log-rank test p-value, as compared to results obtained using all of the modules, modules from one data type, or the clinical stage information. These results suggest instead of focusing on individual genes, using gene co-expression network analysis (GCNA) to cluster genes into co-expressed modules and summarize each module into an “eigengene” is a better way to deal with large number gene data. Module unique analysis and lasso-cox model will further help us choose unique survival-associated eigengenes. Integration of unique and survival-associated eigengenes of both data types provides more complementary information will help achieve a more accurate survival prognosis. Also, we identified one subgroup of patients with very poor survival among high-risk patients. Currently, the underlying reasons for the differences between the GCN structures of the two data modalities are still being investigated.The goal of this paper is to find eigengenes that can serve as potential biomarkers for improving the prognosis of high-risk patients and to give a biological description of these eigengenes. Overall, the authors’ methods and approach are valid (but see major recommendation 1), and their results are promising.In the methods section, it isn’t clear whether the lmQCM algorithm for determining modules and corresponding eigengenes was applied to the entire dataset or only to the training data. If the former, then the cross validation performed later to assess the performance of the lasso-cox model might be biased. The concern is that, even though the lmQCM is unsupervised , if the eigengenes are not stable then using the whole dataset to construct them may lead to underestimation of the error rate during CV (because we are selecting features favorable to both the train and test data). It would be good to check that similar eigengenes are obtained from just the training data alone.As correctly pointed out by reviewer, lmQCM does not use any information about the survival and thus it is an unsupervised method. As the reviewer suggested that the eigengenes stability is very important. We therefore tested the correlation of the genes in testing sets use concordance index (a metric we developed previously to test the correlation of genes in a co-expressed module) to test their stability. The result shown in Additional file 3: Figure S1 and Additional file 4: Figure S2 below. The co-expression modules were first detected from training set, and then the concordance indices were calculated for each gene module in the testing set. The observation is that the concordance indices are stable between the training and testing sets for all the modules and are significantly higher than randomly selected gene sets, which demonstrated the stability of the modules and our approach.Author’s response: The primary tool for assessing the prognostic ability of the eigengenes is through Kaplan-Meier (KM) curves and the log-rank test. The KM curve using INSS stage  is used as a baseline, however this is not adequate. The stratification of patients into risk groups in practice takes other clinical into variables. For example, MYCN amplification is well known to be highly predictive of high-risk patients. A fair evaluation of the authors’ method would be to use the KM curve constructed using the  high-risk indicator that is already provided for each patient. Alternatively, since high-risk patients are of primary interest, the authors can subset on these patients and see whether their method can significantly sub-classify those patients. As it stands, it is not clear if the eigengenes provide any prognostic value beyond that provided by clinical variables currently in use.The patients of focus are already labeled as high-risk, which are the patients with stage 4 disease more than 18 months at diagnosis and patients of any age and stage with MYCN-amplified tumors. The MYCN cannot make more contribution for classification of the high-risk patients. But our workflow can give a better classification than use the clinical stage with these patients.Author’s response: This study uses overall survival as the outcome, but how does this approach perform for predicting event-free survival? Are there eigengenes that are associated with this outcome as well? And if so, are they different from the ones associated to overall survival.We thank the reviewer for this important point. In this paper with the selection of data we focus on overall survival, the event-free survival for events such as relapse and metastasis will require more comprehensive set of data beyond the scope of this paper even though but our methods will be applicable on these data.Author’s response: Since copy number variation (CNV) data is available for these patients, and the authors suggest (page 3 line 8) that lmQCM can find modules that are association with structural mutations (like CNV). The CNV data provides an opportunity to verify that claim. It was also mentioned (page 5 line 32) that some M36 genes are “co-localized on the same cytoband, which indicates a potential structural variant in NB patients.” the CNV data can be used to investigate this.We totally agree and the integration/comparison with CNV data is part of our ongoing work.Author’s response: Page 3, eq. : Is thisYes, we clarified this in the revision.Author’s response: p-values is done by “randomly selecting K genes for 1000 times”. Is this sampling done within the given module or among all genes? If the latter, is it sampling with replacement.Page 3, line 46: Computing This sampling is performed within the given module. We provide a more detailed description in the paper.Author’s response: Page 4, line 27: “We found that by setting and o be 30, 0.8, 20 respectively, …” contains typos. Consider “We found that by setting the three parameters to 30, 0.8, and 20, respectively”.We revised the description.Author’s response: 8. Page 5–6: The figure references do not match. Figure We modified the figure captions in the paper.Author’s response: The authors state that “based on the clinical data, 259 patients were assigned in low risk group while 239 were assigned to the high risk group”: Unfortunately, this is not correct for two reasons. First, according to the provided clinical characteristics file, the high-risk patients are 176; secondly, the patients not marked as “high-risk” are not “low-risk” but can be either low or intermediate risk, thus they should be considered as “non high-risk”. Based on this classification, there are 13 patients among the non high-risk group that are not alive, differently from what stated in the paper. Please clarify this point and revise the results.We thank the reviewer’s thoughtful comment. In the original version of the paper, the 239 patients in the high-risk group was labeled based on our classification result from a companion paper using our algorithm. In this revision instead we focused on the 176 high-risk patients which are provided by clinical characteristics labeling from the CAMDA competition. And we recalculated the result showed substantial improvement over clinical staging. We have clarified this in the revision.Author’s response: About data preprocessing, were the microarray probes summarised at the gene level? If so, how? Parameter tuning in lmQCM was “based on previous work”, but this is unreferenced: Please provide a reference if available.We provided a reference to our previous paper in this revision.Author’s response: What about the rationale behind parameter tuning? Was it used in a similar condition? Was cross-validation used?Based on our extensive previous work, we have empirical knowledge about the range of four the parameters. We compared the different parameter in this range, lmQCM method used these parameters in the paper as they often led to balanced sizes of the gene modules with clear biological interpretations for individual modules.Author’s response: Regarding the parameter tuning in SNF: Did the authors try a grid search over the three SNF parameters, using cross-validation to evaluate the performance? How were the classification results evaluated in practice?We applied a grid search over the three SNF parameters.Author’s response: The references to figures in the main text are out of sync with the actual figure numbers, i.e. there are references up to Fig. We modified the figure captions.Author’s response: Please address minor typos such as missing spaces (as in the title of the methods section about SNF) and missing symbols (as the parameters in the SNF section). Some long sentences may be simplified  or the clinical stage information.” in conclusions).We corrected the typos and simplified long sentences.Author’s response: Major merit of the study is the originality of the used methodology in the context of the applied procedures and approaches for emerging the dependance between the co-expressed genes and the potential of survival time prediction of the patients studied. All those methodological steps are composed in a workflow which has a potential capacity to be used in another cancer studiesWe thank the reviewer for the encouraging comments on this work.Author’s response: The suggested approaches for data integration based on mining gene co-expression network (GCN) is known and already applied in the studies, but the problem here is related to the selection of features in the context how to build and how to apply such a model (i.e. GCN) my remarks here can be related not to the applied method but again to the “tuning” of initial parameters and the potential of possible validation of them. And finally the method of similarity network fusion (SNF) for merging the eigengenes and to test their potential for functional biomarkers drops in semantics of the results in particular to the poorly explained functional annotation through the gene ontology enrichment. As it was presented and described, the workflow demands some clarification in terms of functionality of each step in it as well the total idea for validation of the functionality of the prognosticated biomarkers concerning the risk assessment for the survival time of the studied patientsWe provided more clarification for the functionality of each step in the workflow.Author’s response: There are also some potential remarks in using “our recently developed wighted network mining algorithm” based on local maximum click optimisation - where is not so clear for the point of view of defining of some initial parameters and their comparabilityBased on our extensive previous work, we have empirical knowledge about the range of for the parameters. We compared the different parameter in this range, lmQCM method used these parameters in the paper as they often led to balanced sizes of the gene modules with clear biological interpretations for individual modules.Author’s response: The submitted material needs of a thorough revision in English - both grammar and morphology which will improve significantly the and semantics of sentences. The illustrations are possibly the most questionable part of the study. I think the authors can renew the design of some of the figures which can be related in quite better manner to the obtained results We checked the grammar and layout of the paper. Since Fig. 5 was confusing to readers, it was removed in the new version of the paper.Author’s response: The number and inclusion of references are limited and not enough for such an original workWe added more references to support our work.Author’s response: Conclusions are as well recommended to be corrected in the context of the suggested workflow and the completeness of the work provided by that workflowWe revised the description.Author’s response: Also avoiding for example such freely hanging phrases having obvious lack of comparability as “...which not only help achieve a more accurate survival prognosis...” will give the work better meritWe revised the text accordingly.Author’s response: There are some obvious errors in grammar - in particular in the use of complex sentences and verbs with different tenses. The style can be improved also as a result of correction of the text in the context of spelling and grammar.We checked the spelling and grammar and made revisions accordingly.Author’s response: The level of the submitted material will be improved significantly by renewing some of the graphics Since Fig. 5 was confusing to readers and was redundant to Fig. 2, it was removed in the new version of the paper.Author’s response: The data preprocessing and subsequent clusterization: Due to the highly unbalanced nature of the data there might be problems in defining categories as high or low risk. How the authors overcome the unbalancedness and the heterogenity of the data? Do the authors measure in someway the possible errors due to this problem?We thank the reviewer to point out the unbalanced data problem. If the reviewer refers to the clinical stage and clinical risk. Yes, there is unbalance issue. The number of patients labeled as stage 4 s and high risk are smaller/higher? (check it to be specific). However, we want to find survival-associated features. After we combined the deceased patients, the 105 patients deceased among total of 498 patients (21%), and among them, 92 patients are clinical high-risk in total of 176 clinical high-risk patients (55%). We think the sample sizes and proportions are appropriate for our statistical analysis. Furthermore, we used Regularized Cox proportional hazards model to calculate the risk indices of all patients. The median of risk indices of the training examples was used as a threshold to split patients into low-risk and high-risk groups. The same threshold was applied to classify the single held-out patient into one of the two groups, which means we were not using the same clinical categories as originally curated, which does not incur the unbalanced data issue. At last, we tested if these two groups have distinct survival outcome using Kaplan-Meier estimator and log-rank test. We divided patients into two groups (low and high group) where the median of each feature was used as a cut-off point. By using median as cutoff in the above two steps, we mitigated the unbalanced data issue in our survival association analysis.Author’s response: The suggested lmQCM approach for the purposes of defining GCN modules is interesting and having in mind some previous publications of the authors - it is a well tested method. However in the submitted material will be worth to explain what are exactly in this study the suggested four parameters Lambda, Alfa, t , and Beta. Definitely the fine tuning of these parameters can influence the final result in a large scale - it will be good to have authors explanation for these problems.Yes, as the reviewers pointed out, lmQCM has been applied to various types of cancer studies previously, and the meanings of the parameters were discussed in details in the previous publications [. To further explain them, we added the following section to the manuscript: There are four parameters for lmQCM: γ, λ, t, and β. Among them, γ controls the threshold for the initiation of each new module, λ and t define the adaptive threshold of the module density to ensure proper stopping criterion for the greedy search for each module, and β is the threshold for overlapping ratio for merging. We used the same settings for our GCN module mining as in [for those parameters, which have been proved to generate functionally meaningful modules from multiple cancer datasets.Author’s response: ications , 16. To The used Lasso-Cox model is a reasonable approach for defining the so-called risk index of the patients as it is given in the submitted material. The problem with such models as lasso regression  can arose when they are applied to multivariate space parameters. Although the reduced parameter space by the eigengenes give some relaxation of such models it will be worth to explain the options how to control the Lasso-Cox risk index estimates from certain bias and what is the best way to validate this process?We thank the reviewers to point out this. To address the problem of applying lasso regression to multivariate space, we used a two-level cross validation (CV) strategy. The first level was leave-one-out CV. Namely, a single patient was chosen as test set, with the rest as training set. Then in the training set, we performed 10-fold CV to select the best regularization parameter. Regularized Cox proportional hazards model was built on the training set using the selected parameter, and based on the model, risk indices of all patients were calculated.Author’s response: The data preprocessing and subsequent clusterization: The Gene Ontology enrichment analysis might be not the major objective of the study but it is presented in a very limited manner. Using only a single tool for enrichment from an external knowledge source provokes a lot of questions about the accuracy of the defining (co)-expressed genes and in particular the accuracy of their annotation. My suggestion is that such an ontology enrichment can be extended at least to the major knowledge sources as Gene Ontology, NCBI, other. This can open some parallel to the study problems but from other view angle can extend and enrich all the suggested workflow by the authors.The online gene list enrichment tool ToppGene (http://toppgene.cchmc.org) developed by Cincinnati Children’s Hospital Medical Center [was used for all of the module functional enrichment analysis. ToppGene not only carries out enrichment analysis on standard Gene Ontology, it also generates enrichment results from more than 20 different sources including pathway databases, human and mouse phenotypes, NCBI PubMed, transcription factor binding sites, and drug information. We clarified in the revision.Author’s response: l Center  was usedThe last two part of the results section “Survival-associated feature selection using lasso-regularized Cox proportional hazard model” and the next one “Prognostic prediction based on integrative analysis” are written mostly as material and methods part. There are some problems again how are selected the features for Lasso-Cox model. The selection and subsequent clusterization of the selected eigengenes for obtaining some confidential biomarkers possibly needs some more methodological work. Nevertheless it would be good to get some explanation by the authors about the methodological solution and the obtained results more clearly: why it was done in this way?We thank the reviewer’s comment, it helps for us to rethink and better elucidate our purpose of study. To address this, we moved part of contents of the Results section “Survival-associated feature selection using lasso-regularized Cox proportional hazard model” and the “Prognostic prediction based on integrative analysis” to the Materials and methods section. We also added the details of our method and written in a more methodological form to explain our workflow.Author’s response: Additional file 1:Table S1. The table shown Gene symbols in unique co-expression modules, a comparison was performed between each pair of modules from microarray and RNA-seq. 17 GCN modules from microarray and 10 from RNA-seq are unique to its own data type. (XLS 36 kb)Additional file 2:Table S2. Enrichment analysis for unique co-expression modules which are enriched with different biological processes, molecular functions, or specific pathways related to cancer physiology or to neurological functions (XLS 39 kb)Additional file 3:Figure S1. The concordance index of the co-expression 17 unique microarray modules obtained from the training set in three conditions: the training set, testing set, and randomly selected gene modules with equal number of genes in testing set. (ZIP 75 kb)Additional file 4:Figure S2. The concordance index of the co-expression 24 unique RNA-seq modules obtained from the training set in three conditions: the training set, testing set, and randomly selected gene modules with equal number of genes in testing set. (ZIP 101 kb)


We consider that this web server can help other researchers with no previous bioinformatic knowledge, to perform their analyses in a simple manner.The current DNA sequencing technologies and their high-throughput yield, allowed the thrive of genomic and transcriptomic experiments but it also have generated big data problem. Due to this exponential growth of sequencing data, also the complexity of managing, processing and interpreting it in order to generate results, has raised. Therefore, the demand of easy-to-use friendly software and websites to run bioinformatic tools is imminent. In particular, RNA-Seq and differential expression analysis have become a popular and useful method to evaluate the genetic expression change in any organism. However, many scientists struggle with the data analysis since most of the available tools are implemented in a UNIX-based environment. Therefore, we have developed the web server IDEAMEX . The IDEAMEX pipeline needs a raw count table for as many desired replicates and conditions, allowing the user to select which conditions will be compared, instead of doing all-vs.-all comparisons. The whole process consists of three main steps (1) Data Analysis: that allows a preliminary analysis for quality control based on the data distribution per sample, using different types of graphs; (2) Differential expression: performs the differential expression analysis with or without batch effect error awareness, using the bioconductor packages, NOISeq, limma-Voom, DESeq2 and edgeR, and generate reports for each method; (3) Result integration: the obtained results the integrated results are reported using different graphical outputs such as correlograms, heatmaps, Venn diagrams and text lists. Our server allows an easy and friendly visualization for results, providing an easy interaction during the analysis process, as well as error tracking and debugging by providing output log files. The server is currently available and can be accessed at  This approach has been used since the implementation of Northern blots hybridization analysis and was scaled up by the development of microarray technology. However, transcriptomics has been improved with the aid of sequencing technologies which recently have been replacing microarrays by using RNA sequencing (RNA-Seq) experiments to evaluate gene expression at a genome-wide scale. Therefore, either microarrays or RNA-Seq technologies have generated a massive amount of data results that demands To quantify the transcription levels and identify differential expressed genes under different conditions, using RNA-Seq data from high-throughput sequencing technologies, a general workflow can be described: (1) quality control of RNA-Seq reads ; (2) read trimming or filtering ; (3) mapThe gene expression profile contains the information regarding genes related to the organism response to a certain condition. To retrieve such information, the differential expression analysis has to be performed and it requires statistical methods to differentiate between expression changes due to the tested conditions and biological “noise” or variability. Currently, several computational tools have been developed mainly in the programming language R and packages are available at the Bioconductor project repository . HoweverHere, we introduce the IDEAMEX web server  that uses as input an RNA-Seq raw count table in text format and generates results using bioconductor packages NOISeq, limma-voom, DESeq2 and edgeR. These packages have been constanlty benchmarked and presented the most reliable results with different datasets and gold-standards . In thishttp://www.uusmb.unam.mx/ideamex/ where the documentation and example input files are provided. Our server offers a web server-based analysis that can help researchers with no previous bioinformatic knowledge, to perform their transcriptomic analyses in a simple manner, in order to interpret the biological data contained in their RNA-Seq experiments.Our server has been used in several projects and has been visited from different world-wide locations as recorded in our site tracker. IDEAMEX is available and can be accessed at The web page is hosted by the “Unidad Universitaria de Secuenciación Masiva y Bioinformática” core lab facility, at the “Instituto de Biotecnología” of the “Universidad Nacional Autónoma de México, Campus Morelos located in Cuernavaca, Morelos, México.” A Linux box computer with Ubuntu 14.04 LTS with the following hardware main characteristics: Intel Core i7 4770 processor; 32 Gbytes of DDR3 RAM and 1 Tbyte of hard disk storage.http://www.uusmb.unam.mx/ideamex/.The deployment was implemented using the Apache HTTP server version 2.4.7 with a PHP v5.5.9 front-end that coordinates the writing of the input and output files to a SQL database through a POSGRES Relational Data Base Manager (RDBM) server  (64-bit); Mozilla Firefox Quantum 63.0 (64-bit). Using MacOS Sierra 10.13.6: Safari 12.0.3; Google Chrome 71.0.3578.98 (64-bit). Using Linux Ubuntu 16.04 LTS: Mozilla Firefox Quantum 65.0.https://github.com/leticiaVega/IDEAMEXAdditionally, the scripts and binaries used in the web server can be found in the public repository Arabidopsis thaliana RNA-Seq data  CDS dataset downloaded from Ensembl repository database  and filtering for the raw data was performed using the FASTQC software  and contamination and adapter removal was carried out using in-house Perl scripts. QC’ed reads were mapped using the Bowtie 1.1.234 aligner  to the adatabase  using thdatabase . Total edatabase . The resBased on the parameters defined by the user, 4 different R (version 3.5.2) packages for differential expression analysis are run: edgeR version 3.24.3 , using TThe general workflow used in the IDEAMEX web server can be observed in Optionally, the user can perform a full registration at the IDEAMEX homepage, in order to keep track of all projects results. The sample name format should have a suffix_[0-9] structure: nameCond1_1, nameCond1_2, … , nameCond1_n, nameCond2_1, nameCond2_2, … , nameCond2_m. Once the input file is validated, the server can infer the replicates from the suffix before the underscore symbol and the replicate number will be the digit after the underscore symbol. However, during the input loading, the user can edit these names. In case of samples being prepared in different batches, this information can be specified in the same window the sample names are edited. Indicating samples in different batches will turn on the batch effect error correction of different methods. Importantly, use this option only if you have knowledge of samples from a given condition, being prepared in a different batch which can give the experiment an extra variability. The user manual has a case of study for samples with batch effect.In this work, the samples were named liverC_1, liverC_2 for replicates of control condition (no treatment) and liverT2_1, liverT2_2, liverT3_1, liverT3_2 for replicates that correspond to each of the 3,5-T2 and 3’,3,5-T3 (T2 and T3) thyroid hormones treatments. A raw count table , normalization (CPM) and transcript abundance difference (logFC) can be selected. Also, the comparison matrix can be defined to establish which samples or conditions will be compared.The next step is to select the differential analysis method(s), the data quality analysis and the result integration by clicking on each box. It is recommended to click on the “select all” box to perform a full analysis. Afterward, the cut-off values for statistical confidence  Data Analysis” section. A series of plots are displayed, allowing the user to have a preliminary analysis for quality control based on the data distribution per sample. All conditions defined in the raw count table are depicted as boxplots, CPM bar plots, density plots, principal components analysis (PCA) plots and multi-dimensional scaling (MDS) plots. Inspection and evaluation of these plots are essential steps for the interpretation of the differential expression analysis.In gene expression analysis, only a fraction of genes is expected to show differential expression between experimental conditions. The Count per million (CPM) plot shows the number of genes within each sample, having no counts (CPM = 0) or more than 1, 2, 5, or 10 CPM. This plot could help the user to decide the threshold to remove very low expressed genes in any of the experimental conditions. The default CPM cut-off value of 1 can be changed according to the user judgment, but it has to be done by re-running the analysis.As observed in M-values (TMM) method which is used for the differential expression analysis in edgeR and NOIseq packages. As observed, TMM normalization adjust the data according to the sequencing yield of each sample. The boxplot is an easy way to visualize the data distribution since it shows statistical measures such as median, quartiles, minimum and maximum values. Whiskers are also drawn extending beyond each end of the box with points beyond the whiskers typically indicating count outliers. In the log2 boxplot, the sequencing yield difference per sample is very evident. In this case, the control samples have fewer reads than the other samples. However, TMM normalization can fix this problem and this is why several differential expression methods have implemented this normalization procedure.It is important to mention that the user will find a pair of boxplots, PCA and MDS graphs, since the data is plotted using pseudo-counts and TMM values.The normalized count distributions can also be summarized by means of a density plot. Density plot provide more detail by enabling the detection of a dissimilarity in replicate count distribution. Ideally, the density plot for each replicate for a given condition, should greatly overlap indicating lower variability between replicates. This type of plot is useful for visualizing the overall effect of experimental covariates and batch effects. In the context of RNA-Seq analysis, PCA shows groups of samples that ideally will correspond to each condition. Clustering first by the most significant group, then by progressively less significant groups. Multi-dimensional scaling (MDS) is a technique that is used to create a visual representation of the pattern of proximities  among a set of objects. In the context of RNA-Seq analysis, MDS plot shows variation among RNA-Seq samples, the more is the distance between sample, the higher is their dissimilarity. Therefore, samples belonging to the same condition or treatment should be closer to each other and distant to other conditions. However, if different conditions are grouped together, this could mean that those treatments or conditions have a very similar effect. Worst-case scenario, the user can suspect of a sample mislabeling. Conceptually, MDS and PCA plots can provide the same information and as observed in The “(2) Differential Expression Results” section has links with the name of each selected method, where the user can display the analysis output. A detailed description of each method output can be found in the User Manual at the IDEAMEX web page. However, here we describe the generated graphs for a better interpretation. These plots depict all expressed genes but those with differential expression are represented in other color than black. Basically, in all of them we can see the distribution of the gene expression according to a certain value. For example, in the expression plot  the averIn the MA-plot , the logy-axis in limma or M for NOISeq). Again, values declared as differentially expressed are highlighted in red.The mean-difference (MD) plot  shows thThe smear plot allows to visualize the results of the analysis in a similar manner to the MA-plot, this plot shows the logFC against log-CPM, where genes declared as differentially expressed highlighted in red.In summary, all these plots compare the expression rate or difference between conditions and the normalized values. The proportion of black and highlighted dots gives an idea of the expression change magnitude between the treatment and the control or untreated conditions.p-value). It is a scatter-plot of the negative log10-transformed p-values from the gene-specific test (on the y-axis) against the logFC (on the x-axis). The graph depicts datapoints with low p-values (highly significant) appearing toward the top of the plot. The logFC values are used to determine the change direction (up and down) appearing equidistant from the center. Features declared as differentially expressed are highlighted in red, according to the selected cut-off values.Arguably, the volcano plot  is the mFinally, the “(3) Results Integration” section of the Analysis Results in the IDEAMEX web page contains several text files and graphs that integrates the results from all selected methods. In p-adj, FDR or Probability) to determine if a gene is differentially expressed, the correlograms can help the user to evaluate the correlation of these values among the different used methods. Also, heatmaps are created to observe samples clustered by their fold change, allowing the user to spot groups of genes with a similar expression change.As mentioned, other generated plots are heatmaps  and corrAmong all the text file results that are explained in detail in the User Manual . The comparison between T2 and T3 has to be performed by comparing the results from comparing each one to the control or untreated condition. A direct comparison between T2 and T3 could miss several results since even if we can observe a gene with a certain expression change, the difference could not be statistically significant. Let’s say that “gene A” has a differential expression of 10 times in T2 vs. C comparison and of 12 times in the T3 vs. C comparison. Roughly, the difference between T2 vs. T3 comparison for the same gene, will be 2 times which might not be statistically significant. For this reason, is very important to select the which comparisons make sense, instead of performing all possible comparison.The results in the “Data Analysis” section, are several plots that allow the user to inspect the distribution of their data based on different metrics. This quality control check point is very important, since biological data tend to be very noisy. It is expected that the data from biological replicates within a certain condition, will have the same distribution and a similar trend than those in other conditions. In particular, PCA and MDS plots allow the users to see if biological replicates of a certain condition are grouped together and if each condition forms a separate group. In this particular case, it was known that the samples didn’t present any batch effect but as observed in The distance or dispersion of the replicates and groups indicates how reproducible was the tested condition in different individuals or how variable were individuals despite the treatment. The more replicates available, the better statistical significance is observed. Having very disperse groups or samples from different conditions grouping together, should be considered as noisy or highly variable results that can skew the analysis and lead to misinterpretation of the experiments. However, NOIseq could be a good option when no biological replicates are available and as reported elsewhere, it delivers reliable results that have been confirmed by using quantitative PCR (qPCR) reactions.The results from different methods are not mutually exclusive. From the statistical point of view, one of them, neither or all may be true. Therefore, working with the intersection or the union of all results is a decision that the user has to evaluate after exploring them based not only on the statistical significance but on the biological meaning that will depend on the gene annotation. The main problem with all statistics is the “fakeness” and misrepresentation of the results. However, if four different methods agreed with a certain result it could be assumed that those genes are differentially expressed, bearing in mind that an experimental orthogonal validation using a different technology like qPCR, should be necessary to confirm the result.In the “Results Integration” section, there are several text lists and graphs that can guide the users to make sense out of the results from their experiments. As mentioned, the Venn diagram  shows thIn this work, we provide heatmaps and correlograms for different values obtained from each method. For example, heatmaps  are usefFinally, there are several other methods to continue the differential expression analysis, that can help users to put their results in a certain biological context. Probably the most popular methods are those based on Gene Ontology (GO) terms enrichment  which wiWe consider that the IDEAMEX web server can help other researchers with no previous bioinformatic knowledge, to perform their analyses in a simple manner. Also, more experienced users with some bioinformatics skills can use the results and perform a more detailed analysis and a different integration of them, since all the results are provided in simple text files which are very convenient to parse and handle using regular expression searches.https://submit.ncbi.nlm.nih.gov/subs/sra/), under the SRA identifiers: SRX2630485, SRX2630486, SRX2630487, SRX2630488, SRX2630489, and SRX2630490.The datasets analyzed for this study can be found in the NCBI SRA repository (VJ-J and LV-A developed the web deployment and scripts for the IDEAMEX server. VJ-J, LV-A, and AS-F conceived the web server workflow. AS-F wrote the manuscript. All authors read and authorized the publication of this manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


ANPEP, mir-217, mir-592, mir-6715b) were found to distinguish between the two PCa stages and were highly correlated (average r = ± 0.75) with corresponding aggressiveness-related imaging features in both tumor stages. When combined with related clinical features, these biomarkers markedly improved the prediction accuracy for the pathological stage. Our prediction model exhibits high potential to yield clinically relevant results for characterizing PCa aggressiveness.Prostate cancer (PCa) is a genetically heterogeneous cancer entity that causes challenges in pre-treatment clinical evaluation, such as the correct identification of the tumor stage. Conventional clinical tests based on digital rectal examination, Prostate-Specific Antigen (PSA) levels, and Gleason score still lack accuracy for stage prediction. We hypothesize that unraveling the molecular mechanisms underlying PCa staging via integrative analysis of multi-OMICs data could significantly improve the prediction accuracy for PCa pathological stages. We present a radiogenomic approach comprising clinical, imaging, and two genomic (gene and miRNA expression) datasets for 298 PCa patients. Comprehensive analysis of gene and miRNA expression profiles for two frequent PCa stages (T2c and T3b) unraveled the molecular characteristics for each stage and the corresponding gene regulatory interaction network that may drive tumor upstaging from T2c to T3b. Furthermore, four biomarkers ( Prostate cancer (PCa) is the second most common cancer and affects millions of men every year ,2. ArounThe prognosis and determination of best treatment strategies for PCa patients depend on the correct estimation of PCa TNM (Tumor-Node-Metastasis)-stages, based on the universal TNM tumor stage classification, which refer to the degree by which cancer has spread inside the prostate, to the nearby tissues such as seminal vesicles and bladder, and beyond . TraditiThe ability to predict the pathological PCa stage clinically as precisely as possible would enable clinicians to better determine optimal treatment strategies. Furthermore, PCa patients would benefit from avoiding the potential side effects that are associated with surgical overtreatment ,10. BaseRecent advances in high-throughput Omics and imaging technologies and their applications in precision medicine have created new avenues for better characterizing cancer stages and for detecting molecular biomarkers for early diagnosis and prognosis ,21. For ANPEP, mir-217, mir-592, mir-6715b), which markedly improved the prediction accuracy for pathological stage, when combined with conventional clinical features. Notably, these biomarkers are highly correlated with aggressiveness-related imaging features such as the tumor volume intensity and the tumor texture features. Furthermore, we unraveled the potential molecular interactions underlying PCa progression/upstaging from stage T2c to stage T3b via construction of a PCa-GRN network and highlighted four central hub miRNAs that could conceivably drive the upstaging process between these two stages. This proof of principle study demonstrates a plausible association of carcinogenic gene and miRNA expression with PCa upstaging and progression, establishes the potential of improving the prediction accuracy of pathological stages of PCa patients via integrating clinical and molecular features, and finally sheds light on the promise of integrating imaging with molecular and clinical data (radiogenomics) for clinical decision-making.In this study, we performed a comprehensive integrated analysis of gene and miRNA expression profiles for PCa patients with the two frequently reported yet highly distinctive clinical stages T2c and T3b. We characterized the exclusively differentially expressed genes and miRNAs, and the associated functional terms and pathways for each stage. We further identified candidate biomarkers  and two types of transcriptomic data  for 298 primary PCa tumor cases (T2c and T3b) and 52 healthy cases. The clinical and pathological characteristics of the PCa tumor cohort are listed in Differential expression (DE) analysis resulted in 127 genes and 5 miRNAs that characterize  the T2c stage. This was done by comparing the T2c samples to the healthy samples and selecting the DE genes/miRNAs which were not identified in the comparison of the T3b samples to the healthy samples, nor identified in the comparison of all tumor samples to all healthy samples, see mir-592, mir-587, mir-147, mir-661) that dominate the PCa-GRN network and maintain the interactions between these miRNAs and their neighboring genes. Hence, they could act as driver miRNAs and genetic regulators for PCa progression across the two stages. Remarkably, the aberrant expression patterns of the 4 candidate driver miRNAs have been connected to pathogenesis of various cancer types such as breast carcinoma and colorectal cancer via regulating cancer–specific pathways such as AKT/mTOR signaling pathways [Next we performed differential expression analysis of the gene and miRNA expression data between the two stages. This yielded 125 differentially expressed (DE) genes and 19 DE miRNAs. We constructed a GRN network that combines transcriptional and post-transcriptional regulatory interactions between the DE genes and DE miRNAs (see Methods). We refer to this network as the PCa-GRN network although it represents the progression between two PCa stages only. The PCa-GRN network comprises nine miRNAs regulating 30 target genes, see pathways ,39. Howep-values < 1.2 × 10−4, Kolmogorov–Smirnov test), see Furthermore, we evaluated the biological evidence for the PCa-GRN network in more depth to better assess the functional integrity of the biological processes underlying the etiology of PCa progression and upstaging. We estimated the functional homogeneity of the PCa-GRN genes and the target genes of the nine miRNAs by calculating the functional similarity scores between all gene pairs and comparing the resulting distribution to the similarity score distribution of randomly selected gene pairs from the PCa-GRN network. Interestingly, the PCa-GRN genes have significantly more cellular functional homogeneity than randomly selected ones  with other features from DNA methylation, somatic mutations, and protein expression levels of PCa samples, see ANPEP was inversely proportional to the hypermethylation of many differentially methylated regions (DMRs) and CpG islands  in T2c patients, and max r = 0.94 and min p-value = 0.025 in T3b patients. Significant correlations are marked by an asterisk. It is also noteworthy that the C3 features exhibited significant differential correlation between T2c patients T3b patients. For the T3b patient group, the C3 features (except the entropy) were found to be much better than the C2 features in correlating with the biomarker expression signatures. This also matches the findings about the plausible diagnostic power of the C3 feature class especially in advanced PCa stages [To assess the biological relevance of our candidate biomarkers with respect to PCa progression and upstaging, we investigated their correlation to aggressiveness-related imaging features extracted from corresponding MRI images of PCa patients with the two stages under investigation T2c and T3b), see Methods. We extracted the feature classes C2 and C3 which are related to PCa aggressiveness and upstaging as described by Stoyanova et al., 2016 [b, see Mea stages .We randomly partitioned the data into two stratified sample sets, the training data (70%) and the test data (30%) and performed the biomarker identification again only on the training dataset (the 70% of the data). We repeated the aforementioned procedure for multiple runs (10 times), see Methods. In fact, the set of biomarkers did not change and we explain this invariance by the strict selection criteria (2.5 fold and FDR of 0.001) we used for identifying them, and we also note their power in separating the two classes as shown by the PCA analysis in ANPEP, mir-217, mir-592, mir-6715b), and (3) the combined set . We used three machine-learning methods to ensure the reliability of the prediction efficacy of all used features. Theoretical backgrounds about the three used machine-learning methods and the parameters used for training and testing processes are described in full details in the In order to evaluate our candidate biomarkers as diagnostic features for predicting the corresponding pathological stages, we used three machine-learning methods  and Random Forest) on three feature sets: (1) the clinical features only , (2) the molecular features only  with an AUC prediction accuracy of 0.82 [The results revealed that the predictions using the molecular features only , thereby outperforming the prediction models based on either clinical or molecular feature sets separately. Hence, this highlights the potential usefulness of combining features from heterogeneous datasets to achieve better prognosis for PCa patients.https://portal.gdc.cancer.gov), namely the TCGA-PRAD project. For consistency, we only considered matching samples, which were common between the two datasets . Each TCGA sample refers to a tissue biopsy taken from a unique individual. This resulted in a total of 546 samples composed of 52 healthy control samples and 496 tumor samples with different tumor stages. The matching imaging traits for the examined tumor stages  were obtained from the Cancer Imaging Archive (TCIA) [RNA-Seq, miRNA-Seq, and clinical data for normal and prostate tumor samples were downloaded from the GDC data portal (e (TCIA) , i.e., ae (TCIA) . The datThe gene and miRNA expression datasets were obtained in raw read counts and were consequently normalized, corrected for library size, and log2-transformed using the Bioconductor  package The clinical data were also normalized using quantile normalization  to removThe MRI imaging data for the examined two stages were subjected to median filters for noise reduction and further segmented using the NIH ImageJ software  to deterThe DESeq2 Bioconductor package  was usedThe molecular interactions between the differentially expressed (DE) genes and the DE miRNAs were compiled from the regulatory databases of the TFmiR webserver . We consIn order to assess the biological relevance of the identified molecular interactions to PCa phenotype (here represented by tumor progression), we used the GoSemSim R package  to estimThe functional annotation tool DAVID  was usedThe pairwise correlations between the expression signatures of the identified biomarkers and the extracted aggressiveness-related imaging features was performed using Pearson correlation. The significance was computed using the R method cor.test followed by the Benjamini–Hochberg adjustment for controlling the false discovery rate (FDR), with a cutoff value of 0.05.We performed multiple runs (10×) with random data partitions (70% of the data for training and 30% for testing). The sampling of all partitions was stratified, so that the distribution of the two classes is proportional to the original distribution in the whole dataset. The molecular biomarkers were then identified by differential expression, see “Differential Expression and Association Analysis”, above, with the strict thresholds of at least 2.5-fold changes and FDR < 0.001. The molecular biomarkers and/or the clinical features already available, were then used to train three machine-learning methods, Naïve Bayes, Support-Vector Machine (SVM) and Random Forest. Model training and data partitioning were performed using the R caret package  with theThis whole learning process was performed for every method after removing the NA entries from all datasets. Classifiers were trained based on three datasets; the clinical features only, the molecular features only and finally the combined set of both clinical and molecular features, resulting in nine classifiers that were trained and tested. The AUC (Area Under Curve) characteristic was used as the evaluation metric for the prediction results. We used the pROC package  for ROC The current study presented an integrated regulatory analysis of gene and miRNA expression data for PCa samples to unravel molecular features, related GO functional terms, and pathways underlying PCa progression, and to identify potential biomarkers that can distinguish different PCa stages. The biomarkers we found belong to genes and miRNAs that play critical roles in PCa and other cancer types and showed high correlation with aggressiveness-related imaging features extracted from mp-MRI images. When combined with the traditional clinical features and using the power of machine learning, these biomarkers were able to improve the prediction accuracy of the corresponding PCa pathological stages.To this end, future research work is demanded for predicting the best treatment strategy, such as chemotherapy, radiotherapy, or surgery. Other future research includes the development of similar integrative approaches such as those based on patient similarity networks that better take into consideration the molecular heterogeneity of PCa. This is actually one major limitation of our approach. Another limitation is the insufficient number of imaging samples that prevented us from combining the imaging features together with clinical and molecular features for enhancing the prediction of the PCa stages. If more imaging samples were provided for the different PCa stages, then such an analysis would have been easily possible. Further joint analysis of the associated molecular features  with the four identified biomarkers, as well as wet- lab experiments may enable to characterize more clinically relevant OMICs features that can potentially be used for diagnosis and prognosis of PCa. Finally, our approach can be applied to other cancer types or complex diseases with progressive stages and might be also extended for studying cellular developmental stages as well.