Single-cell RNA-seq (scRNA-seq) data simulation is critical for evaluating computational methods for analysing scRNA-seq data especially when ground truth is experimentally unattainable. The reliability of evaluation depends on the ability of simulation methods to capture properties of experimental data. However, while many scRNA-seq data simulation methods have been proposed, a systematic evaluation of these methods is lacking. We develop a comprehensive evaluation framework, SimBench, including a kernel density estimation measure to benchmark 12 simulation methods through 35 scRNA-seq experimental datasets. We evaluate the simulation methods on a panel of data properties, ability to maintain biological signals, scalability and applicability. Our benchmark uncovers performance differences among the methods and highlights the varying difficulties in simulating data characteristics. Furthermore, we identify several limitations including maintaining heterogeneity of distribution. These results, together with the framework and datasets made publicly available as R packages, will guide simulation methods selection and their future development. Simulation is useful for developing and evaluating computational methods. Here, the authors develop a comprehensive evaluation framework, SimBench, to benchmark Single-cell RNA-seq simulation methods through a diverse collection of experimental datasets. To effectively utilise scRNA-seq data to address biological questions2, the development of computational tools for analysing such data is critical and has grown exponentially with the increasing availability of scRNA-seq datasets. Evaluation of their performance with credible ground truth has thus become a key task for assessing the quality and robustness of the growing array of computational resources. While there exist certain control strategies such as spike-ins with known sequence and quantity, data that offer ground truth while reflecting the complex structures of a variety of experimental designs are either difficult or impossible to generate. Thus, in silico simulation methods for creating scRNA-seq datasets with desired structure and ground truth (e.g. number of cell groups) are an effective and practical strategy for evaluating computational tools designed for scRNA-seq data analysis.Single-cell RNA-sequencing (scRNA-seq) is a powerful technique for profiling the transcriptomes at the single-cell resolution and has gained considerable popularity since its emergence in the last decade5 as it has been the typical choice for modelling gene expression count of RNA-seq6. Its variant, zero-inflated NB (ZINB) model takes account of excessive zeros in the count data and is chosen by other studies to better model the sparsity in single-cell data8. In more recent years, alternative models have been proposed with the aim to increase modelling flexibility including gamma-normal mixture model9, beta-Poisson10, gamma-multivariate hypergeometric11 and the mixture of zero-inflated Poisson and log-normal Poisson distributions12. Other studies argued that parametric models with strong distributional assumption are often not appropriate to scRNA-seq data given its variability and proposed the use of a semi-parametric approach as the simulation framework13. Similarly, a recent deep learning-based approach14 leverages the power of neural networks to infer underlying data distribution and avoid prior assumptions.To date, numerous scRNA-seq data simulation methods have been developed. The majority of these methods employ a two-step process of using statistical models to estimate the characteristics of real experimental single-cell data and using the learnt information as a template to generate simulation data. The distinctive difference between them is the choice of underlying statistical framework. Early methods often employ negative binomial (NB)15. Given that simulation datasets are widely used for the evaluation and comparison of computational methods16, deviations of simulated data from properties of experimental data can greatly affect the validity and generalisability of the evaluation results. With the increasing number of scRNA-seq data simulation tools and the reliance on them to guide other method development as well as choosing the most appropriate data analytics strategy, a thorough assessment of all currently available scRNA-seq simulation methods is crucial and timely, especially when such an evaluation study is still lacking in the literature.A common challenge of simulation methods is the ability to generate data that faithfully reflect experimental data17 in the evaluation framework to enable the large-scale quantification and comparison of similarities between simulated and experimental data across univariate and multivariate distributions, and thus, avoid visual-based criteria which are often used in other studies. To assist development of new methods, we study potential factors affecting the simulation results and identify common strength and weakness of current simulation methods. Finally, we summarise the result into recommendation to the users, and highlight potential areas requiring future research.Here, we present a comprehensive evaluation framework, SimBench, for single-cell simulation benchmarking. Considering that realistic simulation datasets are intended to reflect experimental datasets in all data moments including both cell-wise and gene-wise properties, as well as their higher-order interactions, it is important to determine how well simulation methods represent all these values. To this end, we systematically compare the performance of 12 simulation methods across multiple sets of criteria, including accuracy of estimates for 13 data properties, the ability to retain biological signals and to achieve computation scalability, as well as their applicability. To ensure robustness of the results, we collect 35 datasets across a range of sequencing protocols and cell types. Moreover, we implement measure based on kernel density estimationOur SimBench framework evaluates 12 recently published simulation methods specifically designed for single-cell data Fig. . Using f18. Supplementary Fig. 17 (KDE statistic), in our current study as it is applicable to both univariate and multivariate distributions.The first set of evaluation criteria, termed data property estimation, aims to assess how realistic is a given simulated data. To address this, we first defined the properties for a given dataset with 13 distinct criteria and then developed a comparison process to quantify the similarity between the simulated and real data  genes obtained in the simulated data using DE detection methods designed for bulk and single-cell RNA-seq data, as well as four other types of gene signals of differentially variable (DV), differentially distributed (DD), differential proportion (DP) and bimodally distributed (BD) genes (see ‘Methods’). A similar proportion to the real data would indicate an accurate estimation of biological signals present in the data. Scalability reflects the ability of simulation methods to efficiently generate large-scale datasets. This is measured through computational run time and memory usage with respect to the number of cells. Applicability examines the practical application of each method in terms of whether it can estimate and simulate multiple cell groups and allow simulation of differential expression patterns. Overall, our framework provides recommendations by taking into account all aspects of evaluation Fig. .Through ranking the 12 methods on the above four sets of evaluation criteria, we found that no method clearly outperformed other methods across all criteria  that were not ranked the highest in data properties estimation performed well in retaining biological signals Fig. . scDesig19 with 50–8000 cells (see ‘Methods’). However, some top performing methods, such as SPsimSeq and ZINB-WaVE revealed poor scalability  . Through regression analysis, we found certain data properties such as mean–variance relationships were more accurately estimated under datasets with larger numbers of cells, as shown by the positive regression coefficients Fig. . Neverth20. Figure To examine the impact of sequencing protocols, we utilised datasets consisting of multiple protocols applied to the same human PBMC and mouse cortex samples from the same studyWhile the key focus of our benchmark framework is assessing methods’ performance across multiple criteria, we can further use these results to identify criteria where most methods performed well or were lacking Fig. . CompariFirst, we compared the accuracy of maintaining each data property, where a larger KDE score indicates greater similarity between simulated and real data. Figure The ability to recapture biological signals was quantified using the metric symmetric mean absolute percentage error (SMAPE), where a score closer to 1 indicates greater similarity between simulated and real data (see ‘Methods’). We found differentially distributed (DD) and differential proportion (DP) genes exhibited a greater difference between simulated and real data Fig. . We alsoWe presented a comprehensive benchmark study assessing the performance of 12 single-cell simulation methods using 35 datasets and a total of 25 criteria across four aspects of interest. Our primary focus was on assessing accuracy of data property estimation and various factors affecting it, ability to maintain biological signals and computational scalability, as well as applicability. Additionally, using these results we also identified common areas of strength and weakness of current simulation tools. Altogether, we highlighted recommendations for method selection and identified areas of improvement for future method development.21 to capture unknown cell-level covariates. Therefore, while both powsimR and ZINB-WaVE use ZINB distribution to fit the count data, the actual model differs. Interestingly, while deep learning methods have dominated various fields and applications, cscGAN, a deep learning-based model, for scRNA-seq data simulation only had moderate performance compared to the other models. This may be due to the large number of cells required for training the deep neural network in cscGAN as was demonstrated in their original study14.We found that various underlying models were used for different simulation methods Table . Each ofBased on the experiments conducted, we identified several areas of exploration for future researchers. Maintaining a reasonable amount of biological signal is desirable and was not well captured by a number of methods. We also observed the genes generated by some methods Table  were ass22 can help to extend the use of simulation methods.As expected, we identified that none of the simulation methods assessed in this study could maintain the heterogeneity in cell population that was due to patient variability. This is potentially related to the paradigm used by current simulation techniques, as some methods implicitly require input to be a homogeneous population. For instance, some simulation studies inferred modelling parameters and performed simulation on each cell type separately when the reference input contains multiple cell types. However, experimental datasets with data from multiple samples, for example multiple patients, would be characterised by sample-to-sample variability within a cell type. This cellular heterogeneity is an important characteristic of single-cell data and has key applications such as identification of subpopulations. The loss of heterogeneity can thus be a limiting factor, as in some cases the simulation data could be an oversimplified representation of single-cell data. Future research such as phenotype-guided simulationFinally, we found there exists various trade-offs between the four aspects of criteria and having a well-rounded approach could be more important than a framework that performs best on one aspect but limiting in the other aspects. For example, as single-cell field advances and datasets with hundreds of thousands of cells become increasingly common, users may be interested in simulating large-scale datasets to test the scalability of their methods. As a result, methods that rank highly on scalability while also performing well on other aspects  may be more favourable than other methods under these scenarios. We also note that due to the primary intended purpose of each method, not all methods allow users to customise the number of cell groups and the amount of differential expression between groups. Method that offers a well-rounded approach across multiple aspects of interests is therefore a direction of future research.23 is able to simulate regulation of genes by transcriptional factors, and therefore requires gene regulatory networks as one of the inputs. Both PROSSTT24 and dyngen25 are designed to simulate scRNA-seq data with trajectory information and require user-defined lineage trees. Lun’s26 was originally designed to tackle confounding plate effects in DE analysis and it requires plate information to be specified in the input. These simulation methods may need special considerations and evaluation criteria that could not be captured by the general framework in this study. Although the choice of DE detection methods could affect the evaluation of the simulation methods, our evaluation using both limma, a DE method originally designed for bulk RNA-seq data, and DEsingle, a DE method specifically designed for scRNA-seq data demonstrate a high agreement of the rankings of simulation methods based on the two DE methods . SimBench allows any new simulation methods to be readily assessed under our framework. It requires two inputs including the simulated data generated by any simulation method and the real data that was used as the reference input to generate the simulated data. SimBench then runs the evaluation procedure as performed in this study. We also provide all datasets used in this study as a Bioconductor data package SimBenchData (https://bioconductor.org/packages/devel/data/experiment/html/SimBenchData.html). Together these two packages enable future simulation methods to be assessed and compared with the methods benchmarked in this study. Additionally, we provide a Shiny application for interactively exploring the results presented in this study hosted at http://shiny.maths.usyd.edu.au/. The application allows users to select datasets of their interest, such as within a certain range of cell numbers, and examine methods performance based on the specified datasets. Furthermore, we will provide updates to the website to include the benchmark results from new simulation methods when they become available so that our comparative study will stay up-to-date and will support future method development.In conclusion, we have illustrated the usefulness of our framework by summarising each method’s performance across different aspects to assist with method selection for users and identify areas of further improvement for method developers. We advise users to select the method that offers the functionality best suited to their purpose and developers to address the limitations of current methods. The evaluation framework has been made publicly available as the R package SimBench  count matrix was obtained from each study and quality control was performed by removing potentially low-quality cells or empty droplets that expressed less than one percent of UMIs. For methods that require normalised count, we converted the raw count into log2 counts per million reads (CPM), with addition of pseudocount of 1 to avoid calculating log of zero.Note the Tabula Muris dataset was only used to benchmark speed and scalability of methods. Properties estimation was evaluated on all other datasets. For evaluating biological signals, 25 datasets containing multiple cell types or conditions as specified by Supplementary Data An extensive literature review was conducted and a total of 12 published single-cell simulation methods with implementation available in R and Python was found. The details of each method, including the version of the code used in this benchmark study and its publication are outlined in Table To ensure the simulated data is not simply a memorisation of the original data, we randomly split each dataset into 50% training and 50% testing . The training data was used as input to estimate model parameters and generate simulated data. The real data was used as the reference to evaluate the quality of the simulated data, by comparing the similarity between the simulated data and the real data. The same training and testing subset was used for all methods to avoid the data splitting process being a confounding factor in evaluation.All methods were executed using a research server with dual Intel(R) Xeon(R) Gold 6148 Processor . For methods that support parallel computation, we used 8 cores and stopped the methods if the simulation was not completed within 3 h. For methods that run on a single core, we stopped the methods if not completed within 8 h.18, which is an R package developed to evaluate the similarities between two RNA-seq datasets, either bulk or single-cell and evaluated a total of 13 data properties across univariate and bivariate distribution. They are described in detail below:Library size: total counts per cell.M-values normalisation factor28.TMM: weighted trimmed mean of Effective library size: library size multiplied by TMM.z-score standardisation of the variance of gene expression in terms of log2 CPM.Scaled variance: Mean expression: mean of gene expression in terms of log2 CPM.Variance expression: variance of gene expression in terms of log2 CPM.Fraction zero cell: fraction of zeros per cell.Fraction zero gene: fraction of zeros per gene.Cell correlation: Spearman correlation between cells.Gene correlation: Spearman correlation between genes.Mean vs variance: the relationship between mean and variance of gene expression.Mean vs fraction zero: the relationship between mean expression and the proportion of zero per gene.Library size vs fraction zero: the relationship between library size and the proportion of zero per gene.We adapted the implementation from countsimQC (v1.6.0)Note that properties relating to library size, including TMM and effective library size can only be calculated using unnormalised count matrix and could not be obtained from methods that generate normalised count. As a result, these scores were shown as a blank space in all relevant figures.17 (KDE test) to compare the data properties between simulated and real data. The discrepancy between two distributions is calculated based on the difference between the probability density functions, either univariate or multivariate, which are estimated via kernel smoothing.In this study, we used a non-parametric measure termed kernel density based global two-sample comparison testf1f2 are the kernel density estimates of sample 1 and sample 2, respectively. The implementation from the R package ks (v1.10.7) was used for the KDE test performed in this study.The null hypothesis of the KDE test is that the two kernel density estimates are the same. An integrated squared error (ISE) serves as the measure of discrepancy and is subsequently used to calculate the final test statistic under the null hypothesis. The ISE is calculated as:x is the raw value before transformation. The transformation is applied on the KDE scores obtained from all methods across all datasets, thus the xminimum and xmaximum are defined based on those values. The purpose of the transformation is to follow the principle of, the higher the value the better and enable easier interpretation.We used the test statistic from the KDE test as the measure to quantify the extent of similarity between simulated and real distributions. We applied a transformation rule by scaling the absolute value of the test statistic to  and then taking 1 minus the value as shown in the equation below:To assess the validity of the KDE statistic and compare it against other measures, for example, the well-established KS test for univariate distribution, we utilised the measures implemented in countsimQC package. It includes the implementation of the following six measures: Average silhouette width, average local silhouette width, nearest neighbour (NN) rejection fraction, K-S statistics, scaled area between empirical cumulative distribution functions (eCDFs) and Runs statistics. For ease of comparing between the six measures and with the KDE test, we applied transformation rules where applicable such that the outputs from all measures are within the range of 0–1, where value closer to 1 indicates greater similarity. Similarly, the transformation is calculated from all methods across all datasets.Average silhouette widths(i) was then calculated using the following formula:b(i) is the mean distance between feature i and all other features in the simulation data, a(i) is the mean distance between feature i and all other features in the original dataset.For each feature, the Euclidean distances to all other features were calculated. The feature was either gene or cell, depending on the data properties evaluated. A silhouette width s(i) of all features is then averaged to obtain the average silhouette width. The range of silhouette width is . A positive value close to 1 means the data point from the simulation data is similar to the original dataset. Value close to 0 means the data point is close to the decision boundary between the original and simulated. A negative value means the data point from the original dataset is more similar to the simulation data. The same transformation as described in Eq.  statisticThe K-S measure is based on K-S statistic obtained from performing K-S test, which measures the absolute max distance between the eCDFs of simulated and real dataset. The K-S statistics is in range . The K-S measure was obtained by log-transformation followed by the transformation rule defined previously.Scaled area between empirical cumulative distribution functions (eCDFs)The difference between the eCDFs of the properties in simulated and real dataset. The absolute value of the difference was then scaled such that the difference between the largest and smallest value becomes 1. The area under the curve was calculated using the Trapezoidal Rule. The final value is in the range of , where a value closer to 1 indicates greater differences between the data properties distributions of the real and simulation data. The value was then reversed by taking 1 minus the value such that it follows the general pattern of higher value being better.Runs statistics. The Runs statistics is the statistic from a one-sided Wald-Wolfowitz runs test. The values from the simulated and real dataset were ordered and a runs test was performed. The null hypothesis is that the sequence is a random sequence with no clear pattern of values from simulated or real dataset next to each other in position.The measures and their transformation rules are:In order to summarise the results from multiple datasets and multiple criteria, we implemented the following multi-step procedure to aggregate the KDE scores.n is the number of cell types in the simulated or original datasets, xi is the evaluation score of the ith cell type and wi is the cell type proportion of the ith cell type.First, we aggregated the KDE scores within each dataset. For most methods, each cell type in a dataset containing multiple cell types was simulated and evaluated separately for the reason mentioned in the previous section. This resulted in multiple KDE scores for a single dataset, one for each cell type. To aggregate the scores into a single score for a dataset, we calculated the weighted sum by using the cell type proportion as weight, defined as the follows:Since each method was evaluated using multiple datasets, we then summarised the performance of each method across all datasets by taking the median score. This resulted in a single score for each method on each criterion, which then enabled us to readily rank each method by comparing the score. Cases where a method was not able to produce result on particular dataset were not considered in the scoring process. The reasons for failing to simulate a data include not completing the simulation in the given time limit, error arising in the simulation methods during the simulation process, and special cases in which the simulation method is limited to an input dataset containing two or more cell types and cannot generate result on datasets containing a single cell type. The breakdown of the number of datasets successfully simulated and the number of failed cases are reported in detail in Supplementary Fig. Finally, the overall rank of each method was computed by firstly calculating its rank for each criterion and then taking the mean rank across all criteria.29 and their descriptions are detailed below.DE (limma)30 was performed to obtain the log fold change associated with each gene. We selected genes with log2 fold change > 1.This is the typical differentially expressed genes. LimmaDE (DEsingle)31 that is specifically designed for scRNA-seq data.This finds the differentially expressed genes using a DE detection method DEsingleDVP-value associated with each gene.DV stands for differentially variable genes. Bartlett’s test for differential variability was performed to obtain the DDP-value associated with each gene.DD stands for differentially distributed genes. K-S test was performed to obtain the DPDP is defined as differential proportion genes. We considered genes with log2 expression greater than 1 as being expressed and otherwise as non-expressed. A chi-square test was then performed to compare the proportion of expression of each gene between two cell types.BDThe five categories of biological signals evaluated in this study were adapted fromm1 and m2 are the mean expression of genes in the two cell types, respectively, s is the standard deviation and p is the proportion of cells in the first cell type.BD is defined as bimodally distributed genes. Bimodality index defined using the below formula was calculated for each gene:P-value < 0.1 (Benjamini-Hochberg adjusted) were selected. This higher threshold was used instead of the typical threshold of 0.05 to result in a higher proportion of biological signals, as larger value would enable clearer differentiation of methods’ performance. For the last category, we used bimodality index32 >0.03 as the cut-off to yield a reasonable proportion of BD genes (Supplementary Fig. For the first five categories, genes with 33:Ft is the proportion of biological signals in simulated data and At is the proportion in the corresponding real data, n is the number of data points, one from each dataset evaluated. The proportion was calculated as the number of biological signal genes divided by the total number of genes in a given dataset.To quantify the performance of each method, we used SMAPETo reduce potential confounding effect, we measured scalability using the Tabula Muris dataset only. The dataset was subset to the two largest cell types and random samples of the cells without replacement were taken to generate datasets containing 50, 100, 250, 500, 750, 1000, 1250, 1500, 2500, 3000, 4000, 6000 and 8000 cells with equal proportion of the two cell types.Running time of each method was measured using the Sys.time function built-in R and the time.time function built-in Python. Tasks that did not finish within the given time limit are considered as no result generated. To record the maximal memory for R methods we used the function Rprofmem in the built-in utils Package in R. For Python methods we used the psutil package and measured the maximal Resident Set Size. All measurements were repeated three times and the average was reported.In the majority of methods, simulation was performed in a two-step process. In the first step, a range of properties is estimated from a given dataset. This set of properties are then used in the second step of generating the simulation data. For these methods, the time and memory usage of the two steps was recorded separately and shown in Supplementary Fig. In order to compare and rank the methods as shown in Fig. We selected a subset of datasets to examine the impact of the number of cells and sequencing technologies. Briefly, each dataset was split into 50% training and 50% testing. Transformed KDE score was then calculated from the raw score obtained from all methods across the selected datasets, resulting in values ranging between 0 and 1.To assess the impact of the number of cells on the accuracy of data property estimation, we used the Tabula Muris dataset subset to the two largest cell types and sampled to create datasets of 100, 200, 500, 1000, 1500, 2000, 2500, 3000, 5000, 6000, 8000, 12,000 and 16,000 cells. Each dataset was split into 50% training and 50% testing as previously described.Linear regression was fitted using the lm function in the built-in stats package in R for each of the 13 data properties. This resulted in a total of 13 regression models with the formula defined as:y was the KDE score corresponding to the data property and the exploratory variables x1 was the number of cells measured in 1000.The response variable 20 that sequenced the same tissue type using multiple protocols. It contains human PBMC data generated using the following six protocols, 10x Genomics, CEL-seq2, Drop-seq, inDrops, Seq-Well and Smart-seq2 and mouse cortex cells using the following four protocols of sci-RNA-seq, 10x Genomics, DroNc-seq and Smart-seq2.To assess the impact of the sequencing protocols while avoiding potential batch effect, we utilised two sets of datasets from the same studyP-values were displayed on the figures.ANOVA was fitted using the built-in stats package in R to examine whether there was significant change in mean KDE score across the above datasets of different sequencing technologies for each simulation method. Further information on research design is available in the Supplementary information.Peer Review File.Description of Additional Supplementary Files.Supplementary Data 1.Reporting summary.


A significant correlation between the expression of key immune genes between individual capture-based methods and FF-TruSeq  was observed. All exome capture-based methods reliably detected outlier expression of actionable gene transcripts, including ERBB2, MET, NTRK1, and PPARG. In urothelial cancer samples, the Agilent assay was associated with the highest molecular subtype concordance with FF-TruSeq . The Agilent and IDT assays detected all the clinically relevant fusions that were initially identified in FF-TruSeq. All FFPE exome capture-based methods had comparable performance and concordance with FF-TruSeq. Our findings will enable the implementation of RNA-seq in the clinic to guide precision oncology approaches.The availability of fresh frozen (FF) tissue is a barrier for implementing RNA sequencing (RNA-seq) in the clinic. The majority of clinical samples are stored as formalin-fixed, paraffin-embedded (FFPE) tissues. Exome capture platforms have been developed for RNA-seq from FFPE samples. However, these methods have not been systematically compared. We performed transcriptomic analysis of 32 FFPE tumor samples from 11 patients using three exome capture-based methods: Agilent SureSelect V6, TWIST NGS Exome, and IDT XGen Exome Research Panel. We compared these methods to the TruSeq RNA-seq of fresh frozen (FF-TruSeq) tumor samples from the same patients. We assessed the recovery of clinically relevant biological features. The Spearman’s correlation coefficients between the global expression profiles of the three capture-based methods from FFPE and matched FF-TruSeq were high (rho = 0.72–0.9,  One of the barriers to the clinical implementation of RNA-seq is the need for fresh-frozen tumor samples to obtain optimal results. However, in the clinical setting, the vast majority of specimens are preserved as formalin-fixed, paraffin-embedded (FFPE) tissues for long-term storage. Unfortunately, this preservation process is associated with a rapid decline in RNA quality2. Several adverse factors impact the quality of RNA extracted from FFPE, including ischemia, formaldehyde fixation, embedding in warm paraffin, and the duration of the storage of tissue blocks7.RNA sequencing (RNA-seq) has provided deep insights into gene expression patterns in biological samples, including transcript abundance levels, isoform expression, alternative splicing, and chimeric transcripts resulting from gene fusions. There is growing interest in leveraging RNA-seq as a clinical tool, especially in oncology, to match patients to targeted therapy and improve outcomes8. Recently, several commercial RNA capture kits have become available. However, a systemic comparison of their ability to recover clinically relevant biological features from real-world FFPE samples has not been performed. The lack of an optimal method for transcriptomic profiling of FFPE tumor samples has hindered clinical application. To address this knowledge gap, we compared the sequencing metrics and biological readouts from the Agilent SureSelect V6 (Agilent), TWIST NGS Exome (TWIST), and IDT XGen Exome Research Panel (IDT) capture-based methods from FFPE tumor samples. For each sample, we compared the three FFPE capture-based methods to TruSeq RNA-seq of the fresh frozen (FF) sample from the same tumor (hereafter referred to as FF-TruSeq). We studied the potential clinical utility of FFPE capture-based methods to discover clinically useful readouts. The comparison focused on genes with outlier expression, the expression of key immune genes, molecular subtype classification, and the detection of gene fusions.RNA capture is potentially more suited to the transcriptomic analysis of FFPE tumor samplesWe designed this study to answer two main questions: First, what are the differences in the performance characteristics between the three commercially available FFPE capture-based methods ? Second, what are the performance characteristics of FFPE capture-based methods compared to TruSeq RNA-seq of matched FF tumor samples Fig. ?Fig. 1StTo answer these questions, we compiled a cohort of 32 FFPE tumor samples from 11 patients. For each patient, a matching FF tumor sample was available. We included several tumor types, namely urothelial cancer, gastroesophageal junction (GEJ) adenocarcinoma, oligodendroglioma, cancer of unknown primary (CUP), leiomyosarcoma, papillary thyroid cancer, and colorectal cancer (Supplementary Data p > 0.05) and IDT and TWIST (Wilcoxon rank p > 0.05)   and TWIST   Fig. . TWIST w03) Fig. . The perp < 0.0001) and Agilent (p < 0.0001)  from the capture-based methods. TWIST showed the highest median log FPKMs compared to IDT  . Overall, the global gene expression pattern of FFPE tumor samples clustered with the corresponding FF sample in 8/11 of the matched sample sets in the t-distributed stochastic neighbor embedding plot   Fig.  (Supplem05) Fig. . In one 12. We examined the concordance of the expression of clinically relevant outlier genes in FF-TruSeq and whether the same outliers could be recovered from FFPE capture-based methods. For outlier detection, the mean and standard deviation of a gene were calculated across the Weill Cornell Medicine (WCM) RNA-seq cohort consisting of 650 multiple tumor samples. Outlier expression was defined as 1.5 times the interquartile range, z-score ≥ 2, and FPKM ≥ 20 (see “Methods” section). ERBB2 was found to be an outlier in samples from three patients with urothelial cancer, including all three FFPE capture-based methods and FF-TruSeq  between the FFPE exome capture-based and FF-TruSeq methods. A heatmap of the Spearman’s correlation scores across the expression profiles obtained from the three FFPE capture-based and FF-TruSeq methods is shown in Fig. PD-L1 (CD274) and CTLA4 from the FF-TruSeq method significantly correlated with their expression from the Agilent method , the IDT method , and the TWIST method , respectively  and the IDT (p = 0.003) methods , luminal nonspecified (LumNS), luminal unstable (LumU), stroma-rich, basal/squamous (Ba/Sq), and neuroendocrine-like (NE-like)17. To assess the applicability of using RNA-seq data from FFPE tumor samples for molecular classification consensus, we measured the concordance of the classifier outputs between the three FFPE capture-based and FF-TruSeq methods in five patients with urothelial cancer.A consensus mRNA expression-based single-sample classifier of muscle-invasive bladder cancers was recently publishedThe three FFPE capture methods showed significant agreement with the FF-TruSeq method (50–80%) in classifying the molecular subtypes  were orthogonally confirmed by targeted sequencing using the Archer FusionPlex platform from the FF samples18. The three FFPE capture-based methods successfully detected all the fusions except the MKRN2-PPARG fusion, which was missed by the TWIST capture method in one sample . The Spearman’s correlation coefficients between junction read count and expression levels were 0.99 , 0.92 , and 0.85 , respectively. The junction and spanning read counts supporting each fusion across the three capture methods were comparable with FF-TruSeq . Four fusions . We benchmarked these methods to TruSeq from matching FF tumor samples. We tested these methods on a diverse tumor cohort chosen to represent tumors from real-world FFPE biobanks. This supports the generalizability of our results to different cancer types. Overall, the FFPE capture-based methods showed consistent performance in identifying biological signals, including outlier gene expression, oncogenic fusions, or quantifying the expression of key immune genes. On the other hand, more complex readouts, such as the molecular subtype classification were less consistent and thus need to be interpreted cautiously when using these platforms. The three capture-based methods successfully generated sequencing libraries from all tumor samples. The peak density of the DV200 and RIN were within the accepted quality range to proceed with library prep. Moreover, samples with low-quality metrics from degraded specimens did not adversely impact the sequencing output or the number of uniquely mapped reads using these methods. We observed that low DV200 and RIN values did not predict whether samples should be excluded from sequencing using these capture methods.19.The minor differences in the total number of captured genes among the three capture-based methods did not significantly affect the global gene expression profiles. In fact, the global expression profiles of the FFPE capture-based methods positively correlated with the FF-TruSeq method across 11 matched tumor sets. We performed several downstream analyses to demonstrate the clinical utility of FFPE capture-based RNA-seq. We focused our analyses on clinically meaningful biological readouts, including the detection of expression outliers and oncogenic gene fusions, which are both amenable to therapeutic intervention. We also analyzed expression-based molecular subtyping of tumors which carries potential prognostic valueERBB2 was identified as an outlier gene in three tumor sample sets from three urothelial cancer patients in our cohort. One patient showed an exceptional clinical response to trastuzumab following the detection of outlier ERBB2 expression20. Outlier expression of the targetable oncogene MET was detected in a patient with GEJ adenocarcinoma and of NTRK1 in a patient with medullary colon cancer, respectively. Capmatinib is a MET inhibitor -approved for non-small cell lung cancer patients21, and larotrectinib is an NTRK inhibitor for NTRK fusion-positive solid tumor patients22. Our data suggest that outlier gene expression measurements from FFPE samples can be potentially useful for identifying patients who may benefit from oncogene-targeted therapies.Identifying targetable outlier genes from RNA-seq has important clinical applications. 23. The detection of fusions from FFPE is potentially challenging because of low coverage and the potential for false-positive calls25. Interestingly, the three capture methods we tested identified all the clinically relevant fusions detected by FF-TruSeq except one fusion that was not captured by the TWIST platform. RET fusions were potentially clinically actionable in two patients with CUP and papillary thyroid cancer. In addition, an NTRK1 fusion was identified in a colon cancer patient. NTRK1 fusions are a tumor-agnostic marker with an FDA-approved indication for treatment with larotrectinib.Gene fusions are important therapeutic targets1. The availability of FFPE-derived robust transcriptomic data will enable translational studies linking transcriptomic data to clinical phenotypes. This approach can also expand correlative studies to include FFPE tumor samples obtained from large multi-center clinical trials, mainly because many participating sites may not have the infrastructure for banking frozen tissues.Our study opens the door to the interrogation of FFPE tissues from archival pathology repositories. Fixation and paraffin embedding are commonly used to preserve tissue morphology and enable histomorphological, immunohistochemical, and other in situ studies. A particular advantage of FFPE tissues is that they can be stored for a longer duration allowing the analysis of long-term patient outcomes28 and were mainly focused on gene expression28.To the best of our knowledge, our study is the first to provide a comparison of three different FFPE capture-based methods applied to RNA from the same tumor sample. Previous reports attempted to examine the direct comparison of FF and individual FFPE capture methods from the same sample. The majority of these studies had smaller sample sizes (4–9 tumor samples)29. Using a dataset of 39 FFPE melanoma tumor samples, Kwong et al. compared RNA-seq to two NanoString gene expression panels3. They found that genes with low absolute expression showed poor correlation across platforms. This is consistent with our results across the FFPE capture-based methods suggesting that expression values of low abundance genes should be interpreted cautiously. We identified significant differences in molecular subtype membership assignment of urothelial cancers across the three capture-based methods, but this analysis was limited by the small number of urothelial cancers in the overall cohort. The current study reflects the tissue processing conditions at a single institution. The fixation and storage protocols in our study and the degree of degradation of FFPE samples may differ from those used by pathology departments at different institutions. The effects of these variations need to be studied. Another limitation of our study is that we did not evaluate all the available RNA exome platforms, such as the Illumina Exome library platform. Validation of our findings in multi-center studies that include diverse banking methods and different tissue types is warranted.Digital counting technologies  can be used to interrogate FFPE samples. Unlike RNA-seq, which captures the expression of thousands of genes, these methods are currently restricted to a lower number of mRNA targetsIn conclusion, we compared three capture methods for transcriptomic profiling of FFPE tumors using a range of sequencing metrics and functional readouts. For outlier and immune gene expression, all capture-based methods demonstrated comparable performance. In other areas, namely, multigene-based subtyping and fusion detection, we identified platform-specific differences. Careful consideration of the biological and clinical questions and the intended use case would enable the optimal selection of the best-suited FFPE RNA capture method. Our results demonstrate the feasibility of using RNA exome capture-based methods and their broad clinical applications.Patients signed informed consent (Weill Cornell Medicine IRB #1305013903). Banked excess tissue was collected from surgical specimens of patients with a diagnosis of cancer. All pathology specimens were reviewed by study pathologists . Clinical charts were reviewed by the authors  to record patient demographics, treatment history, anatomical site, and stage using the tumor, node, metastasis system published in the AJCC Cancer Staging Manual (8th edition).® 16 LEV RNA FFPE Purification Kit was used as previously described30. This kit provides a high yield of pure RNA from FFPE tissue  samples. This protocol takes 60 min after macrodissection of the unstained FFPE slides and Proteinase K digestion to complete. Prior to macrodissection, hematoxylin and eosin (H&E) stained slides were cut and annotated by a pathologist to identify the tumor’s location in the corresponding unstained slides to be used in the extraction. Ten unstained slides of 10 µm thickness per case were cut for the extraction along with one H&E stained slide. The annotated locations on each slide were then macro-dissected with a sterile razor blade to obtain tissue for RNA extraction.For RNA extraction from FFPE tissues, the Maxwell 16 ® instrument with the Maxwellhttps://www.promega.com/products/rna-purification-and-analysis/rna-purification/maxwell-16-lev-rna-ffpe-purification-kit/?catNum=AS1260), the Roche High Pure FFPET RNA isolation kit (https://lifescience.roche.com/en_us/products/high-pure-ffpet-rna-isolation-kit.html) and the Qiagen RNeasy FFPE kit . The RNA yields (ngs) and RIN numbers (~2.4) obtained from all three kits were similar. However, DV200 values were more variable. Two-thirds of the samples extracted using the Qiagen method had a DV200 <30. All the samples extracted using the Roche and Maxwell methods had DV200 >45. The Maxwell platform was chosen for extraction based on the availability of an automated workflow at our institution with the potential for scaling up the extraction of RNA from a large number of clinical samples. For extraction from frozen tissue, the Maxwell 16® instrument with the Maxwell® 16 LEV simplyRNA Tissue Kit was also used. Similarly, H&E stained slides were cut from the corresponding frozen block and annotated by a pathologist to identify the tumor location. Tissue from these annotated locations was removed using 1.5 mm diameter punch biopsies to core the block. Tissue homogenization was aided by introducing stainless steel beads to the tissue/homogenization solution and using the Qiagen Tissue Lyser LT set at 1/50 s for 2 min to physically break up the tissue before the lysis buffer was added.A side-by-side comparison using three specimens  was performed between the Promega Maxwell kit (200 measurement (the % of RNA fragments >200 nt) was used to determine the degree of RNA fragmentation for the FFPE samples ; SureSelectXT RNA Direct Protocol Provides Simultaneous Transcriptome Enrichment and Ribosomal Depletion of FFPE RNA, Agilent Technologies, Technical Note, publication number PR7000-0679. (https://www.agilent.com/cs/library/applications/5991-8119EN.pdf)The quantity of RNA was determined using a Nanodrop 2000 for nucleic acid absorbance measurement and a Qubit Fluorometer to confirm RNA concentration . Quality was assessed using a Bioanalyzer2100  with a high-sensitivity RNA chip. The RIN number was used to decide which RNA library prep kit to use for the frozen tissues; the DVr = 0.54, p = 0.001)   . Both RIN and DV200 had a limited utility for excluding low-quality samples for exome capture-based methods.The relationship between two critical quality metrics was analyzed, the percentage of fragments >200 nucleotides  and RNA Integrity Number (RIN). For all FFPE tumor samples, DV200 and RIN ranged between 22 and 87 (median 45) and 2–2.7 (median 2.4), respectively . However, no significant correlation between blocks’ age and RIN  was observed. There was no significant correlation between FFPE blocks’ age with the percentage of uniquely mapped reads .The median FFPE block age was 3.25 years (range 1.6–4.9 years) . Briefly, poly A+ RNA was purified from 100 ng of total RNA with oligo-dT beads and fragmented to ~200 bp. cDNA was synthesized using random priming, then end-repair, dA-tailed, and ligated to Illumina TruSeq adaptors containing unique sequencing indexes. Libraries were amplified with 15 cycles of PCR and pooled for sequencing . Briefly, rRNA was depleted from 200 ng of total RNA with Ribo-Zero and fragmented to ~200 bp. cDNA was synthesized using random priming, and transcript orientation was preserved by using dUTP during second-strand cDNA synthesis. After end repair, A-tail, and ligation to Truseq adapters, libraries were generated by amplification with 15 cycles of PCR. Library pools were clustered at 6.5pM on a paired-end read flow cell and sequenced for 75 cycles on an Illumina HiSeq 2500 to obtain ~50 M reads per sample. . Transcriptome enrichment was achieved by the hybridization of the pre-capture library to the exome panels tested. Since the probe baits were biotinylated, hybridized libraries were captured using streptavidin beads  and PCR amplified-on-beads to generate a post-capture library. All post-capture libraries were subjected to quality control on an Agilent Bioanalyzer and normalized to 2 nM. The post-capture libraries obtained from each capture platform were pooled, and each pool was sequenced on one lane of a paired-end read flow cell for 2 × 100 cycles on a HiSeq4000 to obtain ~40 M reads per sample. The primary processing of sequencing images was done using Illumina’s Real Time Analysis software. CASAVA 1.8.2 software was then used to demultiplex samples and generate raw reads and respective quality scores : Non-indexed pre-capture libraries were made using SureSelect XT RNA Direct kit (PN-G9691-90050) with 200 ng of RNA, using the % DV200 obtained with the Agilent Bioanalyzer to determine fragmentation times and amplifying 14–16 PCR cycles. Hybridization was carried out with 200 ng from each pre-capture library for 24 h × 65 °C on RNA-biotinylated probes. Indexes were added during post-capture libraries amplification using 12 cycles.Sure Select200 of the starting material. Pre-capture libraries were uniquely indexed for Illumina sequencing, using 11–16 amplification cycles. A total of 1.5 µg of pooled indexed libraries  were hybridized to the biotinylated double-stranded DNA probe panel for 16 h at 70 °C. Post-capture libraries were amplified for eight cycles (DOC-001014).Twist Human Core Exome : Libraries were made using the NEBNext Ultra II Directional kit  with 100 or 200 ng depending on the % DVIDT xGen Exome Research Panel v1.0 : Libraries were made as described above using the NEBNext Ultra II Directional kit . A total of 5 µg of pooled indexed libraries (500 ng each) were hybridized to the biotinylated oligo probes for 4 h at 65 °C. Post-capture libraries were amplified for seven cycles (NGS-10122-PR 01/2020).The Agilent capture-based method targets 91 Mb of genomic DNA sequence in addition to 5′ and 3′ UTR sequences. IDT and TWIST methods target 39 and 33 Mb of the coding sequences (CDS) of human coding genes, respectively. The three capture-based methods use 120-base RNA probes to capture known CDS. The total number of captured genes is 20,456 for Agilent, 19,075 for IDT, and 19,542 for TWIST.31 for sequence alignment against the human genome sequence build hg19, downloaded via the UCSC genome browser [http://hgdownload.soe.ucsc.edu/goldenPath/hg19/bigZips/], and SAMTOOLS v0.1.1932 for sorting and indexing reads. Cufflinks (2.0.2)33 was used to estimate the expression values (FPKMS) and GENCODE v1934 GTF file for annotation. Since the sequenced samples were processed using different library preps, batch normalization of FPKMs from WCM Frozen samples was done using ComBat from the sva Bioconductor package35. For fusion analysis, we used STAR-fusion (STAR-Fusion_v0.5.1)37. Fusions with significant support of junction reads (≥1) and spanning pairs (≥1) were selected. For outlier detection, the FPKMs from batch normalized frozen WCM samples were combined with the FPKMs from FFPE samples. We only selected the druggable genes from drugbank38 as well as cancer genes from Oncokb39, which resulted in a list of 138 druggable cancer genes. The mean and standard deviation of each gene were calculated across the WCM RNA-seq cohort (multiple cancer types). An outlier was defined as having 1.5 times the interquartile range, z-score ≥2, and FPKMs ≥20.All reads were independently aligned with STAR_2.4.0f140, which suggests the following interpretations. Below 0.0: Poor, 0.00–0.20: Slight, 0.21–0.40: Fair, 0.41–0.60: Moderate, 0.61–0.80: Substantial, 0.81–1.00: Almost perfect. RStudio (1.0.136) with R (v3.3.2) and ggplot2 (2.2.1) were used for statistical analysis and generating plots. A p value <0.05 was considered significant. All tests were two-sided.For pairwise comparisons, we used the Wilcoxon signed-rank test. For comparison of the post-alignment statistics among the three capture methods, the Kruskal–Wallis test was performed. Correlation analyses between gene expression profiles were performed using the Spearman’s rank correlation test. To measure the inter-classifier concordance, the Cohen’s kappa statistic measure of inter-rater agreement was calculated. The kappa-statistic measure of agreement was scaled to 0 when the amount of agreement is what would be expected to be observed by chance and 1 when there is perfect agreement. We used the Landis and Koch methodFurther information on research design is available in the Supplementary Data 1Supplementary Data 2Supplementary Data 3Supplementary Data 4Supplementary Data 5Supplementary Data 6Supplementary Data 7Supplementary InformationReporting Summary


Here, we present K2Taxonomer, a novel unsupervised recursive partitioning algorithm and associated R package that utilize ensemble learning to identify robust subgroups in a ‘taxonomy-like’ structure. K2Taxonomer was devised to accommodate different data paradigms, and is suitable for the analysis of both bulk and single-cell transcriptomics, and other ‘-omics’, data. For each of these data types, we demonstrate the power of K2Taxonomer to discover known relationships in both simulated and human tissue data. We conclude with a practical application on breast cancer tumor infiltrating lymphocyte (TIL) single-cell profiles, in which we identified co-expression of translational machinery genes as a dominant transcriptional program shared by T cells subtypes, associated with better prognosis in breast cancer tissue bulk expression data.As high-throughput genomics assays become more efficient and cost effective, their utilization has become standard in large-scale biomedical projects. These studies are often explorative, in that relationships between samples are not explicitly defined   Furthermore, since its first published application in 2009  estimation of data-driven groups of observations via application of an unsupervised learning procedure, followed by (ii) annotation of each group based on the identification of distinct patterns of gene expression relative to other groups. While most approaches focus on discovering a ‘flat’ set of non-overlapping groups or subtypes, in this manuscript we present an alternative approach, devised to emphasize ‘taxonomy-like’ hierarchical relationships between observations to discover nested subgroups.Seurat (Scran (Neighborhood Joining (trajectory inference approaches (partition-based graph abstraction (PAGA)  , was deveduction , thus prpcaReduce  algorithms at face value address the need for a multi-resolution representation of the relationship among observations, and while originally adopted for the analysis of bulk gene expression data , numeroucaReduce , ascend    profiled by scRNAseq (https://montilab.bu.edu/k2BRCAtcell/).In this manuscript, we assess the performance of ETABRIC)  and the ETABRIC) . We concscRNAseq . Our anascRNAseq ,31, and K2Taxonomer are not limited to cancer-centric studies. For example, we applied an earlier prototype of K2Taxonomer to the analysis of a toxicogenomic study on the effects of environmental exposures on adipocyte activity, and the tool proved to be instrumental to the identification of chemical subgroups and the pathways contributing to either their deleterious or beneficial effects on energy homeostasis , which further undergoes hierarchical clustering, followed by a K = 2 tree cut.To estimate each partition, ustering . More spK2Taxonomer package implements separate clustering methods tailored to analysis of either observation-level and group-level data input. For observation-level data, the perturbation-specific partitions are estimated via hierarchical clustering of the Euclidean distance matrix, followed by a K = 2 tree cut. By default, Ward's agglomerative method is performed at this step because it has been shown to generally perform well compared to other hierarchical methods , limma (v3.42.2), and GSVA (v1.34.0). All P-values reported in this manuscript are two-sided.The implementation of K2Taxonomer was assessed in comparison to Ward's agglomerative method for recapitulating induced hierarchical structure in simulated data. See supplementary methods for a comprehensive description of the strategy implemented for data generation and performance assessment for observation- and group-level analyses.The performance of K2Taxonomer was evaluated in its observation-level modality for its ability to recover the Pam50 subtypes, as well ER-, PR-, and HER2-status, and the aggregate three-gene genotype of ER-, PR-, and HER2-status, in the METABRIC and TCGA breast cancer datasets, independently  of features. This should be kept in mind when comparing performances, since the best-performing pre-filtering level is not known a priori, and it is in general dataset dependent. For every pre-filtering level, the median absolute deviation (MAD) score was used for feature selection, and Euclidean distance was used to estimate observation-level distance. Performance was assessed as the entropy of each of the phenotypes , induced by the inferred sample sub-grouping, with lower entropy indicating ‘purer’ subgroups, hence better performance  python package. Neighborhood graphs were estimated from the component-based dimensionality reduced data sets using ‘pp.neighbors’. Finally, the PAGA algorithm was run, setting the ‘group’ argument of ‘tl.paga’ to the cell type labels.Analyses with ascend were carried out using the ascend (v0.99.69) R package. The ascend-based hierarchical models were estimated from the component-based dimensionality reduced data sets using the ‘runCORE’ function. Given that, unlike K2Taxonomer and PAGA, ascend can only be carried out as a fully unsupervised analysis without group-level input, concordance of these models with those of other methods was evaluated based on purity-based assignment of cell type labels to individual subgroups of each ascend model. Cell types were assigned to individual partitions based on two criteria: first, if at least 75% of the profiles of a given cell type were assigned to the subgroup and, second, if the total number of cell profiles across individual cell types that met the first criteria made up at least 75% of the total cell profiles in the subgroup.Analyses with GEO, accession number GSE110938 (ZINB-WaVE (v1.8.0) (Seurat (v1.3.4) was modified by setting the ‘resolution’ argument of ‘FindClusters’ to 1.1, rather than the default, 0.8  . To enabult, 0.8 . This reult, 0.8 .K2Taxonomer was applied in its group-level modality to partition these 13 estimated cell subtypes based on the normalized count matrix estimated by ZINB-WaVE. According to the developers, ZINB-WaVE, normalized count estimates are not recommended for differential analysis (bayNorm (v1.4.14) R package (mSigDB (v7.0)  and minimum subgroup-specific expression, (mean[log2 counts] > 0.5), then restricted to a maximum of 50 genes.analysis , hence d package . PathwayB (v7.0) . SignatuK2Taxonomer estimated model and that estimated by a fully unsupervised scRNAseq hierarchical clustering method, we ran ascend on the same ZINB-WaVE reduced dimension data set that was used as input for Seurat. Subgroup-level cell cluster assignments in the resulting ascend estimated model was carried out using the same criteria employed for the healthy airway tissue scRNAseq analysis.To investigate the concordance between K2Taxonomer, we performed survival analysis based on gene signature projection scores, as well as on selected genes in the METABRIC breast cancer primary tumor gene expression data set. Gene set projection was carried out using GSVA , and groups of samples (group-level). The latter corresponds to scenarios where the goal is to define a taxonomy over sample groups, such as cell types in single-cell experiments, or chemical perturbations profiled in multiple replicates , while Ward's performed better for 17 combinations  ..K2Taxonon (PAGA) , which hn (PAGA) .K2Taxonomer was remarkably accurate in capturing the higher-order organization of the 28 cell types. The first partition separated all epithelial cell subtypes from non-epithelial cell types , from the two other Treg cell subtypes in the second partition. Moreover, the ascend-based model segregated individual profiles of CD4+ Trm cells, such that this cell type ‘dropped-out’ out of the model early in partitioning.We compared the subgroup discovery of this model to that of ascend   , while CCL5 is associated with better patient prognosis . However, in the combined model both genes are associated with patient survival, with TNFRSF4 associated with worse patient survival .TNFRSF4 and CCL5 were found to be the top two markers constitutively up- and down-regulated, respectively, within Treg subgroups, with TNFRSF4 the top marker further discriminating between the two non-mitotic Treg subgroups, + Figure , C, D. F) Figure , support) Figure . When moTaken together these results indicate that, in bulk gene expression data, markers of Treg cell activity are highly correlated with markers of overall tumor immune infiltration, confounding associations between expression of these markers and patient survival.P-value = 0.014)  Figure . FurtherK2Taxonomer to characterize the up-regulation of translational machinery as the dominant transcriptional program shared by a diverse subgroup of TILs. These findings informed additional analyses, which demonstrated that association between expression of translational machinery genes and better patient survival tracked with their over-expression in a subgroup of TILs. Taken together these findings suggest that the up-regulation of specific translational machinery genes is widespread across TILs, serving as a predictor of the level of immune infiltration in breast cancer tissue from bulk gene expression data.In summary, we employed K2Taxonomer, a novel unsupervised recursive partitioning algorithm for taxonomy discovery in both bulk and single-cell high-throughput transcriptomic profiles. An important distinctive feature of the algorithm is that each partition is estimated based on a feature set selected to be most discriminatory within that partition, thus permitting the use of large sets of features to be used as input, without pre-filtering or dimensionality reduction approaches. Additionally, to minimize generalization error, each partition is based on an ensemble .In this manuscript, we presented extensive assessment and practical applications of ensemble  of partiK2Taxonomer’s high accuracy, its superior performance when compared to representative other methods, and its capability to (re)discover known nested taxonomies. As we have shown in its multiple applications, K2Taxonomer may be applied in a fully unsupervised mode to partition individual-level data, or it can take group-level labels as input to estimate inter-group relationships among the known groups. In the group-level settings, K2Taxonomer is not directly comparable to fully unsupervised ‘flat’ or hierarchical clustering methods. In our simulations, the comparison to Ward agglomerative clustering was attained by averaging the profiles within each simulated ‘group’ and by then clustering these ‘average profiles.’ Side-by-side comparison with any of the single cell algorithms discover subgroups of cell types characterized by shared lineage. Remarkably, our analysis accurately recapitulated the known taxonomic structure relating the different cell types to an extent not matched by the other methods evaluated. This example illustrates a prototypical use of the tool: in those cases where a data set and its associated cell type estimations are publicly available, K2Taxonomer facilitates their immediate repurposing for additional insight and discovery. This is a defining advantage over fully unsupervised scRNAseq hierarchical clustering methods, such as ascend (ascend analysis of these data was complicated by the fact that cell profiles belonging to the same cell type label were split across multiple partitions, and often to an extent that it made it challenging to assign a representative cell type label to individual subgroups. Moreover, even for annotated subgroups, none of the ascend models were able to recapitulate taxonomic relationships between cell types to the same level as the K2Taxonomer model. This was also true when comparing the K2Taxonomer and ascend models generated from the breast cancer TILs scRNAseq data set.In our analysis of healthy airway cell types’ annotation , we empls ascend , which cK2Taxonomer cannot identify precursor relationships between cell types, the strategy of pairing recursive partitioning with local feature selection allows the discernment of relative relationships between groups rather than ‘all-or-nothing’ connections as in graph-based trajectory models. The advantages of this strategy are exemplified by the analysis of the healthy airway, in which the data included samples of multiple individuals and airway locations. PAGA analysis of these data produced highly-connected graphs with no decipherable trajectories or subgroups beyond that of disconnected subgraphs. On the other hand, K2Taxonomer recapitulated clear lineage-driven hierarchies of airway cell subgroups. While the subgrouping of endothelial cells as a subset of immune cells does not reflect the expected hierarchy, K2Taxonomer generated this model without parameterization. The ‘best’ PAGA model had the unfair advantage of having been chosen from a set of distinct models generated from combinations of feature selection and dimensionality reduction parameters. In practice, the a priori choice of the optimal values of these parameters is challenging.It is important to emphasize that in many data sets continuous lineage trajectories are non-existent or obscured by phenotype-driven inter-group transcriptional relationships. While K2Taxonomer in an advanced in-silico study that yielded significant novel insights. In contrast to bulk gene expression, which captures average expression across all cells, identifying dominant transcriptional programs driving phenotypic similarities between subgroups of cell populations offers additional insights to deconvolute the cellular microenvironment of these samples beyond their individual transcriptional signatures. Molecular convergence of cells of disparate lineages is exemplified by subpopulations of CD8+ and CD4+ T cells, each of which exists in various functional states as naïve, effector, and memory subpopulations (K2Taxonomer-based analysis showed that concordant subpopulations of CD8+ and CD4+ T cells share transcriptional signatures that may outweigh those arising from their shared lineage. For example, both CD8+ Trm and CD4+ Trm cells have been reported to express surface molecules, CD69 and CD11a (Our extensive analysis of single-cell data from breast cancer TILs showcased the incorporation of ulations . Importand CD11a . Concordnd CD11a ,66.Unlike Trm cells, the presence of immune-suppressing Treg cells in the microenvironment has been associated with poor prognosis in breast cancer . After iK2Taxonomer identified a diverse subgroup of breast cancer TILs characterized by consistent up-regulation of translational genes. Increased ribosomal biogenesis has been previously implicated in increased tumorigenesis (Finally, igenesis , but hasigenesis  and expaigenesis . Unlike igenesis  and proligenesis  signaturK2Taxonomer demonstrated a remarkable ability to discover biologically relevant taxonomies when applied to the analysis of both bulk gene expression and scRNAseq data and to outperform standard agglomerative methods. In multiple practical applications, we showcased the versatility of K2Taxonomer to analyze scRNAseq data toward the characterization of genes and pathways distinguishing specific subgroups, thereby generating hypotheses that were then in-silico validated in independent bulk gene expression data. As noted, while we here focused on the analysis of transcriptomics data, the proposed approach is equally applicable to other bulk and single-cell ‘omics’ data, such as those generated by high-throughput proteomics and metabolomics assays.In summary, K2Taxonomer is publicly available as an R package on GitHub, https://github.com/montilab/K2Taxonomer. Results are available through the interactive portal at https://montilab.bu.edu/k2BRCAtcell.https://www.cbioportal.org/study/summary?id=brca_metabric. The bulk RNAseq gene expression and subject data from the TCGA BRCA project is available through the Genomic Data Commons, https://gdc.cancer.gov/access-data/. The single-cell RNAseq gene expression and subject data of healthy airway tissue is available through the UCSC Cell Browser page, https://www.genomique.eu/cellbrowser/HCA/?ds=HCA_airway_epithelium. Finally, the single-cell RNAseq gene expression and subject data of breast cancer immune cells is available in the Gene Expression Omnibus, GSE110938.The data that support these findings are publicly available and were accessed from several repositories. The bulk microarray gene expression and subject data from the METABRIC breast cancer study is available through the cBioPortal, gkab552_Supplemental_FilesClick here for additional data file.


Gene co-expression clustering in scRNA-seq data presents certain challenges. We show that commonly used methods for single-cell data are not capable of identifying co-expressed genes accurately, and produce results that substantially limit biological expectations of co-expressed genes. Herein, we present single-cell Latent-variable Model (scLM), a gene co-clustering algorithm tailored to single-cell data that performs well at detecting gene clusters with significant biologic context. Importantly, scLM can simultaneously cluster multiple single-cell datasets, i.e., consensus clustering, enabling users to leverage single-cell data from multiple sources for novel comparative analysis. scLM takes raw count data as input and preserves biological variation without being influenced by batch effects from multiple datasets. Results from both simulation data and experimental data demonstrate that scLM outperforms the existing methods with considerably improved accuracy. To illustrate the biological insights of scLM, we apply it to our in-house and public experimental scRNA-seq datasets. scLM identifies novel functional gene modules and refines cell states, which facilitates mechanism discovery and understanding of complex biosystems such as cancers. A user-friendly R package with all the key features of the scLM method is available at https://github.com/QSong-github/scLM.In gene expression profiling studies, including  Co-expressed genes work in concert in biological pathways and processes The recently developed single-cell RNA sequencing (scRNA-seq) technology provides high resolution of gene expression at the single-cell level e.g., mRNA quality and pre-amplification efficiency), and extrinsic biological variabilities.Rapid advances in scRNA-seq technologies have made it feasible to perform population-scale studies in which the transcriptome is measured for thousands of single cells from multiple samples or conditions Classical methods designed for analysis of bulk transcriptome data such as weighted gene co-expression network analysis (WGCNA) Herein, we have developed a novel method, single-cell Latent-variable Model (scLM), to simultaneously extract co-expressed genes that exhibit consensus behaviors from multiple single-cell datasets. The scLM method accounts for both cell-level covariates and sample-level batch effects. We assessed the performance of scLM in both simulated data and experimental data. scLM achieves the best performance over other commonly used methods. We then applied scLM to our in-house scRNA-seq data generated from four non-small cell lung cancer (NSCLC) tumor tissues and their corresponding adjacent normal tissues. The scLM method identified tumor-specific co-expressed gene modules with significant prognostic values. Furthermore, these co-expression modules contributed to the subtle characterization of lung tumor cell states. In addition, we applied scLM to analyze a set of malignant cells from NSCLC, head and neck squamous cell carcinoma (HNSCC), and melanoma. We discovered a common co-expressed gene program across different cancer types, providing insights into fundamental mechanisms of carcinogenesis.We proposed a latent-variable model to explicitly disentangle different sources of variabilities in population-scale scRNA-seq data. Our goal was to perform simultaneous detection of co-expressed genes across multiple single-cell conditions/datasets. Specifically, let As multiple recent studies If Therefore, the probability function converts to:As Let i.e., the GLM,To alleviate the impact of extreme values, we utilized logarithm form in the linear model that has been frequently used ariables .Figure 1B distribution, the conditional log-likelihood function of To estimate the parameters in our model, we used the maximum likelihood approach. As is assumed above that For the latent variable To control model complexity and overfitting, we applied the least absolute shrinkage and selection operator , to the following penalized joint log-likelihood estimation:Then the above parameters are estimated by maximizing the penalized joint log-likelihood function, that is, maximizing the following penalized joint log-likelihood,i.e.,To estimate the parameters With the estimated latent variables n = 1), D2 contained two individual datasets (n = 2), …, and D9 contained nine individual datasets (n = 9). Each individual dataset contained 180 genes belonging to three clusters, with 60 co-expressed genes in each of the three clusters. For each gene cluster Based on the single-cell data characteristics, we used the NB distribution to simulate two synthetic cohorts (synthetic cohort 1 and synthetic cohort 2). Each synthetic cohort contained 9 sets of simulated gene expression data with an increasing number of datasets (D1–D9). That is, D1 contained one individual dataset  into Miltenyi Tissue Storage Medium . Tissues were then processed to single-cell suspensions using the Miltenyi Human Tumor Dissociation Kit  and the gentleMACS Octo Dissociator with Heaters . Red blood cells were removed by negative selection using Miltenyi CD235a (Glycophorin A) microbeads  and LS Columns . Recovered cell numbers were determined by trypan blue exclusion using a LUNA II automated cell counter . In preparation for scRNA-seq, cells were thawed and washed according to the demonstrated protocol developed for human peripheral blood mononuclear cells (PBMCs) by 10X Genomics .https://onlinelibrary.wiley.com/doi/full/10.1002/cam4.2113All scRNA-seq procedures were performed by the Cancer Genomics Shared Resource (CGSR) of the Wake Forest Baptist Medical Center Comprehensive Cancer Center (WFBMC-CCC). Viable cells in suspensions were loaded into wells of a 10X Genomics Chromium Single Cell A Chip Kit . Single-cell gel beads in emulsion (GEMs) were created on a Chromium Single Cell Controller and scRNA-seq libraries were prepared using the Chromium Single Cell 3′ Library and Gel Bead Kit v2 . Sequencing libraries were loaded at 1.3 PM on an Illumina NextSeq500 with a High Output 150 cycle Kit  for paired-end sequencing. A total of 11,813 single cells were captured, with the number of cells recovered per channel ranging from 369 to 2502. Low-quality cells were discarded if the cell with expressed genes was smaller than 200. Only malignant cells from four tumor samples and epithelial cells from three adjacent normal samples were used in this study. The scRNA-seq data were deposited in the Gene Expression Omnibus (GEO) of National Center for Biotechnology Information (NCBI) database (GEO: GSE117570) at https://www.ncbi.nlm.nih.gov/pubmed/27124452We downloaded the expression matrix data of melanoma from the GEO of NCBI database (GEO: GSE72056) at https://www.sciencedirect.com/science/article/pii/S0092867417312709We downloaded the expression matrix data of the HNSCC dataset from the GEO of NCBI database (GEO: GSE103322) at https://www.nature.com/articles/s41467-018–06052-0We downloaded the expression matrix data of breast cancer (BR) scRNA-seq dataset from the GEO of NCBI database (GEO: GSE118390) at Each clustering result produced by different methods was assessed using clustering evaluation indices, including the Adjusted Rand Index (ARI) With the co-expressed gene modules, we utilized mean value of the modules in each single cell as input for graph-based clustering. Uniform manifold approximation and projection (UMAP) was used to visualize cell clusters. Graph-based clustering was performed using the Seurat package (v3.1), and UMAP analysis was performed using the “umap” package (v.0.2.3.1) http://cran.r-project.org/web/packages/survival/index.html). Log-rank test was used to test the differences of survival curves. When evaluating the performance of scLM, P value was calculated by t-test.Kaplan-Meier (KM) analysis was performed using the “survival” R package  http://www.reactome.org) was a manually curated open-data resource of human pathways and reactions, which was an archive of biological processes and a tool for discovering potential functions. Gene sets derived from the Reactome Reactome  databases was assessed by hypergeometric test, which was used to identify a priori-defined gene set that showed statistically significant differences between two given clusters. Enrichment test was performed by the clusterProfiler package We developed a new method, scLM, for simultaneously identifying consensus co-expressed genes from multiple scRNA-seq datasets. Our hypothesis was that co-expressed genes coordinating biological processes could be captured across multiple different datasets. In our model, we assumed that latent variables captured the intrinsic signals of the co-expressed genes regardless of technical variances and batch effects among different datasets. To evaluate the performance of scLM, we benchmarked it against other methods, including LTMG n = 1), D2 contained two individuals of datasets (n = 2), and so on. Each set contained three co-expressed gene clusters as ground truth. Additionally, we utilized the Splatter package We first generated two synthetic data cohorts (synthetic cohorts 1 and 2) from NB distribution. Each cohort contained 9 sets (D1–D9) of simulated data with an increasing number of samples. That is, D1 contained one individual dataset  to identify the co-expression clusters. To assess and quantify clustering accuracy, we used the ARI To further demonstrate the performance of scLM, we compared scLM with other methods  on experimental scRNA-seq datasets. For comparisons, we used two in-house datasets from lung tumor and adjacent normal tissues as well as three public datasets from BR, HNSCC, and melanoma. The data pre-processing procedures were described in the Method.P = 1.75E−07) and MNN-wgcna (P = 0.02), demonstrating that scLM achieved better cluster validity than other methods based on average between- and within-cluster sum of squares. In addition, compared to other methods, scLM also achieved significantly higher Dunn index scores representing better inter-cluster separation and intra-cluster compactness  ranged from 15 to 184 . SCENIC identified relatively high number of enriched GO terms in the BR dataset, whereas low number of enriched GO terms in other datasets; CSN identified relatively high number of enriched GO terms in the lung Tumor dataset but low number of enriched GO terms in other datasets; MNN-wgcna identified relatively high number of enriched GO terms in the melanoma dataset but low number of enriched GO terms in other datasets. LTMG and Seurat-wgcna showed lower number of enriched GO terms in five datasets. Similar results were observed when we strengthened the enriched significance by the adjusted P value < 0.01 . Taken together, these results demonstrated that scLM outperforms other methods in functional discovery of co-expressed genes.In addition to GO terms, we also examined the enriched pathways in the Reactome database, based on the co-expressed genes identified by different methods B. DifferIn real-world scenarios, samples from different patients or different data sources often demonstrated highly different cell numbers, largely due to strong batch effects and technical issues. The scLM method was designed to address such highly unbalanced data that outperformed other competitors on such datasets. To validate the effectiveness of scLM, we intentionally selected patient samples that varied with respect to cell number, which could create challenges for this method. As a case study, we used scLM to analyze our in-house scRNA-seq profiling from 4 NSCLC patients (P1–P4) Using the 12 co-expression modules, the single cells were separated into two major clusters. In each cluster, cells from different patients mixed well without interference from batch effects C. These Similarly, in normal single cells, we observed 13 co-expressed gene modules (N-m1–N-m13) that showed concordant expression across individual patients . Then weTo explore the underlying mechanism of carcinogenesis, we next extended the application of scLM to HNSCC and melanoma. In addition to the 12 co-expressed gene modules identified in NSCLC, we identified 11 modules in HNSCC and 14 modules in melanoma. To determine the similarities of these co-expression modules, we performed a pair-wise comparison using weighted Jaccard similarity, followed by hierarchical clustering. As shown in the diagram A, we fouMYC targets v1 and hypoxia were the top enriched terms, involving the genes FOS, GAPDH, HLA-A, and NFKB1A, which suggested the common oncogenesis mechanism regardless of cancer types  scLM identifies co-expressed genes that reveal novel biological processes. An example is the lung tumor-specific module that highlights cell–cell communication in tumor microenvironment . 2) scLMlignancy .Given the merits of scLM, several potential limitations warrant further study. First, zero-inflated genes are excluded during pre-processing. The main reason is that, genes with inflated zeros are not informative and have negligible meaningful contribution to co-expression. The other reason is, with the fast advance of scRNA-seq technology, zero-inflation issue will be very minimal in near future. Second, in future work, we will examine the necessity of providing zero-inflation models, which specifically deal with data of poor sequencing depth and strong dropout effects. Third, the computational cost of scLM can be further reduced. We have already utilized C programming and parallel computing to dramatically boost the efficiency of scLM. However, considering that scRNA-seq data are growing into million-cell level, we will explore the use of GPU computing and cloud-based approaches to catch up with the scale of future scRNA-seq data.Co-expressed genes with coordinate expression indicate functional linkages between genes. Genes with coordinate biological functions are frequently co-transcribed, resulting in co-expression profiles. Thus, co-expressed genes can be used to intuitively associate genes with biological processes, to reveal disease-related genes, and to discern transcriptional regulatory mechanisms. Accumulative evidence supports the reliability of co-expression analysis for annotating and inferring gene functions SMAD2/3/4 transcriptional activity, with identified upstream transcriptional factors including TEAD1 and FOXA1. We further show the clinical prognostic significance of these discoveries in clinical samples. Moreover, we explore the common co-expressed genes, i.e., the common module, across three cancer types and offer intrinsic mechanisms of tumor malignancy. The common module is highly enriched in the MYC targets v1 and hypoxia, suggesting the presence of common intrinsic oncogenesis mechanisms. Additionally, the common module is shown to be related with clinical response to ICI in melanoma patients, suggesting that the common module provides predictive value of ICI therapy response.In this study, we introduce a novel method, scLM, to simultaneously identify co-expressed genes across multiple single-cell datasets. The scLM algorithm uses the conditional NB distribution with latent variables to disentangle co-expression patterns across multiple datasets. To our knowledge, scLM is the first available tool that is capable of leveraging multiple scRNA-seq datasets to accurately detect co-expressed genes. We provide an overview of scLM and illustrate how scLM can be used to further characterize cell states and identify tumor-specific modules in lung cancer. We demonstrate that the tumor-specific modules are enriched in pathways, including cell–cell communication and Compared with other methods, scLM has several key advantages: 1) scLM accounts for data heterogeneity and variances among multiple datasets, such as unbalanced sequencing depths and technical biases in library preparation. 2) scLM leverages information across datasets for detecting stable and conserved co-expression modules with high accuracy and reproducibility. 3) scLM is an integrated pipeline that uses raw count matrix without prior batch-correction as input, thus can be easily applied to scRNA-seq data. Overall, scLM opens possibilities for further investigation and mechanistic interpretation of co-expressed genes. With the growing scRNA-seq data, scLM is poised to become a valuable tool for elucidating co-expression studies in single-cell transcriptomics.For our in-house data, fresh remnant tumor and adjacent normal tissues were collected at the time of elective curative resection by the Tumor Tissue and Pathology Shared Resource (TTPSR) of the Wake Forest Baptist Medical Center Comprehensive Cancer Center (WFBMC-CCC), USA. Collections by the TTPSR adhere to Institutional Review Board approved procedures . Acquisition of de-identified samples from the TTPSR for single-cell isolation and research use was in accordance with approved IRB protocol 00048977. All patients provided written consent to participate in the study, which was approved by the institutional reviewing board.https://github.com/QSong-github/scLM.A user-friendly R package with all the key features of the scLM method is available at Qianqian Song: Data curation, Methodology, Software, Writing - original draft. Jing Su: Methodology, Software, Validation, Writing - review & editing. Lance D. Miller: Investigation, Writing - review & editing. Wei Zhang: Conceptualization, Supervision. All authors read and approved the final manuscript.The authors have declared no competing interests.


Cancer metastasis into distant organs is an evolutionarily selective process. A better understanding of the driving forces endowing proliferative plasticity of tumor seeds in distant soils is required to develop and adapt better treatment systems for this lethal stage of the disease. To this end, we aimed to utilize transcript expression profiling features to predict the site-specific metastases of primary tumors and second, to identify the determinants of tissue specific progression.We used statistical machine learning for transcript feature selection to optimize classification and built tree-based classifiers to predict tissue specific sites of metastatic progression.We developed a novel machine learning architecture that analyzes 33 types of RNA transcriptome profiles from The Cancer Genome Atlas (TCGA) database. Our classifier identifies the tumor type, derives synthetic instances of primary tumors metastasizing to distant organs and classifies the site-specific metastases in 16 types of cancers metastasizing to 12 locations.We have demonstrated that site specific metastatic progression is predictable using transcriptomic profiling data from primary tumors and that the overrepresented biological processes in tumors metastasizing to congruent distant loci are highly overlapping. These results indicate site-specific progression was organotropic and core features of biological signaling pathways are identifiable that may describe proliferative plasticity in distant soils.The online version contains supplementary material available at 10.1186/s12920-021-01122-7. Metastasis accounts for 90% of cancer associated mortality . While dMetastatic organotropism is a long-standing problem in cancer research and characterizing the metastatic patterns of primary tumors is a critical step towards treating patients with advanced disease , 12. ExpStudies using molecular information for retrospective analyses of tumor metastatic sites have been xenograft selection studies that extrapolated organotropic features from metastasis microarray data. Studies leveraging RNA transcript profiling data have been designed for single tumor type progressing to a single site. We have found no significant study has been developed on classifying site-specific metastasis from human primary tumor transcriptomic profiling data , 19–28. Despite the significant progress made from previous modeling methods, a unified approach to predict site specific metastasis in multiple cancer types that learns the biological determinants of dissemination has not been resolved. We have leveraged the publicly available omics data and clinical annotations in the TCGA database to investigate metastatic organotropisms of multiple cancers. In this study, we build off the previous work and establish a machine learning architecture that models organotropic metastases by distinguishing the tumor type and in multiple cancer types predicts the loci of distant tumor metastases. We detail a migration from the canonical pipelines using differential expression for feature assessment and use statistical machine learning for feature selection to optimize classification. Our model systematically predicts site-specific metastases of primary tumors and our methods captured conserved core biological processes overrepresented in tumors of varying origin that seeded in concordant anatomic locations.The TCGA data portal has the clinical data commons that are publicly available for data mining in the clinical databank . These dSynthetic samples were generated to balance positive and negative classes using the SMOTE algorithm; where positive classes were tumors that developed a metastasis in the tested location and negative classes were tumors that did not develop a metastasis in the tested locaiton . BrieflyFeature selection is a method in model building to reduce the dimensionality of a dataset. Overfitting can occur when the number of columns (features) outnumber the rows (instances) we can use for the model. To reduce the dimensionality of the problem we have employed three kinds of feature selection methods: Filter based, Wrapper-based and Embedded feature selection. Chi-square filtering calculates the chi-square metric between the target and the numerical variable and only reduces the features for the variables with the maximum chi-squared values. The SelectKBest, Chi2 and MinMaxScaler Libraries from Sklearn and feature_selection module were used . A RecurIterative Feature selection was conducted by splitting the 60,483 transcript features into 100 blocks of approximately 600 features to be assessed by the above algorithms. We extracted support values for each feature from each selection method. Each block was assessed independent of all other blocks in each classification. Transcripts were filtered for features that showed the highest cross-validated support in multiple or all algorithms. Dimensionality was finally reduced by filtering out co-linear features. The top 10% of highest scoring features were kept from each block for a total number of approximately 5000 candidate transcripts (50 transcripts × 100 blocks). The remaining transcripts were used as the input features in each binary classification. Tree-based models were selected as the best fit for the classification to account for the variability in number selected features in each classification and to allow model attributions to be extracted post-hoc.Random Forest classification and Gradient boosted tree classifiers were built to classify site specific progression from primary tumors. The selected features in each binary classification were used as input attributes into model classification. The model is set to report rounded value for classification but is capable of posterior probability for class likelihood. The code and the pretrained models are available through the documented Github. Model building and usage is documented on the Github wiki page.Feature recapture was the final phase of model building and analysis. Testing the statistical significance of feature recapture in independently generated lists following bioinformatic analysis is an indirect however well documented technique to determine non-random enrichment . Two setThe clusterprofiler package was used to conduct an overrepresentation test in the GO database . The selWe used public data sets drawn from the TCGA database using the GDC data commons for this project and its analyses , 42. We https://github.com/michaelSkaro/Classification_of_organotropic_metastases. The docker image containing all relevant environment variables, dependencies and a demo test data set is also made publicly available on docker hub and integrated into the Github actions. We have a documented wiki page that is available, demonstrating the installations, displays visualization and describes script usage within the pipeline.We have provided a general usage script that runs the entire metastatic classification pipeline. At the command line it can be ran using the metastasis_pipeline.py script within the built docker container. We have provided a general usage feature selection pipeline Feature_selection.py. We have provided the organotropic features sets for all cancer types selected in this study in the Additional file Our code is currently available for view and use in a public Github repository: Each tumor type is unique and potential metastatic sites of progression are limited based on the tumor gene expression profile, anatomic location, and blood circulation . We hypoOverall, the cancer type classification model performed in the excellent range with a macro average precision of 94.2, macro average recall of 91.98 and macro average F1 score of 92.77. The classified results were used to carry forward for site specific metastases prediction. The classification of the primary tumor type significantly decreased the complexity of predicting possible sites of metastatic progression for each primary tumor. We annotated 125 metastatic locations in the ten thousand patient samples separated in twenty-three TCGA projects containing transcriptomic and clinical data , more projects will have clinically annotated tumor and normal transcriptomes. Further, the TCGA database documentation has become more unified and is continuously growing in its clarity. This will allow us to incorporate multiple data types into a multiomic approach that may illuminate genetic, genomic, epigenetic and transcriptomic features working to provide proliferative plasticity in metastatic soils. Finally, if the public data grows by a significant margin, we can approach characterizing organotropic metastasis with an All vs. All model.Our machine learning architecture expands the understanding of the cancer metastasis. The leading cause of cancer associated death is metastatic progression of disease, however incorporating this tool into the clinical timelines for patients may offer clinicians opportunities for pre-metastatic therapeutic interventions. We demonstrate our model can detect if and where metastases will arise. Our methods of synthetic sample generation and feature selection produced a clear and concise biological data-based model of metastatic progression in multiple tumor types. Our recaptured features are offered as candidate biomarkers of site-specific metastatic organotropism.Additional file 1.MOT data tables. Table S1: Tumor vs. Normal metrics: For six tumor types we show the model can detect tumor vs. normal transcriptomic expression profiles. Table S2: Site Specific classification: Site specific metastatic classification metrics. Table S3: Positive and Negative metrics for T/N transcriptomes site specific metastases. Table S4: All binary classification confusion matrices. Table S5: Top 1K selected features for site specific metastases predictions. Table S6: Enriched GO IDs in organotropic metastases. Table S7: Intersection of overrepresented GO processes measured by Fisher’s exact test.