Single-cell data integration can provide a comprehensive molecular view of cells. However, how to integrate heterogeneous single-cell multi-omics as well as spatially resolved transcriptomic data remains a major challenge. Here we introduce uniPort, a unified single-cell data integration framework that combines a coupled variational autoencoder (coupled-VAE) and minibatch unbalanced optimal transport (Minibatch-UOT). It leverages both highly variable common and dataset-specific genes for integration to handle the heterogeneity across datasets, and it is scalable to large-scale datasets. uniPort jointly embeds heterogeneous single-cell multi-omics datasets into a shared latent space. It can further construct a reference atlas for gene imputation across datasets. Meanwhile, uniPort provides a flexible label transfer framework to deconvolute heterogeneous spatial transcriptomic data using an optimal transport plan, instead of embedding latent space. We demonstrate the capability of uniPort by applying it to integrate a variety of datasets, including single-cell transcriptomics, chromatin accessibility, and spatially resolved transcriptomic data. Integrating heterogeneous single-cell multi-omics as well as spatially resolved transcriptomic data remains a major challenge. Here the authors report a unified single-cell data integration framework using an unbalanced optimal transport-based deep network. The computational integration of single-cell datasets is drawing heavy attention toward making advancements in machine learning and data science7 have been devoted to integrating multiple datasets simultaneously profiled from the same cells 8). However, these paired datasets are technically challenging and costly to obtain. Therefore, a vast number of integrative methods have been developed for data profiled from different cells taken from the same, or similar, populations. For example, the celebrated platform Seurat9 projected feature space into a common subspace using canonical correlation analysis (CCA), which maximizes inter-dataset correlation. LIGER10 and DC311 employed non-negative matrix factorization to find the shared low-dimension factors of the common features to match single-cell omics datasets. Harmony12 iterated between maximum diversity clustering and a mixture model-based linear batch correction, providing a latent space in which batch effects are removed. However, these methods rely on linear operation, thus lacking the ability to handle nonlinear deformations across cellular modalities. In addition, they only leverage filtered common genes, while ignoring the importance of dataset-specific genes for the identification of cell populations, which usually capture cell-type heterogeneity not present in common genes13. To address these shortcomings, manifold alignment methods are emerging and have achieved promising results in integrating single-cell multi-omics datasets17. However, manifold alignment methods are limited by relatively high computational complexity, and they are not scalable to large-scale datasets.Among existing single-cell integration methods, tremendous efforts18 and Cobolt19, to utilize cell-paring information. When cell-paring information is unavailable, the alternative is simultaneous training of different autoencoders and aligning cells across different modalities in a latent space. However, this option still makes computation a challenging exercise. Recently, an emerging number of methods have been developed to account for unpaired data. For example, methods like scDART20 and cross-modal autoencoders21 attempted to learn a latent space by autoencoders and align latent representations via kernel-based or discriminator-based discrepancy. However, these methods require global alignment which is often too restrictive for integrate heterogeneous cellular populations. In addition, the transfer learning-based methods were also developed to establish a source atlas via one modality for knowledge  transfer to another modality by learning a modality-invariant latent space23. Although having achieved encouraging results, these methods are restricted to using source modality with annotated cell labels.With the development of deep learning, many autoencoder-based approaches have been proposed and demonstrated their power in data integration across modalities. However, most of them require paired datasets profiled from the same cells, such as DCCA24 and SCALEX25 showed state-of-the-art performance on batch effect correction of one modality, but they have not been benchmarked on single-cell multi-omics data integration. GLUE26, another state-of-the-art method for single-cell multi-omics  integration and integrative regulatory inference, based its development on advanced graph autoencoders. Meanwhile, many other methods are proposed for integrative analysis of spatial transcriptomics (ST) and scRNA data. Among these methods, gimVI27 and Tangram28 achieved the most advanced performance29. However, to the best of our knowledge, no method has been developed for a unified integration of single-cell multi-omics as well as spatially resolved transcriptomic data.Recently published methods for single-cell genomics integration such as scMC30  in the breast cancer region, as well as reveal cancer heterogeneity in microarray-based spatial data.To address this gap, we herein advance the field by developing uniPort, an accurate, robust, and efficient computational platform for integrating heterogeneous single-cell datasets with optimal transport (OT). To overcome the limitation thwarting conventional VAE for single-cell heterogeneous and/or unpaired data integration, we propose a unified computational framework by combining a coupled variational autoencoder (coupled-VAE) and Minibatch Unbalanced OT (Minibatch-UOT)30 Fig. . This fr32 layers. The other is a highly variable gene set through a dataset-specific decoder corresponding to each dataset  and employs a dataset-free encoder to project highly variable common gene sets of different datasets into a generalized cell-embedding latent space. Then uniPort reconstructs two terms. One is input by a dataset-free decoder with dataset-specific batch normalization (DSBN)33 to evaluate the extent of mixing cells across datasets and the Silhouette coefficient34 to evaluate the separation of biological distinctions. To benchmark annotation clustering accuracy, we adopted the adjusted rand index (ARI), the normalized mutual information (NMI), and the F1 scores using cell-type annotations (Methods). Then, for paired datasets, we employed the average fraction of samples closer than the true match (FOSCTTM)14 to measure the preservation of cell–cell correspondence across datasets.We employed a series of scores to assess the performance of single-cell data integration. To quantify dataset mixing and cell-type separation, we computed two scores used by SCALEX: the Batch Entropy score35 on one dataset of paired scATAC and scRNA (the paired PBMC dataset36) and two datasets of unpaired scATAC and scRNA (the microfluidic-based PBMC dataset37 and the mouse spleen dataset13). We employed Uniform Manifold Approximation and Projection (UMAP)38 to visualize the integration results.We benchmarked uniPort against current state-of-the-art single-cell genomics integration methodsWe first applied uniPort to integrate the paired PBMC dataset Fig. . The paiIn addition to integrating the paired PBMC dataset, we further evaluated uniPort on an unpaired microfluidic-based PBMC dataset . Second, we removed the same cell types from scRNA data, while keeping scATAC data unchanged, and denoted the integration task as RNA unbalanced matching (“UBM-RNA”). For comparison, we also defined the integration of complete mouse spleen data as balanced matching (“BM”).uniPort accurately identified and separated the cells of “DC”, “Granulocyte”, “Macrophage” and “NK” from other cell types in the two unbalanced matching cases, while still aligning modality-shared cell types well Fig. a, b. We 31.We further considered the integration of ST and scRNA data. Two main types of ST sequencing technologies are high-plex RNA imaging-based and barcoding-based. High-plex RNA imaging-based spatial sequencing has the advantage of single-cell precision with greater depth, but it is restricted to partial measurement with lower coverage. To test the performance of uniPort over high-plex RNA imaging-based data, we applied uniPort to integrate MERFISH and scRNA data27 and MultiMAP  genes in MERFISH as training genes and reserved the remaining 20%  genes as testing genes. We repeated the above steps twelve times and obtained 12 training and testing gene sets. Afterwards, we trained the uniPort network with each training gene set, and then imputed the corresponding testing gene set. We compared our results with two state-of-the-art gene-imputing methods29: gimVI27 and Tangram28. We applied uniPort, gimVI and Tangram to impute testing genes, and used UMAP to visualize both training and testing genes , as well as the median and average Pearson correlation coefficients (mPCC and aPCC) over imputed and ground truth testing genes. uniPort provided a significant improvement over the two compared methods on the MERFISH dataset. For example, uniPort separated different cell types in the UMAP visualization of imputed genes with a better result  while above SpaOTsc  , structural similarity index (SSIM), root mean square error (RMSE), and Jensen-Shannon divergence (JSD) introduced by a recent paper that benchmarked spatial and single-cell transcriptomics integration methods43.We then applied uniPort to deconvolute real-world barcoding-based ST examples. To estimate cell-type composition for each captured spot and decipher typical organizational structures, we first integrated the anterior slice of adult mouse brain ST data (10× Visium)44. Despite the complexity of its anatomy, uniPort accurately remodeled and arranged the L2/3-L6 subclusters extending from the boundary to the central area ) 4A1 Fig. d, e. Mye49. TLS, a kind of ectopic lymph-like organ recently discovered at sites of tumor or inflammation, are considered as a prognostic and predictive factor for cancer patients. Although TLS are inhabited by multiple cell types, the major residents are T and B cells, implicating the TME by their joint colocalization50. Through decomposing cell-type proportion of each spot, we identified TLS signals via colocalization of T and B cells, rendering an identical expression intensity with T and B cells  tissues for integration, the diameter of which stretches for 100 μm51. Cell-type deconvolution was applied on 428 spots paired with 1926 single cells, measuring 19,736 genes respectively.The area of Visium-based ST data is limited to a 55 μm diameter for each captured spot, which reaches a moderate resolution that translates to 3–30 cells52, preserving dramatically different distributions and genetic characteristics against those of cancer cells. As for malignant pancreatic cells, we grouped them as cancer clone A and B clusters based on genetic differences. Once again, histological annotations of normal and cancerous regions were, overall, in line with their data-driven labels , T cells, and natural killer (NK) cells, in cancer clone A significantly increased , where the weight is an exponentially decreasing function of distances of REs and target genes. The integration result showed that uniPort achieved better performance on the gene activity score by MAESTRO with all evaluated scores higher than that by Signac  approximated by a probabilistic encoder ψ(z∣x). Conversely, a probabilistic decoder ϕ(x∣z) produces a distribution over the possible corresponding values of x given z. The coupled-VAE jointly learns ψ and ϕ to maximize the evidence lower bound (ELBO) with a balanced parameter λ:uniPort inputs each dataset though a coupled variational autoencoder (coupled-VAE) framework and learns its p(z). We set the prior distribution to be the centered isotropic multivariate Gaussian μ and standard deviation vector σ are outputs of the encoder. Then, the latent vector z can be obtained through reparameterization by z = μ + σ × ν, where ν is sampled from The ELBO consists of a reconstruction term that encourages the output data to be similar to the input data, in addition to a Kullback-Leibeler divergence regularization term which regularizes the variational posterior to follow the prior distribution nx cells and dx genes, and ny cells and dy genes. We first select the top k highly variable common genes in both datasets to form Xc and Yc, and the top kx and ky highly variable genes in X and Y to form Xs and Ys, individually. We project both Xc and Yc into a generalized cell-embedding latent space using a dataset-free probabilistic encoder ψ and a decoder ϕ with two Dataset-Specific Batch Normalization (DSBN) layers32 corresponding to two modalities. Then, to leverage the information of dataset-specific highly variable genes, we also introduce two decoders, ϕx and ϕy, to reconstruct Xs and Ys from the latent variables, as well. Overall, the modified ELBO* loss for coupled-VAE is given byx ∈ Xc ∪ Yc, xs ∈ Xs, ys ∈ Ys, and λs, λ are balanced parameters. To better integrate heterogeneous single-cell datasets in the latent space, we design an alignment term for different datasets using Minibatch Unbalanced Optimal Transport (Minibatch-UOT)30. uniPort computes the Minibatch-UOT loss between datasets X and Y described as follows. For two K-dimensional Gaussian distributions xi and yj, where ψ, respectively, the Minibatch-UOT cost between cell xi and yj is defined as  and , and then reparameterize zx and zy by zx = μx + σx × ν and zy = μy + σy × ν;Input both ϕ, and ϕx and ϕy, from zx and zy.Reconstruct C via Eq.  with the ReLU activation function. The decoders have no hidden layer, but directly connect the 16-dimensional latent variables to the output layers with the Human PBMC multi-omics dataset. The paired multi-omics PBMC dataset measuring both DNA accessibility and gene expression data were downloaded from the publicly available 10× Genomic datasets36. The raw gene expression data were processed using the Seurat package (v4.1.0)9. Cells and genes were filtered under default parameters. SCTransform was used for normalization. For paired scATAC-seq data, the fragment files were qualified using Signac (v1.5.0)58 under default parameters. Peaks were called with MACS261. Then latent semantic indexing (LSI) was used for dimensionality reduction, resulting in the binary cell-by-bin accessibility as input for TF-IDF weighting. Consequent dimensionality reduction used Singular Value Decomposition (SVD). The gene activity count matrices were derived using MAESTRO37. Cell types were arranged by transferring labels from an annotated PBMC reference dataset6 using Seurat. Finally, 11,259 cells with 28,307 features in scATAC and 11,942 genes in scRNA were used for the integrative analysis.microfluidic-based PBMC dataset. The unpaired microfluidic-based PBMC dataset was obtained from MAESTRO37, including 1919 cells with 28307 genes in gene activity score of scATAC and 1985 cells with 1477 genes in scRNA. The gene activity score was processed by MAESTRO.Mouse spleen dataset. The processed and annotated scRNA data of mouse spleen was directly obtained from MultiMAP13, containing 4382 cells with 13,575 genes. The gene activity score of scATAC62 was processed by SnapATAC63, containing 3166 cells across 19,410 genes.Synthetic STARmap dataset. The synthetic STARmap dataset was obtained from the benchmarking paper29, containing 14249 cells with 34041 genes in scRNA data and 189 cells with 882 genes in the spot data.Brain scRNA and MERFISH dataset. The scRNA-seq (10×) dataset from the preoptic region of the hypothalamus in six mice was obtained from NCBI GEO accession number GSE11357664. The MERFISH data and annotations were downloaded from Dryad repositories [https://datadryad.org/stash/dataset/doi:10.5061/dryad.8t8s248]. Naive female mouse #1 in the MERFISH data was chosen for integration, and the mouse #2 was extracted to validate the result of online imputation. All cells labeled as ‘Ambiguous’ or ‘Unstable’ were removed from both datasets. scRNA data were qualified and processed using Seurat as above.10× Visium dataset and reference. The series of 10× Visium spatial datasets, including sagittal mouse brain slice  and human breast cancer , were obtained from 10× Genomic datasets36. Spaceranger outputs were obtained and processed with Seurat to generate the standard gene expression matrix. For the mouse brain dataset, the annotated scRNA dataset from adult mouse cortical cell taxonomy from the Allen Institute was chosen as reference, and the subset used was extracted from SPOTlight tutorial43. The processed and labeled scRNA reference for breast cancer slice came from65.Pancreatic ductal adenocarcinoma dataset. The microarray-based spatial transcriptomic dataset of pancreatic ductal adenocarcinoma (PDAC) was acquired from NCBI GEO accession number GSE11167251. The spatial dataset of batch A was adopted together with scRNA-seq generated from the same sample. Cell types in the scRNA reference had already been assigned. These two expression profiles were set as input for uniPort.uniPort preprocessing steps. uniPort preprocessed data as follows: (1) We filtered out cells with fewer than 200 genes and filtered out genes observed in fewer than 3 cells for PBMC data. No cells or genes were filtered out for other data. (2) We normalized total counts of each cell using the scanpy.pp.normalize_total function in the scanpy package66 in Python. (3) We performed log-normalization of all datasets with an offset of 1 using the scanpy.pp.log1p function. (4) We identified k = 2000 highly variable common genes across cells of all datasets, and identified kx = ky = 2000 highly variable genes for each dataset using sc.pp.highly_variable_genes function, respectively. (5) We normalized values of each gene to the range of 0-1 within each dataset by the MaxAbsScaler function in the scikit-learn package in Python. The processed matrix was used as input for uniPort.n_neighbors=15 and min_dist=0.1. All methods adopted gene activity matrices derived from the above data preprocessing steps as input for scATAC-seq. Detailed parameters used in each method are as follows.Seurat. The Seurat package (v4.1.0) was used for all datasets. The raw gene expression profile and unnormalized gene activity matrix were set as input. The matched matrices were log-normalized using the NormalizeData function in Seurat with scaling at 10000 for cell-level normalization separately. Then the FindVariableFeatures function was used to pick the top 2000 HVGs for scRNA. The anchors between scRNA and scATAC were acquired using the FindTransferAnchors function, where ‘cca’ was set as the reduction method with features from HVGs in the scRNA dataset. The imputed scATAC data used the TransferData function, where anchors were weighted by latent semantic indexing. Then scRNA and imputed scATAC datasets were merged and reduced using PCA with the RunPCA function under default 50 principal components (PCs). UMAP was adopted for visualization at 30 PCs using the RunUMAP function. For MERFISH data, the FindIntegrationAnchors and IntegrateData functions were used for integration after common feature selection with the SelectIntegrationFeatures function. Then the integrated matrix was scaled using the ScaleData function, and PCA and UMAP were applied to all at 30 PCs. For the 10X Visium datasets, the cell-type assigning probabilities of spots were derived using the FindTransferAnchors and TransferData functions, where spatial datasets were reduced using PCA at 30 PCs.Liger. The R package rliger (v1.0.0) was used for integrating scATAC and MERFISH data with scRNA data. The normalization and selection of HVGs are the same as above steps in Seurat. For all datasets, the number of factors was set to 20 at default in the optimizeALS function, which was done by calling the RunOptimizeALS function in the SeuratWrappers package (v0.3), and then building a shared factor neighborhood graph and quantile normalization of corresponding clusters applied through the RunQuantileNorm function.Harmony.The R package harmony (v1.0) was used for integrating scATAC and MERFISH data with scRNA data. The normalization and selection of HVGs are the same as above steps in Seurat. Then PCA was performed using the RunPCA function in Seurat at 50 PCs. Harmony refined the results of PCA at top 30 PCs.SCALEX. The Python package scalex (v0.2.0) was used for integrating scATAC and MERFISH data with scRNA data. The data preprocessing steps are the same in uniPort and SCALEX. Therefore, we input the same preprocessed data without common genes as those in uniPort. Embedded preprocessing of the scalex package was ignored. We ran SCALEX using the SCALEX function with parameters set as default.MultiMAP. The Python package MultiMAP (v0.0.1) was used for integrating scATAC and MERFISH data with scRNA data. We followed the tutorials of MultiMAP on GitHub. We first applied the sc.pp.normalize_total and sc.pp.log1p functions to raw counts without scaling. We then processed data with the MultiMAP.TFIDF_LSI function for dimensionality reduction of ATAC peaks, sc.pp.scale and sc.pp.pca functions in Python package scanpy for dimensionality reduction of scRNA and MERFISH data, and the MultiMAP.Integration function for integration with all parameters set as default, as suggested in the pipeline in the GitHub repository.scVI. The Python package scvi-tools (v0.17.1) was used for integrating scATAC and MERFISH data with scRNA-seq data. We followed the tutorials of scVI on GitHub. We first performed sc.pp.normalize_total and sc.pp.log1p functions for input data. Then we selected the top 2000 highly variable genes using the sc.pp.highly_variable_genes functions with flavor="seurat_v3". After that, scvi.model.SCVI and scvi.model.SCVI.setup_anndata functions were used for model initialization.scMC. The R package scMC (v1.0.0) was used for integrating scATAC-seq and scRNA-seq data. The normalization and selection of HVGs are the same as the above steps in Seurat. Then we scaled data by ScaleData function and performed scMC through RunscMC function. We searched parameter similarity.cutoff of RunscMC function within {0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9} for the best integration result. All other parameters were set as default.GLUE. The Python package scglue (v0.2.3) was used for integrating PBMC and mouse spleen examples. We followed the tutorials of GLUE on GitHub. We first performed sc.pp.normalize_total, sc.pp.log1p and sc.pp.scale functions for input data with default parameters for scRNA data. Then we used sc.tl.pca function with n_comps=100 and scglue.data.lsi function with n_components=100 for scRNA and scATAC data, respectively. We chose prob_model=‘ZINB’ for the PBMC example and prob_model=‘Normal’ for mouse spleen example in scglue.models.configure_dataset function. All other parameters were set as default.SCOT. We downloaded SCOT (v1.0) from https://github.com/rsinghlab/SCOTfor integrating scATAC-seq with scRNA-seq data. Because SCOT did not provide a data preprocessing tutorial, we compared different approaches and used the one with the best result. To be specific, we performed uniport.TFIDF_LSI function with default parameters for ATAC peaks, which reduced the dimension to 49. Then we performed sc.pp.normalize_total, sc.pp.log1p and sc.pp.pca functions with default parameters for RNA data. We set all parameters as default in scot.align function.Pamona. The Python package pamona (v0.1.0) was used for integrating scATAC-seq and scRNA-seq data. All data preprocessing was performed in the same manner as SCOT.Tangram. The Python package tangram-sc (v1.0.3) was used for imputing MERFISH through scRNA-seq data. We followed the tutorial of Tangram on GitHub. First, tg.pp_adatas function was applied to pre-process both training MERFISH and scRNA data. Then, we performed tg.project_genes function to find the project matrix, which was used to impute MERFISH test genes. All parameters were set as default.gimVI. The model gimVI in the Python package scvi-tools (v0.17.1) was used for imputing MERFISH through scRNA-seq data. We followed the tutorial of scvi-tools in GitHub, and used GIMVI.train function to train the model in 200 epochs. All parameters were set as default.For the integration among scATAC-seq, scRNA-seq and spatial transcriptomic datasets, we benchmarked the performance of uniPort (v1.1.1) against the following methods: Seurat (v4.1), Liger (v1.0.0), and Harmony (v1.0) in R environment (v4.0.2), SCALEX (v0.2.0), MultiMAP (v0.0.1), scVI (v0.17.1), scMC (v1.0.0), GLUE (v0.2.3), SCOT (v1.0), Pamona (v0.1.0), and Tangram (v1.0.3) in Python environment (v3.8.13). We compared their co-embedding UMAP visualization with parameters k-Nearest-Neighbor (kNN) classifier by the sklearn.neighbors.KNeighborsClassifier function based on cell-type annotations and UMAP coordinates of reference data  embeddings in the common latent space. Then, we applied the well-trained kNN classifier to predict cell-type annotations of query dataset  embeddings, and calculated the ARI, NMI and F1 scores by real and predicted query cell type annotations. All results are based on UMAP visualization with parameters n_neighbors=15 and min_dist=0.1.Adjusted Rand Index. Adjusted Rand Index (ARI) is introduced to determine whether real and predicted cell-type clusters are like each other. The Rand Index (RI) computes the similarity by taking all points identified within the same cell-type cluster. The Adjusted RI (ARI) is the chance-corrected version of the Rand index and calculated with RI asNormalized Mutual Information. Normalized Mutual Information (NMI) is a variant of a common measure in information theory called Mutual Information. It is calculated asF1. The F1 score combines the precision and recall of a classifier into a single metric by taking their harmonic mean. It is calculated ask-NN classifier.The accuracy of cell-type assigning was quantified by the adjusted Rand Index (ARI), Normalized Mutual Information (NMI) and F1 score. To be specific, we trained a 15 to assess cell-to-cell neighborhood preservation for paired datasets profiled from the same cells. All these systematic benchmarks were applied in scRNA-seq, scATAC-seq and MERFISH datasets. All results are based on UMAP visualization with parameters n_neighbors=15 and min_dist=0.1.Silhouette coefficient. The Silhouette coefficient is calculated using the mean intra-cluster distance (a) and the mean nearest-cluster distance (b) for each sample. The Silhouette coefficient for a sample is calculated asBatch Entropy score. It was derived from SCALAX25 inspired by “entropy of batch mixing”33. It evaluates the sum of regional mixing entropies at the location of randomly chosen cells from different datasets where a high score indicates cells from various datasets mixing well. This can be calculated asPi is the proportion of cell numbers in each batch to the total cell numbers, and pi is the proportion of cells from batch i in a given region. We calculated the Batch Entropy score only based on cells from cell types that are common in different batches.average FOSCTTM. FOSCTTM refers to “fraction of samples closer than the true match”. It was used to evaluate the relationship preservation of cell-to-cell pairings’ neighborhood. It was calculated on two datasets with known cell-to-cell correspondence information. The average FOSCTTM is calculated asn is the number of both dataset X and Y, si and ti are the number of cells closer to the i-th cell in a dataset than its true match in another dataset. The average FOSCTTM ranges from 0 to 1, and lower values indicate higher accuracy.To assess the separation of clusters and batch mixing, the Silhouette coefficient and Batch Entropy score were adopted. Furthermore, we employed average FOSCTTMhttps://cran.r-project.org/). The detailed statistical tests were indicated in figures or associated legends where applicable. No statistical method was used to predetermine sample size. No data were excluded from the analyses. Complete randomization was performed for allocating groups. Our study does not involve group allocation that requires blinding.All statistical calculations were implemented in R (v4.0.2; Further information on research design is available in the Supplementary InformationPeer Review FileDescription of additional Supplementary FileSupplementary Data 1Reporting Summary


Ac; and the bicluster average probability p. Three simulation experiments and two real biology mRNA and protein data analyses conducted on the well-known Cancer Genomics Atlas (TCGA) and The Cancer Cell Line Encyclopedia (CCLE) verify that the proposed Bi-EB algorithm can significantly improve the clustering recovery and relevance accuracy, outperforming the other seven biclustering methods—Cheng and Church (CC), xMOTIFs, BiMax, Plaid, Spectral, FABIA, and QUBIC—with a recovery score of 0.98 and a relevance score of 0.99. At the same time, the Bi-EB algorithm is used to determine shared the causality patterns of mRNA to the protein between patients and cancer cells in TCGA and CCLE breast cancer. The clinically well-known treatment target protein module estrogen receptor (ER), ER (p118), AR, BCL2, cyclin E1, and IGFBP2 are identified in accordance with their mRNA expression variations in the luminal-like subtype. Ten genes, including CCNB1, CDH1, KDR, RAB25, PRKCA, etc., found which can maintain the high accordance of mRNA–protein for both breast cancer patients and cell lines in basal-like subtypes for the first time. Bi-EB provides a useful biclustering analysis tool to discover the cross patterns hidden both in multiple data matrixes (omics) and species. The implementation of the Bi-EB method in the clinical setting will have a direct impact on administrating translational research based on the cancer cell screening guidance. Although several biclustering algorithms have been studied, few are used for cross-pattern identification across species using multi-omics data mining. A fast empirical Bayesian biclustering (Bi-EB) algorithm is developed to detect the patterns shared from both integrated omics data and between species. The Bi-EB algorithm addresses the clinical critical translational question using the bioinformatics strategy, which addresses how modules of genotype variation associated with phenotype from cancer cell screening data can be identified and how these findings can be directly translated to a cancer patient subpopulation. Empirical Bayesian probabilistic interpretation and ratio strategy are proposed in Bi-EB for the first time to detect the pairwise regulation patterns among species and variations in multiple omics on a gene level, such as proteins and mRNA. An expectation–maximization (EM) optimal algorithm is used to extract the foreground co-current variations out of its background noise data by adjusting parameters with bicluster membership probability threshold  Co-regulated gene module detection will assist us in identifying its biological functions or molecular pathways. Conventional clustering methods uncover co-expressed genomic profiles across all samples; however, they cannot detect shared patterns in a subset of genes and among a subset of samples, called co-clusters or biclusters. The bicIn these empirical Bayesian models, Gibbs sampling schemes were developed and implemented to estimate the model parameters and detect biclusters for the underlying probabilistic distribution inner data. The empirical Bayes mixture model is a valuable alternative approach for Bayesian models. Computationally, its expectation–maximization (EM) algorithm is usually less expensive than Bayesian model’s Gibb sampling approaches. Chekouo and Murua (2015)  recentlyAs compared to all the existing clustering algorithms, in this paper, we made four major innovative contributions to the empirical Bayes mixture model to detect biclusters. Firstly, our model provides a different variance structure between the biclusters and the background. Secondly, our model assumes that sample means and row means follow normal distributions too. In other words, we use random-effect model formulation instead of fixed effects. Thirdly, for the first time, we provide comprehensive EM algorithm derivation for the biclutering mixture model. Fourthly, we recognize that the EM algorithm itself only provides the probabilistic estimates for the biclustering memberships, but it does not really group the data into biclusters. We further develop an algorithm that automatically searches and groups data into biclusters based on the probabilities estimated after the EM algorithm.The Cancer Genomics Atlas (TCGA) [The Cancer Cell Line Encyclopedia (CCLE) [Large-scale omics profiling has been conducted to investigate the molecular signatures of diseases. Using fast-evolving high-throughput technologies, including transcriptome, DNA copy number alterations, and proteomic data, can provide us with tremendous opportunities to examine disease-specific biological pathways and molecular functions. For example, s (TCGA)  and The a (CCLE)  are exema (CCLE) ,13, and Cancer subtype stratification has become a critical component of disease characterization. Research efforts have focused on how the classification of these subtypes could provide information on influence treatment planning . ClusterTumor-derived cell lines have long been used to study the underlying biologic processes in cancer, as well as screening platforms for discovering and evaluating the efficacy of anticancer therapeutics. Proper cell models for cancer primary tumors have long been the focal point in cancer-based research . The ideIn this paper, an empirical Bayesian biclustering (Bi-EB) algorithm is proposed to identify translational gene sets shared between cancer cell lines and primary tumors based on mRNA and proteomic data or copy number variations (CNVs) and mRNA data. An EM algorithm is developed to conduct estimation and inference in the bicluster analyses. Our bicluster searching starts from a seed, such as a druggable target gene, and it detects interesting gene modules shared between cancer cell lines versus patient tumor samples. Using Bi-EB, gene modules of mRNA and proteomics are explored in two breast cancer subtypes: luminal A/B and basal-like subtypes.The article is organized as follows. In the Results section, Bi-EB is used to search for shared gene modules between patient tumor versus breast cancer cell line samples in datasets TCGA and CCLE. In http://www.broadinstitute.org/ccle, accessed on 2 February 2022) [http://cancergenome.nih.gov/, accessed on 1 March 2022) [https://cancergenome.nih.gov/abouttcga/overview, accessed on 1 March 2022) has now provided detailed molecular compositions for over 11,000 cancer patients’ whole-genome sequencing, RNA-seq, and RPPA data from at least 33 anatomic sites. The RPPA and mRNA expression, copy number, and mutation profiles from TCGA and CCLE are used to calculate similarities between tumors and cell lines. To simulate breast cancer, these subtypes and data refer to literature datasets [Emerging next-generation sequencing (NGS) and microarray techniques, as well as large-scale cancer screening data, can help to achieve this goal. Databases such as the database CCLE ry 2022)  providesch 2022)  project datasets ,11. Then22  providatasets  models fI genes (rows) and J samples (columns). It consists of the grand mean; the between-gene (row) variation; the between-sample (column) variation; and noise from the background and bicluster, respectively. Please note that the sample set contains both primary tumors and cancer cell lines. Denote Y can either be the transcriptome, proteome, or their ratios. We assume that ijy follows a mixture model.The Bi-EB model is used to identify co-regulated biclusters across tumors and cancer cells. B. The between-gene variationB (if In model (1), The EM AlgorithmE step is an iterative marginal distribution used to find the  maximum likelihood (Equation (7)) in the assignment of genes and arrays to biclusters using Formulas (4)–(6). The M step is used to look for the optimal parameters of the model using Formula (8) until the difference of  maximum likelihood reaches the threshold in iterations t and t+1 (Formula (9)). The EM algorithm follows four steps: In the expectation–maximization (EM) algorithm, the (i) Starting Values:(ii) E-Step:E step as follow:t = 1, 2, …, N is the number of iterations; i and j refer to the gene and sample set, respectively, with parameter space of We calculate the M step is used to calculate the maximum log-likelihood (iii) The t, i, and j refer to the number of iterations, the gene set, and the sample set, respectively, as in (8). Based on the E step, the posterior distribution Convergence criteria: The algorithm stops when the relative change in the log-likelihood is sufficiently small.(iv) P according to the following conditions: (i) the bicluster contains as many data points as possible under a certain probability threshold; (ii) the bicluster includes points with different conditions, such as tumor samples and cancer cell lines; (iii) special genes have a higher probability to members of the bicluster. Typically, drug target genes, oncogenes, or tumor suppressers are of interest. The algorithm requires user-defined three parameters in order to search for the specific bicluster. (i) The probability thresholdsign functionAc is a sensitivity parameter. A high-value c increases the accuracy of the bicluster, but results in fewer genes and samples in a bicluster. (ii) The average of probability values in the bicluster, Ac = 0.8 and The EM algorithm fits the empirical Bayes biclustering model and produces a probability matrix of the biclustering membership P = {P is then transformed into a binary matrix U based on the probability thresholdi and condition j belong to the bicluster, and 0 otherwise. Genes and samples with the highest number of 1s will be arranged to the right corner of matrix U. If samples are from two different groups , the sorting process will be applied separately on them. The final bicluster B is defined by a sub-matrix of ,U in which at least To evaluate the Bi-EB algorithm’s performance, we compare seven different bicluster algorithms to ours. The seven algorithms, including Cheng and Church (CC) , Plaid , xMOTIFsThe performance of bicluster algorithms heavily depends on their parameter setting. In order to optimize the biclustering result, parameters are specifically set for each synthetic data in multiple steps. In each algorithm, its parameters are given a vector of values. A final value is chosen based on the performance of the algorithm under that value and its combination with other parameters. The measurements used to evaluate the performance of each simulation are discussed in the following sections.Three synthetic datasets with known patterns of biclusters are used to evaluate the Bi-EB model’s performance. They include constant, row scale-shift, and column scale-shift biclusters. The performances of Bi-EB and seven other algorithms are compared for these simulation data. All simulations are run in R version 3.2.4 and RStudio version 0.99.902.N. The background noise is randomly chosen from the Gaussian distribution of N independently. (ii) The row scale-shift bicluster has a size of 70 × 70 with scaled-shifted rows. The bicluster rows are shifted and scaled from base rowThree synthetic datasets of size 200 × 300 with one embedded bicluster are simulated. (i) The constant bicluster has a size of 25 × 25 with the standard Gaussian distribution (i.i.d.) sets see  are usedJaccard coefficient, recovery, and relevance are used for this comparison. The Jaccard index indicates the relative overlap between two biclusters. Let b1 and b2 be two biclusters, and the score s is defined to compare two biclusters by the function:The algorithms’ performances on synthetic data are evaluated by comparing extracted biclusters with pre-defined biclusters. We follow methods proposed by Eren et al.  and PengT in the simulation model and bicluster R, estimated from the data. The recovery function is defined by:The recovery function refers the comparison between the true bicluster The score ranges from 0 to 1. A recovery score is interpreted as the percentages of the true set that is recovered from the bicluster analysis results. Relevance is another function used to evaluate the similarity between a true bicluster and a bicluster result set. The relevance function is calculated by:R ⊆ T, the score is 1. The relevance score indicates the percentage of the bicluster result set that is shared with the true biclusters.Similar to recovery, relevance score ranges between 0 and 1. If all the bicluster result sets are in the true set, meaning Ac’ is set as 0.8 and 0.6 for constant-, row-, or column (col)-scales biclusters, respectively. Parameter ‘curve) in θ are set based on the estimates from one time run of the likelihood function. With the initial values of θ is given a vector of 4 values for each parameterAc’ to 0.6, a recovery score of 1 is achieved. θ is kept the same as the constant pattern. The Bi-EB algorithm successfully reports a recovery score of 0.9 and a relevance score of 0.989. A similar simulation in col-scaled data results in a recovery score of 1. As The Bi-EB parameter ‘Ac andAc are chosen from (0.2 to 0.8) in implementing our Bi-EB algorithm. When Ac is 0.2, the recovery score identified by the Bi-EB algorithm is 0.85. To further investigate the impact of parameter settings in bicluster identification, we compare simulation results over a vector of values for and pave d,e underIn order to evaluate the performance of our algorithm, we compare our biclustering results with seven other bicluster algorithms. We adopt the comparison framework recommended by Eren  and Sun Ac results in higher percentages of identified true points. The highest recovery value of 0.99 is observed when a Ac value reaches 0.8 and stays constant afterwards. However, at c = 0.8, a recovery score drops by 5% compared to c = 0.6. Thus, we chose Ac= 0.6 which extracts higher percentages of true points with a recovery score of 0.98. Next, we fix the value of Ac at 0.6 and change On the other hand, as for the parameter setting to Bi-EB, as In the row/column scale-shifted simulation datasets, our algorithm outperforms all other algorithms in finding scale-shifted row and column biclusters. Bi-EB has a recovery and relevance score of 1. In row scale pattern data, BiMax, Plaid, and xMotif algorithms have recovery scores of 0.008, 0.0183, and 0.002, respectively. In the column scale-shifted simulation, the Cheng and Church (CC) algorithm has a recovery score of 0.6, while the resulted biclusters in row scale-shifted data using the CC algorithm are very poor (recovery score = 0.0011 and relevance score = 0.00014). The FABIA can find 18% of true biclusters (recovery score = 0.18) with a scale-shifted pattern in rows, whereas the result with the same pattern in columns only has a recovery score of 0.07. QUBIC also fails to find scale and shift patterns in either row or column (with the average of recovery and relevance around 7%). Spectral only has a relevance score of 2% and a recovery score of 30%.We develop a novel biclustering algorithm using the empirical Bayes mixture model, called Bi-EB. The model can search for a bicluster in two directions simultaneously: multi-omics data  and the protein amount (PA)) and multi-conditional samples (such as tumors and cancer cell lines). The bicluster is characterized through a hierarchical model structure built upon normal distributed log-transformed ratios between mRNA expressions and protein expressions (GEs/PAs). Bi-EB is used to search these block clusters from the ratio matrix across multi-conditional samples.The Bi-EB algorithm is used to seek shared molecular profiles in order to facilitate the translational research between breast cancer cell lines and tissue samples in different subtypes on GE/PA ratio from both cancer cell lines and tissue samples. Here, it is our best interest to use the Bi-EB algorithm to identify shared GE/PA ratios to seek common variation patterns in luminal A/B and basal-like subtype breast cancer cell lines and tumors. Gene expression data are derived from TCGA RNA sequencing, and the protein amount is taken from the reverse-phase protein array (RPPA).https://gdac.broadinstitute.org/, accessed on 1 March 2022). The protein amount is taken from the RPPA data, an antibody-based protein assay platform, which is compared to the number of gene features in the gene-level data.The TCGA breast cancer dataset is obtained from the Broad Institute GDAC Firehose  to be found. The iterative Bi-EB bicluster selection requires initial values for two parameters: the bicluster membership probability threshold (i)The luminal A/B subtypeCancer Genome Atlas breast invasive carcinoma (TCGA-BRCA) data collection 269 tissue samples [Luminal A/B is one the well-known subtypes of breast cancer with the most successful targeted therapy drugs comparing to the other subtypes. Since the ER status of luminal subtype is positive, we expect to see ER-α’s GE/PA ratio in our bicluster. Therefore, the bicluster containing ER-α’s GE/PA in the luminal subtype shall serve as a positive control in our Bi-EB analysis. The input data of the Bi-EB algorithm represent a matrix of 45 gene GE/PA ratio with 279 samples (CCLE breast cancer 10 cell lines and the  samples ). Since cell line and tissue measurements have different platforms, we normalize both protein and mRNA expressions separately for cell line and tumor. To explain the Bi-EB algorithm, hierarchical clustering is used to cluster each mRNA and RPPA protein cluster in tumors and cancer cells separately, as shown in p < 0.05) as potential targets.We further investigate the gene expression co-variations between GE and PA within cancer cells and tumors extracted by our Bi-EB algorithm . Figure (ii)The Basal-like subtypeIn the basal-like subtype breast cancers, they are usually triple-negative, i.e., they lack the estrogen receptor (ER), the progesterone receptor (PR), and the human epidermal growth factor receptor 2 (HER2) . Due to Then, we further explore the variations in the GE/PA ratios of this bicluster. The overall GE/PA ratio pattern of these 10 genes from 83 cell lines and tissue samples is shown in Therefore, our Bi-EB algorithm uses genes CCNB1 and RAB25 as examples. A covariation of the ratio for CCNB1 (golden line) in The genes selected by the Bi-EB algorithm are important genes in the basal-like subtype. In this intrinsic subgroup, CCNB1, CDH1, KDR, RAB25, and PRKCA have shown significant correlations of mRNA-RPPA with d p<0.05 . AccordiInput data preparation is a crucial step in biclustering algorithms. It is important to use appropriate normalizing method that suits the integration of multiple-omics data analysis ,34. Bi-Eavep: the accuracy of a bicluster and Ac: the probability of belonging to a bicluster) are designed to regulate the joint and marginal variations and obtain the optimal bicluster. At first, the Bi-EB method uses an EM algorithm, and conducts a data-driven search for bicluster patterns in the expression data. The algorithm outputs the probability of each data point belonging to a potential bicluster. If we set Ac = 0.85 as the probability to assign a gene to the bicluster, all genes in the bicluster shall have an 85% membership probability or higher. Bi-EB is used for a dot possibility calculation in a bicluster. Hence, its speed is not as fast as other biclustering models, which will be improved in future. Bi-EB is an automatic feature detection model on a pixel level (dot level) in a matrix, where the dot variation in the matrix is a joint variation from its associated row and column variation . Two important parameters  with their related protein. In a recent study, Li et al. [DESeq2 and identified differential mRNA expressions in TNBC patients. Some of the genes reported in Li’s study, such as KDR, COL6A6, PGR, and CCNB1, were also found in our basal-like bicluster. KDR, COL6A6, and PGR are identified as differentially down-regulated expressed genes in TNBC, while CCNB1 is identified as a significant up-regulated gene [We utilize the Bi-EB algorithm to determine genes with a shared pattern across the subgroup of TCGA and CCLE breast cancer samples. From our results, extracted genes in the basal-like bicluster are all highly correlated (i et al.  performeted gene . A heatmCurrently, data integration approaches used to efficiently identify subtypes’ genomic variations among existing samples have recently gained attention. Tensor factorization and multi-view correlation analysis methods were applied for either dimension reduction or clustering to provide more amenable data representations for cancer classification and fusion patterns across multi-omics data types among different cancer types ,32, suchThe genome molecular features shared between cell lines and tumors offer valuable insight into discovering potential drug targets for cancer patients. Our previous studies demonstrate that these important drug targets in breast cancer, ESR1, PGR, HER2, EGFR, and AR have a high similarity in mRNA and protein variations in both tumors and cell lines ,29. Base


A challenge in bulk gene differential expression analysis is to differentiate changes due to cell type-specific gene expression and cell type proportions. SCADIE is an iterative algorithm that simultaneously estimates cell type-specific gene expression profiles and cell type proportions, and performs cell type-specific differential expression analysis at the group level. Through its unique penalty and objective function, SCADIE more accurately identifies cell type-specific differentially expressed genes than existing methods, including those that may be missed from single cell RNA-Seq data. SCADIE has robust performance with respect to the choice of deconvolution methods and the sources and quality of input data.The online version contains supplementary material available at (10.1186/s13059-022-02688-w). The past three decades have seen rapid development in gene expression analysis using microarray and sequencing technologies, where bulk samples are analyzed to answer specific biological questions, e.g., to identify genes with different expression levels between cancer samples versus controls. Because bulk samples contain many distinct cell types, such analyses only provide limited granularity. Most differential expression analyses on bulk samples often assume that the measured gene expression is from the primary cell type, e.g., tumor cell in bulk tumor sample.Recent progresses in single-cell RNA-sequencing (scRNA-seq) techniques have demonstrated substantial heterogeneity in bulk samples. However due to the high cost and complexity for scRNA-seq, most available data have remained to be from bulk samples. To make better use of bulk sample data, many in silico deconvolution methods have been proposed to infer cell type proportions from bulk data. Most deconvolution methods assume that the observed bulk gene expression profile is a convex mixture of cell-type specific gene expression profiles, i.e., Y is the bulk gene expression matrix with m genes and n samples, W is the cell type-specific gene expression matrix of the k component cell types, and each column in H represents the cell type proportions for the corresponding bulk sample. All entries in these matrices are non-negative, which is indicated by the “+” sign in the notations.Here The principle behind the designs of most existing deconvolution methods is to utilize genes that have distinct expression levels across cell types to infer cell type proportions. To this end, some methods curate a signature matrix ll types , 29, 41 W¯~∈Rm×k . Both geW¯~∈Rm×k , 40, 42.Y1=W1H1 and Y2=W2H2, when an analysis is performed assuming W1=W2, it intrinsically over-attributes changes to cell type proportion changes.Although enormous insights on cell type proportion changes have been drawn from the applications of these deconvolution methods, most of these downstream analyses were performed under the scheme of single signature matrix, i.e., the same signature matrix was used for different groups of bulk data. In real data analyses, a more appropriate model would be that the observed differences in the bulk samples result from not only cell type compositional changes, but also from changes in cell type-specific gene expression profiles. In mathematical terms, it is Ws and Hs in a two-group comparison setting, thus to accurately infer cell type-specific differentially expressed genes (DEGs) as well as cell type proportion changes. To this end, we present a smoothly clipped absolute deviation-based (SCAD) iterative estimation (SCADIE) framework that can address this challenging problem.In this article, we aim to simultaneously estimate group-specific ℓ1 penalty using the derivative of SCAD are widely used in the penalization methods [a,ζn>0 are parameters to be tuned. SCAD can be viewed as a hybrid of ℓ0 and ℓ1 regularizers in the sense that it resembles the ℓ1 norm in a neighborhood of the origin, but stabilizes to constant at larger values [ℓ1 penalty.The SCAD penalty and weighted  methods , 21. SCAWs and Hs. Its underlying assumption is that the cell type-specific W1,W2 are reasonably similar but not exactly the same, thus it is possible to initialize with the same W and use an iterative algorithm to search for optimal group-specific Ws. Through comprehensive simulation and real data analyses, we demonstrate that SCADIE is capable of identifying cell type-specific DEGs between Ws while maintaining high accuracy in estimating Hs.At high level, SCADIE is built on existing supervised deconvolution methods. It takes bulk gene expression along with a common signature matrix or initial cell type proportions as input and then estimates group specific W1,H1 and W2,H2 from bulk data Y1,Y2 and then perform hypothesis tests to identify cell type-specific DEGs. In the most common deconvolution scenario, only Y1,Y2 and a shared signature matrix Wsub with only signature genes to group-specific W1,W2 containing all genes remains to be a challenge.The goal of the SCADIE framework is to estimate matched W1 and W2 are not differentially expressed. This should especially hold for signature genes in Wsub for initialization, nor is SCADIE applicable. Second, the compositional cell types in W1,W2 should remain the same; otherwise, different models should be used for group 1 and group 2. Common applicable scenarios include tumor microenvironments between different responding groups, or different subtypes of the same disease.In view of this challenge, we assume the following conditions hold in our estimation setting: First, most entries in Jointly estimate cell type proportions for both groups, obtaining mentclass2pt{minimWs and Hs for a few rounds using non-negative least squares (NNLS).Obtain the initial estimates of E used for the main estimation procedure with the derivative of SCAD function [Derive the weight matrix function   penalty-based iterative estimation procedure (SCADIE) that consists of the following steps: The above procedure outputs W-based H update might result in reduced H estimation accuracy. To evaluate SCADIE’s performance on cell type proportion estimates, we benchmarked SCADIE against four deconvolution algorithms, including DWLS[W. We tested these four methods on a simulated data set, a pseudo-bulk data set[Hs: K-L divergence and root-mean-squared error (RMSE). K-L divergence is a suitable measure because it measures the distance between two sum-to-1 discrete distributions, which are the same format as cell proportions, while RMSE is widely used by previous deconvolution methods.One concern of SCADIE’s algorithm is that its iterative procedure and full ding DWLS, CIBERSOding DWLS, MuSiC[4ding DWLS, and a n data set, and a b data set. We usedH accuracy compared to CIBERSORTx, DWLS, MuSiC, and NNLS-iteration, measured by K-L Divergence and RMSE, respectively. The overall result patterns between these two metrics are very consistent. In terms of performance, SCADIE showed equal or better accuracies than the other four methods except in the mouse ISC pseudo bulk dataset, where MuSiC substantially outperformed all other methods. MuSiC is specifically tailored for single cell count data, and its significantly better performance in the single cell data suggests the same. Besides, although the NNLS iteration only differs from SCADIE in its W-update step, the results were substantially inferior. This was due to the uncontrolled changes in Ws over iteration and it highlights the importance of using the SCAD penalty  for each method on each dataset in Fig. Here we benchmarked SCADIE against the above four methods on one simulated dataset and two pseudo-bulk datasets generated from scRNA-seq data. Because these were simulated data with known DEG statuses, we were able to measure the true-positive and false-positive rates , SCADIE’s output is highly robust in terms of H estimates, W estimates, and the DEG identifications  is a chronic lung disease and a leading cause of death. Many efforts have been made to profile the transcriptomes from COPD patients , 28, 32.Although COPD causes pathological changes in several myeloid, epithelial, and endothelial cell types, previous studies did not find any systematic changes in cell type proportions associated with COPD , 32. Reap values are shown in volcano plots in Fig. We next evaluated SCADIE’s performance in identifying cell type-specific DEGs from bulk data. “Ground truth” cell type-specific DEGs were first obtained by performing differential expression analysis on the COPD-Control single cell data cohort for the Note that from Supplementary Finally, we asked the question whether SCADIE can infer DEGs that were not identified in single cell data. To answer this question, we looked into the top 10 SCADIE-inferred down DEGs in each cell type. Among the 50 genes, only 8 of them were also identified from single cell DE analysis. Among the 42 DEGs unique to SCADIE, 39 were also present in the single cell dataset, and 33 showed concordant cell type-specific directional changes as SCADIE. This finding suggests that the cell type-specific expression changes from SCADIE were likely true signals, and the main reason they were missed in single cell data was the limited power due to data sparsity  is the most common type of non-Hodgkin lymphoma and can be classified into two main subtypes based on gene expression: germinal center-like (GCB) and activated B cell-like (ABC) DLBCL , 19. TraHere we analyzed a dataset consisting of 414 samples from both subtypes who received either CHOP or R-CHOP . InspireNext, we compiled a list of validated markers for GCB and ABC from published studies , 10  is a leading threat to global elder population and has been under intensive research over decades. However, gene expression analyses remain challenging due to the difficulty in accessing samples and the low quality of post-mortem RNA-Seq samples. To examine SCADIE’s performance on these challenging data, we applied SCADIE to an AD bulk RNA-Seq dataset where the cell type proportions had been measured by immunohistochemistry (IHC) . The datHC study  and DEG HC study , we onlyWs, respectively. In the initial Ws, 34%, 20%, and 14% of known DEGs were estimated to have fold-changes in the opposite directions -DEGs were estimated to be down(up)-regulated) with | log2FC|>1) in astrocyte, oligodendrocyte, and microglia. After SCADIE procedure, these ratios reduced to 5%, 9%, and 7%, respectively  bulk dataset with both issues present. FL is a common type of B cell lymphoma and can be classified into two subtypes by the presence of CREBBP mutation . We re-aH via deconvolution using CIBERSORTx’s LM22 signature matrix. Since this H contains 22 cell types while our sample number and leave-one-out jackknife require the number of cell type not exceeding 9, we merged H into eight main cell types based on their abundances and similarities. Then, the initial Ws were obtained for CREBBP-mutant and CREBBP-wild type groups respectively. To measure the qualities of the initial Ws, we made the volcano plot for known DEGs in initial Ws. The results show few correctly identified down-DEGs with some false positives  in the initial Ws. After iteration, the final Ws correctly identified more DEGs in both directions . This is consistent with the table in Fig. We inferred the initial Ws can identify more DEGs in both directions, though at a cost of higher false positives.We next compared SCADIE with CIBERSORTx’s high resolution mode and TOAST. Since CIBERSORTx cannot perform whole-transcriptome imputation, we only input the 467 known DEGs for imputation. The results suggest that although SCADIE’s estimations had more directional errors, its overall power Fig. a and numH’s quality, we recommend having sample size at least 1.5x the number of cell types when performing DEG identification with SCADIE.The above results suggest that incorrect initialization and limited sample size do lower the estimation quality of SCADIE; however, SCADIE can still improve and maintain competitive performance in these scenarios. It is recommended to consider SCADIE’s applicability since poor initialization quality or limited sample size is common in real applications. Though it is usually not possible to evaluate initial Recent years have seen rapid developments of technologies and computational methods to enable researchers to identify genes with different expression levels across conditions, either through bulk samples or single cells. When a gene is differentially expressed between two groups of bulk samples, should we attribute it to cell type proportions changes, or cell type-specific gene expression changes?H and cell type-specific gene expressions W. SCADIE features an iterative update procedure of W and H, a SCAD-based penalty for similarity control, as well as a jackknife-based standard error estimation. This framework enjoys several advantages over existing methods. First of all, compared to scRNA-seq, SCADIE’s cell type proportion estimate is not affected by technical dropouts. Second, SCADIE has shown better sensitivity and specificity compared to existing methods. In addition, SCADIE also has certain advantages in functionality compared to other methods mentioned in this paper: when compared to CIBERSORTx, SCADIE’s design can estimate and test all the genes to identify DEGs (while CIBERSORTx only imputes a subset of given genes); it can also accommodate more than microarray data and output matched cell proportions (while csSAM only works on microarray data); when compared to TOAST, SCADIE can not only perform hypothesis testing for DEGs, but produce matched point estimates in the meantime. It is worth mentioning that recently two additional methods, CellR [To this end, we have developed SCADIE, an estimation framework to simultaneously estimate both cell type proportions s, CellR  and CARss, CellR , have beThrough extensive comparisons using simulated and real datasets, we demonstrated that SCADIE can not only maintain high cell proportion estimate accuracy, it can also effectively identify cell type-specific DEGs. In the cases where initialization quality is poor or with barely sufficient sample size, SCADIE’s performance will be affected; however, its iteration procedure can still recover signals to certain extent. We have provided guidelines regarding SCADIE’s applicability under these extreme situations. Besides, its performance is highly robust with respect to a range of parameter values. These features all ensure that SCADIE can be broadly applicable to different settings.H-update step. We may expand its modularity and enable it to fully take advantage of other deconvolution methods  component, we put a high but bounded penalty on their difference in our procedure, whereas if the components are quite different, we penalize less on the difference. Specifically, we penalize ℓ2 penalty using the derivative of SCAD has not been investigated. By doing so, we can incorporate the structure pattern in W1 and W2 in a more adaptive manner  ridge regression imposing similarity between W1 and W2, and (3) SCAD-penalty. The results suggest that SCAD-penalty can keep sensitivity high while in the meantime better control false positives through its precise penalty , and plug in the output To obtain W and H, which corresponds to step (b) in Fig. In this subsection, we provide details of updating n throughout analyses in this paper, i.e., Y1,Y2∈Rm×n)++, we will discuss our handling of this issue specifically.Note that for the simplicity of notation, we intrinsically assume the sample sizes of group 1 and group 2 are both H in the main iterative procedure, we simply use NNLS to solve for the problems below for the two groups separately: For the update of Ws instead of signature genes only, and this alteration still produces good estimation accuracy . We denote n1 and n2 as column numbers of Y1,Y2, respectively, and let n0= min{n1,n2}. Then we run the iterative procedure n0 times—each time leaving one sample out from Y1,H1 and Y2,H2, respectively; this will give us n0 different ijth entry from the W1 and W2.For standard error estimation in most real data, n0Ŵ1−Ŵ2s . With thWe compared the jackknife standard error estimates with those from bootstrap, and both led to highly consistent results (Additional File W1∈R5000×5 was generated with all its entries following the log-normal distribution with mean 8 and standard deviation 3; then to generate W2, 2.5% of the entries in W1 were upregulated to 1.5× or 2×, with another 2.5% downregulated to 0.67× or 0.5×; H1 and H2 were generated using two distinct Dirichlet distributions, each group consisting of 20 samples; bulk expression matrices Y were generated by W·H+ε, where ε is a Gaussian white noise matrix with sd=4, if negative entries were present after adding noise, these entries were reset back to 0.The simulation data used in the section “H’s benchmarking, all methods support H estimation with bulk gene expression and signature matrix as input; since MuSiC only supports count data, we rounded the data before input into MuSiC.For signature matrix generation, we first obtained W1,W2,W3 matrices were generated by averaging over all cells in each major cell type of the same treatment group. H1 to H3 were generated the same way as previous section, also with 20 samples in each group. Finally, Y1 to Y3 were generated by Yi=Wi·Hi+ε,i=1,2,3, where ε is a Gaussian white noise matrix with sd=4.The pseudo-bulk data used in the section “buildSignatureMatrixUsingSeurat function from the DWLS package [Signature matrix was derived using the  package  using alDEAnalysis function in the DWLS package [Ground truth DEGs were derived by performing differential expression analysis for each cell type between group 2 and group 1 and group 3 and group 1, using the  package .In the sections “affy package in R and normalized using the rma method. We used rma normalization to keep the data comparable to [H’s benchmarking, since MuSiC only supports count level data, we rounded the bulk matrices and signature gene matrices before input into MuSiC.In thesection “rable to . The sigrable to . In H’s W. “Groundtruth” DEGs were identified for each cell type between control and COPD using the DWLS package[Raw scRNA-seq data were obtained from . Data prS package.Signature matrices were generated using the CIBERSORTx  and DWLSBulk RNA-Seq data in FPKM was obtained through GEO with accession code GSE57148, the FPKM matrix was further transformed into log scale to accommodate the scale of signature matrix.Hs as SCADIE; the control and COPD groups were denoted as groups 0 and 1, respectively. We used the output effect size as indication of DEGs’ direction, and p-value to identify significant DEGs.For benchmarking with TOAST, we input the bulk gene expression and the same initial z-scores. For single cell DEGs, the z-scores were obtained from normal distribution quantiles of their p-values, while for SCADIE output, since we know the point estimates and standard error estimates for all genes, we directly calculated their z-scores. The z-score lists for both groups were sorted and input for pre-ranked GSEA analysis using R’s fgsea package, and Molecular Signatures Database v7.4 database [p values, then the top 5% or 10% were chosen and compared accordingly. The overlapping p-values were calculated using a binomial distribution, where the model parameter N equals the total number of shared pathways, while p=0.01 for top 10% overlapping and p=0.0025 for top 5% overlapping under the null hypothesis.For GSEA analysis, we first transformed the single cell and SCADIE-derived DEGs into database . For theaffy package in R and normalized using mas5 method. Sample treatment and subtype information was retrieved using the GEOquery package. LM22 matrix from [Raw bulk microarray data were obtained through GEO with accession number GSE10846. Raw data were preprocessed with rix from  was usedBulk post-mortem RNA-Seq samples of prefrontal cortex were downloaded from the ROSMAP cohort . We onlyH into SCADIE. For the DEGs comparison, unlike SCADIE’s whole transcriptome approach, we ran CIBERSORTx’s high resolution mode only inputting those known DEGs. For TOAST run, we input bulk gene expression and initial the same initial Hs as SCADIE and CIBERSORTx; WT group was denoted as group 0, and MT group was denoted as group 1. The output DEGs’ directions were determined by their effect sizes; p-values were directly from output. Noted that in our analysis for COPD, we kept its original effect sizes as fold change measure, instead of trying to transform to log2 fold change.The raw bulk microarray data were accessed through GEO with accession number GSE127462; preprocessing and normalizing were performed the same as the above procedure for DLBLC. The initial deconvolution used LM22 matrix from  and we mAdditional file 1 Includes notations, additional figures, theoretical properties of the proposed method, simulation models, details of the simulation and real data processing.Additional file 2 Supplementary Table S1 contains details on the DEGs exclusively identified by SCADIE as discussed in Section ‘SCADIE accurately infers cell proportions and cell type-specific genes in Chronic Obstructive Pulmonary Disease’.Additional file 3 Review history.


Using this tool, we present a multi-dataset analysis of PDAC transcriptomics that confirms the basal-like/classical model over alternatives.Pancreatic ductal adenocarcinoma (PDAC) is an aggressive disease for which potent therapies have limited efficacy. Several studies have described the transcriptomic landscape of PDAC tumors to provide insight into potentially actionable gene expression signatures to improve patient outcomes. Despite centralization efforts from multiple organizations and increased transparency requirements from funding agencies and publishers, analysis of public PDAC data remains difficult. Bioinformatic pitfalls litter public transcriptomic data, such as subtle inclusion of low-purity and non-adenocarcinoma cases. These pitfalls can introduce non-specificity to gene signatures without appropriate data curation, which can negatively impact findings. To reduce barriers to analysis, we have created pdacR  gene expression datasets. pdacR is used to interrogate the specificities of popular PDAC molecular subtypes. Over the past decade, several genomics studies of pancreatic ductal adenocarcinoma (PDAC) have contributed a wealth of molecular-level data to guide investigation of the disease. A subset of these studies have sought to define transcriptomic subtypes. Although the datasets generated from these studies are accessible as part of the public domain, bioinformatic hurdles ranging from decentralized data storage to insufficiently annotated samples complicates multi-dataset investigation, resulting in inappropriate and inconsistent application of datasets. Thus, there is an unmet need for a PDAC transcriptomic compendium that provides an easily accessible interface for analysis by the PDAC research community, while retaining the capacity for diverse analytical methods in dataset- and application-specific situations.7. Collisson et al. first defined three prognostically-relevant PDAC-specific subtypes , and exocrine-like) using non-negative matrix factorization consensus clustering (NMF-CC) on a merged microarray dataset comprised of laser capture microdissected PDAC tumors (n = 66)2. We later decoupled NMF and consensus clustering to create a two-subtype model , using microarrays from bulk primary and metastatic PDAC tumors, normal pancreas and distant site normal tissue, and PDAC cell lines (n = 357)3. Similar to Collisson et al., Bailey et al. subsequently used NMF-CC to analyze RNA sequencing (RNAseq) data from a combination of PDAC and other uncommon pancreatic cancer histological types, including acinar cell carcinomas, finding four tumor subtypes , and immunogenic, n = 96)4. Then, in 2018, Puleo et al. proposed a five-subtype model using a consensus clustering approach on 309 formalin fixed paraffin embedded whole PDAC tumor samples . However, they confirmed two-subtypes  when using only samples with purity  in the upper quartile of their dataset (n = 78)5.Multiple PDAC gene expression subtype models have been proposed toward the goal of using individualized therapeutic approaches to treat patients with this disease6. Separate work by Nicolle et al. then utilized Independent Component Analysis (ICA) to generate a molecular gradient that correlates with histological differentiation and seeks to assign a continuous value to tumor samples using weighted valuations of between 4000 and 20,434 genes7.In 2020, researchers have sought to further refine these definitions. Chan-Seng-Yue et al. utilized NMF to identify four gene signatures, subdividing basal-like and classical subtypes into “basal-like A”, “basal-like B”, “classical A”, and “classical B”8. Furthermore, Tiriac et al. showed that the basal-like/classical system could be applied to transcriptomic data from patient-derived PDAC organoids to infer patient tumors which would respond to either oxaliplatin or gemcitabine9, and, shortly afterward, Aguirre et al. demonstrated utility using the Moffitt subtypes to characterize metastatic PDAC biopsies for treatment guidance in a future clinical care setting10. In 2019, Camolotto et al. showed that knockout of the transcription factor Hnf4a in mice resulted in a loss of classical expression, a decrease in tissue differentiation, and worse prognosis11. Even more recent work by Hayashi et al. was able to show concordance between the Moffitt subtypes, morphological presentation, and clinical course12. In addition, O’Kane et al. used GATA6 as a surrogate for the Moffitt basal-like/classical identification and recapitulated robust response to tumor and overall survival in the COMPASS cohort13. To enhance the clinical applicability of the basal-like/classical model, we previously generated a single-sample classifier termed purity independent subtyping of tumors (PurIST)14. This classifier showed a robust ability to predict a patient’s molecular subtype and response to therapy across multiple technologies and without the need for a cohort8.Independent studies have comparatively evaluated the clinical utility of these proposed schemas. The Comprehensive Molecular Characterization of Advanced Pancreatic Ductal Adenocarcinoma for Better Treatment Selection (COMPASS) trial’s earliest results showed that the Moffitt et al. classification system provided accurate prediction of the tumors which would respond or not respond to modified FOLFIRINOX or gemcitabine/nab-paclitaxel30. The mRNA dataset includes 182 samples, but only 150 have been confirmed histologically as PDAC. The remaining 32 samples are classified as either other types of pancreatic cancer, no evidence of cancer, or adjacent normal samples. Including such samples in derivation or validation of expression signatures results in misleading results. Finally, a growing consensus surrounding the two-subtype model has led researchers to collapse other 3+ subtype models into two in an attempt to harmonize with the basal-like/classical model33. However, this approach does not address confounding factors related to purity and tissue specificity inherent in the approaches used to derive the models. Given that the two-subtype and collapsed 3+ subtype signatures are not completely redundant, attempting to interchangeably label a subset of the data as basal-like, squamous, or QM is not equivalent.While a limited number of studies have begun comparisons of subtype performance in clinical care, diversity of opinions in the field and difficulty in obtaining data has led to confusion. Indeed, even recent studies show mixed strategies among investigators choosing which datasets or gene signatures to use. In particular, we highlight three common issues. First, validation is frequently only performed on one publicly available dataset. Second, there are multiple examples of inappropriate use of The Cancer Genome Atlas (TCGA) Program PDAC dataset36. This GUI is available as web-hosted software for ease of access and use. The flexibility of pdacR, combined with its inclusion of a wide range of PDAC datasets that are focused on the most reliable, large, and commonly referenced studies, expands upon on previous work by Marzec et al. and Tan et al. to develop the Pancreatic Expression Database (PED) and Human Pancreatic Cancer Database (HPCDb), respectively, for pancreatic cancer data centralization and user interface development for genomics analysis40.To facilitate standardized investigation of PDAC data, we have created an R package, termed pdacR, for public use. As part of this package, we have gathered and carefully annotated thirteen human PDAC datasets from the past decade for a thorough comparison of the performance of subtyping schemas in identifying only the tumor cells within PDAC tumors Table . In keepshiny-based graphical user interface (UI) for point-and-click analysis of public and user-generated data  or categorical  data, and enables the user to define continuous data as categorical by toggling a quantile cutoff. The differential expression component allows users to generate volcano plots with full user freedom to define head-to-head comparisons and select which genes or gene sets to highlight.The UI is composed of four primary functional components that enable a range of investigative methods: heatmapping, dimension reduction, survival analysis, and differential expression. Users begin by choosing their dataset of choice from the list of those already curated or, on a local instance, may choose to load a user-generated dataset. User-generated datasets may be imported as R-formatted data which has been converted to a Other graphical user interfaces have been developed to facilitate transcriptomics investigation in pancreatic cancer, though there are clear advantages offered in pdacR  as does the Collisson exocrine gene set  from the PACA-AU RNAseq and PACA-AU microarray datasets, we determined that expression of the Bailey ADEX, Bailey Immunogenic, and Collisson exocrine gene signatures are lost in cell lines derived from primary PDAC tumors with a range of expression of these same signatures Fig. . Contrasp < 0.0001), suggesting that the non-tumor component driving ADEX, Exocrine, and Immunogenic gene expression is unlikely dominated by CAFs , suggesting a pancreas cell intrinsic ADEX/Exocrine gene expression origin , suggesting that a non-tumor cell component drives expression of this gene set across several tissue types , and normal and metastatic PDAC tissue from liver, lung, peritoneum/stomach/intestine, and spleen/lymph node. We found that the ADEX, Exocrine, and Immunogenic gene sets are highly expressed in primary PDACs compared to either PDAC cell lines or subcutaneously implanted patient-derived xenografts (PDXs). Notably, primary PDACs also demonstrate increased expression of these gene sets compared to CAF cell lines  and the associated code and data (github.com/rmoffitt/pdacR) will be used by other researchers to facilitate robust external validation of future analyses.We have presented data to suggest that the basal-like/classical subtyping schema is both the most clinically reliable model and the most appropriate in describing only the tumor cells within a non-purified PDAC tumor. This data and others are now compiled in a public R package, pdacR, for open use by the research community. While this work has helped further our understanding of PDAC subtypes, future studies on these subtypes in the context of patient outcomes and response to therapy are necessary to apply our understanding to clinically meaningful intervention strategies. We hope this tool (https://github.com/rmoffitt/scOh). This atlas does not include a web interface, but covers many substantial hurdles in scRNA analysis, including parsing, integration, and normalization. Another limitation inherent to this sort of data collection is the inability to robustly merge datasets. Preliminary versions of this tool did have the option for data aggregation. However, in addition to being too computationally intensive for a point-and-click interface, we found that aggregation introduced more problems than it solved. If we cannot assume that datasets come from comparable distributions and cell type compositions , then we necessarily confound our analysis. In addition, our datasets come from a variety of platforms, including bulk RNA sequencing and microarrays, adding another layer of complexity to aggregation and batch correction. Believing that providing the option of poorly-merged data would be more harmful than helpful, we ultimately decided was beyond the scope of this project. It is our belief that the rapidity with which a researcher could perform serialized analysis on different datasets overcomes much of the limitation of not being able to merge the data.Recent attempts to improve our understanding of PDAC have emphasized the utility of single-cell RNA sequencing (scRNA). Given that the datasets provided here are all bulk analyses, we are limited in our ability to fully separate cellular populations. We partially address this shortcoming using datasets derived from a multitude of pure cell type sources or those enhanced by LCM. While scRNA sequencing is expanding in its utilization, it remains prohibitively expensive for inclusion in the realm of clinical characterization and application. While incorporation of new scRNA datasets would improve this tool from a purely investigative standpoint, the computational burden and memory required are beyond the scope of a responsive GUI. We have, in parallel, made available a parsed single-cell atlas , datasets were trimmed down to include only PDAC samples and only the genes present in the relevant gene sets. We then applied the ConsensusClusterPlus function with kmeans clustering and euclidean distance to call the appropriate number of cluster (k = number of gene sets).45.To account for the potential disproportionate impact of outliers on correlations, correlation values were calculated using Spearman Rho. Difference in expression of gene sets by cell types was calculated using a non-parametric Wilcoxon Rank-Sum test. Reproducibility is one of our main motivators for this publication. To that end, we have provided a web application wherein all the analyses can be replicated with a series of clicks. We have also provided all our analysis code, along with parsed and ready-to-analyze datasets, in our githubshiny R package. Datasets were chosen for inclusion in the pdacR package based on public availability or investigator permission, as well as importance in the field as determined by the authors. Differential expression analysis on RNAseq or Array experiments is performed using the DESeq246 or limma47 packages respectively. The logFC is calculated by dividing the mean of cohort A by the mean of cohort B for each gene. Survival analysis is performed using the survival and survminer packages in R. Censor and survival data are obtained from the initial publications, and are a combination of overall and disease-specific survivals. User-generated data may be converted from a SummarizedExperiment container style to a UI-compatible format using our ‘./R/Convert_GUI_data.R’ wrapper function.The pdacR UI was designed using the Further information on research design is available in the Supplementary InformationReporting Summary