It is now freely accessible via https://server.wei-group.net/scIMC/.With the advent of single-cell RNA sequencing (scRNA-seq), one major challenging is the so-called ‘dropout’ events that distort gene expression and remarkably influence downstream analysis in single-cell transcriptome. To address this issue, much effort has been done and several scRNA-seq imputation methods were developed with two categories: model-based and deep learning-based. However, comprehensively and systematically comparing existing methods are still lacking. In this work, we use six simulated and two real scRNA-seq datasets to comprehensively evaluate and compare a total of 12 available imputation methods from the following four aspects: (i) gene expression recovering, (ii) cell clustering, (iii) gene differential expression, and (iv) cellular trajectory reconstruction. We demonstrate that deep learning-based approaches generally exhibit better overall performance than model-based approaches under major benchmarking comparison, indicating the power of deep learning for imputation. Importantly, we built  With the growing complexity of interaction network, using a priori network appropriately has become a key element of the imputation methods. scNPF shares information between similar cells, and uses the prior knowledge of the interaction network to determine gene expressions for a given cell  model-based approaches and (ii) deep learning-based approaches. Model-based imputation methods that borrow prior knowledge and information across cells to predict the missing values, including cell-cell interaction network, gene-gene interaction network, and the integration of both of them for imputation. These methods restore the expression of the target gene in terms of similar cell information, similar gene information, and mathematical computation using prior knowledge. For example, SAVER  is a method that optimizes the whole gene expression counts, using information across genes and cells to impute zeros as well as to improve the expression values for all genes . Similarest data . Gong et  for biological network inference, etc , a method that utilizes stochastic optimization and deep neural networks to aggregate the information between similar cells and genes, and estimates the basic distribution of the count matrix . The Deep Count Autoencoder network (DCA) captures the non-linear gene-gene correlation by introducing the negative binomial noise model with zero-inflation, while considering the count distribution, over-dispersion and sparsity of the data to impute the missing values  for scRNA-seq Imputation), based on generative adversarial networks , also using graph neural network to learn cell-cell relationships and combine three autoencoders to impute ‘dropout’ events and cell clustering , the first computational platform that allows researchers of interest to do data imputation and downstream comparative analysis of the state-of-the-art imputation methods on their customized datasets, and provides visualization result analysis to find out which method is most appropriate for their datasets in specific downstream tasks. We expect this platform can be convenient and useful especially for the researchers without any computer science or programing skill background in this field.Although a large number of imputation approaches have been proposed and most of them achieved good performance in different scenarios, comprehensively comparing the performance of state-of-the-art imputation methods are still lacking ,34–37. IIn this work, we constructed an unbiased framework to quantitatively evaluate and compare the ability of available state-of-the-art imputation methods for scRNA-seq data. Based on this framework, we surveyed the performance of the imputation methods in terms of multiple widely used metrics on six simulated and two real datasets. The general overview of our benchmarking framework is illustrated in Figure (1) 1st step—Data Preprocessing. We performed the benchmarking on six simulated and two real datasets (refer to ‘Benchmark datasets for details’). For each dataset, a raw gene expression matrix (before imputation) is generally preprocessed in two sub-steps  2nd step—Missing Value Imputation. Table (3) 3rd step—Downstream Comparison Analysis. To quantitively compare how good the imputed matrixes generated by the compared methods are, we evaluated the performance in recovering actual gene expression. As seen in Figure In this study, we used six simulated datasets and two real datasets to evaluate the performance of different imputation methods. The six simulated datasets were generated with different zero expression ratios. Two real datasets are Human Embryonic Stem Cells (ESCs) dataset  and Timegroup.prob = c, mean.shape = 0.3, mean.rate = 0.02, de.prob = 0.5, de.facLoc = 0.5, dropout.shape = 0.5, dropout.type = “experiment”, and dropout.mid parameter ranging from 1 to 6 in step of 1, in order to simulate six datasets with zero expression rate of 0.78, 0.71, 0.63, 0.55, 0.48 and 0.42, respectively. It is worth noting that scNPF failed to run on the simulated datasets, we only compared the other 11 methods.Splatter  is a R BP-value  is <0.05, we consider it to be differentially expressed, so-called DEG. We performed edgeR (We utilized a dataset with both bulk and scRNA-seq on human ESCs and differentiated definitive endoderm cells (DECs), including six samples of bulk RNA-seq (four for H1 ESC and two for DEC) and 350 cells of scRNA-seq (212 cells for H1 ESC and 138 cells for DEC) . The pered edgeR  on both We employed the time-course scRNA-seq data derived from the differentiation from H1 ESC to DEC . This daFor the scRNA-seq data imputation, the standard data preprocessing procedure contains three steps: filtering, normalization, and log-transformation. For all datasets used in this study, we firstly filtered out those genes that are expressed in less than two cells, which are called low-expressed genes. Next, we normalized the dataset by a normalization method ‘scanpy. pp. normalize_total’ from Scanpy (1.4.4) with being divided by library size and multiplied by the median of library size across cells. Finally, the normalized matrix is log-transformed. It's worth to point out that five out of the 12 compared methods, including scImpute, DrImpute, scTSSR, AutoImpute, and scGNN, have the data preprocessing module in their source codes. Besides, DCA and DeepImpute only accept the dataset without normalization and log-transformation. Therefore, we input the raw gene expression matrix to DrImpute, AutoImpute, and scGNN and did only the filtering step to the raw gene expression matrix for the four methods . To keep data consistence, we followed the above data preprocessing steps to process the data before feeding to the rest methods  for imputation. The shape of imputed matrix from different imputation methods are shown in Table X (true gene expression matrix in RMSE and PCC), and In this section, the gene expression matrix is denoted as It is to measure the difference between the imputed matrix and the raw matrix, which calculates the deviation between the observed values and true values. RMSE is defined as:X, It is to examine the degree of correlation between the imputed matrix and the raw matrix, which is defined as:It refers to the degree of correlation between two random variables. We denote a, the probability that the sample belongs to the cluster b, and the probability that the sample belongs to both a and b, respectively.m cells which are cluster into k clusters. i and j enumerate the k clusters, and It measures the degree of agreement between the two data distributions . We assui-th sample and all other samples in the same cluster, ith sample and all samples in a given cluster (clusters that does not contain the ith sample).It is used to evaluate the cell clustering performance of imputation methods. It combines cohesion and separation, which can evaluate the clustering results on the same data. The closer the Si score is to 1, the more accurate the clustering is; the closer it is to –1, the worse the result is. The Si score is defined as:A and B are two sets. Jaccard is the ratio of the size of the intersection of A and B to the size of the union of A and B.Moreover, we utilize Jaccard similarity coefficient (Jaccard)  to evalum samples belonging to K clusters, respectively. Purity is defined as:i; i, but belongs to cluster j, which is calculated as:i, but belongs to cluster j.It is a commonly used evaluation metric for clustering. We assume that there are n is the number of samples, ith and jth cells in the ordered path matches their expected order based on the external information  in ALRA, we set “k = choose_k”, k represents the rank of low-rank approximation, and choose_k is the method of selecting rank k of low-rank approximation based on continuous singular value spacing statistics designed in advance in ALRA;  model  in MAGIC, we set the parameter “in ALRA;  Set the  method.  Set “LTMG) model  is used A good imputation method should recover the true gene expression of scRNA-seq data. Due to the lack of ground truth of expression values in the real datasets, we generated six simulated datasets with different parameters using Splatter. It was worth noting that scNPF failed to run on the simulated datasets, we only compared the other 11 methods. We firstly visualized true counts data (data without dropouts), raw data (data with dropouts but not imputed) and imputed data from 11 imputation methods, using UMAP , a commoy = x curve.To quantitively compare the ability of different imputation methods for recovering true gene expression, we further used two metrics, RMSE and PCC, to evaluate the performance of the methods. In order to explore the performance of different methods on datasets with different zero expression rates, we simulated six datasets with different zero expression rates. We performed the imputation methods on the six simulated datasets and calculated the RMSEs and PCCs between true count matrix (matrix without dropouts) and the imputed matrix, respectively. The results are illustrated in Figures Via the visualization and quantitively comparison results, it can be concluded that DCA and DeepImpute significantly outperform other methods for maintaining true expression distribution. Notably, both of them are deep learning-based methods, demonstrating the power of deep learning algorithms for recovering true gene expression.Recently, many clustering methods have been developed to deal with cell clustering in single-cell sequencing data ,51. It pet al.  dataset. It is worth to note that DCA failed to be performed on this dataset, as the count matrix was regarded to have been normalized. A tool called edgeR  is commoP-value in bulk samples as reference for further research. Heat maps of these 10 genes in different imputed data were ploted . Due to the gene selection strategy of AutoImpute and scGNN, only one gene overlapped with the standard gene respectively, so we did not ploted the corresponding heat maps. We found that four genes  had significant differential expression in two cell types (H1 and DEC) in bulk data. SAVER, MAGIC and ALRA could modify the values of the four genes with obvious differential expression, and other methods could only identify half of them, while all genes of scIGANs showed high expression values in different cell types, which was significantly different from the results of other methods. However, we cannot confirm an imputation method's ability to detect differentially expressed genes by the number of differentially expressed genes it inferred. It is necessary to quantify how accurately the differentially expressed genes it detects compare with the gold standard. To facilitate discussion, we calculated four metrics  to compare the accuracy of predicting the differential expression genes. The results were shown in Figure Besides, we extracted the top 10 genes with highest Reconstruction of cellular trajectories is critical to explore the pattern of cell cycle dynamics by time course in scRNA-seq data. Cellular trajectories analysis includes three main steps: dimension reducing, clustering, and trajectory reconstruction. Despite the wide use of cellular trajectories reconstruction in scRNA-seq data , it is sIn this study, we comprehensively evaluated and compared a total of 12 state-of-the-art imputation methods for scRNA-seq analysis under different scenarios. We observed that every imputation method has its own advantages and disadvantages, with no outstanding method. In general, recent deep learning-based approaches exhibit better performance than model-based in most aspects. However, it's worth to note that some model-based methods achieved satisfied performance in some experiments (i.e. reconstruction of cellular trajectory), which might be due to their good ability of maintaining the relationships between cells and genes.We firstly highlighted the advantage of imputation methods for recovering gene expression. We visualized the shape of true counts data, raw data and imputed data obtained by 12 imputation methods on six simulated datasets, and calculated RMSEs and PCCs to evaluate the performance. The results showed that DCA and DeepImpute outperformed other methods. DeepImpute used highly correlated genes of the target genes to impute the missing values, and DCA can capture the nonlinear gene-gene correlation. Moreover, the two methods tend to recover the missing values by deep learning algorithms, demonstrating that using deep learning algorithms reasonably can recover gene expression more effectively. In addition, we found scIGANs performed better when zero expression rate became lower, accounting for the application of GAN in scRNA-seq analysis. Generative adversarial network consists of a generative model and a discriminative model. Its advantage is that the parameter update of the generative model comes from the back propagation of the discriminative model. This mechanism may have improved the ability of GANs to capture some intrinsic information of scRNA-seq data.Then, we focused on evaluating the performance of imputation methods for improving the downstream analysis, including cell clustering, gene differential expression analysis, and reconstructing cellular trajectories.k-means and UMAP and k-means on six simulated datasets in terms of NMI, ARI, Si scores, and Purity. With the recovery of the missing values in terms of autoencoder, DCA focused on the distribution of scRNA-seq data. ALRA maintained the biological zeros in scRNA-seq data successfully, which was beneficial to preserve the original data distribution. Besides, while zero expression rate decreasing, DrImpute, DeepImpute, and scIGANs also showed a performance improvement. Based on the clustering results, DrImpute optimized the distribution pattern of cells from the same cluster, leading to the improvement of cell clustering. Highly correlated genes of the target genes were indispensable to impute the missing values in DeepImpute. DCA, DeepImpute, scIGANs, and ALRA belong to deep learning-based methods, demonstrating that deep learning-based methods are beneficial to the performance of cell clustering. In conclusion, similarity information in genes and cells plays an important role in cell clustering. Moreover, we also investigated the impact of different dimension reduction tools and found that the tools have similar results for cell clustering analysis.For cell clustering, we evaluated the results obtained by t-SNE and For gene differential expression analysis, we ran edgeR on scRNA-seq data and the matched bulk RNA-seq data, and then visualized the results by a volcano figure. In addition, the number of differential expression genes detected by the compared methods is listed in Table To investigate the reconstruction of the cellular trajectories, we performed Monocle3 and TSCAN on scRNA-seq data. The results suggest that scImpute outperformed all other imputation methods, and proved scImpute showed better performance in imputing data with collinearity.It is worth noting that, as for AutoImpute, in different aspects, its performance showed consistently worse than other methods. That is probably due to the gene selection in AutoImpute for preprocessing scRNA-seq data, AutoImpute only keeps the top 1000 high-dispersion genes for each expression data. The gene selection may significantly affect the existing gene expression structure, causing unreasonable results for downstream analysis.In addition, we analyzed three possible reasons why deep learning-based imputation methods are better than model-based imputation methods. Firstly, the size of scRNA-seq data increased from hundreds to millions because of the widespread use of data, which caused the problem of high dimensionality. Based on the features of large sizes and high dimensionality, deep learning algorithms are more adaptable. Next, rather than recovering missing expression values, it is more important to recover data characteristics that are more meaningful for further analysis in scRNA-seq data. Therefore, deep learning-based methods are more advantageous for learning the features of data than model-based methods. Finally, deep learning algorithms are good at constructing gene-gene and cell-cell relationship networks in scRNA-seq data. With these prior information, the overall structure of data can be recovered through continuous optimization.scIMC for comparison and visualization analysis, which would allow researchers of interest to perform the comparison analysis of all the available imputation methods on their specific scRNA-seq datasets. In particular, our platform can provide visualization comparison results for downstream analysis, and give users useful guidance to see which imputation method is more appropriate on their specific datasets.Most importantly, we built the first platform namely scIMC to conduct the experiments.Finally, it should be pointed out that the comparative results and corresponding conclusions in this study are all based on the datasets mentioned above, which might be not fully applicable in all situations. Those who wish to further explore the performance of imputation methods in different datasets, can use our scIMC (single-cell Imputation Methods Comparison platform) so as to help readers perform different imputation methods and downstream analysis .Imputation module of scIMC. The matrix is preprocessed as described in Data Preprocessing and then is used to be imputed by different methods . We perform the imputation methods with default parameters  to impute gene expression matrix, in order to compare the performance of them impartially. Users can select imputation methods required and perform them on the input matrix. After submitting all the necessary datasets, scIMC will provide a Job ID to query the imputation results. When the imputation completes, scIMC will send an email to the user's mailbox (which need to be provided together with datasets), notifying to preview and download the generated results. The results of imputed matrix for different methods can be downloaded from Job List page by clicking the button ‘Details’ corresponding to Jod ID.Imputation is a common approach to recover gene expression affected by ‘dropout’ events. Users can employ a total of 12 state-of-the-art imputation methods in Imputation module, scIMC will provide a Job ID to users. When the experiments complete, scIMC will send an email to the user's mailbox (which need to be provided together with datasets), notifying to preview and download the generated results. The results will show on Detail page, and can be downloaded directly on this page. To display Detail page, click the button ‘Details’ corresponding to Jod ID.In this module, there are four kinds of experiment analysis, including Recover gene expression, Cell clustering, Gene differential expression and Reconstruct cellular trajectories. To start experiments, csv-formatted files should first be uploaded as prompted: true counts matrix, raw matrix, imputed matrices  and the cell labels. We performed four experiments and showed their results, respectively. To summarize the performance of recovering gene expression, RMSE, PCC and scatter plots by PCA, t-SNE and UMAP are used. When it comes to cell clustering, its performance is shown by scatter plots and four metrics: NMI, ARI, Si scores and Purity. We use edgeR to detect DEGs from different gene expression matrix and evaluate the ability of them across volcano figures and numbers of DEGs detected. Cellular trajectories reconstructed are placed to reflect the performance of reconstructing the cellular trajectories for different imputation methods. The same as scIMC website (https://server.wei-group.net/scIMC/) for more details. Users can find User Guide on scIMC, and we prepared a user guide video on how to use scIMC. It is expected to be a useful platform for researchers in this field.Please refer to the https://server.wei-group.net/scIMC/.The authors declare that the data supporting the findings of this study are available within the article and its supplementary information files. Besides, the benchmarking datasets were also available for downloading at gkac317_Supplemental_FileClick here for additional data file.