As time progresses and technology improves, biological data sets are continuously increasing in size. New methods and new implementations of existing methods are needed to keep pace with this increase. In this paper, we present a high-performance computing (HPC)-capable implementation of Iterative Random Forest (iRF). This new implementation enables the explainable-AI eQTL analysis of SNP sets with over a million SNPs. Using this implementation, we also present a new method, iRF Leave One Out Prediction (iRF-LOOP), for the creation of Predictive Expression Networks on the order of 40,000 genes or more. We compare the new implementation of iRF with the previous R version and analyze its time to completion on two of the world’s fastest supercomputers, Summit and Titan. We also show iRF-LOOP’s ability to capture biologically significant results when creating Predictive Expression Networks. This new implementation of iRF will enable the analysis of biological data sets at scales that were previously not possible. Due to innovation in the areas of genome sequencing and ’omics analysis, biological data is entering the age of big data. As opposed to other fields of research, biological data sets tend to have large feature quantity, but much smaller sample counts, such as in a GWAS population where there are typically hundreds to thousands of genotypes, but millions of SNPs. The number of independent features of biological systems is large, and would require many lifetimes of the entire scientific community to sufficiently study . The abiMany methods exist to determine feature importance and feature selection, such as Pearson Correlation, Mutual Information (MI), Sequential Feature Selection (SFS) , Lasso, Arabidopsis thaliana SNPs in relation to a gene expression profile, as part of a genome-wide explainable-AI-based eQTL analysis.Iterative Random Forest  (iRF) isThe base learner for the Random Forest (RF) and Iterative Random Forest (iRF) methods is the decision tree, also known as a binary tree. A decision tree starts with a set of data: samples, features, and a dependent variable. The goal is to divide the samples, through decisions based on the features, into subsets containing homogeneous dependent variable values, generally following the CART (Classification and Regression Trees) method , where eA Random Forest is an ensemble of decision trees. However, the trees in a Random Forest differ from standard decision trees in that each tree starts with a subset of the samples, chosen via random sampling with replacement. Also differing from standard decision trees, the features being considered at each node are a random subset, with the number of features provided by a parameter. Once a forest has been generated, the importance of each feature can be calculated from node impurity, such as the Gini index for classification or variance explained for regression, or permutation importance. Unlike a single decision tree, a Random Forest is a strong learner, because it averages many weak learners and avoids putting too much weight on outlier decisions. The number of trees in a forest is a parameter that is chosen by the user and influences the accuracy of the model.i times, where i is set by the user. Due to the ability to easily follow the decisions that these models make, they have been deemed explainable-AI (X-AI) [Iterative Random Forest expands on the Random Forest method by adding an iterative boosting process, producing a similar effect to Lasso in a linear model framework. First, a Random Forest is created where features are unweighted and have an equal chance of being randomly sampled at any given node. The resulting importance scores for the features are then used to weight the features in the next forest, thus increasing the chance that important features are evaluated at any given node. This process of weighting and creating a new Random Forest is repeated I (X-AI) , which dFor both Random Forest and Iterative Random Forest, the total number of features, samples, trees, and iterations (for iRF) all influence run time to differing degrees. Most of the computation time is spent creating the decision trees, so run time scales with the number of trees. The number of samples influences the number of decisions within a tree needed to divide the data into homogeneous subsets. This influences the number of nodes created in a tree, and thus the run time per tree. Furthermore, the number of features influences the amount of time required to find the feature that best divides the samples at any given node. Finally, for iRF, a whole new forest must be generated for each iteration, though the run time for subsequent forests tends to diminish as many feature weights are set to zero. The number of iterations allows the user to find a balance between over- and under-parameterization of the model, by progressively eliminating features.We used Ranger , an openWithin our implementation, each decision tree is initialized with a random subset of the data sample vectors and is then built in an independent process. Groups of trees (sub-forests) are built on compute nodes and then sent to a master compute node that aggregates them into a full Random Forest. This is done by giving each sub-forest a randomly generated seed number which determines the random subset of data each tree in a sub-forest uses, allowing for a higher likelihood of unique random data subsets on each tree. Once in the form of a single Random Forest, Ranger’s functions for forest analysis are used, including feature importance aggregation.On a distributed system, where parts of the forest are created on different compute nodes, the aggregation of results requires most of the inter-node communication. This process relies on MPI (the Message Passing Interface), an internode-communication standard for parallel programming, and Open-MPI , an openi iterations, the forest creation and aggregation is performed i times. After the completion of each iteration, the feature weights are written to a file. At the beginning of the next iteration, this file is read into an array that is used to create the weighted distribution from which features are sampled during the decision process at each node on each tree. This process, while potentially slow due to the file I/O, uses the preexisting functionality of weighted sampling from Ranger.To implement n features and m samples, iRF Leave One Out Prediction (iRF-LOOP) starts by treating one feature as the dependent variable (Y) and the remaining n − 1 features as predictor matrix (X) of size m x n − 1. Using an iRF model, the importance of each feature in X, for predicting Y, is calculated. The result is a vector, of size n, of importance scores . This process is repeated for each of the n features, requiring n iRF runs. The n vectors of importance scores are concatenated into an n x n importance matrix. To keep importance scores on the same scale across the importance matrix, each column is normalized relative to the sum of the column. The normalized importance matrix can be viewed as a directional adjacency matrix, where values are edge weights between features. See Given a data set of A typical use case of iRF, with a matrix of features and a single target vector of outcomes, becomes comparable to an X-AI-based eQTL analysis, when the matrix of features is a set of single nucleotide polymorphisms (SNPs) and the dependent variable vector is a gene’s expression measured across samples. This analysis determines which set(s) of SNPs are important to variation in the gene’s expression.Arabidopsis thaliana SNP data from the Weigel laboratory at the Max Planck Institute for Developmental Biology, available at https://1001genomes.org/data/GMI-MPI/releases/v3.1/. We filtered the SNPs using bcftools 1.9 [We obtained ools 1.9  to keep Arabidopsis thaliana expression data [We obtained ion data  from 727The C++ iRF code was run using these data as input with five iterations, each generating a forest containing 1000 trees. The number of trees was chosen as a value close to the square root of the number of features (a common setting for this parameter), where each feature has a 95% chance of being included in the feature subset within the first two layers in at least one tree. This helps to guarantee that all features are considered at least once across an iteration. HPC node quantities of 1, 2, 5, and 10 were used to show run time changes as the amount of resources increases.Due to the large number of SNPs, the full data could not fit in memory on a standard laptop, for use in the R iRF program. Instead, small subsets of features of sizes 2000, 1000, 500, 100 and 50 features were run, each using the full 666 samples and the same AT1G01010 gene expression dependent variable. Each feature set was run three times and averaged to account for the inherent stochasticity in the algorithm. Only one run was performed for each parameter set on Summit with the full data set due to limited compute time availability.Given a matrix of gene expression data, there are a multitude of approaches for inferring which genes potentially regulate the expression of other genes, ranging in complexity from pairwise Pearson Correlations to advanced methods such as Aracne , Genie3 Populus trichocarpa (Black cottonwood). The RNAseq data [www.ncbi.nlm.nih.gov/sra). Reads were aligned to the Populus trichocarpa v.3.0 reference [We applied iRF-LOOP to a matrix of gene expression data measured in 41,335 genes across 720 genotypes of seq data  from wereference . Transcreference . The adjt-statistics for each threshold.To determine the biological significance of the PEN produced by the iRF-LOOP, we compared each of the four thresholded networks to a network of known biological function, created from Gene Ontology (GO) annotations. GO is a standardized hierarchy of gene descriptions that captures the current knowledge of gene function. It is accepted that there is both missing information and some error, nevertheless it is useful as a broad truth set for large sets of genes. We calculated scores for each network by intersecting with the GO network. We then evaluated the probability of achieving such scores relative to random chance by creating null distributions of scores produced by random permutation of the iRF-LOOP networks and calculated n is the number of genes with the shared term. This weighting attempts to balance the scoring metric so that correctly identifying rare edges is not out weighted by simply capturing many generic GO terms. Genes that share a GO term create connected sets of genes, where each gene has an edge with each other gene in the set. A predicted network’s intersect score is a summation of all GO edge weights from edges that appear in both the GO and predicted networks. For example, if the predicted network has an edge between gene A and gene B, and the GO network also has an edge between gene A and B with a weight of X, then the intersect score would increase by X. If two genes share more than one GO term, then the largest weight is used for the edge. To avoid edges between very loosely associated genes, only GO terms with less than 1000 genes were used for this analysis. The resulting network provides the relationship between genes that share some level of known functionality.We created a Populus trichocarpa GO network for the Biological Process (BP) GO terms, using annotations from . Genes at-statistics were calculated for the score of each thresholded Predictive Expression Network.To generate the null distribution of intersect scores for each thresholded PEN, the node labels of each network were randomly permuted 1000 times, and each random network was scored against the GO network. From these null distributions, For comparison purposes, the noted GO scoring process was also done on a Pearson’s Correlation-generated network, creating co-expression scores. The input data was the same expression data, and Pearson’s Correlation was calculated for each pair of genes. The top 0.1% of correlation values was then analyzed with the GO scoring process, and compared to the top 0.1% of edge scores from the PEN.Populus trichocarpa expression data mentioned above was used as the feature set.To compare the original iRF R code to the new implementation, both were run on a single node of Summit with a variety of running parameters. These parameters included all combinations of 100, 1000, and 5000 trees and 1, 2, 3, and 4 threads for 1000 features. All combinations were run three times and the scores were averaged. Due to the R code’s doParallel  back-endThe computational resources used in this work were Summit, Titan, and a 2015 MacBook Pro laptop. Summit is an Oak Ridge Leadership Computing Facility (OLCF) supercomputer located at Oak Ridge National Laboratory (ORNL). It is an IBM system with approximately 4600 nodes, each with two IBM POWER9 processors, each with 22 cores (176 hardware threads) and six NVIDIA VOLTAV100 GPUs, and 512 GB of DDR4 memory. Titan was a former OLCF supercomputer, recently decommission, and was a Cray system that had approximately 18,688 nodes, each with a 16-core AMD Opteron 6274 processor and 32 GB of DDR3 ECC memory. The 2015 MacBook Pro has a 3.1 GHz Intel Core i7 Processor and 16 GB of DDR3 memory.Previously, the only published iRF code existed as an R library. This library uses R’s ’doParallel’ functionality, generally allowing for multi-core thread parallelism on shared memory CPUs. However, this system did not function on Summit, so our analysis was limited to running differences on a single Summit CPU.To compare the R code to the C++ code, both programs were run on a single CPU on Summit. While this is a small resource set, it was sufficient in showing trends and making comparisons between the two implementations. Arabidopsis thaliana was used to predict the expression of gene AT1G01010. To show how well our implementation of iRF handles large feature sets, a set of 1.7 million SNPs from To try to determine approximately how long this calculation would take on a standard laptop, multiple smaller runs were completed. Populus trichocarpa. We used iRF-LOOP to produce Predictive Expression Networks for t-statistic for each network was calculated from these values, giving the values shown. All PEN t-statistic values had a p-value of effectively zero, as the iRF intersect scores were all significantly larger than the null distributions. The t-statistic for the co-expression comparison network also had a small p-value, but it is worth noting that even with more edges than the comparable PEN network, it had a lower GO score. This result confirms that the PENs created using iRF-LOOP are finding more biologically annotated GO edges than would be found by chance., with respect to known GO annotations. Quantile-quantile plots for each of the PEN null distributions are provided in Also shown in Arabidopsis thaliana, assuming each run took approximately the same time as the shown above, would take approximately 705 days using 5 nodes, or 84,680 compute node hours, or 18 h using 4600 Summit nodes. To complete the analysis for all 24,175 gene expression predictions on a laptop using the R code would take approximately 2191 years. While this comparison seems impressive, it should also be noted that for a larger feature set and larger number of trees the large resources will be even more appropriate as the scaling factor will be even less effected by overhead. For cases where the number of SNPs is 10 million and higher, the code should be even more efficient on HPC systems. However, as not everyone has access to the fastest computers in the world this code could still run efficiently on a smaller system using a smaller set of SNPs.We have presented a high-performance computing-capable implementation of Iterative Random Forest. This implementation uses Ranger, C++, and MPI to use the resources available on multi-node computation resources. We have shown that our implementation can perform X-AI-based eQTL-type analyses with millions of SNPs and have shown its ability to scale with multiple parameters. Using iRF to complete a whole analysis of the 24,175 gene expression profiles for Using this new implementation, we developed iRF-LOOP and used it to produce Predictive Expression Networks which were shown to have biologically relevant information. The process of iRF-LOOP has the potential to be used for a wide variety of data analysis problems. With an appropriate amount of compute resources, it would be possible to build connected networks for each level of ’omics data available for a given species. The same concept could also be used to connect ’omics layers to each other as shown by using the SNPs in the X-AI-based eQTL analysis. This machine learning method is not limited to genetics or biology and has uses in other fields where systems can be represented as a matrix.Downstream of any iRF analysis, there is the possibility of finding epistatic interactions among features from the resulting forests, using Random Intersection Trees (RIT) . This mehttps://github.com/Jromero1208/RangerBasediRF.The Ranger-based Iterative Random Forest code is available at


SVMs). Furthermore, we investigate combined classifiers that integrate DBNs with SVMs. Such a classifier utilizes a DBN as representation learner forming the input for a SVM. Overall, our results provide guidelines for the complex usage of DBN for classifying gene expression data from complex diseases.Genomics data provide great opportunities for translational research and the clinical practice, for example, for predicting disease stages. However, the classification of such data is a challenging task due to their high dimensionality, noise, and heterogeneity. In recent years, deep learning classifiers generated much interest, but due to their complexity, so far, little is known about the utility of this method for genomics. In this paper, we address this problem by studying a computational diagnostics task by classification of breast cancer and inflammatory bowel disease patients based on high‐dimensional gene expression data. We provide a comprehensive analysis of the classification performance of deep belief networks (DBNs) in dependence on its multiple model parameters and in comparison with support vector machines ( ANNartificial neural networkBpropbackpropagationCDcontrastive divergenceCNNconvolutional neural networksDBNdeep belief networksDLdeep learningIBDinflammatory bowel diseaseMLPsmultilayer perceptronsRBMrestricted Boltzmann machineRNNsrecurrent neural networksRpropresilient backpropagationSE1DCNNsample expansion‐based 1DCNNSGDstochastic gradient descentSVMsupport vector machinesTechnological progress in the generation of genome‐scale high‐throughput data has led to a flood of data on the DNA, RNA, and protein levels Deep learning is a new methodology currently receiving much attention In this paper, we will study deep belief network (DBN), a particular form of DL methods. A DBN is an artificial neural network (ANN) model that is trained in two phases. In the first phase, called pretraining, a restricted Boltzmann machine (RBM) is used to initialize the network model. This phase is unsupervised. In the second phase, called fine‐tuning, this model is then processed in a supervised manner. We examine two algorithms for computing the error gradients of stochastic gradient descent (SGD), used for optimizing the model in the fine‐tuning phase. These two algorithms are called backpropagation (Bprop) and resilient backpropagation (Rprop), whereas the latter is a more efficient advancement of Bprop Deep learning methods are known to be very complex models compared to conventional methods, for example, SVMs or random forests A second major objective of this paper was to investigate the integration of DBNs with SVMs. Put simply, this means we are using a hidden unit of the learned network structure as input layer for a SVM. This can be seen as a feature selection mechanism for the SVM because the DBN is used as a representation learner. Specifically, we investigate the integration of DBN with Bprop and SVM, DBN with Rprop and SVM, RBM‐learned representations and SVM, and autoencoder‐learned representations and SVM.In order to obtain robust results, for our analysis we are using two gene expression data sets for complex disorders: (a) breast cancer and (b) IBD. In contrast to single‐gene disorders, for example, sickle cell anemia or cystic fibrosis, complex or multifactorial disorders are caused by the synergy of genetic, environmental, and lifestyle factors In contrast to previous investigations analyzing DL for genomics data, our study is different with respect to the following points. First, we are using gene expression data from DNA microarray experiments, which are currently understudied in genomics. This complements studies using DNA sequence data, for example, Our paper is organized as follows. In the next section, we present the methods we use to analyze the data. Then, we present our results and a discussion thereof. We finish this paper with concluding remarks.There are a number of different learning algorithms available that can be used to build DL models for supervised learning problems. Examples for such models are convolutional neural networks (CNNs), DBNs, multilayer perceptrons (MLPs) and recurrent neural networks (RNNs) Currently, CNNs are the dominating model for tasks involving computer vision Recurrent neural networks are commonly used in tasks involving sequential input data, such as speech data, music, or text data Only recently, DL models have been used in computational biology. For instance, in Considering the brief history of DL models for general applications and specifically for problems in computational biology, there is currently no verdict about the best DL model for gene expression data. From the available information of previous studies, it looks that RNNs are not the best choice for analyzing gene expression data, because RNNs have been used mainly for sequential data with a correlation structure of nearby input data. Gene expression data do not possess such a sequential ordering and, hence, lack the properties of sequential data entirely. Similar arguments can be raised against CNNs In the following sections, we discuss DBNs in more detail, and then, we study their abilities in isolation and combination with SVMs.Neural networks have been studied since many years Neural networks can be trained via purely supervised learning methods; however, a suitable initialization of the model parameters, that is, the weights and biases, can make the learning faster and improve the performance Pretraining of DBNs consists of stacking RBMs, so that the next RBM in a chain is trained using the previous hidden layer as its visible layer, in order to initialize parameters for each layer. It can be shown that this is an efficient approach W and the biases b – have been initialized by RBMs, the parameters can then be tuned more in order to improve the model further. This second stage of DBN learning is called fine‐tuning, which uses the class‐label information of the training data set that was omitted in pretraining.After the neural network parameters for each layer – the weights We want to build models that can fit new samples well, that is, generalize well. This requires mathematical optimization. We achieve this by minimizing an error function . The mean squared error (MSE) is given by:oi = f(xi) is the ith output from the network function f:Rm→Rn given the ith input xi from the training set D=Dtrain={,…} and ti is the desired (target) output.In Eqn Similarly to maximizing the log‐likelihood of RBM via gradient ascent, we use gradient descent to find the parameter configuration that minimizes the error function.Here, the parameters are the learning rate η, the weight‐cost λ, and momentum ν.D at once, but instead via SGD with smaller mini‐batches. For estimating the gradient of the error function with respect to the weights and biases in each hidden layer and output layer, the backpropagation algorithm is the standard approach for this Usually, the gradient is not calculated using the whole training data set ail the activation of the ith unit in the lth layer , bit the corresponding bias, and wijl the weight for the edge between the jth unit of the (l − 1)th layer and the ith unit of the lth layer. If the neuron has an activation function, φ, then the activation of the lth layer with the (l − 1)th layer as input is al=φ(z(l))=φ(w(l)a(l-1)+b(l)). The following four equations can be derived, see Let us denote δL is the vector of errors of the output layer (L), δl of the lth layer, * is the elementwise product of vectors, and φ′ derivative of the activation function. Thus, the activation function is required to be differentiable. The gradient of the error with respect to the activations for the output layer is:In Equation ∂E∂aj(L)=(aj-tj).For instance, for the MSE one obtains Using the previous definitions, we can write a pseudocode for the backpropagation algorithm that is presented in Algorithm Algorithm 1−, Rprop+ and Rprop−  algorithm It has been shown that the backpropagation algorithm with SGD can learn good neural network models even without a pretraining stage, when appropriate activation functions are used, and an adequate amount of data are available for the training At present, there does not seem to be a general consensus among DL researchers about the shape of the architecture of a neural network. In some studies, a decreasing architecture is used ti ϵ {0, 1} or ti ϵ {p, 1 − p} with p ϵ . If we use the former form, the predicted class is the single output node value rounded, that is, 0 or 1. If we use the latter form, the predicted class is the index of the output vector yielding the higher value. From studies, we observed that the difference between both was negligible.For ANNs, the last output layer can be of arbitrary size, but for a binary classification, a good choice is either one node or two nodes. If the activation function is chosen to be the logistic function, the values are in the range . The outputs are then set to be The idea when combining deep networks with SVMs is to utilize the neural network as a representation learner compressing the original input vector. In this way, the SVM can utilize processed information. In our study, we investigate the influence of different types of deep neural network representations on the combination with a SVM. Specifically, we study RBMs, DBNs, and autoencoders. Fundamentally, all of them perform a dimensionality reduction, since they gradually transform the original representation into higher level representations.Regarding the choice of the input layer for the SVM, there are different possibility. For instance, for DBNs we can use the last hidden layer with as an input for the SVM. For autoencoders, a good choice is to use the code layer as input. In Fig. DBN^ and SVM^ denote the learned classifiers and DBN^(i‐th hidden layer of DBN|Xtraining) is a mapping from an input vector, given by Xtraining to an output, which is defined as the i‐th hidden layer of the DBN. These steps summarize the visualization shown in Fig. In Algorithm Algorithm 2r. The r package darch (versions 0.9.1 and 0.10.0) r package e1071 (version 1.6–7) All calculations were carried out in For our analysis, we use two DNA microarray data sets, one from breast cancer and one from IBD. In the following two sections, we provide a brief description of both.GSE2034.The breast cancer DNA microarray data we are using for our analysis are from r‐package The data set consists of 286 samples for which raw CEL files are available. We processed the raw data with the affy The data set includes the following clinical patient parameters: lymph node status , relapse (yes or no), estrogen receptor status (ER+ or ER). The clinical parameters are summarized in Fig. GSE3365. The array used was a Affymetrix Human Genome U133A Array. For these data, no raw CEL files are available in GEO, but the data available are preprocessed with the mas 5.0 algorithm. We transformed the data into a logarithmic scale.The second data set we are using provides DNA microarray data for IBDs For assessing the performance of the classifiers, we use the following error measures.These values can be obtained from the contingency table providing information about TP, TN, FP, and FN D is once randomly divided into k disjoint sets. If |D|=n, then each subset is of size n/k. The classifier is then trained k times, each time using one of the k subsets to test the classifier and the remaining k − 1 sets in training. For our analysis, we used k = 10, that is, 10‐fold CV.For assessing the variability in the data and for estimating the standard error of the error measures, we are using cross‐validation (CV). CV is the gold standard approach in estimating the prediction error n> samples and the smaller class of n< sample, we randomly drew n< samples from the larger class to balance the classes.It is known that the imbalance of classes can lead to problems in the error estimations In order to obtain meaningful results for the classification of the disease data, the parameters of our models need to be estimated from training data. In this respect, overfitting is a common problem in supervised learning that can negatively effect the results W ‖2=1 holds for each weight matrix Wi), we assessed the performance for two different classifications tasks: (a) ER status, comparing ER+ vs ER−, and (b) relapse status, comparing yes vs no.† indicates that for this method feature selection was used. For *, 1000 genes having the highest variance were selected and for †, 10 918 genes showing the largest differential expression by utilizing a t‐test. For instance, DBN: Bprop* means that a DBN has been trained with the backpropagation (Bprop) algorithm and 1000 genes with the highest variance have been selected as input features for the classifier.The results for the best performing DL classifiers, SVMs and other classifiers are summarized in Table Overall, our results show that the prediction of breast cancer relapse is substantially more difficult than predicting the ER status. This is consistent with previous findings t‐test (significance level of 0.05), hence removing over half of the features. In comparison, our SVM model without feature extraction performs equally. The Boosting result by For the relapse task, our DBN with Rprop and SVM without feature selection obtained the best results. For this data set, the differences are in general larger and the standard errors are higher, but also here several other combinations perform similarly well. We want to highlight that a SVM without feature section performs remarkably well. The reference study by The results in Table There are a few differences between the methods of the reference studies and our methods we would like to mention. Only we and The gene expression data for IBD consists of samples for Crohn's disease (CD), UC, and normal samples. We tested all three binary classifications, that is, UC vs CD, UC vs Normal, and CD vs Normal. In addition, we classified them combined, that is, UC vs CD vs Normal. In Table genecluster 2.0 gene expression analysis software was used. They tested their classifier for different feature sizes of the input vector varying between 1 and 200 genes and found Acc values between 65% and 94%. The highest Acc of 94% was achieved for a feature size of 14  can be considered of reasonable size for gene expression data allowing the application of CV. Importantly, none of our results was data set (disease) specific, but both independent data sets lead to the same overall conclusions regarding the applied classification methods. Hence, one data set could be considered a validation case for the other with respect to our technical results.In this paper, we studied the classification of high‐dimensional gene expression data from genomics from breast cancer and IBD. This is an important computational diagnostics task for translational research with possible applications in personalized and precision medicine. We provided a comprehensive analysis of the classification performance of DBNs in dependence on its multiple model parameters and in comparison with SVMs. Based on this analysis, we found the combination of DBN and SVM performs tendentially best, but requires a substantial analysis effort and a thorough technical understanding of DL. In contrast, the LIBSVM implementation of a SVM provides compatible results, which are much easier to attain. Classifiers using only a DBN led to a middle performance but require a similar effort as the combination of DBN and SVM.Whether other DL classifiers perform differently to DBNs or whether sample expansion methods as, for example, suggested by The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.FES conceived the study. JS implemented all scripts and analyzed the data. JS, MD, and FES wrote the paper. All authors approved the final version.


Translational medicine (TM) is an emerging domain that aims to facilitate medical or biological advances efficiently from the scientist to the clinician. Central to the TM vision is to narrow the gap between basic science and applied science in terms of time, cost and early diagnosis of the disease state. Biomarker identification is one of the main challenges within TM. The identification of disease biomarkers from -omics data will not only help the stratification of diverse patient cohorts but will also provide early diagnostic information which could improve patient management and potentially prevent adverse outcomes. However, biomarker identification needs to be robust and reproducible. Hence a robust unbiased computational framework that can help clinicians identify those biomarkers is necessary.We developed a pipeline (workflow) that includes two different supervised classification techniques based on regularization methods to identify biomarkers from -omics or other high dimension clinical datasets. The pipeline includes several important steps such as quality control and stability of selected biomarkers. The process takes input files (outcome and independent variables or -omics data) and pre-processes  them. After a random division of samples into training and test sets, Least Absolute Shrinkage and Selection Operator and Elastic Net feature selection methods are applied to identify the most important features representing potential biomarker candidates. The penalization parameters are optimised using 10-fold cross validation and the process undergoes 100 iterations and a combinatorial analysis to select the best performing multivariate model. An empirical unbiased assessment of their quality as biomarkers for clinical use is performed through a Receiver Operating Characteristic curve and its Area Under the Curve analysis on both permuted and real data for 1000 different randomized training and test sets. We validated this pipeline against previously published biomarkers.We applied this pipeline to three different datasets with previously published biomarkers: lipidomics data by Acharjee et al.  contains supplementary material, which is available to authorized users. Translational medicine (TM) –3 is an In -omics literature, there has been a recent trend towards the identification of data pre- and post-processing steps. For example, Satagopam et al.  developeIn this study, we develop a pipeline that includes two machine learning algorithms, inspired by simple linear models, coupled with follow-up approaches for systematic data analysis. Our systematic analysis includes data quality checks, identification of important features, as well as combinatorial and stability analyses. We applied and validated our pipeline with three different previously published -omics datasets. Our approach successfully identified the markers reported in the literature as well as potential novel markers.Our pipeline is composed of statistical machine learning modules whose methods are described below. Additionally, we applied and validated our pipeline against three independent published datasets; two RNA microarray datasets and one lipidomics experiment.x number of predictor variables (features) and We used two feature selection methods, LASSO  and Elasentclass1pt{minimadded Eq.  and so tIn Ridge regression , the prementclasspt{minimaions Eq. . A diffeAll analyses were performed in the R statistical computing (R version 3.4.3) environment . All R pTo assess the performance of our framework, we employed a previously published lipidomics dataset from Acharjee et al. . The lipFrom the CBGS cohort we used 3 different datasets, namely CBGS-1, CBGS-2 and POPS (Pregnancy Outcome Prediction Study). All data was obtained from dried blood spots and generated with direct infusion high-resolution mass spectrometry (HRMS).A summary of the cohort is listed below Table .Table 1CA pancreatic ductal adenocarcinoma (PDAC) microarray expression dataset  GSE15471, , as wellhttps://github.com/jaw-bioinf/Biomarker_Identification/.All code and functions are available on our hosted GitHub repository: A graphical depiction of our workflow can be seen in Fig. The data quality module consists of different checks on both features and samples to exclude or data based on the amount of missingness, and to standardize data to make measurements of features in different experiments comparable. Missing value imputation and normalisation steps can be implemented as needed by end users. Normalization methods can be bypassed if RNA microarray datasets are downloaded pre-processed directly from repositories. For these and other microarray datasets, optional filtering steps to reduce dimensionality include many-to-one probe-to-gene mapping. Features may be further reduced by using external tools or expert biological insight to exclude features, but such reduction is optional. In all cases, the input for the ‘feature selection’ segment of the pipeline is a matrix (file or data frame) of features  as well as a set of samples along with a target or outcome variable.In order to apply the LASSO and EN algorithms for biologically relevant feature selection, we need to optimize the penalty parameter associated with each of the methods in an unbiased way. To achieve this, the pipeline divides samples randomly into a training set composed of 75% of the total number of the samples and a test set consisting of the remaining 25% samples. To ensure optimal training in real-world datasets, all data splits retained class balances of the target variable, so each split reflected a proportion of the target observed in the underlying dataset. We note that class balancing measures, such as boosting or under sampling, are not used to artificially balance training/testing data in each split (outer loop set). Then we apply a 10-fold cross validation on the training set (inner loop set) aiming to have an optimised penalty parameter that can be plugged into the LASSO and EN models. Mathematically, LASSO and EN models can be defined by using a single penalty function “ument}α”  Eq. 6)..\documenOur pipeline iterates model creation 100 times and selects the features that appear more than 90 times in the analysis, as these we deem to be the more significant for the classification model. Moreover, in order to better understand the relationship between the features selected and the outcome variable analysed, a display of the weight  and the Pregnancy Outcome Prediction Study (POPS). Our objective was to identify potentially nutritional lipid biomarkers for the classification of babies fed with Formula, Human or a mix of Human and Formula milk. In Fig. For those selected features, performance evaluation was then performed. Results can be seen in Fig. Senescence and Autophagy in CancerIntegrated Pancreatic Cancer PathwayTGF Beta Signalling Pathway (mouse)miRNA targets in ECM and membrane receptorsTGF Beta Receptor Signalling (human).In the pancreatic cancer dataset GSE15471, top features selected included the following 20 genes: SULF1, COL10A1, MIR34AHG, INHBA, COL8A1, FN1, THBS2, NOX4, NTM, RASAL2, ADAMTS12, CAPG, CTHRC1, FAP, VCAN, SLPI, WISP1, LTBP1, GPRC5A, TIMP1. Our biological pathway analysis revealed a class of biomarkers enriched in several known cancer pathways. After stringent multiple testing correction, the following pathways were identified as being enriched:The gene set was further enriched in the Human Proteome Map in adult Esophagus, Lung, and Pancreas tissues, indicating a potential cross-talk among tissue-specific cancer pathways. The full results of our pipeline, including feature ranking graphs and ROC AUC, sensitivity, specificity, and accuracy scores of each variable, as well as all the executed code are included in Additional file In addition to the validation of our method against pancreatic cancer, a validation was performed against an acute myeloid leukaemia (AML) cohort. This analysis revealed sixty genes contributing significantly to AML  to validate our approach by identifying potential markers or signatures and comparing with existing markers found in the literature.Acharjee et al.  investigIn order to assess the wider applicability of our approach for identifying target molecules in different types of -omics data, we also applied it in two transcriptomic datasets, one for Acute Myeloid Leukemia and one for Pancreatic Ductal AdenoCarcinoma (PDAC). Whereas the lipidomic analysis could be validated by expert curation, our transcriptomic analyses were validated via external pathway and ontology gene set enrichment tools.AML driver gene analysis revealed a set of genes known to be enriched for targets of the MYB transcription factor. MYB is known to play a crucial role in hematopoietic stem cell cycles, including proliferation and survival, and recent research has shown that AML-specific microRNAs target c-MYB . AdditioThe genes identified by our pipeline are often discussed in pancreatic cancer literature . Not onlIn high-dimensional -omics data analysis, we are interested in finding a relevant smaller subset of variables that are associated with the response . Procedures to identify such smaller subsets are called variable or feature selection procedures. By employing such procedures, it is possible to reduce the dimensionality of the data . MoreoveOne advantage of using feature selection algorithms is that the final model is built automatically, including only those biomarkers which are useful in predicting patient condition. Thus, we do not have to rely on the cut off for selection of genes and metabolites upfront. All the estimates are decided based on either biomarkers’ effects. However, one of the drawbacks of this method lies with the selection of the appropriate penalty parameters. Failure to decide on appropriate penalty factor will result in underfitting or over fitting of the results. To address this, we split the data into two subsets, training and testing. Within the training subset, we estimated the penalty factor by using ten-fold cross validation. The optimized model was then fit to the unseen testing subset.While either LASSO or EN can be used for both classification and regression tasks, our method focused on validation tasks based on binary outcomes (classification). In a planned update of the software accompanying our method, we will enable users to switch between classification and regression tasks. Users will also be able to choose between different feature selection algorithms and machine learning models including Random Forests, Artificial Neural Networks, and Deep Leaning which can capture alternative patterns of interactions in the data that we might miss out with regularized linear models . We alsoA unique strength of our approach lies with the provision of automated pre-processing and feature selection. Based on our approach, we were able to reduce the number of potential causative genes in each experiment to under 100  whilst the high confidence selections were reduced to less than 15. This robust selection creates a useful feature for end users, eliminating the need to pre-filter data based on perceived biological knowledge thus eliminating bias.To completely understand the underlying biological mechanisms driving diverse phenotypes, a multi-omics approach is often necessary. However, this is a challenging step due to the data size, measurements, and data analysis involved. Different approaches are currently suggested in the literature to link or integrate them. For example, Shen et al. used multi-omics datasets which include copy number, gene expression, and methylation data from TCGA in an unsupervised matrix factorization algorithms using the software i-Cluster . Seoane In addition to -omics datasets, there are other unstructured clinical phenotypic datasets such as medical images, electronic health records, and medical questionnaires. These pose new challenges for data integration and reproducibility that needs standardization and to put into clinical practices . ProposeIt is worth mentioning that certain areas of precision medicine benefit greatly from incorporating single-cell sequencing data, especially cancer. While multiple -omics approaches can be used with single-cell sequencing , 41, RNAThe application of single-cell omics to our pipeline may be useful in model organism and basic research to guide future translational projects or prioritize experiments for biomarker validation.We present a data-driven, generalizable, robust, low-bias machine learning workflow that generates easily interpretable outputs and focus on simple visualizations aiming at actionable biomarker discovery. We believe that our workflow will help researchers to identify significant explanatory features of experimental -omics data, reducing the search space for good candidates for experimental validation and follow up. Robustly optimizing feature selection to changes in data perturbation provides a high confidence in the selection of potential novel features, which forms a crucial advantage in translational medicine applications.Additional file 1. R Markdown analysis results from the workflow developed on GSE15471.Additional file 2. Lipids identified in three cohorts are listed with different category. Category A: HM vs. Mixed (FM and HM combined) feeding; Category B: FM vs. Mixed (FM and HM combined ) feeding; Category C: HM vs. FM.Additional file 3. R Markdown analysis results from the workflow developed on GSE15061.


W) algorithm. In this approach, the normal samples being completely and exclusively clustered into one class is considered to be the standard of reasonable clustering subtypes, and the feature selection pays attention to imbalances of samples among subtypes. With this approach, we identified the molecular subtypes of colon cancer on the mRNA gene expression dataset of 153 colon cancer samples and 19 normal control samples of the Cancer Genome Atlas (TCGA) project. The colon cancer was clustered into 7 subtypes with 44 feature genes. Our approach could identify finer subtypes of colon cancer with fewer feature genes than the other two recent studies and exhibits a generic methodology that might be applied to identify the subtypes of other cancers.Identifying molecular subtypes of colorectal cancer (CRC) may allow for more rational, patient-specific treatment. Various studies have identified molecular subtypes for CRC using gene expression data, but they are inconsistent and further research is necessary. From a methodological point of view, a progressive approach is needed to identify molecular subtypes in human colon cancer using gene expression data. We propose an approach to identify the molecular subtypes of colon cancer that integrates denoising by the Bayesian robust principal component analysis (BRPCA) algorithm, hierarchical clustering by the directed bubble hierarchical tree (DBHT) algorithm, and feature gene selection by an improved differential evolution based feature selection method (DEFS Identifying the molecular subtypes of colorectal cancer (CRC) may allow for more rational, patient-specific treatment in the future. Various studies have been done to predict molecular subtypes for CRC based on gene expression data. Fearon and Vogelstein utilized four different genomic and epigenomic instabilities to identify four subtypes of CRC: chromosome instability (CIN), microsatellite instability (MSI), CpG island methylator phenotype (CIMP), and DNA global hypomethylation . Using cW) method [W algorithm. In view of the usual imbalance of samples among subtypes, we use the naive Bayes (NB) classifier with empirical prior probabilities and weight accuracy to evaluate classification ability. The empirical prior probabilities estimate the prior probabilities from the relative frequencies of the classes in training, which can lessen the influence of the imbalance of samples. Weight accuracy is a special assessment measurement for classification of imbalance samples.One popular approach to identifying cancer subtypes is clustering the gene expression data of patient samples, as expression data can give a comprehensive snapshot of transcription activities for whole genomes . Because) method , which nW. We applied this approach to the Cancer Genome Atlas  mRNA gene expression dataset of colon cancer and identified 7 subtypes with 44 feature genes. The results deliver finer subtyping with fewer feature genes than in the other two recent studies.In this study, we integrated these state-of-the-art techniques of denoising, clustering, and feature selection to identify molecular subtypes in human colon cancer using gene expression data. Our integrated approach incorporates denoising by the BRPCA, hierarchical clustering by the DBHT, and selecting feature genes by DEFSThe microarray mRNA gene expression dataset we used to identify the subtypes of colon cancer is from TCGA. It includes 153 cancers samples, which have been used by Muzny et al.  and Ren W to select the feature genes for the clusters. A summary of our approach is shown in We first applied the BRPCA to denoise the gene expression data by getting the sparse component but removing the low-rank and noise components, and then we used the DBHT to cluster the sparse components in the BRPCA. The normal samples completely and exclusively clustered into one class were considered as the standard of reasonable clustering. If the standard was not reached, we continually tuned the setting of the hyperparameters of the BRPCA until the clustering was up to the standard. Finally, we used the DEFSIn the BRPCA model , the obsIn our study, the observed data matrix For Since the density function at one layer is conjugate to the density function at the layer above it, the posterior density function is easily computed via Markov chain Monte Carlo (MCMC) implemented using a Gibbs sampler. The details for calculation of the BRPCA algorithm are described in algorithm 1 in Ding et al. .The directed bubble hierarchical tree (DBHT)  algorithIn our study, a sample is a vertex and the Pearson’s correlation coefficient matrix is used as the similarity matrix of the vertexes.The dissimilarity matrix of the vertexes we used is:Based on the similarity and dissimilarity matrix of samples, the DBHT constructs the PMFG to perform clustering and get hierarchies of samples. The details for the DBHT algorithm are described in Song et al. .O [W [O, the desired feature subset size is predefined by the user, while in the DEFSW, the optimal feature subset size can be discovered automatically only by setting an upper limit. In the two algorithms, a wrapper classifier such as K nearest neighbor (KNN), support vector machine (SVM), or naive Bayes (NB) classifier is needed. The wrapper classifier and an assessment measurement such as classification accuracy are used together to evaluate the classification ability of features.To identify the subtypes using gene expression profiling, feature selection can be used to reduce the high-dimensionality of huge amounts of otherwise meaningless data. Khushaba and Al-Ani et al. proposed a powerful feature selection method that utilizes the differential evolution (DE) float number optimizer in the combinatorial optimization problem of feature selection, named DEFSO , followeO [W . In the W algorithm and its parameter setting that we used are listed in In our study, some subtypes have a small number of samples and others have a lot, i.e., there are imbalances of samples among subtypes. Therefore, a wrapper classifier and an assessment measurement that can cope with the class imbalance have to be used to avoid the “larger subtypes win.” Meanwhile, for the DE algorithm, the computation cost is generally huge because of its iterative evolution, so a fast and simple classifier is desired. Not only can the NB classifier be trained very efficiently under the condition of a small amount of training data and take only linear time, but its empirical prior probabilities can lessen the influence of the imbalance of samples. To assess classification ability, we used weight accuracy instead of the usual classification accuracy. The weight accuracy of classification is defined by Draminski et al.  as:(13)wWe first used the BRPCA to denoise. In the BRPCA model, we set the largest possible rank as ters see , in whicWe also tried to directly cluster the mRNA gene expression dataset using DBHT without denoising . It sugge result b suggestw algorithm. Considering that the latent premise of identifying subtypes is that the samples are from cancer patients, we expected that the selected feature genes could not only identify cancer subtypes, but also discriminate between tumor and normal samples, i.e., we expected to select out the feature genes that could discriminate all eight sample clusters. It was rational to select the feature genes from the differentially expressed genes (DEGs) of tumor vs. normal samples. We obtained 5897 DEGs by t-test  and fold change cutoff 1.5, and then selected the feature genes from these DEGs using the DEFSW algorithm described in w algorithm could deliver an optimal size of feature genes. We then determined our final size of feature genes to be that which gave the maximal average weight accuracy of the eight sample clusters. In this way, we ended up with 44 feature genes: THBS2, NOX4, KIAA1199, SLC16A4, CCDC19, ZNRF3, GOLT1A, HYAL3, C15orf26, KIFC1, TIPIN, CTNNAL1, CALU, TAF1A, MCM2, MSH6, FLAD1, GCG, SCRG1, PTGER2, TIMD4, MUC1, PLOD2, LIMS2, ADH1B, PTN, PTPN7, AQP1, PSD3, CRAT, ATOH8, CGN, C6orf204, FTHP1, KCNMB1, LIG4, PPFIBP2, PPP2CB, ALAS2, ZZEF1, ATXN7, GRLF1, FAM102A, and C1orf152. Among them, MSH6 is known to be related to CRC, and is located in the Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway of CRC [THBS2 is a potential prognostic biomarker in CRC [NOX4 predicts poor prognosis and promotes tumor progression in human CRC [NOX4 plays a role in PhIP-induced colon carcinogenesis, especially during the early stages before tumor onset [NOX4 is highly predictive of relapse in stage II left-side colon cancer [KIAA1199 in human colorectal tumors  is markedly higher than that in the normal colonic mucosa [KIAA1199 plays a critical role in maintaining an aggressive phenotype of tumor cells, and suppression of KIAA1199-related motilities of tumor cells contributes to reduced tumor metastasis in CRC [MCM2 is correlated with the cell proliferation state in colon cancer [Ki-67 in identifying colorectal mucosal proliferation [MUC1 is aberrantly overexpressed in human colon cancers and is associated with invasion, metastases and a poor prognosis [ZNRF3 is important in serrated tumorigenesis and has identified a potential therapeutic strategy for CRC subtype [LIG4 may represent a new epigenetic marker for CRC independent of known markers [ADH1B displays decreased expression during progression from adenoma to early and more advanced stage of colorectal carcinomas [We then selected the feature genes for the identified subtypes by the DEFSr in CRC  and can r in CRC ; Overexpuman CRC , and NOXor onset , and morn cancer ; KIAA119c mucosa ,33 and ic mucosa , and couc mucosa ; Furthers in CRC ; MCM2 isn cancer  and is mferation ; MUC1 isrognosis ,40; ZNRF subtype ; LIG4 ma markers ; ADH1B drcinomas . In contWe used the same NB classifier to validate the classification ability of these feature genes. We did 10%, 20%, and 30% cross-validation (CV) with 1000 repeats. The mean classification accuracy, the mean weight accuracy of classification, and the mean classification accuracy for each class of the 1000 CVs are listed in w algorithm can discover the optimal feature subset size. Considering the unstable stochastic search, we repeated the DEFSw algorithm multiple times with different upper limits of the feature size and selected the feature genes from the DEGs. This may be helpful for selecting out the optimal size of feature genes and getting higher accuracy for discriminating tumor and normal samples. Overall, the proposed approach can identify finer subtypes of colon cancer with fewer feature genes than the other two recent studies and exhibits a generic methodology for identifying cancer subtypes based on gene expression data by common processes.To identify cancer subtypes based on gene expression data, the proposed approach innovatively integrated state-of-art denoising, clustering, and feature selection algorithms. In BRPCA, the low-rank component of gene expression data may be the same as the background of an image and the noise component simulates unknown and nonstationary noise, whereas the sparse component may be the same as the foreground and is the key information for clustering. The DBHT is intrinsically a correlation-based clustering method. Through building the PMFG, the method does not need any prior tuning and provides both intracluster hierarchy, which describes the way clusters are composed, and intercluster hierarchy, which describes how clusters gather together. To assess whether the clustering is meaningful, we draw in the concept of “reference object” from classical physics. If the reference objects, the normal samples, are correctly clustered together, we consider the clustering as reasonable; otherwise, we need to tune the parameters in the BRPCA. Tuning the parameters means adjusting the degree of details of component Inter-tumor diversity of CRC complicates the prediction of disease and treatment outcomes. Subtypes of colorectal cancer identified by classifying gene expression profiles with defined prognostic markers would predict individual patient outcomes more precisely and therefore provide valuable guidance on appropriate therapeutic intervention . It is p


Considering that colorectal cancer is the third leading cancer-related cause of death within the United States, it is important to detect colorectal tumors early and to prevent the formation of tumors. Short-chain fatty acids (SCFAs) are often used as a surrogate for measuring gut health and for being anticarcinogenic because of their anti-inflammatory properties. We evaluated the fecal SCFA concentrations of a cohort of individuals with different colonic tumor burdens who were previously analyzed to identify microbiome-based biomarkers of tumors. We were unable to find an association between SCFA concentration and tumor burden or use SCFAs to improve our microbiome-based models of classifying people based on their tumor status. Furthermore, we were unable to find an association between the fecal community structure and SCFA concentrations. Our results indicate that the association between fecal SCFAs, the gut microbiome, and tumor burden is weak. Colonic bacterial populations are thought to have a role in the development of colorectal cancer with some protecting against inflammation and others exacerbating inflammation. Short-chain fatty acids (SCFAs) have been shown to have anti-inflammatory properties and are produced in large quantities by colonic bacteria that produce SCFAs by fermenting fiber. We assessed whether there was an association between fecal SCFA concentrations and the presence of colonic adenomas or carcinomas in a cohort of individuals using 16S rRNA gene and metagenomic shotgun sequence data. We measured the fecal concentrations of acetate, propionate, and butyrate within the cohort and found that there were no significant associations between SCFA concentration and tumor status. When we incorporated these concentrations into random forest classification models trained to differentiate between people with healthy colons and those with adenomas or carcinomas, we found that they did not significantly improve the ability of 16S rRNA gene or metagenomic gene sequence-based models to classify individuals. Finally, we generated random forest regression models trained to predict the concentration of each SCFA based on 16S rRNA gene or metagenomic gene sequence data from the same samples. These models performed poorly and were able to explain at most 14% of the observed variation in the SCFA concentrations. These results support the broader epidemiological data that questions the value of fiber consumption for reducing the risks of colorectal cancer. Although other bacterial metabolites may serve as biomarkers to detect adenomas or carcinomas, fecal SCFA concentrations have limited predictive power. Colorectal cancer is the third leading cancer-related cause of death within the United States . Less th5–Many microbiome studies use the concentrations of SCFAs and the presence of 16S rRNA gene sequences from organisms and the genes involved in producing them as a biomarker of a healthy microbiota , 10. Depn = 172) and those with colonic adenomas (n = 198) or carcinomas (n = 120)  were treated and sampled a year later . Finally, we pooled the SCFA concentrations on a total and per molecule carbon basis and again failed to observe any significant differences (P > 0.077). Although some of the P values from our analyses were close to 0.05, the effect sizes were all relatively small and inconsistent given the disease progression  . We weregression . These rPadenoma < 0.001 and Pcarcinoma < 0.001). However, the AUROC values to detect the presence of adenomas or carcinomas were only 0.54 and 0.55, respectively, indicating that SCFAs had poor predictive power on their own  based on 97% similarity or into genera enabled us to classify individuals as having adenomas or carcinomas using random forest machine learning models , 16. We heir own . When we10.1128/mBio.01454-19.1FIG S1FIG S1, EPS file, 0.09 MB.Comparison of training and testing results for classification models shows that the models are robust and are not overfit. The random forest classification models were generated to differentiate between individuals with healthy  colons and those with adenomas or carcinomas using 16S rRNA gene sequence data that were clustered into genera or OTUs by including or not including the three SCFAs as additional features. The random forest classification models were generated by partitioning the samples into a training set with 80% of the data and a testing set with the remaining samples for 100 randomizations. Download Copyright © 2019 Sze et al.2019Sze et al.Creative Commons Attribution 4.0 International license.This content is distributed under the terms of the R2 between the observed SCFA concentrations and the modeled concentrations was 0.14, which was observed when using genus data to predict butyrate concentrations   . We binner = 26) . Similartrations . Becausetrations . SCFA cotrations . The ina10.1128/mBio.01454-19.2FIG S2FIG S2, EPS file, 0.1 MB.Comparison of training and testing results for regression models shows that the models are robust and are not overfit. Random forest regression models were generated to predict the concentration of each SCFA using each individual’s microbiome data generated using 16S rRNA gene sequence and metagenomic sequence data. These regression models were generated by partitioning the samples into a training set with 80% of the data and a testing set with the remaining samples for 100 randomizations. Download Copyright © 2019 Sze et al.2019Sze et al.Creative Commons Attribution 4.0 International license.This content is distributed under the terms of the Our data indicate that fecal SCFA concentrations are not associated with the presence of adenomas or carcinomas and that they provide weak predictive power to improve the ability to diagnose someone with one of these lesions. Furthermore, knowledge of the taxonomic and genetic structure of gut microbiota was not meaningfully predictive of SCFA concentrations. These results complement existing literature that suggest that fiber consumption and the production of SCFAs are unable to prevent the risk of developing colonic tumors. It is important to note that our analysis was based on characterizations of SCFA and microbiome profiles using fecal samples at a single time point. Furthermore, observations along the mucosa near the site of lesions may provide a stronger association. This may be a cautionary result to temper enthusiasm for SCFAs as a biomarker of gut health more generally. Going forward, it is critical to develop additional hypotheses for how the microbiome and host interact to drive tumorigenesis so that we can better understand tumorigenesis and identify biomarkers that will allow early detection of lesions.nadenoma= 41 and ncarcinoma=26) were sampled after treatment of the lesion . Tumor diagnosis was made by colonoscopic examination and histopathological review of the biopsy specimens  Measuring specific SCFAs. The measurement of acetate, propionate, isobutyrate, and butyrate used a previously published protocol that used high-performance liquid chromatography (HPLC) (y (HPLC) . Two cha(iii) 16S rRNA gene sequence data analysis. Sequence data from Baxter et al.  Metagenomic DNA sequence analysis. A subset of the samples from the samples described by Baxter et al. . These data were generated by Hannigan et al.  Random forest models. The classification models were built to predict lesion type from microbiome information with or without SCFA concentrations. The regression models were built to predict the SCFA concentrations of acetate, butyrate, and propionate from microbiome information. For classification and regression models, we preprocessed the features by scaling them to vary between zero and one. Features with no variance in the training set were removed from both the training and testing sets. We randomly split the data into training and test sets so that the training set consisted of 80% of the full data set, while the test set was composed of the remaining data. The training set was used for hyperparameter selection and training the model and the test set was used for evaluating prediction performance. For each model, the best performing hyperparameter, mtry, was selected in an internal five-fold cross-validation of the training set with 100 randomizations. The mtry parameter represents the number of features randomly sampled from the available features at a question point in the classification tree  that, when answered, lead to the greatest improvement in classification. Six values of mtry were tested, and the value that provided the largest AUROC or R2 was selected. We trained the random forest model using the selected mtry value and predicted the held-out test set. The data-split, hyperparameter selection, training, and testing steps were repeated 100 times to get a reliable and robust reading of model prediction performance. We used AUROC and R2 as the prediction performance metrics for classification and regression models, respectively. We used the randomForest R package (version 4.6-14) as implemented in the caret R package (version 6.0-81) for developing and testing our models.(vi) Statistical analysis workflow. Data summaries, statistical analysis, and data visualizations were performed using R (v.3.5.1) with the tidyverse package (v.1.2.1). To assess differences in SCFA concentrations between individuals with healthy colons and those with adenomas or carcinomas, we used the Kruskal-Wallis rank sum test. If a test had a P value below 0.05, we then applied a pairwise Wilcoxon rank sum test with a Benjamini-Hochberg correction for multiple comparisons. To assess differences in SCFA concentrations between individual samples before and after treatment, we used paired Wilcoxon rank sum tests to test for significance. To compare the median AUCROC for the held-out data for the model generated using only the SCFAs, we compared the distribution of the data to the expected median of 0.5 using the Wilcoxon rank sum test to test whether the model performed better than would be achieved by randomly assigning the data to each diagnosis. When we compared the random forest models generated without and with SCFA data included, we used Wilcoxon rank sum tests to determine whether the models with the SCFA data included did better.(vii) Code availability. The code for all sequence curation and analysis steps, including an Rmarkdown version of this article, is available at https://github.com/SchlossLab/Sze_SCFACRC_mBio_2019/.


N ≥ 500. A common issue when analyzing limited samples of RNA-seq count data is that the data follows an over-dispersed distribution, and thus a Negative Binomial likelihood model is often used. Thus, we have developed a Negative Binomial model-based (NBMB) clustering approach for application to RNA-seq studies.Unsupervised clustering represents one of the most widely applied methods in analysis of high-throughput ‘omics data. A variety of unsupervised model-based or parametric clustering methods and non-parametric clustering methods have been proposed for RNA-seq count data, most of which perform well for large samples, e.g. N = 96) and TCGA study of ovarian cancer (N = 295). Simulation results showed that NBMB outperforms GMM applied with different transformations in majority of scenarios with limited sample size. Additionally, we found that NBMB outperformed GMM for small clusters distance regardless of sample size. Increasing total number of genes with fixed proportion of differentially expressed genes does not change the outperformance of NBMB, but improves the overall performance of GMM. Analysis of type 2 diabetes and ovarian cancer tumor data with NBMB found good agreement with the reported disease subtypes and the gene expression patterns. This method is available in an R package on CRAN named NB.MClust.We have developed a Negative Binomial Model-Based (NBMB) method to cluster samples using a stochastic version of the expectation-maximization algorithm. A simulation study involving various scenarios was completed to compare the performance of NBMB to Gaussian model-based or Gaussian mixture modeling (GMM). NBMB was also applied for the clustering of two RNA-seq studies; type 2 diabetes study  contains supplementary material, which is available to authorized users. A common goal of RNA-seq studies is unsupervised clustering –3. UnsupOver the past decade a substantial body of research has emphasized the importance of over-dispersion in the statistical modeling of RNA-seq data. Durán Pacheco et al.  comparedGaussian Model-Based clustering with logarithm or Blom  transforSimilar to the algorithms by Si et al. , we combX is a N × G matrix, where N is the sample size and G is the number of genes being considered for the clustering analysis. Each row of X represents RNA-seq gene counts for a sample, denoted by xi, i = 1, …, N, where each element of xi is denoted by xig, i = 1, …, G. In this study we assume independence between all genes and independence between all samples or subjects. Suppose the samples of RNA-seq counts x1, …, xN can be grouped into K clusters. NBMB assumes xi belongs to cluster k, k = 1, …, K, and xi~NB, where μk = , θk =  are the parameters of cluster k. The density function of xig isk for each sample xi is unknown and cannot be observed. The prior probabilities for each sample belonging to each component are p1, …, pK and p1 + … + pK = 1. The log likelihood function for x1, …, xN is f is the density function for sample i when it is assigned to cluster component k; pk is the prior probability for belonging to component k; μk, θk are the mean and dispersion of component k. The NBMB method assumes that RNA-Seq counts without batch and the library size effects follow the NB distribution, hence, NBMB must be applied to normalized counts without any transformation.The observed RNA-seq data set L and estimate μk and classification of samples. The optimal value of K is determined by Bayesian Information Criterion (BIC) [We adopt the following EM algorithm to optimize likelihood function on (BIC) , 23.Calculate subject-specific posterior probability and value of expectation function.s − 1) is given by:i belongs to cluster component k at iteration (s − 1). Ink at iteration (s − 1);k at iteration (s − 1) and i when it is assigned to component k.The expectation function at iteration  by multiplying G univariate Negative Binomial density functions. However, the existence of zero value of fg cannot be avoided when G is large (e.g. G ≥ 1000), which might result in the denominator of fg to. In applying DA, we used the values τ0 = 2 or 10, τs + 1 = rτs and r = 0.9, similar to the values proposed by Rose et al. [τ0 = 10 providing an optimal solution in most of simulation scenarios. Assessment of the robustness via a simulation study found that these values do achieve the best performance across all scenarios.Due to the assumed independence between genes and subjects, we construct the multivariate density function  (DA) by . We chooe et al. , with τ0K, because we assumed at least two cluster components based on prior knowledge, similar to [∆μ = 0.1, 0.5, 1; ∆θ = 0, 1 ); percentage of differentially-expressed (DE) genes ; sample size ; and number of genes . For each scenario, 100 datasets were generated. Model-based clustering usually requires filtering to a set of genes, as the performance is poor when too many ‘null’ genes  are included. Recent gene expression studies often filtered genes down to 1000–5000 genes for clustering by some measure of variability [G= 1000 and 5000.To assess the performance of NBMB, an extensive simulation study was completed in which NBMB was compared to GMM with no transformation, GMM on log transformed counts, and GMM on Blom transformed counts . For eacmilar to . The simiability , 28 or biability , 30. Henμ1 and θ1 were the MLE of μ and θ for the expression of the most variable genes in ovarian cancer tumor samples from TCGA. The first cluster component was generated by μ1 and θ1. The parameters for each of the remaining cluster components, i.e. μk and θk, k = 2, …, K were computed as μk = μ1ek − 1)∆μ∆θ, while the shift step of dispersion θ was set at either zero or one . The value of ∆μ was similar to the log fold change of RNA-Seq counts between the subgroups of TCGA ovarian cancer tumor samples.The simulated data sets were generated by baseline parameters and the shift step between clusters. Baseline parameters values To assess the performance of NBMB to the commonly used GMM with transformations, the clustering methods were applied to two transcriptomic studies. For clustering analysis of each dataset, the most variable genes were selected based on the Median Absolute Deviation (MAD) as discussed in  and then, GSE63887. This dataset does not contain any known batch effects as described in [edgeR package within R statistical software to calculate library size normalized counts based on the upper quartile normalization factor, followed by computation of the counts per million (CPM) for which clustering analysis was applied. The range of possible number of clusters (K) for the analysis were 2 to 5, since the four groups of subjects in the T2D study can be reclassified as case and control.The first application dataset is a longitudinal transcriptomic study of obesity and type 2 diabetes (T2D) in which RNA was extracted from isolated skeletal muscle precursor cells from 24 subjects. RNA was sequenced on the Illumina HiSeq 2000 platform, with data downloaded from Gene Expression Omnibus (GEO) at GSE81965ribed in . We use N = 295). Tissue site effect within this dataset is removed by Empirical Bayes method [http://bioinformatics.mdanderson.org/tcgambatch/ by selecting Disease ‘OV’, Center/Platform ‘illuminahiseq rnaseqv2 gene’, Data Level ‘Level 3’ and Data Set ‘Tumor-corrected-EBwithParametricPriors-TSS’. The downloaded data (X) was the normalized expression abundance on the log scale; therefore the data was converted to the original scale by eX. The range of possible clusters (K) was from 3 to 5, as multiple groups have reported that there are between 3 and 5 subtypes of serous ovarian cancer [The second dataset contains normalized RNA-seq counts from the TCGA study of ovarian serous cystadenocarcinoma tumors , and Additional file ∆θ = 0). In general, NBMB outperformed the GMM in most of the scenarios, especially when total sample size or cluster distance is small. The simulation results also indicate that for the scenarios assessed, the none and logarithmic outperformed the Blom transformation for the Gaussian model-based approach.There were 480 simulation scenarios being assessed, with 100 datasets simulated per scenario. Unsupervised model-based clustering was completed on each simulated data set using the Negative Binomial mixture model (NBMB) or the Gaussian mixture model (GMM) with log, Blom or no transformation. To assess performance, adjusted rand index (ARI) was computed with mean ARI for the scenarios presented in Fig. N = 50, 100), regardless of the number of genes or the shift step in parameters, except for a few scenarios. This exception might be the result of simulation randomness or less optimal annealing rate. ARI of GMM on log transformed data increases sharply for the K > 3 scenarios at larger sample sizes  and higher proportion of DE genes (10%). The results in larger sample sizes also imply that GMM with log-transformation is an ideal approach for a large-scale study and may outperform NBMB in this scenario.Figure The contrasts between NBMB and GMM methods are consistent across the scenarios with different number of genes . Performance of these approaches also improved for certain scenarios when the number of genes changes from 1000 to 5000. However, this improvement does not imply that including more genes will definitely lead to better performance in the clustering of real data. If the percentage of DE genes decreases while still preserving the number of genes included in the clustering analysis, the clustering performance may decrease. Summary statistics for ARI per simulated dataset across different scenarios were listed in Additional file In the Obesity and type 2 diabetes (T2D) study each subject was sampled at 4 time points: 0, 0.5, 1, and 2 h after insulin stimulation. Among the 24 subjects there are 6 normal glucose tolerant, 6 obese, 6 type 2 diabetic, and 6 obese and type 2 diabetic. We performed clustering by NBMB and Gaussian model-based methods on library size normalized RNA-seq counts of 96 samples and compared the clustering results to the known disease/phenotype groups. The comparison between the clustering methods is presented in Table The heatmaps in Fig. Research by TCGA  and TothThe heatmaps in Fig. It should be noted that the genes and methods used in the development of the CLOVAR subtypes are not exactly the same as those used in the application of NBMB and GMM methods. The CLOVAR subtypes are determined by the top 100 genes of the signatures in  that werIn this paper, we presented a method and algorithm using a Negative Binomial mixture model (NBMB) to complete unsupervised model-based clustering of transcriptomic data from high-throughput sequencing technologies. In doing so, we assess the NBMB method using both an extensive simulation study and transcriptomic studies of type 2 diabetes and ovarian cancer. In general, we found that the NBMB outperforms Gaussian mixture model (GMM) applied to transformed data, particular when the same size was small or when difference in cluster means were small. In this study we choose to compare NBMB to other model-based approaches, as model-based methods are known to outperform nonparametric or heuristic-based methods when the correct model is specified , 39, espNB.MClust. The computation efficiency of NBMB, as implemented in R package NB.MClust, and GMM, as implemented in R package mclust with the function “Mclust”, was assessed by clustering the TCGA OV dataset with 295 samples and 1000 genes. User/system time (in seconds) for clustering with K = 3 for NB.MClust was 29.01/0.03, while Mclust had a time of 0.97/0.19. For the case when clustering was run for K selected from K = 3, 4, 5, the time (in seconds) for NB.MClust was 88.06/0.01, while Mclust had a time of 3.31/0.33. It should be noted that the package mclust was being well-developed and upgraded in computation for the past decades [The strength of this research include the following: the ability to model the over-dispersion in transcriptomic data through the use of a Negative Binomial distribution for development of the mixture modeling framework for model-based clustering; the completion of an extensive simulation study to assess the performance of NBMB; the application of NBMB to two existing transcriptomic studies; and the ability for researchers to apply this method using the R package  decades , 42.N = 50 the suggested range is K = 2, 3, 4 as the mean ARI by each method is below 0.5 for all K > 4 scenarios in Fig. N ≥ 150, it is necessary to include K = 5, 6 into the expected range of K in the application of NBMB or GMM with log transform. Hence, we used this guidance to set the range to select the optimal K in the clustering analysis of two transcriptomic studies. Furthermore, NBMB is superior to the other methods when a transcriptomic study contains no more than 3 subgroups. However, if there are K ≥ 4 subgroups in the T2D small-cohort study, the clusters given by each model-based method may deviate from the true subgroup assignment due to the limited sample size (N = 96). In contrast, if we expect K ≥ 4 subtypes in the TCGA OV large-scale study (N = 295), GMM on log-transformed data may be the optimal method to discover latent subtypes and the result is valid. The heatmap patterns in Fig. In this study, the simulation results present a decrease in performance metric ARI for each method along with an increase in cluster components, as shown in Fig. While this research provides a framework for model-based clustering of samples using a Negative Binomial distribution, future research is needed to extend NBMB in the following ways. First, research is needed to implement a gene selection approach within the NBMB model with a semi-surprised approach or a variable selection / shrinkage approach . SecondlThe NBMB clustering method fully captures the over-dispersion in RNA-seq expression and outperforms Gaussian model-based methods with the goal of clustering samples, particular when the sample size is small or the differences between the clusters (in terms of the mean) is small.Additional file 1:Figure S1. Simulation Results for Zero Shift in Dispersion. Plot of the mean Adjusted Rand Index for 100 simulated datasets in each of the scenarios with zero shift in dispersion parameter. (PNG 246 kb)Additional file 2:Table S1. Summary Statistics of ARI for G = 1000 and Non-zero Shift in Dispersion. Table S2. Summary Statistics of ARI for G = 5000 and Non-zero Shift in Dispersion. Table S3. Summary Statistics of ARI for G = 1000 and Zero Shift in Dispersion. Table S4. Summary Statistics of ARI for G = 5000 and Zero Shift in Dispersion (XLSX 129 kb)


Single-cell RNA sequencing (scRNA-seq) technology provides an effective way to study cell heterogeneity. However, due to the low capture efficiency and stochastic gene expression, scRNA-seq data often contains a high percentage of missing values. It has been showed that the missing rate can reach approximately 30% even after noise reduction. To accurately recover missing values in scRNA-seq data, we need to know where the missing data is; how much data is missing; and what are the values of these data.To solve these three problems, we propose a novel model with a hybrid machine learning method, namely, missing imputation for single-cell RNA-seq (MISC). To solve the first problem, we transformed it to a binary classification problem on the RNA-seq expression matrix. Then, for the second problem, we searched for the intersection of the classification results, zero-inflated model and false negative model results. Finally, we used the regression model to recover the data in the missing elements.We compared the raw data without imputation, the mean-smooth neighbor cell trajectory, MISC on chronic myeloid leukemia data (CML), the primary somatosensory cortex and the hippocampal CA1 region of mouse brain cells. On the CML data, MISC discovered a trajectory branch from the CP-CML to the BC-CML, which provides direct evidence of evolution from CP to BC stem cells. On the mouse brain data, MISC clearly divides the pyramidal CA1 into different branches, and it is direct evidence of pyramidal CA1 in the subpopulations. In the meantime, with MISC, the oligodendrocyte cells became an independent group with an apparent boundary.Our results showed that the MISC model improved the cell type classification and could be instrumental to study cellular heterogeneity. Overall, MISC is a robust missing data imputation model for single-cell RNA-seq data. Single cell genomic analysis has made it possible to understand cellular heterogeneity . AdvanceAlthough single-cell RNA sequencing (scRNA-seq) data analysis provides us an opportunity to study the heterogeneity of cells and the genes that are differentially expressed across biological conditions, it is a challenging process to perform the analysis. With the fast-increase in scRNA-seq data, computational methods need to overcome challenges ranging from handling technical noise to constructing and characterizing cell identities, and to cell lineage analysis through computing high-dimensional sparse matrixes. Therefore, innovative, efficient, robust, and scalable computational analysis methods are essential to this new frontier.Currently, the main obstacle in scRNA-seq data analysis, stems from low capture efficiency and stochastic gene expression, which increases gene dropout events in genome-wide scRNA-seq data. We designate these dropout events as the missing data events of single-cell data. Previous studies indicate that the overall missing rates are consistently high in some single-cell data. For example, in a mouse embryo cell, the missing rate can reach nearly 30%, even after noise reduction  With a hTo solve the key problems in missing value imputation, we proposed a novel model with a data-driven machine learning method, namely, missing imputation on single-cell RNA-seq (MISC). The MISC was designed to address three problems: where the missing data is?; how many pieces of data are missing?; and what their values are?. Its initiation involves modeling the problem to transform the missing data imputation into two machine learning problems for detection and imputation of the missing data events. Then, we proposed a model based on classification and regression methods to solve the aforementioned problems. Finally, we evaluated the missing imputation method on two real datasets for studies of differentiating cells and cell - type detection.η. In which, η can be computed using the rate of classification results and the counts of the test dataset. Finally, to determine their values, we used a regression model to impute the data in the missing elements.There are four modules  in our scRNA-seq missing data discovery flowchart Fig. . First, In the second module, the problem modeling, single-cell missing data was first transformed into a binary classification set. The hypothesis is: if the classifier finds a group of richly expressed genes, whose expression values are equal to zero, than these expressions should be non-zeros and missing values. For the different data, the richly expressed genes can be projected on different gene sets from other genomics data. We used the expression values of these genes as a training set to guide the binary classification model and detect the missing elements in the whole RNA-seq matrix. First, to pursue the latent patterns of the missing data, we constructed a training set based on the matrix transformation of richly expressed genes. All the genes are split into richly expressed gene sets and non-richly expressed gene sets. With these two gene sets, we can construct the richly expressed gene expression matrix as training data and the non-richly expressed gene expression matrix as test data. The positive set is all the gene expression values larger than zero in a single-cell RNA-seq expression matrix and the negative set is all the values equal to zero.x in which X indicates the expression matrix of the richly expressed genes, 0 < i < m, 0 < j < n, where m indicates the number of genes, and n is the number of cells. In generated training set, each element x is a sample and the its features j’ are j’ ≠ j, 0 < j’ < n. The missing data value yi,j of a typical gene j in one cell i can be predicted with the gene expression values.F is the machine learning function. Therefore, the training set s has m × n samples, and the feature set f contains n-1 features. In our case, we took the mouse cortex and hippocampus data as an example for the process. The training set has 406 genes (m), 3,005 cells (n), 1,220,030 samples (m x n = 406 × 3005) and 3,004 features. Similarly, the test set contains t × n samples and t is the number of non-richly expressed genes. In the example, the test set has 19,566 genes (m), 3,005 cells (n), 58,795,830 samples and 3,004 features.Suppose an element O(mn2). Considering the fast development of the single cell experiments, which can perform up to tens of thousands of single cells [F to discover the missing data, which is of much efficiency for the large data set. The method involves solving the following optimization problem:s is the sample, y is the class label for the classification and the expression value for regression, w is the weight vector and w∈Rn, C is the penalty factor, C > 0. We adopted two popular binary linear classifiers, named Logistic Regression (LR) and a Support Vector Machine (SVM) with a linear kernel. LR with L2-regularization employs the following unconstrained optimization function.In the third module, with the aforementioned problem modeling, it can be seen that the computational complexity reaches le cells , we emplThe correspondence dual form isThen, the problem can be solved with a trust region Newton method  or dual The dual form isThen, the problem can be solved with a coordinate descent algorithm .c, the reads rc were modeled as a mixture of “drop-out” data with Poisson (λ0) and “amplified” components with NB(e), where e is the expected expression magnitude, and the background read frequency for dropout was λ0 = 0.1. To fit the mixture model, a subset of genes should be selected. First, given a subpopulation of cells, all the pairs of individual cells  were analyzed with the following model.To further validate the missing data and their percentage, we employed our linear classification model, the zero-inflated model  and falsm = log(ri) + log(rj)) and an expectation–maximization algorithm was used to fit the model. The genes that were assigned to the “amplified” components could be noted, and the set of genes appearing in the amplified components in at least 20% of all the comparisons of the same subpopulation of cells were used to fit the model.Then, a multinomial logistic regression (the mixing parameter Fc(μ) whose odds quantify the cell’s technical detection efficiency [μ* is conditioned to be detected and 1- Fc(μ*) is the missing probability of this gene in cell c.False-negative curves employ housekeeping genes to fit a logistic regression function ficiency  In a givThe differences among the three methods for missing data detection are the training set (subset of genes) and training (fitting) method. First, all three methods need a subset of genes to train or fit the model. From the biology view, the false negative model and large linear classification use the richly expressed genes. However, from the statistical view, the zero-inflated model uses a mixture model of Poisson and negative binomial (NB) to select a subset of genes. Moreover, both the zero-inflated model and false negative model employ logistic regression to fit a model for each cell RNA-seq expression value. The large linear classification uses a linear model instead of a logistic function, which is efficient for big data. Therefore, all three methods try to detect the missing data from different views, which satisfied the heterogenous rule of ensemble learning.After obtaining the ensemble learning and obtaining the missing positions in the RNA-seq expression matrix, we employed a linear regression model to recover the missing values. In our experiments, we employed the support vector regression (SVR) model with a linear kernel. The training set is the same as the classification task; however, the label of the training samples using normalized RNA-seq expression values, such as reads per kilobase per million (RPKM). For the regression optimization function, we employed three L2-regularized methods, which are the dual problem solutions of L1-loss support vector regression, the primal problem solution and the dual problem solution of the L2-loss support vector regression. The L2-regularized SVR is modeled using the following optimization problems:p = 1 indicates the L1 loss and p = 2 is the L2 loss, and ε ≥ 0 is the sensitiveness of the loss. The dual forms of the problem are:where e is the vector of all ones, Q’ = Q + D, Qij = xiTxj, D is the diagonal matrix and p = 1, Dii = 0; p = 2, Dii = 1/2C; 0 ≤ αi+,αi+ ≤ U, i = 1,…,l, U=C when p = 1; U = ∞, and when p = 2. We use LIBLINEAR tool to solve this problem [where  problem .γ of the previous and following cells (γ = 3 in our experiments).In addition, based on the classification results (which show the missing positions in RNA-seq expression matrix), a mean-smooth curve with the neighbor cell method on the cell trajectories is also proposed to make a comparison with the MISC. This method recovers the missing values with the expressions of the For the fourth module, we employed the trajectory analysis and subpopulation analysis to directly show the effectiveness of our MISC method.Two real scRNA-seq datasets were used to verify the effectiveness of our model. One is chronic myeloid leukemia (CML) data (Gene Expression Omnibus: GSE76312)   cutoff of 0.05 and an absolute log fold change cutoff of 1. The training dataset of our MISC machine learning model is based on the richly expressed gene set, which employs human housekeeping genes from reference [C = 2) due to better accuracy. The missing rate threshold η = 0.35 for the false negative curve . The final missing rate of CML data (the overlap of the missing data sets between MISC and FNC method) is 13.6%. After several parameter selection experiments, we selected L2-loss support vector regression with primal problem solution (parameter C = 0.125) due to its lowest mean-square error among the three regression methods.The CML data includes 2,287 stem cells throughout the disease course and 23,384 genes. To analyze the heterogeneity of the stem cells from normal HSCs, we selected 1,102 stem cells without tyrosine kinase inhibitor treatments. Then, the eference  for CML For single-cell trajectory analysis, five different types of stem cell chronic-phase CMLs (CP-CML), normal hematopoietic stem cells (HSCs), pre-BC samples taken from the patients who were presented in CP (pre-BC) 12 months and 3 months before transformation to myeloid and lymphoid blast crisis (BC), blast crisis CML (BC-CML), K562 human erythroleukemic cell lines derived from a patient in CML blast crisis appear in branches in trajectories during cell development in Fig. t-SNE analysis, all five different types of stem cells are visualized in Fig. t-SNE visualization on the single-cell RNA-seq data using MISC imputation  due to better accuracy. The missing rate threshold η = 0.397 for the false negative curve (FNC) and zero-inflated model (ZIM). The final missing rate of the primary somatosensory cortex and hippocampal CA1 region of mouse data is 23.4%  due to its lowest mean-square error among the three regression methods.For the primary somatosensory cortex and hippocampal CA1 region, the single cell data contains 19,972 genes, including 406 housekeeping genes (using the same list in reference ) and 3,0For single-cell trajectory analysis, seven different types of cells, astrocytes-ependymal, interneurons, oligodendrocytes, pyramidal SS, endothelial-mural, microglia and pyramidal CA1, appeared in branches in trajectories in Fig. The complex brain cognitive functions, such as social behaviors and sensorimotor integration, rely on a diverse set of differentiated cells . TherefoThe dropout events are abundant in the single-cell sequencing data , 22. TheThe linear classification was used to achieve efficiency in computational time in our method. A more sophisticated model might provide better performance at the cost of computational expense. Hence, the method coupling parallel computing and advanced modeling could help to enhance the efficiency and accuracy of single cell data imputation. Our missing position refinement via ensemble learning may potentially exclude true missing positions. With a better model, we can also address this limitation.t-SNE on MISC imputed data proves the evolution from CP to BC stem cells as our trajectory analysis and presents more compact clusters. On the primary somatosensory cortex and the hippocampal CA1 region of mouse brain cells, it clearly divides the pyramidal CA1 into different branches, it is a direct evidence of pyramidal CA1 has subpopulations. In addition through the use of MISC, oligodendrocyte cells became an independent entity with an apparent boundary. Furthermore, for filtered CML data, the MISC model can present a clear trajectory and cell type classification. For the scRNA-seq data with a large number of genes,, MISC can also help us study the cellular heterogeneity. All this indicates that MISC is a robust missing data imputation model for single-cell RNA-seq data.Single-cell RNA-seq expression profiling offers a static snapshot of the gene expression, provides estimates of cell heterogeneity and rare cell type detection. Through successfully solving the three problems of missing data, the proposed model MISC can effectively recover the missing values in the scRNA-seq data. Regarding the chronic myeloid leukemia data, MISC discovered a trajectory branch from CP-CML to BC-CML, which provides direct evidence of evolution from CP to BC stem cells. Meanwhile,