The characterization of microRNAs (miRNA) in recent years is an important advance in the field of gene regulation. To this end, several approaches for miRNA expression analysis and various bioinformatics tools have been developed over the last few years. It is a common practice to analyze miRNA PCR Array data using the commercially available software, mostly due to its convenience and ease-of-use.https://bio.tools/mirkit. Overall, miRkit offers the full analysis from the raw RT-PCR data to functional analysis of targeted genes, and specifically designed to support the popular miScript miRNA PCR Array (Qiagen) technology.In this work we present miRkit, an open source framework written in R, that allows for the comprehensive analysis of RT-PCR data, from the processing of raw data to a functional analysis of the produced results. The main goal of the proposed tool is to provide an assessment of the samples’ quality, perform data normalization by endogenous and exogenous miRNAs, and facilitate differential and functional enrichment analysis. The tool offers fast execution times with low memory usage, and is freely available under a ΜΙΤ license from  MicroRNAs are small non-coding RNA molecules with a critical role in gene expression regulation . They arwww.miRBase.org).Various approaches have been used to profile the expression of miRNAs  such as Raw RT-PCR data are typically analyzed using the manufacturer’s software. In fact, several open-source packages analyze RT-PCR data with the traditional Ct (threshold cycle) quantification approach , ignorinCurrently, and going beyond miRNA data, there are plenty of options for differential expression analysis in which the user can employ a larger range of more sophisticated methods, ending up with values for logFC and adj-pvalue. To assess the potential benefits of analyzing this data with an open source tool, we developed miRkit, a framework written in R, specific for miScript miRNA PCR Array (Qiagen) technology. The proposed toolset offers the whole analysis of the raw RT-PCR data completely automated, including quality control of the samples, normalization of endogenous and exogenous controls, differential expression analysis and functional analysis of targeted genes. Finally, the package has fast execution time and uses very low memory.Count table: This is the main data file, containing the different samples on columns and the measurement of each well on the rows. The proposed tool is applicable on miScript miRNA PCR Array (Qiagen) which contains 384 wells and examines 372 miRNAs, 12 controls. Specifically, each well of 372/384 contains a miScript Primer Assay for a miRNome or pathway/disease/functionally-related mature RNA. Moreover, 2 wells contain replicate C. elegans miR-39 miScript Primer Assays and can be used as an alternative normalizer for array data (Ce), 6 wells contain an assay for a different snoRNA/snRNA that can be used as a normalization control for the array data. Finally, there are two wells which contain replicate miRTC Primer Assays (RTC) and two wells that contain positive PCR controls (PPC).Metadata: This file includes a list of sample IDs and the corresponding group e.g. normal/tumorAnnotation of miRNAs well: A file that links the information of the well with the examined miRNA.The main R script of the workflow reads the input data from three distinct files stored in a folder:QC and normalization.the maximum percentage of not detected or not available values (NA’s) in each column. This quality threshold is defined by the user. If a sample fails the comparison then it is excluded from the rest of the analysis.the ratio between reverse transcription control (RTC) assay, which detects an artificial RNA template, and positive PCR controls (PPC), which monitor for PCR inhibitors, is calculated and a standard threshold is used to validate the reverse transcription efficiency which descripted by Qiagen. Specifically, if the ratio is < 5 the sample is passed this quality step.The quality control process examines two different aspects:The data normalization module includes the option of endogenous and exogenous miRNA approach. Normalization of miRNAs using the endogenous controls corrects for factors that could influence the quantification such as different quantities of input RNA, RNA degradation, presence of inhibitors, errors in sample handling. Exogenous controls are typically used on difficult samples such as plasma/serum or other biofluids. Many exogenous controls are not present in humans so it is a good exogenous control for human samples. MiScript PCR Controls are primers designed to quantify a panel of 5 snoRNAs  and the snRNA RNU6B (RNU6-2) as endogenous controls and cel-miR-39 as exogenous.The output of this step is the normalized data matrix that includes the samples which passed the NA’s criterion. Additionally, a visualization option is available, which allows to generate figures that are automatically stored within the analysis folder, and include an upset plot for the NA’s distribution and boxplots with counts before and after the normalization.Differential analysisThis module is performed using the limma package in R . The outFunctional analysisenrichR package [The downstream analysis links the differentially expressed miRNAs with the regulated genes using the multiMIR package, which includes several databases such as mirtarbase , tarbase package . FinallyAt the end of the entire process miRkit produces a list of tables containing all significant events along with the corresponding plots, separated into different folders. Additionally, a report file is automatically exported. The report contains information of the particular execution process, including the user-defined criteria, the rationale for the excluded samples, the overall time required for execution and the total memory usage.The framework is implemented into three distinct phases, as shown in Fig. https://bio.tools/mirkit) as well as the sample input and the output presented above.The miRkit was applied on artificial data provided by Qiagen. The selected NAs percentage threshold was set to 10% and all samples passed the control of NAs and RTC. The data normalization step was performed using the endogenous miRNAs. The total execution time was 18 min and 51.02 s  perform quality control of the samples and data normalization, (ii) identify significant differences on the expression profiles of miRNAs, and (iii) link the significant miRNAs with the targeted genes and biological processes. In each step of the process, the tool produces also the relevant visual representations of the results.Compared to the traditional commercial software for analyzing RT-PCR data, miRkit aims to become fully aligned to the FAIR principles  for Research Software . With thAlthough the miRkit is focusing on one platform technology, there are many examples of commercial array platforms for which researchers developed tools in R in order to analyze their data with open source tools that are more automated and provide a better workflow for complete analysis. Using the array of 450 K Illumina as an example, there are plenty of options in R such as minfi, RnBeads, shinyMethyl, etc. Having alternative options to analyze the data, especially through workflows supporting completely automated processes from raw data to the association with genes and pathways like the proposed tool, is a critical element for the scientific community.It is freely available on GitHub and is accompanied by detailed documentation and examples, in order to facilitate the reproducibility of the presented results. Our method provides a new perspective towards analyzing RT-PCR data. Also, it supports efficient data discovery using the gold standard approach of limma analysis and linking the information with publicly available databases to extract the biological meaning.miRkit focuses on the data from miScript miRNA PCR Array (Qiagen) technology. Other miRNAs technologies which produce count tables i.e. samples on the columns and miRNA of interest on the row, could be possibly analyzed using our tool by following the format of our input files. To this end, a list of conversion scripts will be developed so that a user can use them to convert outputs from other PCR array technologies than Qiagen, as an input to miRkit. This limitation may be addressed in future versions of the miRkit.


Cell-free DNA (cfDNA) serves as a footprint of the nucleosome occupancy status of transcription start sites (TSSs), and has been subject to wide development for use in noninvasive health monitoring and disease detection. However, the requirement for high sequencing depth limits its clinical use. Here, we introduce a deep-learning pipeline designed for TSS coverage profiles generated from shallow cfDNA sequencing called the Autoencoder of cfDNA TSS (AECT) coverage profile. AECT outperformed existing single-cell sequencing imputation algorithms in terms of improvements to TSS coverage accuracy and the capture of latent biological features that distinguish sex or tumor status. We built classifiers for the detection of breast and rectal cancer using AECT-imputed shallow sequencing data, and their performance was close to that achieved by high-depth sequencing, suggesting that AECT could provide a broadly applicable noninvasive screening approach with high accuracy and at a moderate cost. Plasma cell-free DNA (cfDNA) is an intensively investigated biomarker that has been widely used for noninvasive cancer evaluation and prenatal testing –4. BecauComputational approaches have been designed to improve the measurement of genomic or transcriptomic spectra generated from low-coverage sequencing data, particularly from single-cell RNA-seq data and single-cell ATAC-seq data –15. ThesAlthough existing methods may fail to account properly for TSS coverage data, they highlight the potential to capture accurate TSS coverage profiles and extract data structures from shallow sequencing data. One of the most popular such methods, using autoencoder frameworks, may apply to TSS coverage data due to its flexibility and scalability , 14. An Here, we introduce the Autoencoder of cfDNA TSS (AECT) coverage profile, a method of denoising TSS coverage profiles generated by shallow cfDNA sequencing. A set of pre-processing steps for cfDNA sequencing data, including GC bias adjustment and copy number normalization, are also integrated. The effectiveness of AECT was validated using multiple datasets. Outperforming other tools designed for single-cell sequencing data, AECT generated comparable accuracy of TSS coverage profiles as high-depth cfDNA sequencing data, and it was sufficiently powerful to uncover the latent biological features in shallow cfDNA sequencing generated from healthy donors and tumor patients. In sum, AECT greatly improves the performance of shallow sequencing-based cancer detection and sheds a light on the clinical use of cfDNA sequencing at an acceptable cost.AECT is a deep neural network autoencoder, implemented with the Keras framework and TensorFlow in the backend. It uses TSS profiles as its input layer and predicts imputed profiles as the output layer. By default, five fully connected hidden layers with 128, 64, 32, 64, and 128 neurons are used to compress and reconstruct the data using the MSE loss function . The recA few related studies review to our study. A set of deep learning-based denoising model for single cell sequencing data, including DCA , DeepImpThe samples used in the study have been described in previous studies , 17. TheSingle-end sequencing data were generated from the Ion Proton platform . After low-quality sequencing reads were removed, high-quality reads were aligned to the hg19 human reference genome using TMAP (v5.4), and PCR duplication was removed. Because AECT uses the read frequencies of each TSS region, AECT users could also use hg38 as reference genome. GC-bias correction and copy number change normalization were performed with the deepTools correctGCBias algorithm , 19, andp-values were calculated using the Wilcoxon rank-sum test and then were adjusted to the FDR, using the Holm procedure. TSSs with FDR < 0.1 were selected as altered TSSs for downstream functional analyses, and gene-set annotation and functional enrichment analysis (on GO database) were performed using Metascape with default parameters , and lambda values were determined by 10-fold cross-validation. For each model, 1,000-times bootstrapping was used to test the robustness of the candidate genes chosen by the model. To reduce overfitting, we constructed the final prediction model using 100 candidate genes that were most often seen in the bootstrapping. The performance of the classifiers was evaluated on the training cohort and validation cohorts using receiver operating characteristics generated by the pROC R package .For ease of comparison, we used the latest version of MAGIC (v1.5.5) , DCA v0, DeepImpGC content bias influences the number of reads that are mapped to a genomic region, confounds the quantification of TSS coverage profiles, and is a major cause of batch effects in cfDNA sequencing data . To addrWe developed AECT to impute precise TSS coverage . The AECAs a proof-of-principle and to explore the properties of our approach, we applied AECT to mimic low cfDNA-sequencing data. Five high-depth sequenced samples (>400 M reads) , includiWe also used four representative algorithms designed for single-cell sequencing imputation, namely, MAGIC , DeepImpTo test whether AECT captured common features in the real data, we sequenced the cfDNA of 159 healthy donors with an average of 8.6 M reads (range 7.2 to 10.6 M). PCA was performed on GC-adjusted TSS profiles, and samples were into two groups by donor sex to reflect the differences in TSS profiles between males and females . Howeverp < 2.2 × 10−16, Wilcoxon rank-sum paired test), which implies a reconstruction of gene–gene relationships (r| > 0.8), suggesting an over-adjustment in the TSS profiles  and increased the detection numbers of breast cancer-associated TSSs with AUROC > 0.7 . Another 45 healthy women donors were also included in the analyses . To reduOC > 0.7 , while mBRCA1, which did not change significantly between breast cancer and non-cancer samples in the raw data , showed significantly lower TSS coverage in breast cancer patients after imputation , which is consistent with the high expression level of BRCA1 in breast cancer tissues  showed lower TSS coverage in Her-2(+) subtype and luminal B subtype with Her-2 expression  and identified more altered TSSs  and NODE .Publicly available datasets were analyzed in this study. This data can be found here: SRA , Medical Scientific Research Foundation of Guangdong Province of China (B2017006), and China Postdoctoral Science Foundation funded project (2019M662998).KL was employed by the company Guangzhou XGene Co., Ltd., and G-JO, R-TW, and SX were employed by the company Guangzhou Darui Biotechnology Co., Ltd. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


The identification of efficient and sensitive biomarkers for non-invasive tests is one of the major challenges in cancer diagnosis. To address this challenge, metabolomics is widely applied for identifying biomarkers that detect abnormal changes in cancer patients. Canine mammary tumors exhibit physiological characteristics identical to those in human breast cancer and serve as a useful animal model to conduct breast cancer research. Here, we aimed to provide a reliable large-scale metabolite dataset collected from dogs with mammary tumors, using proton nuclear magnetic resonance spectroscopy. We identified 55 metabolites in urine samples from 20 benign, 87 malignant, and 49 healthy control subjects. This dataset provides details of mammary tumor-specific metabolites in dogs and insights into cancer-specific metabolic alterations that share similar molecular characteristics.  Although breast cancer has a higher survival rate than other types of cancer, early and accurate diagnosis is essential. Among various diagnostic tools for breast cancer, mammography has remained the gold standard. However, it poses the risk of exposure to radioactive material and involves high false-positive rates4, especially in benign stages during which non-cancerous cells show characteristics similar to those in malignant cells, obscuring diagnosis. To overcome these constraints, ongoing research has sought to identify biomarkers reflective of the physiological states of the disease, and of use in differentiating between normal and cancer specimens5. Such biomarkers are expected to facilitate steady monitoring of cancer and provide a better understanding of cancer phenomena. However, the identification of efficient and sensitive biomarkers requires a sufficiently large number of samples.According to the American Cancer Society, breast cancer ranks first in the incidence rate in women6. Canine mammary tumors (CMTs) are the second most common neoplasias in dogs, and they exhibit similar clinical and molecular characteristics to breast cancer, including onset age, course of the disease, and estrogen dependency8. In this regard, we recently elucidated cross-species oncogenic signatures obtained from pet dogs as reference points for comparative analysis with human cancers for developing novel diagnostic and therapeutic technologies. As confirmed in a previous study, there is a striking resemblance of genomic characteristics, including frequent PIK3CA mutations (43.1%), aberrations of the PI3K-Akt pathway (61.7%), and key genes involved in cancer initiation and progression10. Here, we aimed to provide reliable large-scale proton nuclear magnetic resonance (1H NMR) spectroscopy data for cancer research on dogs.Owing to the lack of human samples, and the prevalence of ethical issues, animal models are commonly used to confirm drug efficacy and stability before clinical trials in humansIn this study, we obtained a large number of urine samples from dogs diagnosed with benign and malignant tumors, and also from healthy controls for detailed analysis of tumors and their effects. We then utilized NMR to investigate metabolites in the different grades of samples. The overall process described in this study is summarized in Fig. The urinary metabolite dataset from a large-scale canine cohort provides potential advantages for further research. To the best of our knowledge, this is the first study to describe a metabolomics dataset containing a large number of urine samples from dogs with mammary tumors and healthy controls. Despite the prevailing differences between breeds of dogs included in the dataset, their diversity will help cover the general characteristics of CMTs. The inclusion of both benign samples and highly divided tumor grades enables the efficient identification of biomarkers for the early detection of cancer. As mentioned earlier, the canine model is a good animal model for human disease, as it can be used for cross-species research that applies to the discovery of human breast cancer biomarkers.Urine samples were collected from 117 dogs with CMTs and 49 healthy dogs. Urine specimens from the dogs with CMTs obtained as a part of routine screening tests  before surgery under fasting conditions from client-owned dogs via private veterinary clinics with informed consent. Only available cases diagnosed with CMT based on histological assessment were included in the study. Dogs with CMT received surgical treatment without adjuvant chemotherapy. Healthy control urine samples were obtained from surplus urine samples collected from client-owned healthy dogs or a healthy research beagle during a regular medical check-up. Only dogs with no history or clinical signs of tumors or any other disease were included as controls. Urine samples were collected by free catch or cystocentesis (surplus samples from the regular test). Residual samples were refrigerated at 4 °C for no longer than 24 h, frozen at −20 °C, transferred to −80 °C, and stored for up to 2 y before analysis. Stored residual samples were used in the current study and the procedures were conducted in accordance with the guideline of the Institutional Animal Care and Use Committee of Konkuk University (KU18168).10. Malignant samples were separated into three grades: grade 1 (early-stage cancer) was highly prevalent in 47 samples, whereas grades 2 and 3 had the same number of samples (20 each). Both benign and malignant subjects frequently showed positive expression of estrogen receptor (ER) and human epidermal growth factor receptor 2 (HER2). Information about the breeds included in each category is provided in Table All characteristics of the samples used in this study are summarized in Table 1H NMR-based metabolic profiling was performed using one urine sample from each dog. The experimental methods used for NMR analysis have been published elsewhere11. After thawing at 4 °C overnight, proteins in urine were removed with cut-off membrane filters  by centrifugation at 18,213 × g and 4 °C for 20 min. Next, 330 μL of filtered urine was mixed with 350 μL of 0.2 M deuterated sodium phosphate buffer (pH 7.0). Deuterated sodium phosphate buffer, NMR solvent, was prepared by mixing 0.2 M sodium phosphate monobasic  in deuterium oxide and 0.2 M sodium phosphate dibasic (Sigma-Aldrich) in deuterium oxide. The pH of the NMR solvent was then adjusted to 7.0 ± 0.1. After adjusting the pH to 7.0 ± 0.1 for each urine sample, a total of 630 μL of urine sample was mixed with 70 μL of deuterium oxide containing 5 mM TSP-d4 (3-(trimethylsilyl) propionic 2,2,3,3-d4 acid sodium salt (98% atom %). A total of 17.23 mg of TSP-d4 was dissolved in 20 mL of deuterium oxide to prepare 5 mM of TSP-d4. Finally, 600 μL of each sample was transferred into a disposable 5-mm NMR tube . All NMR samples were handled before or after the NMR measurement on Sample Jet (Bruker BioSpin), which is an automated device. Prior to NMR measurement, urine samples were maintained at 4 °C in the Sample Jet and equilibrated at 25 °C for 3 min.To quantify urinary metabolites from dogs with CMTs and healthy controls, 1H) experiments were carried out at 25 °C on a Bruker Avance III HD 800-MHz NMR spectrometer (Bruker BioSpin) equipped with a Bruker 5-mm CPTCI Z-GRD probe. The NOESY PRESAT pulse sequence with water presaturation was used to acquire 1H NMR spectra collected into 65,536 data points, with 64 transients, a relaxation delay of 2.0 s, a spectral width of 20 ppm, and an acquisition time of 2.0 s. A 0.3-Hz line broadening function was applied to all spectra before Fourier transformation.All one-dimensional proton  and AMIX  software, respectively. Afterward, urinary metabolites were identified and quantified using Chenomx NMR Suite . TSP-d4 was used as a chemical shift reference with a value of −0.016 ppm. Fifty-six metabolites were identified and confirmed using the 800 MHz library of Chenomx, 2D NMR spectra, and spiking experiments. A representative 13 after excluding metabolites based on the modified 80% rule14. To apply the most proper imputation method for metabolomics data, we tested four types of imputation algorithms: an iterative Gibbs samplerbased left-censored missing value imputation approach (GSimp), quantile regression imputation of left-censored data (QRILC), half minimum (HM), and zero imputation method. To validate the performance of each imputation method, we extracted the complete dataset that eliminated all missing values from the original data and randomly generated approximately 5–40% of missing values in the dataset. We then selected the method with the lowest normalized root mean squared error and the Procrustes sum of squared error which represent how well the methods could recover the missing values against the complete data with the minimum differences.All missing values in raw data were imputed by three groups and evaluated using MetImp 1.22 transformation was performed to make skewed distributions more symmetric15 and principal component analysis (PCA) was used in R.The CMT samples were separated into benign and malignant specimens for comparison with normal samples. To visualize the distribution of the samples, log16 MTBLS2550 (https://www.ebi.ac.uk/metabolights/MTBLS255017).The dataset supporting the conclusions of this article is available in the MetaboLights repositoryAll samples were rearranged in random order and most experiments such as sample preparation, data acquisition, and data processing were accomplished in this order to avoid any artificial variations. During sample preparation, all samples were kept on ice to avoid metabolite degradation. The pH of the samples was finely adjusted to minimize variation in chemical shift among samples (7.0 ± 0.1). After sample preparation, all samples were kept at 4 °C in the autosampler rack before NMR data acquisition.After spectral processing, fumarate and citrate containing more than 20% of missing values in each group were eliminated. Among the four imputation methods, QRILC showed robust performance in the different proportions of missing values Fig. . As a reTo identify the outliers and visualize the distribution of the dataset, we drew the dendrogram and PCA score plot colored by groups. In Fig. Furthermore, to validate the potential effect of minor breeds, and to check whether they will affect further statistical analysis by contributing to the noise, we checked the breeds of the outliers and colored the samples according to the breeds. The breeds of three outliers in the malignant group and a sample in the healthy control group were maltese, schnauzer, poodle, and chihuahua, respectively, which were not unique breeds in the dataset. In addition, various breeds in the dataset were spread widely Fig. . Compreh1H NMR. Although we excluded samples that contained concurrent tumors in other organs to prevent the impurity of metabolites, the information about these samples can be found in MetaboLights, along with the clinical notes.In this study, we described the preprocessed metabolite dataset extracted from CMTs using 18. Even though this dataset is not directly related to metabolic dysregulation diseases, researchers can explore the raw 1H NMR profiling data without creatinine normalization for flexible usage.The metabolites in the urine samples from the dogs with mammary tumors and healthy controls were normalized with creatinine which was considered as a standard method. However, there are some cases where creatinine normalization is not suitable, such as when the samples were not ensured the required creatinine clearance due to metabolic dysregulation8. However, their average age is three times more than that of healthy controls. As previously mentioned, healthy control samples were collected from client-owned healthy dogs or a healthy research beagle, making it hard to match the age of the control group with the test groups. To deal with feature imbalances, researchers should perform the sampling while considering the ratio of other factors.Client-owned dogs with CMTs were housed in a family environment, and not in a controlled environment, which made it difficult to control for all other possible environmental factors. However, results from client-owned dogs with spontaneous cancer might more closely reflect reality. Besides, there were differences in age and breed between the sample groups as summarized in Tables


To the Editor:Numerous prognostic signatures to bladder cancer (BCa) have been reported, but many of them limited to either nonmuscle‐invasive (NMIBC) or muscle‐invasive BCa (MIBC), and inherent technical biases across platforms impeded clinical application. As cancer immunity plays a critical role in tumor progression,https://github.com/xlucpu/BCaller), which acts as an independent prognostic factor after adjusting other clinicopathological features in metatraining, metatesting, validation 1 and 2 datasets . Next, cohort‐specific HRisk groups were more likely to be basal/squamous subtype, whereas most samples within LRisk were predicted as luminal‐papillary , while seven (77.8%) HRisk groups were susceptible to paclitaxel  LRisk groups were highly sensitive to methotrexate  was used to calculate an immune‐related genes pair index (IRGPI) from single‐sample perspective using transcriptome profiles for bladder cancer.Raw data for this study were generated at corresponding archives. Derived data supporting the findings are available from the corresponding author (Fangrong Yan) upon reasonable request. An R package “Supporting InformationClick here for additional data file.


Acute myeloid leukemia (AML) is the second most frequent leukemia entity in children and adolescents, and definitely the most aggressive variant. Multiparameter flow-cytometry is one of the methodologies most useful to monitor the number of remaining leukemic cells in bone marrow  in AML patients, because it is widely available and applicable to most patients. However, AML flow cytometry data show very complex patterns and identifying leukemic cells in the data is subjective, time-consuming and requires experienced operators who are not available world-wide. In this paper, we approach automatic assessment of AML flow cytometry samples with a novel semi-supervised machine learning model, leveraging implicit expert knowledge stored in a collection of manually assessed samples. Because AML data exhibit a high degree of variability in the patterns of blast cell populations that is difficult to model, the model detects anomalies starting from the appearance of normal cell populations.Leukemia is the most frequent malignancy in children and adolescents, with acute lymphoblastic leukemia (ALL) and acute myeloid leukemia (AML) as the most common subtypes. Minimal residual disease (MRD) measured by flow cytometry (FCM) has proven to be a strong prognostic factor in ALL as well as in AML. Machine learning techniques have been emerging in the field of automated MRD quantification with the objective of superseding subjective and time-consuming manual analysis of FCM-MRD data. In contrast to ALL, where supervised multi-class classification methods have been successfully deployed for MRD detection, AML poses new challenges: AML is rarer (with fewer available training data) than ALL and much more heterogeneous in its immunophenotypic appearance, where one-class classification  methods seem more suitable. In this work, a new semi-supervised approach based on the UMAP algorithm for MRD detection utilizing only labels of blast free FCM samples is presented. The method is tested on a newly gathered set of AML FCM samples and results are compared to state-of-the-art methods. We reach a median  Leukemia accounts for one-third of malignancy in children and adolescents, resulting in the most frequent childhood cancer <18 years) [8 years . ThroughA well established technique for MRD assessment is the detection of leukemia-associated immunophenotypes by multi-parameter flow cytometry (FCM-MRD) ,11.The amount of cells (events) per FCM-MRD sample varies from stencies . If cellIn the background of the technological advancements of flow cytometers, capable of producing FCM data with more and more parameters resulting in high dimensional data spaces with complex distributions of events, numerous machine learning (ML) approaches have been developed to automate FCM data analysis, while manual gating is restricted to 3 features that can be visualized at once, automated gating can utilize the full multidimensional parameter space. Holistic approaches that take a whole sample as input are able to capture the spatial relation of cell populations to each other within the sample. These methods have been shown to be superior over classification methods learning fixed decision boundaries applicable to different samples ,14. A hiHowever, with AML data there are additional challenges to overcome. AML is less common than ALL, which limits the use of supervised methods due to the scarcity of training data. Moreover, blast populations are much more heterogeneous than in ALL see . In addiIn this paper, a novel one-class classification approach based on Uniform Manifold Approximation and Projection (UMAP)  is preseThe Set-Transformer model , which ifound in ).UMAP has proven to find and preserve meaningful clusters in cell data while embedding the data in a lower dimensional space, where the multidimensional information is incorporated ,19,20. TIn this work, a semi-supervised anomaly detection method for automated AML MRD quantification is presented using labels of healthy cell populations in blast-free samples only. Its main building blocks are UMAP and Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN) . Our metIn A randomly selected subset of events coming from multiple MRD free control samples is mixed to the input FCM sample, which is to be assessed. The result is a mixed set of events.The UMAP embedding of the mixed set is created. In this step, possible differences in the appearance of healthy cell populations in the control samples and the input sample are reduced by exploiting UMAP properties as explained in Clusters are identified in the low-dimensional UMAP representation with HDBSCAN.The clusters formed from cancer cells should not contain any or only a few events from control samples, as control samples do not contain blasts. Hence, clusters with a very low amount of control-events are most likely blast clusters. An empirical evaluation on hold-out samples led to a threshold of The method proposed is able to predict blasts in AML FCM data by using blast free samples (control samples) only. Blast detection is a binary classification task: each event of a sample is assigned either the label blast or non-blast. The presented classification pipeline is divided into four steps  from 10 patients with positive MRD levels from different timepoints of therapy. For a full listing of timepoints of therapy present in the data set see The data were collected between 2016 and 2021 at St. Anna Children’s Cancer Research Institute (CCRI) in Vienna using a Navios  flow cytometer. The samples were stained using customized dried format tubes .To obtain MRD negative control samples, we stained bone marrow (BM) samples of pediatric patients without any history of myeloid malignancy. In addition we used BM samples from pediatric patients with AML at a later stage of therapy and with proven MRD negativity (via flow cytometry and/or molecular methodology (RT-PCR)). The data were collected between 2015 and 2021 at St. Anna Children’s Cancer Research Institute. Samples were stained as indicated above using LAIP and CFU tubes. We used a total of 80 data files . Based on expression of the stem cell marker CD34, blasts can be divided in CD34 positive or CD34 negative blasts. The latter are further defined as events with low side scatter (SSC) properties and as such fall in the so called bermude area on the CD45/SSC bi-dimensional plot. Cells within the bermude area are further categorized in monocytes, granulocytes, proerythrocytes or promyeloctyes, after exlusion of basophiles, plasmacytic dendritic cells, mast cells and plasma cells.The events of all samples in the two data sets were labelled using manual gating by at least two experts to obtain objective and reliable ground-truth data. Whenever available, results were confirmed using an independent molecular methodology (RT-PCR). Kaluza 2.0 software  was used for manual data analysis. All obtained samples are manually gated following the same procedure. First the events are filtered by excluding non-viable cells, debris and doublets. Remaining events (cells) are selected based on expression of CD45 in a gate called N denotes the number of measured events, which can be different for every sample, and the number of columns m denotes the number of features extracted. Each sample is compensated and transformed with a logical transformation in a pre-processing step.The resulting output of a sample analysed by FCM is a matrix For LAIP tubes drop-in markers are sample- (patient-) specific and mostly not present in control samples. Those have to be omitted, as the input vectors need to be of same length and corresponding to the same features for the experiments in this paper. On the contrary, for CFU-tubes drop-ins are fixed and can therefore be included.p, recall r and p, recall r and For the evaluation of experiments precision Four different experiments for MRD quantification in FCM data of AML patients are conducted. First, we apply the supervised method based on the self-attention mechanism of the Set-Transformer. Then the method proposed is tested and the impact of pre-filtering of events as well as variations of control samples is explored. A detailed description of all experiments conducted is given in In this section, the results of the experiments are presented. Average precision (p), average recall (r), average F-Score (avg The approach based on the self-attention mechanism (see ) works wWith only 15 control samples, the proposed classification pipeline, reaches a 1.44 times better performance than the Set-Transformer approach with respect to average rm worse . Figure Due to the transparency of our method samples can be inspected at any stage of the pipeline. We take advantage of this property and examine samples that score Other issues identified are false positive clusters when healthy cell populations are not sufficiently covered by control-events. Since drop-in markers are included for CFU samples, for most samples blasts form a seperated cluster in the UMAP embedding space and are correctly identified as blasts. However, additional false positive clusters arise, which is also noticeable in the low precision As stated in  markers .When applying the proposed method to the distilled test sample overall mean and median ticeable . The perOne way to cope with variations of healthy cell populations is to reduce them beforehand in a pre-processing step like in the previous experiment. Another way would be to improve the coverage of possible variations by using more control samples. We test our method for double as many (30) and all control samples available. The performance for LAIP samples increases using 30 control samples instead of 15 but stagnates when adding more. For CFU samples we see a slight drop when increasing the number of control samples. When examining the results for each element in the test set separately, it is revealed that the samples that yield the lowest scores are the same as with 15 control samples with minor fluctuation. Looking at the results for both tubes combined adding more control samples shows a similar result as the pre-filtering experiment above, with a lower mean With respect to the question whether capturing more variety within one sample or between samples is of bigger importance, the results indicate that it is more beneficial to include less events from more control samples than vice versa. Nevertheless, we observed that with increasing number of control samples the control-events additionally transformed into the embedding space tend to be more spread between existing clusters, which is explicable by the following. The number of events selected per sample decreases when more control samples are added. If a cell population is rather heterogeneous within one sample, the additionally selected events run risk to be too different and to not adhere to the healthy cell populations already embedded similar to as demonstrated in In this work, a novel method based on UMAP and HDBSCAN for MRD detection in AML FCM samples is introduced. The approach does not require model-training and only uses control samples that do not contain leukemic cells. It falls in the category of one-class classification approaches, since only labels of healthy cell populations in control samples are utilized. We compare our method to a supervised approach based on the Set-Transformer that has proven to be successful for ALL data . With onThe method presented is subject to some limitations that we outline in this section.Since the sampling of control-events is a random procedure and UMAP as well as HDBSCAN are non-deterministic algorithms, results can differ slightly from run to run. The variability of results can be reduced by setting a seed-value or repeating experiments several times and looking at the average of results.As discussed in Given that UMAP is a graph-based algorithm and given its potential to form biological meaningful clusters, Graph Neural Networks (GNN) seem worthwhile exploring. The graph constructed by UMAP, which is an approximation of and thus inherits information of the manifold the data relies on, could be used as input of a node classification or anomalous subgraph detection task. This would be an holistic approach since not only the feature vectors but also the relation of cell populations within one sample are taken into account through the node connectivity matrix of the graph.As mentioned in Currently, the fixed global threshold of the proportion of control-events to detect blast clusters represents a limitation of the method. It could be beneficial to use an adaptive sample specific threshold determined by the statistics of the control-events proportion of all clusters of a sample.Another promising branch of future development is to utilize all drop-in markers of a sample during processing. Utilizing all drop-in markers requires to take the relation between events without drop-in markers  to neighbouring events with drop-in markers  into account. A common way is to predict an event’s unknown marker expression level by nearest neighbour imputation based on the overlapping markers ,26,27,28We propose a method that only avails itself of healthy cell populations, is easy to implement and provides transparency and interpretability through out the classification process.By exploring alternative clustering methods commonly used within flow cytometry we found that the push-pull characteristic of UMAP together with its use of local distance metric is crucial for the success of our pipeline.Further, the low scores of the Set-Transformer are an indication that one-class classification methods are more suitable than supervised methods for MRD detection in AML due to data scarcity and heterogeneity of blast populations.When looking at the results in combination with run-time we recommend pre-filtering of the events prior to classification as presented in We demonstrate that despite the challenging nature of MRD detection in AML, automated gating is not just a future dream but realistically feasible.On a general note, we want to emphasize the importance of international and inter-laboratory collaboration in combination with the establishment of standards for data acquisition, marker panels, gating strategies and flow cytometry settings, in order to produce comparable FCM data across laboratories and countries. We believe that future steps in this direction are crucial for increasing data availability and hence for the development and maintenance of high quality automatic MRD detection methods.


Xuanwei area, in southwestern China, harbors the highest female lung cancer rate in the country (Supplementary Table 1) . The genThe patients recruited for this study were 117 non-smoker females with untreated primary lung adenocarcinoma (LUAD) from the Xuanwei area, who were receiving surgical treatment at Yunnan Cancer Hospital (Supplementary Table 2). Samples were taken for whole-exome sequencing (WES) , and 33 normal and 115 tumor samples were sequenced with mRNA-Seq technology. Datasets of 168 TCGA-LUAD female smokers (TLSF) and 102 TCGA-LUAD female non-smokers (TLNF) were adopted from The Cancer Genome Atlas (TCGA) program for genomic comparison between lung cancer associated with cigarettes and that associated with smoky coal . RNA-seq dataset for mouse_cigarette model analyses were downloaded from GEO accession: GSE76205 [https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE76205]. RNA-seq and somatic mutation datasets of TCGA-LUAD and TCGA-LUSC were downloaded from XENA (https://xena.ucsc.edu/). All other relevant data are available within the article, Supplementary data or available from the authors upon request.Clinical data (deidentified) are listed in Supplementary Table 2. Raw sequencing data including WES and RNA-seq datasets from human samples have been deposited in the Genome Sequence Archive under the accession code HRA000124. The RNA-seq dataset from rat model samples has been deposited in the Gene Expression Omnibus (GEO, nwab152_Supplemental_FilesClick here for additional data file.


Hepatocellular carcinoma (HCC) is one of the leading causes of cancer death in the world owing to limitations in its prognosis. The current prognosis approaches include radiological examination and detection of serum biomarkers, however, both have limited efficiency and are ineffective in early prognosis. Due to such limitations, we propose to use RNA-Seq data for evaluating putative higher accuracy biomarkers at the transcript level that could help in early prognosis.To identify such potential transcript biomarkers, RNA-Seq data for healthy liver and various HCC cell models were subjected to five different machine learning algorithms: random forest, K-nearest neighbor, Naïve Bayes, support vector machine, and neural networks. Various metrics, namely sensitivity, specificity, MCC, informedness, and AUC-ROC (except for support vector machine) were evaluated. The algorithms that produced the highest values for all metrics were chosen to extract the top features that were subjected to recursive feature elimination. Through recursive feature elimination, the least number of features were obtained to differentiate between the healthy and HCC cell models.From the metrics used, it is demonstrated that the efficiency of the known protein biomarkers for HCC is comparatively lower than complete transcriptomics data. Among the different machine learning algorithms, random forest and support vector machine demonstrated the best performance. Using recursive feature elimination on top features of random forest and support vector machine three transcripts were selected that had an accuracy of 0.97 and kappa of 0.93. Of the three transcripts, two were protein coding (PARP2–202 and SPON2–203) and one was a non-coding transcript (CYREN-211). Lastly, we demonstrated that these three selected transcripts outperformed randomly taken three transcripts , hence were not chance findings, and could then be an interesting candidate for new HCC biomarker development.Using RNA-Seq data combined with machine learning approaches can aid in finding novel transcript biomarkers. The three biomarkers identified: PARP2–202, SPON2–203, and CYREN-211, presented the highest accuracy among all other transcripts in differentiating the healthy and HCC cell models. The machine learning pipeline developed in this study can be used for any RNA-Seq dataset to find novel transcript biomarkers.www.github.com/rajinder4489/ML_biomarkersCode: The online version contains supplementary material available at 10.1186/s12885-021-08704-9. The liver, one of the largest organ in the body, performs various important functions, such as filtering harmful substances from the blood to be then excreted from the body, producing bile to help in the digestion of fats from food, or storing glycogen (sugar) that will be used for energy. Due to its continuous exposure to harmful substances, it is prone to the amplitude of diseases that can eventually cause liver failure and/or liver cancer. Cirrhosis, long-term infection with hepatitis B virus, and hepatitis C virus, alcoholic liver disease, and nonalcoholic fatty liver disease (NAFLD) are leading risk factors for primary liver cancer . MoreoveReducing the global burden of HCC is, therefore, a primary concern and it can be achieved by improving early detection and management . CurrentThough the combinations of various biomarkers are better predictors than the individual biomarkers, sensitivity or specificity is still low for all biomarker combinations , 20–23. Using RNA-Sequencing (RNA-Seq), the whole transcriptome can be quantified. Moreover, different types of transcripts (protein coding and non-coding) can also be identified. Most transcriptomics analyses focus on gene expression by aggregating the expression of all transcripts for the given gene. However, in this study, we will focus on the transcripts because alternative-splicing defects in cancer are well documented –35 and dMachine learning (ML) is a multidisciplinary field that makes use of computer science, artificial intelligence, computational statistics, and information theory to build algorithms that learn from existing data and make predictions on new data . It has From the transcriptomics data, three datasets were assembled: all transcripts, protein coding only, and non-coding only. The goal of making these three datasets was to see if one of them provides a better prediction. Consecutively, the efficiency of the known protein biomarkers Table  was alsoData collectionHCC cell models: The list of all HCC human cell models was obtained from Cellosaurus  Suppl. .RNA-Seq data: Using the names and synonyms of these cell models, RNA-Seq datasets were searched on the European Nucleotide Archive (ENA) and were filtered for baseline expression, instrument model (Illumina HiSeq 2000 or HiSeq 2500 or NovaSeq 6000), and paired-end library layout  were first trimmed of their adapter sequences using Trimmomatic , mapped 3.Preparing different datasets: We analyzed the known protein biomarkers and complete data  separately. Furthermore, the transcriptomics data consists of protein coding and non-coding transcripts and it provided the opportunity to investigate the efficiency of different types of transcripts in identifying healthy and HCC cell models. We made three datasets, namely all transcripts (protein coding and non-coding), protein coding only, and non-coding only for both – all data and known protein biomarkers Fig. .Machine learning algorithms: On these six datasets Fig. , machineThe data was first divided into 70:30 for training and testing, respectively. A separate validation set was not created because we used k-fold cross-validation to tune the model’s hyper-parameters. In the case of datasets  from all data, all transcripts that had a total expression for all samples below 10,000 were removed. This expression filter was applied to take into account only the highly expressed transcripts. However, in the case of known biomarkers, no such filter was used since we wanted to retain all information. Furthermore, using the ‘findCorrelation’ feature from the Caret library, highly correlated transcripts (> 0.75) were identified and removed, except one . Each algorithm’s performance is assessed on all datasets by evaluating various metrics, namely sensitivity, specificity, accuracy, Matthew’s correlation coefficient (MCC), and informedness (eqs. 1–4) using R library ‘MLeval’   were extracted using “varImp” from the Caret library. Then to find the minimum set of features to differentiate between healthy and HCC cell models, “RFE” (Recursive Feature Elimination) from the Caret library was applied using the method cross-validation (CV).Machine learning:TP is true positive.TN is true negative.FP is false positive.FN is false negative.MCC is Mathew’s correlation coefficient4.Re-training the model: The features (transcripts) selected using RFE were used to train the final model. Taking these features, exhaustive k-fold cross-validation was run by setting the repeats to 100 and number to 10; implying 1000 instances will be evaluated.5.Chance findings: There were a total of ~ 200 k transcripts and to establish that the features (transcripts) selected using RFE were not chance findings, 15,000 iterations were performed taking three random transcripts out of the highly expressed transcripts to compare their prediction accuracy. The results from randomly taken transcripts were compared to the selected features (transcripts from RFE).To obtain an exhaustive list of all HCC in vitro cell models, Cellosaurus  was usedThe samples’ quality was assessed using FastQC, and it was observed that all samples passed the “Per base sequence quality” metric. However, one sample (PHH_024_1) did not pass the library size filter and was discarded. The samples passing the filters were then processed and the transcript expression was normalized using DESeq2 for different studies.We first investigated the expression patterns of the known biomarkers at the transcript level to see if the protein coding transcripts demonstrate a similar expression pattern as known protein biomarkers. Each gene can have multiple protein coding transcripts, only the ones mapped to manually annotated and reviewed Uniprot identifiers were considered and their expression pattern was examined  and all data , in predicting the correct labels for the cell models. We focused only on highly expressed transcripts and hence, to remove the lowly expressed ones, an expression filter was introduced  Table . HoweverThe results obtained from the algorithms show that the area under the curve-receiver operating characteristics (AUC-ROC) values was the highest for RF and the lowest for KNN, across all datasets Fig. . AUC-ROCBased on the values of different metrics used to assess the performance of the algorithms on various datasets, RF and SVM performed the best for all datasets; primarily for all transcripts, protein coding transcripts, and non-coding transcripts datasets for all data. To further get the least number of features required to differentiate between the healthy and HCC cell models, the top 20 important features (transcripts) from RF and SVM when applied to all data-all transcripts were taken Fig. A. There The PARP2–202 transcript is the second-largest protein coding transcript from the PARP2 gene. It shares 97.8% coding sequence (CDS) identity and similarity with the longest protein coding transcript (PARP2–201) from the same gene. In the case of SPON2–203, it is the largest protein coding transcript from the SPON2 gene . Lastly,An investigation of the gene ontology terms  obtained using DAVID  highlighTo assess the strength of the model, it was re-trained using the three predicted features (transcripts) but with an increased number of cross-validations . High values for all metrics were observed with RF and SVM (sensitivity: 0.968 and 0.944 (RF and SVM), specificity: 1 and 1, MCC: 0.973 and 0.936, informedness: 0.968 and 0.944, and AUC-ROC: 0.99 (RF only)). Moreover, the confidence interval for sensitivity and specificity in the case of RF was 0.84–0.99 and 0.92–1, respectively. Finally, to establish that these transcripts  were not chance findings, random combinations of three transcripts (highly expressed) were made and their efficiency was assessed and compared to the three transcripts selected using RFE. It was observed that out of 15,000 combinations created, none of the combinations exhibited higher or equal values for the metrics for RF and only 0.12% cases (18 cases) demonstrated higher or equal value for the metrics in the case of SVM (Suppl. Table Hepatocellular carcinoma (HCC) has a huge global burden and the challenge lies primarily in its early detection owing to the limited accuracy of serum biomarkers and inefficiency of radiological examinations. With advancements made in machine learning over the last few years, we investigated if it can assist in finding better biomarkers for HCC. We took RNA-Seq data from HCC and healthy liver cell models and used various machine learning algorithms to highlight key features that can differentiate between the healthy and HCC cell models with high accuracy. A set of three transcripts were identified, namely PARP2–202, SPON2–203, and CYREN-211; proposed as novel putative transcript biomarkers.Though widely studied, RNA-Seq data for HCC at baseline is not abundantly available. Out of 250 HCC cell models listed in Cellosaurus, data could only be obtained for 33 cell models. Many studies were discarded in the process of selection due to single-end library layout, low coverage, exposure to drugs various treatments, and insufficient metadata. For the 33 cell models taken in this study, 28 had only one replicate. This could have been a limiting factor if these were to be analyzed per cell model, however, in this study the focus was on HCC and all cell models were combined to define the transcriptome profile of HCC. Using the transcriptome profile, the cell type and/or condition  can then be accurately assessed  and thenFor HCC, many biomarkers are extensively studied Table , AFP beiMultiple machine learning algorithms  were used to analyze the data, and all exhibited high efficiency. It was surprising to see how well these algorithms performed, despite significant variations in the sample and library preparation by different labs. Though all exhibited high efficiency, we observed some differences among them across all datasets as illustrated by various metrics calculated for them Figs.  and 3. TThe highest values for all metrics were demonstrated by RF and SVM on all data-all transcripts dataset and the confidence intervals were smallest for RF for the mentioned dataset. NB also exhibited high values for all metrics for all data-all transcripts dataset however it performed poorly for other datasets and hence was not considered for further analyses. Hence top 32 important features were extracted from the algorithm-dataset combination  to find the least number of features using RFE. RFE employs a backward selection of the predictors, starting with all and removing the ones with the least importance in the model. Three transcripts were identified with maximum accuracy and kappa Fig. B. None oRe-training the model using the three selected transcripts by applying exhaustive cross-validation helped in establishing their potency in discriminating the healthy from the HCC cell models. A final comparison with randomly selected highly expressed transcripts further established that these three transcripts were not chance findings; with values for all metrics always higher than the random combinations for RF and only 6 cases exhibited higher values for SVM.The catalytic activity of PARP2, one of the poly-ADP-ribose polymerase (PARP) enzymes, has been shown to be induced by DNA-strand breaks. This provides evidence for its cellular response to DNA damage . FurtherWhile SPON2 knockdown cell lines exhibit higher hepatoma cell migration and invasion, overexpression repressed them . At the Though these transcripts are validated through in silico approaches and their role in HCC are defined in the literature, an extensive validation in the HCC patients still needs to be done. If established, such an approach can also be used to identify transcript-level biomarkers for various diseases and conditions, thus providing us an opportunity to look beyond proteins and maybe help in the identification of the disease or the condition at an early stage. One drawback of the current study was that the data was taken from the liver and to predict HCC, an invasive approach has to be taken to extract the sample. To look for transcript biomarkers for HCC that are non-invasive, data from HCC patient’s blood serum/plasma will be required. At this moment, the scarcity of such data limits us from exploring the circulating mRNAs from HCC to find novel and potent biomarkers through in silico approaches. A thorough follow-up study would be required to look for non-invasive/circulating transcript biomarkers in the blood of the HCC patients, by generating and analyzing the data as discussed in this study.In our investigation of the healthy liver and various HCC cell models to find novel biomarkers, we analyzed RNA-Seq data using machine learning. Comparing the known HCC biomarkers with all other possible transcripts, we first concluded that using the exhaustive transcript list displayed better accuracy, thus implying that better biomarkers exist. Similarly, between all existing transcripts, protein coding transcripts only, or non-coding transcripts only, it was illustrated that all transcriptomics data improved also the overall accuracy. From this observation, it can be concluded that both protein coding and non-coding transcripts hold important information and are regulated under internal and/or external stimuli. This is further supported by the identification of two protein coding (PARP2–202 and SPON2–203) and one non-coding (CYREN-211) transcript as novel and potent biomarker for HCC. However, the findings would have to be validated in vivo.The pipeline developed in this study to identify transcript level biomarkers for HCC can be applied to other RNA-Seq datasets as well.Additional file 1: Fig. S1: Log2FC for protein-coding transcripts of the known biomarkers. The known biomarkers were mapped to Ensembl gene ids and for these genes, log2FC for the longest protein-coding transcript  was observed. It can be seen that most transcripts demonstrate upregulation as established by protein assays. The genes where two or more transcripts were the longest protein-coding, all of them were taken. Fig. S2: Overlap between important features: Top 20 important features were taken from RF and SVM for all data-all transcripts. A total of 8 features overlapped between the two algorithms.Additional file 2: Suppl. Table 1: The list of all HCC human cell models that were obtained from Cellosaurus.Additional file 3: Suppl. Table 2: Random combinations of three transcripts to assess their accuracy for correctly identifying the healthy and HCC cell models using ML algorithms.


Single-cell RNA sequencing (scRNA-Seq) experiments are gaining ground to study the molecular processes that drive normal development as well as the onset of different pathologies. Finding an effective and efficient low-dimensional representation of the data is one of the most important steps in the downstream analysis of scRNA-Seq data, as it could provide a better identification of known or putatively novel cell-types. Another step that still poses a challenge is the integration of different scRNA-Seq datasets. Though standard computational pipelines to gain knowledge from scRNA-Seq data exist, a further improvement could be achieved by means of machine learning approaches.Autoencoders (AEs) have been effectively used to capture the non-linearities among gene interactions of scRNA-Seq data, so that the deployment of AE-based tools might represent the way forward in this context. We introduce here scAEspy, a unifying tool that embodies: (1) four of the most advanced AEs, (2) two novel AEs that we developed on purpose, (3) different loss functions. We show that scAEspy can be coupled with various batch-effect removal tools to integrate data by different scRNA-Seq platforms, in order to better identify the cell-types. We benchmarked scAEspy against the most used batch-effect removal tools, showing that our AE-based strategies outperform the existing solutions.scAEspy is a user-friendly tool that enables using the most recent and promising AEs to analyse scRNA-Seq data by only setting up two user-defined parameters. Thanks to its modularity, scAEspy can be easily extended to accommodate new AEs to further improve the downstream analysis of scRNA-Seq data. Considering the relevant results we achieved, scAEspy can be considered as a starting point to build a more comprehensive toolkit designed to integrate multi single-cell omics.The online version contains supplementary material available at 10.1186/s12859-021-04150-3. Single-cell RNA sequencing (scRNA-Seq) was named the “Method of the Year” in 2013, and it is currently used to investigate cell-to-cell heterogeneity since it allows for measuring the transcriptome-wide gene expression at single-cell resolution, enabling the identification of different cell-types. scRNA-Seq data are prevalent generated in studies that aim at understanding the molecular processes driving normal development and the onset of pathologies [One of the most important steps in scRNA-Seq analysis is the clustering of cells into groups that correspond to known or putatively novel cell-types, by considering the expression of common sets of signature genes. However, this step still remains a challenging task because applying clustering approaches in high-dimensional spaces can generate misleading results, as the distance between most pairs of points is similar . As a coet al. showed that low-dimensional representations of the original data learned using latent variable models preserve both the local and global neighbour structures of the original data [Although commonly used approaches for dimensionality reduction achieved good performance when applied to scRNA-Seq data , novel anal data . Autoencet al. integrated a Linearly Decoded VAE (LDVAE) into scVI [Deep Count AE network (DCA) was one of the first AE-based approach proposed to denoise scRNA-Seq datasets  by consinto scVI , enablinnto scVI . More imThe framework called Decomposition using Hierarchical AE (scDHA) is composed of two modules . The firet al. proposed a deep VAE for scRNA-Seq data named VASC [AEs coupled with disentanglement methods have been used to both improve the data representation and obtain better separation of the biological factors of variation in gene expression data . In addimed VASC , a deep med VASC .et al. proposed an AE-based cluster ensemble framework to improve the clustering process [Dimensionality Reduction with Adversarial VAE (DR-A) has been recently proposed to fulfil the dimensionality reduction step from a data-driven point of view . Compare process . As a fiAutoImpute was proposed to deal with the insufficient quantities of starting RNA in the individual cells, a problem that generally leads to significant dropout events. As a consequence, the resulting gene expression matrices are sparse and contain a high number of zero counts. AutoImpute is an AE-based imputation method that works on sparse gene expression matrices, trying to learn the inherent distribution of the input data to assign the missing values . scSVA wOther two methods based on non-parametric AEs were proposed to address the imputation problem . LearninGraphSCI combines Graph convolution network and AE, and it is the first approach that systematically integrates gene-to-gene relationships with the gene expression data into a deep learning framework. GraphSCI is able to impute the dropout events by taking advantage of low-dimensional representations of similar cells and gene-gene interactions .https://github.com/fchollet/keras), TensorFlow [Generally, in the existing AEs the input data are usually codified in a specific format, making their integration into the existing scRNA-Seq analysis toolkits  represents the number of neurons composing the hidden layer (H neurons) and latent space (L neurons).We tested PCA and AEs to address the integration of different datasets. Specifically, we used all the AEs implemented in our scAEspy tool: VAE , an AE ok-Nearest Neighbours (BBKNN) [In order to deal with the possible batch-effects, we applied the following approaches, as suggested in , 11 and  (BBKNN) , Harmony (BBKNN) , ComBat  (BBKNN) –40, and  (BBKNN) . Thus, wThe proposed strategies were compared on three publicly available datasets, namely: Peripheral Blood Mononuclear Cells (PBMCs), Pancreatic Islet Cells (PICs), and Mouse Cell Atlas (MCA) by using well-known clustering metrics . It is worth mentioning that generally the cell-types are manually identified by expert biologists starting from an overclustering or underclustering of the data, possibly followed by different steps of subclustering of some clusters. Here, we evaluate how the different strategies are able to automatically separate the cells by fixing the number of clusters equal to the number of cell-types manually identified by the authors of the papers.“Integrating stimulated vs. control PBMC datasets to learn cell-type specific responses” .PBMCs from eight patients with systemic lupus erythematosus were collected and processed using the platform . The datworkflow . The cou“Integration and Label Transfer” .PIC datasets were generated independently using four different platforms: CEL-Seq  .MCA is composed of two different datasets. The former was generated by Han et al.  using Miet al. , while tet al.  using Smet al. . The couet al.  (https:/The Rand Index (RI) is a similarity measure between the results obtained from the application of two different clustering methods. The first clustering method is used as ground truth , while the second one has to be evaluated . The RI is calculated by considering all pairs of samples appearing in the clusters, namely, it counts the pairs that are assigned either to the same or different clusters in both the predicted and the true clusters. The Adjusted RI (ARI)  is the “The Mutual Information Index (MII)  represenThe Fowlkes Mallows Index (FMI)  measuresThe result of the tested clustering approach satisfies the Homogeneity Score (HS)  if all oThe result of the tested clustering approach satisfies the Completeness Score (CS)  if all tThe V-Measure (VM)  is the hNowadays, various scRNA-Seq platforms are currently available  and We merged the control and treated datasets by first using vanilla PCA and AEs, and then PCA and AEs followed by either BBKNN or Harmony, ComBat, and CCA. After the construction of the neighbourhood graphs, we performed a clustering step by using the Leiden algorithm . Since ip-value lower than 0.0001, confirming that the achieved results are statistically different compared to those achieved by the other approaches.Our analysis showed that the CCA-based approach, proposed in the Seurat library, achieved a mean ARI equal to rrection . In all Regarding the AMII, CCA had a mean value of entclass1pt{minimaH, L) in terms of ARI  than in those acquired with droplet-based methods . Thus, combining gene expression data that spread across several orders of magnitude is a difficult task that cannot be tackled by using linear approaches like PCA. To examine how well AEs perform in solving this issue, we combined four PIC datasets acquired with CEL-Seq , CEL-SeqWe integrated the datasets by first using vanilla PCA and AEs, and then PCA and AEs followed by either BBKNN or Harmony, ComBat, and CCA. Since in the original paper 13 cell-types were manually annotated for the PIC datasets , we clusCCA had a very low mean ARI, i.e., Similar results were achieved for the AMII metric: CCA reached a mean value equal to H, L) in terms of ARI  outperformed the other AEs as well as the other strategies, obtaining a mean ARI equal to H, L) in terms of ARI, the vanilla GMMMD with the NB loss function obtained the best results for the dimension , while GMMMD followed by Harmony reached the best results for the dimensions  and , exploiting the Poisson loss function and NB loss function, respectively. MMDAE followed by BBKNN (using the ZIP loss function) achieved the best results for the dimensions  and , exploiting the NB loss function and Poisson loss function, respectively. MMDVAE followed by Harmony resulted the best choice for the dimensions  and  when coupled with the NB loss function and Poisson loss function, respectively. Finally, VAE followed by Harmony with the Poisson loss function obtained the best results for the dimension . As for the the integration of the PBMC datasets, we used two Gaussian distributions because we merged two different datasets.Comparing the best AE for each of the tested dimension  and 16 GB of RAM, running on Ubuntu 20.04 LTS. In all the tested datasets, we used the top 1000 HVGs. The executions lasted 95.02 seconds, 101.05 seconds, and 223.45 seconds for PIC, MCA, and PBMC, respectively. Considering that PIC, MCA, and PBMC are composed of 6321, 6954, and 14039 cells, respectively, scAEspy scales linearly with the number of cells.Non-linear approaches for dimensionality reduction can be effectively used to capture the non-linearities among gene interactions that may exist in the high-dimensional expression space of scRNA-Seq data . Among tHere, we proposed scAEspy, a unifying and user-friendly tool that allows the user to use the most recent and promising AEs . We also designed and developed GMMMD and GMMMDVAE, two novel AEs that combine MMDAE and MMDVAE with GMVAE to exploit more than one Gaussian distribution. We introduced a learnable prior distribution in the latent space to model the dimensionality of the subpopulations of cells composing the data, or to combine multiple samples.We integrated AEs with both Harmony and BBKNN to remove the existing batch-effects among different datasets. Our results showed that exploiting the latent space to remove the existing batch-effects permits for better identification of the cell subpopulations. As a batch-effect removal tool, Harmony allowed for achieving better results than BBKNN in the majority of the cases. When different droplet-based data have to be combined, our GMMMD and the MMDAE, coupled with the constrained NB and Poisson loss functions, obtained the best results compared to all the other AEs. In order to combine and analyse multiple datasets, generated by using different scRNA-Seq platforms, both GMMMD and MMDVAE, mainly used together with the NB and Poisson loss functions, outperformed the other strategies. However, also GMVAE and the simple VAE obtained outstanding performance, highlighting that the Kullback-Leibler divergence function can become fundamental to handle data spreading various orders of magnitude, especially the high values (up to millions) introduced by plate-based methods. It is clear that using more than one Gaussian distribution allows for obtaining a better integration of the datasets and separation of the cell-types when more than two datasets have to be integrated, as clearly shown by the results reached on the PIC datasets.As a good practice, we suggest filtering the gene expression matrices using the top HVGs , calculated separately within each batch and merged to avoid the selection of batch-specific genes. The original counts, corresponding to the top HVGs, should be the input data of scAEspy. GMMMD coupled with Harmony should be used to integrate different droplet-based datasets as well as to merge datasets generated by using different scRNA-Seq platforms. Harmony should be applied to the latent space produced by GMMMD to take into account the possible batches among the datasets. When a single dataset has to be analysed, the users should use MMDAE with droplet-based data, while MMDVAE should be preferred for data acquired with plate-based methods. In all cases, using a single hidden layer composed of 256 neurons, a latent space with 32 neurons, and the constrained NB or Poisson loss function generally allows for obtaining satisfactory results.Considering the achieved results on the identification of the clusters, scAEspy can be used at the basis of methods that aim at automatically identifying the cell-types composing the scRNA-Seq datasets under analysis . As a maIn this study, we proposed an AE-based and user-friendly tool, named scAEspy, which allows for using the most recent and promising AEs to analyse scRNA-Seq data. The user can select the desired AE by only setting up two user-defined parameters. Once the selected AE has been trained, it can be used to generate synthetic cells to increase the number of data for further downstream analyses . In scAEspy, the latent space is easily accessible and thus allows the user to perform different types of analysis, such as the correction of possible batch-effect in a reduced non-linear space or the inference of differentiation trajectories. In this case, the latent space can be utilised to generate the “pseudotime” that measures transcriptional changes that a cell undergoes during the dynamic process.Thanks to its modularity, scAEspy can be extended to accommodate new AEs so that the user will be always able to utilise the latest and cutting-edge AEs , which cAs an improvement, prior biological knowledge about genes from ontologies can be incorporated into scAEspy. Ontologies can introduce useful information into machine learning systems that are used to solve biological problems. They allow for integrating data from different omics  as structured representations of semantic knowledge, which is commonly used for the representation of biological concepts. This approach has been successfully applied to predict the clinical targets from high-dimensional low-sample data . SpecifiSince AEs showed outstanding performance in the integration of multi-omics of cancer data , we planWe developed scAEspy so that it can be easily integrated into both Scanpy and Seurat pipelines, as it directly works on a gene expression matrix see Fig. . We inte InfoVAE  and scVA InfoVAE . In addik  . We appli) the cells are grouped into multiple-dataset clusters by exploiting a variant of the soft k-means clustering, which is a fast and flexible method developed to cluster single-cell data; (ii) a centroid is calculated for each cluster and for each specific dataset; (iii) using the calculated centroids, a correction factor is derived for each dataset; (iv) the correction factors are then used to correct each cell with a cell-specific factor.We also applied Harmony  to removet al. [k-nearest neighbours in a reduced space , where the nearest neighbours are identified in a batch-balanced manner using a user-defined distance . The neighbour information is transformed into connectivities to build a graph where all cells across batches are linked together. We used both Harmony and BBKNN to correct the PCA and AE spaces.As a further batch-effect removal tool, we applied BBKNN . Polansket al.  showed tet al. , and mnnet al. . In addiinit_pos equal to random, and random_state equal to 10 of the umap function).As a final step, we calculated the UMAP spaces starting from the built neighbourhood graphs and using the default parameter settings proposed in Scanpy, except for the initialisation of the low dimensional embedding  we calculated the neighbourhood graph in both PCA and AE spaces by using the default parameter settings proposed in Scanpy. Then, we clustered the obtained neighbourhood graphs with the Leiden algorithm as described before. Finally, we calculated all the metrics for each found resolution; (ii) we performed a similar analysis where we firstly corrected the PCA and AE spaces using Harmony [https://github.com/slowkow/harmonypy; (iii) we performed the same analysis described in (i) by replacing the neighbourhood graphs with those generated using BBKNN, using the default parameter settings.After that, we applied three different strategies Fig. : (i) we  Harmony  with theIt is worth mentioning that in all the tested batch-effect removal tools, we used the different samples that have to be merged as batches. Specifically, the number of batches is equal to 2, 4, and 2 for the PBMC, PIC, and MCA datasets, respectively.In this work, we used the notation proposed in  to extenWe consider K corresponds to the number of desired Gaussian distributions. As general strict divergence function, we considered the We remind that in VAEs, the input function .The ELBO term proposed in this work, which is the measure maximised during the training of AEs, is:vergence  between Equation  allows thttps://gitlab.com/cvejic-group/scaespy under the GPL-3 license. scAEspy can be easily installed using the Python package installer pip, which allows the user to use scAEspy either as a standalone tool exploiting the command line interface, or as a Python package that can be integrated into Python scripts and Jupyter Notebooks.scAEspy is written in Python programming language (v.3.6.5) and it relies on TensorFlow (v.1.12.0), an open-source and massively used machine learning library . scAEspyThe repository contains all the scripts, code and Jupyter Notebooks used to obtain the results shown in the paper. In the provided Jupyter Notebooks, we show how easy it is to integrate scAEspy and Scanpy, and how the data can be visualised and explored by using both scAEspy and Scanpy’s functions. We also provide a detailed description of scAEspy’s parameters so that it can be used by both novice and expert researchers for downstream analyses.Additional file 1. Mathematical formulation of the proposed autoencoders. We provide the mathematical derivation of GMMMD and GMMMDVAE as well as the generalised formulation that we derived by following the notation proposed in [posed in .Additional file 2. Excel file of the metrics calculated for the PBMC datasets. Each tab is related to a tested approach and shows the calculated metrics and used method.Additional file 3. Excel file of the metrics calculated for the PIC datasets. Each tab is related to a tested approach and shows the calculated metrics and used method.Additional file 4. Excel file of the metrics calculated for the MCA datasets. Each tab is related to a tested approach and shows the calculated metrics and used method.


Deep generative models, such as variational autoencoders (VAEs) or deep Boltzmann machines (DBMs), can generate an arbitrary number of synthetic observations after being trained on an initial set of samples. This has mainly been investigated for imaging data but could also be useful for single-cell transcriptomics (scRNA-seq). A small pilot study could be used for planning a full-scale experiment by investigating planned analysis strategies on synthetic data with different sample sizes. It is unclear whether synthetic observations generated based on a small scRNA-seq dataset reflect the properties relevant for subsequent data analysis steps. We specifically investigated two deep generative modeling approaches, VAEs and DBMs. First, we considered single-cell variational inference (scVI) in two variants, generating samples from the posterior distribution, the standard approach, or the prior distribution. Second, we propose single-cell deep Boltzmann machines (scDBMs). When considering the similarity of clustering results on synthetic data to ground-truth clustering, we find that the  This raises the question of whether such techniques could also be trained on data with a rather small number of samples, e.g., obtained from pilot experiments, for subsequently generating larger synthetic datasets. Such synthetic observations could inform the design of single-cell RNA sequencing (scRNA-seq) experiments by exploring planned subsequent analysis steps, such as clustering, on synthetic datasets of different sizes.Deep generative models, such as variational autoencoders (VAEs)8. As indicated, another interesting property of these generative approaches is that they can provide synthetic data once trained on some dataset. However, the quality of the data generated from these models is challenging to evaluate and requires cautious examination, depending on the field of application and the research question9. In the following, we investigate the quality of synthetic data from generative models and illustrate various obstacles for its application based on an example of the design of scRNA-seq experiments.ScRNA-seq experiments result in data reflecting gene expressions for individual cells in tissues, leading to an improved understanding of cell-type composition. Due to the underlying complexity, deep generative approaches are increasingly used to investigate the structure of scRNA-seq data by learning a low-dimensional latent representation of gene expression within cells. Often, the focus of these applications is on exploring latent features—representing cell types—after which they are used for clustering, imputation, or differential expression analysis10.Single-cell experiments are often costly and time-consuming; hence, it is crucial to plan these experiments carefully to avoid wasting resources. Based on small pilot experiments, deep learning models could generate synthetic data to make judgments about potentially larger experiments, thus offering the possibility of gaining confidence in planning. The apparent advantage to this would be that scientists could generate a practically infinite amount of synthetic data, preventing experiments from being too small or too large. Moreover, synthetic data can also be valuable for experimental design by investigating in-silico generated cell perturbations15, synthetic data could be useful, e.g., when training a generative approach on some pilot data. Sampling from latent representations of generative models then allows for generating in-silico expression patterns, ideally reflecting the most important patterns from the pilot data, and can subsequently be utilized for planning experiments. More specifically, researchers would specify different numbers of cells to be simulated, then apply downstream analyses to the simulated data, after which they evaluate the number of cells needed for detecting patterns of interest, such as clusters comprising rare cell types.As the experimental design of scRNA-seq studies is often based on simulationsTo investigate the authenticity of the synthetic data using experimental design as an example, we follow the procedure shown in Fig. We specifically selected three evaluation methods to investigate the extent to which synthetic data from the models considered can help recapitulate the multivariate structure of gene expression data. This is particularly relevant for clustering, and consequently, for the discovery of cell types. Accordingly, we examine both internal and external clustering evaluation metrics. More specifically, we chose the Davies–Bouldin index because it provides a value for the appropriateness of data partitions. Suppose synthetic data based on a small sub-sample of the original dataset yields similarly small values for the Davies–Bouldin index (DBI). In that case, we can assume that this clustering is also of similar quality, allowing the DBI of the clustering partition on the original data to be used as a benchmark. The DBI could be used to identify a saturation point in the number of generated cells beyond which the DBI does not substantially improve. Also, scientists can use the variability of the DBI across multiple sub-sampling iterations to infer an appropriate sample size, as lower variability indicates a more stable clustering solution. Furthermore, we use the adjusted Rand index (ARI) as an external evaluation criterion. The ARI gives us information about the similarity of two clusterings based on the original data labels. As a further criterion, we examine the cluster proportions to determine at which cluster size the methods encounter problems in recognizing and generating cells from this cluster. We use the univariate distributions of gene expression as another measure since the assignment of similar cells to a cell type is typically performed based on marker genes and may be of importance for discovering cell types.7, DBMs still need to be adapted. We show how this can be achieved using a negative binomial distribution and incorporating a regularized Fisher scoring algorithm to estimate the inverse dispersion parameter. We chose DBMs because synthetic observations are generated by Gibbs sampling, which has theoretical properties that are potentially advantageous for working with smaller sample sizes than variational inference in VAEs17. Since we use the single-cell variational inference (scVI) approach in reduced form, i.e., we do not use batch correction or zero-inflation, the method is comparable to a vanilla VAE with a negative binomial loss function, except for the estimation of the size factors for normalization.While VAEs have already been proposed for scRNA-seq dataVAEs reconstruct their input through a bottleneck layer that corresponds to a low-dimensional latent representation. They offer two ways of generating samples from the latent representation. Most commonly, samples are generated from the posterior, which is the latent variables’ probability given the original data. In a pilot study setting, this will typically mean that multiple copies of the original observations have to be used to obtain a larger synthetic dataset. This might lead to an amplification of sampling bias, as patterns or random fluctuations from single cells could be over-emphasized. In contrast, sampling from the prior might produce samples from a diverse region of the latent space. In our evaluation together with DBMs, we therefore not only investigate the performance of VAEs when feeding in the original data multiple times for obtaining a larger number of cells but also when sampling directly from the prior, which has—to our knowledge—not been considered in the scRNA-seq literature so far.19. The optimal architecture for variational autoencoders with application to scRNA-seq data has also been investigated by comparing different count likelihood functions. The authors also propose an adapted architecture to include clustering information into the latent space20. Furthermore, VAEs were used to generate previously unseen responses to single-cell perturbations based on variables learned in the latent space10. scVI was one of the first fully-integrated models for the use of VAEs in application to scRNA-seq data. It offers the possibility to learn the scRNA-seq data using the negative binomial distribution and add a zero-inflation term. It is useful for many applications in scRNA-seq data analysis and is regularly extended to other omics technologies and applications24. Because of its widespread use, it is interesting to examine this model for settings with small sample sizes.Variational autoencoders are already widely established in application to scRNA-seq data. Researchers have developed many models to visualize scRNA-seq data based on the VAE architectureTo examine the quality of the data generated by scVI and single-cell DBMs (scDBMs), we used the example of designing a scRNA-seq experiment. By mimicking a situation where we want to plan an experiment using a pilot study with a small number of cells, we investigated the impact of varying amounts of cells and generative approaches on the clustering performance, measured by the DBI. We took 30 sub-samples of 384, 768, 1152, 1536, 1920, and 2304 cells of the original dataset, trained the scDBM and scVI on these sub-samples, and generated synthetic data. More precisely, we sampled from the scDBM using Gibbs sampling and from scVI using the prior and posterior distribution, respectively. The sub-samples’ size is based on the number of cells that can be captured by a 384-well plate, which allows us to get an indication of the required number of plates for a sequencing experiment. We then applied UMAP and acquired the cluster labels by mapping the synthetic observations to the original data based on the Euclidean distance in UMAP space  and Uniform, respectively. We used the values drawn for each cell as rate parameters in the Poisson distribution and added the resulting values to the sub-sampled pilot data. To be more precise, we have merged several noisy pilot datasets to achieve the original dataset’s size. For example, for a pilot dataset with 384 cells and an original dataset of 3840 cells, we replicated the noisy dataset ten times and then merged them. One would expect the DBI (see “Methods”) to be very high in a scenario with small pilot datasets since potential artifacts could be strongly amplified.Segerstolpe data, which were generated with the Smart-seq2 protocol and therefore exhibit less sparsity, the DBIs of all models show less variability and are closer to the DBI of the original data  for each gene between the synthetically generated data and the original data for a range of metrics. More specifically, we compare medians, interquartile ranges, coefficients of variation, and the proportion of zero counts. In the ses Fig. a. A simiize Fig. b. We alsize Fig.  and 4. WSegerstolpe dataset across all models, and second, we set 20Segerstolpe dataset to zero. We were able to show that all models also learn the Segerstolpe data with artificially added zeros properly without imputing the omitted expression values , which utilizes the structure of VAEs to encode the transcriptome onto a lower-dimensional representation from which the input is reconstructed. Just as the scDBM, scVI is also based on the (zero-inflated) negative binomial distribution7.Lopez et al.7 use four neural networks for encoding the size factors and the latent variables using the variational distribution n, and 7. The variational distribution can be written as:7. scVI additionally learns a neural network to account for technical dropouts.The model comprises two components, the encoder and the decoder parts of the network. Lopez et al.p(z|x). A straightforward approach for generating more samples than were used during training is to create (multiple) copies of the original data. For example, for scVI trained on 384 cells, we sampled from the model seven times and stacked the resulting samples together to make inference about a larger number of cells. As an alternative, we adapted scVI to sampling from the prior distribution p(z) instead of the more common sampling from the posterior p(z|x). To do that, we changed the inference procedure to sample latent z from Normal and library sizes from Observations are generated from the scVI approach by using original data as input and then sampling from the posterior distribution 3, to the negative binomial distribution. Specifically, we employ the exponential family harmonium framework31 that allows restricted Boltzmann machines (RBMs), the single-hidden layer version of DBMs, to deal with any distribution from the exponential family as input. This framework was further extended and simplified by Li et al.32.We adapted deep Boltzmann machines (DBMs), an unsupervised neural network approach with multiple hidden layers33:We use a parametrization of the negative binomial probability mass function that has been suggested by Risso et al.The mean of the distribution is denoted as 34. The first and second hidden layers are denoted as For simplicity, we describe a three-layer DBM where the visible layer corresponds to an input of unique molecular identifier (UMI) counts for M genes, which can be modeled by a negative binomial distribution32, we define the energy function of the state Following Li et al.a, 3. According to this, the conditional distributions over the visible and the two sets of hidden units are given as:m 44.Moreover, we consider the adjusted Rand index (ARI). The ARI provides a similarity metric between two cluster groupings.To examine whether the models learn to adequately represent frequencies of different cell types, we also compare the number of cells per cluster in the original data and the synthetic observations.9. In contrast, we focus on properties, such as cluster quality, which are important for experimental design.It should be noted that an in-depth evaluation of samples, instead of comparing model fit based on the log-likelihood, is indispensable because it was shown that comparing deep generative models based only on the log-likelihood can be misleading. In particular, even when log-likelihood is low, the quality of generated samples can be good and vice-versa45. It follows that we also slightly increased the number of epochs. Other than that, we keep the hyperparameter settings largely the same for all models  mice processed with the CEL-Seq2 protocol54 and extends the packages’ scope to scRNA-seq data which is available at https://github.com/MTreppner/scDBM.jl. The code to reproduce all analyses and figures can be found on the following Github repository: https://github.com/MTreppner/scDBM-paper.The scDBM implementation is based on the Julia package ‘BoltzmannMachines.jl’https://github.com/YosefLab/scVI), which we adapted to be able to sample from the prior distribution.Furthermore, we used the Python implementation of scVI (Supplementary Information.