A limitation of traditional differential expression analysis on small datasets involves the possibility of false positives and false negatives due to sample variation. Considering the recent advances in deep learning (DL) based models, we wanted to expand the state-of-the-art in disease biomarker prediction from RNA-seq data using DL. However, application of DL to RNA-seq data is challenging due to absence of appropriate labels and smaller sample size as compared to number of genes. Deep learning coupled with transfer learning can improve prediction performance on novel data by incorporating patterns learned from other related data. With the emergence of new disease datasets, biomarker prediction would be facilitated by having a generalized model that can transfer the knowledge of trained feature maps to the new dataset. To the best of our knowledge, there is no Convolutional Neural Network (CNN)-based model coupled with transfer learning to predict the significant upregulating (UR) and downregulating (DR) genes from both trained and untrained datasets.We implemented a CNN model, DEGnext, to predict UR and DR genes from gene expression data obtained from The Cancer Genome Atlas database. DEGnext uses biologically validated data along with logarithmic fold change values to classify differentially expressed genes (DEGs) as UR and DR genes. We applied transfer learning to our model to leverage the knowledge of trained feature maps to untrained cancer datasets. DEGnext’s results were competitive (ROC scores between 88 and 99DEGnext can classify DEGs into UR and DR genes from RNA-seq cancer datasets with high performance. This type of analysis, using biologically relevant fine-tuning data, may aid in the exploration of potential biomarkers and can be adapted for other disease datasets. Transcriptomic profiling is important in understanding how genes regulate biological functions and control the underlying mechanisms of diseases. Differential expression (DE) analysis is used to identify the genes which undergo changes in biological patterns across healthy and disease conditions. This analysis can help researchers identify the differentially expressed genes (DEGs) which behave differently in disease conditions and help them prioritize these condition-specific genes as potential biomarkers for a particular disease. Numerous parametric and non-parametric statistical methods have been developed for DEG analysis based on logarithmic values of fold change (logFC) of gene expression in control state to disease state . For insn) as compared to the number of genes (g) [Deep learning (DL) coupled with transfer learning, on the other hand, has the ability to classify novel data by directly learning complex non-linear relationships among the features of the training data in one end-to-end classification system . A Convoenes (g) . Kakati enes (g) , proposeenes (g) –22 have enes (g) –27. HoweIn this paper, we propose a robust CNN-based model, DEGnext, in conjunction with transfer learning to classify the UR and DR genes from RNA-seq cancer datasets.In this section, we report the experimental results of DEGnext for both general and transfer learning. In the Methods section , we describe each step of the DEGnext workflow and CNN architecture in detail, so we will only give a brief overview here to aid in the interpretation of the results. Fig. We used TCGABiolinks R package  to downlFor labeling the genes in the datasets, we used logFC values in addition to disease-related knowledge from Ingenuity Pathway Analysis (IPA) to divide the data into biologically or non-biologically validated data. First, we use general learning on DEGnext, to predict the directionality of DEGs from all 17 datasets. Second, we divide 17 datasets into 9 training datasets and 8 testing or untrained datasets and use transfer learning to leverage the knowledge  acquired from the previously trained DEGnext model to predict UR and DR genes from rest 8 testing datasets. Third, we evaluate the performance of the DEGnext model for general and transfer learning against five ML methods, namely Decision Tree (DTC), Linear Regression(LR), Naïve Bayes (NB), Random Forest (RFC), Support Vector Machine (SVC), and XGBoost in terms of accuracy, recall, precision, F-measure, Matthews correlation coefficient (MCC), and Receiver Operating Characteristic (ROC) scores. Fourth, we test the robustness of DEGnext by augmenting the datasets with seven levels of Gaussian noise data  and compare it with other ML methods. Fifth, we obtain the Gene Ontology (GO) term enrichment and pathway enrichment of the predicted up/down regulated genes from cancer datasets. Finally, we identify the potential biomarkers mapped to the significant pathways related to BRCA and UCEC datasets. Throughout the text and figures, we will be using the following abbreviations for the testing and training data. The non-biologically validated data is labeled as P (“non-bio data”), and the biologically validated is labeled as Q (“bio data”). The P data were split as non-biologically validated train data (“non-bio train data” or T1) and non-biologically validated test data (“non-bio test data” or T2) in the ratio of 80:20. Similarly, the Q data were split in the ratio of 80:20 as fine-tune (F1) and biologically-validated test data (“bio-test data” or T3), respectively.In the general learning experiment, we first trained the model with non-bio train data (T1) for all 17 datasets with fivefold cross-validation. Then, we took the best fold models for each dataset and further trained the model with fine-tune data (F1). To evaluate the overall DEGnext output quality in general learning, we calculated the mean of five metrics, namely accuracy, recall, precision, F-measure, and MCC across fivefolds of bio-test data (T3). In Table General learning results for DEGnext are nearly perfect. However, most RNA-seq datasets do not have appropriate labels and have smaller sample sizes (n) compared to number of genes (g). In those situations, a general model cannot be obtained using supervised learning and we must rely on models trained on another datasets. This motivated us to make DEGnext generalizable for RNA-seq datasets, irrespective of dataset size or appropriate labels. Instead of training a CNN from scratch for any new dataset, we wanted to use the pretrained DEGnext model on new datasets without labels to predict significant UR and DR genes. Transfer learning can leverage the knowledge of trained feature maps from trained model to untrained cancer datasets . Moreover, general learning on 17 datasets is time-consuming since we need to train and fine-tune the model for each dataset separately to predict UR and DR genes from bio-test data of respective dataset. In order to analyze the effectiveness of DEGnext in transfer learning, we divided the 17 datasets into two groups based on the sample sizes: 9 training datasets with large size  and 8 testing or untrained datasets with smaller size . We used this dataset splitting strategy so that features learned during training generalized to any unknown or new dataset regardless of size. The testing datasets comprised of 100From Table The MCC scores for these datasets, such as BLCA, CHOL, HNSC, KICH, and STAD was above 0.9, which signified the better prediction of UR and DR genes from the untrained datasets. This process leverages the optimization and reduces the amount of data and time required to train new models for new datasets. Thus, we conclude that DEGnext was able to transfer the knowledge of learned feature maps from the trained datasets to untrained datasets effectively.We assessed the performance of DEGnext for both general and transfer learning in predicting UR and DR against five ML methods. In particular, we compared DEGnext to DTC, KNC, RFC, SVC, and XGBoost in terms of mean of accuracy, recall, precision, F-measure, MCC, and ROC scores. For general learning, we trained and fine tuned DEGnext and the ML methods with non-bio train data T1) and fine-tune data (F1) for each dataset. Then, we tested the models with bio-test data T3) of all 17 datasets with fivefold cross validation. In Additional file  and fine of all 1For transfer learning, we trained DEGnext and the ML methods on 80mentclass2pt{minimTo demonstrate that DEGnext is robust to noise, we tested the performance of DEGnext with increasing Gaussian noise in 7 levels: 1After classifying the DEGs into UR and DR genes using our DEGnext model, we assessed the GO enrichment of the predicted UR and DR genes using ToppGene Suite. As shown in Table cellular adhesion and cell morphogenesis, which are associated with cancer cell invasion and metastasis [reproductive processes, reproduction, and positive regulation of plasminogen activation and had more significant p-values and q-values. There is evidence [p-values and q-values associated with cancer.For instance, we observed that the predicted UR and DR genes from BRCA datasets were related to GO terms such as tastasis . On the evidence  that actRhoGDI signaling: The main functions of the Rho family of GTPase involve promoting cellular adhesion, proliferation, and metastasis of breast cancer cells. RhoB exerts positive effects on increasing expression of estrogen receptor alpha  promotes proliferation and growth of breast cancer cells . Under nBRCA dataset PTEN signaling: We found that within the significant PTEN signaling pathway that some mapped genes, such as ITGA4, MCRS1, and SOS1 were predicted to be upregulated by the DEGnext model. In , it has Ephrin receptor signaling: In a significant pathway called the Ephrin Receptor Signaling pathway, we found several mapped genes, namely ATF4, EPHA6, ITGA4, and SOS1, which were predicted to be upregulated by DEGnext and were related to carcinogenesis. For example, the gene Erythropoietin-producing human hepatocellular (EPH) receptors, such as EPHA6 has pro-tumorigenic effects and induces a number of cellular processes, such as adhesion, proliferation, differentiation during carcinogenesis of cervical cancer .UCEC dataset We performed a pathway enrichment analysis of the predicted UR and DR genes obtained from the bio-test data of BRCA and UCEC datasets using IPA. In Table We developed a CNN model called DEGnext to classify UR and DR genes from the DEGs of TCGA RNA-seq cancer datasets. We established that transfer-learning combined with the DEGnext model made the model effective in classifying UR and DR genes from untrained datasets. We compared the performance of DEGnext with 5 other ML methods, and DEGnext is competitive in terms of accuracy, sensitivity, specificity, F-measure, MCC, and ROC scores. In particular, for general learning, in Fig. p-values and q-values. Similarly, for biological pathways, we found that the predicted UR and DR genes were enriched in pathways associated with breast cancer, such as the ILK pathway and the Rho GTPase signaling pathway. Pathways mapped from the predicted UR and DR genes of the UCEC dataset also play significant roles in carcinogenesis of cervical cancer such as PTEN signaling and Ephrin receptor signaling pathways.We validated the biological enrichment of the predicted UR and DR genes from the BRCA and UCEC datasets in terms of GO and pathway enrichment. We found that the predicted UR and DR genes were enriched with GO terms related to cancer with significant The proposed CNN model, DEGnext provides a novel approach for prediction of UR and DR genes from both trained and untrained datasets using both logFC values and disease-related biological knowledge. The downstream analysis of the predicted UR and DR genes has provided insights into the underlying mechanisms and aided in the identification of the prime regulators of carcinogenesis of breast cancer and uterine cancer. Therefore, through the prediction and classification of DEGs, DEGnext may aid in the exploration of potential biomarkers of a disease from other RNA-seq datasets.Although each of the 17 downloaded datasets had different sample sizes, there were 60,483 mRNA transcripts in each dataset. We used the following queries and data categories: Transcriptome Profiling; data type: Gene Expression Quantification; workflow type: HTSeq-Counts; sample type: Primary Tumor and Solid Tissue Normal; legacy: FALSE, to download the cancer datasets from the TCGA portal. First, we prepared the dataset to represent it as an expression matrix with genes as rows and samples as columns. Out of 60,483 mRNA transcripts, we obtained 56,493 mRNA transcripts, which were mapped to the human genome .TCGAanalyze_Preprocessing with a gene expression cut off threshold = 0.6 and found that 56,493 number of mRNA transcripts above this threshold were within the inter-quartile range.We preprocessed the gene expression data using We mapped the ENSEMBL identifiers of the mRNA transcripts, and kept only those mRNA transcripts which had valid HGNC gene symbols. From the 56,493 mRNA transcripts, we found that 37,614 genes had valid HGNC symbols.TCGAanalyze_Normalization to adjust several gene-level effects, such as GC-content and sequencing depth. Internally, TCGAanalyze_Normalization utilizes the EDASeq package [In RNA-seq data, different sample conditions have different sequencing depths and RNA compositions, which may cause complications in downstream analysis . We perf package  to perfo package . 8686 geTCGAanalyze_Filtering, with a quantile cut off 0.25 and found 6514 filtered genes (FG) above this threshold. We used a strict parameter cut off for preprocessing and filtering of the genes before DE analysis, because the main objective of our approach is to find significant predicted UR and DR genes related to cancer progression.For each of the 17 datasets, we filtered the genes using TCGAanalyze_DEA to perform DE analysis on the filtered genes (FG) across normal versus tumor conditions with a false discovery rate (FDR) cutoff 0.01, yielding significant labeled DEGs (SDEGs) for each cancer dataset. Out of the 6514 FGs, the non-significantly differentially expressed for FDR cutoff of 0.01 were labeled “2” as neutral genes.We used The number of SDEGs for each dataset was different as shown in Table We input the SDEGs for each dataset into the IPA tool  to checkWe split P and Q data as shown in Fig. We performed two experiments: one for general training and testing and the other experiment for transfer learning. We leveraged the knowledge  learned from the previously trained DEGnext model to predict UR and DR genes from 8 untrained datasets.For general learning, we tested the effectiveness of DEGnext model to classify UR and DR genes from respective bio-test data (T3) of all 17 datasets.We checked the generalizability of DEGnext by using transfer learning of the significant feature-maps into bio data (Q) of 8 untrained datasets.We compared the performance of our DEGnext against five ML methods, DTC, KNC, RFC, SVC, and XGBoost in terms of accuracy, recall, precision, F-measure, MCC, and ROC scores.We tested the robustness of DEGnext by augmenting the datasets with seven levels of Gaussian noise data  and compared results with other ML methods.We used two tools, ToppGene Suite and IPA, for Gene Ontology (GO) and pathway enrichment analysis of the predicted UR and DR genes for BRCA and UCEC datasets, respectively.In Table We used TCGABiolinks R package  to downlThe DEGnext is a CNN model proposed to predict UR and DR genes from RNA-seq cancer datasets. We implemented the CNN model using Pytorch in Python DL platform , as showThe input to the model is a 1D input vector : In the first experiment, we used all 17 cancer datasets to train, fine-tune and test the corresponding bio-test data (T3) from each dataset. Since the non-bio train data (P) has three labels ‘0’, ‘1’, and ‘2’, this training is for a three-class problem. For the first-level of training, DEGnext runs for 50 epochs with a batch size of 256 and it uses CrossEntropyLoss as a loss function and optim.Adam as an optimizer to compute the cross entropy loss between the output  of each dataset. The second level of training incorporates both prior disease-related biological knowledge and log2FC estimates (sample variance) of the data to the CNN model, which enables capture of non-linear gene expression patterns and enhances prediction performance of the model in determining UR and DR genes. The major advantage of our CNN model is that it allows performing very efficient transfer learning by reusing the feature-map signatures learned from the trained model.For the second level of training, we use fine-tune data (F1) on the best model from first level of training for each dataset. Since fine-tune data (F1) have ‘0’ and ‘1’ labels, the second level of training is a two-class problem. Here, we used the Experiment 2 (Transfer learning): For the second experiment, we divided 17 datasets into two groups: training datasets  and testing datasets . The training datasets are larger in size than the testing datasets. The testing datasets comprised of 100 CrossEntropyLoss as a loss function and optim.Adam as an optimizer, with a batch size of 256 to train the model on the 9 training datasets one after another. For fine-tune, all we needed to do was to customize and modify the output layer L5 and remove the final softmax layer to classifying the DEGs as ‘0’ or ‘1’. We used the BCEWithLogitsLoss loss function to fine-tune the model again with the fine-tune data (F1) for all 9 training datasets. For testing, we did not retrain the model, but instead used the pretrained model to predict UR and DR genes from all 8 testing datasets.Additional file 1. Comparison of ROC scores of DEGnext with other ML methods for general learning and transfer learning.Additional file 2. Readme.Additional file 3. Source codes for general learning.Additional file 4. Source codes for transfer learning.Additional file 5. Source codes for test case.


In order to correctly decode phenotypic information from RNA-sequencing (RNA-seq) data, careful selection of the RNA-seq quantification measure is critical for inter-sample comparisons and for downstream analyses, such as differential gene expression between two or more conditions. Several methods have been proposed and continue to be used. However, a consensus has not been reached regarding the best gene expression quantification method for RNA-seq data analysis.In the present study, we used replicate samples from each of 20 patient-derived xenograft (PDX) models spanning 15 tumor types, for a total of 61 human tumor xenograft samples available through the NCI patient-derived model repository (PDMR). We compared the reproducibility across replicate samples based on TPM (transcripts per million), FPKM (fragments per kilobase of transcript per million fragments mapped), and normalized counts using coefficient of variation, intraclass correlation coefficient, and cluster analysis.Our results revealed that hierarchical clustering on normalized count data tended to group replicate samples from the same PDX model together more accurately than TPM and FPKM data. Furthermore, normalized count data were observed to have the lowest median coefficient of variation (CV), and highest intraclass correlation (ICC) values across all replicate samples from the same model and for the same gene across all PDX models compared to TPM and FPKM data.We provided compelling evidence for a preferred quantification measure to conduct downstream analyses of PDX RNA-seq data. To our knowledge, this is the first comparative study of RNA-seq data quantification measures conducted on PDX models, which are known to be inherently more variable than cell line models. Our findings are consistent with what others have shown for human tumors and cell lines and add further support to the thesis that normalized counts are the best choice for the analysis of RNA-seq data across samples.The online version contains supplementary material available at 10.1186/s12967-021-02936-w. RNA-sequencing (RNA-seq) has replaced gene expression microarrays as the most popular method for transcriptome profiling , 2. Vari. , while normalized count data using either DESeq2  for DESeq2 and  for TMM]. Furthermore, FPKM data had lower ICCg values than DESeq2 and TMM-normalized count data in the above four models. Those four models were the same models identified in hierarchical clustering using Euclidean distance whose replicates did not cluster with each other. These results indicate that the normalized count data were more reproducible across replicate samples, in the sense of having generally higher between-gene variance relative to the total variation (across genes and replicate samples) across PDX models.Figure m for each gene to examine the impact of each quantification measure on both within-model error variance (between replicate samples for the same gene) and between-model variance for each gene (model ICCm). Similarly, larger ICCm indicates that the replicate error variance is relatively small compared to the biological differences across PDX models for each gene.We also calculated ICCm when using different RNA-seq quantification measures on all 28,109 genes. Normalized count from DESeq2 or TMM, as well as FPKM performed similarly well with median ICCm around 0.69, while TPM performed the worst with median ICCm of 0.64. These results indicate that the normalized count data were more reproducible across replicate samples, in the sense of having generally higher between-model variance relative to the total variation (across models and replicate samples) across genes.Figure We further checked whether Z-score transformation or an additional normalization step would help to resolve the potentially problematic issue of TPM data, especially for PDX model 475296-252-R. We found that even after Z-score normalization of TPM data, the replicate samples for PDX model 475296-252-R remained separated following hierarchical clustering —the model for which replicate samples did not cluster with each other in the hierarchical cluster analysis . Among them, while three out of the five models showed minor differences <  5%) in CVs between the replicates, two of the models still displayed relatively high differences between replicates  and TMM normalized count data generally performed better than the other quantification measures.Each normalization method comes with a set of assumptions; thus, the validity of downstream analysis results depend on whether the experimental setup is congruent with the assumptions . For ins. . TPM and FPKM/RPKM may be acceptable to use if the ranks of genes in each sample are used, as opposed to their quantitative expression values. For example, The Broad Institute’s gene set enrichment analysis (GSEA) tool allows users to perform pathway analyses by uploading single rank-based gene list , 45. FinAs described above, each normalization method is based on its own assumptions. When the assumptions are violated, the method could fail . In thisOur results strongly support the notion that normalized count data are the preferred quantification measure for between-sample analysis of RNA-seq data generated from tumors grown in PDX models. These quantifications exhibit greater comparability among replicate samples and are more robust to technical artifacts; hence, they should be the first choice whenever cross-sample comparisons are of interest. Further data transformations or normalizations on TPM-level data are not able to resolve potential issues inherent in TPM quantifications. We hope that our findings will promote the use of normalized count data instead of TPM or FPKM/RPKM in PDX studies using RNA-seq to avoid inaccurate results arising from sub-optimal gene expression quantification.Additional file 1: Table S1. Details of the patient-derived xenograft samples used in this study . Table S2. Summary statistics for CV values including interquartile range for different quantitative measures. Table S3A. Percentage of transcripts representing each of the top five most abundant genes in four PDX models whose TPM data had the highest median CV values. Table S3B. Percentage of transcripts representing each of the top five most abundant genes in five PDX models whose TPM data had the lowest median CV values. Figure S1. (A) Hierarchical clustering of 61 PDX samples using TMM normalized gene-level count data. (B) Hierarchical clustering of 61 PDX samples using FPKM data. (C) Hierarchical clustering of 61 PDX samples using Z-score on TPM-level data. Distance metric 1-Pearson correlation was used to generate the dendrogram in each right panel and Euclidean distance was used for the dendrogram in each left panel. Discordant models are highlighted with different color labels. Figure S2. Maximum distance (1-Pearson correlation) between replicate samples for the four PDX models with high median CV values using different gene expression quantification measures. Figure S3. (A) Hierarchical clustering of 61 PDX samples using TMM normalized TPM data. Distance metric 1-Pearson correlation was used to generate the dendrogram in the right panel and Euclidean distance was used for the dendrogram in the left panel. (B) Pairwise scatter plots comparing normalized TPM values for all genes among replicates of PDX model 475296-252-R. The x- and y- axes are normalized log2 counts on all pairwise scatter plots. Plots along the diagonal represent the density of the respective variable. Figure S4. Bar plot of median CVs for gene expression levels from replicate samples of each PDX model using different quantification measures. Figure S5. (A) Pairwise scatter plots comparing TPM values for all genes between replicate samples of PDX model 695221-133-T. (B) Pairwise scatter plots comparing DESeq2 normalized count values for all genes between replicate samples of PDX model 695221-133-T. The x- and y-axes are normalized log2 counts on all pairwise scatter plots. Plots along the diagonal represent the density of the respective variable. Figure S6. (A) Pairwise scatter plots comparing TPM values for all genes between replicate samples of PDX model 821394-179-R. (B) Pairwise scatter plots comparing DESeq2 normalized count values for all genes between replicate samples of PDX model 821394-179-R. The x- and y-axes are normalized log2 counts on all pairwise scatter plots. Plots along the diagonal represent the density of the respective variable. Figure S7. (A) Pairwise scatter plots comparing TPM values for all genes between replicate samples of PDX model K98449-230-R. (B) Pairwise scatter plots comparing DESeq2 normalized count values for all genes between replicate samples of PDX model K98449-230-R. The x- and y-axes are normalized log2 counts on all pairwise scatter plots. Plots along the diagonal represent the density of the respective variable. Figure S8. (A) Pairwise scatter plots comparing TPM values for all genes between replicate samples of PDX model 983718-287-R. (B) Pairwise scatter plots comparing TPM values for all genes between replicate samples of PDX model 884782-307-R. The x- and y-axes are normalized log2 counts on all pairwise scatter plots. Plots along the diagonal represent the density of the respective variable.


Standard preprocessing of single-cell RNA-seq UMI data includes normalization by sequencing depth to remove this technical variability, and nonlinear transformation to stabilize the variance across genes with different expression levels. Instead, two recent papers propose to use statistical count models for these tasks: Hafemeister and Satija  recommend using Pearson residuals from negative binomial regression, while Townes et al.  recommend fitting a generalized PCA model. Here, we investigate the connection between these approaches theoretically and empirically, and compare their effects on downstream processing.We show that the model of Hafemeister and Satija produces noisy parameter estimates because it is overspecified, which is why the original paper employs post hoc smoothing. When specified more parsimoniously, it has a simple analytic solution equivalent to the rank-one Poisson GLM-PCA of Townes et al. Further, our analysis indicates that per-gene overdispersion estimates in Hafemeister and Satija are biased, and that the data are in fact consistent with the overdispersion parameter being independent of gene expression. We then use negative control data without biological variability to estimate the technical overdispersion of UMI counts, and find that across several different experimental protocols, the data are close to Poisson and suggest very moderate overdispersion. Finally, we perform a benchmark to compare the performance of Pearson residuals, variance-stabilizing transformations, and GLM-PCA on scRNA-seq datasets with known ground truth.We demonstrate that analytic Pearson residuals strongly outperform other methods for identifying biologically variable genes, and capture more of the biologically meaningful variation when used for dimensionality reduction.The online version contains supplementary material available at (10.1186/s13059-021-02451-7). The standard preprocessing pipeline for single-cell RNA-seq data includes sequencing depth normalization followed by log-transformation , 2. The Here, we show that the model used by Hafemeister and Satija  has a tohttp://github.com/berenslab/umi-normalization. Analytic Pearson residuals will be included into upcoming Scanpy 1.9 [Our code in Python is available at anpy 1.9 .g takes up a certain fraction pg of the total amount nc of counts in cell c [Xcg are then modeled as Poisson or negative binomial (NB) [μcg=pgnc without zero-inflation [A common modeling assumption for UMI or read count data without biological variability is that each gene n cell c , 6–10. Tial (NB)  samples nflation : 1\docuThe Poisson model has a maximum likelihood solution see “” that caFor the negative binomial model this holds only approximately. Using this solution, the Pearson residuals are given by θ→∞ gives the Poisson limit. The variance of Pearson residuals is, up to a constant, equal to the Pearson χ2 goodness-of-fit statistic [correspondence analysis [where tatistic  and quantatistic , singulaanalysis –18, a meanalysis .Hafemeister and Satija  suggesteg in this dataset, the authors fit an independent NB regression Hafemeister and Satija  used theθg is the gene-specific overdispersion parameter, βg0 and βg1 are the gene-specific intercept and slope. The natural logarithm follows from the logarithmic link function that is used in NB regression by default. The original paper estimates βg0 and βg1 using Poisson regression and then uses the obtained estimates to find the maximum likelihood estimate of θg. The resulting estimates for each gene are shown in Fig. Here, mentclass2pt{minimρ=−0.91), especially for weakly expressed genes  is used as predictor. Not only does this suggest an alternative, simpler parametrization of the model, but it also explains why Hafemeister and Satija [nc)= ln(nc)/ ln(10) instead of ln(nc) as predictor, and so obtained ln(10)≈2.3 as the average slope.We see that under this assumption, the slope d Satija  found thnc) as predictor with a fixed slope of 1, a so-called offset . Also, many of the weakly expressed genes may be filtered out during preprocessing in actual applications. We note that large errors in NB overdispersion parameter estimates have been extensively described in other fields, with simulation studies showing that estimation bias occurs especially for low NB means, small sample sizes, and large true values of θ [The above arguments show that the overdispersion parameter estimates in Hafemeister and Satija  for geneues of θ –23, i.e.θ value shared between all genes. Of course, any given dataset would be better fit using gene-specific values θg. However, our goal is not the best possible fit: We want the model to account only for technical variability, but not biological variability, e.g., between cell types; this kind of variability should manifest itself as high residual variance.To avoid noisy and biased estimates, we suggest to use one common θ value from a biologically heterogeneous dataset such as PBMC, we think it is more appropriate to estimate the technical overdispersion using negative control datasets, collected without any biological variability [θ∈{10,100,1000,∞}. For the simulations, we used the empirically observed sample sizes and sequencing depths. We found that across different protocols, negative control data were consistent with overdispersion θ≈100 or larger , but likely overdispersion parameter values (θ≈100) were large enough to make the Poisson model acceptable in practice [Rather than estimating the iability . We analmentclass2pt{minimmentclass2pt{minimpractice , 25. A ppractice .Both Hafemeister and Satija  and Townθ=100 led to several high-expression genes selected as biologically variable that would not be selected with a lower θ , but overall, using θ=10,θ=100, or even the Poisson model with θ=∞ led to only minor differences  followed by one of the square-root transformations has been advocated for UMI data processing [It is instructive to compare the variance of Pearson residuals to the variance that one gets after explicit sequencing depth normalization followed by a variance-stabilizing transformation. For Poisson data, the square root transformation terature , such ast}2x+3/8  and the nt}x+x+1 . Normaliocessing , 31.Tubb1 have low average expression (because platelets are a rare population in the PBMC dataset) and do not show high variance after any kind of square-root transform (another example was given by the B-cell marker Cd79a). At the same time, Pearson residuals correctly indicate that these genes have high variance and are biologically meaningful  was very similar to the Fano factor of counts after median normalization -transform as well as by the variance of deviance residuals as suggested by Townes et al. [Lastly, gene selection by the widely-used log) are known to introduce spurious variation into the distribution of normalized counts [We applied these methods to the full PBMC dataset n=2 million cells  argued td counts . For theHowever, on closer inspection, embeddings based on Pearson residuals consistently outperformed the other two. For example, while the Pearson residual embeddings clearly separated fine cell types in the full-retina dataset , the squEmbedding the organogenesis dataset  using PeTo quantify the performance of dimensionality reduction methods, we performed a systematic benchmark using the Zhengmix8eq dataset with known ground truth labels  Fig. 5)5). This The pipeline that used analytic Pearson residuals for both gene selection and data transformation outperformed all other pipelines with respect to cell type classification performance. In contrast, popular methods for HVG selection  comFor this dataset, not using gene selection at all performed similarly well to HVG selection using Pearson residuals Fig. c, but inThe studied normalization pipelines differ in both space and time complexity. UMI count data are typically very sparse  and can be efficiently stored as a sparse matrix object. Sequencing depth normalization and square-root or log-transformation do not affect the zeros, preserving the sparsity of the matrix, and PCA can be run directly on a sparse matrix. In contrast, Pearson residuals form a dense matrix without any zeros and so can take a large amount of memory to store (4.5 Gb for the PBMC dataset). For large datasets this can become prohibitive (but note that a smart implementation may be able to avoid storing a dense matrix in memory ). In conComputational complexity can be greatly reduced if gene selection is performed in advance. After selecting 1000 genes, Pearson residuals do not require a lot of memory (0.3 Gb for the PBMC dataset) and so can be conveniently used. Note that the Pearson residual variance can be computed per gene, without storing the entire residual matrix in memory. GLM-PCA, however, remained slow even after gene selection . The authors also refer to the bulk RNA-seq literature, where it has been observed that the overdispersion parameter grows monotonically with gene expression [θ is assumed to be constant or is allowed to vary between genes, has very little effect on the resulting Pearson residuals.Following our manuscript, scTransform was updated to scTransform v2 and now uses the offset model formulation . At the lmGamPoi  to estimpression , 48, 49.A parallel publication  suggesteOn the practical side, we showed that Pearson residuals outperform other methods for selecting biologically variable genes. They are also better than other preprocessing methods for downstream analysis: in a systematic benchmarking effort, we demonstrated that Pearson residuals provide a good basis for general-purpose dimensionality reduction and for constructing 2D embeddings of single-cell UMI data. In particular, they are well suited for identifying rare cell types and their genetic markers. Applying gene selection prior to dimensionality reduction reduces the computational cost of using Pearson residuals down to negligible. We conclude that analytic Pearson residuals provide a theory-based, fast, and convenient method for normalization of UMI datasets.The log-likelihood for the model defined in Eqs. 2can be, up to a constant, written as p(x)=exeμ−/x!. Taking partial derivatives with respect to nc and pg and setting them to zero, one obtains where we used the Poisson density This is a family of solutions. Setting This derivation does not generalize to the negative binomial model with density θ), up to a constant, is where the log-likelihood . It follows that for the Poisson model deviance residuals  are giveθ, the deviance residuals follow from Eq. Similarly, for the negative binomial model with fixed θ→∞. When computing deviance residuals, we estimated It is easy to verify that this formula reduces to the Poisson case when n cells containing a biologically distinct rare population P of size m≪n. Let this population have a marker gene with expression following Poisson(λ) for the cells from P, and zero expression for all n−m remaining cells. For simplicity we assume the Poisson model here, and further assume that all cells have the same sequencing depth.Clipping Pearson residuals to d Satija  is needementclass2pt{minimλm/n and so the expected Pearson residual value for this gene for the cells from P is The expected average expression of this gene is λ>m, i.e., when the population P is either very small or has very large UMI counts. For example, a population of 10 cells having a marker gene with the within-population mean expression of 20 UMIs, will result in clipped residuals, as if the within-population mean expression were ∼ 10 UMIs. This may have a large effect on the leading principal components (even PC1) if the data contain a very small number of cells with strong marker gene expression.With the clipping threshold μ in half of the cells and is upregulated by a factor of two in the other half of the cells. Then its expression mean is 1.5μ, and the Pearson residuals are close to μ. This makes sense because for higher-expressed genes there is more statistical certainty about over-Poisson variability, but at the same time highlights that Pearson residuals do not aim to estimate the underlying (log-)expression, unlike, e.g., Sanity [It is instructive to observe the effect Pearson residuals have on genes that have the same variance of log-expression but different expression means. Consider a gene that has expression , Sanity .https://github.com/berenslab/umi-normalization.Used datasets are listed in Table For gene selection using sqrt(CPMedian), Pearson residuals, and deviance residuals, we applied the respective data transformation and used the variance after transformation as selection criterion. For Seurat and Seurat_v3 methods, we used the respective Scanpy implementations. In brief, these two methods regress out the mean-variance relationship, and return an estimate of the “excess” variance for each gene , 35. Forx+1) transform; log(CPMedian + 1) + standardization — same as log(CPMedian + 1), but followed by centering each gene at mean zero and scaling it to unit variance; log(CPM + 1) — normalization by sequencing depth, followed by scaling by one million (“counts per million”), followed by log(x+1) transform. Pearson residuals were computed with Eq. We used the following abbreviations to denote data transformations: sqrt(CPMedian) — normalization by sequencing depth, followed by scaling by the median depth across all cells (“counts per median”), followed by the square-root transform; log(CPMedian + 1) — normalization by sequencing depth, followed by scaling by the median depth across all cells, followed by log or Pearson residuals (θ=10). For these two cases, we had to increase the penalty to 10. On the organogenesis dataset, the Python implementation did not converge within reasonable time, so for this dataset, we resorted to the R implementation. It uses a different optimization method and employs stochastic minibatches. All reported GLM-PCA results for this dataset are for batchsize 10,000, as batchsizes 100 and 1,000 (default) resulted in considerably longer runtimes. Because the R implementation does not support NB GLM-PCA with fixed theta, for this dataset, we used GLM-PCA with jointly fit Further, we used three variants of GLM-PCA to transform raw counts and reduce dimensionality down to 50 in a joint step: Poisson GLM-PCA, negative binomial GLM-PCA with estimation of single overdispersion parameter s et al. , the autθ used θ=100. Whenever gene selection was performed prior to a data transformation that required sequencing depths, we computed those depths using the sum over selected genes only.Unless otherwise stated, all residuals and GLM-PCA with fixed nip), where ni is the sequencing depth of the ith selected cell , and expression fraction p was set to 0.001.We used the Zhengmix8eq dataset with known ground truth labels obtained by FACS-sorting of eight PBMC cell types , 43. Then−1 cells. This resulted in a class prediction for each cell based on the majority vote of its k=15 neighboring cells. We quantified the performance of this prediction by computing the macro F1 score . We used the sklearn implementations for kNN classification and the F1 score [We then applied the 70 normalization pipelines shown in Fig. F1 score .All runtimes given in Table n/100, where n is the sample size [All t-SNE embeddings were made following recommendations from a recent paper  using thple size . For theple size .Additional file 1 Contains supplementary Figures S1–S7.Additional file 2 Review history.


Single-cell sequencing technology can address the amount of single-cell library data at the same time and display the heterogeneity of different cells. However, analyzing single-cell data is a computationally challenging problem. Because there are low counts in the gene expression region, it has a high chance of recognizing the non-zero entity as zero, which are called dropout events. At present, the mainstream dropout imputation methods cannot effectively recover the true expression of cells from dropout noise such as DCA, MAGIC, scVI, scImpute and SAVER.In this paper, we propose an autoencoder structure network, named GNNImpute. GNNImpute uses graph attention convolution to aggregate multi-level similar cell information and implements convolution operations on non-Euclidean space on scRNA-seq data. Distinct from current imputation tools, GNNImpute can accurately and effectively impute the dropout and reduce dropout noise. We use mean square error (MSE), mean absolute error (MAE), Pearson correlation coefficient (PCC) and Cosine similarity (CS) to measure the performance of different methods with GNNImpute. We analyze four real datasets, and our results show that the GNNImpute achieves 3.0130 MSE, 0.6781 MAE, 0.9073 PCC and 0.9134 CS. Furthermore, we use Adjusted rand index (ARI) and Normalized mutual information (NMI) to measure the clustering effect. The GNNImpute achieves 0.8199 (ARI) and 0.8368 (NMI), respectively.In this investigation, we propose a single-cell dropout imputation method (GNNImpute), which effectively utilizes shared information for imputing the dropout of scRNA-seq data. We test it with different real datasets and evaluate its effectiveness in MSE, MAE, PCC and CS. The results show that graph attention convolution and autoencoder structure have great potential in single-cell dropout imputation. With the development of single-cell RNA sequencing (scRNA-seq) technology, it provides an easy way to process tens of thousands of single cells in parallel while providing gene expression data with single-cell-level resolution –3. The tThe scRNA-seq technology can produce single-cell-level resolution data. As a result of defects such as low capture rate and low sequencing depth, the sequencing library data contains a lot of noise , 10.Compared with the next-generation sequencing data, scRNA-seq data usually contains a lot of zero expressions. These zero expressions can arise in two ways: One is that the genes are not expressed in the corresponding cells and the other is that some genes with low expression cannot be detected due to technical limitations. These events are called dropout events . There aIn the downstream analysis of scRNA-seq data, the dimensional reduction and unsupervised clustering are always be used to infer cell development trajectories and identify rare cell clusters , 15. HowRecently, many methods have been developed to impute dropout in scRNA-seq data. For example, MAGIC  is basedTherefore, we propose a novel structure neural network named GNNImpute, which is an autoencoder structure network that uses graph attention convolution. By building a graph from the scRNA-seq data, GNNImpute uses graph attention convolutional layer to make a targeted selection of similar neighboring nodes. Then, it aggregates these similar neighboring nodes. The nodes in the graph can continuously transmit messages along the edge direction until stability is reached. In this way, GNNImpute enables the expression of the cells in the same tissue area to be embedded in low-dimensional vectors through the autoencoder structure. GNNImpute can not only capture the co-expression patterns between similar cells but also remove sequencing technical noise from imputing dropout, which improves the downstream analysis of scRNA-seq data.In scRNA-seq data, each cell has its own expression profile, and the expression profile of each cell is different and unique. But cells from the same tissue or with the same function usually have similar features. Therefore, when a dropout event occurs in any cell, it can be recovered by the gene expression profile of similar cells.GNNImpute is a deep learning method based on a graph attention neural network. Different from MAGIC, GNNImpute introduces the attention mechanism that can assign weights to different similar cells according to attention coefficients. It can establish nonlinear relationships between genes by learning low-dimensional embedding of expressions through the autoencoder structure network. Compared with DCA, GNNImpute can learn the gene co-expression patterns of similar cells by aggregating information from multi-level neighbors. The co-expression patterns can help recover low-expressed genes. GNNImpute reduces the dropout noise and improves the gene expression profile of cells.The overall structure of GNNImpute is shown in Fig. GNNImpute uses the expression matrix of scRNA-seq data as input. As shown in Table In the data preprocessing, we use SCANPY  to filteK closest cells to construct graph edges. The K edges display a similar relation of cells. Through the above steps, we construct a cell-to-cell connection graph (K-nearest neighbor graph). We set the GNNImpute is a dropout imputation method based on a graph attention network. It is used to obtain gene expression from similar cells to recover the dropout event. In order to aggregate the cells with similar expressions, it is necessary to define a connection graph between the cells. In this graph, we use nodes to represent cells and we use edges to represent the similarity between cells. In this graph, cells can transmit information to adjacent cells. As shown in Fig. K cells with the most similar expression. We call these K cells the “first-level” neighbors. These K cells are adjacent to the origin cells. Similarly, there are also K cells in the “first-level” neighbors. We named the “second-level” cells. It doesn’t exist edges between origin cells and “second-level” cells, but the transferability of intermediate nodes can still indicate similarity. Figure After constructing the graph, all graph cells have k is the number of layers of graph convolution. W is the trainable weight. I is the identity matrix, which means add self-connections to the adjacency matrix. The degree matrix In order to aggregate the information of cells, we need a connection graph and graph attention convolutional layers. The essence of the graph convolutional layer is not to aggregate information around the original nodes, but aggregate nodes connected by edges. The calculation process of graph convolutional layers is as follows:i and node j, which is the gene expression profile of the cells. And j to cell i. a is the attention calculation formula. It is used to calculate the similarity of every two nodes. we use the dot product as the attention calculation formula to calculate the similarity. W is a shared weight matrix. It transforms the input features into more advanced features, so that each node can obtain sufficient expressive ability. We only calculate the attention coefficient of cell i and cell j, where i in the cell-to-cell connection graph. Further calculation of multiple independent attention coefficients can be extended to the multi-head attention mechanism. In this attention mechanism, the attention coefficients are combined by calculate the average values to stabilize the learning process. The formula is as follows:Through the superposition of multiple graph convolutional layers, the information aggregation of multi-order neighbors can be achieved. We use two-layer graph convolution in the encoder, and the output of the encoder is as follows:mentclass2pt{minimGNNImpute builds the model with an autoencoder (encoder and decoder). The input layer and output layer of the model have the same number of nodes. In the hidden layer, the nodes are much lower than encoder and decoder nodes. Different from the traditional autoencoder structure, we make an improvement on the GNNImpute encode layer. We use graph attention networks in the GNNImpute encode layer instead of linear networks. As shown in Fig. Further, GNNImpute adds dropout layers to combat the over-fitting problem of the model. GNNImpute introduces a multi-head attention mechanism to achieve targeted selection when aggregating neighbor nodes. This multi-head attention mechanism can stabilize the learning process and provide robustness for the model. It is noted that GNNImpute uses a semi-supervised learning method to recover from dropout events. The advantage of semi-supervised learning is that some labeled cells can provide soft labels for many other unlabeled cells, and it can help the model recover from dropout events more accurately.The model can learn the potential features by minimizing the error between the reconstructed expression matrix and the original expression matrix. Meanwhile, the hidden layer can capture the distribution of the matrix and ignore invalid changes in the low-dimensional environment.Because the dropout event is random, there are few dropout benchmarks. Therefore, we used a fair measurement method , 29. ThiIn the model training phase, we divide the PBMC dataset according to the ratio of 6:2:2. There are 1706 cells in the training set, 569 cells in the validation set, and 568 cells in the test set. Training set used to train the model, the validation set would be used to test the trained model, and the test set would evaluate the final model. The total parameters of GNNImpute will be adjusted according to the size of the dataset. When using the PBMC dataset, the total parameters of the model are 26.75 M. The loss function of the model is set to the mean square error loss function. The optimizer is Adam, and the learning rate is 0.0001. The maximum number of iterations is set to 3000. If the loss value of the validation set does not decrease in 200 consecutive iterations, it will interrupt training early. The training processes of the model are shown in Fig. In the experiment, we use four metrics to measure the imputation ability of GNNImpute with the other four methods. The four metrics are mean square error (MSE), mean absolute error (MSE), Pearson correlation coefficient (PCC) and Cosine similarity (CS), respectively. MSE and MAE are used to show whether the imputed gene expression values are the same as the labels. PCC and CS are used to measure whether the express trend of the imputed matrix is consistent with the raw matrix. In downstream data analysis, we employ the Adjusted rand index (ARI) and Normalized mutual information (NMI) to measure the clustering results.MSE:MAE:PCC:A and B represent the gene expression profile of the cell after imputation and the real gene expression profile of the cell respectively. And they are represented in the form of vectors. CS:Human Frozen Peripheral Blood Mononuclear Cells (PBMCs), which from 10X GENOMICS. It contains 2900 cells and 32738 genes.Mouse Brain cells published by Campbell (GSE93374). It uses Drop-seq technology to perform single-cell analysis on brain cells of adult mice, which contains 21,086 cells and 26,774 genes.Mouse Brain cells published by Chen (GSE87544). It is the diversity analysis of mouse hypothalamic cells, which contains 14,437 cells and 23,284 genes.Mouse embryo cell analysis published by Klein (GSE65525). It contains 2717 cells and 24,021 genes.We use four different real datasets in experiments. The real datasets list as following: In order to validate the dropout imputation performance of different methods, we compare GNNImpute results with other five methods, including DCA, MAGIC, scVI, scImpute and SAVER. DCA uses a zero-inflated negative binomial distribution model to denoise the autoencoder network. This denoising network can solve the problem of count distribution, over-dispersion and sparsity. MAGIC is an imputation method based on Markov affinity-based graph, which imputes dropout values by sharing information among similar cells. scVI can capture the basic low-dimensional structure in the scRNA-seq data by introducing a robust latent variable model, which can eliminate the noise in the data. scImpute is a statistical method that can automatically identify possible dropout events and recover them. It can also exclude outliers without introducing new bias. SAVER uses regularized regression prediction and empirical Bayesian methods to recover the gene expression profile in noisy and sparse data. We conduct experiments on four single-cell sequencing datasets of humans and mice. To illustrate the imputation performance of methods, we use dropout recovery index, clustering, robustness to evaluate the results.By randomly masking the expression matrix on the four real datasets, we can obtain positive and negative training data. We compare the imputation performance of GNNImpute with the other five imputation methods using four real datasets. Figure Overall, In the Fig. The purpose of imputation is to improve the downstream analysis of scRNA-seq data. Therefore, we use clustering results to evaluate the downstream analysis. We use two metrics  to measure the performance of cell clustering after imputation.In the clustering analysis, we used the data published by Klein. They analyzed mouse embryonic stem cells, revealing in detail the population structure and the heterogeneous onset of differentiation after leukemia inhibitory factor (LIF) withdrawal. The cluster labels are determined by the intervals of LIF withdrawal . The t-distributed stochastic neighbor embedding (t-SNE) algorithm is used to reduce the dimension of the expression matrix. And it can realize the visual analysis of clustering. In Fig. By calculating the gene heat map of all cells in the imputed expression matrix, the results can also be visualized to determine which methods can improve the downstream analysis of scRNA-seq data. Because the PBMC dataset does not have real cluster labels, we use Leiden algorithm to calculate pseudo labels. Then we find highly differentiated marker genes in each cluster by t-test based on the pseudo-labels. Finally, we select 50 marker genes most relevant to cluster classification in the PBMC dataset to measure the performance of GNNImpute and other five methods. Figure Next, we evaluate the ability of the imputation method for scRNA-seq data under different dropout rates. The dropout rates are 10%, 20%, 30%, 40%, 50%, and 60%, using PBMC dataset with random mask expression matrix.Figure GNNImpute uses a semi-supervised learning method to train the model and learn the dropout knowledge. The advantage of semi-supervised is that it can use only a small amount of labeled data and a large amount of unlabeled data for training, which greatly reduces the requirements for manually labeling data. In this experiment, we use 80 ~10% data with labels for model training. Even if only 10% labeled data is used, the model still shows great imputation performance, as shown in Fig. To further evaluate the performance of GNNImpute, we evaluate the performance in simulated data. Following the previous work, we used the Splatter package  to generIn order to verify the effectiveness of the attention mechanism, we specially added experiments to evaluate our method and GCN architecture model (without attention). As shown in Tables On the high level, imputing the dropout of scRNA-seq data is a process of denoising the expression matrix data of the raw scRNA-seq library. Many existing approaches show that deep learning methods, especially autoencoders, can effectively denoise data.Our GNNImpute method extends this high-level approach to the case of Non-Euclidean spatial data like cell graphs. By reconstructing the expression matrix of scRNA-seq data, GNNImpute can establish a learning mechanism between the input and output of the model. It allows our model to capture non-linear relationships between genes and make better utilization of data.Usually, the dropout imputation performance is affected by the sparseness of scRNA-seq data. For noise data, it is usually more difficult to impute dropout values. However, GNNImpute shows excellent performance compared with other methods. GNNImpute compensates for the lack of low expression intensity of some genes by aggregating the features information of similar cells. Meanwhile, it can recover the dropout events in the scRNA-seq data and remain the specificity between cells to avoid excessive smoothing of expression.Compared with other dropout imputation methods, GNNImpute has great adaptability for addressing the different sizes of datasets . The difficulty in processing large datasets is that there are many cell types, and each cell contains a lot of gene information. Since GNNImpute uses a neural network model, it can capture important features from all information, and then reduce the dimensionality to ignore unimportant features.Moreover, GNNImpute is a semi-supervised learning model, which does not require manually labeled data. It can be trained with few labeled data. This model only uses 10% of the dataset for training can still achieve great results.In this paper, a novel imputation method based on graph attention convolution is proposed, which is a semi-supervised learning method using an autoencoding structure network. GNNImpute focuses on determining the similarity between cells and constructs a connection graph to capture the features of similar cells. This method also introduces an attention mechanism of weighted neighbor nodes to select the cell node with the most useful features information. In the experiments of four datasets, the performance of GNNImpute is better than other existing methods for four metrics of MSE, MAE, PCC and CS. When we explore the limits of GNNImpute, we find that it cannot provide the interpretability of cell clusters. Future investigations will focus on how to make GNNImpute more explainable.


Liquid chromatography-mass spectrometry-based metabolomics studies are increasingly applied to large population cohorts, which run for several weeks or even years in data acquisition. This inevitably introduces unwanted intra- and inter-batch variations over time that can overshadow true biological signals and thus hinder potential biological discoveries. To date, normalisation approaches have struggled to mitigate the variability introduced by technical factors whilst preserving biological variance, especially for protracted acquisitions. Here, we propose a study design framework with an arrangement for embedding biological sample replicates to quantify variance within and between batches and a workflow that uses these replicates to remove unwanted variation in a hierarchical manner (hRUV). We use this design to produce a dataset of more than 1000 human plasma samples run over an extended period of time. We demonstrate significant improvement of hRUV over existing methods in preserving biological signals whilst removing unwanted variation for large scale metabolomics studies. Our tools not only provide a strategy for large scale data normalisation, but also provides guidance on the design strategy for large omics studies. Mass spectrometry-based metabolomics is a powerful method for profiling large clinical cohorts but batch variations can obscure biologically meaningful differences. Here, the authors develop a computational workflow that removes unwanted data variation while preserving biologically relevant information. There are a number of large-scale cohort studies that have performed metabolomic analyses, such as the Consortium of Metabolomics Studies (COMETS)2, and the Framingham Heart Study (FHS)3.Liquid chromatography–tandem mass spectrometry (LC–MS/MS) is a preferred method of metabolomic acquisition given its high sensitivity and dynamic range. Typically, a range of metabolites can be separated on a single high performance LC column and their relative abundance quantified in MS/MS. This enables capture of fingerprints of specific biological processes that are critical in precision medicine applications such as studying complex metabolic diseases, and discovering new therapeutic targets and biomarkers4. Due to the data acquisition time of studies with a large sample size, prolonged study recruitment and potentially multiple samples at various time points for each participant, the data acquisition process may require the samples to be divided into multiple batches, and may span anywhere from months to years5. Signals often drift over extended periods due to multiple factors including buffer changes, pooled quality control (QC) sample solutions, instrument cleanliness, and machine scheduled maintenance6. Common intra-batch variations include changes in LC–MS/MS performance due to instrument-dependent factors such as component failure or inconsistency, and fouling of the column, LC or MS source. Common inter-batch variations include time-dependent instrument variations such as instrument cleaning, tuning, column change, or inconsistent sample preparation factors including change in equipment and operator. These technical factors have a substantial impact on downstream analytics and need to be appropriately accounted for to maximise the opportunity to identify true biological signals.Despite a rapid increase in the number of large-scale metabolomics studies, the normalisation of metabolomics data remains a key challenge7 and NormalyzerDE8). However, most of them adapt common normalisation methods developed for other omics platforms and do not account for signal drift across extended time or inter-batch variations which are distinct unwanted variations commonly observed in metabolomics studies. A number of metabolomics-specific normalisation methods have been developed in recent years. Many of these methods share conceptual similarities, such as regression-based methods12, machine learning approaches15 and other matrix factorisation approaches18. The common assumption behind the majority of these approaches is that the signal drift across extended time can be robustly estimated based on one specific sample alone, the pooled QC sample. This dependence on the pooled sample is suboptimal for several reasons: (1) even in the most careful hands, there will be batch variation in the thawing and extraction of the pooled sample; (2) long-term storage of plasma leads to variable stability of the metabolite pool, e.g. increases of amino acids, decreases in phosphatidylcholines19; (3) long-term storage leads to even greater changes in the protein pool20, which in turn variably influence metabolite levels; and (4) there are many logistical issues with long-term pool storage including refrigeration malfunction, sample loss, and sample exhaustion. Currently, there are no existing experimental designed strategies or methods to robustly account for normalisation over an extended period.Several workflows have been designed for analysing metabolomics data  was conducted based on the proposed sample arrangement strategy, and we utilise this to assess the normalisation on a number of criteria including retention of biological signal, low variability among replication, and reproducibility of results in comparison to other existing methods. More specifically, we compare against the performance of a number of recently developed and commonly used methods in popular pipelines when applied to large cohort studies, such as Support Vector Regression (SVR)5, Systematic Error Removal using Random Forest (SERRF)15, and Removal of Unwanted Variation based approaches23 , these are the (i) classical pooled QC samples, (ii) single sample replicates in each row of a batch from a random selection of non-replicated samples in previous rows, which we call ‘short replicates’, and (iii) five randomly selected non-short replicated samples from each batch replicated to the next batch, which we call ‘batch replicates’. Figure The classical pooled QC consists of a mixture of 10 μL of each of the 1002 samples, pooled together into a single tube. The pooled QC sample was aliquoted and frozen, and a fresh aliquot was thawed for each batch to minimise the impact of repeated freeze–thaw cycles. The spacing of the technical short replicates ~10 samples apart capture variation within a short time . This is a good measure of the variation of the metabolomics experiment. In contrast to pooled QC samples, where one sample is repeated many times, short technical replicates are duplicates of different samples; this increase in heterogeneity of samples for the estimation of unwanted variation is more robust compared to estimation with pooled QC. Finally, the batch replicates measure the variation that occurs across different batches. These replicates are typically 60–70 samples apart, capturing variation over a longer time period of 48–72 h.This design was used to generate data for a large metabolomics study consisting of 1002 individuals from the BioHEART-CT biobank and quantification of 100 metabolites per individual. After pre-processing, 53 metabolites were detected at adequate levels in plasma to be included in the analyses  as well as to minimise the overall number of pooled samples used. In practice, one could select any number of samples, subjects to the tray capacity of the autosampler as the unit for a batch. The notions of short and long (batch) replicates can be applied to any batch size to assess variation over a variety of distances.To enable effective data harmonisation across large cohort studies or across an extended period of time, we propose a hierarchical approach to correct for the unwanted variation between smaller subsets of batches individually, and to sequentially expand to the next set of batches. The two key components of the hRUV can be summarised as follows: (i) signal drift correction within batches with a robust smoother that captures the irregular patterns affecting each metabolite as illustrated in Fig. The signal drift within each batch was corrected using a robust smoother that captures the trends visible by run order Fig. . We expl2(n) RUV adjustments to deal with n batches, while the concatenating tree approach requires n − 1 RUV adjustments. The concatenating approach requires more computation than the balanced tree approach but has an advantage when future integration with new batches is necessary. For once the initial batches are normalised, the additional RUV adjustments are needed are only as many as the number of new batches. While the balanced tree approach is quicker for large n, if m new batches are introduced in the future, the data will require additional log2(n + m) RUV adjustments from the individual batch level. Both approaches of hRUV normalisation method have been evaluated to assess retention of biological signal, low variability among replication, reproducibility and distributional characteristics . Here, we applied both linear and non-linear smoothers to two sets of sample types; pooled QC samples, or all biological samples within a batch. In general, all four adjustment approaches  give adjusted values that have effectively removed any signal drift associated with experimental run order  between all sets of replicates, with lower values indicating better performance as the replicates should theoretically be identical. Figure To examine the extent to which our method removes only unwanted variation and retains known biological signals, we performed supervised machine learning to illustrate our ability to identify known biological signals for disease prediction. Here we have chosen hypertension as a response variable and performed supervised machine learning classification to detect hypertension status from metabolomics abundance. We anticipate that a normalisation method that retains biological signals has a higher classification accuracy. The differential expression (DE) analysis to identify corresponding biomarkers (DE metabolites) measures the interpretability of the signal.loessAllShort_batch_Hc method showed the best performance in prediction accuracy. This approach first adjusted for signal drift by fitting a loess line through all the samples and RUV corrected with short technical replicates within a batch, followed by applying RUV in a hierarchical fashion using batch technical replicates.We observed that the average accuracy of hierarchical based normalisation methods was generally higher compared to one-step methods Fig. . The loe24. While not all these metabolites were associated with hypertensive status in our cohort, we did see several biologically-plausible associations. The top-associated metabolite with hypertensive status in 1000 patients in hRUV normalised data was dimethylguanidino valeric acid (DMGV) , whose role in cardiometabolic disease we previously discovered27. The second-most associated metabolite, cAMP , is a long-recognised second messenger in central, peripheral, and essential hypertension30. Although less well-established, proline is reported to elevate blood pressure: dietary studies showing association of intake of proline with blood pressure31, and preclinical studies demonstrating pressor actions of proline32. The hydroxylated form of proline, trans-4-hydroxyproline, was the third-most associated metabolite with hypertension in our cohort . Together, these metabolites that are significantly associated with hypertension in our hRUV normalised data demonstrate the interpretability of the signal.There are many metabolites reported to be associated with hypertension, including vasoactive metabolites (tryptophan and its derivatives), nitric oxide related metabolites , microbiome-derived metabolites , TCA cycle intermediates, ketone bodies, and bile acidsIn general, we found that hRUV performed favourably in terms of maintaining strong biological signal and reducing unwanted variation such as signal drift and batch-specific noise in this large study Fig. . Our evaWe have investigated a number of parameters under the hRUV framework, including the various kinds of technical replication, types of negative control metabolite and types of hierarchical structure. We have explored different combinations of replication including pooled QC, inter-batch, and intra-batch replicates. We found that corrections with only pooled QC sample replicates over-estimates the unwanted variation and thus removes biological signals from the data  were placed at the start of the current batch. Ideally, we would expect to select samples with this strategy whose positions were evenly distributed across the batch, but it is possible by chance to select samples whose positions are from only the first half or only the second half of the previous batch  and all participants provided informed, written consent. Briefly, patients undergoing clinically indicated CT coronary angiogram for suspected coronary artery disease were recruited from multiple sites in Sydney, Australia. The inclusion criteria for BioHEART-CT study are patients aged 18 or older who had been referred for investigation of suspected CAD by CTCA, and who were willing and able to provide informed consent. Patients who were highly dependent on medical care who were unable to provide informed consent, as well as patients who were unwilling or unable to participate in ongoing follow-up were excluded. The BioHEART-CT discovery cohort utilised for this analysis included the first 1002 patients recruited to the BioHEART-CT study who had technically adequate CTCAs, sufficient stored blood samples for all planned biomarker discovery platforms, and who did not have a cardiac stent in situ or a prior history of coronary artery bypass surgery. Patients were advised to fast for 2 h prior to the procedure. 20–30 ml of venous blood was collected at the time of CTCA. After standard processing, plasma samples including replicates were aliquoted and stored at −80 °C until analysis.The samples used were from the BioHEART-CT discovery cohort, a study which has been described in detail previouslym/z was determined by Qtrap5500 (Sciex)34 driven by Analyst software . Elution window and mass transitions used to identify eluted metabolites were pre-determined using pure compound . The resulting solution was vortexed and spun at 14,000 rpm for 20 min to precipitate and remove plasma proteins. The metabolite containing supernatant (70 μl) was then transferred to a glass HPLC sample vial and resolved on an Agilent 1260 Infinity HPLC System with Atlantis Silica HILIC Column,  while BioHEART-CT is an ongoing multi-platform imaging and biomarker discovery cohort. The results from initial analyses are currently under peer review with additional studies and analyses planned or underway. The data reported in this study is from the discovery cohort and has been approved by the study investigators. The current analysis is in line with this original approval and consent. BioHEART is a registered Australian and New Zealand Clinical Trial (ACTRN12618001322224).35 [version 4.0.3]. The replicate design is available as a function ‘expDesign’ in the hRUV package, and also in our shiny application http://shiny.maths.usyd.edu.au/hRUV.For each batch, three pooled QC samples were run to fine adjust the acquisition setting of the HPLC/MASS-Spec system, then a single pooled QC sample was repeated after every 10 runs. For each row of the 11 samples  from the 12th sample onwards (second row), we randomly selected a single biological sample from the previous row to be replicated at a random position in a current row (short replicate). For each batch, after the first three pooled QC samples, a random selection of 5 biological samples from the previous batch was repeated (batch replicate) and short replicates are embedded at each row. All randomisation was performed using the sample function in RTM version 3.0.3 (SCIEX), with manual peak integration was performed when necessary. This ensures the consistency of all the peaks integrated. The signal intensity of the ions were log2 transformed and metabolites that were not present in at least 50% of the samples were filtered out. Missing values were imputed using k-nearest neighbour with default parameters implemented in DMwR236 package [version 0.0.2] in R35 [version 4.0.3]. We examined the three consecutive QC samples embedded at the start of each batch and removed any outlying measurements.Targeted metabolomics based on scheduled multiple reaction monitoring optimised to the metabolite of interest using authentic standards was applied in this study. Metabolite abundance peaks were integrated using the area under the curve for calibrated peaks from MultiQuant37 [version 1.20.0] objects in R35 [version 4.0.3], a specific intra- and inter-batch normalisation method, the structure of the tree and the parameters for RUV. hRUV performs repeated RUV procedures to sequentially adjust the data over a large collection of batches, with the number of unwanted variation factors (k) defaulting to 5. Further details of the workflow is described in Supplementary Note 1.The hRUV algorithm was designed for experiments with a large number of batches. The process of hRUV consists of two key steps, including (i) within batch signal drift adjustment with robust smoothers and/or RUV; and (ii) the adjustment of the datasets with an unwanted variation using an RUV in a hierarchical approach. The main inputs to hRUV are raw signal matrix, with rows corresponding to metabolites and columns to samples as SummarizedExperimentPart I: Signal drift adjustment within a batchIn the present setting, batch refers to one 88 sample run. However, this can be any pre-specified number of samples. After pre-processing of metabolomics data, hRUV begins intra-batch normalisation by fitting a linear  smoothers. An optional RUV step is then applied using sample replicates when specified by the user.L and L + M, where i at run number l.(i) Standard adjustment (ratio): The signal ratio was calculated by dividing the sample signal by the signal of its nearest pooled QC sample run. Let us denote 2 transformed signal for sample j in metabolite j. Here, the loess line uses the loess function in the stats35 [version 4.0.3] package.(ii) Loess line: A loess line was fitted to all the biological samples for each metabolite within a batch with the default span parameter of 0.75. The differences between the fitted line to the median of each metabolite across all samples per batch were calculated for adjustment of each samples as follows:2 transformed signal using the rlm function from the MASS38 [version 7.3-53.1] package for all the biological samples, with maximum number of iterations set to 100. The adjustments to each sample were calculated as with to the loess approach where (iii) Linear line: A robust linear model (rlm) against the run index was fitted to the log(iv) Loess line fitting with pooled QC samples: Similar to (ii), we fit the loess line to the pooled QC samples only. The adjusted value for each sample were calculated using the predicted values from the model ((vii) Linear line fitting with pooled QC samples: Likewise, we fit the rlm to the pooled QC samples. The adjusted values for each sample were calculated using the predicted values from the model.21. These sample replicates are utilised to estimate the unwanted variation as the signals of these replicate samples should theoretically be identical. All metabolites were used as the negative controls for RUV and the number of unwanted factors to use (k) was taken to be 5.(viii) RUV based approaches: We incorporated sample replicates into the design matrix of RUV introduced by Molania et al.Part II: Hierarchical batch integration design(i)n batches, this will requires log2(n) RUV adjustments.Balanced tree. The balanced tree approach to normalisation removes unwanted variation between pairs of batches at different levels of the tree. In this approach, we begin by removing unwanted variation between pairs of neighbouring batches. In the next layer of adjustment, we pair the two neighbouring groups of integrated batches (sets of 2 batches) and repeated the process to expand the number of batches per set until the last layer, where we have a single group of all the normalised, as illustrated in schematics in Fig. (ii)n batches, this will require n−1 number of RUV adjustments.Concatenation. As with the balanced tree approach, the concatenating approach removes unwanted variation between pairs of batches, but sequentially. We begin with the first two batches, and sequentially introduce subsequent batches, removing unwanted variation as illustrated in schematics in Fig. After intra-batch normalisation is complete, hRUV performs batch correction of multiple batches using either the balanced tree or the concatenating approach.For both the balanced tree and the concatenation methods, we apply RUV at each layer as follows:B the pair of batches of interest, M as the number of metabolites, and S as the number of samples in B. The mean adjusted sample m across samples S and batches B calculated by:Let us denote by X is the matrix of factor of interest; p is the number of factors of interest; W is the unobserved design matrix corresponding to the unwanted factors; k is the linear dimension of the unwanted factors, which is unknown; and The mean adjusted data After the RUV, P-values per batch for the hypertension response variable. We carried out this analysis with the limma39 [version 3.46] package in R35 [version 4.0.3].We explored an adaptive data-driven selection of negative control metabolites in comparison to the use of all metabolites in an RUV method. Adaptive selection is performed by ranking non-differentially expressed metabolites by We evaluate hRUV methods including 13 publicly available metabolomics data normalisation methods (Table e107140 [version 1.7-4] package in R35 [version 4.0.3].(i) Skewness: The skewness of samples were calculated with skewness function from x, n for the number of samples, s for the sample standard deviation, and r. The skewness then can be calculated as:Let us denote stats35 [version 4.0.3] package in R35 [version 4.0.3].(ii) Normality metric: The normality tests were performed with Shapiro–Wilk normality test implemented in shapiro.test function from the e107140 [version 1.7-4] package to predict the hypertension status of participants of the study. We measured the average accuracy via a 30-repeated 10 folds cross-validation strategy.(iii) Predictability with accuracy: To assess the predictability of a normalised dataset, we utilised a binary diagnosis of hypertension as the response variable. This was chosen as it had a reasonably balanced class distribution, as 39% of the cohort had hypertension. We used a Support Vector Machine (SVM) implemented in the (iv) Signal strength with AUC: We use the same prediction model from (iii) and calculate the area under the ROC curve (AUC) values.(v) Standard deviation of replicates (SD replicates): To demonstrate the variation between the replicate samples after normalisation, for each set of replicated sample, we calculated the standard deviation for each metabolite and visualised the results as a boxplot. A low standard deviation indicates a small variability between the replicates and thus illustrates that the replicates are close to identical.k-means clustering (hclust and kmeans in stats35 [version 4.0.3] package in R35 [version 4.0.3] respectively) where we set the number of clusters k to the number of batches. The cluster output is evaluated using adjusted rand index (ARI):(vi) Clustering by batch (Reduction in batch effect): To assess the removal of batch effects, we performed unsupervised hierarchical and limma39 [version 3.46.0]. We identified a set of metabolites with a 5% level of significance and verified their association with hypertension from the literature.(vii) Differential expression (DE) analysis of hypertension: To assess the biological signal in the normalised data, we performed DE analysis with the R package 41; (3) metabolite run plots.To graphically assess whether the normalisation method or the choice of parameters of hRUV has effectively corrected the batch effect, we have provided three kinds of diagnostic plots: (1) PCA plots; (2) relative log expression (RLE) plotsPCA plots were generated using all metabolites. We show the first and second principal components.i in sample j. The samples from different batches should have a similar distribution, and the medians of the boxplots should be close to zero if the unwanted variations are removed.RLE plots are a useful tool to visualise unwanted variation. RLA plots are boxplots of RLA for each sample, calculated as Metabolite plots are a useful diagnostic visualisation to visualise the signal drifts. The run plots are a scatter plot of signals for each metabolite against the run order of all the samples. The overall shape of the scatter plot should be a flat horizontal bar. All other shapes of trend in the scatter plot is an indication of signal drift.Further information on research design is available in the Supplementary InformationDescription of Additional Supplementary FilesSupplementary Data 1Supplementary Data 2Reporting summary


Genome wide gene expression analysis has revealed hints for independent immunological pathways underlying the pathophysiologies of phlegmonous (PA) and gangrenous appendicitis (GA). Methods of artificial intelligence (AI) have successfully been applied to routine laboratory and sonographic parameters for differentiation of the inflammatory manifestations. In this study we aimed to apply AI methods to gene expression data to provide evidence for feasibility.Modern algorithms from AI were applied to 56.666 gene expression data sets from 13 patients with PA and 16 with GA aged 7–17 years by using resampling methods (bootstrap). Performance with respect to sensitivities and specificities where investigated with receiver operating characteristic (ROC) analysis.ERGIC and golgi 3, regulator of G-protein signaling 2, Rho GTPase activating protein 33, and Golgi Reassembly Stacking Protein 2. ROC analysis showed a mean area under the curve of 84%.Within the experimental setting a best performing discriminatory biomarker signature consisting of a set of 4 genes could be defined: Gene expression based application of AI methods is feasible and represents a promising approach for future discriminatory diagnostics in children with acute appendicitis. Due to new conservative modalities for the treatment of uncomplicated acute appendicitis, current strategies for the distinguishment of high risk and low risk acute appendicitis are under pressure. Successful and safe application of conservative antibiotic treatment for clinically uncomplicated appendicitis has been demonstrated . Even frNew evidence on the pathophysiology of acute appendicitis has led to the concept of risk being given more weight without complications having actually  occurred. This attitude is especially due to evidence on substantial epidemiological and immunological differences between histopathological phlegmonous and gangrenous appendicitis –7. HowevIn a prospective study with genome wide gene expression analysis in children with acute appendicitis showed clearly distinguishable pictures within an immunological pathway analysis . Gene exHowever, safe differentiation for clinical decision making is still a great challenge. Clinical examination is characterized by a great interobserver variability with low discriminatory capacity , laboratIn a recently published study, laboratory cellular parameters from full blood counts, C-reactive Protein (CRP) and the sonographically measured appendiceal diameter were used within modern algorithms from machine learning and artificial intelligence for the differentiation of phlegmonous and gangrenous appendicitis . The preThe aim of the current study was to investigate the applicability of algorithms from machine learning and artificial intelligence to the extended pool of data from whole genome gene expression analysis for pretherapeutic differentiation of phlegmonous and gangrenous appendicitis in children and adolescents.This prospective study included children aged 7–17 years, which presented with signs of acute appendicitis at the Department of Pediatric Surgery of Charité – Universitätsmedizin Berlin, Germany, between April 2019 and August 2019. Ethical approval was provided by the local ethics committee (reference number ES2/130/16). Due to ethical concerns regarding a gene expression study with underaged patients, the study population was restricted to a total number of 30 patients with an additional expected exclusion rate of 10%. The ability to give informed consent based on age adapted information sheets within this pilot study was expected at an age of 7 years. Thus, younger children were not considered. Patients with signs of acute appendicitis in ultrasound examination and planned appendectomy were enrolled in the study using respective information sheets for informed consent . Inclusion criteria were written informed consent, performed appendectomy with histopathological examination, sufficient RNA quality (RNA integrity number over 7) and time period from blood sample collection till peripheral blood mononuclear cell (PBMC) isolation less than 1 h. Exclusion criteria were previous conservative treatment of acute appendicitis, any concomitant disease or antibiotic treatment in the past 2 weeks.All patients underwent standard laparoscopic appendectomy in general anesthesia with infraumbilical insertion of a 12 mm Hasson trocar for a 5 mm camera and of two 5 mm working trocars in the left and mid lower abdomen for 5 mm instruments. After establishment of a capnoperitoneum (10-12 mmHg) and bipolar dissection of the appendicular artery, the appendix was ligated with Röder-slings at its base, divided with scissors and removed via retrieval bag. A 5 mm stapler device was used for complicated appendicitis with necrotic appendix base. In case of free fluid irrigation with Ringer solution and respective suctioning was performed. Drains were consequently avoided.According to the ethical vote, blood samples (> 5 ml) were only collected during routine blood test at the emergency department. Isolation of PBMCs was carried out within 1 h after blood collection. Therefore, blood from each patient was suspended in phosphate-buffered saline  followed by density gradient centrifugation . Thereafter, the monocyte layer was re-suspended with PBS and centrifugated , followed by a final re-suspension with 1 ml PBS, centrifugation and removal of the supernatant. Native cells were frozen at − 20 °C  and finally stored at − 80 °C in liquid nitrogen. By using the NucleoSpin RNA Plus kit  total RNA was isolated from the PBMCs. RNA quality control  and RNA quantity control  were performed. Samples displaying a Bioanalyzer’s RNA integrity number (RIN) above seven fulfilled quality control standards and were labelled by generation of fluorescent cRNA . 1st strand synthesis using random primer/oligo-dT primer mixture, 2nd strand synthesis and synthesis of cRNA labelled with cyanine 3-CTP and were performed. Cy3 labelled cRNA (600 ng) was hybridized  followed by microarray wash and scanning . According to manufacture’s protocols a resolution of 3-μm was used to generate 20 bit TIF files.phlegmonous appendicitis: transmural infiltration of the appendix by neutrophilic granulocytes, serositis, microabscesses and oedema without gangrene or perforationgangrenous appendicitis: ischemic areas in the appendix with transmural myonecrosis.perforated appendicitis: gangrenous alterations with transmural defect of the appendix wallAfter appendectomy, the appendices were histopathologically examined by two pathologists of which the second examination was a blinded evaluation by a specialized pediatric pathologist. Thereafter, primary diagnosis had been corrected in two cases. Appendicitis was classified in accordance with Carr (9):p < 0.05.To obtain microarray data, the Feature Extraction Software V11 (Agilent Technologies) and GE1_1105_Oct12 protocol was used to extract TIF files. At OakLabs GmbH, Henningsdorf, Germany, the raw data was further analyzed using DirectArray Software. Box plots were used to visualize signal distributions of raw data and to identify potential issues for individual samples. For statistical analysis of gangrenous versus perforated appendicitis, samples were first quantile normalized and then analyzed using a Welch’s test and calculating log2 fold changes for each gene. The level of significance was discovery and validation [n samples, each of these described by a set of p variables – represented by the biomarker values. Concretely, the data matrix consisted of n lines and p columns. Then, relevant biomarkers were identified: a sequence of distinct biomarker signatures {bm1,. . ..,bmj,.. ., bmm} was built and subsequent implementation of a binary classification problem was performed, fitting the parameters of a logistic regression model on the discovery data Xdiscovery whose colums pbmj were filtered according to the biomarker signatures. The quality of each biomarker signature was measured on the discovery data, while all performance values were obtained by measurement of the trained model on the validation data [A supervised learning algorithm was used to analyze gene expression data and to build a prediction model for diagnosis of appendicitis based on relevant biomarkers. Usually, this is a two-step process summarized as lidation . Due to ion data .After definition of the best model, it was used to predict the diagnostic status of a patient with class probability. Output class probabilities can be interpreted as separation thresholds between class prediction. The thresholds represent trade-offs for the model to predict true/false positive and true/false negative rates. The diagnostic ability of the model with regard to sensitivities and specificities was tested on the validation dataset: true and false positive rates were counted at different thresholds . For illThe primary endpoint of the study was the pretherapeutic distinctness of the histopathological entities demonstrated by an area under the curve (AUC) in the ROC analysis exceeding at least 50%. A relevant decisive degree was assumed at an AUC of at least 80%.Regarding evaluation of the artificial intelligence approach concerning effective differentiation of the inflammatory entities, bootstrapping allowed for the calculation of standard errors and confidence intervals.p < 0.05. Statistical analysis was performed with GraphPad Prism software .Statistical analysis regarding epidemiological and routine laboratory data was performed with Mann-Whitney-U-test for continuous and Chi-Square-test for categorical parameters. Welch’s t-test was used for statistical analysis of gene expression values after quantile normalization as previously described . Values p < 0.05). Mean eosinophilic granulocytes were significantly upregulated in patients with PA compared with patients with GA  without any other significant differences in the differential blood counts.Epidemiological data have already been published elsewhere . Acute aPatient demographics, distribution of histopathological entities and mean symptom durations are illustrated in Table ERGIC and golgi 3, regulator of G-protein signaling 2, Rho GTPase activating protein 33, and Golgi Reassembly Stacking Protein 2. Figure As published previously, out of a total of 56,666 analyzed genes 3594 (6.3%) were significantly differentially expressed . ResamplReliable differentiation of the inflammatory entities within acute appendicitis is mandatory for safe performance of new conservative treatment strategies. Several studies have attempted to increase the validity of currently established routine methods with regard to the pretherapeutic differentiation between complicated and uncomplicated appendicitis. But especially the great number of publications which are concerned with this question might be a sign for the fact that this problem is still unsolved.In particular, we identified two obstacles in previous attempts to resolve this issue. First, up to now the level at which the distinction should be made has not been convincingly shown – at the level of clinical judgement or at that of histopathological examination. And second: The discriminatory capacity of the established diagnostic methods is far too low with respect to the needed sensitivities and specificities. Clinical examination and imaging techniques are severely limited by interobserver reliability and need for interpretation, currently used laboratory values are restricted by a substantial indistinctness , 13, 14.Increasing epidemiological, immunological and even gene expression evidence strongly suggests that acute appendicitis should be differentiated with respect to expected courses (uncomplicated vs. complicated) at the histopathological levels of phlegmone and gangrene. Phlegmonous appendicitis is most probably an own entity based on a distinct pathophysiology, possibly related to viral infection , 5, 12. Application of supervised learning algorithms to gene expression data with focus on targets for therapeutic intervention and biomarkers for early diagnosis has already been successfully performed in other contexts like sepsis . It was ERGIC and golgi 3, regulator of G-protein signaling 2, Rho GTPase activating protein 33, Golgi Reassembly Stacking Protein 2) are involved in cellular functions affecting cytoskeleton formation, regulation of GTP binding, membrane trafficking and cell signalling [ERGIC and golgi 3 is not significantly differentially regulated (p > 0.05). This finding demonstrates that the diagnostic horizon, which is usually limited to standard parameters when purely statistical methods are used, is expanded through the use of artificial intelligence. However, as these cellular functions take place on a very basic level, interpretation is not easy and cannot be provided at this stage.Interestingly, the particular underlying expressions of the particular genes in the biomarker signature (gnalling . Unlike The comparability of the present approach with other studies is limited. It represents a purely experimental procedure in terms of a proof-of-concept study with a limited number of patients.Although the presented investigation is best characterized as a simulation with artificial augmentation of data sets, the results justify the application of supervised learning algorithms with regular training and validation sets to gene expression data of a greater number of patients in order to gain the necessary level of distinguishability of the inflammatory entities. Although the presented results are already very promising, the combination of gene expression data with other largely objective parameters like the sonographically measured appendiceal diameter within artificial intelligence strategies might even substantially improve the efficacy of the method.