The increasing quantity of multi-omic data, such as methylomic and transcriptomic profiles collected on the same specimen or even on the same cell, provides a unique opportunity to explore the complex interactions that define cell phenotype and govern cellular responses to perturbations. We propose a network approach based on Gaussian Graphical Models (GGMs) that facilitates the joint analysis of paired omics data. This method, called DRAGON , calibrates its parameters to achieve an optimal trade-off between the network’s complexity and estimation accuracy, while explicitly accounting for the characteristics of each of the assessed omics ‘layers.’ In simulation studies, we show that DRAGON adapts to edge density and feature size differences between omics layers, improving model inference and edge recovery compared to state-of-the-art methods. We further demonstrate in an analysis of joint transcriptome - methylome data from TCGA breast cancer specimens that DRAGON can identify key molecular mechanisms such as gene regulation via promoter methylation. In particular, we identify Transcription Factor AP-2 Beta (TFAP2B) as a potential multi-omic biomarker for basal-type breast cancer. DRAGON is available as open-source code in Python through the Network Zoo package . A regulates the expression of two genes, B and C. In this case, a correlation network will contain an edge between gene B and gene C because correlation indicates a relation between the two genes. However, that relationship is only a consequence of their mutual relationship with the transcription factor A and thus the observed edge in the correlation network represents an indirect association. The problem of such erroneous correlations was discussed by Pearson and Yule in the early twentieth century, where the term ‘spurious correlation’ was introduced to distinguish indirect from direct relationships; a historical review of this question has been summarized by Aldrich and colleagues  provides data on RNA expression, methylation levels, and copy number variations for many individual tumor samples. More recently, single cell multi-omic data have become available as it has become possible to assay different omics data types from individual cells; for example, Cao le cells . Such mup typically exceeds the number of samples n (or both are of the same order of magnitude), which presents challenges for network inference , which is strongly associated with the basal-like breast cancer subtype.X be a n × p data matrix of n observations (samples) and p features . Assume that the observations N, where Σ is a positive definite covariance matrix. Further, Θ = (θij) = Σ−1 is the inverse covariance matrix  where vanishing entries i and j. A Gaussian Graphical Model (GGM) is a conditional dependence graph in which nodes represent variables and edges connect conditionally dependent pairs of variables  of Equation  yields \mentclass2pt{minimEquation . Anothermentclasspt{minimaand Wolf  and T ismentclasspt{minimaX(1) be a n × p1 data matrix that represents the first omics layer and let X(2) be a n × p2 data matrix for the second layer, where p1 and p2 are the number of variables from omics layers 1 and 2. We further assume paired data, meaning that measurements (rows) i but differ in their features. We define the empirical sample covariances k, l ∈ {1, 2}. Now, we can generalize the shrinkage estimator tok ∈  and 2 = 1. Here, Equation  = ((1 − λ1)S + λ1T(1))−1 is the shrinkage estimator of the precision matrix using only the features with k = 1. Thus, if λ2 = 1, technology 1 decouples from technology 2.In DRAGON, we extend covariance shrinkage to account for two different omics layers by introducing layer-specific regularization terms. Let imator to\document1 = λ2 = λ. Then Equation  for n ≫ p, or can be fitted by MLE for the ill-posed case p > n or for p ≈ n \docor p ≈ n . For an \docor p or p ≈ n . Let(11)H0: ρ = 0) for given sample size n, and estimate corresponding partial correlations using DRAGON with λ1 and λ2 given from the original data.Simulate data under the null hypotheses , P and P, separately.Fit Equation  for P,  and  for multiple testing, separately, using the method of Benjamini and Hochberg . Note thn ≫ p, n − 1 − (p − 2) and (i–ii) are not necessary.For p-values without computationally expensive resampling, and have been published in peer-reviewed journals.To benchmark the performance of DRAGON, we selected five methods for GGM estimation, each of which has distinct advantages and disadvantages , based oGGM: we implemented an omics-layer agnostic DRAGON model, which we simply denote as Gaussian Graphical Model (GGM) in the following. Note, in contrast to the GeneNet implementation described below  via an empirical Bayes approach , we adjusted p-values using the procedure of Benjamini and Hochberg  ‘variable’. The regulator might belong to another omics layer representing a different biological factor such as the chromatin state in the DNA region of A and B. We generated artificial data representing such a process for n = 100 observations . This correlation is statistically significant although gene A and gene B are not directly related – rather, they are only co-regulated by the regulator. In contrast, the partial correlation between A and B takes into account the effect of the regulator, resulting in a non-significant partial correlation of ρ ∼ 0.02  as well as models in which there are different probabilities for direct relationships among variables within and between different omics layers. Table A to D, with fixed numbers of molecular variables, p1 = 100 and p2 = 500 for omics layers 1 and 2, respectively. Direct relationships can be simulated by appropriate choice of the entries of the precision matrices We performed four simulation studies, p-dimensional identity matrix, where p = p1 + p2,start with a k, l) to each entry of the data matrix.Each precision matrix was used to sample from a multivariate normal with mean vector μ = 0 and covariance R, Equation  = η = η = 0.05. Figure R in dependency of λ1 and λ2 for the analytical estimate and the ground truth, respectively, for a fixed sample size of n = 5000. We observed that estimate and ground truth agree remarkably well, and that they provide almost identical estimates for the position of the minimum of the selected penalty parameters, X =  corresponds to the diagonal red lines, such that the two data layers are treated as they would belong to one layer and a single regularization parameter n = 500 and n = 1000, respectively.In simulation study 1) = η1,  = η to η = 0.005. Again, we observed a remarkable agreement between estimated and true R, shown in Figure In Simulation B, we investigated the influence of the edge densities ηC and D, both of which have unbalanced edge densities with results shown in Supplementary Figures S3, S4, and S5 for n = 500, n = 1000, and n = 5000, respectively. Here, we also found that DRAGON correctly estimates R and that λ1 and λ2 are chosen by the algorithm accordingly to minimize R.We repeated this analysis for two further simulation studies, A to D 20 times for different sample sizes n and evaluated the log-likelihood on n = 1000 test samples . While we recognize that few omics studies have 1000 or more samples, single-cell experiments generally assay thousands of cells and large cohort studies are beginning to develop omics databases that include populations of this size or larger.We next analyzed how the DRAGON regularization scheme, which augments covariance shrinkage by omics layer-specific regularization parameters, affects model inference. We repeated simulation studies A, but in simuations B, C, and D, DRAGON outperformed the standard GGM as indicated by positive values for the difference. The greatest improvements were seen for the unbalanced simulation B. We can also see that as the sample size n increases with the number of predictors p remaining fixed, DRAGON and GGM estimates coincide, as demonstrated by a vanishing log-likelihood difference.We found similar results on an absolute scale . Howeverp-values  were assigned as described in the Materials and Methods section and were used to assess the edge-recovery performance in simulations A to D using receiver operating characteristic (ROC) curves for DRAGON, GGM, GeneNet, B-NW-SL, D-S-NW-SL and D-S-GL. Variables were standardized for performance comparisons.Adjusted n for simulation study A. We saw almost identical performance for DRAGON (blue) and GGM (red) for all sample sizes in our simulation, a result consistent with our previous findings. For simulation studies B, C and D, the corresponding results are shown in Figure B, DRAGON achieved an AUC for n ≈ 250 samples that is compatible to that of the GGM for n ≈ 1000 samples. This improved performance of DRAGON is further illustrated by Figure B), ∼0.05 (study C) and ∼0.11 (study D). In the balanced simulation scenario A where we expect the GGM and DRAGON to have similar performance, the GGM negligibly outperformed DRAGON, as shown by small negative values in Figure B to D, respectively.We first compared the results of DRAGON and GGM inference methods. Figure A, B and D that DRAGON’s improved estimates are of the same size (for GeneNet and B-NW-SL) or substantially larger (for D-S-NW-SL and D-S-GL) than we found in comparing DRAGON to GGM. In simulation study C, we found comparable performance of GeneNet and DRAGON across a large range of sample sizes  for each of the simulation studies required to reach network confidence levels AUC >0.8. As can be seen, DRAGON reached confidence for substantially smaller n and larger p/n ratios than the other methods.Analogous AUC analyses were performed comparing DRAGON with GeneNet, B-NW-SL, D-S-NW-SL and D-S-GL with results shown in Figures  Figures  and 4G bP-values and false discovery rates (FDRs). First, we performed simulations under the null hypothesis of no partial correlation (ρ = 0) and verified that the P-value distributions are flat for different sets of the regularization parameters λ1 and λ2 . The simulated data sets were motivated as follows: we recorded the λ1 and λ2 values for each simulation run in study A to D, which yielded the results shown in Supplementary Figure S12. We extracted the corresponding parameter sets at n = 256, n = 1024 and n = 4096 samples, which correspond respectively to a highly regularized estimate, a moderately regularized one, and one with low regularization. This resulted, in total, in 12 pairs  with the associated P-value distributions under the null hypothesis that are shown in Supplementary Figure S13; as desired, these distributions are flat. Further, we recorded the FDRs corresponding to studies A to D  for the empirical estimate of κ and the theoretical value κ = n − 1 − (p − 2), respectively. As expected, the observed FDR for DRAGON approaches 0.05 as the sample size increases. Analogous plots for the comparison methods can be found in Supplementary Figure S9e–t. DRAGON generally outperforms the other methods, some of which show inflated FDRs indicating an overly liberal method (GeneNet) or deflated FDRs indicating an overly conservative method (D-S-NW-SL and D-S-GL).We also verified that DRAGON correctly estimates Epigenetic aberrations such as DNA methylation are associated with altered patterns of gene regulation during the development and progression of complex diseases such as cancer . Given thttps://docs.gdc.cancer.gov/Data/Bioinformatics_Pipelines/Methylation_Pipeline/, Accessed: 2022-05-25), which uses SeSAMe  to produce gene expression levels; this pipeline uses the Spliced Transcripts Alignment to a Reference (STAR) (The GenomicDataCommons  R packags SeSAMe  for signs SeSAMe . RNA-seqe (STAR)  algorithMethylation and expression data for the transcription factors for this breast cancer data set were loaded into DRAGON and a partial correlation network was calculated. A thresholded version of the DRAGON-estimated network consisting of all edges with FDR <0.005 and all nodes with degree >0 is shown in Figure DNA methylation can regulate gene expression by blocking TF binding or by altering the binding of other regulatory proteins . In cancBecause of the complex, correlated nature of methylation and gene expression, a multi-omic correlation network based on these data may be too noisy to identify meaningful methylation-expression relationships. To illustrate this, we created a Pearson correlation network on the methylation-gene expression data with 824,599 significant methylation-gene expression edges based on the criteria FDR <0.05. Notably, of these edges, only 861 (0.1%) were edges between the expression of a gene and the methylation of its promoter – the associations we expect functionally to be the most significant (175 (20%) positive and 686 (80%) negative edges). In contrast, when using DRAGON to estimate the partial correlation network on the same data, the result was a much sparser network consisting of 769 significant methylation-gene expression edges (FDR <0.05). Of these edges, 333 (43%) were associations between expression of a gene and methylation of its promoter (26 (8%) positive and 307 (92%) negative partial correlations). The high proportion of negative partial correlations is an important ‘sanity check’ on the performance of DRAGON, as promoter methylation suppresses transcription for most genes . To furt−100, ρ = −0.25), followed by methylation of the SKOR2 promoter , and methylation of the ZNF121 promoter . A comprehensive list of edges is shown in Table Returning to the DRAGON network, we ranked gene expression nodes according to the number of significant edges to methylation nodes they possessed Table . The top−34, ρ = −0.175), followed by ZNF266  and KLF2 . ZNF334 was recently identified as tumor suppressor of triple-negative breast cancer and higher ZNF334 expression was shown to be associated with better survival outcomes  and IKZF3 . FOXO6 is a transcription factor known to play multiple roles in breast cancer. Its downregulation is implicated in promotion of the epithelial–mesenchymal transition, in migration and proliferation of breast cancer cells, and in reduced cell resistance to the anti-cancer drug paclitaxel through the PI3K/Akt signaling pathway  and MYRFL, both showing 10 edges to methylation variables . Unlike the other transcription factors we identified, NR6A1 was not related to its own methylation site . Its strongest methylation-gene expression edges were observed to methylation sites attributed to the genes FOXO6 -methyltransferase 3B) to induce promoter methylation of the genes Oct-3/4 . MYRFL encodes a transcription factor that is required for central nervous system myelination and it has been identified as a member of a regulatory cluster of genes on chromosome 12 that has been associated with elevated risk of breast cancer . For the analysis presented here, it should be noted that each transcription factor gene may appear twice, once based on its expression and once based on its methylation. Therefore, the universe of possible genes considered for the ORA (and used to set background expectations) is twice the size of the number of TFs included in the DRAGON model. We also performed an ORA assessing enrichment for methylation only and one assessing enrichment for expression only. A Reactome pathway was identified as over-represented in a community if its FDR was <0.05 in at least one of these three ORAs (Benjamini-Hochberg FDR as implemented in fgsea). Ten of the 59 communities of size ≥5 genes were enriched for at least one REACTOME pathway at FDR <0.05; complete results of the enrichment analysis are available in To assess the enrichment of methylation-gene expression communities for functions potentially related to breast cancer, we performed an over-representation analysis (ORA) for Reactome gene sets  within e the ORA . ReactomCommunity 5 consists of 39 TF nodes, 14 based on TF gene expression and 25 based on TF promoter methylation Figure . The TF PanCancerAtlas_subtypes function from the TCGAbiolinks R package  for the tumor samples using the  package . We thenp = 3.3 × 10−4; TFAP2B p < 2.2 × 10−16) as did TFAP2A, TFAP2B and TFAP2C expression . However, TFAP2C methylation, which was notably excluded from this community, did not differ significantly between subtypes (p = 0.17). To illustrate the multi-omic phenotyping possible with DRAGON communities, we investigated TFAP2B further. The median TFAP2B promoter methylation beta level was 0.55 in basal samples versus 0.17 in non-basal samples; the median TFAP2B expression among basal samples was 0.49 TPM versus a median of 36.46 TPM in non-basal samples. This multi-omic phenotype of increased methylation and decreased expression in basal samples follows the canonical paradigm that promoter methylation results in gene silencing. To explore the predictive power of this joint information, we constructed a logistic regression model for multi-omic subtype classification, regressing basal vs. non-basal subtype against TFAP2B methylation, expression, and an interaction term between the two. In the resulting model, TFAP2B methylation and the interaction term between TFAP2B methylation and expression were both significant  while TFAP2B expression was not . The AUROC of this model was 0.85; in contrast, the AUROC of a similar model using TFAP2B methylation alone as a predictor was 0.82 and using TFAP2B expression alone, 0.79, demonstrating the synergistic power of multi-omic features for class discrimination. Although this classification model does not outperform the expression-based subtype classification used by the TCGA, these results are nonetheless impressive given that they are based on two measurements of the omic state of a single gene.TFAP2A and TFAP2B methylation both differed significantly based on subtype , estrogen-dependent gene expression (FDR = 1.5 × 10−3), PTEN regulation (FDR = 0.02), and MAPK family signaling cascades (FDR = 0.02). The nodes primarily driving these enrichments include EGR1, EGR2, EGR3, FOS, FOSB, JUNB and JUND. These seven nodes comprise a tightly co-expressed subgraph of Community 38 and are well-known players in cancer signaling . To provide a benchmark for differential methylation between subtypes, we performed the same statistical test for a ‘housekeeping’ transcription factor (ATF1), which showed no significant difference in methylation , indicating that the differential methylation of these key transcription factors may play a substantial role in breast cancer beyond those already discussed, particularly given their association with the EGR/FOS/JUN expression cluster and the importance of these transcription factors in a wide range of cancer processes.NR4A2 methylation and KLF6 methylation are both degree-one nodes in Community 38; their only edges indicate negative partial correlation with their own expression. It may be that differential methylation of NR4A2 and KLF6 drives subtype-specific activity in the pathways of this community by modulating the expression of NR4A2 and KLF6, which are connected via FOS and JUN expression to the eight-node cluster of coexpressed TFs. In comparing these nodes between subtypes, we found that both KLF6 and NR4A2 are significantly differentially methylated  and uses a regularization scheme to optimize the trade-off between the network’s complexity and its estimation accuracy while explicitly taking into account the distinct data characteristics of the various omics data types used in the model. DRAGON accounts for differences in edge densities and feature sizes, enabling improved estimation of partial correlations compared to layer-agnostic GGMs. The advantages of DRAGON are particularly evident in simulations when the number of variables We recognize that DRAGON has some limitations. DRAGON’s GGM framework assumes multivariate normally distributed data which, for biological data, generally does not hold. For continuous distributions, data transformations such as the nonparanormal transformation  can be ua priori removing edges from the model based on known regulatory relationships by, for example, estimating the inverse biased covariance matrix with a priori specified zero entries (Tk).https://github.com/katehoffshutta/DRAGON-TCGA-BRCA (doi:10.5281/zenodo.7336963).DRAGON is available through the Network Zoo package  and a vignette to demonstrate its use can be found in netBooks (netbooks.networkmedicine.org). Both are also available as supplementary material. Code to reproduce the TCGA breast cancer methylation analysis is available on Github at gkac1157_Supplemental_FilesClick here for additional data file.


The constant evolving and development of next-generation sequencing techniques lead to high throughput data composed of datasets that include a large number of biological samples. Although a large number of samples are usually experimentally processed by batches, scientific publications are often elusive about this information, which can greatly impact the quality of the samples and confound further statistical analyzes. Because dedicated bioinformatics methods developed to detect unwanted sources of variance in the data can wrongly detect real biological signals, such methods could benefit from using a quality-aware approach.We recently developed statistical guidelines and a machine learning tool to automatically evaluate the quality of a next-generation-sequencing sample. We leveraged this quality assessment to detect and correct batch effects in 12 publicly available RNA-seq datasets with available batch information. We were able to distinguish batches by our quality score and used it to correct for some batch effects in sample clustering. Overall, the correction was evaluated as comparable to or better than the reference method that uses a priori knowledge of the batches . When coupled to outlier removal, the correction was more often evaluated as better than the reference .In this work, we show the capabilities of our software to detect batches in public RNA-seq datasets from differences in the predicted quality of their samples. We also use these insights to correct the batch effect and observe the relation of sample quality and batch effect. These observations reinforce our expectation that while batch effects do correlate with differences in quality, batch effects also arise from other artifacts and are more suitably  corrected statistically in well-designed experiments.The online version contains supplementary material available at 10.1186/s12859-022-04775-y. Batch effects arise from differences between samples that are not rooted in the experimental design and can have various sources, spanning from different handlers or experiment locations to different batches of reagents and even biological artifacts such as growth location. In the context of sequencing data, two runs at different time points can already show a batch effect . AlthougVarious methods have been developed to detect or even remove batch effects in genomics data, particularly RNA-seq data and cDNA microarrays. For example, the sva package from Bioconductor  can deteQuality control of next-generation sequencing data is an essential but not trivial step in functional genome and epigenome analysis. Many tools have been designed to shed light on the quality of a given sample or file. In previous studies, we used 2642 quality-labeled FASTQ files from the ENCODE project to derive statistical features with different bioinformatics tools well known in the scientific community. We showed that these features have explanatory power over the quality of the data from which they were derived, and built a machine learning classification tool that uses these features as input , 12. WitAs the quality of biological samples could be significantly impacted by batch processing, we hypothesized that an automated machine-learning-based quality classifier could also detect batches in gene-expression datasets and be used to correct the batch effect. Here, we leverage the explanatory power of our machine learning algorithm, to successfully detect batch effects based on quality differences in the samples. In addition, we provide a method for batch correction based on predicted sample quality that we evaluate with 12 published RNA-seq datasets.low by biological sample is depicted in Fig. low is a machine-learning derived probability for a sample to be of low quality, as derived by the seqQscorer tool [low and this significantly reduced computing time.Our workflow to process the data and derive low-quality scores Prer tool . From 12low scores between the batches. In 5 datasets, the differences are not significant . We can observe a strong batch effect in the uncorrected principal component analysis (PCA)  where samples from batch 1 cluster together on the right-hand side, and samples from batch 2 on the left-hand side (panel: PCA Abundance). It is supported by poor clustering evaluation scores  and very few differential genes (DEGs = 4). Thanks to the reference method using a priori knowledge of the batches to correct the analysis, samples on a batch-corrected PCA cluster by group and not anymore by batch (panel: PCA corrected Batch). It is supported by more differentially expressed genes (DEGs = 12) and better clustering evaluation scores , with further improvement on another version of the batch-corrected PCA where an outlier sample could be manually identified and removed . Without using a priori knowledge but only using the automatically derived quality scores to correct the analysis, samples are also clustered by group and not anymore by batch (panel: PCA corrected Plow). Clustering results and statistics are very comparable to the reference that uses a priori knowledge of the batches, though the number of differentially expressed genes is even higher (DEGs = 21) and removing outliers manually or based on quality scores did not improve the results further. Finally, applying a correction based on both a priori knowledge and automatic quality scores, together with outlier removal, showed the best clustering statistics (panel: PCA corrected Batch and Plow and no outlier) .Figure low correction have is mostly comparable, although the true batch correction seems to work better in a few cases (e.g. dataset GSE120099). However, if outliers of the base PCA or depending on sample quality are removed, the impact improves and even overcomes the simple batch correction. It should be noted that these clustering metrics are also sensitive to the removal of large proportions of samples, which is sometimes the case here. The impact of outlier removal alone is negligible (data not shown). All PCAs that are discussed here are given as panel plots similar to Fig. In Fig. An additional view on the data is shown in Table low correction was mostly evaluated as comparable to or better than the reference method that uses a priori knowledge of the batches . When coupled to outlier removal, the Plow correction was more often evaluated as better than the reference . Combining true batch and Plow could improve the results further but not systematically ) and notably with a strong association with the imbalance of quality between groups of samples (designBias): the lower the bias, the better this further improvement. Although performing well, the Plow correction could not systematically remove the batch effect as well as the reference method using a priori knowledge. This result is in agreement with the multifaceted nature of batch effect, which is not only explained by quality differences.Overall, the Plow (GSE120099) or vice versa a correction with Plow would improve the clustering, but batch correction would not (GSE117970) , we observed very different outcomes. The combination of the methods could perform better or worse in comparison to the individual approaches. We hypothesize that the improvement of a combination of two confounding factors depends on whether they model different artifacts in the data or not. We did not take into account other confounding factors, since they are generally not well documented in public datasets. Poor results in this study from true batch or quality-based correction could be explained by such unaccounted confounding factors.Batch effects that would impact data quality, such as effects explained by different handlers, sequencers or reagents during RNA extraction, will most likely be detected by our software , 13. Biolow. However, they must be guided with information about the biological groups of interest beforehand in order to avoid their detection as unwanted variation to be corrected [low or even the true batch information, it could be observed that in 25% of the cases investigated the correction would skew or even remove the group clustering  to inflammatory diseases (e.g. periodontitis and parasitic diseases with leishmaniasis) and also different cancer types and cardiovascular diseases, showing a broad overview over biological data despite the relatively small number of datasets. Table low [low for each sample. Among the predefined models of seqQscorer some were trained on certain subsets of a large main training set: 2642 samples labeled for low- and high-quality. We used the generic Random Forest model that was trained on the complete training set. With this generic model, Plow is computed from the ratio of votes by trees in the random forest. To derive the differential gene expression, we used Salmon [We used our software seqQscorer to derive the quality features and to predict the quality of a sample, assigning the probability of low-quality Plow . Our soflow , Bowtie2low , ChIPSeelow  and ChIPlow . FastQC d Salmon  to quantd Salmon  for statlow with R and ggplot2 and computed a Kruskal–Wallis’s test to confirm if there are significant differences of Plow values between batches.We plotted the batches against Plow values could be used for batch corrections, we observed the first two dimensions of a PCA either without any correction, or with correction by Plow as well as correction by the real batch identifier as confounding factors. We used the AC-PCA package for this, which allows to add confounding factors to the computation of principal components [low as well as the combination of all correction methods and outlier removal. Outliers removed according to batch correction were identified by the uncorrected PCA as well as the PCA with correction by the true batch. If a sample or a small group of samples was clearly skewing the clustering it was removed. The same was done for the Plow correction, with the help of the bar plot showing Plow values by samples . For the combination of all outliers, the outliers of Plow were used, since they always contained the outliers used for the batch correction and normal PCA. To have a metric of evaluation for these plots, we used the clusterstats function from the fpc package and employed the wbratio, pearsongamma and dunn1 statistics [To investigate if Pmponents . We alsoatistics .The dunn statistic considers the ratio of the minimum separation of all clusters by the maximum diameter of all clusters. That makes it a worst-case indicator, since it shows the smallest index, even if some of the clusters are well defined and only one cluster is scattered.The pearsongamma is the correlation between distances and a 0–1-vector in which 0 indicates the same cluster, while 1 indicates a different cluster. It is derived from the Normalized gamma of Halkidi et al. .The wbratio is the average distance within the clusters divided by the average distance between the clusters. While the other two metrics increase with clustering improvement, wbratio decreases.low (either with or without outliers) worked similarly to a batch correction . The same information is given for the combination of both corrections, but the reference is then the best overall PCA plot. One main point considered was the change of the grouping of the clusters from the second to the first principal component and if the groups clustered together. Especially a widely scattered cluster which can often be found with cancer versus normal tissue would not score high with the metrics used, but it would be easily identified at a glance. It is furthermore indicated if the experimental design is poor, good, or very good, according to the manual observation of the balance of samples between batches. If there is information about confounding factors available, it is also taken into account.In Table low to biological groups, utilizing Pearson gamma or “normalized gamma” [We computed a design bias representing the agreement of Pd gamma” , to haveWe compared our results with the results of the sva method of the package of the same name . We usedAdditional file 1. Results of Correction Experiments for all Datasets.Additional file 2. Comparison of Plow and SVA Correction.


Oryza sativa L.) is a widely consumed food source, and its geographical origin has long been a subject of discussion. In our study, we collected 44 and 20 rice samples from different regions of the Republic of Korea and China, respectively, of which 35 and 29 samples were of white and brown rice, respectively. These samples were analyzed using nuclear magnetic resonance (NMR) spectroscopy, followed by analyses with various data normalization and scaling methods. Then, leave-one-out cross-validation (LOOCV) and external validation were employed to evaluate various machine learning algorithms. Total area normalization, with unit variance and Pareto scaling for white and brown rice samples, respectively, was determined as the best pre-processing method in orthogonal partial least squares–discriminant analysis. Among the various tested algorithms, support vector machine (SVM) was the best algorithm for predicting the geographical origin of white and brown rice, with an accuracy of 0.99 and 0.96, respectively. In external validation, the SVM-based prediction model for white and brown rice showed good performance, with an accuracy of 1.0. The results of this study suggest the potential application of machine learning techniques based on NMR data for the differentiation and prediction of diverse geographical origins of white and brown rice.Rice ( Oryza sativa) is a primary food source for almost 50% of the global population because of its high caloric content and various nutrients, such as minerals and vitamins [O. sativa, indica and japonica, with an enormous number of varieties. Japonica is the most commonly cultivated crop in East Asia, particularly in Korea, Japan, and China. Genotype and environmental factors, such as rainfall intensity, soil, and temperature, heavily influence rice metabolite profiles [Rice  and discriminant analysis [1H-NMR spectroscopy coupled with partial least square–discriminant analysis (PLS-DA) and independent component analysis [Several studies have been conducted to determine the types or geographical origins of rice samples to prevent adulteration and substitution, and to standardize their safety and quality assurance. For example, various rice samples were distinguished by multi-element fingerprinting using high-resolution inductively coupled plasma mass spectrometry (ICP-MS) , elementanalysis , and 1H-analysis . Mass spanalysis .1H-NMR and inductively coupled plasma atomic emission spectroscopy/ICP-MS techniques coupled with ML algorithms have been applied to discriminate the geographical origins of medicinal plants from Korea and China  [Asparagus officinalis L.) from six different countries  was successfully distinguished using 1H-NMR spectroscopy coupled with ML algorithms [Ixeris denticulata samples from eight different origins were differentiated using ultra-high performance liquid chromatography-quadrupole time-of-flight mass spectrometry followed by ML algorithms [Recently, machine learning (ML) algorithms have been demonstrated to significantly enhance the predictability and validity of prediction models for the geographical origin of various crops and medicinal plants. biflora) . The geogorithms . Ixeris gorithms . Differegorithms .1H-NMR analysis coupled with ML techniques.The total amount of rice imported into Korea in 2021 was 492,901 tons (accounting for 12.7% of domestic production), of which 40.8% was imported from China . Most of the imported rice from China into Korea was brown rice because of its advantages in storage and variety of utilization ,18. It iIn this study, we collected white and brown rice samples from different regions of the Republic of Korea (hereafter referred to as Korea) and China. We analyzed the data using NMR spectroscopy coupled with various ML algorithms such as PCA, orthogonal PLS-DA (OPLS-DA), random forest, decision tree, support vector machine (SVM), logistic regression, and k-nearest neighbors. The main objective of our study was to explore the utilization of NMR spectroscopy coupled with various ML algorithms to develop a convenient method for predicting the geographical origin of rice.Rice samples, collected from Korea (44 samples) and China (20 samples) with two types of milling (white and brown rice) as shown in 2O, 99.9% atom D) including 0.05% 3-(trimethylsilyl) propionic-2,2,3,3-d4 acid sodium salt (TSP), deuterium oxide , and monopotassium phosphate (KH2PO4) were purchased from Sigma-Aldrich . Sodium deuteroxide solution  was purchased from Cambridge Isotope Laboratories, Inc. .Deuterium oxide (D2O (0.1 mM TSP) were transferred into a 2 mL centrifuge tube , vortexed for 1 min, and sonicated for 15 min. Subsequently, the suspension was centrifuged at 17,000× g, 4 °C for 10 min. A buffer solution of 90 mM KH2PO4 was prepared from D2O, and NaOD was used to adjust the pH to 6.0. The clear supernatant was filtered using a 0.45 µm PVDF filter , and 600 µL of the sample was transferred into a 5 mm NMR tube .Pooled samples from each location were promptly frozen in liquid nitrogen, pulverized using a blender, and stored in a deep freezer until NMR analysis. Then, 100 mg of rice powder and 1.5 mL of 100% D1H-NMR spectra, 64K data points were obtained with a relaxation delay of 2.0 s and a spectral width of 10,775.9 Hz. A total of 128 scans and an acquisition time of 3.0 s were used. Water suppression was conducted to exclude the region between δ = 4.7 and 5.0 using a pre-saturation pulse sequence (Bruker 1D noesygppr1d). For two-dimensional NMR spectra, 1H–1H correlation spectroscopy (COSY) spectra were acquired under the following conditions: 32 scans, relaxation delay of 2.0 s, and 7812.5 Hz (for white rice) and 6465.5 Hz (for brown rice) spectral widths. 1H–13C heteronuclear single quantum correlation (HSQC) spectra were obtained with 32 scans, 2.0 s relaxation delay, and spectral widths of 5122.9 Hz and 36,235.5 Hz in the F1 and F2 dimensions, respectively (for white rice), and 6465.5 Hz and 36,150.3 Hz in the F1 and F2 dimensions, respectively (for brown rice). Baseline correction and assignments of all 1H–NMR spectra were performed using Chenomx NMR suite software . Metabolites were further identified based on the HMDB database (http://www.hmdb.ca/) (accessed on 2 March 2022). Non-overlapping peaks were used for peak assignment. MestReNova  was employed to measure the peak J values and identify the peaks of the 1H–1H COSY and 1H–13C HSQC spectra.A 600-MHz Bruker Avance spectrometer  was employed to analyze rice samples at 25 °C to record all NMR spectra. For the 1H–NMR spectral data were performed using the Chenomx NMR suite software. Baseline-corrected NMR spectral data ranging from 0.08 to 10.00 ppm were segmented into a series of small bins  with widths of 0.04 ppm, while excluding the water suppression region (4.70–4.86 ppm). The raw NMR spectral data were normalized using total area and standardized area normalization techniques. The total area normalization method was used to compute the relative intensities of the binned spectral data by dividing the spectral data by the total area of all bins. In contrast, in standardized area normalization, the relative intensities of the binned spectral data were calculated by dividing the spectral data by the area of the reference peak. Subsequently, the results of the binned datasets were converted to Microsoft Office Excel in a suitable format to quantify each compound by its loading value, and the binning values of compounds with numerous non-overlapping peaks were summed. All the pre-processed NMR spectral data with peak values were converted into comma-separated value (CSV) files for ML analysis.Binning and normalization of the After normalization of the NMR data, SIMCA-P+ software  was used to perform multivariate statistical analysis. PCA and OPLS-DA were performed using the SIMCA software. PCA is a clustering approach that minimizes the dimensions of multivariate data, while retaining the majority of its variance without any prerequisite information about the dataset, whereas the OPLS-DA model is a supervised classification method . The aut2Y and Q2Y parameters. The R2Y and Q2Y values are expected to be close to 1. The 10-fold cross-validation and permutation test were performed to prevent the overfitting of the model. Intercept values of R2Y and Q2Y below 0.4 and 0.05, respectively, were regarded as valid models.Mean-centering was performed, unit variance (UV) and Pareto (Par) scaling were applied in both PCA and OPLS-DA, and the outcomes were compared to determine the best scaling method. The goodness-of-fit and predictability of the model were evaluated using RPython is a scripting language widely used in data science . Differe“GridSearchCV” is an algorithm in the SciKit-Learn library that selects optimal hyperparameters for each ML algorithm to identify the best differentiation model. A range of hyperparameter values can be assigned to the algorithm as inputs. The algorithm then builds models using each possible hyperparameter set from the ranges of the hyperparameters and shows the best hyperparameter settings for the selected ML algorithm. It also uses a CV method to find optimal hyperparameter values over k-fold CV .Leave-one-out cross-validation (LOOCV) was used to evaluate the performance of machine learning algorithms. Most often used cross-validation techniques are k-fold and LOOCV. For larger datasets, k-fold is preferable to LOOCV. Data are split into K sets for k-fold cross-validation, with one set serving as the validation set for each iteration. In comparison, LOOCV is a special case of k-fold that employs test and training data from each sample in the dataset. LOOCV chooses one sample from the data as a validation set so that each sample can reflect the test data. However, utilizing several trained and testing models by LOOCV estimates more reliable outcomes, thus it is suitable for small datasets ,27.In ML algorithms, true positives (TP) are the positive classes that the model correctly classifies, and true negatives (TN) are the negative classes that are classified correctly by the model. False positives (FP) are the classes that the model incorrectly classifies as positive, and false negatives (FN) are the classes that are incorrectly classified as negative by the model .To evaluate the ML algorithms, six evaluators compared the performance of the established models, including accuracy, receiver operating characteristic (ROC)–area under the curve (AUC), specificity, precision, recall, and F1_score. Accuracy measures the ratio of correctly predicted samples to the total number of samples evaluated ((TP + TN)/(TP + FP + TN + FN)) . Specifi1H-NMR spectra using three experimental replicates from white rice  and brown rice  samples, respectively.The putatively assigned peaks for white and brown rice are presented in Twenty-four metabolites, including 12 amino acids, 5 organic acids, 3 sugars, 1 alcohol, and 3 others, were identified in the rice samples using a one-dimensional NMR technique. Among the 12 amino acids, isoleucine, leucine, methionine, threonine, and valine were identified as essential amino acids. Acetate, malate, fumarate, glycolate, and succinate were found to be the organic acids. Sugars found in the rice samples included glucose, maltose, and sucrose. Two-dimensional NMR spectroscopy  was performed to support the identification of various metabolites by one-dimensional NMR .Like other agricultural crops, metabolic profiles of rice are influenced by genotype and various environmental factors, such as rainfall, temperature, and soil ,30. This1H-NMR data. In the PCA score plot  collectively accounted for 68.7% of the total variation, with values of R2X = 0.984, and Q2 = 0.692.In this study, PCA was used to objectively analyze the ore plot , the whi2Y and Q2Y) and parameters  of permutation for predicting Korean and Chinese white and brown rice samples with different normalization and scaling methods. For white rice samples, the model established with total area normalization and UV scaling showed the highest R2Y and Q2Y values, 0.673 and 0.566, respectively. Therefore, we selected this model as the optimal model. The permutation test was also satisfied with R2Y and Q2Y intercept values of 0.0731 and −0.196, respectively. OPLS-DA-derived score plots showed an explicit separation between the Korean and Chinese rice samples . Par-scaled data were used for brown rice to establish the model, and the best differentiation model was developed by applying an SVM (accuracy and ROC-AUC of 0.96 in the test set). AUC values of 0.99 (for white rice) and 0.96 (for brown rice) were determined through the ROC curve analysis for predicting the geographical origin of rice, which suggested that the discovered 24 metabolites might be utilized to distinguish between rice samples from Korea and China. Thus, the SVM model developed in this study can be used to distinguish and predict Korean and Chinese rice samples.In total, 105 and 87 rice sample spectra were analyzed using NMR. SVM showed better performance than other ML algorithms, as it offers great generalization ability for a small sample size ,54. Genen = 105) was randomly divided into development (n = 90) and validation (n = 15) samples. For brown rice, the entire dataset (n = 87) was randomly divided into development (n = 74) and validation (n = 13) samples. The performance scores (white and brown rice) were 0.96 or higher for the development set, whereas they were 1.0 for the validation set.Compared with mass spectrometry (MS)-based metabolic profiling, NMR-based metabolic profiling has advantages in rapid sample preparation and higher reproducibility . HoweverThe limitation of machine learning techniques compared with the widely used multivariate statistical analyses, such as PLS-DA or OPLS-DA, is the lack of information about the contributing factors (metabolites) for the differentiation of each group. The machine learning techniques should be employed when the main aim is the practical differentiation or prediction, rather than revelation of contributing factors.For practical use of established methods in this study, the expensive cost of the NMR equipment and its maintenance should be considered, especially in developing countries. Establishment and effective management of a nationwide centralized laboratory system can be a promising approach for the high-cost problem. In future studies, an extensive sample collection and analysis could be performed to establish a robust differentiation model for discriminating rice samples from various countries worldwide.This is the first study to discriminate the geographical origin of rice from Korea and China with two milling types (white and brown) using NMR spectroscopy coupled with the most widely used ML algorithms. The SVM-based classification showed the best results in the LOOCV and external validation of the white and brown rice samples. This study can be employed as a complementary and alternative approach to previously reported analytical techniques for the geographical discrimination of rice samples. The concept and results of this study could be used for establishing a robust model for differentiation of rice samples from various countries worldwide in future studies.


Gastric cancer (GC) is one of the deadliest cancers in the world, with a 5-year overall survival rate of lower than 20% for patients with advanced GC. Genomic information is now frequently employed for precision cancer treatment due to the rapid advancements of high-throughput sequencing technologies. As a result, integrating multiomics data to construct predictive models for the GC patient prognosis is critical for tailored medical care. F1 = 0.913) than traditional machine learning models. Furthermore, the GCS-Net not only enables accurate patient survival prognosis but also provides model interpretability capabilities lacking in most traditional deep neural networks to describe the complex biological process of prognosis. The GCS-Net suggested the importance of genes  and biological pathways  to GC, which is consistent with the results revealed in biological- and medical-related studies of GC.  In this study, we integrated multiomics data to design a biological pathway-based gastric cancer sparse deep neural network (GCS-Net) by modifying the P-NET model for long-term survival prediction of GC. The GCS-Net showed higher accuracy (accuracy = 0.844), area under the curve (AUC = 0.807), and F1 score ( The GCS-Net is an interpretable deep neural network built using biological pathway information whose structure represents a nonlinear hierarchical representation of genes and biological pathways. It can not only accurately predict the prognosis of GC patients but also suggest the importance of genes and biological pathways. The GCS-Net opens up new avenues for biological research and could be adapted for other cancer prediction and discovery activities as well. Gastric cancer (GC) is one of the deadliest tumors in the world and gastric adenocarcinoma (GAC) is the most common type of gastric cancer , with 95Many types of genomic data have been acquired as a result of the advancements of next-generationhigh-throughput sequencing technology, including DNA methylation , mRNA 66, miRNA Deep learning (DL) algorithms have recently demonstrated remarkable performance in handling multiomics nonlinear data and numerous DL-based cancer multiomics analysis methods have been developed. Based on the combination of clinical and multiomics data, Tong suggested an integrative predictive model for colon cancer . Using aAlthough these models have revolutionized the diagnosis and predictions of cancers, they tend to be black boxes with poorly interpretable models. Conversely, machine learning models based on interpretable biomedical information may contribute to cancer genomic discovery and clinical prediction –16. Hao Using multiomics data to analyze the complex biological mechanisms of cancer patient survival is crucial; however, high-dimensional, nonlinear data pose computational challenges for survival analysis. In this study, we integrated multiomics data and designed a gastric cancer sparse deep neural network (GCS-Net) by modifying the P-NET model for gastric cancer prognosis, which can not only perform patient survival prognosis but also describe the complex biological process of prognosis. The GCS-Net is biologically interpretable with nodes in the neural network corresponding to biological genes and pathways, which can capture the nonlinear and hierarchical effects of biological genes and pathways on gastric cancer patient survival. Applying the GCS-Net to long-term survival prediction of GC, GCS-Net's accuracy, area under the curve (AUC), and F1 score are all higher than those of traditional machine learning models. Furthermore, genes and biological pathways discovered to be significant in the GCS-Net were validated as important genes and pathways for GC in previous biological and medical studies.The remainder of the paper is organized as follows: R tool “TCGA-assembler 2” [https://tcga-data.nci.nih.gov/tcga/). The dataset contains two types of multiomics data: copy number variation (CNV), somatic mutation, and clinical data. Integrating copy number alteration and somatic mutation data helps to reveal and predict survival time due to genomic variation in gastric cancer. The dataset has 295 samples, including 295 mutation data and 293 CNV data.We used the mbler 2”  to downlhttps://reactome.org/download-data, which contains three files: the gene matrix file ReactomePathways.gmt, the pathway name file ReactomePathways.txt, and the pathway parent-child relationship file ReactomePathwaysRelation.txt. From the parent-child relationship file, we create a hierarchical network with four levels of pathways, one layer of genes, and one layer of characteristics.The GCS-Net network architecture is constructed based on the biological pathway database Reactome . We downLong-term survival (LTS) samples were those who lived for more than 60 months , while short-term survival samples were those who died in less than 60 months (non-LTS). We obtained 183 non-LTS samples and 42 LTS samples, of which approximately 20% were LTS patients.The CNV data were standardized to −2, −1, 0, 1, 2. CNV deletion was defined as −2 and CNV amplification as 2. Somatic mutation data were normalized to 1 and 0, with 1 denoting a gene with at least one site mutation and 0 denoting a gene with no mutation.We read the Reactome pathway file ReactomePathwaysRelation.txt, which contains the parent-child relationships in the pathway, and chose the human relationships by the keyword “HSA.” Then, we used the Python package NetworkX  to buildTo capture the relationship between gastric cancer information pathways and reduce network operations, we selected the first four layers to construct the pathway layers in the GCS-Net. In the directed acyclic graph, the directed edges point from parent pathways to child pathways they depend on, while in the GCS-Net, this is reversed, with the outputs of child pathway nodes serving as inputs of parent pathway nodes. Thus, the fourth layer of the directed acyclic graph is the first pathway layer in the GCS-Net, while the first layer of the directed acyclic graph is the last pathway layer in the GCS-Net.As shown in Based on the Reactome-based network relationship built by NetworkX, we use TensorFlow's high-level API Keras to build multiple linear layers, with each layer followed by dropout and then an activation function.x={x1, x2,…, xm}.The input layer represents feature data that need to be fed into the network for training, which is mutations and copy number variation data (encompassing copy number amplifications and copy number deletions) in this study. Each input node represents a feature and they are combined to form an m-column vector, denoted by A ∈ Pn×m, where n is the number of pathways in the first pathway layer and m is the number of genes in the gene layer, to encode the connections between the gene layer and the first pathway layer. We set the value of the element aij of A to one if gene j belongs to some pathway i, and zero otherwise. This is a sparse coding model established based on the relationship between genes and pathways.The gene layer consists of genes involved in the pathways of the first pathway layer. The connection between the input layer and the gene layer is established based on the fourth layer of the pathway database. Each node in the fourth layer of the pathway database is made up of a set of genes; thus, the connection between the input layer and the gene layer is a sparse connection, but not a full connection. We construct a binary adjacency matrix, M, where M is a binary matrix created from parent-child relationships between the Reactome pathways. During the forward propagation calculation of the network, the output vector y of each layer is jointly determined by the input vector x, the weight matrix W, and the mask matrix M, forming a sparse network model. The calculation formula is as follows:f is the activation function. For each node, we use the following tanh activation function:In the subsequent pathway layers, the connections between two adjacent pathway layers are determined by the pathway parent-child relationship in the Reactome pathway dataset and are stored in a binay mask matrix https://github.com/kundajelab/deeplift).To measure the importance of each node in the network model, we use the DeepLIFT  gradientt represents the target output and let x1, x2,…, xn represent some intermediate layer neurons that are necessary to compute the target output. Let t0 denotes the reference activation of t.We define Δt as the difference-from-reference:In this work, to calculate the importance of nodes in each layer, each node needs to be assigned a score. Let CxiΔtΔ to Δxi s.t.:CxiΔtΔ can be thought of as the amount of difference-from-reference in t that is attributable to the difference-from-reference of xi.DeepLIFT assigns contribution scores N represents the total number of samples, yi is the label corresponding to sample i, and pi is the LTS probability of sample i calculated according to the sigmoid function.We split TCGA gastric cancer data set (containing somatic mutation and copy number data) into 80% training set, 10% validation set, and 10% test set for predicting survival. To make the model training converge smoothly, we initialize the learning rate to 0.001 and reduce it actively after every 100 epochs. The model is trained using the Adam optimizer . We perfIn this work, we investigated the effectiveness of four traditional machine learning approaches in predicting the prognosis of stomach cancer . We utilized the scikit-learn package to implement these algorithms and used the default settings .wl is the number of weights in a layer l,  nl is the number of nodes in the same layer, and nl−1 is the number of weights in the previous layer. The formula for calculating the number of weights in a layer in the sparse network is as follows:M is the mask matrix of each layer, with each element in M being 1 or 0 depending on whether or not the corresponding connection path with the parent-child relationship exists. W is the weight matrix of the layer.There are much fewer weights in the GCS-NET sparse model than in a fully connected dense network with the same number of nodes. The number of weights of the sparse model is slightly higher than 83,347 , while tTraditional machine learning models such as decision trees, support vector machines, logistic regression, and random forests perform worse than the GCS-Net method. We trained the GCS-Net and these traditional machine learning models for long-term survival prediction of gastric cancer (GC), and the GCS-Net showed higher accuracy, area under the curve (AUC), and F1 score than previous traditional prediction classifiers (area under the receiver operating characteristic (ROC) curve (AUC) = 0.807, area under the precision-recall curve (AUPR) = 0.949, and accuracy = 0.844)  Figure .Evaluated on the test set, the GCS-Net model achieved a true negative rate of 75% (TN) and a true positive rate of 100% (TP), indicating that the model has a certain generalization and can classify samples that are not in the training set .To understand the connections and interactions between different mutations, copy number variations, genes, and biological pathways from input to output after training, we visualized the entire structure of the GCS-Net using a Sankey diagram .From the figure, we can see that compared with copy number variation, mutation has a greater impact on the prognosis, which is consistent with the related studies of gastric cancer. To obtain the importance of each node, we use the DeepLIFT attribution method to calculate the node's contribution score to rank the nodes. UBE2C, JAK2, RAD21, NUP210, PTPN1, CDC27, NUP188, and PLK4 were the top-ranked genes, and they have been reported in related gastric cancer studies .At the same time, in the hidden layer of pathways, we found that mitotic anaphase, antigen processing, recruitment of NuMA to mitotic centrosomes, neddylation, centrosome maturation, SUMO E3 ligases, G2/M transition, M phase, SUMOylation, and cell cycle have an important impact on the prognosis of gastric cancer. These pathways involve cell cycle checkpoints, posttranslational modification, and transcriptional regulation. These pathways have been studied in the relevant gastric cancer prognostic literature .The expression level of the mitotic checkpoint BUB gene family is closely connected with tumor cell proliferation, according to the literature , and theCompared with traditional machine learning methods, the GCS-Net has better performance and significantly reduces the number of learning parameters. More importantly, it has an excellent model interpretability. Using the DeepLIFT method to measure the importance of different genes and pathways in predicting results, the GCS-Net found known genes related to gastric cancer, such as UBE2C, JAK2, RAD21, CEP250, NUP210, PTPN1, CDC27, NINL, NUP188, and PLK4. In addition, the GCS-Net also discovered important biological pathways, such as mitotic anaphase, resolution of sister chromatid cohesion, and SUMO E3 ligases. These important genes and pathways are documented in relevant gastric cancer biology literature.Although our method has proved to be robust and reliable in predicting the prognosis of gastric cancer, there are still some concerns that need to be addressed. First, we found that the false-positive rate was high. One possible reason was the imbalance of samples in the dataset. Among them, there were only 42 samples with a good prognosis of gastric cancer with long-term survival greater than 5 years. Second, this experiment uses mutation data and copy number variation data in the multiomics data. If more omics data such as RNA and methylation data had been added, there might have been a higher prediction accuracy. Third, studies [Multiomics data analysis can be used to forecast cancer survival information. In this study, we developed the GCS-Net for predicting gastric cancer prognosis. The GCS-Net utilizes a biological pathway-based architecture and integrates multiomics data for prognosis prediction of gastric cancer.In the future, we will add more omics data for prediction, use cross-validation to reduce the performance impact of low sample size, and collect more sample data for modeling. In addition, we will optimize the interpretability of deep neural networks through optimization algorithms, such as loss functions, to further improve the accuracy of the model. We will also consider applying this model to the prediction of gastric cancer types, such as diffuse and intestinal types .Finally, the GCS-Net is a deep neural network with interpretable biological pathways for accurate gastric cancer prognosis. Neural networks based on biological information pathways offer a novel approach to biological discovery that might be used for a variety of additional cancer prediction and research applications. To more precisely assess the prognosis of gastric cancer patients, we will combine clinical data and multiomics data and analyze the effect of heterogeneity generated by diverse clinical characteristic data  on the prognostic risk of gastric cancer patients.


The ever-growing number of methods for the generation of synthetic bulk and single cell RNA-seq data have multiple and diverse applications. They are often aimed at benchmarking bioinformatics algorithms for purposes such as sample classification, differential expression analysis, correlation and network studies and the optimization of data integration and normalization techniques. Here, we propose a general framework to compare synthetically generated RNA-seq data and select a data-generating tool that is suitable for a set of specific study goals. As there are multiple methods for synthetic RNA-seq data generation, researchers can use the proposed framework to make an informed choice of an RNA-seq data simulation algorithm and software that are best suited for their specific scientific questions of interest. RNA sequencing (RNA-seq) technology has revolutionized the way we analyze the dynamic transcriptome. This technology overcomes many of the limitations of microarray platforms—it has the ability to detect unknown genes, is unbiased by antisense probe sequences, has a wider dynamic range to quantify expression and a more advantageous signal-to-noise ratio ,2. The mWhat must be acknowledged is that bulk RNA-seq does not consider the fact that tissues are composed of various cell types and that there is mounting evidence that gene expression in similar cell types is not homogeneous ,5,6. TheBulk RNA-seq could be used to obtain an estimate of the mean of the gene expression from a cell population for detecting the presence and measuring the quantity of transcripts in a tissue sample at a given moment. This makes bulk RNA-seq best suited to identifying differences between sample conditions. The standard workflow includes wet lab experiments and downstream computational analyses. The wet lab part consists of RNA extraction and isolation, library preparation , and sequencing to a read depth of many millions of reads per sample, with the newest technology producing billions of read-pairs per lane on a high-throughput sequencing machine. Bulk RNA-seq usually requires a small initial amount of RNA. The computational analysis part incorporates filtering out low quality reads and adapter sequences, alignment to reference transcriptome, read abundance quantification, gene-based read counting and filtering, and normalization between samples and batches. The relevance of bulk RNA-seq is supported by large public databases  and its common use in translational research .While scRNA-seq experiments share some laboratory preparation steps with the bulk RNA-seq processing pipeline, they have several specific steps involving single cell dissociation and capture, which is critical for the quality of the obtained data and the downstream analysis. The main technologies of scRNA-seq are plate-, microfluidic- and droplet-based . The comhttps://singlecell.broadinstitute.org/single_cell (accessed on 8 November 2022), the Hemberg collection [scRNA-seq data are still not as widely available as bulk RNA-seq data and are much more expensive to produce. The main factors adding to the cost at the time of writing are the costs of the reagents as single cells are processed in individual reactions; the cost of sequencing as many more reads need to be generated; and the technology itself, which requires more research and development . scRNA-sllection , SCPortallection , and scRllection ), a largllection .After the initial computational processing, researchers are most commonly interested in the analysis of differentially expressed genes (DEGs). The main difference in DEGs analysis between bulk RNA-seq and scRNA-seq is that in the case of bulk RNA-seq it is often aimed at differentiating between case vs control conditions , while in the case of scRNA-seq it is aimed at detecting biomarkers across cell types ,23. An eThe second most common type of analysis aims to discover sets of transcripts that can discriminate between different biological phenotypes, i.e., statistical methods, and supervised or unsupervised machine learning algorithms; thus, these methods attempt to hypothesize about the presence of phenotypes and their relation to experimental conditions or disease states. The common setup of bulk RNA-seq experiments, with samples in the hundreds and measured transcripts in the thousands, leads to the so-called “curse of dimensionality” problem which is further exacerbated by rapidly increasing computational costs. Because of these issues, the analytical approaches are usually coupled with dimensionality reduction techniques . An asseGene expression data can be studied not only at the level of individual genes, but also as gene lists, regulatory pathways or networks because genes interact to support biological processes. Gene regulatory network inference could be based on simple correlations among the measured variables (co-expression) or non-linear associations among subsets of the transcripts represented in the RNA-seq data. Due to the fact that co-expressed genes can often be functionally related, controlled by the same set of transcriptional factors, or part of the same pathway, deciphering co-expression networks can help in clarifying biological modules governing specific biological processes . InteresAdditionally, small changes in the expression of some genes may not be detected as statistically significant or informative when each gene is considered in isolation. However, their joint activity could have impactful biological consequences. Therefore, the differential enrichment analysis of biological pathways can sometimes provide a better biological interpretation than focusing on individual genes . An exteWith the increased interest in developing new methods for analysis of bulk and scRNA-seq data, the need for evaluating methodologies and benchmarking the implementations of the associated algorithms also increases. Some of the most popular metrics used for such benchmark performance evaluation are the false discovery rate (FDR) and sensitivity , as wellAs the amount of transcripts in the RNA-seq raw data is measured in non-negative integer counts, and taking into consideration the systematic biases of the sequencing platforms, the Poisson distribution was first proposed to model RNA-seq data, e.g., in . That diA well-known property of scRNA-seq datasets is that they contain a relatively large proportion of genes with zero counts: the proportion of zeros in bulk RNA-seq data is 10–40% ,49, whilCurrently, there are a multitude of scRNA-seq data simulators available, with the majority of them estimating features of a real single-cell dataset in order to generate data. The earliest methods in this direction utilize the negative binomial distribution ,53,54, oVarious synthetic data generators for bulk RNA-seq data have been proposed since the inception of the technology. The early methods simulating bulk RNA-seq data utilize FASTA, SAM or BED files as the input and have goals such as the evaluation of alignment algorithms, studying the experimental biases of the sequencing platforms by mimicking the major experimental steps, or estimation of transcript expression when comparing de novo transcriptome assemblers . SynthetThe following sections describe a general framework, as shown in As there is a variety of available software aimed at simulating RNA-seq data, it is important to compare performance in a systematic fashion. Our paper introduces a general computational framework that could be used to select a synthetic RNA-seq simulator to generate a sufficient number of samples, appropriate for the evaluation of a specific analysis method or to benchmark several such algorithms against each other, as shown in When selecting algorithms for benchmarking, one should consider their underlying parametric models for data generation, or lack thereof in the nonparametric case. Parametric approaches use experimental datasets to infer the parameter values for an assumed distribution, whereas nonparametric approaches use experimental datasets as baseline data add add effects to it, and are model-free; therefore, they are considered to be fast . The evoIn this article, we focus on one specific application of the proposed computational framework and consider the following important benchmarking criteria in the case of bulk RNA-seq data: (i) distributional characteristics of generated synthetic data; (ii) number of DEGs; and (iii) class-covariance structure, as shown in Because the software packages of interest were originally designed to have real bulk RNA-seq data as input, we used the AD RNA-seq dataset  for comphttps://github.com/Felitsiya/Comparative-study-of-synthetic-bulk-RNA-seq-generators (accessed on 8 November 2022)), for purposes of comparison with the real RNA-seq data used in our benchmarking, the Alzheimer’s disease (AD) dataset. The NGSSPPG software utilizes the following two-step procedure to generate data: (1) modeling mRNA concentrations with a multivariate Gaussian model (MVN-GC); and (2) modeling NGS-reads using a Poisson process that takes the MVN-GC data as its input. The user can specify the following parameters: number of features, number of samples in each group, number of subgroups in group 1, number of global, heterogeneous and non-markers, average expression and average standard deviation for each group, number of correlated variables in a block structure, strength of block covariance, sequencing depth, and noise.We also used synthetically generated RNA-seq data with a known covariance structure  as tm. As in , the mea(i)We calculated the parameters needed for the benchmarked packages using the NGSSPPG synthetic RNA-seq datasets or the real RNA-seq data (AD dataset) as input for textbfcompcodeR: mean gene expression of class 1, and specific dispersions of class 1 and class 2, as recommended in the package manual. We used the default values for effect size and the minfact and maxfact parameters for the simulated samples’ individual sequencing depths.(ii)powsimR’s parameter estimation step was performed with the recommended settings for bulk RNA-seq data; the simulation step was performed with the DESeq2 differential testing method. For this particular simulation, we truncated the values above (iii)seqgendiff, we took either one of the two classes from the NGSSPPG dataset or the control group from the AD dataset. The added signal is from an exponential distribution with a rate of 0.5 and effect size of 1.5.As the input of (iv)SimSeq runs as a one-step procedure; therefore, we used its default parameters.(v)SPsimSeq, we set the genewiseCor parameter to FALSE; therefore, we chose not to retain the gene-to-gene correlation structure of the input data. This was to avoid the high computational cost of calculating/keeping that structure.For The procedure we used to simulate data consists of the following steps:The read count matrices produced by the respective synthetic data generation packages were used for the comparisons, including quantile-quantile (Q-Q) plots  and descOur first comparative metric is based on the Q-Q plots visualization of the similarities between the distributions of the input and the respective outputs of the five benchmarked packages. Q-Q plots illustrate the similarity between two given distributions. If the distributions of two datasets are similar, their quantiles should be close and the scatter plot should be close to the diagonal, as shown in Our second comparative metric is the Dispersion vs Biological Coefficient of Variation (BCV) plots, as shown in The third metric deployed in our benchmarking application of the general framework is based on the mean-variance scatter plots, as shown in The next comparative metric used in our application is based on the Spearman correlation coefficients distribution for pairs of features presented in the feature-feature correlation plots, Another metric used in our benchmarking application is the ability of the synthetic data generators to recapture the number or the percentage of DEGs observed in the input dataset. Note that there is an overwhelming presence of DEGs in the AD dataset, possibly due to the intrinsic nature of the sampled tissues. The synthetic data generators have been set to produce data with 5% DEGs i.e., 500 DEGs for the NGSSPPG-based data and 1731 DEGs for the AD-based data, as shown in As part of the testing for DEGs, we also generated volcano plots. Volcano plots illustrate the statistical significance of the expression difference relative (y-axis) to the magnitude of difference (x-axis) for every transcript in a comparison between two groups . Figure Our final tool for benchmarking the five synthetic bulk RNA-seq data generators is based on principal component analysis (PCA). It demonstrates the separability of groups or classes potentially present in the respective datasets, as shown in One of the critical problems related to the evaluation of statistical and machine learning methods for RNA-seq data analyses is the lack of knowledge about the true real data multidimensional distributional properties. Therefore, the generation of synthetic RNA-seq datasets, which can serve as ground truth for such an evaluation, has become a topic of significant interest in the research community. Currently, there is an ever-growing number of methods and software packages for generating synthetic RNA-seq data. Many studies focus on the performance of methods analyzing DEGs, typically comparing the results, e.g., sensitivity, specificity and related performance metrics, and in some cases the statistical power. However, little attention has been paid to using a systematic approach for comparing the quality of the RNA-seq simulated data. In this paper, we propose a general framework to address the problem. We highlight several metrics that can be considered when comparing synthetic RNA-seq data, while also taking into account the properties present in the data used as input for the generation algorithms. The application of the proposed approach is illustrated by using five currently available software packages for the generation of synthetic bulk RNA-seq data. In our opinion, it is important to include both synthetic data (NGSSPPG) with a known structure, and real datasets (AD) in the proposed evaluation. The application of our general computational framework, as shown in