The tremendous rate with which data is generated and analysis methods emerge makes it increasingly difficult to keep track of their domain of applicability, assumptions, and limitations and consequently, of the efficacy and precision with which they solve specific tasks. Therefore, there is an increasing need for benchmarks, and for the provision of infrastructure for continuous method evaluation. APAeval is an international community effort, organized by the RNA Society in 2021, to benchmark tools for the identification and quantification of the usage of alternative polyadenylation (APA) sites from short-read, bulk RNA-sequencing (RNA-seq) data. Here, we reviewed 17 tools and benchmarked eight on their ability to perform APA identification and quantification, using a comprehensive set of RNA-seq experiments comprising real, synthetic, and matched 3′-end sequencing data. To support continuous benchmarking, we have incorporated the results into the OpenEBench online platform, which allows for seamless extension of the set of methods, metrics, and challenges. We envisage that our analyses will assist researchers in selecting the appropriate tools for their studies. Furthermore, the containers and reproducible workflows generated in the course of this project can be seamlessly deployed and extended in the future to evaluate new methods or datasets. Alternative polyadenylation (APA) is a mechanism of RNA processing that generates distinct 3′ termini, allowing the expression of multiple transcript isoforms from a single genomic locus . The choGiven the importance of APA, identifying and quantifying the usage of polyadenylation sites on a transcriptome-wide scale is critical for understanding both the underlying mechanisms and functional implications of APA-mediated gene regulation. Early studies of APA used microarray platforms and discovered widespread changes in PAS usage . While tArabidopsis in their analysis. While the study provided assessments of the precision and sensitivity of PAS site prediction, the RNA-seq data were pre-processed using different tools, making the results difficult to interpret and compare. Moreover, the code was not presented in a manner that made it easily reproducible and extendable. Other benchmarking efforts have similar shortcomings  sites from RNA-seq data. This international effort in the RNA community had several goals: (1) To bring together RNA biologists, bioinformaticians, and developers within the RNA Society to improve crosstalk between these communities; (2) to provide informative benchmarking results for current methods for APA detection and quantification; and (3) to develop a framework for reproducible, cloud-based benchmarking for bioinformatic tools. This benchmarking infrastructure was designed to be modular, extendable, and standardized at all levels, with the idea that additional tools or metrics could be added in the future, or the infrastructure applied to the benchmarking of other types of tools.https://openebench.bsc.es/benchmarking/OEBC007).de novo identification of sites based on changes in the read coverage along the mRNAs, as this is expected to drop at mRNA 3′ ends. This category includes DaPars, DaPars2, GETUTR, TAPAS, IsoSCM, and APAtrap  sites, including QAPA  and PAQR APAtrap .One of the main goals of APAeval was to make all benchmarked methods accessible for the larger RNA community. Thus, beyond the performance of the method, we assessed additional factors that included the ease of installation, tool accessibility for new users, the breadth of use of the method (a metric biased towards older methods), the responsiveness of the authors to email questions or GitHub issues, and whether or not the method is currently being maintained (a metric biased towards recent methods). Based on these criteria, we evaluated 17 methods for their ability to perform our benchmarking challenges . This liTo be able to evaluate predictions of a computational method, a ground truth, independent  dataset is necessary. In our case, the ideal dataset would have RNA-seq data for samples where the precise abundance of transcript isoforms is known and can be directly compared with the abundance inferred by the method for APA site inference. Quantifying transcript abundances genome-wide with a method different than RNA-seq is not generally done. However, there are studies in which both RNA-seq and 3′ end sequencing data have been obtained from the same experimental system. Therefore, we used two types of “ground truth” estimates of PAS usage: 1) from targeted sequencing of mRNA 3′ ends; and 2) from “realistic” simulated data, where transcript isoform expression levels were sampled from the distribution constructed by RSEM in a specific GTEX sample .Based on the above ground truth definitions, we searched for publicly available RNA-seq datasets with matching 3′-end sequencing data. To represent diverse biological conditions, we selected both human and mouse datasets, originating both from cell lines (HEK293 or P19 cells) and primary tissues (cortex and immune cell populations). Moreover, to avoid potential biases caused by technical characteristics of datasets, we chose datasets with varying sequencing depths (30 to 200 million reads) and from both paired and single-end sequencing. The ground truth data was obtained with a broad range of techniques for 3′ end sequencing, namely 3′-seq , MACE-sede novo poly(A) sites; 2) quantification of the usage of individual poly(A) sites within the transcriptome; and 3) quantification of the relative usage of poly(A) sites compared to others within the same terminal exon (TE). For each of these events, specific metrics were defined (see below) and then computed for each dataset pair separately.APAeval benchmarking was divided into three “events”, according to the different tasks the evaluated methods perform: 1) identification of https://github.com/iRNA-COSI/APAeval/) and additional details can be found in the Material and Methods section. We used standard data formats, BAM files for aligned reads, FASTA files for nucleotide sequences, GTF files for gene/transcript model annotations and BED files for reference poly(A) sites. We included all the code necessary to make the inputs compatible with a method as well as to generate the standardized outputs in the respective method workflow. To ensure reproducibility and extensibility of our work, whenever Docker or Singularity containers for a method were not readily available, we custom-built them. As some methods were developed for a specific genome annotation, we carried out the analysis both with the preferred annotation of the tool and the corresponding GENCODE annotations   and workflows to compute the benchmarking metrics for all evaluated methods (“benchmarking workflows”). Detailed input and output specifications are available in the APAeval Github repository (r mouse) . A graphIdentification of poly(A) sites based on the profile of genome coverage by RNA-seq reads is challenging to achieve with nucleotide-level precision. For this reason we tested associating poly(A) sites identified from RNA-seq data by the benchmarked method (PD - prediction) to poly(A) sites detected in an orthogonal 3′-end sequencing dataset (GT - ground truth) with various degrees of tolerance in calling corresponding sites. Specifically, the coordinates of ground truth sites were extended by n nucleotides  in both directions  and the de novo poly(A) sites. We tested the tools both on our reference annotation  provided by a given tool. For simplicity of terminology, we call this “absolute quantification”. Three tools provided such values and were thus included in this analysis: PAQR, APAtrap, QAPA. QAPA and APAtrap were each tested with two different annotations, QAPA with GENCODE and a custom annotation provided by the authors, and APAtrap with GENCODE  and RefSGiven that the site identification from RNA-seq data does not generally have nucleotide-level precision, if one PD site matched multiple GT sites its score (expression level as TPM) was split between the GT sites into shares inversely proportional to the distance of the PD site to the respective GT sites. If multiple PD sites matched one GT site they were merged and their scores were summed up. By default, we included all sites predicted to be used, i.e., with expression level > 0, but we also explored the impact of filtering the data, using in the analysis only predicted and ground truth sites with TPM > 1. Additionally, we determined the fraction of the gene expression coming from unmatched sites (“pct-FP”), to evaluate whether the accuracy of quantification is associated with the ability of a method to correctly identify used PAS.de novo and quantifies their usage, both PAQR and QAPA, methods that quantify the usage of annotated sites, achieve better performance, as they focus on PAS that have been found to be functional in prior data  and PAS (GT-PAS), there are a number of parameters to consider for benchmarking. These include the window size for an allowable match between GT-PAS and predicted PAS (PD-PAS) as well as which transcriptome annotation to use when running each tool . Additionally, upon matching GT-PAS and PD-PAS, one must consider how to handle multiple matches and which GT-PAS values are most relevant for comparisons: values from the proximal PAS (GT-pPAS), the distal PAS (GT-dPAS), or all PAS considered together ). We describe GT-pPAS, GT-dPAS, and GT-allPAS in more detail below.Given a specific combination of the above parameter settings, we assessed how well the predicted PAS quantifications matched the high-confidence set of PAS by measuring correlation coefficients between the two, by plotting the distribution of absolute differences between prediction and ground truth relative quantification values, and by reporting the number of high-confidence TEs each tool reported on.Importantly, different tools report different types of relative usage. Some, like DaPars and DaPars2, report on each version of each TE in the annotation that was expressed, which can lead to the same PAS coordinate having multiple quantification values. Other methods, like QAPA, only report on the most distal, composite TE in a gene and report all annotated PAS regardless of expression level. Other tools we also considered, like LABRAT and APAlyzer, quantify PAS usage per transcript, but do not provide specific coordinates of the potentially collapsed PAS clusters or inferred PAS and therefore we did not include these in the benchmarking. These disparate approaches to produce PAS quantifications can lead to multiple valid matches between prediction and ground-truth PAS quantifications for one or more PAS within each TE. See, for example, a priori from the outputs, therefore we focus on the results for all tool predicted output PAS matches to GT-PAS . The results for the best possible tool predicted PAS quantification to GT-PAS (best-PD) are shown in To overcome this, we performed the benchmarking based on all PD-PAS matches output for a given window  and also on only the best possible GT- to PD-PAS matches (best-PD). For a tool that outputs multiple quantifications for a single PAS, it is difficult for a user to prioritize the best version of the TE or PAS The effect of choosing to correlate all-PD matches compared to only the best-PD matches was apparent at different window sizes, where most tools had higher correlations using a window size of 100 nt considering only the best-PD match, but similar or worse correlations when considering all-PD matches at this window size . PAQR and TAPAS performed similarly for all window sizes when considering PD-all or only PD-best matches . Given tDue to partial, incomplete, or redundant matches between a tool’s predicted PAS and the high confidence set of GT-PAS, choosing which set of GT-PAS value(s) to correlate with the predicted values can also have an effect. We considered all GT-PAS together , GT-dPASWe also considered how annotation may influence method performance. We found that the more conservative annotation, RefSeq, which was often mentioned in many tools’ documentations , led to better correlations for many of these tools, particularly on the simulated dataset for APAtrap, DaPars, and DaPars2 Compare . We noteGiven the above results, we chose to visualize the distributions of absolute differences between all-PD matches to different GT-PAS values using each tool’s preferred annotation with an overlap window of 50 nt  sites from RNA-seq data, as well as set up an extendable framework for carrying out continuous benchmarking. We strove to package tools for easy installation and use, as well as developed workflows that could be applied to new datasets or used for incorporating new tools into the benchmark. We also made an effort to identify appropriate ground truth datasets and preprocess them for use not only in our study, but also in future studies of computational methods for APA analysis. By benchmarking many of the currently available tools, we provide researchers with the basis for making informed decisions on which methods best fit their application(s).From the 17 surveyed tools, we were able to benchmark eight tools across five distinct ground-truth datasets. Our reasons for not being able to include all tools varied, from not being able to install or run them, to the tools generating outputs that could not be transformed into the basic metrics for the respective task, i.e., PAS identification or quantification . That isab initio, but rather use pre-defined PAS to assess their usage , generally have better performance than methods that do not use such information. However, the advantage of considering only known APA sites clearly comes at a price, especially for species that are not as well annotated, or when researchers study polyadenylation in contexts when novel sites are likely to play a role. We found that TAPAS has the highest accuracy for PAS identification, while for quantification, the accuracy varies widely between methods. The correlation of PAQR-predicted PAS usage with the ground truth is consistently higher compared to other methods such as DaPars(2), although PAQR consistently quantifies fewer sites. Thus, if researchers are interested in a higher accuracy set of PAS quantifications PAQR may be preferable, but if a broader coverage of PAS is needed for downstream analyses, this could be obtained with methods such as QAPA, TAPAS and DaPars. Also, users should take into account that in general, proximal PAS are much less accurately quantified and far fewer are properly identified compared to distal PAS, particularly for methods that infer proximal sites de novo, from the RNA-seq read coverage.The results show that, in general, the methods strike different balances between sensitivity and specificity. The highly non-uniform coverage of genes by RNA-seq reads makes it very challenging to reliably identify the drops in coverage that reveal the PAS. Thus, methods that do not identify the PAS The above results are generally in line with other recent efforts to compare methods for calling PAS from RNA-Seq. Specifically, the closest benchmarking effort  includedImportantly, even though we made a strong effort to identify appropriate ground truth data, having diverse “gold standard” data remains a challenge. Any orthogonal method for measuring 3′ end usage will exhibit a different bias compared to RNA-seq, and therefore, the overlap between predictions and ground truth will be inherently limited. Furthermore, even when comparing replicates of the ground truth datasets, the Jaccard index of the identified sites was roughly between 0.5 and 1, and the Pearson correlation coefficient roughly between 0.75 and 1 (data not shown). These values set the upper bound on the performance of the computational methods.Given these limitations of the methods for quantifying APA from RNA-seq data, the question arises as to their utility overall. Ideally, studies of APA would use direct measurements of APA isoform abundance, obtained on platforms such as PacBio or Oxford Nanopore, which are designed for sequencing full-length cDNAs. However, substantial efforts have been put in the past decades into the profiling of samples obtained from tumors by short read RNA sequencing. Such datasets are much more extensive than datasets where 3′ end sequencing has been applied. Thus, the interest in mining RNA-seq data to uncover APA remains, until perhaps 3′-biased single-cell sequencing will have generated comparable coverage of human cell types and diseases.Dataset preprocessing of the RNA-seq data was performed using the nf-core/rnaseq v3.8.1 RNA-seq pipeline  with opt--profile docker --aligner star_salmon --save_reference--gencode --save_trimmed --skip_markduplicates --skip_stringtie--save_unaligned --skip_bbsplit6 and divided by the total read count in the sample to obtain the expression level as TPM. All ground truth files were converted to a BED6 format for benchmarking. If applicable, PAS clusters were collapsed to their representative site to obtain single nucleotide PAS.For ground truths from A-seq2 and 3′-seq protocols, the raw data was downloaded from SRA  where the last nucleotide of each quantified transcript was defined as the poly(A) site.Simulated RNA-seq data based on transcript level expression quantification from GTEx v8 were used from . We seleFor each of the APAeval benchmarking events, specific metrics were defined and then computed for each dataset pair (predicted - ground truth data) separately.https://github.com/iRNA-COSI/APAeval/).The benchmarking infrastructure was broadly separated into two modules: workflows to execute individual methods (“method workflows”) and workflows to compute the benchmarking metrics for all evaluated methods (“benchmarking workflows”). To ensure compatibility between those two modules, the output of the method workflows was required to adhere to a previously defined standardized format, namely the well-known BED format. Thus, 1) the genomic location of poly(A) sites had to be reported as single-nucleotide; 2) the absolute quantification of PAS was done as TPM (Transcripts Per Million); or 3) the relative quantification of PAS was done as fractional usage compared to one or more additional PAS within the same TE. Detailed input and output specifications are available in the APAeval Github repository (A method workflow was developed for each participating method using either the Snakemake  or Nextfhttps://github.com/inab/TCGA_benchmarking_workflow). In the workflows three distinct containerized steps were executed; validation of the file formats created by the method workflows, computation of the metrics defined for the respective benchmarking event, and consolidation of results into OEB compatible JSON files. Those files were used for uploading APAeval results to the OEB database, and the metrics were extracted to create the figures presented in the manuscript. All the code is available on GitHub .Benchmarking workflows were created in Nextflow, based on previously published guidelines from the OEB initiative  sites identified from RNA-seq data by the benchmarked method (PD - prediction) to poly(A) sites detected in an orthogonal 3′end sequencing dataset (GT - ground truth). To achieve this we used BEDTools window . MatchinIn the identification metrics, we define true positives (TP) as predicted sites that fall within windows of specified size (see above) around GT sites. In contrast, false negatives (FN) are GT sites that do not have a matching prediction, and false positives (FP) are predicted sites without a GT match. Accordingly, we calculated the precision (TP / (TP + FP)), sensitivity (TP / (TP + FN)) and Jaccard index (TP / (TP + FP + FN)) for a range of window sizes.Finally, a metric evaluating the prediction per gene was calculated, called “Percentage of genes with correct number of PAS”. For this, we obtained the number of PAS per gene from the GT. Genes with no PAS in the GT were not considered further. Similarly, the number of PAS per gene from unique PD sites was calculated. Finally, the number of genes with the same number of PAS in GT and PD was calculated and divided by the number of genes with at least one PAS and multiplied by 100 to obtain the percentage of genes with the correct number of PAS.matched sites. To assess a method’s ability to focus on relevant sites we also determined the fraction of the gene expression coming from unmatched sites (“pct-FP”). For that, the percentage of TPM expression of non-matched predicted sites (FP) was calculated by summing the TPM expression of all predicted sites without a GT match and dividing by the sum of expression values of all predicted PAS (FP + TP). Although the absolute quantification event aims to assess the performance of methods in the “true” quantification of PAS expression, we provide the sensitivity and precision metrics as defined above, as well as the F1-score (TP / (TP + 0.5*(FP + FN))) to shed light on the sets of PAS each method considers.Given the matching procedure described above, the main quantification accuracy metric we used is the Pearson or Spearman rank correlation of the normalized prediction and the ground truth expression (TPM) values for For relative quantification benchmarking, we first defined sets of high confidence TEs that exhibited alternative polyadenylation (APA) to be detected by the various tools. The procedure is outlined in a priori which TE version or PAS quantification is correct we calculated metrics based on all matched PD-PAS usage values together, including duplicates , or by only considering the best PD-PAS usage value that minimized the difference to GT-PAS usage (best-PD matches). Finally, to shed light on the number of TEs with APA and the types of PAS  each tool detects and quantifies in this relevant subset, we calculated the fraction of GT-TEs with APA that had matches to any PD-PAS as well as the fraction that had a PD match to the GT-pPAS site, the GT-dPAS site, or to both.We applied the PAS window matching strategy described above, to match the filtered ground truth (GT-PAS) and the tool predictions (PD-PAS) and then calculated metrics over these matched sites. We calculated Pearson correlation between matched GT and PD relative usage values and plotted the empirical cumulative distribution functions (eCDFs) of absolute differences between GT-PAS and PD-PAS PAUs. Importantly, for these metrics we also considered different subsets of matches between GT and PD to identify potential shortcomings in the tool’s ability to quantify either the pPAS or the dPAS accurately. Therefore, we calculated these metrics for all GT to PD matches , just the matches to GT-pPAS, and just the matches to GT-dPAS. Some methods provide duplicate quantifications for the same PAS coordinate because they consider each transcript-defined TE separately, leading to some multi-matched sites (see, for example, Supplement 1


Gene expression profiling is increasingly being utilised as a diagnostic, prognostic and predictive tool for managing cancer patients. Single-sample scoring approach has been developed to alleviate instability of signature scores due to variations from sample composition. However, it is a challenge to achieve comparable signature scores across different expressional platforms.singscore. We assessed the reproducibility and the performance in reporting immune profile of singscore based on NanoString assay in advance melanoma. To conduct cross-platform analyses, singscores between the immune profiles of NanoString assay and the previous orthogonal whole transcriptome sequencing (WTS) data were compared through linear regression and cross-platform prediction.The pre-treatment biopsies from a total of 158 patients, who have received single-agent anti-PD-1 (n = 84) or anti-PD-1 + anti-CTLA-4 therapy (n = 74), were performed using NanoString PanCancer IO360 Panel. Multiple immune-related signature scores were measured from a single-sample rank-based scoring approach, singscore-derived signature scores reported significantly high scores in responders in multiple PD-1, MHC-1-, CD8 T-cell-, antigen presentation-, cytokine- and chemokine-related signatures. We found that singscore provided stable and reproducible signature scores among the repeats in different batches and cross-sample normalisations. The cross-platform comparisons confirmed that singscores derived via NanoString and WTS were comparable. When singscore of WTS generated by the overlapping genes to the NanoString gene set, the signatures generated highly correlated cross-platform scores (Spearman correlation interquartile range (IQR)  and r2 IQR ) and better prediction on cross-platform response (AUC = 86.3%). The model suggested that Tumour Inflammation Signature (TIS) and Personalised Immunotherapy Platform (PIP) PD-1 are informative signatures for predicting immunotherapy-response outcomes in advanced melanoma patients treated with anti-PD-1-based therapies.singscore based on NanoString data is a feasible approach to produce reliable signature scores for determining patients’ immune profiles and the potential clinical utility in biomarker implementation, as well as to conduct cross-platform comparisons, such as WTS.Overall, the outcome of this study confirms that The online version contains supplementary material available at 10.1186/s12967-023-04092-9. Other studies have demonstrated the gene expression patterns by NanoString nCounter® PanCancer IO 360™ are highly correlated to other platforms, including WTS and HTG EdgeSeq , , and ), and intercepts close to 0  , 2-transformed count data showed Spearman correlation (r) between 0.57 to 0.86 on 762 overlapping genes in homologous samples, and 33 out of 35 samples had r > 0.7  singscores between the NanoString and WTS platforms in all pairs of comparisons when using all signature scores. The three comparisons, “NS to WTS all genes”, limited to just the overlapping genes “NS to WTS part”, and “NS skewed to WTS all genes”, have high and similar correlations r  and r2    and r2 , while the “NS to WTS part” method had generally higher scores in these two values   Fig. A. The “N2 ,2 ,r ≥ 0.8), and the “NS to WTS part” method displayed more highly correlated signatures (40 signatures with r ≥ 0.8)  low AUC (73.5%) then the values from other methods  when predicting overlapping WTS samples . The model built on singscores from “No stable gene” method to predict WTS samples by “all” method classified the majority of WTS samples as responders, and the singscores from the “HK genes” methods generated a model that misclassified many WTS samples into non-responders  of testing datasets from NanoString samples were similar in three different logistic regression models. In the performance of predicting the overlapping WTS sample, the models built on singscores from “No stable gene” method has better prediction of WTS samples by “all” method when using the optimal threshold  different signatures included multiple immune-related and cell signalling pathways. Many PD-1, T-cell, and IFN-γ-related genes and signatures showed significantly high levels in responders, which correspond to preferable response in immunotherapies , 14, 27.p-values ≤ 0.05, 74 p-values ≤ 0.05). Many signature genes highlighted in the WTS study were also upregulated in responders with an adjusted p-value ≤ 0.05 . Although genes in the NanoString nSolver in-built pathways in our study differed from the genes in KEGG pathways used in the WTS study, some significantly different pathways reported in nSolver analysis were also found in the WTS study, such as Cytotoxicity, JAK-STAT Signalling, and Cytokine and Chemokine Signalling [As a highly sensitive, reproducible, and robust technique for evaluating targeted gene expression profiles, our NanoString assay also reported high correlations to WTS data in matched samples in the log-transformed count data . Differesingscores also support the robustness and reproducibility of singscore. Most of the NanoString nCounter® PanCancer IO 360™ genes are intermediate and high expression genes compared to the WTS ranked profiles. It resulted in that singscores derived from the NanoString platform were generally lower scores under the same scoring method from WTS. While highly correlated, the skew in gene expression distribution will lead to different singscores between platforms and be problematic in some cross-platform analyses, such as model building and prediction. Our study suggests two possible solutions to alleviate this impact. One is to subset the WTS data to analyse only the overlapping genes (“part” method). The other way is to modify the ranks in the NanoString platform by fitting the raw ranks using linear regression coefficients (“Skewed ranks” method). Many samples’ ranks follow a uniform distribution. However, some samples contained more low signal probes (raw count ≤ default background noise 20). The transformation did not change their ranks . Although the “Skewed ranks” method provided highly correlated and similar cross-platform singscores and good prediction, it is not a generalisable method. When crossing multiple expression datasets or another dataset with a different number of genes, the coefficients for skewing need to be adjusted per assay. Therefore, focusing only on the overlapping genes may be a simpler and more feasible option. Additionally, although using a different probability threshold to predict binary responding outcomes  may alleviate this problem, it also leads to a higher risk to misclassify the true responders as non-responders.In cross-platform comparisons, highly correlated singscore variation was to introduce “stable genes” to calibrate across samples. Through analysing the perseveration of gene ranks across multiple platforms, Bhuva et.al introduced a list of stable genes, which is an in-built function in singscore [singscore. Additionally, this study attempted to identify platform-specific “stable genes” using a similar idea on all 762 overlapping genes. However, no suitable stable gene for cross-platform calibration could be identified between the NanoString Pancancer 360 IO panel and WTS. Therefore, when choosing the cross-sample stable genes for singscore, the stability in gene expression is not a good selective criterion. Checking the distribution and preservation of consistency in cross-platform ranks are important before introducing any stable gene for calibration.Another intuitive method to overcome this cross-platform ingscore . Howeveringscore . Howeversingscore based signatures in predicting immunotherapy response. Although such prediction was still not highly accurate, it can be further improved by combining other information, including tumour mutation burden [singscores. The consistent tendency can still be observed when crossing platforms. Site of biopsy used to generate the subsequent singscores should be considered within cohorts prior to model building as some signatures can differ significantly based on the tissue or origin. While the lymph node specimens displayed significantly different scores in multiple signatures compared to other biopsy sites, there were no statistically significant differences in the PIP PD-1 and TIS signatures between sites and or any association with responses. The predictive MCC indicated the good power of these two signatures in classifying response status in the testing datasets. Higher PIP PD-1 and TIS scores were observed in some non-responders, resulting in the internal prediction on training samples misclassifying them as responders. These patients may have immune-excluded tumours, with immune cells enriched in the stroma surrounding the tumour region but lacking infiltration in the intratumoural region [PDCD1, PDCD1LG2, CD274. Differential expression results in nSolver also labelled these genes as significantly highly expressed in responders. Among them, the PD-L1 (CD274) gene expression level is a validated biomarker for anti-PD-1 monotherapy and anti-PD-1 + anti-CTLA-4 in advanced melanoma [The LASSO regression was used to select more informative signatures and to test the clinical utility of the n burden , 28 and n burden . The PIPl region , 31. Themelanoma , 33. TISmelanoma , 34. Themelanoma . The conmelanoma , 37.® PanCancer IO 360™ can generate a similar immune profile to that generated by the WTS platform in advanced melanoma patients, and illustrate that the rank-based scoring tool, singscore, is also a reliable and practical approach to analyse the variations of immune signatures between response status and conduct cross-platform analysis.Consistent with our previous publication , we demoAdditional file 1: Figure S1. Expressional distributions of samples from NanoString and WTS assays; Figure S2. Heatmap of NanoString raw and normalized counts data; Figure S3. PCA plot of NanoString samples; Figure S4. Heatmap of singscores of significant signatures on NanoString samples by the “No stable gene”, “Skewed ranks”, and “HK genes” methods; Figure S5. Linear regression of NanoString repeats; Figure S6. Boxplot of HKGs expressions and ranks in NanoString and WTS samples; Figure S7. Plots of rank distributions of all and overlapping genes in NanoString and WTS platforms; Figure S8, S9. Linear regressions of overlapping samples’ singscores based on all signatures and highly correlated signatures; Figure S10. Signature level singscores consistency; Figure S11. Frequency of selected signatures by LASSO regression in 1000-time repeats.Additional file 2: Table S1. All NanoString samples metadata; Table S2. NanoString sample names map to the overlapping WTS sample names; Table S3. Repeating samples (derived from same patients) in NanoString assay; Table S4. Repeating Details of modification in WTS genes to match NanoString probes; Table S5. Details of Immune signatures used in this study.Additional file 3: Table S6–S8.Mann Whitney Wilcoxon test significance of signatures between response status on NanoString samples using signature scores from different singscore calculating methods. Table S9.Mann Whitney Wilcoxon test significance of signatures between site of biopsy on the lymph node or not.Additional file 4: Results of NanoString samples using advanced analysis in nSolver4.0. Table S10. Differential expression analysis on response; Table S11. Pathway scores. Table S12.Mann Whitney Wilcoxon test significance of signatures between response status on NanoString samples using nSolver pathway scores.Additional file 5: Table S13. Cross-platform Spearman correlations of 35 overlapping samples counts data; Table S14, S15. Gene rank consistency tables of 20 HKGs and all 762 overlapping genes.Additional file 6: Table S16. Cross-platform per signature Spearman correlations of singscores using different calculating methods; Table S17, S18. Cross-platform linear-regression coefficients and Spearman correlations of all and highly correlated signatures’ singscores using different calculating methods; Table S19. Confusion matrixes of cross-platform response predictions by logistic regression models using frequently selected signatures.


The assay for transposase-accessible chromatin with sequencing (ATAC-seq) is the most widely used method for measuring chromatin accessibility. Researchers have included multi-sample replication in ATAC-seq experimental designs. In epigenomic analysis, researchers should measure subtle changes in the peak by considering the read depth of individual samples. It is important to determine whether the peaks of each replication have an integrative meaning for the region of interest observed during multi-sample integration. We developed multi-epigenome sample integration approach for precise peak calling (MESIA), which integrates replication with high representativeness and reproducibility in multi-sample replication and determines the optimal peak. After identifying the reproducibility between all replications, our method integrated multiple samples determined as representative replicates. MESIA detected 6.06 times more peaks, and the value of the peaks was 1.32 times higher than the previously used method. MESIA is a shell-script-based open-source code that provides researchers involved in the epigenome with comprehensive insights. Recently, many studies have shown higher quality results by combining4. Researchers supplement the lack of consistency by including multi-sample replication in experimental designs. In this process, it is important to integrate the information from biological replicates. In epigenome analyses, including ATAC-seq, which require the extraction of peaks from individual samples to measure subtle changes in specific regions, require integrated analysis to determine whether the results of the measured replication are consistent. Furthermore, the measurement of peak epigenome levels via NGS is accompanied by the process of extracting significant signals compared to background signals using complex algorithms, such as hidden Markov models6. Therefore, it is important to determine an optimal peak within a region of interest to ensure the reliability of the research results by measuring the similarity of individual samples rather than by simple merging.Among the many methods for measuring chromatin accessibility, the assay for transposase-accessible chromatin with sequencing (ATAC-seq), which requires relatively little time and a smaller amount of samples for analysis, is widely used7. The length overlap-based approach is a method of determining the same peak based on the degree of overlap between the peaks of each replication, On the other hand, a statistical-based approach uses the reliable boundaries of reproducibility through statistical parameters to determine the reproducibility of peaks. In the case of the length-overlap-based approach, integration is possible regardless of the number of replications by focusing on speed and convenience; However, because reproducibility is derived based on the degree of overlap in length, the process of confirming statistical significance is not carried out. In the case of a statistical-based approach, replication integration is performed after determining the presentation and reproducibility based on the statistical background; however, only two replications are possible7. That means, no statistical background-based methods that are not significantly constrained by the number of replicates exist.Currently, methods used for ATAC-seq replication integration can be divided into two categories. One is the length overlap-based approach and the other is the statistical-based approachhttps://github.com/ERASMUSlab/MESIA).Here, we developed a novel replication integration method called Multi-Epigenome Sample Integration Approach for Precise Peak Calling (MESIA). MESIA determines replication with high representativeness and reproducibility in multi-sample replication and then integrates them to extract more robust and less likely false positive peaks. For researchers whose work may have been difficult to analyze due to a lack of computational background, a shell-script-based open-source code was disclosed to GitHub  were also used to verify the pipeline on other datasets. Sequence trimming was performed using Cutadapt9 and Trimmomatic10. Subsequently, the filtered fastq was aligned to the hg19 genome assembly for GM12878 and mm10 for MEFs using Bowtie211. The generated BAM file was sorted, filtered by MAPQ quality, and PCR duplicates and mitochondrial reads were removed using Samtools12, and multi-mapping control was performed using assign_multimappers.py13, provided by Encode. Finally, the filtered BAM file was divided by histone status using Sambamba and bedpeTn5shift14. MESIA takes BAM files as input and uses MACS215(v2.2.7.1) for peak calling. Through a series of processes, MESIA determines and integrates the reproducibility of multiepigenomic samples. , Samtools12(v1.13), Bamtools16(v2.5.1), Bedtools17(v2.30.0), Biobambam18(v2.0.183), Cutadapt9(v4.0), and Sambamba19(v0.8.2). Expression calling was performed using Salmon20(v1.7.0) and Kallisto21(v0.46.2).RNA-seq data obtained from two biological replicates were analyzed on the GM12878 human lymphoblastoid cell lineMESIA is a method of selecting only reproducible replications in consideration of representativeness in multi-replication ATAC-seq and then deriving an optimal peak within replications Fig. .Figure 222. MESIA stage 1 is process of determining whether two replications are reproducible to each other, indicating that Nt called “peaks consistent between true replicates” and Np called “peaks consistent between poured pseudoreps” are less than twice that of each other. Furthermore, an analysis is conducted to examine potential biases within the sample by generating pseudo-replicates for each replicate through random read splitting. These conditions are reflected in constraints (1) and (2), where the variables i and j can range from 1 to the number of replications being considered as inputs. Constraint (3), MS1pass, refers to the replication set that passed MESIA Stage 1, which is the process of deriving a replication set with reproducibility based on the previous analysis.Subsequently, it goes through MESIA stage 1 based on the IDRMESIA stage 2 uses Nt and Np of the replication set with the reproducibility selected in MESIA stage 1. The intersection of Nt and Np is called the base peak of the replication set, and MESIA stage 2, which passes the multi-replication peak, is obtained using the base peak. The base peak is labeled as BP in constraint (4). Similarly, within the MESIA Stage 1 passed replication set, the peak of a single replication is denoted as SP in constraint (5). High-ranking standards for calculating constraint  include the peak point, the peak point, distance from the start site to the highest signal site of the peak, and the Q value, which indicates statistical significance. Since base peak have already passed MESIA stage 1, the significant criteria for peak point and Q values were set to the medians of the base peak. This process is described in constraints (4) and (5), where the variable n represents the position of the largest value when sorting the peak points in descending order, and similarly, the variable m represents the position of the smallest value when sorting the q values in ascending order. Using these high-ranking standards, we derived a robust peak from the two replicates in the replication set. Constraint (6), MS2pass, refers to the MESIA Stage 2 passed peak.Finally, the MESIA optimal peaks are derived by merging the robust peaks of all replications sets that have passed through MESIA Stage 2. This process is described in constraint (7). Each step mentioned above are summarized into pseudocode Fig. . MESIA s23 (NAIVE)   Fig. A. NAIVE We needed a simulation sample with 7 characteristics to perform this simulation. Simulation samples A and B are simulation samples that are not reproducible with each other. Using the GM12878 replication1 ATAC narrowPeaks file, samples A and B with different trends were created. First, adjacent peaks were combined to form small set. Subsequently, the adjacent peaks were randomly sampled and divided into three large set. Select two large set out of three, and then divide the adjacent peak back into a narrowPeaks form. No matter how different the samples tend to be, there are basically shared peaks which were called body peaks. On the other hand, there must also be peaks that distinguish A and B, which we call function peaks. The body peaks and function peaks are combined to form A and B Fig. . The repTo evaluate the performance differences between the compared algorithms based on the reproducibility of the simulation sets, we defined three different states: standard, hard, and soft. Specifically, in the standard state, we used simulation set F, which has a 50% overlap with either simulation set A or B. However, in the hard state, we used simulation set E, which has only a 30% overlap. Conversely, in the soft state, we used simulation set G, which has a 60% overlap.To evaluate the performance differences of the compared algorithms based on the number of similar replications of the simulation sets, we defined three different states: strong, normal, and weak. In the normal state, we used cases of three and four replication sets that passed through MESIA stage1. In the strong state, we used cases of five and six replication sets that passed through MESIA stage1. Finally, in the weak state, we used the case of one and two replication sets that passed through MESIA stage1.Peaks formed in the promoter region derived from each method were used. Performance was compared based on the difference in the degree of gene expression related to the peak formed in the promoter. Gene expression information was derived from GM12878 polyA- and RNA-seq. it was aligned to the hg19 genome assembly.27 (v4.3). In the hard state, MESIA formed the fewest peaks in the distal intergenic region, while forming the most peaks in the promoter region when the region within 3 KB from the transcription start site was defined as the promoter region  . . .  Fig. B. We pre16) Fig. C. The ex Fig. B. p = 1.117e-13). To further analyze this difference, we conducted a comparison between only compared MESIA and NAIVE. We observed that 98.84% of the genes detected by MESIA overlapped with those detected by NAIVE.  We also compared MESIA with ChIP-R using a similar approach. The expression levels of genes detected using only MESIA were significantly higher than those detected using only ChIP-R. (p = 0.0352), respectively (Fig. 8, which are not human datasets. Therefore, it was revealed that MESIA maintains good performance even in datasets that are not human lymphoblastoid datasets. (Supplementary Fig. The performance was compared using real data. The distribution of the number of peaks in the four GM12878 replications is shown as a Venn diagram Fig. . The VenThe challenges of the diversity and integration of peaks discovered in multi-sample epigenomic analyses still exist. In this study, we proposed a new integrative method for multi-sample epigenomic analysis. This method combines epigenetic data from multiple samples to identify consistent peak patterns and will be valuable for genomic analysis of high-resolution genome and epigenome data.29. Our MESIA analysis identified conserved peak patterns associated with known regulatory elements, suggesting that our method can effectively identify biologically relevant peaks shared across multiple samples. Several studies integrated epigenomic signals from various disease models and computational approaches.Previous studies have shown that shared peaks across different cell types and tissues are associated with regulatory elements, such as enhancers and promoters, and play important roles in the regulation of gene expression32. Moreover, in cancer research, many researchers have used multi-sample epigenome peak analysis to identify epigenetic changes associated with cancer development and progression. Researchers have found that certain epigenetic modifications are more prevalent in cancer patients and have developed new therapeutic strategies targeting these modifications35.Epigenome peak analysis has emerged as a powerful tool for investigating the molecular mechanisms underlying common diseases, and numerous studies have used this approach to identify disease-associated epigenetic changes. Recent studies have used epigenomic peak analysis to identify novel therapeutic targets for Alzheimer’s disease, autism, and type2 diabetes. Researchers have identified unique peak patterns that are associated with the disease and used this information to develop a new diagnostic tool for disease36. The scABC is designed for single-cell epigenomic data and uses a Bayesian clustering approach to identify clusters of cells with similar epigenomic profiles across multiple samples or conditions. It can be used to identify cell type-specific and shared regulatory elements, and to study cellular heterogeneity37. Moreover, Episom uses a multi-sample peak-calling approach to identify shared and sample-specific peaks across multiple epigenomic datasets. It can also be integrated with other genomic data types, such as RNA-seq and ChIP-seq, to identify candidate regulatory regions and their target genes38. EpiTensor uses a tensor decomposition approach to identify shared and sample-specific epigenomic patterns across multiple samples and conditions. They can be used to identify regulatory elements and gene regulatory networks that are conserved or variable across different contexts39.There are several methods for integrating epigenomic peaks. For example, ChromVAR uses a latent variable model to identify shared and sample-specific regulatory elements across multiple epigenomic datasets. It can also perform differential analyses and prioritize disease-associated variantsAlthough some methods use statistical approaches, most use a length overlap-based approach. However, to the best of our knowledge, there are no methods that can integrate multiple samples while considering reproducibility between replicates. Specifically, most methods do not consider the reproducibility between replicates while integrating the peaks. Consequently, a statistical approach is vulnerable to noise caused by significant differences between samples. Our research aims to extract more accurate and reliable peaks by utilizing a statistical methodology that is more active than traditional multi-sample integration methods. To achieve this, we statistically extracted the significance of the individual peaks and integrated the peak information from multiple samples to determine whether each peak was significant. As a result, compared to other existing algorithms in situations where the degree of reproducibility is not sufficiently ensured, MESIA was able to detect peaks in up to 3.37 times more genes. Furthermore, the genes detected only by MESIA showed significantly higher expression levels than those detected by the other algorithms. In situations where the number of similar replications was insufficient, MESIA also identified genes that were significantly more highly expressed. These results show that MESIA, in addition to detecting genes with significantly higher expression by finding strong signals, also filters out false positive peaks that other algorithms miss. It does this by selecting peaks from reproducible replications. This methodology can be applied regardless of the sample size, allowing us to obtain valid results even with small sample sizes. To do this, appropriate statistical adjustments and simulations are necessary. For example, statistical adjustments were performed for all possible sample combinations and accurate results were obtained through simulations of individual combinations. As a result, we were able to obtain robust peaks that were not strongly affected by the degree of reproducibility and the number of similar replicates. Our study has some limitations. MESIA considers the reproducibility between replications before proceeding with Multi-Sample Integration, which may take a considerable amount of time and generate intermediate products proportional to the number of replications. As with all other methods, further validation using additional genomic data or experiments is necessary to increase the reliability of the results.40.The integration of multi-sample epigenome peak analysis has various applications. One is in the development of new therapeutic strategies. Epigenetic modifications have been shown to play a critical role in the regulation of gene expression, and aberrant epigenetic changes have been implicated in the development and progression of various diseases. By identifying disease-specific peak patterns, researchers can develop new therapeutic strategies that target these epigenetic modifications. For example, drugs that target specific epigenetic enzymes or regulatory proteins can be developed to correct aberrant epigenetic changes and restore normal gene expressionMESIA is a user-friendly, open-source tool that enables researchers to integrate various epigenetic changes easily and accurately. By shedding light on the sophisticated regulation of gene expression and disease development, this approach can provide valuable and novel biological insights.Supplementary Information 1.Supplementary Information 2.Supplementary Information 3.Supplementary Information 4.Supplementary Information 5.Supplementary Information 6.Supplementary Information 7.Supplementary Information 8.Supplementary Information 9.Supplementary Information 10.Supplementary Information 11.


Cancer is the second major cause of disease-related death worldwide, and its accurate early diagnosis and therapeutic intervention are fundamental for saving the patient’s life. Cancer, as a complex and heterogeneous disorder, results from the disruption and alteration of a wide variety of biological entities, including genes, proteins, mRNAs, miRNAs, and metabolites, that eventually emerge as clinical symptoms. Traditionally, diagnosis is based on clinical examination, blood tests for biomarkers, the histopathology of a biopsy, and imaging . Additionally, omics biotechnologies help to further characterize the genome, metabolome, microbiome traits of the patient that could have an impact on the prognosis and patient’s response to the therapy. The integration of all these data relies on gathering of several experts and may require considerable time, and, unfortunately, it is not without the risk of error in the interpretation and therefore in the decision. Systems biology algorithms exploit Artificial Intelligence (AI) combined with omics technologies to perform a rapid and accurate analysis and integration of patient’s big data, and support the physician in making diagnosis and tailoring the most appropriate therapeutic intervention. However, AI is not free from possible diagnostic and prognostic errors in the interpretation of images or biochemical–clinical data. Here, we first describe the methods used by systems biology for combining AI with omics and then discuss the potential, challenges, limitations, and critical issues in using AI in cancer research. Delayed diagnoses, misdiagnoses, and missed diagnoses impact patient health and safety, and have great societal consequences. Mistakes in diagnosis may account for up to 60% of all medical errors and are accountable for up to 80,000 deaths in U.S. medical centers each year . TypicalArtificial Intelligence (AI), a field of computer science used for prediction and automation, has emerged as a potential solution to promote a precision approach in healthcare and is expected to reduce errors caused by human judgment in various medical domains .Cancer is the leading cause of death in people, accounting for an estimated 10 million deaths by 2020 . It is aDespite difficulties in providing individualized and data-driven care, advancements in screening, diagnosis, treatment, and survival rate in cancer patients have been remarkable in recent decades . Early dThe development and extensive use of high-throughput technologies has ushered in the era of biological and medical big data. This has led to the accumulation of data sets on a large scale, thereby opening a wide range of potential applications for data-driven methods in cancer treatment, spanning from basic research to clinical practice: molecular tumor characterization, tumor heterogeneity, drug discovery and potential therapeutic strategies. As a result, the data-driven research field of bioinformatics adapts data mining techniques, such as systems biology, machine learning, and deep learning, which are discussed in this review paper. Systems biology uses a data-driven approach to identify important signaling pathways. The pathway-oriented analysis is extremely important in cancer research because it helps researchers comprehend the molecular features and heterogeneity of tumors and tumor subtypes . In thisAI-based technologies applied to oncology aim at improving clinical practice, including but not limited to the early and accurate diagnosis and prediction of personalized outcomes , by acquiring a profound perception of tumor molecular biology through the association of multiple biological parameters .AI is meant to mimic human cognitive abilities in elaborating the information but at a much higher speed and with no emotional interference. The main types of AI that apply to cancer-patient healthcare include machine learning (ML) and its evolved subtype deep learning (DL), which can assist in making a rapid and more accurate diagnosis , in discovering and developing new drugs, in designing personalized therapy, in predicting the therapy response, and in guiding the robotic surgery ,18 Figu.Current AI systems have been involved to be used in a variety of clinical settings, including (i) image-based computer-aided discovery and diagnosis in various medical specialties, (ii) the translation of genomic information for recognizing genetic variants using high-throughput sequencing technologies, and (iii) the prediction and tracking of patient’s prognosis ,20. MoreThe creation of AI models that predict the properties of vast and interconnected networks found in living organisms would allow for a thorough examination of how signaling molecules generate functional cellular reactions. Machine learning (ML) algorithms, a subset of AI, are capable of making decisive interpretations of large, complex data sets, making them an effective tool for analyzing and comprehending multi-omics data for patient-specific observations . We can Omics technologies allow for the in depth analysis of the molecular characteristics of cancer at both bulk and single-cell level, providing a wealth of multi-omics data that challenge the capability of scientists and medical doctor to combine for drawing a consistent picture of the multilayer complexity of cancer biology. Genomic, epigenomic, transcriptomic, proteomic, and metabolomic data can be elaborated using appropriate models for making predictions about prognosis and treatment response in a patient-tailored  manner ,15,22.To find cancer metabolic biomarkers, survival models have been used more frequently than partial least squares (PLS) models, ML models, and gene expression modeling (GEM)  Figure . The KapPartial least squares regression (PLS) was initially created as a regression model that processes numerous independent variables that are correlated and produce numerous dependent variables, which many statistical and ML techniques cannot directly handle. PLS models and their variations, particularly PLS-discriminant analysis (PLS-DA) are frequently used for the analysis of omics data with a focus on metabolomics . PLS-DA PLS-DA and its variants have been used to analyze metabolome data to identify a variety of cancers, including breast cancer, glioma, non-small cell lung cancer, oral precancerous cells, cervical precancerous lesions, and prostate cancer ,28. AmonGene expression modeling (GEM) is a computational model based on the law of mass conservation of metabolites and allows for the prediction of metabolic fluxes for entire biochemical reactions taking place inside a cell by using numerical optimization ,31. TechThe classification task of disease prediction has been thoroughly studied in medical oncology and cancer research, based on well-established machine learning algorithms for dealing with binary or multi-class learning problems. Patient categorization would allow for the development of ML-based predictive models capable of assessing risk stratification with generalizable performance. Based on images and genetic data, DL models were trained to classify and detect disease subtypes. These data-driven approaches demonstrated the superiority of ML-based frameworks for leveraging heterogeneous datasets for improved diagnosis and treatment .Deep neural network (DNN) models are rapidly evolving and becoming more sophisticated. They have been widely used in biomedical research across the board. Initially, large-scale imaging and video data aided its development. While most biomedical data sets are not considered big data, the rapid data accumulation enabled by NGS made it suitable for the application of DNN models that require a large amount of training data . In 2019Graph neural networks (GNNs) have achieved great results and are being progressively employed in a node classification task. It offers a strategy to acquire novel representations of nodes by combining the features of its local neighborhood and connectivity. Recently, some GNN-based approaches have been proposed to forecast the molecular subtyping of cancer. Rhee et al. created a graph convolutional network (GCN)-based model to investigate the gene–gene alliance and information transmission for cancer subtyping . Lee et Single-cell sequencing allows for the study of the molecular changes occurring in individual cells within the tumor mass. Nonetheless, attributing a specific cellular annotation (in terms of cell type or metabolic state) is challenging, in particular to distinguish cancer cells in single-cell or spatial sequencing experiments. The information provided by high-throughput single-cell sequencing provides not only the description of distinct cellular annotations but also the functional annotation of single cells, for example the estimation of the differentiation potential, vulnerability to metabolic changes, and a prediction of cellular crosstalk . HoweverIn addition to the genome data, the transcriptome, proteome, and metabolome data offer snapshots of a cell’s phenotype space. As shown by PCAWG58 and TCGA59, which also provide transcriptome data in addition to genome data, the transcriptome, particularly RNA sequencing (RNA-seq), is the most frequently generated omics data among these. To perform more complex transcriptomic analyses, bulk RNA-seq has evolved into single-cell RNA-seq (scRNA-seq) and spatial RNA-seq. To enable a greater understanding of cell phenotypes, massive amounts of proteome and metabolome data are being generated for various human cells ,49. The Several studies have combined NGS data with ML to propose a novel data-driven methodology in systems biology . SeveralA common aim of NGS data analysis in cancer research is the identification of potential biomarkers that are predictive of specific cancer types or subtypes. A variety of bioinformatics tools and ML models, for example, aim to identify a molecular signature that is significantly altered in cancer cells on a genomic, transcriptomic, or epigenomic level. Statistical and ML methods are typically used to identify the best set of biomarkers, such as single nucleotide polymorphisms (SNPs), mutations, or differentially expressed genes that are important in cancer progression. Previously, those markers had to be discovered or validated using time-consuming in vitro analysis. As a result, systems biology provides in silico solutions to validate such findings by utilizing biological pathways or gene ontology data .Detecting and predicting the course of the disease are key components to controlling tumor enlargement and providing adequate treatment to cancer patients. With the understanding that cancer can affect individuals differently, AI has been utilized to isolate subgroups within the patient population based on prognosis and survival data. Aside from segmentation, AI has pinpointed biomarkers that can indicate the recurrence of the disease. AI has been implemented to prognosticate high-risk neuroblastoma patients. Utilizing combined gene expression and copy number variations, an unsupervised learning algorithm called auto encoder determined significant features, which were then used for division into two clusters . In a seDL-based neural networks have also been applied to breast cancer survival prognosis. To prevent overfitting effects due to the vast size of omics data, the SALMON survival analysis algorithm operates on eigengene matrices of co-expression network modules. To enhance robustness, it brings together traditional cancer biomarkers and multi-omics information and pinpoints key feature genes and cytobands . The useA subset of alternative network approaches to identifying cancer targets are provided by network-based biology analysis algorithms. More importantly, because different algorithms can look at network data from different angles, they can compensate for each other to provide accurate biological explanations .Interactome data can be organized and represented in the form of network structures to explain the molecular mechanisms underlying carcinogenesis, where the nodes are biological entities  while the edges represent the associations–interactions between them  ,68. AI aOne of the fundamental needs of precision oncology is anticipating therapy response for a patient population. The advantages of ML strategies have been tried for treatment response displaying and expectation following both center-based and component choice-based strategies . The proThe Support Vector Machine (SVM) and the Leave-One-Out Cross-Validation (LOOCV) models have been employed to detect significant changes in RNA and miRNA transcriptomics data between from pancreatic ductal adenocarcinoma specimens and normal tissues. These features (selected RNAs and miRNAs) in combination with miRNA target expression data were further exploited to identify efficient diagnostic markers that were validated in other distinct datasets and biologically interpreted by pathway analysis of the corresponding target genes . MoreoveRecently, a comprehensive examination of nine cancers has demonstrated that proteomics data combined with gene expression, miRNAs expression and genomics is more effective in predicting the responsiveness of drugs and molecules specifically designed to target them. This research was conducted across 58 cell lines over nine cancers with Bayesian Efficient Multiple Kernel Learning (BEMKL) models . This coThe DELFI technology, which uses a blood test to indirectly evaluate the packing of DNA in the nucleus of a cell by assessing the bulk and amount of cell-free DNA present in the flow from various regions of the genome, is one example of AI in clinical practice. Cancer cells release DNA into the bloodstream when they die. DELFI uses ML to investigate millions of cell-free DNA pieces for unusual design in order to distinguish the occurrence of cancer. The strategy provides a perspective on cell-free DNA known as the “fragmentome” and only requires low-coverage genome sequencing, allowing the technology to be economically affordable in a screening setting .The DELFI methodology finds that patients who were later diagnosed positive for cancer had a wide fluctuation in their fragmentome profiles, while those who had a negative cancer diagnosis had predictable fragmentome profiles. Overall, the technique was able to distinguish more than 90 percent of patients with lung cancer (including those with early stages) and displaying different subtypes .Another study focused on glioblastoma, whose diagnosis is based on resection or biopsy which can be especially arduous and perilous in the case that the tumor mass is located in a deep position. Moreover, tracking cancer progression also necessitates repeated biopsies that are often impracticable. Consequently, there is an urgent requirement to identify biomarkers to diagnose and follow-up glioblastoma evolution by limiting the invasive approaches. Recently, an innovative cancer detection method has been developed based on plasma denaturation profiles obtained by a novel use of differential scanning fluorimetry. By comparing the denaturation profiles of blood samples collected from glioma patients and from healthy subjects, the researchers demonstrated that ML-based algorithms can automatically distinguish the cancer patients from the healthy individuals (with a precision around 92%). Additionally, this high-throughput workflow can be applied to any type of cancer and may represent a potent pan-cancer diagnostic and monitoring tool that requires only a plain blood test .Among the limitations of the current approaches, tissue biopsy presents a fixed overview of the tumor that fails to record the intratumor distinguishment and dynamic changes occurring during carcinogenesis, also determined by clonal pressure caused by the applied medication . On top Another AI approach is the PinPoint test, a cost-effective AI-driven blood test for cancer that is meant to upgrade rapid cancer referral paths. The test is found on an algorithm that uses ML to investigate regular constituents, as well as the patient’s age and sex. It can calibrate and combine these individual variables into one solid and highly precise result, such as the likelihood that a patient has cancer . The PinIn the field of cancer imaging, AI displays a great utility in three main clinical tasks: tumor detection, characterization, and monitoring . The locCharacterization in general includes tumor segmentation, diagnosis, and staging. It can also include a disease-specific prognosis as well as outcome prediction based on specific treatment modalities. Segmentation determines the extent of abnormalities and can range from simple 2D measurements of the maximum in plane tumor diameter to more involved volumetric segmentations that assess the entire tumor as well as any surrounding tissues. This information could be exploited for future diagnostic purposes as well as for calculating the appropriate dose administration during radiation planning. AI has the capability to significantly improve the efficiency, reproducibility, and reliability of tumor measurements through automated segmentation. In computer-aided diagnosis (CADx) systems, systematic processing of quantitative tumor features is used, allowing for more reproducible descriptors. In the case of inconsistencies in interpretation by different human readers, CADx systems have been used to diagnose lung nodules in thin-section CT and prostate lesions in multiparametric MRI .Staging is another aspect of tumor characterization in which tumors are classified into predefined groups based on the size and spread of the tumor mass, thus providing information regarding the expected clinical course and for the decision of the most appropriate treatment strategies . The appDL systems have been used to simultaneously detect and classify prostate lesions. For training convolutional neural networks (CNNs) for prostate cancer diagnosis by MRI, both de novo training  and the The accuracy and consistency of AI systems are frequently restricted by their training data and the hardware used. We must keep in mind that AI can make mistakes in some situations because its decision-making ability is predictive and probabilistic. As a result, there are no clear regulations or guidelines in place to determine who is legally liable when AI malfunctions occur or causes issues while providing a service. Another factor to take in consideration is that most of the places where the potential of AI in healthcare has been evaluated are basically high-income and resource-driven areas. When used in low-income countries with a shortage of well-trained physicians and oncological specialists, AI-based prediction tools are expected to have a greater impact and increment the success of cancer treatment.The improvement in the AI interpretation is a crucial step toward mitigating this risk and providing a decision-making rationale. One limitation is represented by the lack of a human verification step in the process unless a physician supervises the AI system. As a result, no one expects AI to entirely replace medical professionals. AI-based precision medicine will be critical for cancer treatment in the future. Living databases will exploit extremely complex models capable of making a personalized therapy selection, estimation of the drug dose, follow-up schedule, and so on. However, the transition from artificial narrow intelligence to artificial general intelligence will result in the automation of all the steps involved in cancer prediction, diagnosis, and treatment.Despite its numerous benefits, AI presents several challenges and constraints that hinder it from fully functioning in cancer research. Particularly, three layers of complexity must be considered: (i) cancer is a highly heterogeneous organoid-like structure that, at the time of diagnosis, is made up of many different cancer subclones embedded in a stroma (the tumor microenvironment) that itself contributes to cancer progression; (ii) as cancer progresses, tumor evolution leads to increased intratumor heterogeneity so that by the time therapy is started, the targeted cancer may not respond; (iii) cancers with the same molecular and histological signatures behave differently in each single patient because of individual epigenetic and immunological modulations ,98,99. TThe new era of innovation brings with it many challenges that should be overcome to drastically improve oncology procedures at several levels. The lack of inclusive and different datasets for training represents a significant obstacle to the widespread adoption of AI algorithms and decision-support systems in cancer care. Most of the powerful AI models require a large sample size to efficiently train the tool. Although there are dimensionality reduction and feature selection methods for addressing these aspects, proper implementation is critical for achieving better and reliable results. The number and type of data annotated influences the constructions of algorithms, and an imbalance in data from patients differing for gender, age, race, nutritional state, lifestyle, and environment will affect AI and ML training. Thus, the lack of sensible data may increase the risk of missed diagnosis. Therefore, experts are fundamental in data curation and data annotation to provide reliable datasets to be used for training AI classifier and predictors models.In medical data sets, particularly in the case of cancer data, classes are typically distributed unequally. The continuous use of AI- and ML-based tools for diagnosis and treatment decisions can be risky due to distributional shifts, which means that target data may not match the ongoing patient data employed to train the model, resulting in incorrect outputs. Predictions made by AI at the time of diagnosis likely changes during the course of the therapy and the evolution of the disease along with changes in patient’s habit .Changes in technology, healthcare, and population, such as the gene pool, are likely to have an impact on the relationship between the data items. The actual application of AI models in clinics is not being actively considered. The predictions achieved with these models frequently require validation in the clinical practice to assist medical experts in confirming diagnosis decisions.Significant issues regarding data availability and interpretability caused by AI’s “black box” process, in parallel with the emergence of an inherent bias toward limited cohorts that reduces the reproducibility of AI models and perpetuates disparities in the healthcare, collectively prevent the widespread application of AI in clinics. Additionally, the distribution of AI-based technologies in many developing countries may be hampered by a lack of knowledge in computing algorithms and technologies of the physicians.Taken together, the clinically relevant achievements discussed in the present review need to become more solid to be translated into the right treatment for the right patient. Hence, the rapidly ongoing evolution of AI-based medical data analysis will significantly improve the treatments in cancer.In this paper, we present an overview of the models applied in diagnosing and identifying therapeutic targets, and we discussed the challenges and future perspectives of AI in cancer research . As the


The current study aimed to construct and validate a magnetic resonance imaging (MRI)-based radiomics nomogram to predict tumor protein p53 gene status in rectal cancer patients using machine learning.Clinical and imaging data from 300 rectal cancer patients who underwent radical resections were included in this study, and a total of 166 patients with p53 mutations according to pathology reports were included in these patients. These patients were allocated to the training (n = 210) or validation (n = 90) cohorts (7:3 ratio) according to the examination time. Using the training data set, the radiomic features of primary tumor lesions from T2-weighted images (T2WI) of each patient were analyzed by dimensionality reduction. Multivariate logistic regression was used to screen predictive features, which were combined with a radiomics model to construct a nomogram to predict p53 gene status. The accuracy and reliability of the nomograms were assessed in both training and validation data sets using receiver operating characteristic (ROC) curves.Using the radiomics model with the training and validation cohorts, the diagnostic efficacies were 0.828 and 0.795, the sensitivities were 0.825 and 0.891, and the specificities were 0.722 and 0.659, respectively. Using the nomogram with the training and validation data sets, the diagnostic efficacies were 0.86 and 0.847, the sensitivities were 0.758 and 0.869, and the specificities were 0.833 and 0.75, respectively.The radiomics nomogram based on machine learning was able to predict p53 gene status and facilitate preoperative molecular-based pathological diagnoses.The online version contains supplementary material available at 10.1186/s40644-023-00607-1. Rectal cancer is one of the most common digestive tract malignancies , with apWild-type p53 is an important tumor suppressor gene . HoweverCurrently, magnetic resonance imaging (MRI) has the advantage of high soft tissue resolution, and it has become the first choice for early overall non-invasive evaluation for preoperative staging of rectal cancer , 10. ImpRadiomics is a relatively new field that uses emerging technologies to extract features from medical imaging, quantifying its phenotypic characteristics in a high-throughput manner . Such feTherefore, this study aimed to use machine learning to build a radiomics signature that could predict p53 gene status in rectal cancer. The radiomics signature could then be combined with clinical features to build a visualized nomogram to evaluate p53 gene status.This retrospective study was approved by the Ethics Committee of Zhejiang Provincial People’s Hospital and informed consent was obtained. A total of 1056 rectal cancer patients in the picture archiving and communication system with a definitive clinical diagnosis between February 2018 and May 2022 were identified. Next, the clinical and radiology data from 300 patients were selected for retrospective analysis according to the inclusion and exclusion criteria. The inclusion criteria were: (1) diagnosis of rectal cancer by pathological examination; (2) lack of immunotherapy or neoadjuvant chemoradiotherapy; and (3) surgery was performed within one month after examination by MRI. The exclusion criteria were: (1) poor MRI quality or the lesions were poorly displayed; (2) lack of pathological results upon resection. All patients were allocated at a 7:3 ratio to either the training (n = 210) or validation (n = 90) cohort according to when their examination was completed . The training data set was used to build the radiomics model, and the validation data set was used to verify the performance of the model.Paraffin-embedded tissue sections were deparaffinized in a series of gradient ethanol baths, rehydrated, and immersed in methanol containing 0.3% hydrogen peroxide for 10 min to block endogenous peroxidase at room temperature. Subsequently, the tissue-slides were heated for 30 min in a pH 6.0 antigen retrieval solution to induce antigen retrieval and then incubated overnight with an anti-P53 antibody at 4 °C. Staining was performed using a Prolink-2 Plus HRP rabbit polymer detection kit  according to the manufacturer’s instructions. Images were captured using Aperio ScanScope CS software .The results were evaluated based on the intensity and extent of staining by two independent pathologists (double blinded) as described previously. Briefly, the P53 positive staining area was scored as follows: the staining percentage of positively stained area over total tissue area was defined as 0 = ≤ 5%; 1 = 5–25%; 2 = 26–50%; 3 = 51–75%; and 4 ≥ 75%. The intensity was graded as follows: 0, negative; 1+, weak (yellow); 2+, moderate (light brown); and 3+, strong (dark brown). A final score between 0 and 12 was achieved by multiplication of the staining area and intensity scores. A staining index was used in which a score of 0–2 was considered negative expression, 3–6 was considered low expression, and ≥ 6 was considered high expression.The clinical and histopathological data of all patients were reviewed and recorded from the patients’ electronic medical records, and included sex, age, carcinoembryonic antigen (CEA) levels , and pathological p53 gene status. The clinical imaging features were obtained by consulting the structured MRI rectal examination report, and included tumor node metastasis (TNM) staging, distance from the end of the convex edge of the tumor to the edge of the anus (DIS), circumferential resection margin (CRM) status, and presence of EMVI. A positive CRM status was defined as tumor, metastases, metastatic lymph nodes, or intramuscular vascular invasion within 1 mm of the mesenteric fascia. EMVI was defined as (a) presence of tumor signal intensity within a vascular structure, (b) expanded vessels, (c) tumoral expansion through and beyond the vessel wall, and (d) disrupting the vessel border.All data were assessed independently by two experienced radiologists in a double-blind setting. Radiologist A had three years of experience and Radiologist B had ten years of experience in abdominal imaging. Consensus was reached in all cases of disagreement after all assessments were completed.www.itksnap.org) to segment the entire rectal tumor layer by layer to obtain the volume of interest (VOI) [All patients underwent rectal MRI in a supine position using 3.0T MRI systems  equipped with an eight-channel phased-array coil. In this study, we only used T2-weighted images (T2WI) to extract radiomics features. Not only do T2WI provide clear images of tumor structures that can easily delineate regions of interest, but their high spatial resolution also reduces the impact of image quality on the extracted radiomics features. Export T2WI data for each patient from the Picture Archiving and Communication System (PACS), and then import it into non-commercial Artificial Intelligence Kit software  for preprocessing of T2WI. Image preprocessing was performed by resampling the images with a resolution of 1 × 1 × 1 mm3 using the linear interpolation method. The image gray level was discretized and normalized to order 32. Next, the preprocessed T2WI were imported into ITK-SNAP software (st (VOI) . Blood vA total of 930 features were extracted from the T2WI of each patient, and specific feature information can be found in Supplementary Materials Table Optimal features among the robust features were obtained by dimensionality reduction based on the training set. Firstly, analysis of variance was used to calculate the variance of each feature. The variance value is calculated as the average of the squared differences between each variable’s value and the mean. It is the most important method for measuring the dispersion of numerical data. The larger the variance, the greater the fluctuation of the data, and vice versa. So, it is necessary to eliminate features with a variance of 0 or less as a priority. In this study, we calculated the variance of each feature and retained the features that were greater than the threshold of 1. Secondly, minimum redundancy maximum relevance (mRMR) was used to extract optimal features. The purpose of mRMR was to select features that have the greatest correlation with the pathological p53 gene status. The radiomics features with an inter class CCs > 0.8 were retained. Subsequently, to ensure minimal redundancy, intra-class correlation analysis was performed on the remaining features, retaining features with an intra class CCs ≤ 0.1. Finally, the gradient boosting decision tree (GBDT) algorithm was used to reduce the dimension of the remaining features.− 2, 2− 1, 1, 21, 22} and the kernel spread parameter was varied with values in {10− 2, 10− 1, 1, 101, 102}. The machine learning scores were concatenated from all testing data folds to assess the signature model performance over the entire data set. The signature model quantified the discriminability as the possibility of a p53 mutation for each patient defined as Rad-score. ROC curves were used to evaluate the performance of the signature model, and the validation data set was used for verification. Next, independent predictors from clinical features and the rad-score were determined using multiple factor logistic regression with a backward stepwise selection method. This method ensured that only significant variables were included in the regression equation. The process continued until no significant explanatory variables were added to the equation and less significant variables were no longer eliminated. Akaike information criterion was used to evaluate the models. A integrated model was then established and a visual nomogram was constructed. Finally, the difference between the integrated model and the signature model was evaluated using the DeLong test, and the nomogram goodness-of-fit was assessed using the Hosmer-Lemeshow test. The radiomics workflow was shown in Fig. A support-vector machine (SVM) was used to build the signature model based on the final screened features. Ten-fold cross-validation involved stratifying and dividing the training data set into ten folds of equal size: eight folds (80%) for training, one fold 10%) for tuning model parameters, and one fold (10%) for testing. Stratification was used to ensure a similar distribution of events across the ten folds. This process was repeated for ten iterations, always using a different data fold for signature model training, tuning, and testing. The data from the training subgroup and the data from the tuning model parameters subgroup were used to construct a model, resulting in 100 models. Finally, the data from the test subgroup was selected to test the performance of the model. The SVM parameters of the model with the highest accuracy were then chosen as the final tuning parameters. In this study, SVM with a Gaussian kernel function were implemented. The cost parameter C was varied with values{20% for tut-test or Mann–Whitney U test, and categorical variables were compared using a chi-square test. All statistics were two-sided, and statistical significance was set at P < 0.05.Statistical analyses were performed using MedCalc software (version 11.2), Python (version 3.5), R software (version 3.4.1) and SPSS (version 24.0). Continuous variables were compared by independent sample P > 0.05), as shown in Table P > 0.05), as shown in Table There was no statistical difference in demographic characteristics or conventional radiology characteristics between the training and validation sets . With further screening, 874 features were obtained based on the variance results. After mRMR dimensionality reduction, 24 features were identified, and finally nine features remained after GBDT dimensionality reduction  from an ideal fitting.The area under the curve (AUC) values of receiver operating characteristic curve (ROC) showed that the diagnostic efficacies of the radiomics signature model in the training and validation data sets were 0.828 and 0.795, the sensitivities were 0.825 and 0.891, and the specificities were 0.722 and 0.659, respectively. The calibration curves reveal that the predicted p53 gene status of the signature model in both the training and verification groups was in good consistency with the actual p53 gene status, as shown in Fig. The results of this study showed that a radiomics signature based on features extracted from T2WI combined with machine learning can be used to predict p53 gene status in patients with rectal cancer. In addition, a visual nomograph was created to ensure that the results of the prediction model are easy to comprehend and convenient for clinicians to evaluate the p53 gene status of rectal cancer patients. As such, this study provides a new tool for pathological molecular diagnoses.p53 is an important tumor suppressor gene, and its mutation status may be an important factor for early diagnosis and treatment of rectal cancer . HoweverPreviously, both computed tomography (CT)- and MRI-based radiomics have been used to predict p53 status in different cancers. Iwatate et al. established a CT-based radiologic model to predict p53 status in pancreatic cancer , while SThe results of the current study also showed that tumor size, lymph node metastasis, sex, and the radiomics signature were independent predictive factors of the p53 mutation status. Xu et al. determined that tumor size is the primary radiomic prognostic and predictive feature of microvascular invasion in hepatocellular carcinoma and is independently associated with disease recurrence and mortality . Zhang eIn this study, SVM was used to construct the prediction signature. The kernel function in SVM maps the input parameters to different feature spaces, which can divide the transformed data to achieve more accurate results, improve the robustness of the model, and avoid over fitting in the training process. Especially for research with limited cases, SVM is considered the best choice to balance the variance and bias of input data –30. WangThere are some limitations in the current study. Firstly, this was a retrospective study and patients were not randomized, which may result in selection bias. Secondly, p53 gene status was confirmed using immunohistochemistry rather than genetic sequencing. However, as a conceptual study, this did not affect the final research outcome. Finally, the study data was collected from a single center, and further multi-center studies are needed to validate the results.In conclusion, radiomic features can be used to identify the p53 gene status in cancer, and the nomogram visualization based on this model can serve as a conceptual design for screening high-risk populations with P53 gene status in clinical practice. This may facilitate the pathological molecular diagnosis and risk stratification of locally advanced rectal cancer. In the future, we hope to further validate the model through multidimensional verification in order to expand the detection methods of P53 gene status.Below is the link to the electronic supplementary material.Supplementary Material 1