R-loops are three-stranded nucleic acid structures formed from the hybridization of RNA and DNA. While the pathological consequences of R-loops have been well-studied to date, the locations, classes, and dynamics of physiological R-loops remain poorly understood. R-loop mapping studies provide insight into R-loop dynamics, but their findings are challenging to generalize. This is due to the narrow biological scope of individual studies, the limitations of each mapping modality, and, in some cases, poor data quality. In this study, we reprocessed 810 R-loop mapping datasets from a wide array of biological conditions and mapping modalities. From this data resource, we developed an accurate R-loop data quality control method, and we reveal the extent of poor-quality data within previously published studies. We then identified a set of high-confidence R-loop mapping samples and used them to define consensus R-loop sites called ‘R-loop regions’ (RL regions). In the process, we identified a stark divergence between RL regions detected by S9.6 and dRNH-based mapping methods, particularly with respect to R-loop size, location, and colocalization with RNA binding factors. Taken together, this work provides a much-needed method to assess R-loop data quality and offers novel context regarding the differences between dRNH- and S9.6-based R-loop mapping approaches. R-loops are three-stranded nucleic acid structures formed from the hybridization of RNA and DNA, often in regions with high G (guanine) or C (cytosine) skew, such as CpG islands . The stuChen et al. introduced ‘R-ChIP’, an R-loop mapping modality which relies on catalytically inactive RNase H1 (dRNH) as an alternative to S9.6-based methods   sequencing was the first such technique, utilizing the S9.6 antibody to detect R-loops  , but no and II’) .Chedin et al., who demonstrated that issues of data quality persist even within prominent R-loop mapping studies . From meta-analysis of RL regions, we observed stark contrasts between S9.6 and dRNH-based R-loop mapping results, supporting the concept of different subclasses of R-loops such as proposed with the ‘Class I/II’ hypothesis. We then explored R-loops mapped uniquely by dRNH, finding their enrichment at transcriptionally-silent, bivalent enhancer clusters in pluripotent stem cells. Finally, we characterized R-loop regions based on their level of conservation between independent samples, finding that they differ with respect to pathways relevant for transcription and replication. Moreover, we find that more constitutive RL regions are associated with housekeeping genes.Taken together, our findings support a model in which dRNH-based mapping approaches are unable to measure most of the R-loop-covered genome, likely owing to the presence of RNA binding factors in these same regions. On the other hand, dRNH-based mapping approaches also have greater sensitivity to detect small, native R-loops in promoters and enhancers compared to S9.6 modalities. Though our model necessitates future wet-lab validation, it should prove useful for designing future R-loop mapping experiments.In this study, we also establish novel approaches for the quality control and analysis of R-loop mapping data, which we provide in the collection of software tools, ‘RLSuite,’ which accompany this work (see Availability). While these software packages are not described in this work, we have made them publicly accessible, and we plan to provide a full description in a future manuscript.QmRLFS-finder method (makeRLFSBeds.R script in the accompanying data generation repository (see Availability). Briefly, we downloaded every genome for which gene annotations were available in ‘FASTA’ format from the University of California Santa Cruz (UCSC) genome repository. Then we ran the QmRLFS-finder.py script to generate predictions. Finally, we converted the results table to ‘BED’ format.We predicted R-loop forming sequences using the r method  implemenR-loop data was found by querying the Gene Expression Omnibus (GEO), Sequence Read Archive (SRA), ArrayExpress, and PubMed with the keywords ‘R-loop’, ‘R-loops’, ‘RNA:DNA hybrid’, ‘RNA:DNA hybrids’, ‘RNA-DNA hybrid’, ‘RNA-DNA hybrids’, ‘DRIP-Seq’, ‘R-ChIP’, ‘S9.6’ and ‘D210N’. We included all query results with R-loop mapping data publicly accessible via SRA. For every R-loop mapping sample, we recorded the following details:The sample accession (GEO or SRA).The accession for the corresponding genomic input sample (if available).The sample condition .The mapping mode ).The tissue of the sample. For cell lines, this was the cell line name . For organs and whole organisms, the tissue name ).The sample genotype, where relevant .‘Other,’ a catchall for other relevant metadata provided by the original data submitters, such as drug treatment information.The source publication PMID, if available.The last author on the source publication, if available.RLPipes v0.9.0 software program to query the SRA database for missing metadata related to each sample (see Availability).We standardized the manually curated catalogue via the custom scripts in the accompanying data generation repository (see Availability). First, the ‘condition’ was binarized to one of the following labels: ‘POS’  or ‘NEG’ . The mapping of ‘condition’ to ‘label’ relied on a manually curated regular expressions (REGEX) dictionary and simple string matching . AdditioRLPipes v0.9.0, a CLI tool based on Snakemake . For all upstream processing steps, we used nakemake   using reformat.sh from bbtools  peaks in ‘broadPeak’ format, (B) coverage in ‘bigWig’ format, (C) quality reports for alignments and raw reads, and (D) alignments in ‘BAM’ format (see Availability).edgeR R package  method previously demonstrated for similar sequencing approaches  implemen package . Then, wd method . We theneepTools  with argFollowing data standardization, a quality control model was developed to filter out poor-quality samples prior to meta-analysis of R-loop consensus sites. This involved (1) R-loop forming sequences (RLFS) analysis, (2) quality model building and (3) sample classification.rlfsAnalyze.R, available in the accompanying data generation repository (see Availability). The analysis script calls the analyzeRLFS function from the RLSeq R package (see Availability). This function implements a method for assessing the enrichment of R-loop mapping peaks within RLFS via the following procedure: Permutation testing is implemented via the permTest function from the regioneR . For visualization purposes, the scaled Z score distributions were calculated for each peak set in the reprocessed data and visualized using locally estimated scatterplot smoothing (LOESS) regression with the standard error as the confidence interval.R-loop forming sequences (RLFS) analysis was performed using a custom R script, regioneR  R packagregioneR  within 5models.R script in the accompanying data generation repository (see Availability). The process of model building is semi-supervised and utilizes a graphical user interface to allow operators with minimal coding experience to conveniently use it. The model building procedure involves the following steps: (A) The operator executes the models.R script from the command line, launching an interactive web interface that contains a table with SRA experiment ID and label (‘POS’ or ‘NEG’) for each sample along with a plot showing the Z-score distribution from the RLFS analysis of that sample. This interface also indicates samples that have been previously deliberated about. (B) The operator then decides which samples to exclude from the model building process. A sample should only be excluded when the Z-score distribution drastically differs from the expected Z-score distribution for its label (examples are provided to the operator in the web interface). (C) Once all samples with a label mismatch are selected, the operator clicks the ‘Build model’ button which executes the model building process on the non-excluded data via the RMarkdown notebook, FFT-classifier.Rmd (available from the accompanying data generation repository). This notebook automatically executes the following steps: (C.1) The data are wrangled. (C.2) Then, the engineered features are calculated from the Z-score distributions K-nearest neighboursSupport vector machine 10-fold cross-validation repeated 5 times was implemented during training.confusionMatrix function from the caret R package ((C.7) Finally, the model was evaluated on the test set with the  package .classifySamples.R script from the accompanying data generation repository (see Availability). This script primarily uses the predictCondition function from the RLSeq R package (see Availability). The procedure implements the following steps for each peak set in the reprocessed R-loop mapping data: (i) calculates the Fourier transform of the Z-score distribution, (ii) reduces the dimensions to the engineered feature set , (iii) applies the pre-processing model to normalize these features  and (iv) implements the classifier to render a preliminary quality prediction . Of note, the quality model alone does not provide the final prediction for each sample. Instead, the final prediction is ‘POS’ (positive) only if all the following are true:Following model building, all samples were subsequently classified using the P value is significant (P < 0.05)The RLFS Permutation test Z-score distribution at 0 bp is greater than 0 The Z-score distribution at 0 bp is greater than the value at both −5000 bp (start) and +5000 bp (end) The The quality model predicts a preliminary label of ‘POS.’Z score) also met baseline criteria which indicate enrichment in typical permutation testing .To examine the internal and external validity of the predicted labels, several approaches were implemented: (A) RLFS consensus analysis, (B) genic feature enrichment analysis and (C) sample correlation analysis. All four (prediction:label) conditions: POS:POS, POS:NEG, NEG:POS, NEG:NEG were compared in each approach.This analysis involved four DRIP-Seq samples, ‘TCELL (Input)’ (SRX2455193), ‘TCELL’ (SRX2455189), ‘786-O (RNH)’ (ERX3974965), and ‘786-O’ (ERX3974964), in which both the ‘TCELL’ samples were from the same study (SRP095885), and the ‘786-O’ samples were also from the same study (ERP120322). Both ‘TCELL’ samples and both ‘786-O’ samples respectively were from the same study.TxDb.Hsapiens.UCSC.hg38.knownGene R package  sequencing study (GEO accession: GSE63874) (RLSeq R package (i) peaks were randomly down sampled to a maximum of 10 000 peaks, (ii) down sampled peaks were overlapped with genic features, (iii) then, intersect statistics were calculated using Fisher's exact test via the valr R package  . All sam package , finally package .Chedin et al. (gsgCorr.R script in the accompanying data generation repository (see Availability). First, high-confidence R-loop sites were identified from ultra-long-read R-loop sequencing (SMRF-Seq)  files for each sample were quantified around high-confidence sites  via theMRF-Seq) . Then, h package .prepareConsensus.R script from the accompanying data generation repository (see Availability). The following criteria were used to select samples for R-loop consensus analysis: (i) label of ‘POS’, (ii) prediction of ‘POS’, (iii) at least 5000 peaks called under a P adjusted value of 0.05, (iv) is a human sample. We chose to enforce a label of ‘POS’ as ‘NEG’-labelled are subjected to treatments which should prevent robust R-loop mapping and may, therefore, introduce unwanted technical variance to the consensus analysis. We chose samples with at least 5000 peaks due to the need for random peak down sampling to 5000 ranges.This analysis identified human R-loop consensus sites within catalytically dead RNase H1 (dRNH) and S9.6-based samples. To identify general R-loop consensus regions (RL regions), the union of dRNH and S9.6 consensus sites was calculated. To obtain these data, the following procedure was implemented: First, samples were prepared for R-loop consensus analysis using the rlregions.smk workflow (built on snakemake) from the accompanying data generation repository (see Availability).Two hundred and forty-seven samples met the full criteria for inclusion in the analysis of the 375 ‘POS’-labelled human samples. For each included sample, peaks were randomly shuffled and down sampled to 5000 ranges to ensure high peak-count samples would not dominate the analysis. Finally, samples were partitioned based on whether they used dRNH or S9.6-based R-loop mapping. Then, each down-sampled peak set was processed further using the bedtools  the peak sets were intersected with 10 bp genomic windows and the number of peak overlaps per window was counted (using bedtools ). (ii) Tbedtools . (iii) mpeakcall  was thenpeakUnion.R script from the accompanying data generation repository (see Availability). This script found the union of the peaks in the dRNH and S9.6 narrowPeak files, along with calculating the percent of samples within each group in which each peak appeared (conservation score) and averaging these scores in cases where dRNH and S9.6 peaks overlapped to create a peak union. Finally, RL regions and associated metadata were finalized using the finalizeRLRegions.R script from the accompanying data generation repository (see Availability). R-loops were filtered to remove any regions over 50kb in size, then every hg38 ‘POS’-labelled peak set was overlapped with the regions via the bed_intersect function from the valr R package (RLHub R package reference (see Availability). Moreover, RL region ‘confidence scores’ were calculated for the purposes of colouring the RL region peaks when visualized in the genome browser. These scores are the geometric mean of scaled log2pct_case (percent of samples detecting the RL region which are predicted ‘POS’), nStudies (number of independent studies in which the RL region is detected), log2medQVal , and the log2medSignalVal .Then, consensus RL Regions were generated using the  package  to yieldcomputeMatrix function from the deepTools software program .The following analyses were performed automatically by running the macs3 peak calling. Summited peaks were then overlapped with genomic annotations for downstream analysis via the bed_intersect function from the valr R package.For analyses which involved calculating the overlap  of dRNH and S9.6 consensus peaks with genomic features, peaks were ‘summited’ (length-standardized) prior to overlap to mitigate size biases. Standardization involved finding the position of highest signal within the peaks (peak summit) and extending 250 bp bidirectionally to form 500 bp ranges. Peak summits were provided automatically by the output of TxDb.Hsapiens.UCSC.hg38.knownGene R package  were obtained from the  package . Peak ovbed_fisher function from the valr R package (binOverFeature function from the ChIPpeakAnno R/Bioconductor package (annotatePeak function from the ChIPseeker R/Bioconductor package (Enhancer annotations were developed from three sources: predicted cis-regulatory elements (CREs) provided by ENCODE SCREEN , high-co package . Enrichm package . Distal  package .enrichr function from the enrichR R package  sequencing datasets were obtained from the Gene Expression Omnibus (GEO) in induced pluripotent stem cell (iPSC) (GSE117086) and CUTTL1 (GSE115894) cell lines. These were analysed jointly with the MapR  datasets curated previously via the steps described beforehand (see R-loop mapping data standardization). Samples were selected which matched the same confidence filters applied in the generation of consensus peaks . Gene set over-representation analysis was implemented to identify the pathways relevant to R-loop bound distal enhancers in each cell line. Briefly, the distal enhancers overlapping intergenic MapR peaks were identified. Then, the GeneHancer interaction database was queried to obtain high-confidence gene targets for each enhancer. Gene set over-representation proceeded using the  package ,47 with  package , the MSi package , the ChE package  and the  package .binOverFeature function from the ChIPpeakAnno R/Bioconductor package. CTCF, RAD21 and SMC3 consensus peaks were obtained from the ENCODE3 transcription factor binding site collection, provided by the UCSC table browser. SA1 and SA2 peaks were curated as previously described  cells and from GEO for CUTTL1 (GSE130140) cells. We also obtained HiC sequencing data from GEO for iPSC cells (GSE125540) (CUTLL1 not available). HiC-sequencing data was added to the UCSC genome browser and compared with MapR bigWig tracks. Enrichment analysis of CTCF peaks within MapR peaks was calculated via the escribed , and the utility . R-loop findOverlapsOfPeaks function from the ChIPpeakAnno R/Bioconductor package  enhancers (GeneHancer enhancer annotations were extended to include the labels from the chromHMM 18-state model predictions in iPSC cells (described in preceding section). Briefly, chromHMM enhancer states were overlapped with distal GeneHancer enhancers using the  package . Overlapnhancers .bed_fisher function from the valr R package . sample alignment files (‘BAM’ format) were processed with featureCounts from the Rsubread R package to quantify the read counts from each within RL regions (R-loop region (RL region) abundance was calculated within each human sample using the  regions . Finally regions .DESeq function from the DESeq2 R package  the count matrix was subset to contain only the RL regions found by both dRNH and S9.6 peaks, (2) the  package  was appl package  (and R i service ) was use service  terms wiprcomp function in R was used to calculate principal component analysis plots and the EnhancedVolcano R package  was ranked and bins were selected to partition RL regions by conservation percentiles. Binned RL regions were overlapped with genomic features and enrichment was calculated via the procedure described above (see Consensus site overlap in genes). Pathway enrichment was also performed via the procedure outline in the above section , with the only difference being that the ChEA , KEGG 6, and MSiThe data in this study were hand-curated from public data repositories, yielding 810 R-loop mapping samples from 74 different studies . The datFollowing manual curation of R-loop mapping datasets, we developed a purpose-built R-loop pipeline tool called ‘RLPipes’ and applied it to reprocess all datasets, yielding peaks, read coverage and other processed files (see Availability) Figure .Previous work has demonstrated discrepancies in the number of R-loop sites uncovered by different mapping methods . HoweverRecent studies have identified the presence of poor-quality R-loop mapping datasets within the literature ,17, but Chedin et al. indicated that POS-labelled samples may sometimes fail to map R-loops as expected  due to a mismatch between the sample label and the Z score distribution. Having selected the samples to discard, the operator triggered the automated model building script, which initiated the training process.We built a classifier using a semi-supervised approach that involves a human operator deciding whether to exclude samples from the training set based on the alignment between the RLFS Z-score plot and the sample label , D. The P-value [accuracy > NIR]: 3.58e–15)  , demonstP value below 0.05, (ii) show a Z score above 0 at 0 bp in the distribution, (iii) show a Z score at 0 bp which is greater than both the Z score at –5 kb and +5 kb, and (iv) receive a prediction of ‘POS’ from the machine learning model. In this work, ‘quality model’ refers to criteria i–iv. Of note, the purpose of criteria i–iii is to ensure that a ‘POS’ prediction from the machine learning model also meets baseline standards for assessing permutation testing results  have a permutation testing  results .Z score distribution for each group. The results demonstrate that the classifications robustly represent the RLFS analysis of the data . From univariate analysis, we found that most false negatives  occur in samples treated with RNase  or actinomycin D, suggesting incomplete R-loop degradation . Surprisexternal validity of our quality method through other quality approaches that did not involve RLFS analysis. The first approach was to visualize the normalized read coverage tracks at sites of known R-loop formation. A representative depiction of four DRIP-Seq samples at a region near MALAT1 (a non-coding RNA with constitutive R-loop formation (We next proceeded to assess the ormation ) shows tormation . These rormation , as discWe then implemented enrichment testing for genic features, such as exons and introns, across all human samples in each prediction:label group. Given that R-loops are a by-product of normal transcription, successful R-loop mapping should result in an enrichment of peaks within genic features, as has been shown previously ,16,66. FWe then repeated the same analysis with both predicted and experimentally determined G4 quadruplexes (G4s) . G4s areet al demonstrated that G/C skew (i.e. G or C skew) is a common feature of unmethylated CpG islands (CGIs) with R-loop occupancy .Taken together, these internal and external validations demonstrate that the method we present here is an accurate measure of R-loop mapping data quality. They also reveal the extent of poor R-loop data within published datasets, indicating that the approach described here will be of great benefit to future studies. Of note, we provide access to this quality method in the The locations and prevalence of R-loops genome-wide has been a subject of study since the first R-loop mapping modalities were described ,66. HoweTo address these gaps in knowledge, we used high-confidence human R-loop mapping samples to generate dRNH- and S9.6-specific R-loop ‘consensus regions’ Figure -D. We fiet al employed a consensus analysis approach to identify high-confidence ‘R-loop zones’ , which is 4.32% of the human genome . Moreover, we calculated the overlap of these sites and found 9950 shared sites  Figure . This inCALM2, a gene containing one of the most mapped R-loops in our analysis. Interestingly, we observed that while both S9.6 and dRNH peaks localize around the transcription start site (TSS), only signal from S9.6 was found in the gene body and around the transcription termination site (TTS) and Poly-A regions  and ‘Class II’   . They hyngation) . In the ngation) , K. Morengation) . HoweverTo address these questions, we analysed differential R-loop abundance for each high-confidence sample within the 9950 R-loop regions (RL regions) found in both S9.6 and dRNH consensus sites Figure . We applAvailability). We then visualized the top differentially abundant RL region (RL46345)  Figure , E, whicstudy and then by sample until the S9.6 dataset had the same number of studies as dRNH (9 studies) and the same number of samples or less (∼56 samples). From comparing the downsampled results to the full dataset, we found they agreed well (Pearson R: 0.95)  and dRNH . To address this concern, we randomly downsampled the S9.6 dataset to match the dRNH dataset, and then we repeated the differential analysis. Of note, due to previously observed batch effects between R-loop mapping studies , we randR: 0.95) . To valiR: 0.95) . MoreoveWe also wanted to address the unlikely possibility that dRNH- and S9.6-specific R-loops might coincide with genes belonging to specific biological programs. Therefore, we identified the genes overlapping the top 1000 dRNH- and S9.6-specific differentially abundant R-loops and performed GO term analysis. Unexpectedly, we found that, unlike dRNH, S9.6-based mapping methods detected R-loops in genes related to translation Figure . ConversThe Class I/II hypothesis predicts that dRNH maps R-loops which arise from promoter–proximal pausing with high specificity compared to S9.6 Figure  13). To. To13). Taken together, these results further support the soundness of the Class I/II hypothesis as dRNH displays a strong preference for transcriptionally paused genes compared to S9.6. Moreover, they suggest intriguing differences between dRNH-based and S9.6-based R-loop mapping in several gene programs.From the above analysis, we found support for the Class I/II hypothesis by examining the genic distribution of dRNH and S9.6 consensus R-loops Figure -K and thFrom a preliminary analysis of peak characteristics, we noticed that dRNH-only consensus regions are smaller (mean 754 bp) than dRNH-shared (mean 1588 bp) and S9.6 (mean 3293 bp) Figure . From anThe smaller size and lack of distinct gene set enrichment within dRNH-only peaks suggested that they might be coinciding with specific genomic features. From feature analysis, we observed that while dRNH-shared peaks display strong enrichment in TSS and TTS regions, dRNH-only peaks are strongly over-represented in intergenic regions  Figure . This leYan et al demonstrated that MapR (a dRNH-based technique) detects R-loops at predicted enhancers, a type of CRE  peak pileup within distal enhancers in both cell lines Figure . From ovEnhancer RNA (eRNA) is a non-coding RNA species that forms within enhancer regions ,75 and wTo determine whether the R-loop-bound enhancers pertained to cell type-specific gene programs, we used the GeneHancer interaction database to obtain the downstream gene targets of each enhancer. We then used the CellMarker database to test for cell type-specific gene programs and, interestingly, we observed that the R-loop-bound enhancers in both CUTTL1 and iPSCs activate gene programs specific to each tissue type Figure , F.Taken together, these results further support the existence of eRNA R-loops and suggest that they do not necessarily impede the functionality of enhancers, particularly those which are critical to defining cell identity.In our analysis of iPSC HiC-sequencing data, we observed examples of MapR-detected R-loop formation which occurred nearby topologically-associated domain (TAD) boundaries Figure . PreviouHaving observed a strong enrichment of CTCF binding within cell type-specific dRNH data, we decided to assess whether this enrichment also occurs in a cell type-nonspecific manner. Interestingly, we observed a stark enrichment of dRNH-only (but not dRNH-shared or S9.6) peaks within CTCF consensus peaks Figure . MoreoveTo better understand R-loop-bound enhancers, we analysed chromHMM chromatin state data in iPSC cells (CUTLL1 data not available). Given their global association with eRNA transcription and cell type-specific gene programs, we anticipated that R-loops would be enriched at active enhancers, as was previously reported ,77. CuriPluripotent stem cells contain a large number of bivalent enhancers which prime early developmental programs . To undeHOX genes are arranged co-linearly in four clusters  and act to regulate anatomical patterning during early development . In iPSCWhile the preceding results reinforced the Class I/II hypothesis and characterized the R-loops which dRNH mapping technique uniquely detect, it remained unclear why dRNH consensus sites only cover 26.3% of the RL-region occupied genome, compared to 87.9% for S9.6 Figure . It was P < 2.1E−7) in eCLiP enrichment for S9.6-only genic peaks compared to intergenic peaks. This result aligned with our prior expectations, given that most RNA transcription occurs within gene bodies. However, in dRNH-only peaks, the difference between genic and intergenic enrichment was marginal (P = 0.048), suggesting that dRNH may bind preferentially to R-loops that lack RNA binding factor occupancy, regardless of genic localization. Of note, a recent study by Wang et al. utilized ex vivo and in vivo mapping with S9.6 and dRNH and demonstrated these R-loop sensors map R-loops similarly when analysed under the same experimental conditions  conservation across samples could highlight additional R-loop subtypes, regardless of whether those RL regions derive from S9.6 or dRNH-based mapping samples. For these experiments, we subdivided RL regions to identify those which are more ‘constitutive’ (more conserved across independent samples) or ‘variable’ (less conserved across independent samples). To accomplish this, we calculated RL region conservation percentages, defined as the proportion of high-confidence R-loop mapping samples that identify each RL region.Prior to our main analysis, we wanted to ensure that dRNH and S9.6-specific biases would not dominate the conservation analysis. Therefore, we calculated the conservation percentage distributions for S9.6 and dRNH consensus peaks separately, finding that they tended to be similar, with a noticeable increase in conservation levels within sites found by both dRNH and S9.6 . ComfortWe examined the distribution of conservation percentages across all RL regions and assigned them to bins for discrete analysis Figure . Next, wXRN2 has been associated with R-loops in previous studies, particularly with respect to transcription termination , was also associated with variable RL regions. PRC2 has been previously associated with R-loops in development-specific genes  (R-loop forming sequences (RLFS) are genomic regions which show favourability for R-loop formation ,61–64. Tdiction) ,18. We cdiction) , along wdiction) . Unexpecdiction) . Taken tWhile it has been proposed that roughly 3–13% of the genome may contain R-loops, these findings were derived from an analysis of peaks from a limited number of S9.6-based R-loop mapping samples ,70,71. FCastillo-Guzman and Chedin recently proposed two classes of R-loops: ‘Class I’ and ‘Class II.’ ‘Class I’ refers to R-loops which result from promoter-proximal pausing, and ‘Class II’ refers to R-loops which result from transcriptional elongation  boundaries . Our prein vitro . Moreovein vitro . Howeverin vitro , we deciWe calculated the enrichment of R-loop consensus sites within consensus peaks for CTCF and cohesin complex members SA1, SA2, RAD21 and SMC3. As expected, we observed an enrichment of intergenic R-loop consensus peaks within consensus sites for CTCF and all cohesin subunits Figure . IntriguTaken together, these results confirm and extend our previous findings  by showiWulfridge and Sarma used a dRNH-based method (BisMapR) to demonstrate that R-loops form within active (H3K27ac + H3K4me1/3) and bivalent/poised (H3K27me3 + H3K4me1/3) enhancers . Thus, we hypothesized that access to R-loops within gene bodies might be restricted in vivo by the presence of RNA binding proteins (RBPs). To assess this hypothesis directly would require a follow-up study. However, by leveraging publicly available RBP and transcription factor ChIP/eCLiP profiles, we employed an indirect, associative test.The results presented herein strongly support the Class I/II hypothesis and help to partially explain the presence of dRNH-only peaks, particularly in intergenic regions. However, it is still not known why S9.6-based mapping is capable of efficiently detecting R-loops over a larger proportion of the genome compared to dRNH, or why S9.6-based mapping is more efficient in detecting R-loops which occur in gene bodies downstream of the promoter proximal pause site Figure –K. One pnditions . One posin vivo R-loop mapping for detecting Class II R-loops.We observed an enrichment of S9.6 consensus peaks within RBP eCLiP binding sites . We alsoAnother outstanding question in the R-loop field relates to the dynamics of more constitutive R-loops (those which are consistently abundant across samples) and variable R-loops . To address this question, we analysed sites of R-loop formation, binned by level of conservation across samples Figure . From ouWhile the quality control method proposed in this study is both internally and externally valid Figures , it may We derived R-loop consensus sites from 56 dRNH and 191 S9.6 high-confidence R-loop mapping samples. This represents all the available human data which passed our quality thresholds. However, it still indicates a significant imbalance between dRNH and S9.6, leading to potential inconsistencies in the accuracy of our findings. To address the impact of this discrepancy on the results, we randomly downsampled the S9.6 dataset to match dRNH and demonstrated the stability of our differential analysis results across random downsampling permutations , C. HoweA second caveat of this analysis was the need to select a cut-off for ‘calling’ consensus peaks. In this analysis, we chose a cut-off of 15% such that the ‘summit’  within the consensus peak is supported by at least 15% of high-confidence samples (8 samples for dRNH and 28 samples for S9.6). The choice of 15% was based on the desire to obtain ‘variable’ RL regions, while reducing the potential for spurious peaks. We chose this cut-off based on manual examination of the number and ‘noisiness’ of peaks produced at 5%, 10% and 15% thresholds (not shown). In future studies, an automated approach for selecting this cut-off should be developed.Another caveat for the joint analysis of S9.6 and dRNH-derived data is that there are large discrepancies between the size of peaks and consensus sites between them Figures  and 6B. unstranded mapping method) is still the most well-represented modality and all analyses require conversion to unstranded data for compatibility with it. However, as stranded techniques gain in popularity and begin to outpace DRIP-Seq, we anticipate that future versions of the software used for processing these data will incorporate strand information. Therefore, we expect to eventually update the analyses presented in this study with stranded data to reveal higher-resolution insights into R-loop dynamics.We excluded bisulfite samples from our analysis due to the added complexity of processing them and the small number of available samples . However, these techniques should provide more accurate nucleotide-level R-loop mapping. Therefore, in future versions of the software used for this analysis, we will include methods for analysing these data types. Another limitation of this analysis is that it does not incorporate strand assignment for any of the strand-specific sequencing techniques included in the dataset. This is intentional as DRIP-Sequencing  and the list of differential RL regions between S9.6 and dRNH samples are provided on Figshare: https://doi.org/10.6084/m9.figshare.16920475.v2. UCSC genome browser sessions shown in this study are also provided:No new raw data were generated as part of this study. Processed data files generated as part of this study are provided via RLHub: https://genome.ucsc.edu/s/millerh1%40livemail.uthscsa.edu/NAR_2022_Fig_1BFigure https://genome.ucsc.edu/s/millerh1%40livemail.uthscsa.edu/NAR_2022_Fig_S2Chttps://genome.ucsc.edu/s/millerh1%40livemail.uthscsa.edu/NAR_2022_Fig_3AFigure https://genome.ucsc.edu/s/millerh1%40livemail.uthscsa.edu/NAR_2022_Fig_4HFigure https://genome.ucsc.edu/s/millerh1%40livemail.uthscsa.edu/NAR_2022_Fig_5EFigure https://genome.ucsc.edu/s/millerh1%40livemail.uthscsa.edu/NAR_2022_Fig_5IFigure https://genome.ucsc.edu/s/millerh1%40livemail.uthscsa.edu/NAR_2022_Fig_7GFigure https://genome.ucsc.edu/s/millerh1%40livemail.uthscsa.edu/NAR_2022_Fig_8DFigure https://genome.ucsc.edu/s/millerh1%40livemail.uthscsa.edu/NAR_2022_Fig_S8BFigure S8B: https://github.com/Bishop-Laboratory/RLBase-data. The repository containing the code used to generate all figures and tables in this manuscript is provided here: https://github.com/Bishop-Laboratory/RLoop-QC-Meta-Analysis-Miller-2022. The software developed as part of the present study includes RLPipes (https://anaconda.org/bioconda/rlpipes), RLHub (https://bioconductor.org/packages/release/data/experiment/html/RLHub.html), and RLSeq (https://bioconductor.org/packages/release/bioc/html/RLSeq.html).The code used in this study is provided publicly on GitHub. The repository containing the workflow for recreating all upstream data analyses is provided here: All accessions for data used in this study are listed in gkac537_Supplemental_FilesClick here for additional data file.


Dear Editor,The central contribution of this research is to present highly effective ensemble biomarkers for guiding therapies to ovarian cancer, which has become the major cause of death from gynecologic cancer in industrialized countriesTo enable the development of therapeutic predictive AI systems, digitized whole slide images (WSIs) of tissue microarrays are collected from samples with EOC and PSPC, containing 720 female patient sample tissue cores in total. For the data inclusion and exclusion criteria, patients are eligible for the retrospective study if they are EOC or PSPC patients receiving 1) concurrent bevacizumab therapy, 2) second‐line bevacizumab therapy post recurrence or 3) maintenance bevacizumab therapy. On the other hand, patients are ineligible if they are borderline malignant tumor or not primary ovarian or retroperitoneal carcinoma. The data include papillary serous carcinoma (444 samples), clear cell carcinoma (69 samples), endometrioid carcinoma (39 samples), mucinous carcinoma (10 samples), unclassified carcinoma (69 samples), and PSPC (89 samples) of 12 TMAs. This study has been approved by the hospital research ethics committee with the IRB approval number (TSGHIRB No.1‐107‐05‐171 and No. B202005070), and we construct the database and trace patient data from the period of 2013/3 to 2021/1. Supplemental Table For quantitative evaluation, the experiments were conducted in four parts. Firstly, the whole dataset (with 720 patient sample tissue cores) was randomly sampled and separated into two independent subsets, including 472 cores (66%) for training and 248 cores (34%) for testing as shown in Supplemental Table p = 0.012), and (HR = 0.23) indicates that patients who are predicted with effective treatment responses are 0.23 times as likely to get cancer recurrence, compared to patients predicted with invalid responses (Table (p<0.001)  Figure . See det HR = 0.2 indicate99%±3%, precision 99%±2%, recall 99%±2%, f‐measure 99%±2% and AUC100%±0%. Moreover, the proposed biomarker system based on single PKM2 or Ang‐2 data also obtains high accuracy and precision ; , respectively.Fourthly, to further examine the generalizability of individual models for unseen data, quantitative evaluation was performed based on 5‐fold CV. As shown in Table In conclusion, these findings suggest that the proposed ensemble model could assist treatment planning for personalized medicines and may be used as a new potential biomarker system to predict the bevacizumab therapeutic effect. Further conclusion and discussion are in the supporting information Section .There is no competing interest, which influences the work reported in this study.Supporting InformationClick here for additional data file.


Computational analysis methods including machine learning have a significant impact in the fields of genomics and medicine. High-throughput gene expression analysis methods such as microarray technology and RNA sequencing produce enormous amounts of data. Traditionally, statistical methods are used for comparative analysis of gene expression data. However, more complex analysis for classification of sample observations, or discovery of feature genes requires sophisticated computational approaches. In this review, we compile various statistical and computational tools used in analysis of expression microarray data. Even though the methods are discussed in the context of expression microarrays, they can also be applied for the analysis of RNA sequencing and quantitative proteomics datasets. We discuss the types of missing values, and the methods and approaches usually employed in their imputation. We also discuss methods of data normalization, feature selection, and feature extraction. Lastly, methods of classification and class discovery along with their evaluation parameters are described in detail. We believe that this detailed review will help the users to select appropriate methods for preprocessing and analysis of their data based on the expected outcome. Researchers working in these domains create and use a variety of data such as DNA, RNA, and protein sequences, gene expression, gene ontology, protein-protein interactions (PPI), etc.A genome is a complete set of genes in an organism. Genomics is a study of the information structure and function programmed in the genome. Genomics has applications in multiple fields, including medicine , agriculetc. Common implementations of these applications include splice site detection , DNA methylation, etc. Microarray and NGS technologies are the tools of choice for functional genomics studies. The functional genomics that deals with high-throughput study of gene expression is referred to as transcriptomics.Genomics data can be broadly classified into sequence and numeric data . The DNA sequence information can be determined by first generation , second etection , promoteetection , classifetection , identifetection , biomarketection , etc. Thetc.), contains the expression levels of thousands of genes experimentally evaluated in various conditions. Gene expression analysis helps us understand gene networks and molecular pathways. Gene expression information can be utilized for basic as well as clinical research , ArrayExpress, Genomic Expression Archive (GEA), etc. , etc. .t-test, ANOVA, etc.  is an artificial intelligence-based approach that emphasizes building a system that learns automatically from data and improves performance without being explicitly programmed. ML models are trained using a significant amount of data to find hidden patterns required to make decisions . ArtificML techniques, in general, are broadly categorized into supervised and unsupervised learning methods . Supervietc. The classification algorithms have also contributed to biomarker identification , Missing At Random (MAR), and Missing Not At Random (MNAR)  Figure . In MCARMissing values can be imputed using two different approaches. MCAR/MAR values are either embedded with a fixed value, or mean, median, or mode. However, this method creates lots of similar values if missing data is high. MCAR/MAR and MNAR values can be imputed using advanced computational techniques. The choice of imputation method depends on the accuracy of the results obtained from the downstream analysis. Computational techniques for estimating missing values can be categorized into four different approaches: Global, Local, Hybrid, and Knowledge Assisted  Table 2Table 2.Global approaches assume homogeneity of data and use global correlation information extracted from the entire data matrix to estimate missing values. The Bayesian framework for Principal Component Analysis (BPCA) is based on a probabilistic model that can handle large variations in the expression matrix . In BPCAetc. ANNimpute utilizes MLP (Multi-Layered Perceptron) based architecture that is trained with complete observed data  is a standard ML-based missing-value imputation strategy  Figure . A missiSome studies have compared the global and local approaches for their performances. SVD and KNN require re-computation of a matrix for every missing value, which results in prolonged evaluation time . SVR, BPThe internal correlation among genes affects the homogeneity and heterogeneity of data and, therefore, the performance of global and local imputation approaches . In ordeetc. Integrative MISSing Value Estimation (iMISS)   is one s (iMISS)  is anoth (iMISS)  formulat (iMISS)  utilizes (iMISS) .Although a large number of missing value imputation methods are available to the users, there are still quite a few challenges when it comes to the application of imputation methods to the data. Firstly, there is only limited knowledge on the performance of different imputation methods on different types of missing data. The performance of the imputation methods may vary significantly depending on the experimental settings. Therefore, it is important to systematically evaluate the existing methods for their performance on different platforms and experimental settings . Secondletc., does not get affected by variability in the data. However, the performance of class comparison, class discovery, and classification methods, e.g., KNN, SVM etc., may get affected due to technical variations in gene expression signals. The gene expression signals may vary from sample to sample due to technical reasons such as the efficiency of labeling, amount of RNA, and platform used for the generation of data. It is important to reduce the variability due to technical reasons but preserve the variability due to biological reasons. This can be achieved using data normalization or scaling techniques  is a commonly used entropy-based method for feature selection applied to expression array data . IG calcFeature extraction methods are multivariate in nature and are capable of extracting information from multiple feature genes. Classical Principal Component Analysis (PCA), an unsupervised linear transformation technique has been used for dimensionality reduction  Figure . PCA buiAll filter approaches (both simple filter and feature extraction methods) ignore the interface with classifier which can result in poor classification performance. This limitation can be overcome by wrapper and embedded approaches.The wrapper approach is a feature selection approach that wraps a specific machine learning technique applied to fit the data . The wrap-value, r-square, or residuals sum of squares of the model to maximize performance. A Genetic Algorithm (GA) is a stochastic and heuristic search technique used to optimize a function based on the concept of evolution in biology  is an iterative method of feature selection . It calc biology   and Ridge regression . Both thThe K-means clustering technique is an unsupervised method that is utilized to eliminate redundancy in high-dimensional gene expression data  Figure . In K-meANN-based Autoencoders (AE)  is an unA hybrid approach is considered as a combination of two or more filter and wrapper methods. It can reduce the error rate and the risk of overfitting. A well-known feature selection hybrid approach is Recursive Feature Elimination with a linear SVM (SVM-RFE) . SVM-RFEFeature selection or feature extraction process can generate high quality data for classification and predication analysis. It should be noted that for classification analysis, feature selection is carried out only on the training dataset. For clinical applications, it should be noted that model interpretation is important, and feature extraction technique may cause the model interpretation challenging as compared to feature selection techniques.t-test, ANOVA, etc.  from all training samples using the distance metric. The performance of KNN is dependent on the threshold of the feature selection method and is subject to the distance function (K-Nearest Neighbor (KNN) is one of the techniques that can be utilized for the classification of expression array data . The clafunction . An incrfunction . Anotherfunction . It calcA Decision Tree (DT)  approachetc. In some studies, SVMs performed better than DT and ANN-based techniques  curve and Precision-Recall curve are other standard tools used by binary classifiers as performance measures. ROC and MCC are more robust measures as compared to accuracy since accuracy is affected by class imbalance  or Root Relative Squared Error (RRSE) are examples of error-rate-based evaluation. The accuracy metric is the most common performance evaluation parameter utilized to find the accuracy of classification. However, accuracy alone is not enough for performance evaluation  and thermbalance .The problem of classification of expression data is both biologically important and computationally challenging. From a computational perspective one of the major challenges in analyzing microarray gene expression data is a small sample size. Error estimation is greatly affected by the small sample size, and the possibility of overfitting of data is very high . AnotherThe third type of microarray analysis is class discovery which involves the analysis of a set of gene expression profiles for the discovery of novel gene regulatory networks or sample types. Hierarchical Clustering Analysis (HCA) is a simple process of sorting instances into groups of similar features and is very commonly used for the analysis of expression array data . HierarcK-means is an iterative technique that minimizes the overall within-cluster dispersion. K-means algorithm has been utilized to discover transcriptional regulatory sub-networks of yeast without any prior assumptions of their structure . The advAnother technique for class discovery analysis is the Bayesian probabilistic framework which uses Bayes theorem . This teviz. internal validation index, relative validation index, and external validation index  . The aim


Complex high-dimensional datasets that are challenging to analyze are frequently produced through ‘-omics’ profiling. Typically, these datasets contain more genomic features than samples, limiting the use of multivariable statistical and machine learning-based approaches to analysis. Therefore, effective alternative approaches are urgently needed to identify features-of-interest in ‘-omics’ data. In this study, we present the molecular feature selection tool, a novel, ensemble-based, feature selection application for identifying candidate biomarkers in ‘-omics’ data. As proof-of-principle, we applied the molecular feature selection tool to identify a small set of immune-related genes as potential biomarkers of three prostate adenocarcinoma subtypes. Furthermore, we tested the selected genes in a model to classify the three subtypes and compared the results to models built using all genes and all differentially expressed genes. Genes identified with the molecular feature selection tool performed better than the other models in this study in all comparison metrics: accuracy, precision, recall, and F1-score using a significantly smaller set of genes. In addition, we developed a simple graphical user interface for the molecular feature selection tool, which is available for free download. This user-friendly interface is a valuable tool for the identification of potential biomarkers in gene expression datasets and is an asset for biomarker discovery studies. Despite advances in machine learning methods, few are utilized in the analysis of biomedical data. Currently, the most common approach to identify genes of interest or other molecular features in large datasets is to perform a differential expression analysis where each feature is assessed individually using statistical tests or curve fitting ,2. Howevple size ,4,5. ThuFeature selection is a machine learning technique for identifying features of importance in high dimensional data. The main goal of the feature selection is to reduce the number of features to a small set of most variable predictors required for classification. A typical -omics dataset has a relatively low sample size compared to the number of features. This affects classifier accuracy and increases the risk of overfitting the prediction model by developing an overly complex classifier with perfect classification on a training set, but low performance and generalizability on unseen data ,11,12,13Many of the proposed applications of feature selection in -omics studies use only one type of feature selection algorithm ,17,18,19Molecular Feature Selection Tool (MFeaST) that enables the identification of candidate biomarkers in -omics data by the reduction of features to the most valuable predictors. This novel, ensemble-type feature selection technique does not rely just on one approach, but utilizes multiple univariable and multivariable as well as filter-, wrapper-, and embedded-type feature selection techniques. The algorithm applies a greedy method [MFeaST to identify features of interest in many different -omics datasets [MFeaST in selecting a small set of features as potential biomarkers in prostate adenocarcinoma mRNA profiles using immune-function related genes. Our analysis showed that the biomarkers selected using MFeaST could also be used to build classification models that perform better in all comparisons than the other models built in this study using differentially expressed genes or without feature selection.In this manuscript, we present our y method  to rank datasets ,32,33,34the Molecular Feature Selection Tool (MFeaST) to rapidly identify candidate biomarker molecules in -omics data using an ensemble feature selection approach. This MATLAB-based application accepts a feature matrix of -omics profiles and ranks each feature based on its ability to discriminate between two classes. Features are assessed for classificatory ability using an ensemble of multiple univariable and multivariable filter-, wrapper-, and embedded-type feature selection algorithms  from frozen prostate adenocarcinoma samples were obtained from the TCGA-PRAD dataset  from theOnly primary tumor sample profiles (n = 497) were used. Samples that were solid tissue normal (n = 1) or metastatic (n = 52) in the TCGA-PRAD dataset were identified and removed. Data were preprocessed using an established methodology for data preprocessing and the detection of outlier and batch effects . No batcThe prostate adenocarcinoma samples were categorized according to the Pam50 protocol, an established protocol to determine molecular subtypes in prostate and breast cancer ,38. The 2 transformed. Since the log2 of zero is undefined, the zero values were replaced with a computed low value of the same magnitude as the smallest non-zero value in the dataset.The existence of luminal and basal subtypes across carcinomas was previously reported , howeverMFeaST. On the comparison tab, one of the comparisons was selected . The MFeaST is an ensemble-based algorithm that relies not on one but on multiple univariable and multivariable as well as filter-, wrapper-, and embedded-type feature selection techniques. Therefore, for the identification of potential biomarkers in the current dataset, all available algorithms were selected, with 5-fold validation, optimization, and five iterations for the sequential algorithms. The 5-fold validation reduced the chance of overfitting by performing the feature selection five times, each time on a different subset of data. The ranking results of the MFeaST were reviewed on the Results tab, where each original feature was assigned and ranked based on an ensemble score between 0 and 1, with 1 indicating the perfect ability to discriminate between conditions of interest. The univariable feature selection methods assign a score of 0–1 to each feature. The multivariable methods search for the best combination of features that provide the highest discrimination accuracy. In the final selected set, each feature obtains a score of 1, and features that are not selected each obtains a score of 0. The results were further inspected within the software using its visualization tools to identify and select the most valuable predictors, focusing on the top-ranking 10% of the results that provided the best clustering results.Potential biomarkers were identified for each of the comparison groups (A) LumA vs. Basal, (B) LumB vs. Basal, and (C) LumA vs. LumB as follows. Highly expressed immune-function genes (n = 574) were imported into MFeaST were compared to the conventional approach using differential expression method. The differentially expressed results were computed as follows. Preprocessed data were imported into R version 3.6.0 (“Planting of a Tree”) for analysis with the edgeR package (version 3.25.8), as described in the user’s guide [The results of the biomarkers selected with ’s guide . Briefly’s guide . All difMFeaST selected genes, the second using differentially expressed genes (FDR ≤ 0.05), and the third using all highly expressed genes. All classification algorithms in the MATLAB Classification Learner App (n = 23) were evaluated to identify a family of classifiers that worked best for discriminating (A) LumA from Basal, (B) LumB from Basal, and (C) LumA from LumB. Preprocessed data were scaled to have 0 mean and unit-variance, then imported into Jupyter Notebook with Python 3.7 to construct final classification models with 10-fold cross-validation. For each model, the data were randomly divided into 10 disjoint sets of equal size +/−1. The training of the SVM classifier was performed 10 times and each time it was tested on a different set held out as a validation. The mean and standard deviation of accuracy, precision, recall, and F1-score were computed over the 10 validation sets to evaluate the classification model performance.The selected biomarker candidates were further evaluated for their ability to classify the conditions of interest. Three classification analyses where performed: one using p-value ≤ 0.05 were considered significant.Clinically relevant and prognostically important markers to prostate adenocarcinoma from the TCGA repository were compared between patients of different pathological types using SPSS Statistics . The Kruskal–Wallis test was used to investigate differences between variables with ordinal or ratio data. The Chi-square test was used for variables with categorical data. Results with a MFeaST algorithm on any dataset by end-users with different levels of computational expertise. A detailed user guide is presented in the results. While coded in MATLAB, MFeaST is a completely standalone application that does not require a MATLAB subscription, instead running on the freely available MATLAB Runtime software . MFeaST can be run on most Windows or Apple desktop computers .A desktop application with a simple, intuitive, graphical user interface (GUI) was developed to facilitate easy application of the p = 0.020) and pathological stage (p = 0.009). Differences in histological type could not be assessed as one or more categories had expected counts <5. These results show that only the grade and pathological stage are the potential confounding variables in any future statistical analysis of the molecular subtypes.The clinicopathological statistics for important prognostic markers of prostate adenocarcinoma are presented in MFeaST for (A) LumA from Basal, (B) LumB from Basal, and (C) LumA from LumB  33 genes used to discriminate LumA from Basal, (B) 15 genes used to discriminate LumB from Basal, and (C) 18 genes used to discriminate LumA from LumB  to 0 (worst) using the rom LumB . The toprom LumB . This serom LumB .MFeaST-selected genes. After filtering for statistical significance (FDR ≤ 0.05), differential expression analyses found (A) 136 upregulated and 159 downregulated genes in LumA when compared to Basal, (B) 233 upregulated and 239 downregulated genes in LumB when compared to Basal, and (C) 161 upregulated and 167 downregulated genes in LumA when compared to LumB  selected by both MFeaST and differential expression were listed under “intersection”; and (3) only differential expression selected were listed under “Differential expression only”. All but one MFeaST-selected gene was among the differentially expressed genes for all three comparisons  was between 80 and 97%. The recall, which measures the ratio of true positives to the total actual positives , was between 83 and 97%. Finally, the F1-score, which is a harmonic mean of precision and recall and accounts for both false positives and false negatives, was between 79–97% for predicting the LumA, LumB, and Basal subtypes . The classification accuracy ranged from 77 to 94%, precision 74–94%, recall 75–97%, and F1-score 73–95%. This demonstrates that the MFeaST approach is useful for the identification of a small list of potential biomarkers, which is very important in biomarker discovery studies.When evaluated in the MATLAB Classification Learner App, linear SVM was the top performing classification algorithm in all A–C comparison cases. Therefore, all final classification models for both on folds . Using asubtypes . Comparaubtypes , an ensemble feature selection desktop application that can be utilized to identify biomarkers of disease or other clinical conditions by exploring the high dimensionality of the data. To demonstrate its effectiveness, we applied MFeaST to prostate adenocarcinoma mRNA profiles. Using only immune-related genes, we identified a small set of features that can effectively predict three different molecular subtypes. The classification models that were built using MFeaST-selected genes outperformed other models built in this study using differentially expressed genes and all genes, but more importantly, used a very small set of genes.In contrast, machine learning driven feature selection can be used to detect patterns and interactions in -omics datasets and lend insights into the mechanisms of the conditions studied . In thisMFeaST has several advantages over conventional differential expression analysis and existing feature selection methods. First, MFeaST is an ensemble-type method that combines multiple univariable and multivariable as well as filter-, wrapper-, and embedded-type feature selection techniques. This novel combination of algorithms makes MFeaST more generalizable and reliable for identifying features of importance [MFeaST utilizes the power of machine learning algorithms to rank molecular features. These algorithms characteristically rely on mathematical functions or branching logic that determine the class boundary rather than hypothesis testing, and are therefore less dependent on sample size. Using MFeaST allows one to select a small set of features as potential biomarkers, enabling a more effective way for biomarker discovery. From a machine learning perspective, using a small set of features can potentially avoid issues with overfitting, which is especially important for datasets with a small number of available samples. Third, we developed an intuitive, GUI that allows users, irrespective of computational expertise, to apply MFeaST on any dataset. Additionally, we successfully applied MFeaST to find biomarkers in many different omics datasets including messenger RNA-Seq [portance . Second, RNA-Seq ,25,26, R RNA-Seq ,28, micr RNA-Seq ,30,31,32 RNA-Seq , and pro RNA-Seq . These dMFeaST allows one to identify potential biomarkers based on a binary comparison. However, it can be extended to multiclass problems where the dataset has more than two conditions of interest. In our previous studies, we successfully applied the method in the one-vs.-all approach [MFeaST in a multi-layer classifier by assembling binary decision-layer models according to a predefined hierarchy [MFeaST was used to identify features of interests to address a binary problem. This divide and conquer approach allows one to reduce a complex problem into smaller, manageable tasks. Following this approach, a hierarchical classification of LumA, LumB, and Basal subtypes can be implemented. First the LumA vs. LumB classifier is applied on an unknown sample. If a sample is classified as LumA, then the LumA vs. Basal classifier is applied as a final prediction. On the other hand, if a sample is classified as a LumB subtype, then the LumB vs. Basal classifier is applied as a final prediction.The approach ,34, wherierarchy ,30,31,32MFeaST allows for the selection of a small set of features for further analyses or to build classification models. Not only did the features selected with MFeaST result in a better classification of the three molecular subtypes than the other models in this study, but it also used a significantly smaller set of genes than in the differential expression approach. Since MFeaST ranks all available features—as opposed to compressing or filtering them—the final selection can be further enhanced and interpreted by a domain expert by observing each feature ranking score, assessing it visually or experimentally. Reducing a high-dimensional feature space to a small set of highly informative predictors is valuable for biomarker development and for building generalizable prediction models.Using prostate adenocarcinoma mRNA data, we demonstrated how MFeaST was still able to identify a small set of features that reliably discriminated between the molecular subtypes. In addition, we did not further investigate the selected immune-related genes, highlighting the critical role that domain experts need to play in biomarker development. It is important to note that the prediction of the LumA vs. Basal subtype was relatively lower and less stable (large std) than for other comparisons across all models. This indicates that it was difficult to achieve a good and stable discrimination between the LumA and Basal samples based on the immune-function genes. Therefore, other molecular markers should be examined, and further validation of the classifier on other prostate adenocarcinoma datasets should be explored.The proof-of-principle example presented here was limited by the use of the Pam50 gene set, rather than a gold-standard pathological diagnosis, to label the samples by their molecular subtype. Nevertheless, while some samples were excluded from the study due to labelling uncertainty, MFeaST as an effective approach to identify molecular features of importance as potential biomarkers in high-dimensional -omics data. In our analyses, we focused on immune-function-related genes and investigated their potential as biomarkers using machine learning approaches. In general, MFeaST selected features can be further examined by domain experts through pathway analysis, literature search, and experimentation. We are continuously working on adding more functionalities to MFeaST and future updates may include support for multiclass feature selection and the use of unsupervised feature selection methods. We expect MFeaST, through its effectiveness and easy-to-use GUI, to have a large impact on existing and future -omics based biomarker discovery studies.Here, we introduce


However, analysis and maintenance of high volume of genetic mutation data to build a machine learning (ML) model to predict the cancer type is a computationally expensive task and is often outsourced to powerful cloud servers, raising critical privacy concerns for patients’ data. Homomorphic encryption (HE) enables computation on encrypted data, thus, providing cryptographic guarantees to protect privacy. But restrictive overheads of encrypted computation deter its usage. In this work, we explore the challenges of privacy preserving cancer type prediction using a dataset consisting of more than 2 million genetic mutations from 2713 patients for several cancer types by building a highly accurate ML model and then implementing its privacy preserving version in HE. Our solution for cancer type inference encodes somatic mutations based on their impact on the cancer genomes into the feature space and then uses statistical tests for feature selection. We propose a fast matrix multiplication algorithm for HE-based model. Our final model achieves 0.98 micro-average area under curve improving accuracy from 70.08 to 83.61% , being 550 times faster than the standard matrix multiplication-based privacy-preserving models. Our tool can be found at  One of the biggest components of precision medicine is to incorporate patients’ genetic information to the diagnosis, treatment, and decision making2. This is further bolstered with the availability of genetic data with the decreasing sequencing costs. In the context of precision medicine for cancer, distinguishability between genetic mutations of normal and malignant tissues is the crux of cancer genomics. These somatic genetic changes accumulated during a person’s life are heavily correlated to the progression of several cancer cases3. Somatic single-nucleotide variation (SNV) and copy-number variation (CNV) on protein-coding genes, especially on oncogenes, tumor suppressors and cell cycle regulators are known to cause tumor formation and progress. However, the heterogeneity in various levels makes it difficult to understand precisely which gene is involved in which cancer type. From a data analysis perspective, a statistical inference of cancer type requires analyzing huge volume of genomic data to find correlations between somatic mutations and cancer type.Precision medicine, the process of tailoring diagnosis and treatment for an individual patient, has far-reaching results when analyzing complex diseases4. An ML model is a function (combination of linear and non-linear functions) with trainable parameters which learns about the data by decreasing the loss between prediction and true values. This computationally expensive task of training is often beyond the capabilities of personal machines and is outsourced to powerful servers. After a model is trained, it is hosted in the cloud where different parties can use it as a service to predict cancer type of new patients.Machine learning (ML), has had an unprecedented success in correlating complex data patterns to an inference in several critical infrastructures, including healthcare5. If the sensitive data is always encrypted, then it can be protected during computation as well as in transit. In other words, if an entity (like the cloud) is not trusted, data should not appear in plaintext form.There are privacy concerns in using genetic information for this type of inference. Genetic information, especially in the clinical settings, are subject to privacy protections and may not be used in public server and cloud computing setting due to the restriction. A genomic data leakage may also be permanent, unlike other private data like passwords, credit cards, etc., which can be changed. A partial leak of genomic data may reveal important information about the individual and may also be used to reconstruct their genome6 had prohibitive computational overheads for real-world applications like ML-based inference. During inference an ML model performs linear operations, typically matrix multiplications and additions between the data matrix, the weight matrix, the bias matrix, and non-linear operation on the resultant output. Linear operations in HE become more and more expensive as the number of features increase, i.e. as the amount of information in a dataset increases, the number of operations in HE increases, making encrypted computation impractical. Genomic datasets are inherently high-dimensional and the current privacy-preserving computation literature is limited in exploring these datasets8. Hence, to effectively compute on genomic data in the encrypted domain, we explore feature engineering methodologies to reduce overhead and improve the inference methodology to increase scalability in the encrypted domain. Our data consists of more than 2 million CNV and SNV information of 11 different cancer types9. Using an ML model on the raw cancer dataset would require multiplication of matrices with millions of features (columns), which is extremely expensive in HE.Homomorphic encryption (HE) is a cryptographic technique that allows for computation on encrypted data. In this setting, researchers can use their homomorphically encrypted ML models in public servers without revealing the input, interactions with the input and output, as well as the outputs of the models. The first Fully HE (FHE) scheme for arbitrary computation proposed by Gentry et al.11. For dimensionality reduction, we develop a somatic mutation encoding-based feature engineering methodology involving feature (gene) selection and genetic (mutation) information encoding, leveraging both biological intuition and statistical tests. Trivially using just statistical scores may result in overfitting especially for genomic datasets with the number of predictors (f) several times larger than the number of samples (|X|) i.e. 12. Using our methodology of somatic mutation encoding, we reduce the dimensionality of the task (from over 2 million mutations to 43K features), but still the genomic data remains high-dimensional as compared to benchmark ML datasets used for evaluating homomorphic encryption-based privacy-preserving studies. Since our application needs several thousands of features for accurate predictions, not only our time budget is completely exhausted by the linear operation, but also standard matrix multiplication does not offer the performance needed.The intuitive solution towards faster private inference is to reduce the number of computations. This translates to reducing the number of features using feature selection, as can be seen in other HE applications in genomicsAnother drawback of current HE-based implementations of private inference is that they are designed to maximize throughput, computing on thousands of inputs together to improve efficiency in a cumulative way. However, they suffer in latency, i.e. the algorithms would take the same time to compute on just one input as it would take for thousands of inputs. Therefore, for algorithms that do not optimize for latency would generally wait for thousands of test data points to send a single query to the cloud for the inference on those data points. But the real-world application we consider in this work benefits from improved latency, i.e. it is important for the private inference algorithm to infer on a single data point instead of waiting for a batch of thousands of patients to utilize the query limit. In summary, to enable practical real-world private inference, we need the ability to compute on high-dimensional data in the encrypted domain with low latency and high throughput.honest but curious/semi-honest threat model explored in the cryptography literature13. In this work we focus on privacy-preserving inference following the honest but curious/semi-honest threat model. The goal of such an adversary is to gain (sensitive) information while not hindering the computation at the cloud. In our application, we represent the training data accessible to the cloud as (Cloud server The cloud is honest but curious i.e. it infers cancer type correctly but is curious of the highly identifying patient (genomic) data motivating encrypted computation at the cloud. The cloud, however, has access to public training data collected from several research entities . The resource heavy data collection and maintenance and model training and maintenance are outsourced to the cloud.Client The client  owns sensitive data and wants to perform cancer prediction on the data without revealing the data to the cloud. This data, in our study, is represented by test data.In a nutshell, for a private cancer detection solution to be practicable, it must have three properties: (1) comprehensive high-dimensional genomic analysis for high detection accuracy with a focus on explainability, (2) cryptographically secure privacy guarantees, (3) practical inference time and high throughput. In developing the privacy-preserving tumor prediction model, we list our contributions as follows: predictive genes from our findings and compare it to Gene Ontology enrichment analysis to understand the significance of certain genes in predicting a cancer type.We develop a three-step feature engineering methodology targeted towards practicable encrypted cancer prediction and demonstrate the importance of feature selection and encoding based on biological significance of genetic alterations and statistical tests. We discuss the 15.We build high-performing ML model , and binary models using the same encoding for each cancer type. We compare our methodology with the reported baseline and we achieve a 13% increase in accuracyWe propose a fast matrix multiplication algorithm for high-dimensional matrices, specifically designed to implement HE-based privacy-preserving logistic regression model to ensure high performance.We optimize our private cancer prediction methodology for low latency as well as high throughput.16, an FHE scheme, and demonstrate that we can perform privacy-preserving cancer prediction for We implement our privacy-preserving cancer-prediction model using BFVWe open-source our methodology and implementation.Paper roadmap: We first discuss data encoding schemes and feature selection using biological intuition and statistical tests in “In this work we focus on privacy-preserving inference following the Since computations in the encrypted domain are expensive, private inference on any type of should prioritize towards less number of computations. This translates to a low number of features and smaller ML models . Our methodology can be divided into two parts. In the first part of our methodology “” sectionFor the cancer prediction to be correlated to both CNV and SNV information, the CNV and SNV features can be concatenated together, which can be used to train an ML model. CNV subset has 25 K features. But the SNV subset corresponds to over 2 million mutation rows, which may equate to over 2 million features if each of these mutations is analyzed separately. The concatenated dataset, thus, consists of more than 2 million features. Therefore, instead of representing a mutation as a feature, we represent a gene as a feature with encoded mutation as the value of that feature. However, this approach faces the challenge of compacting mutation information of over 2 million data-points to 25 K data-points (corresponding to 25 K genes).15 on cancer detection using somatic mutations observed that the frequency of mutation of a gene correlates to higher prediction accuracy. We first choose the genes with the highest number of mutations as we hypothesize that there might be an additive effect of the mutations and more somatic mutations will have higher impact on the function of the gene17. We then focus on the mutations with high frequency as recurrent somatic mutations have more statistical power to predict a cancer type due to more patients having them. It is also suggested that recurrent mutations are more informative of the mutational processes in cancer, which might help with the prediction of the cancer types18 if different types are governed by different mutational processes. Therefore, as the first step of feature selection, we choose genes with higher number of mutations. But each cancer type corresponds to a higher mutation in a different gene. We first rank the genes based on their SNV frequency in the patient cohort for each cancer type by also taking into consideration genes with more than one SNVs on them. We then combine the ranked genes from different cancer types and finally select the top 10,000 genes with highest SNV frequency for each cancer type as our features. This step also ensures removal of genes with low SNV frequency and reduce the dimensionality of our feature space. With all cancer types combined, we have a total of 18,606 genes.Previous studies 19. As mentioned in the dataset section, each SNV on a gene is represented by multiple characteristics. This information need to be meaningfully merged with the CNV information of each gene. We explore the following encoding schemes based on a biological intuition as explained below: Using presence of an SNV on a gene The genes selected using frequency are merged for all cancer types. In this encoding we aim to study if a particular gene (mutation) is highly correlated to a cancer type. We assign a binary value  to each of genes of a patient to denote the presence or absence of one or more SNVs. This allows us to encode SNV information in a categorical manner. This binary value per gene is used as a feature.Using the type and impact of an SNV The impact of mutation of a gene is calculated using Ensembl Variant Effect Predictor (VEP) (see 20.VEP) see 20.j. Each of the encoding scheme are given equidistant real values between 0 to 1. We also experimented with different ranges but did not find any improvement in test accuracy. We think that both of these measures are useful in classifying the tumor type. We first encode the first measure by assigning values for  as  for each m is the effect of the j. We then combine this encoding with the second measure as For each SNV in the dataset, we have two types of measure: 1—a qualitative measure indicating whether an SNV has a tolerated or deleterious effect; 2—a quantitative real-value measure (s) of an SNV, the qualitative confidence of the effect of an SNV on a gene is also as a categorical variable with values as . We encode them as  since intuitively we want to assign a higher importance to a high mutation effect. A gene with no mutation is assigned 0 effect. Similar to the previous feature, for a gene with multiple SNVs at different locations, the values are added to finally represent the effective value of all the SNVs in a gene. The resulting value per gene is used as a feature  mutual information, which measures the gain in information or reduction in entropy if a feature is selected, (2) chi-square test, which measures the difference between the actual and the expected output if a feature is selected, and (3) f-score, which measures the Analysis of Variance (ANOVA). We choose chi-square test as a feature selection metric since we achieve the best possible accuracy when compared to feature selection with mutual information or f-score statistics. We choose E represents the expected value, O represents the actual output and i represents each instance of a The previous steps of feature selection incorporate biological intuition. In this step, we explore statistical tests for evaluating feature importance. Statistical methods like n features are selected and are used to train a classification model. We also use this step to analyze the selected genes and their relative importance in cancer type prediction in “We run the While our encoding scheme and feature selection methodologies are targeted towards reducing the number of computations in the encrypted domain, we follow a similar philosophy to select our ML model. We perform a grid search over several small ML models like Support Vector Machine (SVM) , logistic regression, and Deep Neural Networks (DNNs with two fully connected hidden layers with relu activation) as possible classification models see . To captl1) penalty22 to the logistic loss function during training such that the features that are unlikely to contribute to the prediction are penalized and weighted zero. Therefore, if the logistic loss function is given by We find that the logistic regression model performs best for several encoding schemes with the best test accuracy of 83.61%. To tackle overfitting, we take measures in two stages: (a) during training, (b) after training. We introduce a Lasso  can give a false sense of performance as a random guess (of the label with the highest number of samples) may also result in a high accuracy. For a holistic performance evaluation of our classifiers, we plot Receiver Operating Characteristics (ROC) Curve and report the individual area under curve for each class and the Micro-average Area Under Curve (MAUC) for the classifier. Since ROC curves reflects the entire range of probability threshold, it is a more robust metric and is used in genetic analysis.15). Note that training of individual classifiers is to show the effectiveness of the gene selection and encoding schemes. Since the sample sizes for training individual classifiers are relatively low, the performance may not translate to larger, more diverse datasets. These models are supplementary to our main prediction model and focus on one type of cancer. These binary models can also serve as independent models since genomic datasets are not balanced. The features for this single cancer type model are chosen following the steps described above and the classifier is trained using a binary label: 0 for the all of the other cancer types and 1 for the cancer type of interest. We create separate models for each of the 11 cancer types and call these models as binary models since the prediction is converted into a binary classification task.We also build models to predict each cancer type separately  and the binary models for individual diseases are different because of the feature selection steps. In our binary models, the genes important to a specific disease are selected. However, for our main prediction model, the genes which are cumulatively important for all the 11 labels, are selected. Hence, we report both the analyses in “It should be noted here that the From analyzing the nature of genomic mutation data and the trends in accuracy . This effectively converts our HE operations into fixed-point arithmetic. The scaled matrix of inputs n coefficients. We pack n features of each row into a polynomial. This leads to an encoded matrix of dimensions t. From experimental results, we determined that our inputs and weights require 14 bits of precision. Since our inputs are in the interval n features encoded into a polynomial, we are actually encoding into a pair of polynomials, one with coefficient modulus Figure pk. The encrypted matrix W, which has dimensions Y| is the number of outputs. The transposition is a requirement of our computation. It packs several feature weights of an output in a plaintext polynomial, leading to an encoded matrix of weights sk and obtains the result of the inference in plaintext.The encoded matrix of inputs n. Each row of Our privacy-preserving matrix multiplication algorithm, optimized for implementation in HE, is displayed in Algorithm 1. It receives three arguments: The encrypted matrix of inputs d features, belongs to a class (k) is given by Tumor prediction is a classification problem which we address using multinomial Logistic Regression (LR). An LR model is trained by reducing the logistic loss function. During inference, the probability that an input The evaluation of private cancer prediction methodology is dependent on the precision of the plaintext model and performance of the private inference protocol. We report the evaluation of somatic mutation encoding towards accurate cancer prediction in plaintext in “15. A similar study from 201614 also developed cancer prediction models based on TCGA database but the 2019 study supersedes the former in performance. For privacy preserving implementation of cancer prediction, although the plaintext matrices have thousands of features, they get packed in just a handful of ciphertext . Thus, the matrix multiplication algorithm multiplies small matrices. While there are matrix multiplication algorithms with lower complexity than the standard, their benefit is only for large matrices. For smaller matrices they are worse than the naive matrix multiplication due to costs to prepare the matrices. Furthermore, techniques that improve cache utilization in matrix multiplication do not work for HE because ciphertexts are very large, which precludes keeping many ciphertexts in the cache memory. Other encrypted matrix protocols require back-and-forth communication between the client and the server, which is a different approach from ours (send once/receive once) and therefore, not consistent with our usage model, since communication costs are the bottleneck in these protocols23. Thus, we implement the standard LR in HE and compare our optimized ML model with it in the encrypted domain.Comparison to the state-of-the-art: To the best of our knowledge cancer prediction performance using the exact same dataset has not been published. Therefore, we compare the performance of our plaintext model (accuracy) with the 2019 study on prediction of cancer types since the authors use the same TCGA database14 and of 70.08%15. These methods also used the SNV frequency to prune the number of features.We report different performance metrics and the information on the features used for different models tested in Table We also experiment with just the copy number information for all the 25,128 genes and run a f features using both SNV and CNV information as described above and train several models . All the classes achieve an ROC area under curve of more than 0.9. This experiment also shows that although CNVs are more informative, using both CNV and SNVs result in the highest prediction accuracy. We also test our logistic regression model using mutual information and f-score as feature selection methods but we achieve lower test accuracy of 81.95% with 32,000 features, and 15 due to many factors contributed by our approach including the feature engineering, variant effect encoding, as well as addition of CNV information to the feature set.We select top mentclass2pt{minim15, we built 11 specific models for each cancer type. Although the minimum accuracy for a binary model is lower than the minimum accuracy reported by the authors, we report a higher value in terms of minimum auc achieved by all binary models. We use both CNV and SNV information to select the top 34,000 features. In these experiments, we train 11 different models, where each one detects a particular tumor. We report the performance (test accuracy and micro-average score) in Table To compare with the specific models depicted in the state-of-the-artmore important than their CNV counterparts. But CNV information improves the test accuracy by adding potentially more relevant biological information.We discuss our findings on the top genes selected. A total of 17,962 genes are selected for CNV data and 16,038 are selected for SNV data as the most informative features. However, more than 60% of these genes are common, i.e. 11,133 genes are selected based on both CNV and SNV information. Considering the When we investigated the top 10 most informative genes based on the SNV information, we found “PTEN” are “APC” genes, which are known tumor suppressors; “MUC16” gene, which is a biomarker for ovarian cancer; “ZFHX3” gene, which is implicated in prostate cancer; “CCDC168” gene, which is known to be associated with Prostate Carcinoma and Uterine Body Mixed Cancer. The other 5 genes in this list are also implicated in important cellular activities that could potentially be related to cancer. The first gene in the top 10 most informative genes based on the CNV information is the first ever known tumor suppressor “RB1”. Similarly, “CDKN2A” and “CDKN2B” genes are also known tumor suppressors. “RCBTB2” gene is known to be repressed in prostate cancer. The “CDKN2B-AS1” gene has the silencing power of many other genes in the genome and strongly implicated in various cancer types. The other 5 genes in this list are also implicated in important cellular activities that could potentially be related to cancer. These genes and their corresponding Gene Ontology (GO) term enrichment are depicted in Supplementary.The first insight is that biological intuition combined with high-dimensional data analysis methods can together achieve high accuracy (MAUC) while reducing the effective number of features. We further validate, using other studies, that our ML model indeed selects features that are biologically relevant. Using domain expertise, our ML model achieved a high performance of 0.98 MAUC with logistic regression ML model.We further plot the variation of test accuracy (of a model trained using best hyper-parameters) with the number of features in Fig. 24 with the underlying Microsoft SEAL library25 and encryption parameters set as: polynomial degree We evaluate our privacy-preserving cancer prediction model on an AMD Ryzen Threadripper 3960X 24-Core Processor with 128 GB RAM using 24 threads running Ubuntu 20.04 LTS. Encryption and computation operations are threaded, while decryption runs on single core. We implement our model using the E3 frameworkAs mentioned in “We report the encryption, decryption, and computation time required for private cancer prediction in Fig. We compare the performance of our model  with the LR with standard matrix multiplication while keeping the features, test dataset, and the plaintext methodology same. To study the scaling effect of computational costs with the increment of the number of individuals in the test data (i.e. the number of patients with sensitive data), we generate synthetic data for upto 8192 individuals to represent new individuals since our original test data consists of only 543 individuals.We measure time for encryption, computation, decryption operations. We plot the timing results in a log-log graph in Fig. 26) and thus, can be reused for datasets requiring HE-based high-dimensional inference. The transferability of our private inference algorithm across applications is an added advantage.Healthcare models are difficult to port trivially across datasets (as discussed in “Current solutions for HE-based privacy preserving inference suffer from impractical overheads; which are further aggravated when dealing with high-dimensional genomic data. In this work we develop a solution for privacy preserving cancer inference on genomic data. We first leverage biological intuition to structure the mutation data and reduce the dimensionality to a practicable limit. For our privacy preserving ML model, we propose a matrix multiplication algorithm to implement logistic regression model, optimized for high throughput and low latency. Our analysis on a real-world genomic dataset shows that our solution achieves cancer prediction MAUC of 0.98 on test dataset and can be computed on encrypted genomic data at Supplementary Information.