Variable number tandem repeats (VNTRs) account for significant genetic variation in many organisms. In humans, VNTRs have been implicated in both Mendelian and complex disorders, but are largely ignored by genomic pipelines due to the complexity of genotyping and the computational expense. We describe adVNTR-NN, a method that uses shallow neural networks to genotype a VNTR in 18 seconds on 55X whole genome data, while maintaining high accuracy. We use adVNTR-NN to genotype 10,264 VNTRs in 652 GTEx individuals. Associating VNTR length with gene expression in 46 tissues, we identify 163 “eVNTRs”. Of the 22 eVNTRs in blood where independent data is available, 21 (95%) are replicated in terms of significance and direction of association. 49% of the eVNTR loci show a strong and likely causal impact on the expression of genes and 80% have maximum effect size at least 0.3. The impacted genes are involved in diseases including Alzheimer’s, obesity and familial cancers, highlighting the importance of VNTRs for understanding the genetic basis of complex diseases. Variable number tandem repeats (VNTRs) are implicated in human diseases yet have been difficult to analyse computationally. Here, the authors describe a neural network method, adVNTR-NN, that allows rapid and accurate genotyping of VNTRs from large whole genome sequencing datasets. While multiple resources have been developed for genome-wide analysis of STRs, here we focus specifically on VNTRs, which have been largely missing from genome-wide studies due to technical challenges of genotyping and the computational expense.The human genome consists of millions of tandem repeats (TRs) of short nucleotide sequences. These are often termed as short tandem repeats (STRs) if the repeating unit is <6 bp, and variable number tandem repeats (VNTRs) otherwise. Together, they represent one of the largest sources of polymorphisms in humans3, untranslated regions (UTRs)4, and regulatory regions proximal to a gene6, the variation in length can have a significant functional impact. Not surprisingly, VNTRs have been implicated in a large number of Mendelian diseases that affect millions of people world-wide9. They also are known to modulate quantitative phenotypes in several other organisms10, and have shown pathogenic effects in other vertebrates including dogs11. VNTRs are also an important source of variations in bacteria and have commonly been used to study epidemiology and genetic diversity of Mycobacterium tuberculosis and Yersinia pestis13. They have influenced primate and human evolution through gene regulation and differentiation of great ape populations14. Recent studies have identified VNTRs that have expanded in the human lineage or are differentially spliced or expressed between human and chimpanzee brains15.We define VNTR genotyping in the narrower sense of determining VNTR length (number of repeating units). As VNTRs can be located in coding regions18. Studies have shown that causal variants for diseases often overlap with cis-eQTL variants in the affected tissue19. Therefore, we focus on the specific application of identifying expression-mediating VNTRs (“eVNTRs”), or VNTRs located in regulatory regions whose length is correlated with the expression of a proximal gene. Examples of “eVNTRs” include a VNTR in the 5′ UTR of AS3MT, which is strongly associated with AS3MT gene expression and lies in a schizophrenia associated locus4 and a 12-mer expansion upstream of the cystatin B (CSTB) gene is associated with gene expression and with progressive myoclonus epilepsy20.Single nucleotide polymorphisms (SNPs) that associate with gene expression, often referred to as expression quantitative trait loci (eQTLs), are molecular intermediates that drive disease and variation in complex traits1. While tools for genotyping STRs have been developed22, they generally do not detect or genotype VNTRs, which have non-identical and larger repeat units. Recently, a few specialized computational methods  have been published to tackle the problem of genotyping VNTRs from sequence data24. However, these methods are too computationally intensive to scale to functional studies with hundreds of individuals and 104 VNTR loci (Results). There have also been recent, successful efforts to genotype VNTRs using long-read sequencing technologies such as Pacific Biosciences (PacBio) and Oxford Nanopore Technologies (ONT)26. While these methods (which include adVNTR) are quite accurate, the technologies are currently too expensive for population scale sequencing.Despite their importance, the full extent of VNTRs in mediating Mendelian and complex phenotypes is not known due to genotyping challenges. Traditionally, VNTR genotyping used capillary electrophoresis which did not scale to large cohorts. Despite the advent of sequence based genotyping, repetitive sequences continue to be challenging for genomic analysis. For example, “stutter errors” due to polymerase slippage during PCR amplification change VNTR length and reduce genotyping accuracy28. While the standard whole genome sequencing (WGS) frameworks often ignore repetitive regions, there is some progress towards “harder” variant classes such as eSTRs31 and “eSVs”28. Therefore, “missing heritability”—the gap between estimates of heritability, measured for example by twin studies33, and phenotypic variation explained by genomic variation—remains a limitation for eQTL studies34. It has been speculated that the inclusion of TRs in association analyses may reduce this heritability gap35.For these reasons, large-scale studies of VNTRs and their association with gene expression have been limited when compared to other sources of human variation such as SNPs and CNVsHere, we describe adVNTR-NN, a method that uses shallow neural networks for fast read recruitment followed by sensitive Hidden Markov Models (HMMs) for genotyping. We test the speed and accuracy of adVNTR-NN on extensive simulations to demonstrate accuracy. We use adVNTR-NN to genotype over 10,000 VNTRs in 652 individuals from the GTEx project and associate VNTR length with gene expression in 46 tissues. We additionally validate eVNTRs in blood tissues in 903 samples from an Icelandic cohort and 462 samples from the 1000 genome project with gene expression data (Geuvadis cohort). We compare the strength of genic eVNTR association against proximal SNPs and identified many of the eVNTRs as causal. Our results suggest that it is computationally feasible to genotype VNTRs accurately in thousands of individuals, and multiple eVNTRs are likely to causally impact the expression of key genes involved in common and complex diseases.36, 502,491 VNTRs were identified that contained at least two repeating units in the GRCh38 human assembly and had repeat unit lengths between 6 and 100 bp. Over 80% of these had total length <140 bp , the time to genotype n = 10 K VNTRs was about 631 h per individual. In developing adVNTR-NN, we first made significant improvements to pre-processing time. Next, we deployed a second filtering step with a two-layer feedforward network trained separately for each VNTR that accepted the k-mer composition for each read and filtered it specifically for that VNTR  and recall TP/(TP + FN).While adVNTR was highly accurate by itself, its final accuracy depended upon reads filtered for genotyping, and specifically on false negatives—reads that were incorrectly removed by a filter. Formally, a read sampled from a VNTR was considered to be true positive (TP) if it passed the filter for that VNTR, and false negative (FN) otherwise. False positives (FP)—reads that passed the filter despite not being from the VNTR locus—were a lesser concern because they would eventually be discarded by the HMM for not aligning well to the model. However, high false-positives increase the running time. To account for this, we measured the trade-off between efficiency (1 − (TP + FP)/39. These tools are designed for fast mapping of reads and are accurate for most of the genome, but are not specifically designed for VNTR mapping genotyping (could have high FN). As a second comparison, we used adVNTR23, which has high recall (low FN) for VNTR mapping. We used a mix of real and simulated reads to test performance (Methods).For comparisons with alternative filters, we used Bowtie2 as a representative read-mapping toolr), Bowtie2 was the most efficient retaining only 0.9 in 106 reads for further processing for 90% of the VNTRs. Both adVNTR and adVNTR-NN were slightly less efficient retaining about 1.2 reads per million for 90% of the VNTRs. However, they had significantly better recall. adVNTR-NN filtered reads with at least 90% recall for 99% of the target VNTR loci /23 using trio-consistency, comparison to long reads, and other methods. Similarly, we used a mix of WGS data and simulated reads (Methods) to measure adVNTR-NN accuracy.We had previously measured adVNTR genotyping accuracyThe accuracy of VNTR genotyping using short reads depends critically on total allele length and length of repeat unit itself. adVNTR was 90% accurate on reads up to 90 bp in length, but its accuracy dropped subsequently . We observed 98.4% consistency in the calls obtained by adVNTR-NN. The inconsistent alleles had longer length (median 90 bp) in contrast to the length of the consistent alleles . The GTEx project provided WGS for 652 individuals as well as RNA-seq for each of these individuals from 46 tissue types including whole-blood. A majority (86.0%) of the donors were of European origin; another 11.5% were African American and the remaining were Asian and American Indian. For validation, we used a second cohort of 903 Icelandic individuals42 with associated whole blood RNA expression data and WGS. We also chose a smaller, third cohort from the Geuvadis27 project which provided gene-expression data in lymphoblastoid cell-lines for 462 samples, where the WGS for the samples was available from the 1000 genomes project41. The Geuvadis cohort was dominated by individuals of European ancestry (80.7% of cohort). Most of the remaining (19.3%) were of African ancestry. Due to the match of tissue type and ethnicity, the Icelandic and Geuvadis whole blood data were used for validation of methods for identifying eVNTRs discovered from the GTEx project.To identify expression-mediating VNTR loci (eVNTRs), we primarily used data from the GTEx projectWe genotyped 10,264 VNTR loci in all 652 samples from GTEx to study the role of VNTRs in mediating gene expression of proximal genes. As expected, the most frequent allele matched the reference allele in 96.8% of the cases  followed by UTRs (51%) and coding exons (47%) Fig. a. Each iTo perform association analysis, we excluded 1817  VNTRs that were monomorphic, 1445 (14.1%) VNTRs that violated Hardy–Weinberg equilibrium constraints, and 4330 (42.2%) VNTRs that had minor allele frequency (MAF) <1% after removing individuals in the GTEx cohort with no expression data for the specific gene (Methods). We investigated VNTRs that violated HWE. Similar to trio-inconsistent VNTRs but distinct from all VNTRs, these VNTRs were longer, had long common alleles .The filtering resulted in a set of 2672 VNTRs (26%) available for association analysis. We used linear regression to measure the strength of association between average VNTR length of the two haplotypes, and adjusted gene expression level of the closest gene , we used the Benjamini–Hochberg procedure to identify a tissue-specific 5% FDR cutoff , consistent with their known roles in core-promoters44.Not surprisingly, a larger fraction  eVNTRs were significant in at least half (23) of the tissues tested  showed significance at a similar level and same direction of effect in Icelanders, highlighting the strong reproducibility of the associations. The Geuvadis data were acquired for a smaller cohort compared to the Icelandic data and measured expression in lymphoblastoid cells–transformed B cells, which are a component of whole blood tissue. Therefore, we recomputed 5% FDR cut-offs using the Benjamini–Hochberg method on 100 permuted samples. Despite the caveats, 12 of the eVNTRs were replicated. Combined, 91% (20/22) of eVNTRs could be replicated in an independent cohort where data was available. We also tested for correlation of effect sizes between the Icelandic and GTEx data and found strong correlation . A similarly strong correlation was observed between the Geuvadis cohort and GTEx . In all cases, the direction of effect was also maintained.Twenty-three of the 163 unique eVNTRs showed significant association in whole blood  with SNPs associating with schizophrenia4.Only a small number of examples have been reported where VNTR repeat unit counts have a causative on gene expressionr1); and (b) using the tool CAVIAR47 to measure the causality of association (r2) (Methods). Remarkably, the two rankings were very similar with mean discrepancy 2∣r1 − r2∣/(r1 + r2) = 2.3 × 10−3 across the 163 eVNTRs. We used the harmonic mean To investigate causality, we ranked each eVNTR against all SNPs within 100 kbp by (a) comparing the relative significance of association with gene expression  is a precursor protein for many peptide hormones with multiple roles including regulation of appetite and satiety48. Hypermethylation of POMC (and reduced expression) in peripheral blood cells and melanocyte-stimulating hormone positive neurons was strongly associated with obesity and body mass index49. Surprisingly, POMC over-expression also predisposed lean rats into diet-induced obesity50. Our analysis identified a VNTR in the coding region of the POMC gene as the causal variant governing expression levels in 15 tissues, including adipose and nerve tissues. The 6R allele had 1.8-fold higher expression in blood and nerve cells  had 2-fold higher median expression relative to RU3 , TBC1D7 (lung cancer59), ZNF490 , MSH3 (myotonic dystrophy61). We note that the VNTR in MSH3 is a 9 bp repeat that is distinct from the trinucleotide expansion mediated by MSH362. Taken together, our results suggest that VNTRs mediate the expression of key genes.The 5 individuals in the future.VNTRs are the “hidden polymorphisms.” Despite high mutation rates and known examples of function modifications, VNTR genotyping is not a component of Mendelian or GWAS pipelines. This is primarily due to technical challenges. Here, we use a combination of fast filtering followed by a HMM-based genotyping to accurately determine VNTR genotypes. Our method, adVNTR-NN, can genotype 10K VNTRs for an individual in 50 cpu hours with high accuracy. We used adVNTR-NN to genotype close to 2000 human samples at 10K loci. The use of neural networks as a filtering strategy is novel, and we believe that further improvements could lead to another order of magnitude reduction in compute time, making it practical to genotype ≥10Some VNTRs have complex multi-repeat structure making it difficult to map reads and count the repeating units. However, unlike other VNTR genotyping methods, our method customizes the genotyping for each VNTR. Future research will focus on improving the genotyping for the hard cases, possibly by building HMMs with separate profiles for each distinct repeating unit, as well as the use of long-reads to improve anchoring to the correct locations. We pursue a targeted genotyping approach which has the disadvantage of not being able to discover new VNTRs, and we rely on other methods for the initial discovery of VNTRs. However, we note that the discovery is a one-time process while genotyping must be repeated for each cohort, and therefore, it makes sense to separate the two problems. For maximum sensitivity, discovery of VNTRs could be performed on a new cohort prior to genotyping. Even if the reference contained 0 copies, knowledge of the repeat pattern and location would allow us to genotype donors with multiple repeat units.The relatively large number of VNTRs violating HWE suggests that genotyping accuracy could be improved by filtering problematic VNTRs. We are developing strategies to filter VNTRs based on similarity to other VNTRs, matching sequence of repeat units and flanking regions, and other tests for long alleles. As more data is collected, we will be able to assess the accuracy of these strategies.adVNTR-NN can be used for association of a VNTR genotype chosen from a large collection of target VNTRs, against categorical or quantitative phenotypes. We used it to identify eVNTRs, where VNTR allele changes associated strongly with gene expression. It is possible that the largest allele or some other regrouping has the strongest effect for some VNTRs, and this idea may be used to strengthen the eVNTR association. However, we did not have a consistent strategy for grouping the VNTRs and therefore did not try this approach for the VNTRs in our study. Nevertheless, for individual VNTRs that are on the borderline for significance, this approach could be tried prior to functional tests.We found that VNTRs were strongly associated with the expression of proximal genes with over 6.1% of the 2672 VNTRs tested showing genome wide significant association. Nearly half of the eVNTR loci were more significant compared to neighboring SNPs. While the high fraction of causal eVNTRs can partly be explained by the choice of “genic” VNTRs for testing, we believe that non-genic regions will identify additional causal eVNTRs. In testing for causality, it would be best to compare against all other forms of variation including SNPs , structural variations, and other STRs. However, there is significant complexity in calling these variants. For example many STRs and even VNTRs are mis-annotated as structural variants. We will address these concerns in future work. In summary, ongoing technical innovations in speed and accuracy of VNTR genotyping are likely to improve our understanding of human genetic variation, and provide novel insights into the function and regulation of key genes and complex phenotypes.A(r) denote the HMM genotyping time using r reads. The goal of filtering is to reduce the number of reads supplied to each VNTR HMM. Any filter is characterized by three parameters:Let P(r) denote the running time of the filter for r reads for each VNTR locus;run-time: Let fk denote the fraction of reads that were retained for any VNTR. The efficiency is defined as 1 − fk so that high efficiency implies only a small fraction being retained by the filter.efficiency: Let sensitivity/recall: The fraction of true VNTR overlapping reads that were accepted for each VNTR.r unmapped reads and among the mapped reads, an average of n VNTRs is given by:A(r) = 0.32r seconds per VNTR. The keyword match filter for adVNTR achieved fk = 7.7 × 10−5. For a 55X coverage WGS with r = 4.2 × 106 reads, P(r) = 111.22(s), n = 10K VNTRs is about 631 h per individual.Consider a dataset with V, and each read R, consider a binary classification function f: V × R → {0, 1}, where f = 1 if and only if read R maps to locus V. For each read and each of N loci V1, …, VN, the neural recruitment method computes independent classification functions fi. Note that a read can be assigned to multiple VNTR loci, or to none. As an initial step toward this task, we perform a fast string matching based on prefix tree (trie) to assign each read to the VNTR loci that share an exact match with the read. For an efficient matching, we generate a separate aho-corasick trie63 using every k-mer in VNTR loci as dictionary X. A trie is a rooted tree where each edge is labeled with a symbol and the string concatenation of the edge symbols on the path from the root to a leaf gives a unique word (k-mer) X. We label each leaf with a set of T VNTRs that contain corresponding k-mer. On the other hand, the string concatenation of the edge symbols from the root to a middle node gives a unique substring of X, called the string represented by the node. We add extra internal edges called failure edges to other branches of the trie that share a common prefix which allow fast transitions between failed string matches without the need for backtracking63. Testing whether a query q has an exact match in the trie can be done in O(∣q∣) and we require additional O(∣T∣) time to assign read q to all T VNTR loci that share the keyword. The overall complexity of this algorithm is linear based in the length of original dictionary (VNTRs in the database) to build the trie and recover matches plus the length of queries (sequencing reads). Hence, after construction of the trie, the running time is proportional to just reading in the sequences.For each VNTR locus fi, using a k-mer based embedding to encode DNA strings. Specifically, we use a DNA string w of length k, consider an bijection ϕ that maps w to a unique number in . Each read R can be defined by a collection of overlapping k-mers. We map read R to a unique vector vR[i] = 1 if and only if ϕ−1(i) ∈ R. Details of the neural network architecture and hyper-parameters are presented below.To further reduce the set of reads assigned to each VNTR, we use a 2-layer feedforward neural network to compute v denote the mapping of a read. We use a shallow architecture with an input layer used to present v to the network. We add two layers of fully connected nodes as the hidden layers, with each node being a ReLU function. In the output layer, there are two nodes zero and one which specify that whether read should be classified as true (containing VNTR) or false , mean squared logarithmic error (MSLE), and mean absolute error (MAE), as well as three binary classification loss functions hinge, squared hinge, and binary cross-entropy. We compared the validation performance of our models for these six different loss functions. Each distribution in Supplementary Fig. SN(r) ≃ 0.03r seconds for r reads, greatly increasing filtering efficiency  with Illumina HiSeq 2500 error profile. For each target locus, we modified the number of the repeats to be ±3 of the original count in the reference with setting 1 as minimum number of repeats, and simulated reads from those regions. For each locus, we assigned labels to reads as being true reads or not, based on exact location. We divided the original set of reads into three parts: 70% for training, 10% for validation, and 20% for testing. We trained all neural network models using the training and validation sets, and reported performance on the test dataset.We used ART68. For each sequence in the dataset, we replaced its nucleotides with a random one with probability rm. We set rm = 10−5, the novel base substitution mutation rate within VNTRs69. This method of dataset augmentation helps include “mutated” k-mers in the embedding of reads, making the method more robust.To augment the data, we added random single nucleotide variations in the genome sequences of the dataset before simulating the sequencing reads67 to generate heterozygous samples by simulating 15X coverage reads from each modified haplotype which contained a non-reference allele and combined those with 15X reads that were simulated from reference. The non-reference allele for each VNTR was chosen to be in the range , where c is the reference count. Together, this provided six diploid simulated datasets for each locus, at 30X coverage.To test and compare genotyping accuracy against VNTRseek (v1.10.0), we started with a random selection of 10,000 target VNTR loci (<140 bp) and filtered them out if a VNTR locus was marked as indistinguishable in VNTRseek. As a result, 9638 target VNTRs remained. We used ART40 (v2.4.5) for, we selected VNTR loci with repeat unit length ≤20 bp. A total of 6508 target VNTRs remained. Following the method for VNTRseek comparisons, we used ART67 to generate a homozygous sample and six heterozygous samples by simulating 30X paired-end reads with Illumina HiSeq 2500 error profile.Similarly, to test and compare genotyping accuracy against GangSTRWe measured running time of adVNTR-NN and VNTRseek by running them with default parameters on a single core of Intel Xeon CPU E5-2643 v2 3.50GHz CPU. To measure the accuracy of genotyping, we ran adVNTR-NN and VNTRseek on diploid simulated data of heterozygous VNTRs and measured the number of correct calls divided by total number of VNTR loci.https://www.ncbi.nlm.nih.gov/projects/gap/cgi-bin/study.cgi?study_id=phs000424.v8.p2]). Specifically, we accessed CRAM files containing read alignments to the GRCh38 reference genome through cloud-hosted SRA data using fusera v1.0 and downloaded VCF files containing SNP genotype calls from dbGaP.We accessed 30X Illumina WGS data from the GTEx cohort  through dbGaP  cloud, which allowed us to do the computation in parallel for different samples.As genotyping VNTRs remains computationally expensive, we focused on the smaller set of VNTRs located within coding, untranslated, or promoter regions of genes, which are most likely to be involved in regulation. We identified VNTRs in coding exons and UTRs by intersecting VNTR coordinates with refseq gene coordinates downloaded from UCSC Table Browser. To identify VNTRs that appear within promoter regions, we considered 500 bp upstream of the TSS of genes as the promoter regions. Overall, this procedure identified 13,081 VNTRs, of which 10,262 were within the size range for short-read genotyping  were eliminated. We further removed VNTRs that were monomorphic  in the entire GTEx cohort or had MAF lower than 1% among the individuals with expression data in every tissue. We used the resulting 2672 VNTRs for subsequent analysis  to observe representation of each VNTR in the reference. We also searched for VNTRs with multiple observed alleles to estimate a rate of polymorphism for VNTRs and find how common each allele was. To call a VNTR polymorphic, we set the MAF at 5% and any variation below that frequency was discarded. In addition, we identified the amount of base-pair difference that they make in genome of each individual by comparing the copy number difference of VNTRs between reference and the sample and multiplied that by the pattern length of each locus. We computed how many loci on average differed between an individual and reference by combining all non-reference calls in at least one haplotype from all individuals and dividing it by all called variants. VNTRs whose allele frequencies did not meet the expected percentage of homozygous versus heterozygous calls under Hardy–Weinberg equilibrium (https://www.ncbi.nlm.nih.gov/projects/gap/cgi-bin/study.cgi?study_id=phs000424.v7.p2]) and limited analysis to 46 tissues which had data for at least 100 individuals. “Non-expressed genes”—genes with median RPKM level zero—in each tissue were removed from analysis. For the remaining genes, we quantile-normalized RPKM values of each tissue to a normal distribution. We analyzed VNTR-gene pairs for each VNTR and its closest gene based on refseq annotations in each of the 46 tissues.We obtained processed RNA-expression data  from 54 tissues from dbGaP (phs000424.v7.p2 [https://www.ncbi.nlm.nih.gov/projects/gap/cgi-bin/study.cgi?study_id=phs000424.v7.p2]) and filtered biallelic SNP sites MAF >0.05 using plink70. To correct for non-genetic factors such as technical variations in measuring RNA expression levels , we applied PEER factor correction and used the top 15 factors43. We removed the effect of covariates by regressing them out from the RNA expression matrix of each tissue and subtracting their factor contributions and used the residuals for all eQTL association analyses.Before the analysis of the association of VNTR genotypes and gene expression levels, we adjusted gene expression levels for each tissue in order to control for covariates of sex, population structure, and technical variations in measuring expression. For population structure, we used the top ten PCs from a principal components analysis (PCA) on the matrix of SNP genotypes to provide a correction for population structure. To generate the SNP genotype matrix, we used the VCF files for GTEx cohort  by subtracting the mean and dividing by the standard deviation of the expression values for that cohort. For a gene-VNTR pair v, let yiv denote the normalized expression value of gene in v for individual i and xiv denote the genotype of the VNTR in v for individual i. Then,ik denotes the strength of the k-th principal component, and Rik the value of the k-th PEER factor. We performed the association test for each VNTR-gene pair separately for each tissue type using Python statsmodels linear regression, ordinary least squares (OLS)71, and computed a nominal p value of the strength of association for each VNTR-gene pair using two-sided Fisher’s exact test.We normalized the individual raw gene expression values to −3 to 3.8 × 10−5  and LD structure between every pair of variantsFurther information on research design is available in the Supplementary InformationDescription of Additional Supplementary FilesSupplementary Data 1Supplementary Data 2Supplementary Data 3Supplementary Data 4Supplementary Data 5Supplementary Software 1Supplementary Software 2Reporting Summary


We exemplify the effects of minimizing technical noise on several datasets, across various sequencing assays: coding, non-coding RNAs and interactions, at bulk and single-cell level. An immediate consequence of filtering out noise is the convergence of predictions  across different approaches.High-throughput sequencing enables an unprecedented resolution in transcript quantification, at the cost of magnifying the impact of technical noise. The consistent reduction of random background noise to capture functionally meaningful biological signals is still challenging. Intrinsic sequencing variability introducing low-level expression variations can obscure patterns in downstream analyses. We introduce  High-throughput sequencing (HTS) became a new standard in most life science studies yielding unprecedented insights into the complexity of biological processes. The increase in sequencing depth and number of samples, across both bulk and single cell experiments, facilitated a greater diversity in biological questions , at the While technologies may exhibit different technical biases, the sequencing bias across an experiment was expected to be uniform. This expectation was based on the assumption that sequencing reads would uniformly cover the expressed transcripts, with the algebraic sum of reads from each gene being proportional to the expression of that gene . HoweverThe presence of noise in HTS data has been widely acknowledged, and there have been several attempts to understand and quantify it. A recent study  presenteOn the computational side, several imputation and denoizing algorithms have been proposed, e.g. a machine learning (ML) based deep count autoencoder . Other tExisting approaches for calling DE genes mitigate to various extents the presence of noise, however these are not designed to identify and assess the impact of genes showing random, low-level variation. As a result, some of these are detected by the DE analyses, biasing the biological interpretation of the results. In addition, the choice of tools used for pre-processing steps may influence the relative transcript expression estimation accuracy . These anoisyR, a denoizing pipeline to quantify and exclude technical noise from downstream analyses, in a robust and data-driven way. The approach underlines consistency of signal over a user-defined threshold. noisyR is applicable on either the original, un-normalized count matrix, or alignment data (BAM format). Noise is quantified based on the correlation of expression across subsets of genes for the former, or distribution of signal across the transcripts for the latter, in different samples/replicates and across all gene abundances (Methods). We illustrate the approach on bulk and single cell RNA-seq datasets and highlight the impact of the noise removal on refining the biological interpretation of results.We developed noisyR was generated by Yang et al.  and from Wallach et al. . For both datasets, the reads were aligned to mature and hairpin miRNAs, downloaded from miRBase  dataset, consisting of three biological replicates (GSE113958) presented in Thody et al. .noisyR was generated by Cuomo et al (study of stem cell differentiation) (The single-cell mRNA-seq dataset used to illustrate tiation) . The datA. thaliana , Mus_musculus.GRCm38.98 (Ensembl version 98) and thaliana .M. musculus exon annotations obtained from the Ensembl database (genome assembly GRCm38.p6). Additional quality checks included density plots, (comparable distributions are a necessary but not sufficient condition for comparability), MA plots for the sufficiency check , incremental dendrograms and PCA plots to evaluate the similarity of distributions  and summarized with multiQC (version 1.9) . Alignmeibutions .2(FC)| > 1 and adjusted P-value < 0.05 (Benjamini–Hochberg multiple testing correction). The enrichment analysis was performed using g:profiler   , againstn 0.2.0)  and reacn 0.2.0)  pathway P-value <0.05 (Benjamini–Hochberg multiple testing correction) and |log2(FC)| > 1 were considered before and after noise correction.To assess the effect of noise correction across the multiple options of mRNA quantification, the sequencing reads were aligned to the reference genome using Bowtie2 (version 2.4.2)  and HISATo assess the implications of the noise filter on downstream biological interpretations, we used the bulk and single-cell datasets as inputs for various gene regulatory network (GRN) inference tools and compared the results for filtered and unfiltered inputs. For this purpose, we selected several gene subsets, ranging in size from 49 to 996 genes for the bulk dataset and from 57 to 246 genes for the single-cell dataset, based on enrichment analyses performed on the DE genes according to their inclusion in annotated pathways .https://github.com/drostlab/network-inference-toolbox) and then assembled them into a custom pipeline (https://github.com/drostlab/network-inference-pipeline).We chose a subset of the GRN inference tools benchmarked by BEELINE : GENIE3 edgynode package . The edge weights and node degree distributions for all genes across the selected subsets are then visualized.This pipeline extracts the subsets of genes corresponding to selected pathways and uses them as inputs for the GRN inference tools. The results are rescaled, binarized and compared using the edgynode function network_benchmark_noise_filtering and was visualized using plot_network_benchmark_noise_filtering. For this purpose, the inferred networks were converted to a binary format (presence/absence of an edge) using the overall median edge weight per network as a threshold. In network_benchmark_noise_filtering four different types of matrices are used as input: a weighted adjacency matrix returned by a network inference tool where (i) no noise filter and no quantile normalization  was performed (denoted in the figures as −F −N), (ii) a noise filtering but no quantile normalization was performed (+F −N), (iii) no noise filtering but a quantile normalization was performed (−F +N) and (iv) both, noise-filtering and quantile normalization were performed (+F +N).In detail, the similarity assessment of network topologies was performed using the edgynode, with the same parameter configurations to both bulk  and single-cell  data to retrieve comparable results for direct comparisons. Computationally reproducible analysis scripts to perform all inference steps, data transformations, and visualizations, including the ones used in this study can be found at https://github.com/drostlab/network-inference-pipeline.In a pairwise all versus all comparison, for each gene, the Hamming distance over the binary edge weight vectors was computed using the hamming.distance function from the R package e1071 v1.7-4, yielding a distribution of distances, which captures how many genes gained or lost their connection with other genes. A Kruskal–Wallis Rank Sum Test was performed using the stats::kruskal.test function in R to assess whether comparisons of Hamming distance distributions between original, noise-filtered, and normalized combinations were statistically significantly different. Furthermore, visualizing these distributions across comparisons and for all network inference tools facilitated an evaluation of the overall change of network topologies driven by the network inference tool or the normalization/noise-filtering that was applied. These visualizations were then used to assess the impact and robustness of our noise-filter on the interpretation of biological network topologies. We applied the pipeline, including A. thaliana sRNA samples were assessed using multiQC version 1.9  to bothsion 1.9  and TEs sion 1.9 .M. musculus sRNA samples were processed in a similar way as the plant samples and subsampled without replacement to 3.5M sequences  were QCed (multiQC version 1.9)   and the mples GSE3958 wereion 1.9) ) using Sion 1.9) ), with dNA label .For the single cell SmartSeq2 data, the cellranger software version 3.0  was usedTwo approaches were implemented for the identification of noise. (i) The “count matrix approach” is a simple, fast way to obtain a threshold utilizing solely the un-normalized count matrix (m genes x n samples). (ii) The “transcript approach” is more refined, as it takes into account the distribution of signal across the transcript obtained by summarizing the aligned reads from the BAM alignment files. For both approaches, a variety of correlation and distance measures are used to assess the stability of signal across samples . Most reFor each sample in the count matrix, the genes are sorted, in descending order, by abundance. A sliding window approach is used to scan the sorted genes (genes with similar abundances are grouped into “windows”). The window length is a hyper-parameter that can be user-defined or a single value inferred from the data using a Jensen–Shannon entropy based approach . The sliUsing the transcript coordinates of the aligned reads as input, the expression profile for each individual transcript was built as an algebraic point sum of the abundances of reads incident to any given position ; if the noisyR pipeline, for both the count-matrix and transcript-based approaches, we used a server with 32 cores, specifying 16 and 32 cores as noisyR parameters, respectively. The specification of the server used for benchmarking is: Kernel: Linux 4.19.0-6-amd64 #1 SMP Debian 4.19.67-2+deb10u2 (2019-11-11), Hardware: Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz; RAM: 756GBTo benchmark the The benchmarking focused on several covariates: the number of samples from two to 16, the number of exons used for the transcript approach and the sequencing depth/number of mapped reads for the transcript approach.noisyR on mRNA-seq and smallRNA-seq data. First, we illustrated the advantages of using the pipeline on a subset of mRNA-seq samples from a 2019 study by Yang et al.  P-values marginally below the DE threshold ; the number of retained genes were 19.7k and 15.6k for the counts and transcript approaches, respectively. As a DE pre-processing step, the averaged noise threshold was added to all entries in the count matrix (Methods). The effect of the noise removal is illustrated by the narrower distribution in the MA plots Figure . Next, wd Figure  versus Dd Figure . The numd Figure . Thus, anoisyR transcript approach was also applied on two small RNA (sRNA) datasets, from plants  and animals (M. musculus), respectively. In contrast to the mRNAseq data, sRNAs samples had different correlation vs abundance distributions. Overall low abundance sRNA transcripts/loci contained more noisy entries  sequencing output focusing on a subset of samples from the dataset presented by Cuomo and colleagues  and remained stable over a wide range of abundances , is to infer the complex interactions of genes that lead to biological functions, the cause . Understanding these interactions between genes and the corresponding regulatory elements  dataset . To capedgynode network analytics package  on bulk RNA-seq datasets using non-noise-filtered original, non-noise-filtered normalized, and noise-filtered normalized count matrices. Bulk RNAseq data has been widely used despite its well-known effect to dilute expression signals of individual cells or tissue types. However, in the context of technical noise, the averaging across cells and tissues may buffer the noise effect on general patterns while reducing the possibility to detect weak, but biologically meaningful, expression signals , associated with annotated pathways; we quantify the impact of random noise in altering network topologies and subsequent biological interpretations. To achieve this, we run our Network Inference Pipeline (NIP) and n factor  or transn factor ).edgynode and rescaled to the range  to allow comparisons across inference tools.Using the Yang dataset  in four Next, all rescaled weight matrices  were conIn addition,  shows a Together, these results suggest that across network inference tools noise-filtering has refining effects on the inferred network topologies in original or normalized data, further illustrating the advantages of noise-filtering to magnify biological signals by reducing technical noise .noisyR package is available on CRAN (https://CRAN.R-project.org/package=noisyr) and comprises an end-to-end pipeline for quantifying and removing technical noise from HTS datasets  programming language and is actively maintained on https://github.com/Core-Bioinformatics/noisyR. Also available is a detailed vignette, illustrating the functionality of the pipeline: https://core-bioinformatics.github.io/noisyR/articles/vignette_noisyr_counts.html.The s Figure . The thrFor the sample-similarity calculation, two approaches are available. The count matrix approach uses the original, un-normalized count matrix, as provided after alignment and feature quantification; each sample is processed individually, only the relative expressions across samples are compared. Relying on the hypothesis that the majority of genes are not DE, most of the evaluations are expected to point towards a high similarity across samples. Choosing from a collection of >45 similarity metrics . A slidiThe noise quantification step uses the abundance-correlation (or other similarity measure) relation calculated in step (i) to determine the noise threshold, representing the abundance level below which the gene expression is considered noisy, e.g. if a correlation threshold is used as input then the corresponding abundance from a (smoothed) abundance-correlation line plot is selected as the noise threshold for each sample. The shape of the distribution can vary across experiments; we provide functionality for different thresholds and recommend the choice of the one that results in the lowest variance in the noise thresholds across samples. Options for smoothing, or summarizing the observations in a box plot and selecting the minimum abundance for which the interquartile range (or median) is consistently above the correlation threshold are also available. Depending on the number of observations, we recommend using the smoothing with the count matrix approach, and the boxplot representation with the transcript option. A detailed overview of benchmarking analyses is presented in The third step uses the noise threshold calculated in step (ii) to remove noise from the count matrix (and/or BAM file). The count matrix can be calculated by exon or by gene; if the transcript approach is used, the exon approach is employed. Genes/exons whose expression is below the noise thresholds for every sample are removed from the count matrix. The average noise threshold is calculated and added to every entry in the count matrix. This ensures that the fold-changes observed by downstream analyses are not biased by low expression, while still preserving the structure and relative expression levels in the data. If downstream analysis does not involve the count matrix, the thresholds obtained in step (ii) can be used to inform further processing and potential exclusion of some genes/exons from the analysis.noisyR.A direct side-by-side comparison with other methods is challenging since some are based on wet-lab validations   and the noisyR is the calculation of the signal/noise threshold per sample, then used to remove all genes with counts less than the threshold. The second part of the filtering is the averaging of thresholds across samples and adding the resulting value to the whole matrix  and 100. The results are presented in noisyR-thresholds were in line with the results of using comparable, fixed noise thresholds. In addition, we observed a local minimum of the specific differences of edgeR and DESeq2, achieved using the noisyR approach. The larger intersection between the two methods also led to more enriched terms found for the noisyR-corrected data.In addition, a pivotal characteristic of n offset ). This anoisyR hyperparameters can be used to finely tune the identification of the signal/noise thresholds. To optimize the noise filtering procedure and dampen the stochastically induced differences between samples (e.g. derived from variation in sequencing depth or sample read-complexity) the noise removal step is performed by adding the average of the signal/noise thresholds across samples, on the raw count matrix. Nevertheless, comparable thresholds across the dataset are essential for a meaningful filtering; we recommend the use of consistency and robustness checks throughout the pipeline to ensure that the input samples are comparable, coupled with the data-driven selection of threshold values for setting hyper-parameters. The option of user-defined values is available, however the selected values should be based on observations from the input dataset, rather than exclusively following default recommendations. Next, we discuss in detail the options available for selecting the hyperparameters for a more adaptive noise-filtering based on the structure of the input data.noisyR, tackles this task from a different direction; it relies on optimizing the window length using an entropy-based approach with the Jensen-Shannon divergence to assess the stability achieved as the window length is increased (t-test) for larger windows. The first point of stability is selected as the optimal window length, as it provides the largest possible granularity while maintaining robustness. The results from this approach are also consistent with earlier, empirical findings when applied to the Yang dataset  of five using both counts and transcripts based approaches | > 1) detected by the two methods were compared for outputs produced using STAR  thousands of individual cells expressing thousands of genes each. This limit on the sequencing depth per cell underlines, yet again, the technical noise, whereby the quantification of low-abundance transcripts can be the result of either low biological expression or due to stochastic effects (likelihood) of read capturing. The requirement of an adaptive noise-filtering pipeline is fulfilled by edgynode and rescaled  and also shown for all three network inference tools  and comprises an end-to-end pipeline for quantifying and removing technical noise from HTS datasets. The package is written in the R (version 4.0.3) programming language and is actively maintained on https://github.com/Core-Bioinformatics/noisyR.The gkab433_Supplemental_FilesClick here for additional data file.


Artificial intelligence methods offer exciting new capabilities for the discovery of biological mechanisms from raw data because they are able to detect vastly more complex patterns of association that cannot be captured by classical statistical tests. Among these methods, deep neural networks are currently among the most advanced approaches and, in particular, convolutional neural networks (CNNs) have been shown to perform excellently for a variety of difficult tasks. Despite that applications of this type of networks to high-dimensional omics data and, most importantly, meaningful interpretation of the results returned from such models in a biomedical context remains an open problem. Here we present, an approach applying a CNN to nonimage data for feature selection. Our pipeline, DeepFeature, can both successfully transform omics data into a form that is optimal for fitting a CNN model and can also return sets of the most important genes used internally for computing predictions. Within the framework, the Snowfall compression algorithm is introduced to enable more elements in the fixed pixel framework, and region accumulation and element decoder is developed to find elements or genes from the class activation maps. In comparative tests for cancer type prediction task, DeepFeature simultaneously achieved superior predictive performance and better ability to discover key pathways and biological processes meaningful for this context. Capabilities offered by the proposed framework can enable the effective use of powerful deep learning methods to facilitate the discovery of causal mechanisms in high-dimensional biomedical data. Gene set selection facilitates interpretation of high-dimensional omics data by reducing it to components of greatest relevance and then allowing them to be linked to specific functional themes  and to extract meaningful clues for clarifying biological mechanisms. Other important applications include disease diagnostics, where the goal is to produce gene sets small enough to be used for diagnostic tests in clinical practice.Traditional machine learning (ML) algorithms  are a class of deep learning architectures that have shown promising results and gained widespread attention in all types image analysis . CNN takootprint . In a loootprint . The inpOne possible way CNNs can process nonimage samples is by first converting it to an image sample considering spatially coherent pixels in its local regions, i.e. the pixels close to each other share similar information sometimes with patterns. The arbitrary arrangement of pixel locations can induce an unfavorable impact on the feature extraction and classification performance of CNN architecture. Therefore, the order of neighboring pixels in an image utilized by CNN is no longer independent as they were in ML techniques . The DeeKaggle.com). Recently, a competition was organized by the Connectivity Map, a project within the Broad Institute of MIT and Harvard, the Laboratory for Innovation Science at Harvard (LISH) and the NIH Common Funds Library of Integrated Network-Based Cellular Signatures (LINCS), on the Kaggle.com platform (https://www.kaggle.com/c/lish-moa/overview). The title of the competition was Mechanisms of Action (MoA) predictions. The organizers posed a problem where it was required to develop an algorithm that can classify drugs based on their biological activities. A total of 4373 teams participated and submitted their models. The winning team of Peng et al. [https://www.kaggle.com/c/lish-moa/discussion/201510). The implementation and description of DeepInsight part by Peng et al. [https://www.kaggle.com/c/lish-moa/discussion/195378.The usability of DeepInsight based model has also been noticed in data science online platform  and therefore it is important to highlight some key differences between it and the proposed DeepFeature method. In most typical types of DEG analysis genes are processed individually and the comparisons are made between two or more conditions based on some variation of a linear model. The analysis would return all the genes found to be substantially different, though ability to meet the selection threshold is heavily influenced by the variance and magnitude of the expression, which may not necessarily align with the overall importance of these genes to the condition of interest. In contrast, deep learning classifier at the core of the DeepFeature model performs selection by considering all of the available genes simultaneously  and the priority is solely given to the overall predictive importance of the gene within the context of the selected set. Due to these differences in selection criteria DeepFeature can offer a rich, complementary perspective to that of the traditional methods. As illustrated by the analysis reported in this paper, DeepFeature-selected gene sets are both very different from more traditional approaches like lasso and analysis of variance (ANOVA), but also appeared to be better aligned with meaningful biological mechanisms and therefore consistently achieved higher enrichment for key pathways and functional groups.The hidden layers of CNNs can reveal complex mechanisms (such as pathways) for nonimage samples. The development of CNNs is inspired by biological processes to perform feature extraction from image patterns , 44, 45.The innovation and/or contributions of this paper are as follows. DeepFeature pipeline is introduced where feature selection can be performed for nonimage samples (or tabular data) via CNNs. Snowfall compression algorithm is developed to allow more elements of data in a fixed pixel frame. Region accumulation and element decoder (READ) are introduced to find genes or elements from the activation maps. In addition to delivering high-classification performance, DeepFeature also offers a powerful means for the identification of biologically relevant gene sets. When applied to the task of cancer classification, our DeepFeature approach was able to identify coherent sets with significant enrichment of genes in cancer-associated pathways from MSigDB and a gold-standard reference set. Further analysis of these results suggested biologically meaningful connections of potential interest to our understanding of the differences between major cancer types.The results are produced by following an overall procedure of DeepFeature . The modFirst, we compared the number of selected genes and classification performance for the TCGA cancer study identification task using different ML algorithms . Lasso gi = 1,2, … 10) with coefficients On the other hand, the logistic regression+RF method generates models for each cancer study Therefore, these algorithms, with the exception of logistic regression+RF, provide one subset of genes for all the categories. However, in some cases pairwise analysis is possible (e.g. by using post-hoc Tuckey’s test ). In sucs Seurat , pcaRedus Seurat , TSCAN [s Seurat  and SINCs Seurat , also prFor DeepFeature, three different visualization tools were used/developed to plot feature locations on a 2D-plane. These methods were (1) t-SNE, (2) t-SNE with Snowfall and (3) PHATE  , and 96.8% accuracy when using PHATE.For DeepFeature with t-SNE and Snowfall, four distances  were used which gave a subset of 5228 genes . The two resulting subsets were of 1914 genes and 962 genes  were applied in DeepFeature and the resulting gene subsets were examined for enrichment of genes in pathways of interest.DeepFeature is also capable of finding different gene subsets belonging to different cancer studies. As discussed above, applying different visualization tools resulted in different gene subsets of sizes 962, 1569, 1806 and 1914 genes for each of the 10 cancer studies. We applied enrichment analysis on these 962, 1569, 1806 and 1914 gene subsets. DeepFeature using t-SNE + Snowfall gave a 1806 gene subset and the number of genes for each cancer study ranged between 930 and 1450 see . DeepFeaTo compare these results with previously proposed gene sets suitable for identification of cancer types, an additional comparison was made with a ‘hallmark gene sets’ and ‘chemical and genetic perturbations’ collections from the MSigDB database  that aimTo explore possible underlying biological meaning behind these selected gene groupings, each set of cancer study-specific genes was evaluated for enrichment of specific pathways in Reactome database . This anThe performance evaluation of DeepFeature is gauged by both finding significantly enriched pathways within the selected subset of genes and classification accuracy on the independent test set. As expected, promising results were obtained when compared with the state-of-the-art ML techniques. The gene enrichment results for DeepFeature were reasonably sound compared with the benchmarked ML methods. The classification accuracy on the independent test set was ~98%, which is better than the ML technique. This shows that deep learning architectures have a possibility to provide solutions for biomarker discovery, genomic analysis for a variety of input samples ranging from RNA-seq to various omics data. In general, this method is suitable for applications where given data is in a nonimage form.The three visualization methods used in the DeepFeature pipeline were t-SNE, PHATE and t-SNE + Snowfall. Though the Snowfall algorithm adds distortion in the visualization of t-SNE, t-SNE + Snowfall performed better than t-SNE as measured by gene enrichment analysis. The distortion is introduced when the image is transformed from a Cartesian coordinate system to the fixed size pixel framework, as many features overlap, i.e. various features having the same pixel location. This overlapping reduces the chance of a feature to be selected. In such a case, the application of Snowfall is to find nearby pixel locations for features and expose more features to selection. If we could have very large pixel frame size where no overlaps occur while transforming from Cartesian coordinate system to pixel frame, then perhaps the application of Snowfall will not be useful as distortion is added by it while adjusting the features in the nearby pixel locations. Nonetheless, in a given fixed size pixel framework, the application of Snowfall for DeepFeature was found to be useful in finding more meaningful features or genes. On the other hand, although PHATE  has beenThe feature selection results indicated that DeepFeature was much better than alternative algorithms at recovering biologically meaningful groups of genes that were relevant to classes (phenotypes) of interest. A notable advantage of the method is its ability to narrow down the set of candidate genes even in cases where the differences are very substantial, like different cancer types. On the contrary, under such circumstances approaches like DGE return very large number of significant hits and therefore are not useful in sufficiently reducing the candidate list(s) to allow meaningful interpretation. Therefore, the algorithm is likely to have great utility for tasks like prioritization of diagnostical signatures and interpretation of complex multi-omics data.It is now generally accepted that in order to fully understand the dynamic quintessential complexity of cancer, it is essential to profile and collect data using a wide array of possible methods. As a result, a typical study commonly needs to consider wide array of possible data types, possibly including images, clinical records, as well as protein-coding and noncoding RNA expression profiles and full genome sequencing. Understanding mechanisms and discovery of clinically relevant subtypes frequently require appropriate aligning of these different types of data and, most often, subsequent use of ML algorithms to identify meaningful patterns. Among current generation of approaches, deep learning-based methods offer the greatest flexibility and can potentially allow fuller automation and more comprehensive integration of these different types. In principle, by combining different specialized encoder layers, any types of data can be converted into a set of inter-compatible embeddings and used within the same ML system. This innate flexibility is difficult to match using other methods and is particularly valuable for mining complex multimodal datasets.Good correspondence of identified gene sets with known gold standard cancer-associated pathways is particularly promising as a means of interpreting the deep learning results from a biological perspective. Despite many recent advances, deep learning still commonly has a reputation for generating high quality but ultimately ‘black box’ models, where discovering how an algorithm arrived at a particular conclusion is very challenging. However, given that structures of key cancer pathways are very well understood, their high degree of overlap with DeepFeature results could be crucial for correctly contextualizing the importance of individual genes and offer possible explanations for why they were selected by the algorithm.We obtained RNA-seq data from the TCGA project. To maintain large enough categories, only 10 cancer studies were considered, namely, TCGA-BRCA, TCGA-COAD, TCGA-HNSC, TCGA-KIRC, TCGA-LGG, TCGA-LUAD, TCGA-LUSC, TCGA-PRAD, TCGA-THCA and TCGA-UCEC. A total of 6280 HTSeq-FPKM-UQ expression files were downloaded using the GDC data transfer tool.From these, 64 files were removed for sharing submitter IDs, resulting in a final total of 6126 samples. The FPKM-UQ files contain expression for 60 483 genes. In this study, we only used the 19 086 genes classified as protein-coding genes by the HUGO Gene Nomenclature Committee (download date: 22 November 2019). Next, we will describe the DeepFeature model.An overall procedure for feature selection using CNN . The modThis section defines the proposed DeepFeature methodology. The constituents of the model are (1) image transformation by DeepInsight, (2) Snowfall compression to enable more elements in a pixel-frame, (3) SqueezeNet model of CNN architecture, (4) CAM model to find activation maps, (5) region accumulation to obtain overall activated regions for a category or dataset and (6) element decoder to decode genes from active regions. These steps are discussed below.DeepInsight transforms a nonimage sample to a well-organized image form by effectively arranging elements while considering neighborhood information. The feature extraction and classification tasks are done by CNN. DeepInsight integrates three steps: (1) element arrangement, (2) feature extraction and (3) classification. This approach of element arrangement can be useful in uncovering hidden mechanisms (e.g. pathways). In this way, the relative importance of features for assignment of samples to particular classes can be better understood. An input feature vector is transformed to a feature matrix using t-SNE , kernel d elements, If the dimensionality of a sample with It is important to note that when an image in transformed from Cartesian coordinates to pixel coordinates, it brings distortion due to the limited or fixed size of the pixel frame. Due to limited size, many features may overlap on the same location and it would become difficult to perform feature selection. The Snowfall algorithm tries to find nearby empty pixel locations of such features so that overlapping can be minimized. Therefore, it tries to make features visible to CNN by relocating them to neighboring points thereby helping to perform feature selection.Since class activation maps (CAMs)  cannot bconv1). Therefore, the input image size would be ReLu_conv10). After training the CNN model on the optimal hyperparameters, each new sample could be categorized into one of the classes or phenotypes (new_classoutput) at the output layer. Here, an RNA-seq gene expression sample was first converted to an image by DeepInsight and Snowfall compression algorithm, and then used as an input. At the input, the RNAseq sample can be visualized and a particular region leading to its identification can be analyzed at ReLu layer. The activation map at ReLu layer defines which localities of an image are of interest for decision making process. This activation map has three colors in order of preference as red, yellow and blue. The red zone is the most active and blue is the least  step integrates regions of importance. Let a training set with The output of RA is processed to the element decoder model. The pixels underneath the selected region are considered for this task. For a region For genomic or transcriptomic data, the number of genes is normally very high and it becomes very difficult to fit all genes into a finite image size due to fixed hardware limitations. In this case, it is inevitable to get quantized images, i.e. some image pixels will carry multiple genes in a location. One might wonder, how to perform the selection on those batch genes (where batch gene refers to a set of two or more genes having the same pixel location in the frame). One may also want to reduce the number of selected elements. These issues can be addressed by running DeepFeature iteratively. The first iteration will find a subset of elements, which can be used as the input for the subsequent iteration. Continuing this procedure will reduce the number of elements.Here we developed L2-regularized logistic regression model, i.e. each of the 10 cancer studies has its model as shown below .(2)\docuhttps://www.csie.ntu.edu.tw/∼cjlin/liblinear/).This gives the top Equation . TherefoThe dataset was partitioned into 80:10:10 segments corresponding to training set, validation set and an independent test set, respectively. The training set is employed to achieve model fitting, whereas the validation set is used to evaluate its fitness. This picks the hyperparameters for which the validation error is small. The independent test set is held aside and applied to provide an unprejudiced evaluation of the final model. The DeepFeature model is implement on Matlab R2020a version software. The appropriate links are given under Code Availability section.The main aim of this experiment is to show that a subset of imperative elements or genes of nonimage data can be selected by CNN with the utilization of DeepFeature method. We have used high-quality reference resources for enrichment analysis like Reactome and Molecular Signatures Database (MSigDB) , 64 to dWe compared our methodology with alternative ML techniques. A number of ML algorithms exist to find a subset of genes , 66. We P-values were corrected for multiple testing using Benjamini–Hochberg FDR method. In addition, we report the housekeeping gene counts based on the annotation from The Human Protein Atlas  [As outlined above, both DeepFeature and other exemplar algorithms offer some functionality for reducing the number of genes to a more focused subset enriched for the genes used for correctly identifying the relevant cancer study. This feature selection is of particular importance in biological data analysis where identification of key genes and underlying mechanisms is usually part of the overall goal—especially for tasks like identification of clinically useful sparse diagnostic signatures. To evaluate the utility of DeepFeature from this perspective, we have quantified the enrichment of ‘gold standard’ cancer specific gene sets and pathways from MSigDB (C6 subset). In all instances the enrichment was calculated using Fisher’s exact test and reported las.org)  and a stlas.org) . Here, tAnalyzing a large number of elements will cause overlaps in the small pixel frame and it becomes challenging to perform feature selection. This can cause important elements to be overlooked in the selection process. Therefore, it is useful to perform element reduction to reach a manageable size due to the pixel frame size and hardware limitations.The element arrangement step of DeepFeature utilizes t-SNE  and PHATE. In the case of t-SNE technique, it supports various distance measures, ined see . ThereafIn this paper, we present, DeepFeature, a more advanced version of our previous work, DeepInsight, that can now recover underlying feature combinations specific to each class of interest. DeepInsight , established a novel approach that allows high-throughput biological data to be represented in a form compatible with current state-of-art convolutional neural network (CNN) architectures.In addition to delivering high classification performance, DeepFeature also offers a powerful means for the identification of biologically-relevant gene sets.DeepFeature converts non-image samples of RNA-seq data into image-form, and, furthermore, performs gene selection via CNN. To our knowledge, this is the first approach to employ CNN for element or gene selection on non-image data.When applied to the task of cancer classification, our DeepFeature approach was able to identify coherent sets with significant enrichment of genes in cancer-associated pathways from MSigDB and a gold-standard reference set. Further analysis of these results suggested biologically meaningful connections of potential interest to our understanding of the differences between major cancer types.DeepFeature is available for download and the web-links provided in the paper.Supplement_File_1_2_4_bbab297Click here for additional data file.SupplementFile_3_bbab297Click here for additional data file.


K-nearest neighbor (KNN), decision tree C4.5 (C4.5), and random forest (RF) were used as the primary learners of the stacking ensemble. Given the imbalanced characteristics of cancer gene expression data, the embedding cost-sensitive naive Bayes was used as the metalearner of the stacking ensemble, which was represented as CSNB stacking. The proposed CSNB stacking method was applied to nine cancer datasets to further verify the classification performance of the model. Compared with other classification methods, such as single classifier algorithms and ensemble algorithms, the experimental results showed the effectiveness and robustness of the proposed method in processing different types of cancer data. This method may therefore help guide cancer diagnosis and research.Ensemble learning combines multiple learners to perform combinatorial learning, which has advantages of good flexibility and higher generalization performance. To achieve higher quality cancer classification, in this study, the fast correlation-based feature selection (FCBF) method was used to preprocess the data to eliminate irrelevant and redundant features. Then, the classification was carried out in the stacking ensemble learner. A library for support vector machine (LIBSVM),  Cancer is a malignant tumor originating from epithelial tissues. It is a disease caused by the loss of normal regulation and the excessive proliferation of cells in the body. In recent years, cancer incidence and mortality have increased, thus posing severe risks to human health and life. In addition, because the occurrence and development of cancer are dynamic, most patients are diagnosed with cancer in late stages, thus making clinical diagnosis and treatment challenging , 2. WithAt present, the application of machine learning methods to cancer classification is a significant research field in bioinformatics , 5. ManyBoosting  and baggCompared with the two ensemble classification algorithms of bagging and boosting, stacking  providesC-correlation value of each gene and category through symmetric uncertainty, and the irrelevant genes were eliminated. The F-correlation value between features was calculated to eliminate the redundant genes and obtain the candidate gene subset, so as to simplify the combined classification model. Second, in the multiclassifier combination method based on the stacking algorithm, LIBSVM, KNN, C4.5, and RF were used as primary learners. Given the problem of imbalanced cancer gene expression data, CSNB was used as the metalearner of the stacking ensemble to perform combinatorial learning, which was expressed as CSNB stacking. Nine cancer datasets were tested for experiments and then compared with other single classifier algorithms and ensemble algorithms: cost-sensitive KNN stacking (CSKNN stacking), cost-sensitive C4.5 stacking (CSC4.5 stacking), cost-sensitive LIBSVM stacking (CSLIBSVM stacking), bagging, AdaBoost, CSNB, NB, LIBSVM, KNN, RF, and C4.5. The experimental results demonstrated that the proposed method provided more accurate classification and was effective and robust in handling various cancer classification data.To further explore the effects of ensemble learning applied to cancer gene expression data, we adopted a two-layer classification model using a stacking ensemble learning strategy in combination with feature selection technology to conduct a classification study on binary cancer datasets. First, the original gene expression dataset was standardized and transformed into data with a mean value of 0 and a standard deviation of 1. Then, we used FCBF to calculate the The rest of the paper is organized as follows. The mature development of DNA microarray technology provides important guidance for cancer diagnosis and recognition. At present, many scholars have applied the machine learning method to cancer classification and thus designed various classification models and achieved satisfactory results. For example, Musheer et al.  used a nStacking is widely used as a more flexible combination classification model in ensemble learning. However, in the process of constructing the combination model, how to choose the base classifier and give full play to their effectiveness and how to choose the best secondary learner are problems worthy of attention. In addition, the types and characteristics of experimental data will also affect the effectiveness of the classification model. Therefore, in view of the above problems, first of all, we used the FCBF algorithm to reduce the data dimension and achieve the purpose of simplifying the classification model. In addition, four base classifiers  were used as the primary learners of the ensemble model. Meanwhile, cost-sensitive learning idea was introduced as a secondary learner to solve the imbalance of microarray gene expression data, so as to overcome these problems and achieve high-quality classification results.For evaluation of the effectiveness of the proposed method, we used nine groups of cancer datasets of two classes derived from the Kent Ridge Biomedical Dataset database. These datasets included central nervous system embryonal tumors (NervSys), leukemia, three groups of diffuse large B-cell lymphoma (DLBCL), prostate cancer, ovarian cancer, and two groups of lung cancer. A detailed description of these datasets is shown in Among these sample data, DLBCL1 was derived from Stanford data, including a total of 62 samples of two subtypes. DLBCL2 and DLBCL3 were selected from two sets of data detected by Harvard. DLBCL2 included two types of patients: those with DLBCL and those with follicular lymphoma. DLBCL3 comprised the outcome prediction data, including cured patient samples and relapsed patient samples. Lung cancer1 was derived from the University of Michigan, including ten normal samples and 86 diseased samples. Lung cancer2 contained 181 samples comprising 31 cases of malignant pleural mesothelioma and 150 cases of adenocarcinoma, with 12533 genes detected. Notably, the NervSys and DLBCL3 data were outcome prediction data, whereas the ovarian cancer data were protein data, and the remaining samples were from two categories of data.Stacking, also known as stacked generalization , is a teKNN  is a claK-nearest neighbors in the feature space, the category of the prediction sample was usually determined by most of the categories of the K-nearest neighbors. The effects and performance were optimized by selecting the K value, distance measurement method, and classification decision rules.The basic idea of the algorithm was that if a prediction sample has The three commonly used decision tree algorithms  are IterRF  is an enSVM  is a clax and xi are two sample vectors, K is the value of the RBF kernel function, and σ is a free parameter.For nonlinear samples, SVM used a kernel function to map the original data to a high-dimensional space, thus making the samples linearly separable. The optimal classification hyperplane was constructed to separate the samples correctly. Generally, different forms of kernel functions strongly affected the classification performance of SVM. Among them, the radial kernel function (RBF)  had feweCancer gene expression data are a type of unbalanced data . Traditic0 class be the minority class and the c1 class be the majority class. The misclassification cost can usually be represented by a 2 × 2 cost matrix, and each element in the matrix represents the misclassification cost of samples [Cij represents the cost of mistakenly classifying samples that are i classes into j classes. Generally, for the rationality condition of two classification problems [C00 = C11 = 0 and C01 > C10.First, the definition of misclassification cost was given. Taking a binary classification dataset as an example, let the  samples . (2)C=C0problems , correctx with an unknown category is classified by a classification algorithm, the sample x can be represented by a vector . The probability that it belongs to the category cj is P(cj |  x), which is expressed by the Bayesian formula asAfter the cost matrix is determined, the naive Bayes theory is used to construct the risk function . When a cj, the expected misclassification cost isP(ci |  x) represents the posterior probability that sample x belongs to the ci category. Its value is obtained by the naive Bayes formula. Furthermore, the corresponding categories are determined by minimizing the posterior probability as follows:When its category is determined to be x is finally predicted to be a certain category cj that makes R(cj  |  x) have a minimum value, which can be expressed asThe sample essed asRcj∗ ∣ x=C-correlation and F-correlation, where C-correlation is the degree of correlation between features and categories and F-correlation is the degree of correlation between features. When a feature has high C-correlation with a category and low F-correlation with other selected features, the feature is marked as an important feature. In this algorithm, symmetric uncertainty (SU) was adopted as the standard to measure the degree of correlation. SU is defined as the standardized information gain:X and Y represent two random variables, IG(X | Y) denotes the information gain, and H(X) represents the information entropy.FCBF  is a supIn this article, the FCBF algorithm was first used to reduce the dimensionality of the datasets. Then, LIBSVM, KNN, C4.5, and RF were used as primary learners. In addition, given the unbalanced classification of cancer gene expression data, the naive Bayes with embedded cost sensitivity was adopted as the metalearner of the stacking ensemble. In the experiment, the 5 × 10-fold nested cross-validation method was used to divide the data to prevent overfitting. The experimental process is shown in Cancer datasets contain many genes that are irrelevant and redundant for classification, thus leading to more complex classification tasks and inaccurate classification results. In fact, few informative genes are known to directly affect the classification results. Therefore, to save time and calculation costs and obtain better classification performance, we used the FCBF method to quickly and effectively reduce the dimensionality of the cancer datasets before classification, by discarding features making little or no contribution to classification. In this article, data were preprocessed first, and FCBF was used in WEKA for feature selection. The data information after reduction is provided in As shown in In this article, the nine cancer datasets after feature reduction are used as the input to the proposed method CSNB stacking classifier for classification. To test the quality of the proposed method, we compared this method with CSKNN stacking, CSC4.5 stacking, CSLIBSVM stacking, bagging, AdaBoost, CSNB, NB, LIBSVM, KNN, RF, andC4.5.Among these methods, the primary learners with the CSKNN stacking, CSC4.5 stacking, and CSLIBSVM stacking were the same as with the proposed method, but the metalearners were different. CSKNN stacking adopts embedded cost-sensitive KNN as a metalearner for combinatorial learning. CSC4.5 stacking used embedded cost-sensitive C4.5 as a metalearner for classification, whereas the metalearner of CSLIBSVM stacking applies embedded cost-sensitive LIBSVM. Both bagging and AdaBoost ensemble methods used CSNB as the base classifier. The CSNB algorithm introduced cost-sensitive information into naive Bayes, thus making it sensitive to cost and emphasizing the learning of small samples. The above classification models all used cost-sensitive learning. In addition, several other comparison methods exist. Among them, the naive Bayes algorithm was simple and provided stable classification efficiency, but it was highly sensitive to the expression form of the input data; for example, if the training data error is large, the predictive effect will be poor. The LIBSVM classification algorithm had few parameters, flexible operation, and broad application capability. KNN was simple and effective and had low training costs, but it was computationally expensive. RF adopted an integrated algorithm with high accuracy and fast training speed, but the training required large amounts of time and space. Moreover, the RF model was prone to overfitting for sample sets with high noise. The classification rules generated by the C4.5 algorithm were easy to understand and have high accuracy; this algorithm handled discrete and continuous data, but its computational efficiency was low.F-score, specificity, and receiver operating characteristic (ROC) curves were used to evaluate the effectiveness of the proposed method. The final classification accuracy results are shown in Furthermore, owing to the small training set, we evaluated the classification performance with 5 × 10-fold nested cross-validation to avoid overfitting. That is, the data were first divided by the 5-fold cross-validation method, and then, the datasets were divided according to the 10-fold cross-validation method before the primary classifier of stacking carries out the training data. The classification accuracy, recall rate, As shown in Figures F-score of different classification methods on each cancer dataset. As can be seen from the table, the proposed method obtained the highest F-score on all nine datasets. Since the F-score represents the harmonic mean value of precision and recall rate, the experimental results further verified the effectiveness of the proposed method.In order to display the ROC curves of different classification models more clearly and intuitively, we selected NervSys and prostate datasets from nine datasets for graphing. In addition, we divided all classifiers into two groups to show them more aesthetically and clearly. As a combinatorial learning method, the stacking method is supported by less theoretical research than the boosting and bagging combination method, but it is welcomed by many researchers because of its strong flexibility and scalability in algorithms , 45. HowThe datasets selected in this experiment were diverse, including prognostic data, protein data, and two-category data. The experimental results showed that the proposed method achieved the best classification accuracy in different types of cancer datasets, thus reflecting the superiority of the CSNB stacking classification model in processing cancer datasets. On the other hand, through the experimental verification, the four base classifiers selected in this study constitute the primary learner of stacking, which is a satisfactory combination. At the same time, the cost-sensitive idea was introduced into the metalearner of stacking, which can well deal with the problem of unbalanced data. In addition, the proposed classification combination model was superior to the single classifier and ensemble algorithm in terms of stability and generalization ability, and it achieved better classification results. Furthermore, the proposed method provided a feasible reference scheme for the two-classification problem, and it may potentially aid in cancer prediction, identification, and classification.Owing to its high incidence and mortality, cancer has always been a threat to human health. Its complexity and variability make clinical diagnosis and treatment difficult. The emergence of cancer gene expression profiles, by synchronously tracking the expression levels of many genes, is important for early diagnosis of cancer at the molecular level. However, the analysis and processing of gene expression data are accompanied by problems such as high dimensionality, relatively small samples, high noise, and class imbalance. Therefore, to improve the quality of cancer classification and determine the genes that contribute to classification, we proposed a novel method for gene expression data classification: CSNB stacking, based on a CSNB stacking ensemble. This algorithm is based on the traditional stacking algorithm. In addition, because of the imbalanced characteristics of cancer gene expression data, we adopted the embedding CSNB as the metalearner of the stacking ensemble.In the experiment, FCBF was first used to reduce the dimensionality of the data and discarded the irrelevant and redundant attributes. Then, the feature subset was input into the CSNB stacking classification model for classification. The experimental results for nine sets of cancer datasets demonstrated that the CSNB stacking method achieved the best classification performance among the tested methods. Simultaneously, this method had advantages in processing outcome prediction data, protein data, and two categories of data, and it therefore should have high guidance potential and clinical value for cancer classification and prognosis prediction.In addition, through the comparison and analysis of the experimental results, the superiority of the proposed method in the binary classification problem was verified. Our next step will be to expand to the multiclassification task on the basis of the algorithm, to explore its effectiveness. Simultaneously, to make the algorithm more conducive to solving practical problems, we will also focus on combining multiple types of data to achieve more detailed analysis in the future.