The Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Trial is a prospective cohort study of nearly 155,000 U.S. volunteers aged 55–74 at enrollment in 1993–2001. We developed the PLCO Atlas Project, a large resource for multi-trait genome-wide association studies (GWAS), by genotyping participants with available DNA and genomic consent. Genotyping on high-density arrays and imputation was performed, and GWAS were conducted using a custom semi-automated pipeline. Association summary statistics were generated from a total of 110,562 participants of European, African and Asian ancestry. Application programming interfaces (APIs) and open-source software development kits (SKDs) enable exploring, visualizing and open data access through the PLCO Atlas GWAS Explorer website, promoting Findable, Accessible, Interoperable, and Re-usable (FAIR) principles. Currently the GWAS Explorer hosts association data for 90 traits and >78,000,000 genomic markers, focusing on cancer and cancer-related phenotypes. New traits will be posted as association data becomes available. The PLCO Atlas is a FAIR resource of high-quality genetic and phenotypic data with many potential reuse opportunities for cancer research and genetic epidemiology. In addition to these aims, PLCO has been used widely as a prospective cohort. Over the years, PLCO has collected longitudinal health, dietary and lifestyle risk factor data as well as blood, buccal, and pathology samples. These data and samples have been used for studies of cancer etiology and early detection. PLCO has also contributed to many consortia to study the germline genetic architecture of common adult malignancies. As such, numerous previous genotyping efforts have been conducted within the cohortWith the advent of affordable, high-density genotyping platforms, we developed the PLCO Atlas project to enhance the value of PLCO by genotyping all participants who consented to genetic analysis and had available DNA for genotyping. The goal was to build a research resource for studying genetics and gene by environment interactions important for cancer risk. This resource can contribute to the discovery of susceptibility alleles for multiple traits and further define the underlying architecture of genetic susceptibility to these traits. In addition, the resource will be used to evaluate the impact of genetic risk stratification using polygenetic risk scores (PRS) on the screening trial results; and to evaluate the calibration and risk discrimination of cancer risk prediction models that integrate information on genetic and environmental risk factors. As follow-up continues in PLCO, there will be additional cancer diagnoses and deaths occurring with time, yielding larger sample sizes for future research.11. Almost 155,000 men and women aged between 55 and 74 years were enrolled from 1993 to 2001 at 10 screening centers across the United States . Approximately half of the participants were randomized to the intervention arm and underwent cancer screening, while the other half were in the control arm and received standard medical care. Several self-administered questionnaires were administered at baseline and during follow-up, which collected information on demographics, medical history, family history and various lifestyle and dietary risk factors. Information from these questionnaires have been aggregated and harmonized to produce traits and covariates used in the PLCO Atlas genetic association tests. Blood was collected from screening-arm participants at baseline and at each annual screening visit for up to 5 additional years. In addition, buccal cells were collected from 2000–2003 from control arm participants and again in 2018 from participants in both arms. Cancer incidence and mortality outcomes have been tracked longitudinally with a median follow-up length >18 years for cancer incidence  and >19 years for deaths . All cancer diagnoses were confirmed by medical record review and/or via linkage to cancer registries, as previously described13. All participants provided written informed consent and the study was approved by the Institutional Review Boards at the National Cancer Institute and the 10 screening centers. Additional information about the cohort can be found at https://cdas.cancer.gov/learn/plco/home/.Study participants were from the NCI PLCO Cancer Screening Trial, a large, randomized trial designed to evaluate if screening for prostate, lung, colorectal, and ovarian cancers lead to mortality reduction for these diseases8 with a new round of genotyping using the Illumina Global Screening Array (GSA) for 84,731 participants who had low-density genotype data  or no prior genotyping .The PLCO Atlas genotyping project sought to genotype all PLCO participants with genetic consent and available DNA or source vial N = 117,551)  Laboratory. The predominant DNA sample source was buccal cells (48.4%), followed by buffy coat (39.8%), whole blood (4.2%), and buffy coat and red blood cells (1.4%), as well as previously extracted DNA (2.6%). Of the 91,720 participants whose samples were processed, 3,360 (3.6%) individuals were not genotyped on GSA due to insufficient DNA extracted  or insufficient material from previously extracted DNA . In addition, a total of 3,629 (4.0%) individuals were excluded from the final dataset due to quality control failures described below and summarized in Fig. GSA genotyping was performed at the NCI Division of Cancer Epidemiology and Genetics CGR Laboratory according to Illumina protocols and following internal standard operating procedures. The CGR has extensive experience performing high-throughput Illumina bead-based genotyping having previously genotyped hundreds of thousands of samples. Initial GSA genotyping resulted in an overall failure rate of 1.5% for blood-derived DNA and a failure rate of 13% for buccal-derived DNA. After additional DNA extraction and genotyping to recover sample failures, genotyping was fully completed for a total of 84,731 GSA genotyped individuals.Extensive quality control filtering was performed for each array to ensure a set of high-quality genotype data for subsequent imputation and association analyses. Detailed quality control steps and the reasons and numbers of exclusions for the GSA platform are described below and summarized in Fig. Cumulatively, samples from all platforms were filtered to remove abnormal levels of heterozygosity (N = 12), sex discordance (N = 31), within-dataset unexpected duplicates (N = 130), discordant expected duplicates (N = 14), cross-dataset and cross-platform unexpected duplicates (N = 126), and relatedness check (N = 291). A total of 47 subjects with sex-chromosome abnormalities were retained in the dataset for downstream imputation.After applying QC exclusions to each array, a total of 112,065 DNA samples genotyped across 110,562 unique individuals on a modern, high-density Illumina genotyping array remained (Table https://github.com/ncbi/graf) on a set of 10,000 pre-selected fingerprinting variants. GRAF assigned individuals into the following 9 ancestral groups: “African”, “African American”, “East Asian”, “European”, “Hispanic1”, “Hispanic2”, “Other”, “Other Asian”, and “South Asian”. Hispanic1 included individuals of Dominican or Puerto Rican ancestry whereas Hispanic2 included individuals of Mexican or Latin American ancestry. For parsimony and to facilitate downstream analyses, we merged “African” and “African American” into a “African American (Combined)” group and also “East Asian” and “Other Asian” into a “East Asian (Combined)” group. The largest ancestral sets in the PLCO Atlas included European , African American (Combined)  and East Asian (Combined) .Genetic ancestry for PLCO Atlas participants was determined using GRAF (https://imputationserver.sph.umich.edu), we filtered variants with minor allele frequency ≤ 0.01, variant-level missingness ≥ 0.05, and Hardy Weinberg equilibrium exact test p-value ≤ 1 × 10−6 from the imputation input. Data from each genotyping platform were then analyzed using a community-recommended script for aligning data to reference datasets . The script was modified to support TOPMed 5b as a reference panel using a pre-existing test imputation with 1000 Genomes Project subjects versus the TOPMed 5b reference panel. Data were uploaded to the MIS in GRCh37/hg19 and lifted over by the MIS. Pre-phasing using phased reference data from TOPMed release 5b was conducted using EAGLE 2.4. Imputation was conducted against the same reference panel using minimac4 (https://genome.sph.umich.edu/wiki/Minimac4). The “Population” option was set to “EUR” for GSA batches 1–4 that included European ancestry samples, while the option was set to “Other/Mixed” for all other imputations, which consisted of non-European samples or samples of uncharacterized ancestries. The PLCO imputation process took place over several months and was run in different rounds over the span of those months. In total, 110,562 subjects were successfully imputed to the TOPMed 5b reference panel.For genotype imputation, we used the TopMed reference panel on the Michigan Imputation Server, which is accessible on the Michigan Imputation Server to all TopMed collaborators. To prepare for genotype imputation on the Michigan Imputation Server  bins  and each bin was filtered, starting at the variant with the lowest Rsq, until the average Rsq of remaining variants within the corresponding MAF bin was at least 0.9. In total, more that 78,000,000 high-quality imputed variants were available for association testing. In addition, we observed high concordance between high quality imputed SNPs from the GSA with genotyped variants present on the OmniExpress arrays, with a median correlation of 1.00 and a mean correlation of 0.984.Following MIS imputation, raw imputation data were partitioned into subsets according to predicted GRAF genetic ancestry groups to estimate ancestry-specific imputation quality. Ancestry and chip combinations with less than 100 individuals were deemed to have insufficient sample sizes for association testing and removed. After partitioning by ancestry and recomputing imputation quality Rsq values, each platform and ancestry pair was cleaned according to the filtering method described by Kowalski Filtered imputed data by platform and ancestry were then converted to bgen format v1.2) for compatibility with BOLT-LMM and SAIGE for association testing. The resulting final imputed PLCO Atlas Project dataset for association analyses is detailed in Table  for comphttps://github.com/NCI-CGR/plco-analysis). All variants in non-PAR regions of the X chromosome in males were handled by coding these variants as 0/2. Quantitative phenotypes with a sample size of at least 3,000 subjects were analyzed by BOLT-LMM v2.3.415, using linear mixed models on variants with MAF >0.01. The top 20 principal components (generated separately by ancestry) were included as adjustment variables, as well as participant’s age, sex, and study center. Healthy subjects free of any cancer diagnoses throughout the follow-up period were treated as controls for all cancer analyses. Binary and categorical phenotypes were analyzed with SAIGE 0.43.216. We required more than 1,000 subjects and at least 50 cases for each SAIGE phenotype tested. At the variant level, a minimum variant count of 5 and a MAF >0.01 were required for testing. Association analyses were run separately for every GRAF-defined ancestry group, genotyping array, and imputation group. Ancestry-specific results were aggregated by meta-analysis to create overall summary results as well as sex-specific summary result files. Quantile-quantile (Q-Q) plots were generated and lambda values were calculated for each phenotype by linkage disequilibrium score (LDSC) regression17.Association analyses on the autosomes and X chromosome were carried out using the PLCO pipeline hosted on GitHub (https://github.com/NCI-CGR/annotate_rsids_from_linker.git), in the format of rsid:otherAllele:testedAllele (or chr:pos:otherAllele:testedAllele if there was no matching rsid). Population-specific data from the 1000 Genomes Project imputed with the TopMED imputation 5b panel was used to annotate allele frequencies for each tested variant in the GWAS summary statistics using the annotate_frequency program (https://github.com/NCI-CGR/annotate_frequency). Association analyses for every GRAF-defined ancestry group were run separately for each genotyping array and imputation group. While the genotyping arrays and imputation procedures we implemented in the PLCO Atlas captures trait and disease associations with common variants shared across ancestries, associations with population-specific variants and variants with ancestry-specific differences in allele frequencies may not be well captured by this approach.After association analyses using BOLT-LMM or SAIGE, the SNP column of the GWAS summary files were annotated by a custom tool . For smoking-related cancers, for example, we’ve considered cancers of bladder, ureter, kidney, lip, oral cavity, oropharynx, nasopharynx, hypopharynx, larynx, nasal cavity, paranasal sinuses, colorectum, esophagus, gastric, liver (excluding intrahepatic bile duct cancer), lung, myeloid leukemia, ovarian (mucinous), pancreas, and uterine cervix. By pathology, we’ve organized cancers into solid tumors  and hematologic cancers . Within carcinomas, we further broke down to adenocarcinomas (excluding mixed adenocarcinoma), endocrine or neuroendocrine, and squamous cell cancers.Currently the PLCO Atlas project hosts association results for 90 diseases and traits, including a comprehensive list of cancer types and subtypes defined by organ site, etiology, and pathology Table . For exaWe also include GWAS association results for key cancer risk factors such as baseline status of body mass index, height, cigarette smoking for ≥6 months , and cigarette smoking categories , caffeine consumption from diet, and male pattern baldness at age 45, as well as baseline measures of serum PSA level and serum CA-125 level. These initial traits were selected based on available previous data and represent binary, categorical, and continuous traits for the purpose of analytical pipeline development and validity checking. Analyses of additional traits are in progress and association results will be publicly posted as they become available.https://hpc.nih.gov/) that imported and aggregated participant phenotype metadata and variant association data. Using several parallel processes, each phenotype’s variant association data was aggregated and then indexed. Specific plot views for data visualization, such as the single chromosome summary view in the Manhattan plots and the q-q plots, were generated in this import and indexing process. The results were then pooled into the primary MySQL instance where a snapshot was created in Biowulf using Percona Xtrabackup tool. The snapshot was then uploaded to an Amazon Web Services (AWS) Simple Storage Service (S3) cloud bucket where it was restored to AWS’s Relational Database Service (RDS).After association testing and annotation, summary statistic data was imported into a primary MySQL instance using an import script run on the National Institutes of Health (NIH) High Performing Computation Biowulf cluster (Data Records). Download requests are routed to a dedicated local NCI download server to reduce egress costs. Additionally, a cache layer is configured using Redis and AWS ElastiCache to reduce server load and speed-up popular requests.All PLCO Atlas summary statistic data is publicly posted on the GWAS Explorer Fig. . The GWAThe GWAS Explorer frontend website is built with React NodeJS. All user interface components are developed with Bootstrap. Plots for visualization of participant descriptive characteristics and association data are built in Plotly.js as well as custom solutions; for example, Manhattan plots and gene tracks are built in custom canvas. The quantile-quantile (Q-q), principal component (PC) plots and frequency plots are generated using Plotly.js and the bubble charts in the Browse Phenotypes section are produced using custom D3.js. The Apache service handles and serves all incoming web requests.18 under accession number phs001286.v2.p2 (https://identifiers.org/dbgap:phs001286.v2.p2). This public repository allows researchers to apply for access to the individual genotype and imputation data that we used to create the PLCO Atlas. The informed consent document signed by the PLCO study participants allows use of these data by investigators for discovery and hypothesis generation in the investigation of the genetic contributions to cancer and other adult diseases as well as development of novel analytical approaches for GWAS. No IRB review is required for data use. Public posting of genomic summary results is permitted. Companion phenotype data can be requested through the NCI Cancer Data Access System (CDAS) (https://cdas.cancer.gov/plco/). We note that some of the cancer endpoint data used in the PLCO Atlas cannot be shared through CDAS due to restrictions on data use agreements with certain cancer registries. However, summary GWAS statistic data is directly available from the PLCO Atlas GWAS Explorer website (https://exploregwas.cancer.gov/plco-atlas/) as well as accessed directly through API access (https://exploregwas.cancer.gov/plco-atlas/#/api-access).All PLCO genotype data is available in dbGaPThe Illumina genotyping process performed at the CGRL include many systems that aim to ensure reliable, accurate, and high-quality data. Prior to performing the GWAS, dilution and contamination series of control DNA samples were processed through GSA genotyping to establish thresholds for input requirements and level of sensitivity for contamination using VerifyIDintensity. All specimens, from time of arrival at the CGR laboratory through extraction, DNA sample QC, project plating, and genotyping were tracked through Laboratory Information Management Systems (LIMS) specific to CGR and Illumina processes, including sample plate and well locations and array section locations. LIMS uses this information to generate all manifests for downstream analysis automatically, without manual data entry or intervention. Laboratory staff were blinded to sample information as each tube or plate has a unique barcode to identify instance or work to be performed without any identifying information. All laboratory processing, including sample plating, genotype array preparation in plates and on glass arrays, as well as scanning, was performed using automation, which tracks and records sample plate and glass array barcodes for import into LIMS. All processes were calibrated and validated routinely, using controlled methods to standardize processing and reduce the occurrence of human error. Manifests containing all necessary information and metadata to perform analyses were automatically generated from the CGR LIMS for automated QC analysis.Internal controls were utilized throughout the GWAS, in which at least 1 sample is randomly located per 96-well plate, rotating through a familial trio of subjects and an unrelated subject with DNA extracted from EBV-transformed cell lines. Technical replicates of study subjects were also included, accounting for 2–3% of samples processed, both within and across 96-well plates, also in random well locations. These replicates were located within the 96-well plate to create a unique layout to identify any potential plate mix-ups and distributed across each set of 24 samples run on the GSA chip, so that each chip contained a QC sample. Internal controls and study replicates were utilized during data QC, confirming sex concordance and genotype concordance of internal control and study replicates. For data generated from each 96-well plate to be considered valid, QC samples and study replicates must be >99% concordant with technical replicates.https://exploregwas.cancer.gov/plco-atlas/)19. All modules were designed for rapid query times, ease of use, and ability to interactively visualize the data.The GWAS Explorer provides three main modules for accessing and interactively viewing the PLCO Atlas GWAS association data, all of which can be accessed from the landing page  plot of expected -log10 p-values and observed -log10 p-values as well as the lambda and number of variants analyzed; and a PC plot that displays all individuals included in the analysis and allows for principal components 1 through 20 to be plotted against each other. Sharable links can be generated to share specific plot views with collaborators. The generated interactive table displays the lowest association p-values for a trait and can be sorted and searched for specific variants. Another tab in the “GWAS Results” section also allows for searching for a specific variant or list of variants by RefSeq (RS) number or chromosomal position (GRCh38).The first module is the “GWAS Results” section where a user navigates a tree menu to select the phenotype, or phenotypes of interest when doing pairwise plots, as well as the desired sex and ancestry group of interest before the Submit button will become active. Once submitted, the query generates a header defining the phenotype and number of participants as well as a series of interactive plots and a table of most significant variants Fig. . The intThe second module is the “Browse Phenotypes” section. Here users can get additional information about the participants included in the study by phenotype. A menu tree or bubble plots can be used to browse to a phenotype of interest. Once selected, details and plots for the phenotype will be displayed. Phenotype definition is displayed at the top of the page to clarify how phenotypes were assigned. Plots of overall frequency, frequency by sex and frequency by ancestry are all included. Tables can also be displayed to see the data used to generate the plots. Sharable links can also be generated for the “Browse Phenotype” module.“Download Data” is the final module and allows users to download the association summary data. Like in the other modules, a tree menu allows users to select a phenotype of interest for download. The submit button begins the download process through the web browser. Up to 5 phenotypes may be selected for download at a time.https://exploregwas.cancer.gov/plco-atlas/#/api-access. Users can test queries interactively using the web interface before accessing the API programmatically. Many API endpoints require a phenotype_id, which can be obtained by querying the /api/phenotypes route and will return an array of phenotype objects containing the phenotype_id, name, and other properties. Output is returned in JSON format except when specifically indicated.In addition to the web interface, the GWAS Explorer also allows for API Access to the data accessible through a REST API. The API enables users to retrieve PLCO Atlas data in their preferred environment and offers more flexibility for querying data than the web interface. The syntax needed to perform API calls is described in the documentation available at Further details about the PLCO study and GWAS Atlas are available in the “About” section of the webpage.


Traditional bulk RNA-Seq pipelines do not assess cell-type composition within heterogeneous tissues. Therefore, it is difficult to determine whether conflicting findings among samples or datasets are the result of biological differences or technical differences due to variation in sample collections. This report provides a user-friendly, open source method to assess cell-type composition in bulk RNA-Seq datasets for heterogeneous tissues using published single cell (sc)RNA-Seq data as a reference. As an example, we apply the method to analysis of kidney cortex bulk RNA-Seq data from female (N=8) and male (N=9) baboons to assess whether observed transcriptome sex differences are biological or technical, i.e., variation due to ultrasound guided biopsy collections. We found cell-type composition was not statistically different in female versus male transcriptomes based on expression of 274 kidney cell-type specific transcripts, indicating differences in gene expression are not due to sampling differences. This method of cell-type composition analysis is recommended for providing rigor in analysis of bulk RNA-Seq datasets from complex tissues. It is clear that with reduced costs, more analyses will be done using scRNA-Seq; however, the approach described here is relevant for data mining and meta analyses of the thousands of bulk RNA-Seq data archived in the NCBI GEO public database. This method, which provides a simple method for assessing sampling biases in bulk RNA-Seq datasets with evaluation of cell-type composition, will aid researchers in assessing whether bulk RNA-Seq from different studies of the same heterogeneous tissue are comparable. The additional layer of information can help determine if differential gene expression observed is biological or technical, i.e., cell composition variation among study samples. The described method uses publicly available bioinformatics resources and does not require coding expertise or high-capacity computational processing. Development of tools accessible to scientists without computing expertise will contribute to greater rigor and reproducibility for bioinformatic analyses of transcriptome data. Bulk RNA-Seq is a sensitive and robust method for identifying transcriptional changes in tissues, but it’s often difficult to assess what cell types are represented in transcriptome data. Standard computational pipelines for bulk RNA-Seq data analysis do not typically assess cell-type composition within heterogeneous tissues–4. This While some methods allow for cell type selection for bulk RNA-Seq during sample preparation such as tissue microdissection and flow cytometry, the additional processing requires collection of greater amounts of tissue than simple bulk RNA-Seq–13. SingQuality control (QC) assessment of bulk RNA-Seq data does not typically include assessment of cell-type composition, rather QC is limited to RNA quality,17. ThisSingle cell references use a list of previously identified gene markers based on cell-type specific gene and protein expression to identify cell types,14,19,20We describe an easy-to-use protocol for those with limited computational background to process a previously published single cell dataset for creation of a reference cell-type expression list, and then annotate a bulk RNA-Seq dataset using Microsoft Office, which is commonly available at minimal expense in universities and other workplaces. This method can easily be adapted to R or other software with a simple script.Papio hamadryas) are genetically and physiologically similar to humans and have been used to study the role of the kidney in many complex diseases maintained as part of the pedigreed baboon colony at SNPRC, located on the campus of Texas Biomedical Research Institute, San Antonio, Texas. All animal procedures were reviewed and approved by Texas Biomedical Research Institute’s Institutional Animal Care and Use Committee protocols (IACUC). SNPRC facilities at the Texas Biomedical Research Institute and animal use programs are accredited by Association for Assessment and Accreditation of Laboratory Animal Care International (AAALAC), operate according to all National Institutes of Health (NIH) and U.S. Department of Agriculture (USDA) guidelines, and are directed by veterinarians (DVM). All animal care decisions were made by the SNPRC veterinarians. Daily enrichment was provided by the SNPRC veterinary staff and behavioral staff in accordance with AAALAC, NIH, and USDA guidelines. The KINSHIP program from Pedigree Database System (PEDSYS v. 2.0) was used with the Stevens-Boyce algorithm to calculate kinship coefficients to ensure similar degrees of relatedness among all animals–31 , sedated with ketamine , and maintained on isoflurane (1.3–3.0%) anesthesia throughout kidney biopsy. Kidney biopsies were collected via ultrasound guidance from the cortex region of the left kidney and immediately placed into cooled 1.5 mL tubes then flash frozen in liquid nitrogen. Animals were monitored during collections and throughout recovery. Postoperative care included observation for swelling at the surgical sight and monitoring until animal was conscious. Buprenorphine  was administered for post-operative pain relief immediately once animals were awake. Buprenorphine  was administered as needed 24–48 hours after surgery–31.ad libitum in individual cages during feeding times. Female baboons were housed in an outdoor social group with one vasectomized male (not on study) to provide full social and physical activity. Females were run into individual cages during feeding times. Males were housed individually prior to study start for 6 weeks prior to study initiation. Animals were acclimated to cages for at least 8 weeks prior to study. Animals had free access to 500 g chow r 6 weeks,30,32.RNA was isolated from kidney biopsy samples using the Direct-zol RNA Miniprep Plus Kit . Tissue was homogenized in 600 μL of TRI reagent  using a BeadBeater (Biospec Products) for 3 × 30 sec with RNA purification according to the Direct-zol RNA Miniprep Plus Kit instructions. RNA was quantified by Qubit RNA BR Assay Kit , and RNA integrity was determined with an RNA ScreenTape Kit on the TapeStation 2200 system  and only RIN scores of 8 or higher were used for sequencing. cDNA libraries were prepared using the KAPA Stranded RNA-Seq Kit . Libraries were quantified using a KAPA Library Quantification Kit , and quality was determined with a DNA ScreenTape Kit on the TapeStation 2200 system . Sequencing was done using a HiSeq 2500 (Illumina).® Flow®  with a minimum read length of 25 and minimum Phred score of 330 30. Adapter sequences were trimmed with the resulting average read length above 90 bases in each group. Reads were aligned to the baboon genome . Reads were normalized by TMM+1 and filtered for genes expressed an average of less than 1 across all samples. The list of baboon kidney bulk RNA-Seq quality transcripts for all samples was saved as a .csv file. The list included official gene symbols and abundance values for each gene in each sample.Female and male transcript data were processed together in PartekThe list of transcripts that passed quality filters was annotated to identify kidney-specific cell types by the following steps. A reference list of kidney cell-type specific transcripts was derived from previously published rat whole kidney scRNA-Seq data which included genes specifically expressed in one of 43 kidney cell types. For annusegalaxy.org). The Gal®, Inc.). Data are expressed as mean, mean ± SD, or coefficient of variation and were statistically significant if the FDR adjusted p-value < 0.05.Data, the 274 cell-type specific transcripts were assessed. Multiple unpaired t-tests with FDR two-stage setup  to adjust for multiple testing were performed in GraphPad Prism version 9.5.0 for Mac, (GraphPad Software). Unbiased gene expression analysis by PCA was performed on normalized transcripts with ellipsoids representative of 2 standard deviations away from the mean using Partek Genomics Suite . We performed a standard workflow for quality filtering of bulk RNA-Seq data prior to comparison of females and males to identify differentially expressed genes,32. We aet al., was expressed significantly higher in males compared to females .We assessed cell-type composition of the baboon transcriptome  . et al. among alFAM19A3, LEP, NOS1, were only found to be expressed in female baboon kidney cortex samples.PCA of all quality transcripts in female and male baboon samples revealed the top principal components of 24%, 11.6%, and 9.03%, which account for 44.6% variation in the samples, with two clusters revealing sex-specific profiles . CoefficMany bulk RNA-Seq studies have been performed to date, and rarely is it clear whether gene expression differences are due to tissue sampling  or biological differences. Profiling cell composition strengthens results by ensuring greater rigor and reproducibility. By using gene expression that is exclusively expressed in one cell type, it is reasonable to infer that the cell type is present. Most of the currently available methods to deconvolute cell types in bulk RNA-Seq data require hosting packages and may be inaccessible to scientists with limited bioinformatics training. We present a simple method for cell type deconvolution.This new method ultimately saves time, samples, and money by leveraging existing datasets as a reference. Moreover, collaborative groups often have many people involved from sample collection to final analysis, and cell type profiling provides an additional layer of QC for the person doing downstream data analysis. This technique can also be applied to older datasets, proteomic and metabolomic datasets, and aid in determining study similarity related to tissue and cell-type sampling,34. It iWe performed bulk RNA-Seq on kidney cortex biopsies to assess sex differences in the transcriptome and used a simple cell type deconvolution method to determine whether observed differences were technical or biological. Our results showed consistency in kidney biopsy cell composition among samples indicating the sex differences were biological. This method of assessing sampling consistency of bulk RNA-Seq datasets utilizing an existing scRNA-Seq dataset as a reference with a “computationally light” annotation pipeline, is especially useful for bulk RNA-Seq studies with heterogeneous cell types. Inclusion of these steps provides an additional level of rigor for transcript data quality assessment. Use of official gene symbols, in which orthologous genes use the same symbol, allows cross species comparison to leverage the extensive publicly available scRNA-Seq datasets primarily generated from rodent studies,27,35–37While this method does not tell us the number of cells present for each cell type, it consistently reveals the presence of individual cell types allowing comparison among samples,30,35,37


A fundamental problem of downstream analysis of scRNA-seq data is the unavailability of enough cell samples compare to the feature size. This is mostly due to the budgetary constraint of single cell experiments or simply because of the small number of available patient samples. Here, we present an improved version of generative adversarial network (GAN) called LSH-GAN to address this issue by producing new realistic cell samples. We update the training procedure of the generator of GAN using locality sensitive hashing which speeds up the sample generation, thus maintains the feasibility of applying the standard procedures of downstream analysis. LSH-GAN outperforms the benchmarks for realistic generation of quality cell samples. Experimental results show that generated samples of LSH-GAN improves the performance of the downstream analysis such as feature (gene) selection and cell clustering. Overall, LSH-GAN therefore addressed the key challenges of small sample scRNA-seq data analysis. LSH-GAN is a locality-sensitive hashing based generative adversarial model that can produce realistic cell samples from small sample single-cell scRNA-seq data. The generated cells can be utilized for downstream analysis, like gene selection and cell clustering. The high dimension, and small-sample size (HDSS) data handling is difficult for downstream analysis particularly for feature selection (FS). It affects later stages of downstream analysis such as cell clustering, marker selection, and annotation of cell clusters. A few outliers can drastically affect the FS techniques, and the selected feature sets may not be adequate to discriminate the classes3. Moreover, high dimensionality increases the computational time beyond acceptability.Recently, the emergence of high dimensional biological data such as single-cell RNA sequence (scRNA-seq) data has posed challenges to machine-learning researchers8. Each step has a profound effect on the next stage of analysis. The gene selection step identifies the most relevant features/genes from the normalized/preprocessed data and has an immense impact on cell clustering10. The general procedure for selecting relevant genes which are primarily based on high variation (highly variable genes)12 or significantly high expression (highly expressed genes)4 suffers from a small-sample effect. The general FS techniques also failed to provide a stable and predictive feature set in this case due to an ultra-large size of feature (gene). One way to solve this issue is to go for a robust and stable technique that does not overfit the data. A few attempts14 were observed recently which embed statistical and information-theoretic approaches. Although these methods result in stable features, however, these are not performed well in small-sample scRNA-seq data.High dimensional small-sample (HDSS) data is prevalent in the single-cell domain due to the budgetary constraint, ethical consideration of single-cell experiments, or simply because of the small number of available patient samples. Whatever the reason is, too few observations (cell sample) in the single-cell data may create problems in the downstream analysis. This is because a small-sample size may not reflect the whole population accurately, which surely degrades the performance of any model. The general pipeline of scRNA-seq downstream analysis starts with preprocessing  of the raw count matrix and then going through several steps which include identification of relevant genes, clustering of cells, and annotating cell clusters with marker genes15, Splatter16, SUGAR17 are already developed which uses different techniques  to successfully simulate the samples of specific cell types or subpopulations. The challenge in this task is to handle the sparsity and heterogeneity of the cell populations which define the specific characteristics of scRNA-seq data. In this paper, we propose a generative model to sort out this problem in HDSS scRNA-seq data. We use generative adversarial model to generate realistic cell samples from a small number of available samples of HDSS scRNA-seq data. Generative adversarial network (GAN)21 has already been shown to be a powerful technique for learning and generating complex distributions23. However, the training procedure of GAN is difficult and unstable. The training suffers from instability because both the generator and the discriminator model are trained simultaneously in a game that requires a Nash equilibrium to complete the procedure. Gradient descent does this, but sometimes it does not, which results in a costly time-consuming training procedure. The main contribution here is in modifying the generator input that results in a fast training procedure. We create a subsample of original data based on locality-sensitive hashing (LSH) technique and augment this with noise distribution, which is given as input to the generator. Thus, the generator does not take pure noise as input, instead, we introduce a bias in it by augmenting a subsample of data with the noise distribution.Recently computational researchers are gaining interest in this field. Some methods like cscGAN24, Wasserstein GAN (WGAN)22 try to train the model quicker than the conventional GAN. Unlike PGAN and WGAN, conditional GAN (CGAN)25 operates by conditioning the conventional model on additional data sources  to dictate the data generation. In our model, we direct our attention to the additional sample generation from HDSS data. However, the generated sample size becomes increasingly large with more features, the generation of which may not be feasible for conventional generative models. Augmenting subsample of real data distribution (pdata(x)) with the prior noise (pz(z)) makes the training procedure of our model faster than the conventional GAN. We theoretically proved that the global minimum value of the virtual training criterion of the generator is less than the traditional GAN (<−log4).Researchers are still trying to find improved versions of the GAN to use in different domains. Most of the variations such as progressive GAN (PGAN)Here, we provide the following: (i) The proposed model address the problem of downstream analysis (particularly gene selection and clustering) on HDSS scRNA-seq data. (ii) LSH-GAN is able to generate realistic samples in a faster way than the traditional GAN. This makes LSH-GAN more feasible to use in the feature (gene) selection problem of scRNA-seq data. (iii) LSH-GAN can produce more realistic cell samples than the other existing benchmarks. (iv) Here we derive a training procedure of generator that combines subsamples of original data with pure noise and takes this as input. (v) For a fixed number of iteration LSH-GAN performed better than the traditional GAN in generating realistic samples. (vi) Gene selection and clustering on the generated samples of LSH-GAN provide excellent results for small-sample and large-feature sized single-cell data.In the following, we first describe the workflow of our analysis pipeline. Next, experimental validation of the proposed model is carried out by comparing it with several state-of-the-arts in real-life scRNA-seq data. Finally, we used LSH-GAN to select genes from HDSS scRNA-seq data. We use benchmark gene selection techniques on the generated samples and used one single-cell clustering technique to validate the selected genes.Figure 27 is widely used in nearest neighbor searching to reduce the dimensionality of data. LSH utilizes locality-sensitive hash functions which hash similar objects into the same bucket with a high probability. The number of buckets is much lesser than the universe of possible items, thus reduces the search space of the query objects LSHForest module with default parameters.In this work first, the unique hash codes which depict the local regions or neighborhoods of each data point are produced. For this, we utilized python sklearn implementation of k-nn graph) is constructed by using k = 5 for each data point. This step computes the euclidean distances between the query point and its candidate neighbors. Sampling is carried out in a ‘greedy’ fashion where each data point is traversed sequentially and its corresponding five nearest neighbors are flagged out which never visited again. Thus after one traversing a sub-set of samples is obtained which is further down-sampled by performing the same step iteratively.An approximate neighborhood graph (G) is modified by augmenting its taken input data. Instead of giving the pure noise (pz(z)) as input we augment a subsample of real data distribution (pdata(x)) with it. The sampling of the input data is done in the LSH step. Thus the generator (G) function builds a mapping function from x) as The generator function (pdata(x) and generated data coming from generator (D(x) that represents the probability that the data x is coming from the real data: D(.): x → .Here discriminator (D) takes both the real data D and G form a two-player minimax game with value function L. We train D to maximize the probability of correctly validating real data and generated data. We simultaneously train G to minimize pz) and the sampled data So, the value function can be written as:GLM-PCA28, CV2Index, M3Drop29, Fano Factor30, and Highly Variable Gene (HVG) selection of Seurat V431. Single-cell clustering method (SC3) technique is utilized to validate the selected genes from the generated samples.The generated cell samples are utilized for the gene selection task. We have employed five well-known gene selection methods (with default parameters) of scRNA-seq data adopted for validation: The whole algorithm and the sampling procedure are described in Table k) and the number of iteration (t) are two main parameters of the LSH step , which results in the lowest Wasserstein metric. For example, we take eopt as 10k, 30k, 10k, 15k, and 25k for the dataset Darmanis, Yan, Pollen, Klein, and Melanoma, respectively  and discriminator (D) uses 2-layer multilayer perceptrons with hidden layer dimension as . For traditional GAN, we retain the same settings as LSH-GAN for G and D networks.15, SUGAR17, and Splatter16. For these three methods, We adopted the code (with default parameters) provided on the Github page of the original publications.For benchmarking our method we have utilized three state-of-the-art techniques widely used for sample generation: cscGANGLM-PCA28, CV2Index, M3Drop29, Fano Factor30, and Highly Variable Gene (HVG) selection of Seurat V431. We select the top 500 features (genes) using all three feature selection methods on scRNA-seq datasets. For validation purposes, Wasserstein metric22 is utilized to estimate the quality of the generated data. Clustering of scRNA-seq data is performed using SC332 technique with default parameters. Clustering performance is evaluated using the adjusted Rand index (ARI), normalized mutual information (NMI).Five well-known gene selection methods (with default parameters) of scRNA-seq data are adopted for validation: μ) of the data in the range of 5 to 15 for class-1 and −15 to −5 for class-2. The covariance matrix (Σ) is taken for all the samples using the formula Σ = (ρi−j∣∣), where i, j are row and column index, and ρ is equal to 0.5. We calculate Wasserstein metric to estimate the quality of the generated data. The Wasserstein distance between the real data distribution (pdata) and the generated data distribution (pg) to estimate the quality of the generated data. We use different settings of kth  nearest neighbor to generate subsample of data from LSH sampling procedure. In each case, the sampled data (pz) and given to the generator of LSH-GAN for model training.First, we train the LSH-GAN on HDSS synthetic data and generate realistic samples to compare against the traditional GAN model. For this, we create a 2-class non-overlapping Gaussian mixture data consisting 100 samples and 1000 features by taking the mean (k. A closer look into Table k = 5) is far better than the traditional GAN model with 25,000 epochs. Notably, for less amount of sampling (larger k), LSH-GAN needs more iterations for training. As for particular example, the performance of LSH-GAN achieved on 20,000 epoch and k = 20 is rivaled only at 10,000 epoch for k = 10. Thus it is evident from the results that reducing the amount of sampling needs more epochs and thus needs more training time for the LSH-GAN model to converge. Figure For comparison with the traditional GAN model, we use the data with train: test split of 80:20 and calculate the Wasserstein metric between the test sample and the generated sample. Table 15, splatter16, SUGAR17, traditional GAN18, and its two variants f-GAN23 and w-GAN33. Since the evaluation of the generative model is notoriously difficult35, we first use Wasserstein distance to compare the real data distribution and generated data distribution coming from different competing models. We also used UMAP visualization, and marker genes expression to visualize the generated cell samples. We create UMAP visualization for LSH-GAN, SUGAR, and cscGAN, as these methods are more associated with the generation of single-cell data. Figure We compare LSH-GAN with four existing benchmarks: cscGANWe also validate the generated samples by training a classifier (random forest) to see whether it can able to distinguish the samples coming from two different distributions . The aim is to see whether the model can discriminate between the real and generated cell samples accurately. Table Fano Factor, CV2-index), widely used for the gene selection task in scRNA-seq data and one single-cell clustering method (SC3) technique to validate the selected genes from the augmented data.Here, we aim to address the problem of gene selection in HDSS scRNA-seq data using the generated samples. We augment the generated sample with original data to make the sample to feature ratio as 1.5. The augmented data is utilized for gene selection. Here, we have employed five feature selection methods (Highly Variable Gene (HVG) selection of Seurat V3/V4, M3Drop, GLM-PCA, and LSH-GAN is compared with five other state-of-the-arts in four HDSS scRNA-seq datasets . We exclude Melanoma data for this analysis as it already has larger sample size compared to the feature size (sample:feature is 3.46). The aim is to know whether the selected features/genes from the generated combined data can lead to a pure clustering of cells. Table 32 is adopted for cell clustering. Figure 32. Here two cells are said to be in different clusters if the score is zero (blue color). Similarly, a score ‘1’ (red) signifies two cells that belong to the same class. Thus completely red diagonals and blue off-diagonals represent a perfect clustering. A careful notice on Fig. Here our aim is to investigate whether the selected genes from generated scRNA-seq data are effective for cell clustering. We have utilized the generated data from the four scRNA-seq datasets for gene selection. A widely used single-cell clustering method SC3https://github.com/JinmiaoChenLab/Batch-effect-removal-benchmarking) of Tran et al.36. The data consists of human blood dendritic cell (DC) cells created with Smart-Seq2 technology in two different batches. Both batches contain 96 pDC and 96 double negative cells. Each batch has one biologically similar unshared cell type: CD141 cells in the first batch and CD1C cells in the second batch. The two batches contain a total of 288 cells over 16,594 genes. We have applied LSH-GAN on the first (or second) batch and select genes from the generated data. These genes are utilized to cluster the second (or first) batch. In both cases, we got high ARI scores . For gene selection, we have utilized HVG of Seurat v4.To know how the generated data of LSH-GAN is affected by the data of different batches, we performed this analysis. We first download the processed datasets from a github repository (pg = pdata).In this paper, we present a novel and faster way of generating cell samples of HDSS single-cell RNA-seq data using a generative model called LSH-GAN. We update the training procedure of GAN using locality-sensitive hashing which can produce realistic samples in a lesser number of iteration than the traditional GAN model. We utilized the generated data in the standard procedure of downstream analysis for analyzing real-life scRNA-seq data. Particularly, we demonstrated that the recent and benchmark approaches of gene selection and cell clustering produce excellent results on the generated cell samples of LSH-GAN. Our preliminary simulation experiment also suggests that for fixed number of training iteration the proposed model can generate more realistic samples than the traditional GAN model. This observation is also established theoretically by proving that the cost of value function is less than −log4 which is the cost for traditional GAN at the global minimum of virtual training criterion  and feature selection  which may play a critical role in the clustering performance. However, we found clustering results are always better for the generated data with more than 1p (p is the feature size) sample size. This observation suggests that for feature selection in HDSS data, whenever we produce samples larger than the feature size we will end up with a better clustering. The feasibility of generating such samples is justified by the faster training procedure of LSH-GAN model.One limitation of our method is that for feature selection we hardly found any linear relationship between the clustering results with the sample size of generated scRNA-seq data. The correct sample size should be selected by using a different range of values between 0.25It may interesting to speculate how well LSH-GAN can be useful for generating data of other domains, such as for image analysis, bulk RNA-seq analysis, and spatial transcriptomics. For bulk RNA-seq data, one can directly apply LSH-GAN, keeping the same setup of LSH-based sampling procedure  https://www.ncbi.nlm.nih.gov/geo/. Table We have used five public benchmark scRNA-seq datasets: Melanomac and g represent the number of cells and genes, respectively, is normalized using Linnorm42 Bioconductor package of R. We select cells having more than a thousand expressed genes  and choose genes having a minimum read count more than 5 in at least 10% of the cells. log2 normalization is performed on the transformed matrix by adding one as a pseudo count.The raw count matrix In this section, we first provide a short description of GAN and then explain the theoretical foundation of LSH-GAN model.18 which was proposed to train a generative model. GAN consists of two blocks: a generative model (G) that learn the data distribution (p(x)), and a discriminative model (D) that estimates the probability that a sample came from the training data (X) rather than from the generator (G). These two models can be non-linear mapping functions such as two neural networks. To learn the generator distribution pg over data x, a differentiable mapping function is built by generator G to map a prior noise distribution pz(z) to the data space as G. The discriminator function D returns a single scalar that represents the probability of x coming from the real data rather than from generator distribution pg. The goal of the generator is to fool the discriminator, which tries to distinguish between true and generated data. Training of D ensures that the discriminator can properly distinguish samples coming from both training samples and the generator. G and D are simultaneously trained to minimize G and maximize log(D(x)) for D. It forms a two-player min–max game with value function VGAN is introduced in ref. pz(z). Due to this additional information in generator, we assume that the probability ζ.For LSH-GAN, a sub-sampling of real data The value function of LSH-GAN can be written as:Proposition 1. L is maximized with respect to discriminator (D), for a fixed generator (G), whenProof. Equation  ∈ R2{0, 0} and ζ ∈ . So, the optimum value of D for a fixed generator, G is:D is to maximize the log-likelihood of the conditional probability P(Y = y∣x), where Y signifies whether x is coming from real data distribution(y = 1) or coming from the generator(y = 0). Now the equation  = pdata(x) ), the value ofC(G) is less thanProof. From equation (JSD(pdata(x)∣∣pg(x)) represents Jensen–Shannon divergence between two distributions pdata and pg. Now, if the two distributions are equal, Jensen–Shannon divergence (JSD) will be zero. Thus, for global minimum criterion of the value function (pg = pdata) Eq. (equation  we get8\p(z)) as input instead a subsample of the original dataset is augmented with the noise. By adding this term, without loss of generality we assume that the probability ζ. The assumption is justified by the input of the generator, which takes some amount of real data samples with the noise (p(z)) from the very first step of the training. The output of the generator G(z) generated from the random noise (p(z)). This cause a small increase of the value ζ. This ultimately leads to a lower value of the cost function than (−log4), which is the value of the cost function of original GAN at the global minimum criterion of value function L (pg(x) = pdata(x)).As explained in the above proof, the generator function of LSH-GAN does not take prior noise (Further information on research design is available in the Peer Review FileSupplementary InformationReporting Summary


A fused method using a combination of multi-omics data enables a comprehensive study of complex biological processes and highlights the interrelationship of relevant biomolecules and their functions. Driven by high-throughput sequencing technologies, several promising deep learning methods have been proposed for fusing multi-omics data generated from a large number of samples.In this study, 16 representative deep learning methods are comprehensively evaluated on simulated, single-cell, and cancer multi-omics datasets. For each of the datasets, two tasks are designed: classification and clustering. The classification performance is evaluated by using three benchmarking metrics including accuracy, F1 macro, and F1 weighted. Meanwhile, the clustering performance is evaluated by using four benchmarking metrics including the Jaccard index (JI), C-index, silhouette score, and Davies Bouldin score. For the cancer multi-omics datasets, the methods’ strength in capturing the association of multi-omics dimensionality reduction results with survival and clinical annotations is further evaluated. The benchmarking results indicate that moGAT achieves the best classification performance. Meanwhile, efmmdVAE, efVAE, and lfmmdVAE show the most promising performance across all complementary contexts in clustering tasks.https://github.com/zhenglinyi/DL-mo.Our benchmarking results not only provide a reference for biomedical researchers to choose appropriate deep learning-based multi-omics data fusion methods, but also suggest the future directions for the development of more effective multi-omics data fusion methods. The deep learning frameworks are available at The online version contains supplementary material available at 10.1186/s13059-022-02739-2. Advances in high-throughput techniques have led to an explosion of multi-omics data in biomedical research. Each type of omics data helps researchers to understand the complex biological systems from different perspectives, such as genomics, transcriptomics, proteomics, and metabolomics . ResearcSome researchers reviewed and tested several traditional ML algorithms from a data fusion perspective , 12–15. This paper evaluated the performance of 16 representative and open-source models from all DL-based data fusion methods on three different datasets, i.e., simulated multi-omics datasets, single-cell multi-omics datasets, and cancer multi-omics datasets. These 16 models were grouped into two categories: supervised models (six) and unsupervised models (ten). Accordingly, for each of the datasets, two tasks were designed: classification and clustering. For simulated and single-cell datasets, ground-truth samples were retrieved through classification and clustering by using six supervised models and ten unsupervised models, respectively. For cancer datasets, supervised DL methods were evaluated in the classification tasks on five types of cancer datasets with ground-truth cancer subtypes. Meanwhile, unsupervised DL methods were evaluated in the clustering tasks. Furthermore, the associations of the embeddings with survival and clinical annotations were evaluated. Based on the benchmarking results, we provided recommendations for biologists to choose appropriate methods in different scenarios and give guidelines on methodological improvements for researches focusing on algorithm design of multi-omics data fusion.DL-based multi-omics data fusion methods aim to learn low-dimensional embeddings from the fusion of multi-omics data for various downstream tasks. According to our investigation, various DL-based data fusion methods can achieve this goal, including fully connected neural network (FCNN) –20, convTo make the structural differences of different models easy to understand, we renamed the model according to the structural characteristics (early or late fusion). The evaluation models are described as follows: a late fusion method based on AutoEncoder (lfAE), an early fusion method based on AutoEncoder (efAE), a late fusion method based on Denoising AutoEncoder (lfDAE), an early fusion method based on Denoising AutoEncoder (efDAE), an early fusion method based on Variational AutoEncoder (efVAE), an early fusion method based on Stacked Variational AutoEncoder (efSVAE), an efVAE method whose loss function is a maximum mean discrepancy (efmmdVAE), a late fusion method based on Neural Network (lfNN), an early fusion method based on Neural Network (efNN), a late fusion method based on Convolutional Neural Network (lfCNN), an early fusion method based on Convolutional Neural Network (efCNN), a multi-omics Graph Convolutional Network method (moGCN), and a multi-omics Graph Attention network method (moGAT). To accommodate different evaluation data, this study modified the input of these ten original models proposed by Ma et al. , Lee et Furthermore, considering evaluation completeness, our own frameworks were designed, including a late fusion method based on Variational Autoencoder (lfVAE), a late fusion method based on Stacked Variational Autoencoder (lfSVAE), and a lfVAE method with a loss function of maximum mean discrepancy (lfmmdVAE).InterSIM CRAN package . The larger value of JI, the higher the similarity.JI is a statistic used to compare the similarity and diversity between two finite sets Ck, there are NW represents the total number of such pairs:In the cluster The total number of pairs of distinct points in the dataset isThe C-index is defined as:SW is the sum of the NW distances between all the pairs of points inside each cluster; Smin is the sum of the NW smallest distances between all the pairs of points in the whole dataset, and there are NT such pairs: one takes the sum of the NW smallest values; Smax is the sum of the NW largest distances between all the pairs of points in the whole dataset, and there are NT such pairs: one takes the sum of the NW largest values.where The silhouette score of this sample can be written as:a is the average distance from other samples in the same category, and b is the average distance from samples in the nearest different category.For a sample, For a sample set, its silhouette score is the average of the silhouette score of all samples. The range of the silhouette score is . The closer the samples of the same category and the farther the samples of different categories, the higher the value is. A negative value of the silhouette score indicates a poor clustering performance.The Davies Bouldin score calculates the sum of the average distance of any two categories divided by the center distance of two clusters to obtain the maximum value. A lower Davies Bouldin score means a smaller intra-class distance and a larger inter-class distance. The calculation expression is as follows:si represents the average distance between each point of a cluster and the centroid of the cluster, and dij represents the distance between the centroids of clusters i and j.where The selectivity score can be defined as:Nc is the total number of clinical annotations associated with at least one feature, Nf is the total number of features associated with at least one clinical annotation, and L is the total number of associations between clinical annotations and features. When each feature is associated with one and only one clinical annotation, the maximum value of S is 1, and vice versa; the minimum value of S is 0.where Based on the benchmarking results, to make our evaluation more objective, we defined “unified score” to unify the results of each indicator as the final comprehensive evaluation indicator. The unified score is referring to the rank aggregation scheme for the Synapse Challenge (10.7303/syn6131484). This score is equal to the sum over all normalized ranking measures, it is defined asr is the rank of a method for a specific metric  and N is the total number of methods. Thus, higher scores indicate better performance. If one method was evaluated in more than one scenario , we used its average unified scores.where Additional file 1: Table S1. Performance of six supervised methods in the condition that the clusters have variable random sizes. Table S2. JI of ten unsupervised methods on simulated datasets. The results are presented as mean value of JIs. Table S3. C-index of ten unsupervised methods on simulated datasets. Table S4. Silhouette score of ten unsupervised methods on simulated datasets. Table S5. Davies Bouldin score of ten unsupervised methods on simulated datasets. Table S6. JI, C-index, silhouette score, and Davies Bouldin score of ten unsupervised methods on single-cell multi-omics datasets. The JI index is presented as mean value of JIs. Table S7. C-index of ten unsupervised methods on cancer benchmark datasets used in clustering task. Table S8. Silhouette scores of ten unsupervised methods on cancer benchmark datasets used in clustering task. Table S9. Davies Bouldin scores of ten unsupervised methods on cancer benchmark datasets used in clustering task. Table S10. Selectivity score of ten unsupervised methods on cancer benchmark datasets used in clustering task (selectivity scores greater than the average are bolded). Figure S1. Data reduction experiment on cancer benchmark datasets used in classification task. Accuracy (a), F1 macro (b), F1 weighted (c) of the six unsupervised methods for classification under 20%, 40%, 60%, 80% of the total samples in the original data, respectively.Additional file 2. Review History


As the third-generation sequencing technology, nanopore sequencing has been used for high-throughput sequencing of DNA, RNA, and even proteins. Recently, many studies have begun to use machine learning technology to analyze the enormous data generated by nanopores. Unfortunately, the success of this technology is due to the extensive labeled data, which often suffer from enormous labor costs. Therefore, there is an urgent need for a novel technology that can not only rapidly analyze nanopore data with high-throughput, but also significantly reduce the cost of labeling. To achieve the above goals, we introduce active learning to alleviate the enormous labor costs by selecting the samples that need to be labeled. This work applies several advanced active learning technologies to the nanopore data, including the RNA classification dataset (RNA-CD) and the Oxford Nanopore Technologies barcode dataset (ONT-BD). Due to the complexity of the nanopore data (with noise sequence), the bias constraint is introduced to improve the sample selection strategy in active learning. Results: The experimental results show that for the same performance metric, 50% labeling amount can achieve the best baseline performance for ONT-BD, while only 15% labeling amount can achieve the best baseline performance for RNA-CD. Crucially, the experiments show that active learning technology can assist experts in labeling samples, and significantly reduce the labeling cost. Active learning can greatly reduce the dilemma of difficult labeling of high-capacity nanopore data. We hope active learning can be applied to other problems in nanopore sequence analysis.https://github.com/guanxiaoyu11/AL-for-nanopore.The main program is available at Bioinformatics online. Recently, nanopore sequencing technology has been recognized as the most advanced third-generation sequencing platform due to its long read duration of macromolecules and high resolution of single bases (Mycobacterium smegmatis porin A (MspA), which consists of rigid β-barrel structures. In a previous work  RNA changes in nanopores with high accuracy. (ii) Deep learning methods: 3N4) nanopore data. Indeed, nanopore sequencing technology has played a crucial role in sequencing the entire human genome. Unfortunately, due to the inevitable signal distortion of the sequences generated by nanopore sequencing technology, it is more difficult to analyze the sequence data only by manual processing. Fortunately, with the advances in computing power and machine learning , it has become possible to analyze the long-read nanopore sequences. Machine learning analysis of sequences can completely outperform traditional complex manual processing methods, which is why it is widely used in the field of nanopore sequence analysis. Machine learning algorithms used in the field of nanopore can be divided into (i) Traditional methods: These machine-learning algorithms have enabled an unprecedented breakthrough in applying nanopore sequencing to various biological tasks. The success of the above methods is often based on repeated iterative training with huge annotation datasets. However, data annotation often requires high sample labeling costs . In order to realize the exponential acceleration of labeling efficiency, we are the first ones to introduce active learning (AL)  from previous work .We apply the AL strategy to other nanopore datasets, which confirms that the AL strategy can be used in more biological nanopore scenarios.We add the threshold evaluator to evaluate the model is not locally optimal when optimal performance is achieved.This section will briefly explain the problem of nanopore data analysis and several AL algorithms, concretely including five essential contents (i) explicitly explain the problem of RNA type prediction and the ONT barcode classification; (ii) briefly recommend several commonly used AL algorithms and emphatically dive into the details of the margin sampling AL algorithm; (iii) briefly expound on the AL strategy mechanism applied to the problem of RNA type prediction.Before studying nanopore data, we need to depict the related concepts of nanopore data. Obviously, nanopore data are considered as a continuous time series, which means that several concepts of processing time series can be mapped to the nanopore data analysis. For the problem of RNA type prediction, the origin of the data is generated by sequencing six RNA molecules in the nanopore devices.S represent the original time series and T represent the RNA types. The input long sequence S is truncated to n sub-sequence T is truncated to n sub-targets vi) of the sub-sequence is extracted by feature extract methods, which contains the length, mean, standard deviation and other statistical information. Traditionally, the RNA types prediction problem can be defined as a classification task:y is the predicted RNA type, and f is the mathematically rigorous function mapping .In general, supposing that the symbols L-layer CNN model, the ONT barcode classification problem can be defined as:lθ is the parameters of the lth layer and lth layer. The lx is the output of lth layer, and the l-1x is the input of (l − 1)th layer.In contrast, the data type of the ONT barcode classification problem is the same . QS can be briefly summarized as the following six commonly used strategies: (i) Uncertainty Sampling; (ii) Query-by-committee-based queries; (iii) Query based on Expected Model Change; (iv) Expected Error Reduction based query; (v) Query based on Variance Reduction; (vi) Query based on Density Weighting Methods.QS can be based on a single machine learning model or multiple machine learning models, depending on the actual situation. On the whole, the significance of the AL is to reduce the labeling cost and rapidly improve the effectiveness of the model. Therefore, the AL strategy can be simplified by selecting the best QS for the specific task.In recent years, active learning has been widely studied using different approaches to address the data problem. Classical active learning approaches use either pool-based or query synthesis methods. In query synthesizing approaches, generative models are used to find the most informative samples . Pool-baUncertainty sampling is one sort of QS that selects the ‘indistinguishable’ sample data through the model and provides those select samples to the field specialist for annotation. The key to the uncertainty sampling method is how to describe the uncertainty of samples. Uncertainty sampling commonly contains three select ways: (i) Least Confident, (ii) Margin Sampling and (iii) Entropy Method.θ represents a trained set of machine learning model parameters. For x, the The Least Confident method is to select those samples with the lowest probability and mark them with the mathematical formula:x, respectively.Margin sampling refers to selecting data samples that can easily be categorized into two categories or have a similar probability of being categorized into two categories. Edge sampling is to select the sample with the slightest probability difference between the largest and the second-largest predicted by the model, which is described by the mathematical formula:In mathematics, entropy is used to measure the uncertainty of one system. A higher entropy value means the enormous uncertainty of the system, and contrary, lower entropy means less uncertainty. Therefore, for Entropy Method, sample data with high entropy can be selected as undetermined annotation data, which the mathematical formula can express:x determined by the model. While Least Confident only considers the maximum probability, the margin sample considers the maximum and the second-largest probabilities.Compared with Least Confident and Margin Sampling, the Entropy Method considers the results of all categories of To further illustrate the application of the AL strategy in the nanopore field, we briefly describe the overall process of applying the AL strategy to RNA-CD, as shown in L, C, Q, U, S) in L is set to ten samples to train the model, as shown in C is specially set as the RF algorithm in the RNA types prediction experiment, as shown in Q contains six common strategies: query-by-committee (QBC) is based on the QS , and these samples sent to the model will affect the final classification task. Therefore, in the nanopore field, AL selects samples from the sample pool that are easy to label or learn. Here, we design a specific bias constraint for AL of nanopore data based on the idea in this article  Some samples are easily mislabeled (noise sequence); (ii) Some samples perform negative optimization of the model. Therefore, in nanopore field, the sampled M points should be easy for the model to learn or easy for the expert to label. First, we use the traditional t-SNE method for feature mapping, select the mean and variance of the mapped features, and set the center point of the existing training dataset for feature mapping as The bias constraint is added to the unlabeled sample pool In the experimental section, first, we depict the two datasets (RNA-CD and ONT-BD) in Section 3.1 and briefly illustrate the experimental settings related to the dataset allocation and model configuration in Section 3.2. Secondly, we show the experimental results of six AL comparison methods in Section 3.3 and briefly analyze the strengths and weaknesses of the AL experimental results in the nanopore field in Section 3.4. Finally, in Section 3.5, we elaborate show the effectiveness of the AL in terms of loss of labeling time and difficulty of labeling.Before the AL experiment, we need to briefly present the dataset used to evaluate the effectiveness of AL. We primarily use two nanopore sequencing datasets in this work, RNA-CD and ONT-BD.For RNA-CD, we use the RNA sequencing data from the previous publication .For ONT-BD, the barcode data come from the paper  is the first one, indicating that the labeling cost of the AL method is reduced compared to the full complete sample (FS). The SR can be defined as follow:We report experimental results from two datasets (three models) on the different AL sampling strategies. The eight AL sampling strategies: QBC, Random, QUIRE, Density, LAL, SPAL, EER and UNC. We perform a series of experiments to show the performance of the two datasets (three models) with the different AL sampling strategies. The results of the Confusion Matrix are shown in For the RNA type classification task, we first use the randomly selected initial 10 samples to train the RF and S2Snet model, and use different sampling strategies to select ten samples for specific queries in each iteration. Therefore, the classification performance gradually improves as the iteration time increases. We first check the performance of the different AL sampling strategies for two nanopore data. The results show that UNC performs better than other methods, as shown in For RNA-CD datasets, only about 15% of the labeling rate is required to achieve the optimal performance. However, for ONT-BD datasets, 50% of the labeling rate is required to achieve the optimal performance. We analyze that there are two main situations that cause this difference: (i) The different capacity of the dataset. (ii) The task of the dataset is different. The task of ONT-BD is relatively difficult; the task of RNA-CD dataset is relatively easy.From the experimental results, the performance curve becomes stable after reaching the optimal performance (upper-bound line), which also confirms that the results of the active learning algorithm in the field of nanopore are similar to those of other active learning applications. As the number of iterations increases, the performance does not vary much. For non-convex models, the local optimal performance is similar to the global optimal performance, and the global optimal cost is higher, so many existing models have not been optimized to the global optimal solution. For convex models, active learning (semi-supervised learning method) will not fall into local optimization. Therefore, when the performance of the model is not improving, it is a local optimal solution for the non-convex model and a global optimal solution for the convex model.Obviously, the performance of UNC is better than that of other AL sampling methods. Moreover, the experimental results show that the AL strategies can effectively reduce the labeling rate of the samples and maintain the training performance for the full ground truth. Indeed, the AL strategies can be applied in the field of nanopores to reduce the cost of specialists.The use of active learning in nanopores has two goals. The first is to reduce sample labeling costs incurred by field professionals in manually distinguishing event categories. The second objective is to use AL to deepen the understanding of the selected samples to achieve the performance of the training model with complete ground truth. The experimental results show that the strategies of AL can accomplish the first goal. In this section, we primarily analyze the second objective: how to achieve the best performance of the subsamples selected by AL.In addition, it is interesting to analyze the feature importance of the results output by the full ground truth training model and the sample training model selected by UNC to achieve the full ground truth training performance, as shown in The main function of the AL strategy is to select the whole sample set selectively. The above experimental results show that the AL method UNC achieves the best experimental performance. Therefore, we analyze the samples selected by the AL strategy UNC. In RNA-CD, the samples are obtained by truncating the long sequencing signal. However, due to the influence of the sequencing noise, there are many noise samples in the truncated signal, which are not conducive to the final classification performance. Eliminating the proportion of noise samples after truncation will play a very positive role in the final classification performance. Therefore, we investigate whether AL can guide the initial sequencing signal truncation. We select the samples selected by UNC active learning for analysis, of which Similarly, for When classifying RNA types, we perform the statistical analysis of the selected samples. As shown in 6-carboxymethylguanine (O6-CMG) is a highly mutagenic alkylation product of DNA that induces transition mutations relevant to gastrointestinal cancer. First, we cut the original data of the nanopores used in the paper (To verify the scalability, we use the other two nanopore datasets to implement the AL strategy and analyze how sample labeling costs can be reduced under the premise of ensuring optimal performance. The first dataset is from (6-CMG dataset, 22% of the AL labeling samples are required to achieve the optimal performance. This experimental result also guarantees that the AL strategy is universal and has good scalability for the universal nanopore dataset.The experimental results are shown in In this article, we use bias constraint to restrain the samples that are difficult to label or learn. We perform an ablation study on two variants of UNC, which is the best AL strategy for the three nanopore datasets. The metric is SR, which is the percent reduction in labeling cost of the AL method compared to the full sample. The results are shown in O and radius r, and mapping the 10 samples selected by the current AL to t-SNE according to the mean value and variance of the feature. Finally, the threshold T is evaluated according to the number of samples with the distance d > r between the mapping point and the center of the feature O. If the number of d > r is larger than T, there is a difference between the selected samples and the training dataset. The model needs to be trained for the new samples, which is not optimal for the model. If the number of d > r is less than T, there is no significant difference between the selected samples and the training dataset, and the model no longer needs to be trained. At this time, it can be ensured that the model is in the optimal situation, and the subsequent iteration will have little impact on the model. The number of categories n in each dataset is different, the threshold T needs to be adjusted according to the number of categories n. Through the verification of the experimental results of the three datasets, when the model performance is not improving, the threshold value is set to T = n/2, and the number of AL iterations obtained is similar to the number of AL iterations for the optimal performance obtained from the model training under full ground truth.In fact, because it is not possible to label all samples in the real world, so we do not know where the best performance of the model. When AL is in constant iteration, in order to stop the iteration, the basis of judgment is that the performance of the model is no longer improved. To ensure that the current model is not locally optimal, we set a threshold evaluator to evaluate the difference between the new sample selected in the current round of iteration and the new training dataset. If the difference is smaller than threshold value, the current model does not need to be trained further, and the model has reached the optimal performance. To achieve the above objectives, we carry out the following series of operations. First, we implement AL iteration to make the model reach the performance is no longer improved. Then, mapping the samples in the current training dataset to t-SNE according to the mean value and variance of the feature, construct a circle with center The experimental results are shown in To evaluate the effectiveness of the AL strategy on labeling costs, we experiment on RNA-CD with the UNC AL strategy. The biologists label the entire RNA-CD label and selected samples using the AL strategy. At the same time, they evaluate the complexity of labeling. In the experimental configuration, we set five difficulty indicators: ‘Very easy’, ‘Easy’, ‘Medium’, ‘Hard’ and ‘Very hard’. The RNA-CD labeling information is shown in We use two sets of experimental data, namely the professional labeling of experts in the nanopore field and labeling of non-experts by simple learning. The labeling results show that experts in the nanopore field label relatively quickly, while non-experts take longer to label. For samples with different difficulties in the sample labeling process, the labeling experts perform subjective analysis according to the previously labeled sample rules. Inaccurate labeling occurs in the labeling of samples or the labelers get tired during labeling process and the samples are labeled incorrectly. Therefore, a small number of samples will be mislabeled in all labeled samples.Similarly, biologists need to label the selected samples again and evaluate the complexity of the selected samples. The labeling information of the selected samples is shown in In this work, we apply active learning to the nanopore field. We verify the active learning strategies on the RNA type classification and ONT barcode datasets. The experimental results show that the active learning strategies can drastically reduce the labeling cost. Moreover, in the machine learning phase, the active learning strategies can help nanopore specialists understand which samples are crucial for the classification task. The extent to which active learning can be applied to nanopore dataset analysis has yet to be fully demonstrated, but we believe that the initial validation performed in this work is promising for future applications. This work intends to inform biologists on how to understand and utilize active learning technology.btac764_Supplementary_DataClick here for additional data file.


The identification of effective novel biomarkers is emergently needed in colon cancer patients. In the present study, firstly we predicted that CHGA could be a biomarker for colon cancer based on the protein–protein interaction network of all the reported biomarkers that were collected from our colorectal cancer biomarker database (CBD). Then we verified our results using a diagnostic test in gene expression data and an immunohistochemistry test. The results of this study suggest that a loss of CHGA expression from the normal colon and adjacent mucosa to colon cancer may be used as a valuable biomarker for early diagnosis of colon cancer patients.Background. The incidence of colorectal cancers has been constantly increasing. Although the mortality has slightly decreased, it is far from satisfaction. Precise early diagnosis for colorectal cancer has been a great challenge in order to improve patient survival. Patients and Methods. We started with searching for protein biomarkers based on our colorectal cancer biomarker database (CBD), finding differential expressed genes (GEGs) and non-DEGs from RNA sequencing (RNA-seq) data, and further predicted new biomarkers of protein–protein interaction (PPI) networks by machine learning (ML) methods. The best-selected biomarker was further verified by a receiver operating characteristic (ROC) test from microarray and RNA-seq data, biological network, and functional analysis, and immunohistochemistry in the tissue arrays from 198 specimens. Results. There were twelve proteins  which were predicted by ML as colon cancer candidate diagnosis biomarkers. These predicted biomarkers showed close relationships with reported biomarkers of the PPI network and shared some pathways. An ROC test showed the CHGA protein with the best diagnostic accuracy (AUC = 0.9 in microarray data and 0.995 in RNA-seq data) among these candidate protein biomarkers. Furthermore, immunohistochemistry examination on our colon cancer tissue microarray samples further confirmed our bioinformatical prediction, indicating that CHGA may be used as a potential biomarker for early diagnosis of colon cancer patients. Conclusions. CHGA could be a potential candidate biomarker for diagnosing earlier colon cancer in the patients. Colon cancer contributes to cancer mortality and morbidity . In 2020Biomarkers are biological indicators for special clinical conditions or states, which have been reported many times, improving the diagnosis of colon cancer ,7. In prColon cancer and rectal cancer have many similar features in both genotype and phenotype, which are always grouped as colorectal cancer (CRC) . It has Network topology analysis is an important component of system biology study . Many reMachine learning (ML) has been applied in bioinformatics and complex network analysis for many years . SupportWith its high robustness, low heterogeneity, and extensive adaptability, bioinformatics (dry lab) experiments have become a new focus in the biomedicine field, especially in cancer biomarker discovery . TraditiCHGA) is a member of the grain family of neuroendocrine secretory proteins, and it is located in secretory vesicles of neurons and endocrine cells such as islet beta-cell secretory granules in the pancreas [CHGA gene.Chromogranin A or parathyroid secretory protein 1  test, immunohistochemistry (IHC), and biological network and function analysis were conducted to make the verification and confirmed that CHGA could be a future biomarker for colon cancer diagnosis. Meanwhile, the multiple biomarkers consisting of the 12 predicted biomarkers have been suggested with high diagnostic accuracy. Further, the diagnosis and prognosis value of CHGA in both colon and rectal cancer were evaluated, which indicated that CHGA could be a promising diagnostic biomarker but not a prognostic biomarker in CRC.The present study included 198 specimens consisting of 22 biopsy primary tumors, 55 surgical primary tumors, 22 metastatic lymph nodes, 46 adjacent normal mucosa specimens (adjacent to the primary tumor on the same histologic section), and 53 distant normal mucosa specimens from the proximal or distal margin (4–35 cm from the primary tumor) of the resected colorectum. All the patients were from the Southeast Swedish Health Care region, Sweden. The detailed characteristics of the patients and specimens are presented in CHGA expression was determined by IHC on 5-μm TMA sections as described previously . Brieflyhttp://gepia.cancer-pku.cn/index.html accessed on 20 May 2020), which concluded normalized and comprehensive high-throughput RNA sequencing (RNA-Seq) data from the Cancer Genome Atlas (TCGA) and Genotype-Tissue Expression (GTEx) database. A total of 275 colon cancer patients and 349 normal controls were included. A linear model and the empirical Bayes method were used to calculate the DE genes (DEGs) by the limma package in R. p-value < 0.05, and |Log2FC| > 1 was selected as the cut off for the DEGs. All the DEGs, along with their statistics results, can be found in the We downloaded the colon cancer differential expression (DE) data from the GEPIA  database (The Human PPI network (confidence > 0.7) was downloaded from the String database via the NDEx public server. Colon cancer diagnostic protein biomarkers were downloaded from the CBD database .The Gene Expression Omnibus (GEO) database provided the microarray data named “GSE 44861” for verification of candidate biomarkers, which contained 111 colon tissues from tumors and adjacent noncancerous tissues. These GE data were from GPL3921 Platform.p-value. In the greedy searching, firstly every p-value of DE gene was transferred to a z-score using the Stouffer’s Z-score model based on the inverse normal cumulative distribution. A smaller p-value will have a higher z-score, which means that the genes with higher z-scores are more related to colon cancer. Then a k-subnetwork will be given a z(A):iz is a random gene, and k is the number of genes, on the subnetwork. We selected the subnetwork with the highest summary z (A) after ten iterations. This subnetwork was constructed by every highly scored DE genes along with one of its neighbor genes. Here we named this subnetwork as colon cancer-specific PPI network (CCS-PPIN).The colon cancer DEGs were transferred to protein by searching in the NCBI protein database then mapped to the Human PPI network. The greedy search algorithm of jActiveModules in Cytoscape was used to find the most highly scored subnetwork from the human PPI network according to colon cancer patients DE genes’ Several network topology features were selected from the CCS-PPIN: Average shortest path length, betweenness centrality, closeness centrality, clustering coefficient, degree, eccentricity, neighborhood connectivity, number of directed edges, radiality, stress, and topological coefficient. The definition of these network features was shown in A total of 31 diagnostic protein biomarkers that were collected from the CBD database were found on the CCS-PPIN. A total of 31 non-DE proteins in the CCS-PPIN were randomly selected as the control group. We took 22 biomarkers and 22 non-DE proteins as the training set to establish an SVM model to predict biomarkers and another nine biomarkers and nine non-DE proteins as the test set to test the model performance. A regression tree is an ML method that combines the advantages of a decision tree and regression. The aim of the regression tree is to find the best features and their cut off to classify the target. The regression tree that is implemented by R package “rpart” was utilized to choose the useful network features to distinguish the biomarkers from non-biomarkers. We inputted the 11 original calculated network features together with the category information of biomarkers and non-biomarkers in the training datasets, into the regression tree model. Then the model selected the best network features, which could best distinguish the biomarkers and non-biomarkers. The selected features would be used as the input features for the SVM prediction model.SVM is a popular supervised machine learning method for classification issues. SVMs can efficiently perform nonlinear classification using the so-called kernel trick, which implicitly maps its inputs into a high-dimensional feature space. Using the selected features by regression tree, SVM was used to construct the topology model to predict new biomarkers in the CCS-PPIN, which was conducted by R package “kernlab”. The biomarkers and non-biomarkers in the training data were used to construct the model and test data was used to evaluate the model performance. We tried eight different kernels to train the SVM model to get the best prediction accuracy. The area under the curve (AUC) on the receiver operating characteristic (ROC) curve was used as an indicator to evaluate the models. A total of four kernels showed good performance (AUC > 0.7): Bessel with an AUC of 0.765; Spline with an AUC of 0.728; Hyperbolic tangent and ANOVA RBF with an AUC of 0.716. Another two kernels showed normal performance (AUC > 0.6): Radial Basis and Laplacian. Polynomial and Linear showed low performance. (AUC < 0.6) Finally, Bessel kernel was chosen as the Kernel function in the SVM. A total of 2401 DE proteins in the CCS-PPIN was selected to predict new biomarkers. .The ROC curve was used to identify the predicted biomarkers from the SVM model using the patients’ data that was provided by the GSE 44,861 microarray data. The AUC of the ROC curve was recorded to compare the diagnostic accuracy of candidate biomarkers.PPI network analysis, Gene Ontology (GO) annotation, and KEGG pathway enrichment that were performed by String and Gluego on Cystoscope were conducted to analyze the candidate biomarkers that were calculated from our SVM model and confirmed biomarkers from the CBD database in biological interaction and function level.The predicted biomarkers were collected to combine as multiple biomarkers by logistic regression. The ROC curve was drawn to test the diagnostic accuracy of multiple biomarkers.p-value and Log2FC. In total, 5562 colon cancer DEGs were identified based on the A regression tree was conducted to select useful parameters for the SVM model among the 11 original network features. Finally, the clustering coefficient, betweenness centrality, and stress were selected B.We tried different kernels to train the SVM model using training data and predict the test data. ROC curve was selected to calculate the perdition accuracy C. With iROC analysis that was performed by gene expression data was used to test the diagnostic value of candidate biomarkers that were predicted by the SVM model. A total of 11 predicted biomarkers were found on the GPL, and they all showed high AUC (bigger than 0.5). Among them, CHGA had the best AUC (0.9). Scatterplots and boxplots of the network features of the predicted model for the predicted biomarkers and other genes are shown in We used the String database to explore the relationship between these predicted biomarkers in the PPI network and biological pathways. In order to investigate the relationship between the already reported biomarkers from the CBD database and the newly predicated biomarkers, we mapped them together in the human PPI network B. We fouWe performed the KEGG pathway enrichment analysis for the confirmed and predicted biomarkers and mapped them together with the results in GO annotation in biological process, Cellular component, Immune system process and Molecular function level were conducted , and we We combined the predicted biomarkers as multiple biomarkers via logistic regression and using AUC analysis to test its diagnostic value. The ROC on the AUC curve of multiple biomarkers was 0.964 E.With its best performance in a diagnostic test, CHGA was selected to make further verification. p-values on survival test: 0.24, 0.38, and 0.13, respectively).Colon cancer is one of the most common types of cancers, and patients with advanced stage cancer have a poor prognosis . ColonosAs mentioned in the introduction, both dry and wet experiments have their significant advantages. However, dry experiments are always doubted with their false positives, and wet experiments are limited by their laboratory environments. Hence, more and more scientists suggest combining the wet and dry experiments together, by which to make the results of studies more comprehensive and credible. In the present study, we used two ML methods (regression tree and SVM) to construct the biomarker prediction model based on the PPI network topology features and predicted CHGA as a novel diagnostic biomarker, which was further verified by IHC. A regression tree was used to find the best features of the PPI network, which were selected as the final features for the SVM prediction model. The kernel is an essential part of SVM. We tried eight different kernels in the SVM prediction model and tested their prediction accuracy using the ROC test. Finally, the “Bessel” kernel was selected with its 0.765 AUC.Recently, many bioinformatics studies have used ML algorithms to predict new biomarkers ,23. CompWe have identified the diagnosis value of CHGA as a biomarker in colon cancer using meta-analysis based on gene expression data from RNA-seq and microarray, and most of these sequencing data were from colon cancer tissues . In ordeBiological functional analysis has been conducted to verify the prediction results. We found some overlapping enriched pathways for the predicted and reported biomarkers, which supported our results. Meanwhile, the results of biological function analysis inspire researchers to detect new biomarkers in these pathways.Multiple biomarkers combined by several single biomarkers have been suggested to improve the diagnosis effect in many previous studies ,18. In tWe used ML to predict new biomarkers for colon cancer diagnosis based on the PPI network and found twelve candidate biomarkers, of which CHGA showed good diagnostic performance in both gene expression data and IHC. We combined these predicted biomarkers as multiple biomarkers and showed better performance than when used alone. Further, these predicted biomarkers share some pathways with reported biomarkers, and these pathways may be pivotal pathways for further biomarker discovery for colon cancers. More importantly, CHGA may be a potential diagnostic biomarker for colon cancer patients. The combination of ML and IHC for the protein analyses can provide a more acute prediction of biomarkers in early diagnosis.