The Gamma-Poisson distribution is a theoretically and empirically motivated model for the sampling variability of single cell RNA-sequencing counts and an essential building block for analysis approaches including differential expression analysis, principal component analysis and factor analysis. Existing implementations for inferring its parameters from data often struggle with the size of single cell datasets, which can comprise millions of cells; at the same time, they do not take full advantage of the fact that zero and other small numbers are frequent in the data. These limitations have hampered uptake of the model, leaving room for statistically inferior approaches such as logarithm(-like) transformation.We present a new R package for fitting the Gamma-Poisson distribution to data with the characteristics of modern single cell datasets more quickly and more accurately than existing methods. The software can work with data on disk without having to load them into RAM simultaneously.The package glmGamPoi is available from Bioconductor for Windows, macOS and Linux, and source code is available on github.com/const-ae/glmGamPoi under a GPL-3 license. The scripts to reproduce the results of this paper are available on github.com/const-ae/glmGamPoi-Paper.Bioinformatics online. The parameter θ = 0, in which case the distribution coincides with the Poisson distribution. Larger values of θ correspond to wider distributions.The statistical distribution of sequencing counts from single-cell RNA-Seq can be modeled by son , where yμ vary across the cells. This can be done in different ways: via a generalized linear model, Biological interest is added by extending the model beyond  replicates and letting c > 0, and then proceed with analysis methods that are based on the least squares error, such as used for normal distributed data. However, this approach is fundamentally inferior as it overemphasizes the influence of small count fluctuations  of the overdispersion on datasets with many small counts.No size limitations for the datasets. glmGamPoi supports fitting the model without loading all data into RAM simultaneously (i.e. working ‘on-disk’), by using the HDF5Array  and beacSmall number of R package dependencies to facilitate use as a building-block for higher-level methods, such as factor analysis, dimension reduction or clustering/classification.pseudobulk sample, which Like edgeR, glmGamPoi also provides a quasi-likelihood ratio test with empirical Bayesian shrinkage to identify differentially expressed genes . In addiTo demonstrate how glmGamPoi can be integrated into other tools, we forked the DESeq2 package and integrated glmGamPoi as an alternative inference engine (github.com/mikelove/DESeq2/pull/24).θ (by setting it to zero) reduced the runtime to about a half. The forked version of DESeq2 that calls glmGamPoi was about as fast as calling glmGamPoi directly, indicating that inference carried out by glmGamPoi uses the largest part of the compute resources, while the additional steps carried out by DESeq2 make relatively small demands. Although all methods theoretically scale linearly with the number of genes and cells, we find empirically that glmGamPoi scales sub-linearly with the number of cells, which explains the observed performance benefit . In contrast, glmGamPoi completed after ca. 45 min .P-values from glmGamPoi and edgeR are very similar, consistent with the fact that they use the same statistical test. In P-values of glmGamPoi and DESeq2 are similar and for which genes they are different.This work was supported by the EMBL International PhD Programme. In addition, this work has received funding from the European Research Council Synergy Grant DECODE under grant agreement No. 810296.Conflict of Interest: none declared.btaa1009_Supplementary_DataClick here for additional data file.


Changes in cellular chromatin states fine-tune transcriptional output and ultimately lead to phenotypic changes. Here we propose a novel application of our reproducibility-optimized test statistics (ROTS) to detect differential chromatin states (ATAC-seq) or differential chromatin modification states (ChIP-seq) between conditions. We compare the performance of ROTS to existing and widely used methods for ATAC-seq and ChIP-seq data using both synthetic and real datasets. Our results show that ROTS outperformed other commonly used methods when analyzing ATAC-seq data. ROTS also displayed the most accurate detection of small differences when modeling with synthetic data. We observed that two-step methods that require the use of a separate peak caller often more accurately called enrichment borders, whereas one-step methods without a separate peak calling step were more versatile in calling sub-peaks. The top ranked differential regions detected by the methods had marked correlation with transcriptional differences of the closest genes. Overall, our study provides evidence that ROTS is a useful addition to the available differential peak detection methods to study chromatin and performs especially well when applied to study differential chromatin states in ATAC-seq data. Chromatin states can be seen as the collection of proteins or histone modifications that regulate the openness and activity of a given chromatin region ,2. DynamThe output of both ChIP-seq and ATAC-seq analysis consists of reads enriched in genomic locations  representing the presence of a protein in the case of ChIP-seq or open chromatin state in the case of ATAC-seq. Thus, methods first developed for ChIP-seq data analysis, such as MACS2  and HOMEIn addition to peak calling, it is of specific importance to find significant differences in chromatin states between the biological conditions or groups of interest from the ChIP-seq and ATAC-seq data. A major challenge in such differential ChIP-seq and ATAC-seq data analysis is the large search space, as it is not limited only to, e.g. protein coding genes as in gene expression analysis, but in theory, open chromatin and protein binding events take place across most of the genome. Additionally, the range of the signal does not have intrinsic boundaries like the 0 to 100% in DNA methylation, as there is no theoretical upper limit for the read enrichments. An additional challenge in detecting the differential states in the context of ChIP-seq also lies in the amount of noise generated by the immunoprecipitation step, which makes it difficult to detect subtle changes between conditions ,16. SeveThe existing methods for differential peak calling can be classified into one-step methods that inherently include the initial peak calling, and two-step methods that require the prior use of a separate peak caller to produce the peaks for differential analysis. The one-step methods can be further separated into sliding window methods , but the differential peak calling methods have been previously compared only using ChIP-seq data , constitHere, we introduce the application of reproducibility optimized test statistic (ROTS)  for robuz-statistic k by considering the reproducibility of the k top-ranked features t-type statistics The ROTS is an approach that is based on investigating the inherent characteristics of the data and thus is able to free itself from any distributional assumptions . Specifik top-ranked features over B pairs of bootstrap datasets. The reproducibility for each pair b of bootstrap data matrices    and usesWe selected five datasets for our study: two biological datasets were based on ATAC-seq [Interferon response (IFN) and Yellow Fever vaccine (YF)], two were from ChIP-seq studies (Rheumatoid Arthritis H3K4me3 and H3K36me3), while the fifth was a synthetic dataset. The four biological datasets Table  were selhttps://www.encodeproject.org/chip-seq/histone). For the comparative purpose of this study, the ChIP-seq datasets were compared without subtracting the input chromatin sample, because as such the detection of differential binding between two conditions does not require an input ChIP-seq control  and converted to fastq format with fastq-dump tool from the SRA Toolkit . Reads w control ,41. The We used the GenomicRanges R/Bioconductor package  to detecEach of the biological ATAC-seq/ChIP-seq datasets included in our study contained matching gene expression data. For ATAC-seq datasets, we used the normalized gene expression data available from the original studies; in the YF dataset the available read counts had been corrected for batch effect with ComBat , and in Evaluation of differential peak calling in real biological data is challenging, as there is no existing biological gold standard. To circumvent this, we used the correlation to gene expression data as suggested previously ,45–48. TWe compared the performance of ROTS with five popular differential peak detection methods; DiffBind, diffReps, MAnorm2, PePr and THOR, on a synthetic dataset adapted from a previous study , containWe also tested the speed and the memory consumption of the methods when run on a modern computer cluster managed by the open-source Simple Linux Utility for Resource Management. In our inspection, ROTS was by far the fastest and the most memory efficient method Figure . DiffBinThe number of significant peaks (FDR < 0.05) reported varied considerably between the different methods within each dataset Table . THOR anNext, we compared the overlap of the most significant differential peaks between the methods across each of the biological datasets, with focus on top 2000 peaks Figure .In order to compare the shape of the peaks detected by the different methods, we assessed the intensity and width of the most significant differential peaks called by each method. We used heatmaps, average read counts across the peak regions and average peak widths to evaluate the 2000 most significant differential peaks Figure . AdditioIn the ATAC-seq datasets, all methods detected peaks with clear differences in read count values between the conditions Figure  and B, wP < 2.2×10–16 for the top 2000 peaks)  overall detected narrower peaks than the one-step methods  (0 peaks) . In the 0 peaks)  and B. I0 peaks) .Visual inspection of the example loci further illustrates that results of diffReps, PePr and THOR, based on inbuilt candidate peak identification, included many lower intensity segments on both sides of the peak summit Figure . THOR anIn order to have experiment-specific external evaluation criteria, we used the closest phenotypic data, the transcriptomic data, available for each of the studied chromatin datasets as conceived previously . The undDifferential peaks detected by ROTS consistently had a high overall correlation with differential transcription across the four datasets while the behavior of the other methods was more variable between the datasets Figure . Among tAccurate differential peak calling remains a challenge in the study of chromatin and histone modification states despite recent methodological advancements. ROTS is a computational tool that has been shown to work well in the statistical analysis of several types of high-throughput omics data by bootstrapping over the data to improve the reproducibility of the results and adapting the statistical test according to the intrinsic properties of the data . In thisThe differential peak detection software compared in this study can be defined as one-step methods that include the peak calling step as part of the method, and two-step methods that use a separate peak caller to produce the initial peaks for the differential analysis. By visual inspection of the mean signals Figure  and examBased on our results, the two-steps methods relying on MACS2 input peaks were more consistent in finding the peaks with high-intensity enrichment in the middle of the peak than the one-step methods considered in this study. For instance, many of the THOR and diffReps (one-step methods) ATAC-seq peak regions showed a pattern where the highest signal of the broadest peaks was not in the middle of the peak but on the two sides of the peak centre Figure , suggestper se. However, we observed that ROTS consistently appeared among the methods reporting highest correlations across the datasets and especially in ATAC-seq data the differential peaks detected by ROTS had stronger correlation with differential transcription compared to the other methods  data where cell type or condition specific clusters may be pooled and analyzed in a similar manner as the bulk ATAC-seq data.https://bioconductor.org/packages/release/bioc/html/ROTS.html).ROTS (1.16.0) is a Bioconductor R package for differential testing in omics data THOR is part of the regulatory genomic toolbox (0.11.4) software for differential peak calling (https://github.com/shenlab-sinai/diffreps)diffReps (1.55.6) is a software for differential peak calling (https://github.com/shawnzhangyx/PePr)PePr (1.1.10) is a software for differential peak calling (https://bioconductor.org/packages/release/bioc/html/DiffBind.html)DiffBind (3.0.15) is a software for differential peak calling (https://github.com/tushiqi/MAnorm2)MAnorm2 (1.0.0) is a software for differential peak calling (https://github/elolab/Faux_et_al_NARGAB2021The code used to produce the figures in this study and to perform the differential peak calling with the different methods is available at: The ATAC-seq Yellow-Fever dataset is available on the GEO accession number GSE101609; updated 15 May 2019.The ATAC-seq IFN dataset is available on the GEO accession number GSE100383; updated 15 May 2019.The ChIP-seq RA dataset for H3K4me3 and H3K36me3 is available on the GEO accession number GSE112655; updated 19 March 2019.lqab059_Supplemental_FileClick here for additional data file.


Typical clustering analysis for large-scale genomics data combines two unsupervised learning techniques: dimensionality reduction and clustering (DR-CL) methods. It has been demonstrated that transforming gene expression to pathway-level information can improve the robustness and interpretability of disease grouping results. This approach, referred to as biological knowledge-driven clustering (BK-CL) approach, is often neglected, due to a lack of tools enabling systematic comparisons with more established DR-based methods. Moreover, classic clustering metrics based on group separability tend to favor the DR-CL paradigm, which may increase the risk of identifying less actionable disease subtypes that have ambiguous biological and clinical explanations. Hence, there is a need for developing metrics that assess biological and clinical relevance. To facilitate the systematic analysis of BK-CL methods, we propose a computational protocol for quantitative analysis of clustering results derived from both DR-CL and BK-CL methods. Moreover, we propose a new BK-CL method that combines prior knowledge of disease relevant genes, network diffusion algorithms and gene set enrichment analysis to generate robust pathway-level information. Benchmarking studies were conducted to compare the grouping results from different DR-CL and BK-CL approaches with respect to standard clustering evaluation metrics, concordance with known subtypes, association with clinical outcomes and disease modules in co-expression networks of genes. No single approach dominated every metric, showing the importance multi-objective evaluation in clustering analysis. However, we demonstrated that, on gene expression data sets derived from TCGA samples, the BK-CL approach can find groupings that provide significant prognostic value in both breast and prostate cancers. High-throughput sequencing technologies and next generation omics platforms have enabled molecular profiling of disease at an unprecedented level. Transcriptomics data obtained from RNA-Seq have been successfully used to identify clinically relevant subtypes of a number of cancer types, including breast cancer , colon cPathway-based approaches such as GSVA  and DiffAlthough pathway- and network-based approaches may lead to better patient stratification results in cancer, there is currently no protocol to support a systematic comparative analysis between DR-CL and biological knowledge-driven clustering (BK-CL) approaches. In this study, we aim at filling this gap by providing a computational framework which implements existing DR-CL and BK-CL approaches and a novel network analysis and pathway enrichment method based on a combination of random walk with restart (RWR) and gene set enrichment analysis (GSEA). Disease subtyping techniques are systematically evaluated through different criteria, which include (1) internal evaluation metrics [e.g. silhouette score (SS)]; (2) external evaluation metrics, which are based on association with known disease subtypes and batch effects; (3) functional relevance; and (4) clinical utility. Separability is important for identifying clearly distinct molecular patterns, but assessing the other criteria may be more important in order to obtain more actionable subtyping results. In this study, we decided to apply survival analysis for assessing clinical differences of the clusters. Moreover, we propose a novel disease module-based score which verifies that the clusters correlate with different network-driven gene modules. Many basic machine learning methods have been benchmarked on TCGA data (e.g. ). TherefcuratedTCGAData [caret-package [Normalized RNA-Seq data for BRCA and PRAD was downloaded with the TCGAData  R-packagTCGAData  which we-package .TCGAbiolinks [curatedTCGAData. The molecular subtypes in BRCA are based on the PAM50 assay [TCGAbiolinks PAM50 subtypes are sourced from [Subtypes and other clinical and technical variables were retrieved with biolinks  and cura50 assay . TCGAbioced from . Two samced from , 55. Proced from , 54. Theced from . The tumTCGAbiolinks. Differentially expressed genes (DEGs) were identified by BH-adjusted [The available tumor-control sample pairs were used to perform a differential expression analysis by using the negative binomial generalized log-linear model method implemented in adjusted  \documenrm (OTP)  was usedGene expression data was transformed into low-dimensional representations by using various dimensionality reduction techniques in order to efficiently test different clustering algorithms. We used four different methods: PCA , t-SNE , UMAP 88 and theGSVA  and Diffdnet-package [Next, we outline a novel topology enhanced pathway enrichment method for patient stratification. -package  was usedved from , and genved from . The retved from . For BRCClusterR-package [mclust-package [flashClust-package [cluster-package [The data sets resulting from different feature reduction techniques were then evaluated by means of clustering analysis. In total we tested six different clustering methods: k-means ; gaussia-package  was used-package . GMM fro-package  was para-package  and for -package .We assessed clustering performance by using external and internal validation metrics as well as clustering stability. In our benchmarking studies, we used the normalized mutual information (NMI) method to evaluate the agreement of a clustering result with known class labels. NMI was also used to assess the association between the clustering results and known batch effects. Samples may be processed through different protocols, depending on the practices followed by each independent laboratory. Therefore, the variable indicating the different laboratories that processed the patient samples should be considered as a possible source of batch effect . HoweverThe Cox proportional hazards (PH) model  was usedIn order to assess the mechanistic relevance of clusters, we defined a score based on gene modules derived from WGCNA. Gene modules for each data set were identified from a weighted gene co-expression network constructed from Spearman correlations between DEGs by using the soft power method . The corn-genes’ . We thenThe module score is defined as The fraction is undefined when the denominator is zero which occurs when no clusters are strongly correlated to an eigen-gene, in which case we assign a score of https://github.com/vittoriofortino84/COPS/tree/benchmark).Figure Cluster separation and cohesion were evaluated with the SS which is based on the difference in average distance of data points within clusters and between clusters. Tables On the other hand, clustering stability may be a better metric for comparing different DR-CL and BK-CL approaches, since it does not deteriorate when increasing the number of features. Tables The clustering results were evaluated based on external information of patient samples (e.g. NMI between clusters and known subtypes). Moreover, we measured how robust the grouping results were with respect to know batch effects by compiling the cNMI score . Figure We utilized co-expression driven gene modules to evaluate the mechanistic relevance of patient clusters. Gene modules were identified by using the WGCNA method and were used to assess whether the patient subgroups were associated with distinct disease-relevant gene modules. Details on the gene module detection are reported in To evaluate the survival relevance of clusters we compared Cox-regression model LRT One possible explanation for the differences in the survival relevance of the clusters between BRCA and PRAD could be the set of disease-associated genes. As mentioned in It is currently known that predicting an individual patients’ risk of recurrence or metastatic progression by using gene expression data in PRAD remains challenging . These rtrations , as recetrations , 48. ThuWe selected one of the best clustering results for BRCA to analyze further, by using the first Pareto frontier plot shown in We also selected the best clustering result for PRAD to analyze further, similarly to BRCA, by selecting one of the Pareto optimal solutions . Figure In this study, we aimed to compare data-driven with knowledge-driven methods for patient stratification from cancer gene expression data. Widely used dimensionality reduction techniques, such as PCA, t-SNE, UMAP and VAEs, were compared against methods that reduce gene-level information into pathway activation scores over a sample population. Both strategies were coupled with standard clustering algorithms, such as agglomerative or divisive HC, Gaussian mixture models and k-means. Metric fairness and usefulness are limitations which we attempted to address by introducing the module score and survival-based metrics. Different evaluation metrics were used to assess their performance including several internal and external validation measures, as well as scores aiming to evaluate the clinical utility and the mechanistic relevance of the identified patient groupings. When using the DR-CL approach, the choice of DR method had more impact than the choice of clustering algorithm. In terms of common clustering metrics, such as cluster separability, cohesion and gold-standard agreement, manifold learning methods, especially UMAP, outperformed PCA. This might be attributed to the manifolds’ ability to preserve local as well as global structure in the data . HoweverThe significance of the embedding was also apparent in the BK-CL results: the gene set collection choice was more significant than the clustering algorithm in every metric. This suggests that studying different DR and BK integration methods is relevant. BK-CL methods were outperformed by DR-CL methods when considering SS and the gene modulation score. With some exceptions, BK-CL had better results in stability and clinical relevance of the compiled subtypes. Regarding the cNMI scores, DiffRank and RWR-FGSEA methods outperformed DR-CL methods in PRAD. In BRCA it was interesting to observe that the cNMI score compiled for DiffRank-based solutions increases when considering a higher number of clusters, suggesting that pathway-driven subtyping may help the identification of finer subdivisions in known subtypes. In both BRCA and PRAD, the best clustering results did not adhere to the current clinical divisions and could therefore provide additional value. It should be noted that differences in treatment were ignored; hence, the differences could be related to differences in efficacy of treatments for patients in the clusters.In summary, no single approach dominated every metric, showing the importance of multi-objective evaluation of clustering results. Out of the five metrics, cNMI and survival analysis were perhaps most relevant since the SS, clustering stability and module score were likely to be more affected by technical choices in the methods rather than any biologically linked reasons. Pathway-based approaches are very promising and many different methods have been suggested . AlthougFileS1_Supplementary_Methods_and_Results_Rev_bbab314Click here for additional data file.FileS2_BRCA_metrics_bbab314Click here for additional data file.FileS3_PRAD_metrics_bbab314Click here for additional data file.FileS4_DRCL_DEG_metrics_bbab314Click here for additional data file.


Comprehensive, Computable NanoString Diagnostic gene panel (C2Dx) is a promising solution to address the need for a molecular pathological research and diagnostic tool for precision oncology utilizing small volume tumor specimens. We translate subtyping-related gene expression patterns of Non-Small Cell Lung Cancer (NSCLC) derived from public transcriptomic data which establish a highly robust and accurate subtyping system. The C2Dx demonstrates supreme performance on the NanoString platform using microgram-level FNA samples and has excellent portability to frozen tissues and RNA-Seq transcriptomic data. This workflow shows great potential for research and the clinical practice of cancer molecular diagnosis.The  In this era of precision oncology, there has been a rapid adoption of targeted therapy for detected driver mutations in non-small-cell lung cancer (NSCLC). As a result, there is a growing tension between the increasing tissue demands required for comprehensive molecular diagnosis and the traditional, immunohistochemical and morphology-based lung cancer subtyping approaches. Currently, the treatment of NSCLC is dependent on accurate histological and molecular subtyping as well as other clinical and pathological features , 2. ClinIn advanced and metastatic NSCLC cases, small volume biopsies are generally obtained through the least invasive diagnostic option. The morphology-based subtyping can utilize a significant amount of limited tissue and can lead to insufficient biopsy material for molecular testing. Despite the fast-growing quantity demands for diagnostic tissues, minimally invasive small volume biopsies are commonly utilized for diagnosis. The fine-needle aspirate (FNA) is now the most routinely utilized biopsy approach for NSCLC –7. WhileOver the last decade, we have been introduced to precision oncology initiatives  and novethe Comprehensive, Computable NanoString Diagnostic gene panel (C2Dx), as a promising solution to address the burning clinical need in molecular pathological diagnosis with small volume tumor specimens. The NanoString nCounter® , PARIS , UM , and the DC .The study was approved by the Institutional Review Board of the Wake Forest Baptist Medical Center from 2013 and 2015. Patients completed informed consent forms prior to enrollment. Eligibility requirements were for radiographic evidence for lung cancer or have a previously diagnosed NSCLC with potential recurrence requiring re-biopsy.Patients enrolled on study agreed to provide one additional small volume aspirate pass for RNA collection after completion of standard of care diagnostic tissue collection for presumed NSCLC. Patients underwent FNA of primary or secondary metastatic lesions. Exclusion criteria included patients whose FNA biopsy was unable to provide subtype classification by pathology, consistent with small cell carcinoma, non-malignant etiologies, or returned non-diagnostic. Small volume FNA diagnostic modalities were obtained by: endobronchial ultrasound guided transbronchial needle aspiration (EBUS-TBNA), conventional trans-bronchial needle aspiration (cTBNA) and trans-thoracic needle biopsy (TTNB). Bronchoscopy cases were performed by pulmonogists using rapid on-site cytology evaluation (ROSE) to help decipher results. TTNB were performed by interventional radiologists with training and experience with CT-guided thoracic procedures.For developing the molecular subtyping model, we established a WFBH-FNA cohort by collected 83 FNA samples (1.645 ± 2.298 μg RNA) from tumor or lymph node tissues of the Wake Forest Baptist Hospital (WFBH) non-small cell lung cancer patients, including 47 adenocarcinoma (LUAD), 25 squamous cell carcinoma (LUSC), and 11 non-small cell lung cancer but not otherwise specified (NOS) cases. Two cohorts were used for model validation: 1) WFBH-TB. We collected 44 frozen tissue samples  from WFBH Tissue Bank for internal validation; 2) TCGA. The TCGA RNA-Seq dataset ; data extraction date: September 7, 2017) of 594 LUAD and 551 LUSC samples was used for external validation.Histology classification of the 127 WFBH samples was determined through diagnostic pathological review of Haemotoxylin and Eosin (H&E) stained microscopic slides of tumor tissues by two mutually blinded pathologists and discordant cases were labeled as NOS.A 67-gene panel has been constructed and the nanoString-based transcriptomic data has been generated for the study cohort as previously reported . Among tFor NanoString-based FNA data, the raw Reporter Code Count data was used. For TCGA samples, the expressions of the 67 genes in the form of FPKM-UQ   were usei was from an LUAD patient (p) vs. from an LUSC patient (1 - p) was modeled using a logistic regression model given by:Assuming that the subtypes LUAD and LUSC followed a binomial distribution, the odd of the probability of a given sample ig ≡  was the expression levels of the m genes of sample i, and 0β and β ≡  were the regression coefficients.where iy indicate the true subtype of sample i, iy = 1 for LUAD and iy = 0 for LUSC, the objective function for the elastic net penalized logistic regression model was:Elastic net regularization  and crosn was the sample size, λ > 0 was the tuning factor, 0 ≤ α ≤ 1 was the elastic net mixing factor that controlled the mixing percentage between the ridge regularization (α = 0) and the LASSO regularization (α = 1), ||•||1 and ||•||2 represented the L1 and L2 norm, respectively. To reach a robust solution, 3-fold cross validation was used for 1,000 resampling iterations with α screened from 0 to 1 at step size of 0.1. Evaluating the model performance by accuracy, the 15-signature-gene model was selected with estimated accuracy of 0.902 ± 0.053. All FNA samples was then used to train the final 15-signature-gene classification model using logistic regression with elastic net regularization. where The classification model, with authentic coefficients, was directly implemented on the internal (44 NanoString samples from frozen tumor tissues) and external  to classify TCGA sample subtypes for evaluating model performance.The 11 FNA samples diagnosed as NOS were analyzed with the 19-gene molecular subtyping model and compared with the LUAD and LUSC cases. The logistic probability patterns of NOS, LUAD, and LUSC cases were analyzed using enrichment analysis and Kolmogoro-Smirnov test.α. The receiver operating characteristic (ROC) curves were generated for model performance on the FNA, internal tissue bank, and external TCGA cohorts and the corresponding concordance statistic   –35 were Patients’ demographics and clinical characteristics were obtained for the Exploring, Training, and Validation cohorts summarized in α. Through 1,695,000 training-and-testing rounds, the best performance of an accuracy of 0.902 ± 0.053 was achieved at α = 0.5 with a model complexity of 15 predictive genes α, λ, accuracy, and the model complexity during the first 100 resampling were shown in The original NanoString 67-gene panel data as well as the individually normalized gene profile of the Exploring cohort and FNA cohort were visualized in The final 15 signature genes as well as corresponding coefficients were listed in The intensive resampling was crucial since the sample size was relatively small (n = 72) comparing with the candidate gene set (m = 63). As shown in The model was validated internally with the 67-gene panel transcriptomics generated on the NanoString platform with frozen tumor LUAD and LUSC tissues, and externally with the RNA-Seq transcriptomics from TCGA LUAD and LUSC dataset. The performance on the WFBH tissue bank cohort and on the TCGA cohort were shown in Control of overfitting is a major challenge in molecular subtyping, since comparing with the large size of candidate biomarkers made possible by modern high-throughput or multiplex technologies. The clinical cohorts are usually small. We have significantly reduced the candidate biomarkers from microarray data to 63 genes. Such a focused gene panel still raise overfitting concern for our clinical cohort. With elastic net regularization and intensive resampling, we reduced the model complexity (gene size = 15) and achieved an events-per-variable (EPV) level of about 5. Such EPV level was around the minimum statistically acceptable  ratio foLUADθ and LUSCθ, can be used to reach required confidence. Samples falling between these thresholds need to be examined by pathologists. As shown in LUADθ = 0.84 and LUSCθ = 0.74. About 87.5% clinical samples can be subtyped at a specificity of 97% if thresholds LUADθ = 0.89 and LUSCθ = 0.66 are used.In clinical practice, it is important to identify which cases can be confidently subtyped without pathologists’ review. Subtype-specific thresholds, The classification model demonstrated high accuracy when compared to standard NSCLC clinical subtyping. The TCGA LUAD and LUSC cohort study model outperformed the histopathological image-based artificial intelligence approach using deep convolutional neural network , with an–4 for both NOS vs. LUAD and NOS vs. LUSC). In contrast, there was no similar NOS-like probability density function peak in TCGA cohort  is a promising tool to conserve diagnostic tissue specimens that require molecular pathological diagnosis when limited small volume tumor specimens are available. The C2Dx platform was developed from underlying classification models for molecular diagnosis, the NanoString gene panel to reliably generate targeted genomics data, and the clinical and experimental protocols for obtaining FNA specimens and generating NanoString genomics data. C2Dx platform demonstrates high potential in addressing the challenges in molecular diagnosis on FNA samples. The current advance of clinical practice raises a challenge to traditional pathology approaches: the gap between the dramatically reduced sample volume when less invasive approaches such as FNA are used and the fast-increasing needs of molecular diagnosis required by modern therapies such as immune therapies, targeted therapies, and, of course, chemotherapies. NanoString data generated from the clinical FNA samples can address such clinical need. Our approach provides a reliable clinical solution to this emerging clinical need.Our work proved the concept that We chose to use the LUAD vs. LUSC subtyping to prove the concept of C2Dx. First, the task is representatively complex for typical molecular diagnosis tasks, and requires a large number of genes used in a multivariate model. It is both experimentally and mathematically more complex than the standard PD-1/PD-L1 test which only requires a single or few genes. It is representative for other molecular diagnostic needs by using metagenes and gene signatures. Second, this task is suitable for examining the performance of the C2Dx platform. It has been well-established clinically, pathologically, and molecularly. Finally, subtyping LUAD vs. LUSC cases is still an essential pathological task in clinical practice and impacts treatment decisions.Beyond the use of the C2Dx platform for fresh FNA samples, the computable subtyping model can be applied for other clinical scenarios. Scenario 1: centralized C2Dx laboratory services, as many clinical sites rely on such external resources for genomics profiling tasks. Scenario 2: retrospective calibration and molecular diagnosis for frozen samples stored in tissue banks at low cost and low demands of sample volume. Our work demonstrated that reliable molecular diagnoses can be achieved for frozen samples and thus both scenarios can be archived.Finally, our subtyping model allows linking the emerging clinical FNA samples to previous patient cohorts for integrative analysis. For example, samples collected at WFBH are comparable to TCGA samples using the subtyping probability scores calculated from the logistic subtyping model on the nanoString 19-gene panel data and the TCGA’s RNA-Seq transcriptomic data. Such mapping provides comprehensive, multi-omics molecular contexts to individual clinical cases, and enables translations of knowledge from public biomedical big data to support precision oncology for individual patients.The main goal of this work is to use a well-established molecular subtyping task (distinguishing LUSC from LUAD cases among NSCLC patients) to prove the concept of a crucial step toward a comprehensive, computable gene panel for molecular diagnosis – robust molecular modeling for bench- -to-bedside knowledge translation. To be implemented in real-world clinical settings, the model needs to be expanded to distinguish small-cell lung cancer cases. Another major future direction will be the incorporation of biomarkers representing the pathological and prognostic knowledge from different domains including genomic abnormalities such as EGFR and BRAF mutations, molecular subtypes such as LUSC and LUAD, immunotherapy markers such as PD-1/PD-L1 expression, and other transcriptomic and genomic signatures for precision oncology.Our work using FNA-based nanoString gene panel for robust molecular subtyping of NSCLC patients dramatically can reduce the demand of diagnostic samples for subtyping. It can avoid the complications and risks of re-biopsy when specimens are limited. It allows for more tissues to support molecular diagnoses for emerging targeted and immune therapies, and release pathologists from subtyping burden. It may also be used in the research setting, when utilizing small volume “left over” sampling. This translational modeling work is a crucial step toward comprehensive, computable molecular diagnosis devices and potentially revolutionizes current NSCLC diagnosis procedures for precision oncologyhttp://github.com/Su-informatics-lab/FNASubtype, with sample R code demonstrated in the supplements.The original contributions presented in the study are included in the article/The study was approved by the Institutional Review Board of the Wake Forest Baptist Medical Center. The patients/participants provided their written informed consent to participate in this study.JS, LM, and JR prepared the manuscript. LM and JR designed the experiments. JS and LH analyzed the data. RB developed the github distribution of the tool. GP, JC, CB, TD, LC, IP, JH, HC, and JP contributed to sample collection and data generation. JS, LH, BPr, MC, LM, and JR participated the development of the manuscript. All authors contributed to the article and approved the submitted version.The work is partially supported by the Cancer Center Support Grant from the National Cancer Institute to the Comprehensive Cancer Center of Wake Forest Baptist Medical Center (P30 CA012197). The work is also partially supported by the Indiana University Precision Health Initiative (funded to JS). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


KIAA1549-BRAF fusion oncogenes, and we also found 16 new SIA-associated fusion transcripts. In addition, we bioinformatically simulated activities of targeted cancer drugs in SIA samples and predicted that several tyrosine kinase inhibitory drugs and thalidomide analogs could be potentially effective as second-line treatment agents to aid in the prevention of SIA recurrence and progression.In this study, we report 31 spinal intramedullary astrocytoma (SIA) RNA sequencing (RNA-seq) profiles for 25 adult patients with documented clinical annotations. To our knowledge, this is the first clinically annotated RNA-seq dataset of spinal astrocytomas derived from the intradural intramedullary compartment. We compared these tumor profiles with the previous healthy central nervous system (CNS) RNA-seq data for spinal cord and brain and identified SIA-specific gene sets and molecular pathways. Our findings suggest a trend for SIA-upregulated pathways governing interactions with the immune cells and downregulated pathways for the neuronal functioning in the context of normal CNS activity. In two patient tumor biosamples, we identified diagnostic  Spinal intramedullary astrocytoma (SIA) is a rare subtype of glioma comprising about 2%–4% of all primary central nervous system (CNS) neoplasms and approximately 6%–8% of tumors occurring in the spinal cord. SIAs are mainly observed as low-grade tumors (WHO I and II) . Five-yeSurgical resection remains the main primary treatment for intramedullary astrocytomas of the spinal cord , 5. In tOn the other hand, RNA expression profiles may serve as potent predictors of tumor sensitivity to targeted therapeutics, as shown in clinical investigations for microarray  and RNA In this study, we report 31 new SIA RNA-seq profiles for 25 patients with documented clinical annotations. As far as we know, the current study presents the first clinically annotated RNA-seq dataset of spinal astrocytomas derived from the intradural intramedullary compartment.We compared these tumor profiles with the previous healthy CNS RNA-seq data of spinal cord and brain samples , 25. We SIAs were not previously characterized on transcriptome-wide level, and in this study, we aimed to analyze RNA-seq profiles of SIA samples in comparison with healthy brain and spinal cord samples obtained from Genotype-Tissue Expression (GTEx) Portal  and AtlaOverall, 31 tumor tissue samples were taken from 14 male and 11 female donors who were diagnosed between 2003 and 2018 with SIA . Most reads reached Phred-like quality scores (Q-scores) at the Q30 level, indicating that the probability of an incorrect base call is 0.001%. The average coverage of sequencing depth reached approximately 53.45× of the human transcriptome. After alignment, 98.57% to 99.12% uniquely aligned reads were mapped to the reference human genome.To further characterize SIA transcriptomic data, we compared using principal component analysis (PCA) distributions of RNA-seq profiles among the SIA samples (n = 31) and publicly available datasets of normal spinal cord (n = 159) and brain (n = 9) tissues from GTEx and ANTE databases, respectively. PCA was performed to investigate cross-dataset compatibility in order to select proper reference group(s) for SIA comparison. The profiles from ANTE database were chosen because they were obtained using the same reagents, equipment, and protocols as for the current experimental SIA sampling . The GTE10 transformed quantile normalized gene counts. We observed tissue-specific sample clustering corresponding to the biological nature of the datasets under analysis, where SIA samples formed a separate cluster  of 1,611 molecular pathways  calculatOn the PAL-based PCA plot, we observed similar clustering as for the gene expression-based PCA -adjusted p-value <0.05] between SIA and GTEx spinal cord samples .In total, 32 genes, 23 (71.88%) upregulated and 9 (28.12%) downregulated, were commonly differentially expressed in SIA samples according to both comparisons, which supported the hypothesis that the intersections between the DEGs were non-random . We identified significantly enriched 563 functional GO terms, where 340 terms (60.39%) were for upregulated and 223 (39.61%) were for downregulated DEGs. For statistical estimates, we used Benjamini–Hochberg method for FDR correction  terms enrichment analysis  of them were activated and 196 (41.88%) were suppressed Figure 5p < 0.001  activated and 175 (44.08%) inhibited;  < 0.001 Figure 5In both comparisons, top activated pathways deal with intracellular signal transduction and with the immune response, whereas top downregulated pathways are responsible for translational regulation and neurotransmitter activities Figure 6MGMT gene products  that are currently in use for other CNS tumors could be repurposed as second-line treatment options for SIA. To this end, we utilized Oncobox method for predicting efficiencies of ATDs based on gene expression and molecular pathway activation data . This reproducts .By using Oncobox algorithm, we identified 85 ATDs with positive DS in the SIA–GTEx spinal cord comparison, and 70 ATDs with positive DS in the SIA–ANTE healthy brain comparison 8. In thWe then assessed available clinical trial reports for the top 20 DS-ranked drugs among these common 66 ATDs with the biggest DS values for different CNS tumors Figure 7On the other hand, drugs with the predicted negative drug scores that were, therefore, algorithmically not recommended belonged mainly to cyclin-dependent kinase inhibitors and androgenic and anabolic steroid (AAS) classes.Chromosomal rearrangements resulting in fusion genes and abnormal transcripts in some cases may become clinically actionable targets of specific cancer therapeutics . Fusion STAR-Fusion software  also reflect the top GO terms enriched in the SIA-upregulated gene set, i.e., the cellular response on tumor necrosis factor . Many ty-Ciclibs) and AAS hormones.In contrast, there are some cancer therapeutics we predicted to be potentially harmful for treating SIA, which mainly related to cyclin-dependent kinase inhibitors  family member transcription factors ocytomas . Moreoveocytomas . This procytomas  and overocytomas . In turnocytomas . Also, eocytomas . Thus, oKIAA1549-BRAF fusion detected for two adult SIA patients was previously relatively frequently found in pediatric pilocytic astrocytomas [47] and was reported to confer a clinically less aggressive phenotype in pediatric low-grade astrocytoma . Further quality control metrics for RNA-seq data were obtained with NCBI MAGIC software  and org.Hs.eg.db (v.3.8.2) R packages. Fusion transcripts were detected with STAR-Fusion tool  and prcomp software. Code for data analysis is available at: https://github.com/raevskymichail/SIA_analysis.SIA profiles were processed according to Suntsova et al. . STAR alsoftware –94. RNA-software , and the ] for cancer-targeted drugs was calculated according to Tkachev et al. , whose mp-value of intersection significance was calculated as an expected fraction of random intersects that is equal to or higher than the experimentally observed number of overlapping genes.To test whether an observed number of overlapping differential genes or pathways between the two intersecting datasets is significant, for every comparison, we performed 1,000 random intersections according to Sorokin et al. . In everKIAA1549-BRAF fusion transcripts in two samples and 16 new fusions each present in one SIA patient. For the first time, differential gene and molecular pathway analysis showed that the top SIA-upregulated pathways govern interactions with the immune cells, whereas the top inhibited pathways deal with normal neuronal activities. In addition, we found SIA-specific activation of molecular targets for cancer drugs: several tyrosine kinase inhibitors and thalidomide analogs. While this is a theoretical prediction, we propose that they could be further investigated as second-line treatment agents to aid in the prevention of SIA recurrence and progression.We report here the first clinically annotated RNA-seq dataset for 31 tumor tissue samples of 25 patients with SIA, a rare CNS tumor. Bioinformatic analysis revealed the presence of characteristic Transcriptomic data used across the analyses was deposited in the Sequencing Read Archive under BioProject accession: PRJNA763174. Normalized gene counts are presented in The studies involving human participants were reviewed and approved by the Ethical Committee of the Burdenko Neurosurgical Center. The patients/participants provided their written informed consent to participate in this study.Software: MR. Visualization: MR. Validation: MR. Writing—original draft: MR, MS, and AB. Writing—review and editing: MR, MS, and AB. Investigation: MR. Data curation: MR. Conceptualization: MS and AB. Methodology: MS. Supervision: MS. Project administration: AB. Patient treatment: NK, DA, SK, and ST. Tissue sample collection: NK, DA, SK, and ST. Writing—review and editing: SK and ST. All authors have read and agreed to the published version of the article.This research was funded by The Russian Foundation for Basic Research, grant number 18-29-01042.Author MR was employed by Omicsway Corp and AB was employed by Omicsway Corp. and Oncobox Ltd.The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


Due to the lack of accurate markers of disease progression, many women with detected DCIS are currently overtreated. To distinguish those DCIS cases who are likely to require therapy from those who should be left untreated, there is a need for robust and predictive biomarkers extracted from molecular or genetic profiles. We developed a supervised machine learning approach that implements multi-omics feature selection and model regularization for the identification of biomarker combinations that could be used to distinguish low-risk DCIS lesions from those with a higher likelihood of progression. To investigate the genetic heterogeneity of disease progression, we applied this approach to 40 pure DCIS and 259 invasive breast cancer (IBC) samples profiled with genome-wide transcriptomics, DNA methylation, and DNA copy number variation. Feature selection using the multi-omics Lasso-regularized algorithm identified both known genes involved in breast cancer development, as well as novel markers for early detection. Even though the gene expression-based model features led to the highest classification accuracy alone, methylation data provided a complementary source of features and improved especially the sensitivity of correctly classifying DCIS cases. We also identified a number of repeatedly misclassified DCIS cases when using either the expression or methylation markers. A small panel of 10 gene markers was able to distinguish DCIS and IBC cases with high accuracy in nested cross-validation (AU-ROC = 0.99). The marker panel was not specific to any of the established breast cancer subtypes, suggesting that the 10-gene signature may provide a subtype-agnostic and cost-effective approach for breast cancer detection and patient stratification. We further confirmed high accuracy of the 10-gene signature in an external validation cohort (AU-ROC = 0.95), profiled using distinct transcriptomic assay, hence demonstrating robustness of the risk signature.Ductal carcinoma  To investigate the molecular, genetic, and epigenetic heterogeneity of disease progression, we applied the regularized approach to 40 DCIS and 259 IBC samples, profiled with genome-wide transcriptomics, DNA methylation, and DNA copy number variation. For economic clinical implementation, we further investigated the effect of the number of model features on the classification accuracy with each omics measurements. In doing so, we identified a minimal risk signature of 10 highly predictive and subtype-agnostic transcriptomic markers, originating from a single omics platform (microarrays), which could be developed as a decision support tool in clinical practice. We further validated our minimal risk signature in an independent validation cohort (with RNA-seq data) and studied how the signature predicted also lesions between DCIS and IBC classes, as well as relapsing DCIS cases.As a model training data, we used multi-omics molecular and genomic profiles combined from three patient cohorts, Oslo2, Uppsala, and Milan . Each paN = 44,263 CpGs)  . These C63 CpGs) . The sec10(RNAseq count + 1), and it was centered for each gene around the mean of the pseudocounts. The validation cohort included 18 pure DCIS cases and 20 IBC cases, as well as 16 micro-invasive (MI) DCIS cases, which are DCIS lesions with invasive foci of maximum 1 mm.The validation data set was collected at Institute Curie, France (referred to as Curie Cohort), where the gene expression was profiled using RNA sequencing with the Illumina HiSeq2500 sequencer. The read counts were normalized with the rlog and cpm options in edgeR (v3.1.2) and DESeq2 (v1.4.5) R-packages, respectively . PseudocOur main objective was to identify the most discriminating molecular and genetic differences between DCIS and IBC, regardless of their intrinsic subtype and the nuclear grade. We initially constructed Lasso, Support Vector Machine (SVM), and Random Forest (RF) models based on each type of omics data . We used the R-package “glmnet” to build Lasso models, R-package “e1071” to build SVM models, and R-package “randomForest” to build RF models . To asseTo evaluate the predictive accuracy, we used Area Under the ROC Curve (AU-ROC) and Area under the Precision-Recall Curve (AU-PRC) . MoreoveTo test whether integrating the three types of omics data improved the prediction accuracies, we combined gene expression data, DNA methylation, and DNA copy number data together in a single Lasso model. The CpG-level and gene-level methylation data were combined separately with the other data types to investigate their respective predictive contribution. To unify the scales between the different data types, we applied z-score scaling over each feature (gene or CpG) and then combined the z-scored features into a single model.To test the effect of limiting the maximum allowed number of model features on the prediction accuracy, we adjusted the parameter “dfmax” of the glmnet function, which limits the maximum number of variables in the Lasso-regularized model . We variWe considered the common features identified by the two classification models, SVM and Lasso, as robust biological signatures. To further improve the reliability of these signatures, and to avoid reporting unstable features, we considered only those features that were returned more than five times during the 10-fold CV , where each feature can be selected up to 10 times. This analysis was limited to the gene expression data only (without using z-scoring), since gene expression data was found generally most predictive.In the validation phase, we trained a new Lasso model using the subset of 10 most robust genes on the entire training set and tested its predictive power on the validation set (the Curie Cohort). Only RNA-seq transcriptomics data were available in the validation set. We used z-score scaling over each gene separately in the training and validation sets to normalize their scales between the microarray and RNA sequencing data. The model outcome was the predicted class probability in DCIS vs. IBC classification for each validation case separately.® DCIS Score that has been suggested to quantify the risk of developing an ipsilateral breast event   . The oriSome DCIS cases may never progress to IBC and will remain intraductal, while other DCIS lesions may have future invasive potential but were discovered while still intraductal. We hypothesized that even though some lesions are discovered while still intraductal, they may carry molecular or genomic changes that distinguish them from the low-risk DCIS cases that will never progress. To address the secondary questions of whether we can divide DCIS samples into two groups, low- and high-risk DCIS, and how accurately we can find those higher-risk DCIS cases that might carry the potential for future invasion, we built additional machine learning models based on gene expression and DNA methylation data, and the cases incorrectly classified by more than one model-data combinations were considered for further scrutiny. Next, we used so-called pseudo labeling, where the repeatedly misclassified DCIS cases were relabeled as IBC, then retrained a Lasso model with 10-fold nested CV and checked whether or not its classification accuracy increased, compared to the original Lasso model with the original class labeling.We started by testing various prediction algorithms, including Lasso, SVM, and RF, to classify the patient samples of the training cohort into two groups, DCIS and IBC. These algorithms were evaluated in terms of their classification accuracy and robustness in the heterogeneous omics data . In the initial runs, the classifiers were allowed to freely make use of an unlimited number of the omics features (genes and CpGs), and nested CV was then used to evaluate the predictive power of the models and the selected feature panels. In this section, we show the results of the Lasso model that performed generally the best, while the results of RF and SVM models are provided in Notably, gene expression features provided the best overall accuracy among the single omics datasets when using summary metrics AU-ROC and AU-PRC . InteresThe combined use of the three omics features in a single Lasso model using z-score scaling resulted in similar AU-ROC and AU-PRC values when using the gene expression features alone . HoweverWe next studied the effect of limiting the number of features of the Lasso model on its predictive accuracy, with the aim to investigate what are the minimal panels of biomarkers that could cost-effectively distinguish DCIS cases from IBC. A feature number limit from 2 to 50 was imposed on each data type separately and in combination, and for each limit, 10-fold nested CV was applied to investigate the classification accuracy of the Lasso models with limited number of features. Notably, already two gene features provided an almost perfect AU-ROC of 0.95 when using the expression data only , indicatWhen considering AU-ROC, the CpG methylation model performed initially worse, when compared to the gene-level methylation model, but after 30 CpGs its classification accuracy increased . The varSince the features were selected in 10-fold nested CV at each feature number limit, the model may identify in total more features than the limit, since the different CV folds may select different features. We next investigated whether the multi-omics data and the classification models could identify those DCIS-labeled samples with a potentially higher likelihood for progressing to an invasive state. Even if these DCIS samples have been originally labeled as DCIS in the diagnostic classification, they may still possess molecular changes that promote invasion later in time. In this analysis, we used Lasso and RF models, together with gene expression and CpG methylation profiles, due to their overall good performance. We considered for further investigation those DCIS cases in the training cohort that were repeatedly misclassified by these model–data combinations more than once . MisclasOut of the 40 DCIS cases, there were 19 lesions that were always correctly classified, and 11 DCIS cases were misclassified once, whereas eight and 2 DCIS cases were misclassified two or three times, respectively. We next applied so-called pseudo-labeling, where the repeatedly misclassified DCIS cases were relabeled as IBC, and then trained a new Lasso model with nested CV. Notably, such pseudo-labeling slightly increased the AU-PRC levels in the training cohort, while the AU-ROC levels remained similar to those with the original class labels . The mulSince gene expression was found to be generally the most predictive among the single omics data, we next identified the set of common genes selected by both the Lasso and SVM models using the gene expression features alone. We further required that a gene needs to be selected in more than 50% of the CV folds , with the aim to guarantee robust and stable feature selection. In total, we found 10 such common and robust genes identified as robust risk signature. Notably, each of the 10 genes had a similar direction of differential expression between the DCIS and IBC classes across the established breast cancer subtypes , suggestInterestingly, there were marked differences in the expression levels of the 10 genes across the DCIS cases misclassified as IBC . For ins® DCIS score (see Methods). Our results showed that none of these risk scores was able to accurately distinguish between DCIS and IBC cases in our training cohort , the invasiveness score (64-gene signature) and seven-gene DXg cohort . In part as DCIS . There wThe final step was to validate the 10-gene signature on an external data set, the Curie Cohort, with the aim to investigate whether the DCIS classification model generalizes also beyond the training cohort to an independent validation dataset. The Lasso model of 10 genes estimated in the full training dataset was shown to provide highly accurate classification between the DCIS and IBC cases also in the validation dataset . NotablyWe further tested how the model predicts the microinvasive (MI) DCIS cases in the validation cohort to explore whether the 10-gene signature could also distinguish the MI cases from pure DCIS and IBC cases. Interestingly, the classification probabilities of the MI DCIS cases were in between the pure DCIS and IBC classes but remained significantly closer to the pure DCIS cases , left. HTo further investigate the features of the sparse Lasso model, we plotted the expression distributions of the 10 genes on both the training and validation cohorts . After zIn our multi-omics classification analysis between DCIS and IBC, we found that the gene expression-based model features led to the highest classification accuracy alone; however, methylation data provided a complementary source of predictive signal, and it improved especially the sensitivity of correctly classifying DCIS cases, which is important for clinical application of risk signatures. No better prediction results could be obtained with any of the two-data combinations, and the gene expression data was always required for the best prediction results, indicating its high predictive contribution. Due to the challenges of acquiring fresh frozen DCIS tissue, the number of DCIS cases was much smaller in the training cohort, compared to the IBC cases. We used several computational approaches to take into account such unbalanced classification setting: (i) we used several evaluation metrics to provide multiple views into the predictive performance of the models, including precision–recall analysis, which is often considered more suitable for the unbalanced classification problem; (ii) we included only those omics features in the signature that were robustly identified using multiple algorithms and across several CV rounds; (iii) we carried out the pseudo-labeling approach to investigate whether relabeling of some of the recurrently mis-classified cases could increase the predictive performance of the model and reveal potentially high-risk DCIS cases; and finally (iv) we validated the predictive power of the signature in an external validation cohort with more balanced classes.Previous studies have found only moderate genomic and epigenomic differences between DCIS and IBC . In thisThe comparison between our 10-gene signature and traditional breast cancer risk scores further demonstrated the added value of our 10-gene markers especially for the accurate DCIS classification (high sensitivity). We note that ROR is mostly affected by proliferation, and it is highly associated with breast cancer subtypes . Our resMany of the genes included in the model have previously been identified as differentially expressed between DCIS and IBC , but theThe remaining four genes in our gene list are not directly associated with the microenvironment. For instance, RUFY3 is involved in F-actin-enriched protrusions from the cell surface and it has been shown to be involved in gastric cancer cell migration and invasion . This geSince our analyses were performed across the molecular intrinsic subtypes of breast cancer, the identified genes can detect DCIS cases, regardless of their subtype. The genes therefore represent general invasion processes, while the subtype-specific tumor progression processes may be obscured. A major proportion of breast cancer samples are Luminal A, and this is also the case in the training cohort. We have previously shown that Luminal A DCIS and IBC are highly similar at a molecular level, while basal-like DCIS differ substantially from basal-like IBC . StratifOur results of the classification analyses using the two options to represent DNA methylation (preselected enhancer and promoter CpGs related to breast cancer biology or PCA-derived gene-level methylation) suggests that few individual CpGs cannot capture enough variation for accurate prediction and that a certain number of CpGs (>30 features) are needed to represent a meaningful information identifying DCIS from IBC. Moreover, we observed that CpG-level methylation features show higher sensitivity than gene-level methylation features using the Lasso model. This result highlights the importance of both enhancer and promoter methylation for gene regulation in breast cancer. On the other hand, the gene-level methylation represents many CpGs for each gene and thus it captures more variation, but some important CpGs may be masked by the PCA summarization approach. Furthermore, classification made using only a few individual CpGs may be vulnerable to measurement noise, and this can be overcome by increasing the number of CpGs in the classifier. Using all the 450,000 CpGs led to a poor class prediction performance, likely due to model overfitting (data not shown). Since the optimal processing of DNA methylation data is still poorly understood, we hope these results will provide guidance for the community on how to use methylation features in predictive modeling, either alone or combined with other omics features.We initially tested several classification algorithms, Lasso, SVM, and RF, which all supported the importance of multi-omics profiles for increased DCIS detection sensitivity –3. The lIn conclusion, our results support the use of the 10-gene signature to reliably identify those DCIS cases that are less likely to progress to invasive disease and may therefore have potential for reducing the current overtreatment in breast cancer. Longitudinal follow-up data of the DCIS cases will be needed for prognostic validation of the signature in terms of its accuracy at identifying high-risk vs. low-risk DCIS cases, and to explore how many of the initially DCIS diagnosed cases will eventually progress to an invasive disease or become invasive recurrent.The original contributions presented in the study are included in the article/The studies involving human participants were reviewed and approved by approval numbers: 2016/433 , PG/U-25/01/2012-00001497 , and 2005/118 . The patients/participants provided their written informed consent to participate in this study.HX analyzed the data, implemented the predictive models, prepared the figures, and drafted the manuscript. TL contributed to the data analysis and predictive modeling. HB existed risk scores. HB and TS interpreted the biological results. TF provided the methylation data process. LD and AV-S provided the validation data. LD, AV-S, and TA interpreted the results. TS and TA co-supervised the work. TA designed the study. All authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Machine learning is one kind of machine intelligence technique that learns from data and detects inherent patterns from large, complex datasets. Due to this capability, machine learning techniques are widely used in medical applications, especially where large-scale genomic and proteomic data are used. Cancer classification based on bio-molecular profiling data is a very important topic for medical applications since it improves the diagnostic accuracy of cancer and enables a successful culmination of cancer treatments. Hence, machine learning techniques are widely used in cancer detection and prognosis.In this article, a new ensemble machine learning classification model named Multiple Filtering and Supervised Attribute Clustering algorithm based Ensemble Classification model (MFSAC-EC) is proposed which can handle class imbalance problem and high dimensionality of microarray datasets. This model first generates a number of bootstrapped datasets from the original training data where the oversampling procedure is applied to handle the class imbalance problem. The proposed MFSAC method is then applied to each of these bootstrapped datasets to generate sub-datasets, each of which contains a subset of the most relevant/informative attributes of the original dataset. The MFSAC method is a feature selection technique combining multiple filters with a new supervised attribute clustering algorithm. Then for every sub-dataset, a base classifier is constructed separately, and finally, the predictive accuracy of these base classifiers is combined using the majority voting technique forming the MFSAC-based ensemble classifier. Also, a number of most informative attributes are selected as important features based on their frequency of occurrence in these sub-datasets.To assess the performance of the proposed MFSAC-EC model, it is applied on different high-dimensional microarray gene expression datasets for cancer sample classification. The proposed model is compared with well-known existing models to establish its effectiveness with respect to other models. From the experimental results, it has been found that the generalization performance/testing accuracy of the proposed classifier is significantly better compared to other well-known existing models. Apart from that, it has been also found that the proposed model can identify many important attributes/biomarker genes. Cancer is one of the most fatal diseases around the globe . AccordiWith the rapid advancements in genomic, proteomic, and imaging high-throughput technologies , now it It is well-known fact that in cancer-causing cells, gene expression is either overexpressed or under expressed . So, meaUsing computational or in-silico approaches, gene expression-based cancer sample classification task has been reviewed extensively in different papers . HoweverA fairly large number of works have been already developed for sample classification. These works are divided into two categories. In the first category , the majFilter methods  select avia minimizing the classification error rate or maximizing the classification accuracy of a classifier. The wrapper methods generally achieve better classification accuracy than the filter methods because the selection of feature subset is classifier-dependent. One drawback of these methods is that these are biased to used classifiers and another drawback is that these are computationally more expensive than the filter methods as generation of the best feature subset for the high-dimensional dataset is an NP-complete problem. Due to these reasons, these methods are not applicable for high-dimensional datasets.Wrapper methods , on the In Embedded methods , the optvia selecting the most informative genes from each cluster.Apart from these methods, clustering techniques  are alsoAll the above-mentioned variants deliver comparable feature selection and classification accuracy. Quite often this type of classification models with only a few genes and with a limited number of training samples can classify the majority of training samples correctly, but the generalization capability of such classification models cannot be guaranteed . So, theApart from this problem, the microarray data is related to several uncertainties due to fabrication, hybridization, and image processing procedure in microarray technology. These uncertainties introduce various types of noise in microarray data. Due to the presence of these uncertainties with a limited number of training samples, the conventional machine learning approaches face challenges to develop reliable classification models.To overcome the above-mentioned problems, it is therefore essential to develop general approaches and robust methods. In this regard, researchers are motivated to develop the second category-based model. These are the different robust ensemble classification models  which caEnsemble methods  are a clAlthough different category-based ensemble classification models exist in the literature but these ensemble models are not capable of addressing all the above-mentioned problems  related to microarray data.In this regard, here a new Multiple Filtering and Supervised Attribute Clustering algorithm-based ensemble classification model named MFSAC-EC is proposed. In this model, first, a number of bootstrapped versions of the original training dataset are created. At the time of the creation of bootstrapped versions, an oversampling technique  is adoptvia different filters to form different sub-datasets from every bootstrapped dataset. Finally, every gene subset is modified using a supervised attribute clustering algorithm. In this way, the high-dimensionality problem of gene expression data is handled here. Apart from this, from the MFSAC generated sub-datasets, the frequency of occurrence is counted for every gene and informative genes are ranked accordingly. The prediction capability of the proposed model is experimented with over different microarray datasets and compared with the existing well-known models. Experimental outputs demonstrate the superiority of the proposed model over existing models.The novelty of the proposed MFSAC-EC model is that here the emphasis is given simultaneously on the high dimensionality problem of gene expression data, small sample size problem as well as the class imbalance problem. All of these problems at the same time are not considered in any existing ensemble classification model. First of all, due to the use of bootstrapping method with a class balancing strategy, the proposed model can handle a small sample size and overfitting problem. Second, in MFSAC, different filter methods are used with their unique characteristics. So, different characteristics-based relevant gene subsets are selected The proposed MFSAC-EC model is composed of different filter score functions, a new supervised attribute clustering method, and an ensemble classification method. In the following subsections, first, a brief overview is given on different filter score functions and then the proposed MFSAC-EC model is described.In this paper, a data set  is represented by a data matrix, The filter score functions used in the proposed MFSAC-EC model are modified Fisher score , modifieIn the proposed MFSAC-EC model, initially, bootstrapping (sampling with replacements) with a class balancing procedure of samples is applied on training dataset The MFSAC method of the MFSAC-EC model, which is an integration of multiple filters and a new supervised attribute (gene) clustering method, is applied on every newly created bootstrapped , the frequency of occurrence that means the total number of times it appears in all sub-datasets generated by the MFSAC method for all bootstrapped versions is calculated. Then according to their frequency of occurrence, those genes are ordered or ranked. The top-ranked genes with the highest occurrence frequency are considered the most informative cancer-related genes.The block diagram of the proposed MFSAC-EC model is shown in The experimentation has been carried out over ten publicly available different gene expression binary class and multi-class datasets. Among these datasets, eight datasets are cancer datasets and two arthritis datasets. The eight cancer datasets are Leukemia , Colon , ProstatIn the Lesukemia dataset , the genIn the Colon cancer dataset , gene exProstate cancer dataset  also conThe Lung cancer dataset  consistsIn Rbreast data set , the patIn the Breast cancer dataset , the genMLL  is a typSRBCT dataset  is introversus Healthy Controls is a data set  then it is considered as Leave one out cross-validation method (LOOCV). For k is equal to two, the cross-validation method is named the household method. It has been found that when k is set at a very small value that means the fold size is large then the accuracy of the classification model is affected by low bias and high variance problems. On the other hand, if k is set at a high value that means the fold size is not so large then the classification accuracy of the classification model has a high bias but low variance. It has been found that tenfold cross-validation method outperforms the LOOCV method  and testing set (1/3rd of the dataset) with 50 runs.In training-testing random splitting the dataset is initially randomly partitioned into training set  analysis . It is aClassification accuracy (SN) or True Positive Rate (TPR) can be defined as,The sensitivity  is used to represent the performance of the binary classification model. The higher AUC value of a ROC curve for a particular classification model signifies the better performance of the classification model in differentiating positive and negative examples. The range of AUC value is 0 <= AUC <= 1.A heatmap is a data representation diagram in which the values for a variable of interest are portrayed using a data matrix. In this data matrix, the values of the variable are represented across two-axis variables as a grid of colored squares. The axis variables are divided into ranges and each cell’s color represents the intensity of that variable for the particular ranges of values of axis variables.Here, the performance of the proposed classifier for multi-class datasets is judged using Heat map representation of confusion matrix , where aBefore running the MFSAC-EC, the parameters are settled down. In MFSAC-EC the input training dataset is bootstrapped. The proposed MFSAC-EC model is run here varying the number of bootstrapped datasets , NB, and KNN classifiers individually for forming different ensemble classification models. All the classifiers are implemented using Scilit-learn libraries of Python. For all classifiers, we have set parameters with default parameter values. For DT as default setting we have used splitting function = Gini, Splitting criterion = best, height = none (that means for every sample it reaches a leaf/class node). For SVM, we have used the RBF kernel function. For KNN we have chosen K (number of nearest neighbor) value from three to seven.The overall execution time of a single run of the MFSAC-EC model  and testing time using only tenfold are shown for different datasets in In In To show the generalization property of the proposed ensemble classifiers, the classification accuracy of these classifiers is also measured repeatedly with respect to the random splitting of the dataset into a training set  and test set . Random splitting is done with care such that class proportion is alike in the training set and test set. In From the results of The performance of the proposed model for different two-class datasets with respect to different parameters like SN, SP, PPV, NPV, FPR is shown in In In In In In In In The top eight genes selected by the MFSAC-EC model for Colon cancer and Leukemia are listed in In this paper, a new Multiple Filtering and Supervised Attribute Clustering algorithm-based ensemble classification model named MFSAC-EC is proposed. The main motivation behind this work is to develop a machine learning-based ensemble classification model to overcome the over-fitting problem which arises due to the presence of sample class imbalance problem, small sample size problem, and also high dimensional feature set problem in the microarray gene expression dataset, to enhance the prediction capability of the proposed model.Nowadays, in designing machine learning models, the use of ensemble methodology has been increasing day by day as it incorporates multiple learning algorithms and also training datasets in different efficient manners to improve the overall prediction accuracy of the model. Due to the inclusion of prediction accuracy of multiple learning models and also the use of different bootstrapping datasets, the chances of potential overfitting in training data is greatly reduced in the ensemble models, and as a consequence the prediction accuracy increases. One necessary condition of the superior performance of an ensemble classifier with respect to its individual member/base classifier is that every base classifier should be very accurate and diverse . A classIn our proposed ensemble model, at first, a number of bootstrapped datasets of the original training dataset is created. In every bootstrapped dataset, the class imbalance problem is solved using the oversampling method. Then for every bootstrapped dataset, a number of sub-datasets are created using the MFSAC method (which is a hybrid method combining multi-filters and a new supervised attribute/gene clustering method), and then for every generated sub-dataset, a base classifier is constructed using any existing classification model. After that, a new ensemble classifier (EC) is formed using the majority voting scheme by combining the prediction accuracy of all those base classifiers.The generation of the different bootstrapped versions of training data and also the use of the oversampling procedure to balance the cardinality of majority class and minority class in every bootstrapped dataset reduces the chances of the overfitting problem of a classifier.Different types of filter methods are used in the MFSAC method. It has been already observed that one filter gives better performance for one dataset while the same gives poor results for other datasets. This is because every filter uses separate metrics and so the choice for a filter for a specific dataset is a very complex task. As different filter methods are used in the MFSAC method, so different sub-datasets with different characteristics-based attributes/genes are formed from each dataset. This is shown using Venn diagram in Moreover, the genes selected by different filter methos are good biomarker also. In In MFSAC, at first, a sub-dataset of the most relevant genes is selected by each filter method. Then on each sub-dataset, the proposed supervised gene clustering algorithm is applied and a reduced sub-dataset of modified attributes/features in the form of augmented cluster representatives is generated. In this method, at the time of cluster formation, genes are augmented based on their supervised information. In other words, such augmentation is considered where it increases the class discrimination power. Thus effectively, the class relevance of any augmented cluster representative is greater than that of any single gene involved in that process. So, this modified sub-dataset containing a reduced feature set in the form of augmented cluster representatives is more powerful according to class discrimination power than the sub-dataset containing a subset of the most relevant genes. Apart from this, it is well known fact in gene expression data that two genes are functionally similar if they are pattern-based similar (either positively co-expressed or negatively co-expressed) . So, at Finally, for each sub-dataset with modified attributes in the form of augmented cluster representatives, a classifier is constructed using any existing classifier, and these classifiers are combined using the majority voting technique to form an ensemble classifier (EC). The use of different sub-datasets with optimal gene subsets in the form of augmented cluster representatives and the formation of a classifier for every sub-dataset can solve the overfitting problem of any single classifier. This is due to the reason that not all sub-datasets can consistently perform well on all types of cancer datasets (due to inherent characteristics of the datasets), but due to the use of majority voting in ensemble classifiers, this problem can be solved or reduced.The prediction accuracy of the proposed model is verified by applying it to high-dimensional microarray gene expression data From Another outcome of our proposed model is to rank informative genes for every cancer dataset. For this task, the frequency of occurrence of each gene present in the form of augmented cluster representatives in every sub-dataset is counted and these genes are ranked according to the counted value to measure the importance of those genes for any specific disease, here cancer. To establish the biological significance of those selected genes for every cancer dataset, their contribution has been confirmed by other existing studies where they are referred already. From these existing studies, it is clear that the selected genes are important for cancer class discrimination and also are important as cancer biomarkers for molecular treatment targets.Many machine learning and statistical learning-based classifiers for sample classification already exist in the literature, but these methods are prone to suffer from overfitting due to small sample size problems, class imbalance problems, and the curse of the high dimensionality of microarray data. Although some of the existing methods can mitigate these issues to quite an extent, the problems have still not been satisfactorily overcome. Due to this reason, here a novel feature selection-based ensemble classification model named MFSAC-EC is proposed. It has been shown that the proposed model can handle the above-mentioned issues present in existing models. To check the performance of the proposed MFSAC-EC model, this classifier is applied to test sample classification accuracy in high dimensional microarray gene expression data, a domain that will be beneficial in the field of cancer research. From the experimental results, it has been found that the proposed model outperforms all other well-known existing classification models combined with the different recognized feature selection methods and also the newly developed ensemble classifiers for all types of cancer datasets mentioned here. Apart from this classification task, the proposed model can also rank informative attributes according to their importance. The efficiency of the proposed model in this task is vindicated by finding the most informative genes for the colon cancer and leukemia cancer datasets using this model. These genes are biologically validated based on other well-known existing studies. Consequently, it is clear that the selected genes are vital for sample class discrimination and are also important biomarkers for molecular treatment targets of deadly diseases.10.7717/peerj-cs.671/supp-1Supplemental Information 1Each row of this table illustrates one of the seven filter methods used in the proposed MFSAC-EC model. Every equation along with the description defines every filer score function.Click here for additional data file.10.7717/peerj-cs.671/supp-2Supplemental Information 2For (a) Leukemia Cancer (b) Lung Cancer datasets.Click here for additional data file.10.7717/peerj-cs.671/supp-3Supplemental Information 3For (a)RAOA dataset using fivefold. (b) RBreast dataset using fivefold. (c) Prostate Cancer dataset using tenfold.Click here for additional data file.10.7717/peerj-cs.671/supp-4Supplemental Information 4Classification accuracy(%) of MFSAC-EC model along with seven filter methods using classifiers Naive Bays, KNN, Decision Tree (C4.5), and SVM are represented using different colored bars respectively for (a) SRBCT (b) RAHC (c) Prostate and (d) Colon Cancer datasets.Click here for additional data file.10.7717/peerj-cs.671/supp-5Supplemental Information 5(a) Leukemia Cancer dataset. (b) Prostate Cancer dataset.Click here for additional data file.