The advent of high-throughput transcriptomic screening technologies has resulted in a wealth of publicly available gene expression data associated with chemical treatments. From a regulatory perspective, data sets that cover a large chemical space and contain reference chemicals offer utility for the prediction of molecular initiating events associated with chemical exposure. Here, we integrate data from a large compendium of transcriptomic responses to chemical exposure with a comprehensive database of chemical-protein associations to train binary classifiers that predict mechanism(s) of action from transcriptomic responses. First, we linked reference chemicals present in the LINCS L1000 gene expression data collection to chemical identifiers in RefChemDB, a database of chemical-protein interactions. Next, we trained binary classifiers on MCF7 human breast cancer cell line derived gene expression profiles and chemical-protein labels using six classification algorithms to identify optimal analysis parameters. To validate classifier accuracy, we used holdout data sets, training-excluded reference chemicals, and empirical significance testing of null models derived from permuted chemical-protein associations. To identify classifiers that have variable predicting performance across training data derived from different cellular contexts, we trained a separate set of binary classifiers on the PC3 human prostate cancer cell line.p-values lower than 0.05 and internal accuracies ranging from 0.73 to 0.94 and holdout accuracies of 0.68 to 0.92. High-ranking predictions for training-excluded reference chemicals demonstrating that predictive accuracy extends beyond the set of chemicals used in classifier training. To explore differences in classifier performance as a function of training data cellular context, MCF7-trained classifier accuracies were compared to classifiers trained on the PC3 gene expression data for the same molecular initiating events.We trained classifiers using expression data associated with chemical treatments linked to 51 molecular initiating events. This analysis identified and validated 9 high-performing classifiers with empirical This methodology can offer insight in prioritizing candidate perturbagens of interest for targeted screens. This approach can also help guide the selection of relevant cellular contexts for screening classes of candidate perturbagens using cell line specific model performance.The online version contains supplementary material available at 10.1186/s13040-022-00292-z. Animal-based chemical screening methods have evaluated only a fraction of commercial chemicals for human safety . New appOne challenge of leveraging HTTr for chemical screening is the analysis and interpretation of large data sets in which hundreds or thousands of gene expression levels are measured across many chemicals and concentrations. Interpretation of such datasets requires time-intensive and specific application of expert knowledge or, more favorably, a systematic analysis that discounts noise and indicates the probability of “true” biological signals that are relevant to chemical hazard identification. One common strategy for distilling this wealth of expression information is to use gene set enrichment analysis to identify molecular pathways perturbed by chemical treatment . HoweverMachine learning (ML) methods provide a flexible framework to address this problem. To date, ML methods have leveraged chemical structure and gene expression data to predict chemical targets for pharmaceutical repositioning –14, and Here, we predict MIEs through a ML-based classification approach Fig. . We deveClassifiers were trained using one of six algorithms: 1) support vector machine with linear kernel (SVM_L), 2) support vector machine with radial kernel (SVM_R), 3) support vector machine with polynomial kernel (SVM_P), 4) K-nearest neighbor (KNN), 5) multilayer perceptron with multiple hidden layers (MLP), and 6) Naïve Bayes (NB). We tested several sets of input features for model training, including pathway scoring. Binary classifiers were validated by four complementary approaches: 1) 5-fold cross validation, 2) validation on holdout data, 3) empirical significance testing, and 4) evaluation of exemplar chemical activity predictions. Lastly, in the interest of testing whether MIE prediction from transcriptomic profiles is dependent on cell line, we compared the predictive accuracy of classifiers trained on data from the MCF7 breast cancer derived cell line versus classifiers trained on data from the PC3 prostate cancer derived cell line.To predict MIEs from gene expression, we trained binary classifiers using chemical-target labels from RefChemDB  and geneTo train binary classifiers to predict MIEs from gene expression, we leveraged annotations from RefChemDB , a databSome MIEs in RefChemDB are annotated for similar or identical sets of chemicals. For example, if filtered to include only annotations with a support level of at least 5, there are 83 chemicals linked to Carbonic anhydrase 1 (CA1) and Carbonic anhydrase 2 (CA2) inhibition, 82 of which are shared. This was expected, as members of the same gene family that have considerable homology (60% in the case of CA1 and CA2 ) often hTo train classifiers to predict MIEs using gene expression, we required a gene expression data set to train models that had sufficient overlap with chemicals annotated in RefChemDB. To meet this need, we used the LINCS L1000 gene expression database . This cohttps://jcheminf.biomedcentral.com/articles/10.1186/s13321-017-0247-6] (https://comptox.epa.gov/dashboard/), a web-based application providing access to chemical property data and in vivo and in vitro toxicity data for nearly 900,000 chemicals. Broad_IDs were matched to DTXSIDs in ChemReg using additional chemical descriptors extracted from LINCS metadata files, including canonical SMILES, InChIKeys, and chemical names  were downloaded from the gene expression omnibus  for LINCcaret with “center” and “scale” methods, as done in previous ML-based investigations using LINCS data [Most chemicals present in the LINCS data set have multiple associated gene expression profiles. Some chemicals were screened at multiple concentrations and with varying durations, resulting in multiple profiles for a single chemical, in addition to replicate experiments generating profiles with identical conditions. The number of profiles associated with each chemical varies from chemical to chemical, with a median of 6 profiles in MCF7 cells for the LINCS chemicals mapped to a DTXSID in RefChemDB. Previous ML-based investigations have limited the heterogeneity of training data by restricting training data sets to a single concentration and exposure duration . We sougNCS data  .caret package v6.0–83 in the R statistical framework [kernlab v0.9–27, RSNNS v0.4-12, and naivebayes v0.9.6 R libraries, respectively. Hyperparameter tuning for model optimization was performed automatically by caret using the grid search default in the ‘train_control’ function.To optimize models, we evaluated the performance of six different classification algorithms for model training. A subset of these algorithms were based on Support Vector Machines (SVMs). SVMs are a popular class of algorithm for biological binary classification problems . SVMs trramework . SVM, MLPrevious ML-based investigations using LINCS L1000 data have shown that pre-processing gene-level expression information into pathway scores can significantly improve multi-classifier performance . We therTo explore how the selection of input features affects classifier performance, we trained models using three types of L1000 derived features: 1) Moderated Z-scores for all genes (LINCS landmark and inferred), 2) Moderated Z-scores for LINCS landmark genes only, and 3) Pathway scores.The classifier training and validation pipeline is outlined in Fig. caret package in R [For each MIE with sufficient training data:Generate list of “MIE-active” chemicals associated with MIE from RefChemDBN MIE-active profiles linked to MIE-active chemical treatments in the selected cell lineGenerate collection of N MIE-inactive profiles linked to chemicals that are not annotated for the MIEGenerate collection of Select 20% each of MIE-active and MIE-inactive profiles as holdout dataRemaining 80% of profiles are designated as training data setFor each algorithm in Split model training data into 5 equal subsetsk:k subsets and train model on these datakTest model on Combine non-For each subset Train classifier using 5-fold cross validation:For each unique combination of hyperparameters (grid search)Test accuracy of model with optimal hyperparameters using holdout dataReturn internal  and holdout accuracies for optimal hyperparameter configurationIdentify optimal hyperparameter configuration that returns the highest average accuracy (proportion of correct assignments) across the 5 foldsFor each MIE, training data sets were formed using RefChemDB chemical-MIE linkages to identify and label suitable LINCS gene expression profiles. First, a collection of “MIE-active” LINCS profiles was generated by selecting LINCS profiles associated with chemicals that are linked to a given MIE. Then, an equally sized collection of “MIE-inactive” profiles was generated by randomly selecting profiles that met two criteria: 1) profiles must not be associated with chemical treatments linked to the MIE being modeled at any support level nor with any mode and 2) profiles must be associated with a different MIE with a support level ≥ 5 , 500 null data sets were generated and corresponding classifiers trained. Then, an empirical significance test was performed by calculating eviously . CandidaTo assess model performance independently of internal and holdout accuracies, we excluded a set of “exemplar” chemicals  from all training and holdout data sets, including for null classifiers. To select exemplar chemicals, we used a data-driven approach. First, we generated a list of candidate chemicals by taking the union of the top 10 chemicals for each MIE ranked by support level. We then iteratively selected one exemplar chemical for each MIE that did not yet have an exemplar selected, with the constraint that the exclusion of the chemical could not reduce the remaining training data to < 5 chemicals or < 50 total profiles for any MIE on the list. If a selected exemplar chemical was in the list of top 10 supported chemicals for multiple MIEs, that exemplar chemical was assigned to all these MIEs, and all were removed from the list for further exemplar selection. This process identified 31 exemplar chemicals assigned to 44 of the 51 MIEs. All profiles associated with these exemplar chemicals were subsequently excluded from all classifier training . A subset of associations at this support level are likely spurious , howeverIn the context of transcriptomic differences between cell lines at baseline, we hypothesized that some MIE classifiers would perform better using data from some cell lines relative to others. To identify MIEs that show cell type specific differences in prediction, we trained a separate set of models using LINCS gene expression profiles derived from the PC3 prostate cancer derived cell line. PC3 cells differ from MCF7 cells in terms of baseline gene expression and tissue of origin, and thus exhibit a different repertoire of proteins that can be perturbed by chemical treatment. Also, the PC3 cell line has the second highest number of associated gene expression profiles in the LINCS data set .Classifiers were trained using LINCS L1000 gene expression data in the form of moderated z-scores (LINCS level 5 data). These values are derived from 978 measurements of landmark genes directly measured by the L1000 assay, and inferred expression of an additional 11,350 genes . To detep < 1.2 × 10− 2). 79% of landmark gene classifiers out-performed corresponding classifiers using all LINCS genes and pathway scores for the same MIE and classification algorithm. Due to the magnitude of difference in accuracies between feature types, and the consistency of these findings across classification algorithms, we focused on landmark gene-based classifiers for further analysis.Previous ML investigations of LINCS data have shown improved performance for models trained on gene expression data that are pre-processed into pathway scores . To testp-value = 10− 8), which were the algorithms with the next highest mean accuracies of 0.67 and 0.66 respectively.Focusing on classifiers trained with landmark genes, we then compared performance across training algorithms Fig. B and C. p-value = 0.031), suggesting overfitting. Therefore, classifiers trained using the SVM_P algorithm were excluded from further analysis. In the absence of SVM_P models, there was not a clear winner among ML algorithms with respect to internal accuracies , without capturing a mechanism of action relevant to the MIE. To explore the possibility, we first compared internal accuracy with the number of chemicals used in classifier training . These linkages were previously excluded from training data sets because a previous comparison to in vitro assay results has shown that these associations are more likely to be spurious relative to linkages with a higher support level . On the To test this, we compared prediction ranks for these moderate support chemicals to the background rank distribution using a one-tailed KS test. Of the 9 confirmed high performance classifiers, 8 generated ranked predictions for moderate-support linkages that were significantly higher than the background distribution Table . Thus, c− 5). Additionally, the classifiers with the highest internal accuracies were the same in both cell types (TUB (−) and MTOR/PI3K (−)). On the other hand, a subset of models showed relatively dissimilar accuracies between cell types. The AR (+) model showed relatively higher internal accuracy when trained on MCF7 data (0.85) compared to training on PC3 data (0.49), while ADRB-1/2 (−) showed relatively higher accuracy when trained on PC3 data (0.80) as opposed to MCF7 (0.53). Because MCF7 cells are known to be estrogen-responsive, we suspected that a greater diversity of estrogen receptor-interacting chemicals may have been assayed in MCF7 cells relative to PC3 cells. Such an imbalance in training data could potentially drive differences in model accuracy between the cell types. However, an inspection of the training data for these classifiers reveals similar numbers of chemicals and profiles were available for ESR-1/2 activation in both cell types  had sufficient training data in PC3 cells. To train PC3 classifiers comparable to MCF7-trained counterparts, we used landmark gene features. In order to maximize comparability between classifiers trained in each cell line, we selected SVM_L-based classifiers for comparison. Classifiers trained with SVM_L on MCF7-derived profiles were compared with algorithm and MIE-matched PC3-derived classifiers based on internal accuracies  values from the Human Protein Atlas v20  for MIE In the current study, we trained binary classifiers to predict MIEs from gene expression profiles and optimized both input feature type and classification algorithm. By integrating LINCS and RefChemDB in a ML framework for MIE prediction, our analysis has the following primary findings: 1) We identified landmark genes as an optimal feature type for MIE prediction based on internal accuracies, 2) Of the 51 MIEs modeled, 9 MIEs generated classifiers that performed significantly better than null models and that were further validated using training-excluded chemicals and profiles, and 3) Classifier accuracy varied based on the cell line from which training data were derived, and may be associated with differences in baseline expression of MIE-linked genes.Previous studies of drug-use category prediction showed that classifiers trained on pathway scores outperformed analogous models trained on individual gene features , prompticaret) are limited to a single hidden layer due to compute time constraints. However, the training and validation framework presented in the current work can be used to test additional models and hyperparameter combinations in the future.Previous ML-based investigations incorporating LINCS L1000 data have shown that deep-learning-based approaches yielded superior accuracy in predicting general chemical use categories relative to SVM-based classifiers . We wereConfirmed high performance classifiers assigned relatively high prediction scores to the corresponding exemplar chemicals, which were completely excluded from the training data. However, some classifiers assigned high ranking prediction scores to chemicals for which they were not annotated. For example, Everolimus received a high ranking prediction for MTOR/PI3K (−), to which it is linked through RefChemDB annotations, but was also a high-ranking prediction for TOP2A (−), for which it is not annotated. These MIEs are related in that induction of either can be linked to decreased cell proliferation. MTOR (Mechanistic Target of Rapamycin) and a downstream pathway that includes Phosphoinositide 3-kinase (PI3K) is a major regulator of cell growth and proliferation, while TOP2A , is a nuclear enzyme that alters topology of DNA to facilitate replication. The high ranking prediction of Everolimus for TOP2A (−) may therefore result from the convergence of distinct MIEs onto the same key events through shared molecular consequences. While Everolimus is known to inhibit MTOR but not TOP2A, the inhibition of either of these targets is known to induce apoptosis , 35. SimAn additional limitation of the current work is sparse data on which to train classifiers. Previous ML-based investigations have leveraged more general labels capturing overall toxicity or chemical use categories to train classifiers using expression profiles associated with hundreds of chemicals. In contrast, models produced by the current work are trained using as few as 5 chemicals associated with much more specific biological activities, such as AR inhibition. One reason for this sparseness in the training data is that currently there are relatively few chemicals associated with certain MIEs with high levels of literature support. RefChemDB is the product of an automated curation process and therefore contains some spurious chemical-target associations. To ensure the validity of our training data, we limited MIE-active chemicals to those with a support level corresponding to at least 5 separate sources of evidence for the association. It is likely that a subset of chemical-MIE annotations with lower support levels are valid and could therefore be added to the available training data as additional evidence becomes available. A second contributor to the sparseness of the training data is limited overlap between chemicals annotated in RefChemDB, and chemicals surveyed in LINCS. Of the 1181 chemicals annotated in RefChemDB with a support level ≥ 5, only 765 (65%) have available data in LINCS.We addressed the sparseness of the data by first clustering similar targets into a single MIE, such as HDAC1 (−) and HDAC2 (−), using a data-driven strategy based on overlapping chemical associations in RefChemDB. This strategy generated MIEs that can easily be associated with AOPs, without unnecessarily distinguishing between gene family members for which there are a limited number of chemicals that display family member specific affinity. We also increased the size of our training data sets by leveraging multiple examples of each chemical treatment in the LINCS data, including those tested at multiple concentrations and exposure durations. We also performed an empirical significance test using null models matched to the number of chemicals and profiles used to train each MIE model. This test should identify cases where the model accuracy is likely to reflect spurious commonalities between profiles for a small number of chemicals, rather than the true transcriptional signature of the MIE.p-value = 0.03) but failed to generate a high-ranking prediction for 17-Methyltestosterone (87.66 percentile). On the other hand, 17-Methyltestosterone did return high-ranking predictions for TUB (−), ESR-1/2 (+), and TOP2A (−), none of which are linked to the chemical in RefChemDB. Consideration of the literature reveals 17-Methyltestosterone is estrogenic due to the ease of its conversion to estrogen by aromatase in MCF7 cells, which may explain the high ranking prediction for ESR-1/2 (+) [Another limitation of the current methodology was the data-driven selection of exemplar chemicals. This selection paradigm considered MIE-chemical support level, frequency of annotation across MIEs, and minimal training data set volume to identify suitable exemplar chemicals (see section 2.8). We used this data-driven approach for exemplar selection to remove our own bias of chemical-MIE linkages from the analysis. However, performing this selection without expert knowledge of metabolism and molecular crosstalk may have resulted in less than ideal exemplar chemicals. For example, our strictly data-driven method selected 17-Methyltestosterone as the exemplar for AR (+). AR (+) passed empirical significance testing (-1/2 (+) .To explore if MIE predictions differed as a function of the cellular context of training data, we compared classifiers trained on MCF7-derived data with models trained on PC3-derived data. Of the models with sufficient gene expression profiles in LINCS for training in both cell types, a subset, such as TUB (−), showed relatively high and comparable internal accuracy across cell lines. For MIEs modeled by classifiers that demonstrate comparable performance across cell lines, it may be advantageous to attempt to improve classifier accuracy further by training classifiers with an expanded set of training data derived from multiple cell lines in LINCS.While some classifiers showed relatively similar results across both cell lines, a subset of MIE classifiers showed markedly different internal accuracies as a function of the cell line on which they were trained Table . This diWhile the relative responsiveness to estrogen receptor modulators in MCF7 cells is well studied , the relThe current study used a binary classification approach to predict MIEs induced by chemical exposure based on gene expression data. We explored a variety of modeling parameters including input feature type, classification algorithm, and cellular context of training data. Candidate high performance classifiers were identified with empirical significance testing, and further validated using training-excluded exemplar reference chemicals.Systematic comparison of models generated with MCF7 and PC3 derived profiles identified MIEs that were modeled with markedly different accuracies in each cell type, emphasizing the importance of cellular context in model training. As demonstrated by training-excluded exemplar chemical predictions, we propose that a subset of classifiers offer utility in predicting MIEs from L1000 gene expression data. Methods developed herein could be integrated into EPA’s current tiered testing paradigm to prioritize chemicals for further study within a framework of new approach methodologies [Additional file 1: Supplemental Fig. 1. Distribution of LINCS gene expression profiles across cell lines. Each slice of the pie chart captures the percent of all LINCS phase I and II chemical perturbagen gene expression profiles that are derived from the indicated cell types. 76 Cell lines associated with < 5% of total profiles each were combined into the “all other cell lines” group.Additional file 2: Supplemental Fig. 2. Original and null model accuracies as a function of the number of training chemicals. A) Linear regression of internal accuracy (Y axis) for each MIE using each of the six classification algorithms as a function of the number of chemicals included in model training for MCF7-derived landmark gene classifiers (X axis). B) Linear regression of mean internal accuracy from 500 corresponding null classifiers (Y axis) for each MIE is as a function of the number of chemicals used in training (X axis).Additional file 3: Supplemental Fig. 3. Enrichment of high-ranking prediction scores among training excluded chemicals for confirmed high performance classifiers. For each of 9 confirmed high performance classifier, an Empirical Cumulative Distribution Function (ECDF) is plotted for LINCS chemicals ranked by median prediction. Solid red lines correspond to the ECDFs for MIE-associated chemicals with a support level of 3 or 4 in RefChemDB. Blue lines correspond to the ECDFs for all chemicals tested in MCF7 cells. Red dots and dashed lines indicate the greatest difference between the ECDF functions, which is used as the test statistic for the Kolmogorov-Smirnov (KS) test p-values shown above each plot.


The composition of microbial communities has been known to be location-specific. Investigating the microbial composition across different cities enables us to unravel city-specific microbial signatures and further predict the origin of unknown samples. As part of the CAMDA 2020 Metagenomic Geolocation Challenge, MetaSUB provided the whole genome shotgun (WGS) metagenomics data from samples across 28 cities along with non-microbial city data for 23 of these cities. In our solution to this challenge, we implemented feature selection, normalization, clustering and three methods of machine learning to classify the cities based on their microbial compositions. Of the three methods, multilayer perceptron obtained the best performance with an error rate of 19.60% based on whether the correct city received the highest or second highest number of votes for the test data contained in the main dataset. We then trained the model to predict the origins of samples from the mystery dataset by including these samples with the additional group label of “mystery.” The mystery dataset compromised of samples collected from a subset of the cities in the main dataset as well as samples collected from new cities. For samples from cities that belonged to the main dataset, error rates ranged from 18.18 to 72.7%. For samples from new cities that did not belong to the main dataset, 57.7% of the test samples could be correctly labeled as “mystery” samples. Furthermore, we also predicted some of the non-microbial features for the mystery samples from the cities that did not belong to main dataset to draw inferences and narrow the range of the possible sample origins using a multi-output multilayer perceptron algorithm. The development of next-generation sequencing (NGS) platforms  has enab1. MetaSUB aims to create a global genetic cartography of urban spaces that is based on extensive sampling of mass-transit systems and other public areas across the globe. They partnered with CAMDA to provide data for one of the three annual CAMDA challenges for 2020. These data include over a thousand novel samples from 28 cities and comprise over a terabase of whole genome shotgun (WGS) metagenomic data. The main dataset included nearly 1000 samples from 23 cities in 17 countries. An additional microbial dataset with 10 cities (5 of which were represented in the main dataset and 5 new cities) was provided as a mystery dataset. In addition to the microbial data, CAMDA also provided some city-specific data related to climate, location, and neighboring biomes for the main dataset.The objective of the CAMDA 2020 metagenomic geolocation challenge was to predict the geographic origins of mystery samples by using city-specific microbial and non-microbial data. All raw data in this work was provided by the MetaSUB ConsortiumClassification methods are typically used to construct models for predicting the origin of a novel sample. For the mystery samples that originated from cities included in the main dataset, we were able to use only 16S ribosomal RNA mapped compositional data to perform classification. One limitation of classification models is that they can never predict a novel origin that does not exist within the trained model. To address this problem, we can use city-specific microbial data to predict non-microbial city-specific features such as temperature and humidity . We can The selection of the non-microbial features was driven by our knowledge of factors that may possibly affect microbial composition. Microbial community composition and function are altered when microbes are exposed to different climate conditions . In addiAfter bioinformatics and data preprocessing, the operational taxonomic units (OTUs) were aggregated as counts. The selected taxonomic ranks “order,” “family,” and “species” were used independently for the analysis. https://susmdatta.github.io/software.html. We provide a more detailed description of the implementations in the following sections.The design of the analysis was motivated by our previous CAMDA experiences . In our Samples from MetaSUB were Illumina-sequenced at different depths and delivered in FASTQ format. The bioinformatics and data processing were conducted in the “HiPerGator 2.0” high-performance computing cluster, which includes 50,000 cores with 4GB of RAM in average for each core, at the University of Florida. The run time for each job was highly associated with the size of the raw FASTQ file.q = 38 as the minimum Phred score to keep and p = 50 as the minimum percentage of bases with Phred score ≥ 38. We then transformed the quality filtered FASTQ files into FASTA format for performing open reference OTU picking with QIIME  for the main dataset since some of the cities were with multiple collection years. We used this ordering to determine the cutoff values in downstream analyses. The more times a feature was retained by the models, the more likely the feature was to differentiate the samples of one city from the other cities. By setting different cutoffs, we obtained multiple feature sets that we in turn used for machine learning analysis to obtain the feature set with the best prediction performance. We then used the “voom” function  , supportomForest , e1071  along with the main dataset to serve as the training set. We repeated this procedure 100 times.Since CAMDA provided non-microbial data only for cities within the main dataset, for the non-microbial city-specific data, we used the multi-output MLP to build the model, which enabled us to model the relationship between microbial features and the non-microbial city-specific features from the main dataset, where microbial features were used as the predictors and non-microbial data were used as the response. We used the model based on the main dataset to predict the non-microbial city-specific features for the mystery samples, which allowed us to draw inferences about the sample origins. In this way, we could identify information about the samples in the mystery dataset without having trained the model with labels for cities within the mystery dataset.As discussed in the Feature Selection section in the Materials and Methods section, even after quality control and OTU picking, there were still some samples with extremely low counts compared to other samples within the same city. We removed samples with a sum of counts lower than the cutoff for the further analyses for each city in the main dataset and present the number of features before and after the filtering for each city in We performed feature selection in two steps as discussed in detail in the Feature Selection subsection in the “Materials and Methods” section and ordered the selected features by the number of times the model retained each feature; we used this ordering as cutoffs for downstream analyses. By setting different cutoffs, we obtained different feature sets.After the first step of initial feature selection, we retained 223 features, including 35 orders, 68 families, and 120 species. For the second step of feature selection with dimension reduction with Elastic Net, we present the results with different cutoffs ranging from 6 to 14 . The numWe performed clustering on the non-microbial city features to transform the continuous data into categorical data. Please refer to the Clustering for the non-microbial city data section of section “Materials and Methods” for further details. We present details of group assignment for each city in To find the feature set with the best classification performance, we fit the machine learning models, i.e., RF, SVM, and MLP using feature sets that were generated by using different cutoffs during feature selection. Bradyrhizobiaceae and Enterobacteriaceae and the orders Enterobacteriales, Burkholderiales, and Rhizobiales. Bradyrhizobiaceae and Enterobacteriaceae are families with the corresponding orders Rhizobiales and Enterobacteriales, respectively, and Enterobacteriales contains only the Enterobacteriaceae family   for the samples that passed quality control in the main and mystery datasets can also be found in the e family . The Bra family) . Many ofn humans . The ordironment . Finallybacteria . From thWhen using the feature set with cutoff = 12 for MLP (the classifier with the highest vote of the prediction results from the 100 runs), we found that the error rate was 32.66%. However, for some samples, the highest vote and the second-highest vote tied. Furthermore, some of the second-highest votes were the true label for the samples. When the highest and second-highest votes were used at the same time, the error rate fell to 19.60%, indicating that nearly 13% of the second-highest vote of the samples were the true labels. The details of the error rate for each city are presented in Next, we implemented the same procedure to predict the non-microbial data of the test set by using a multi-output MLP with the selected feature set. The only difference was that we combined samples from the same city collected in different years to predict the non-microbial data, which we then used separately for predicting the sample origins. The error rates were 20.1, 28.6, 24.6, 21.1, 15.6, and 14.1%, respectively, for summer temperature, winter temperature, summer humidity, winter humidity, coastal indicator, and degree of urbanization of the neighboring biomes.Finally, we used the selected feature set to predict the sample origins and the non-microbial features for the mystery samples. There were samples from 10 cities in the mystery dataset, consisting of five cities that have been sampled before and five new cities. These were Bogota, Hong Kong, Kiev, Krakow, Marseille, Naples, Taipei, Tokyo, Vienna, and Zurich. For the samples from the cities that have been sampled before, the prediction was straightforward; we could predict the sample origins directly. The error rates for these five cities, i.e., Hong Kong, Kiev, Taipei, Tokyo, and Zurich, based on the highest and second highest votes were 46.7, 72.7, 18.18, 38.5, and 50%, respectively. The lowest error rate was obtained from Taipei, which showed the best performance when using only data from the main dataset as well. Similarly, the samples from Kiev had the highest prediction error rate for both the main dataset and the mystery dataset among these five cities. For the samples from the cities that do not appear in the main dataset, we used the classifier to see if we could determine whether the test samples were accurately labeled “mystery” . 57.7% of the samples were predicted not to belong to a city in the main dataset (“mystery”). The knowledge of whether the samples have been sampled before provided limited information, so we then predicted a selection of non-microbial features for the samples from the new cities. The average predicted non-microbial features for each city are presented in In this work, we have used a more relaxed preliminary filtration to obtain the common features and followed with an elastic net logistic regression for further selection. Of the three different classification methods we tested, we found that MLP performed the best of the three regardless of the feature sets. For MLP, we obtained a low error rate without including too many features by using the feature set with cutoff = 12. The top five features were common orders and families, which corresponded with our results from the CAMDA 2019 MetaSUB challenge. The error rate was 19.60% based on the highest and the second-highest votes. Compared to last year’s challenge, there were many more samples, with over 1,000 samples included. However, higher error rates were obtained since we included more cities this year, which made the accurate prediction of cities harder. In addition, we found that samples collected from the same city but collected over different years sometimes displayed notably different error rates, including NYC_16 and NYC_17, TYO_16 and TYO_17, and DOH_16 and DOH_17. This could be caused by technical variability such as batch effects. The prediction accuracy could be improved by a better normalization method to avoid the potential batch effects. Furthermore, this difference between samples collected in different years could also be considered as the result of dynamic change in microbial communities within a given city from year to year.According to the results for predicting the mystery samples, Taipei has the best accuracy while samples from Kiev were poorly classified. For the samples of cities that do not exist in the main dataset, we found that the pattern of the samples from Krakow was similar to the pattern of Berlin, and the pattern of the samples from Naples was similar to the pattern of New York. These results made sense to us since Krakow and Berlin are geographically close, and Naples and New York are at nearly the same latitude. This additional information would help us to draw inferences and narrow the possible candidates for the sample origins. Since we did not have the non-microbial city-specific data for the mystery cities, we were unable to validate the error. The predictions for some novel cities including Bogota, Marseille, and Vienna were not ideal; some of the non-microbial city features predictions were not close enough to the preset level , indicatThe average of the error rates of the six non-microbial city-specific features was about 20%; it was necessary to select meaningful and appropriate features to decrease the error rate and thus allow us to build a more reliable model. Currently, the selection of the non-microbial features was based on our knowledge of the factors. We have found that some features such as climate may affect the microbial composition, which motivated us to use the microbial features to inversely predict these non-microbial features and further obtain the city’s pattern. In addition to the literature support, other analyses to determining the main drivers of variance in microbial features would also be worth investigating in future analyses.To date, our work has been based on generating taxonomic information from alignments of the reads to the 16S ribosomal gene in bacterial species . This isIn summary, the results presented in this work show an effective method to process and classify microbial samples by origin. Even when samples were from cities that have never been sampled before, we could map these samples to a specific pattern of climate information and draw inferences from these predictions. By doing this, it would be possible for us to obtain the trace of the object based on the swabs taken from the object. However, there is still much to be improved in our future work.http://camda2020.bioinf.jku.at/doku.php/contest_dataset.Publicly available datasets were analyzed in this study. This data can be found here: SD reviewed the manuscript and provided the theoretical support when required. RZ, DE, and AW designed and ran the analyses. RZ wrote the manuscript. All authors have read and approved the final manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


Lung cancer (LC) prognosis is closely linked to the stage of disease when diagnosed. We investigated the biomarker potential of serum RNAs for the early detection of LC in smokers at different prediagnostic time intervals and histological subtypes. In total, 1061 samples from 925 individuals were analyzed. RNA sequencing with an average of 18 million reads per sample was performed. We generated machine learning models using normalized serum RNA levels and found that smokers later diagnosed with LC in 10 years can be robustly separated from healthy controls regardless of histology with an average area under the ROC curve (AUC) of 0.76 . Furthermore, the strongest models that took both time to diagnosis and histology into account successfully predicted non-small cell LC (NSCLC) between 6 and 8 years, with an AUC of 0.82 , and SCLC between 2 and 5 years, with an AUC of 0.89 , before diagnosis. The most important separators were microRNAs, miscellaneous RNAs, isomiRs, and tRNA-derived fragments. We have shown that LC can be detected years before diagnosis and manifestation of disease symptoms independently of histological subtype. However, the highest AUCs were achieved for specific subtypes and time intervals before diagnosis. The collection of models may therefore also predict the severity of cancer development and its histology. Our study demonstrates that serum RNAs can be promising prediagnostic biomarkers in an LC screening setting, from early detection to risk assessment. Lung cancer (LC) continues to be the leading cause of cancer-related deaths despite declining smoking prevalence . Non-smaLiquid biopsies quantifying molecular biomarkers in circulation, such as tumor-derived DNAs, proteins, and RNAs, can be used to detect cancer . MicroRNOne explanation of the lack of circulating RNAs used in clinical settings is our limited understanding of the prediagnostic dynamics of cell-free RNAs, since studies are usually based on samples at or after diagnosis. Carcinogenesis is a multistep process that turns cell functions from normal to malignant . It can In the present study, our objective was to identify serum RNA-based biomarkers for early diagnosis of LC using prediagnostic samples. We identified the optimal machine learning (ML) algorithm for RNA biomarker modeling. Optimization of prediction models was done with an ML workflow, including cross-validation and testing, which was repeated five times to increase the generalizability of our results. We also investigated the biological relevance of the best RNA separators in the context of cancer biomarkers.In this study, we selected 400 patients with prediagnostic serum samples including multiple samples from the same patients. We also included 525 individuals as controls. After excluding failed or low input samples, we obtained RNA-seq data from 1061 serum samples. However, samples from individuals without any smoking history (i.e. never smokers) or missing information were excluded from initial analyses. This resulted in 535 cases and 263 control samples from 645 current or former smokers for modeling and testing . Non-smoAfter filtering out low-count transcripts, 3306 RNAs were selected as candidate features and used in the models: 202 miRNAs, 1137 isomiRs, 89 miscellaneous RNAs (miscRNAs), 380 piRNAs, 119 small nucleolar RNAs, 530 tRFs, 790 mRNAs, and 59 lncRNAs.We first evaluated the classification performance of the ML algorithms in terms of average AUCs on test datasets, created by five random repeats as explained in Materials and methods.All samples were included in algorithm evaluation regardless of their stage at diagnosis and prediagnostic time which were regarded as full-time standard models . The aveAlthough the models of all algorithms had comparable performances in terms of average AUCs, they differ in terms of total number of non-zero features (i.e. different model complexity). For example, random forest (RF) selected more than 3000 non-zero features while the lasso model selected fewer than 25 features. However, the profiles of the top features, ranked in terms of feature importance, usually consisted of similar RNAs (e.g. miRNAs or tRFs).INTS10 for all histologies; Y-RNA, piR-hsa-28723, and GPB3 for NSCLC; and tRF-BS68BFD2, RN7SL724P, and tRF-947673FE5 for SCLC. An in-depth investigation of selected features by other algorithms also showed common RNAs. For example, Y-RNA and iso-23-8YUYFYKSY isomiR were among the top predictors of the RF, elastic-net, the SGL, and the lasso models for NSCLC; tRF-BS68BFD2 for SCLC. We also performed KEGG pathway enrichment analysis based on the common miRNA, mRNA, and isomiR features. The results showed that many cancer-related pathways were significantly (p < 0.01) enriched such as MAPK signaling, mTOR signaling, and AMPK signaling.Since XGBoost produced the most predictive full-time models, we used it for the remaining analyses. We also investigated the best predictors of the XGBoost models and ranked them based on their importance . The topWe evaluated the classification performance of the XGBoost algorithm by selecting one sample per patient rather than using all samples from the same individuals. Our results showed comparable performance in terms of AUCs for all models . The SCLWe produced XGBoost models that included only a single RNA class  to further investigate important features/classes. This method showed that miscRNA-only and miRNA-only models achieved better classification performance than the other RNA classes regardless of histology and stage at diagnosis . The besGBP3, and piR-hsa-28723 for all histologies; Y-RNA, hsa-miR-423–5p isomiR (iso-20-5KP25HFF), and LINC01362 for NSCLC; HIST1H4E, PTCH2, and tRF-R29P4P9L5HJVE for SCLC , up to 5 years (0–5), and up to 8 years (0–8) before diagnosis. The results showed slight improvement in model performance compared to full-time models see . HoweverWe created models by compiling the best features from the full-time models. Our results showed improved prediction performance for these models despite inclusion of leave-out datasets into the test set . In the test datasets including only smokers, AUC for all histologies was 0.76 ; NSCLC model was 0.78 ; SCLC model was 0.88  . HoweverThe RRs and their associated p-values on the test dataset, with and without non-smokers, are reported in We also investigated the prediagnostic models, using the ML workflow, and selected two pairs of models for NSCLC and SCLC, which showed high performance before and after 5 years prior to diagnosis see . We founIn this study, we showed that ML models of prediagnostic serum RNA levels can be used to predict LC years before diagnosis and manifestation of disease symptoms. Our models achieved clinically relevant performance in terms of AUC, accuracy, sensitivity, and specificity . The modWe previously reported that prediagnostic circulating RNA signals are highly dynamic in LC patients and they can be histology and stage dependent . In the RAB38 and RAB27B (PCP2 in tumor tissues (PIK3R1 to suppress cell growth. Its overexpression inhibits cell proliferation and invasion and it was significantly downregulated in both tissue and serum (We found that some features were considerably more important (and frequent) separators than others with or without feature selection. The list includes piRNAs (e.g. piR-hsa-28723), miRNAs , isomiRs (e.g. isomiR of hsa-miR-423–5p (iso-20-5KP25HFF), hsa-miR-486–5p (iso-23-8YUYFYKSY)), and miscRNAs (e.g. RN7SL181P). Some of them were particularly interesting since they were associated with cancer or proposed as cancer biomarkers. Hsa-miR-30a-5p is a tumor suppressor and downregulated in LC tissues . It regud RAB27B . Another tissues  and has  tissues . Hsa-miRnd serum . Hsa-miRnd serum .TP53 mRNA at the 3'UTR region and downregulates its expression  . This caThe most important features of histology-specific models also showed associations with carcinogenesis-related KEGG pathways, which were common or specific to histology. The common ones include well-known signaling  and cancer-related pathways (e.g. proteoglycans in cancer and pathways in cancer). Choline metabolism in cancer pathway was one of the common ones and enriched in some histology-specific prediagnostic models. Altered choline profiles are characteristics of tumor tissues . MoreoveA strength of our study is the large sample size from prediagnostic cases and a large control group from cancer-free individuals from the same cohort. We have detailed information on histological subtype and stage at diagnosis from the Cancer Registry of Norway (CRN) and smoking history from survey data. We also accounted for other potential confounders  . Some ofThere are some weaknesses in our study that we need to address. First, an independent cohort should replicate our results. However, only a few cohorts include prediagnostic samples that can be used for discovery and validation. We tried to overcome this issue by using training-testing repeats for assessing generalisability. We also reported our results with and without feature selection since some feature selection methods (e.g. lasso and univariate) can cause overfitting. Second, using more than one sample from the same individual can potentially cause overfitting. However, we did not detect any effect related to this issue . Third, In LC screening programs, RNA biomarkers can be used as a tool of initial assessment or combined with LDCT for early detection . We founWe have shown that LC can be detected in both smokers and non-smokers years before diagnosis and the manifestation of symptoms regardless of histological subtype. We also proposed a model on how RNA biomarkers can be utilized in clinical settings. Our top performing models can produce AUCs up to 0.9 before diagnosis suggesting a great potential for LC early prediction.We used the population-based Janus Serum Bank (JSB) cohort containing prediagnostic serum samples . The stuDetailed cancer information was selected from the CRN that has systematically collected mandatory notification on cancer occurrence for the Norwegian population since 1952 . The casWe extracted RNA from 400 µL serum using phenol-chloroform and miRNeasy Serum/Plasma kit . We performed size selection using a 3% Agarose Gel Cassette (Cat. No CSD3010) on a Pippin Prep (Sage Science) with a cut size optimized to cover RNA molecules from 17 to 47 nt in length. Libraries were prepared with the NEBNext Small RNA kit  and sequenced on a HiSeq 2500 platform to on average 18 million sequences per sample .optmatch (v0.9–11) R package (p.adjust function of R).Our bioinformatics workflow includes quality control, adapter trimming, read mapping, read counting, and creation of count tables. We used a large annotation dataset containing several RNA classes available in serum , includi package  selected package . We used package  , RF, and extreme gradient boosting (XGBoost) algorithms. We used fivefold cross-validation (if available) to tune hyperparameters for model training. For the SGL models, RNAs were classified by type.caret (v6.0–84) and glmnet (v2.0–18) packages for elastic-net and the lasso, sglfast (v0.10) and msgl (v2.3.9) for the SGL models and xgboost (v1.0.0.2) for XGBoost. Classifications were performed according to histology and time to diagnosis (for details see next paragraph) using an automated ML workflow. In the ML workflow the datasets were split into training (70%) and test (30%)  . We repeWe refer to models for all histologies, NSCLC and SCLC that do not take time to diagnose into account as standard full-time models . PrediagWe implemented feature selection methods to improve model performances, including single-RNA class, lasso selection, and significant selection. In the single-RNA class method, we dropped all RNA types except one. In lasso selection, all non-zero features selected by the lasso classification models were pooled. Next, we retrained new classification models which were restricted to use only these features. In significance selection, an univariate regression analysis was done per feature and significant features (multiple testing adjusted) were used to train classification models.cutpointr (v1.0.1) and epitools (v0.5–10.1) to calculate RRs.We created models for each histology which utilize the most frequent features identified in the standard full-time models. To assess overfitting and to get a better estimate of these model performances, we split the datasets into training (80%) and test (20%) sets. To the test sets we also added non-smokers and samples from smokers, but not previously used in frequency matching .The datasets generated for this article are not readily available because of the principles and conditions set out in articles 6 (1) (e) and 9 (2) (j) of the General Data Protection Regulation (GDPR). National legal basis as per the Regulations on population-based health surveys and ethical approval from the Norwegian Regional Committee for Medical and Health Research Ethics (REC) is also required. Requests to access the datasets should be directed to the corresponding authors. Our scripts and bioinformatics workflow files can be accessed from our GitHub repo .Thank you for submitting your article "Serum RNAs can predict lung cancer up to 10 years prior to diagnosis" for consideration by The reviewers have seen each other's reviews, and the Reviewing Editor has drafted this to help you prepare a revised submission.Essential revisions:1. The analysis and result of serum RNA sequencing were not clearly presented. RNA sequencing with an average of 18 million reads per sample was performed. How about the mapped rate? Since the samples are long-term stored and may be at different times of collection, it is important that the sequencing results should be consisent. How about the sample correlations? In addition, why many of the RNA candidates (1137 of 3306) are isomiRs? They are derived from miRs, or degradation?2. The authors showed that the candidate biomarkers were invovled in the cancer-related pathways. This result is somewhat overestimated because these RNAs are small RNAs and the KEGG pathway enrichment analysis is based on the targets.3. A strength of this study is the large sample size from prediagnostic cases and a large control group from cancer-free individuals from the same cohort. The authors obtained RNA-seq data from 1061 serum samples. However, only samples from a total of 645 individuals with smoking history were analyzed. It would be helpful to include the samples without smoking to investigate the specificity of the potential biomarkers.4. There are concerns regarding multiple samples from the same cancer subject. If samples from one subject go to both the training and testing set, there would be issues of overfitting. In the revision, the authors need to remove the duplicate samples from the analysis.5. Calculate the relative risk for future cancer development for individuals with positive and negative test results.6. Instead of using sliding windows of fixed duration, use windows of increasing duration.7. Address the issue of over-fitting for the analysis from line 338 onwards.8. The proposed use of the analysis  does not appear to be realistic. The authors need to calculate the percentage of subjects falling into the test positive and negative groups based on their analysis. Then, for each group, what percentage of subjects will eventually develop LC.Reviewer #2:This study by Umu et al., investigated the biomarker potential of serum RNAs for the early detection of lung cancer (LC) in smokers at different prediagnostic time intervals and histological subtypes. They observed that smokers later diagnosed with LC can be robustly separated from healthy controls by using machine learning models with serum RNAs profiles. This work demonstrated that serum RNAs could be promising prediagnostic biomarkers in a LC screening setting. This finding is interesting but not convincing because only one sample cohort was evaluated in current study and the datasets generated for this manuscript are not available. Also there are some concerns as follows.1.The analysis and result of serum RNA sequencing were not clearly presented. RNA sequencing with an average of 18 million reads per sample was performed. How about the mapped rate? Since the samples are long-term stored and may be at different times of collection, it is important that the sequencing results should be consisent. How about the sample correlations? In addition, why many of the RNA candidates (1137 of 3306) are isomiRs? They are derived from miRs, or degradation?2.The authors showed that the candidate biomarkers were invovled in the cancer-related pathways. This result is somewhat overestimated because these RNAs are small RNAs and the KEGG pathway enrichment analysis is based on the targets.3.A strength of this study is the large sample size from prediagnostic cases and a large control group from cancer-free individuals from the same cohort. The authors obtained RNA-seq data from 1061 serum samples. However, only samples from a total of 645 individuals with smoking history were analyzed. It would be helpful to include the samples without smoking to investigate the specificity of the potential biomarkers.Reviewer #3:The authors analyzed archived serum samples which were collected before the development of lung cancer (LC) for different species of RNA.They used machine learning to investigate if the combination of the RNA markers would be useful for the prediction of the future development of LC. In the machine learning analysis, they used 70% samples for training and 30% samples for testing. The process was repeated 5 times so that the samples can be assigned to both training and testing groups.The strength of the study is that they include a large cohort of samples collected years before the development of LC.The weaknesses of the study include1. the questionable biological basis of this approach. As the samples were collected years before cancer diagnosis, the amount of tumor-derived RNA is expected to be very low. In this regard, the aberrations in serum RNA, if any, are likely to reflect the background risk of the body as a whole.2. multiple samples from the same cancer subject. If samples from one subject go to both the training and testing set, there would be issues of overfitting. The authors would need to remove the duplicate samples from the analysis.3. the low diagnostic performance. The sensitivity and specificity of the models were both around 70%. This would not be useful clinically. In fact, the use of relative risk for individuals with positive and negative test results would be more useful for predictive models.4. The authors used a sliding window of fixed duration for the analysis. This analysis is counter-intuitive. If a marker is useful for predicting cancer at 3-5 years before cancer, what is the reason why it cannot predict cancers that occur within 3 years. It would be better to use windows with flexible size e.g. prediction of cancer in the next 2, then next 3, then next 5 years, etc.5. For the analysis on lines 338 onwards, the authors further select features identified in the previous machine learning models for evaluation. What is the testing group in this analysis? Did they use all the samples without separating training and testing groups? If so, the over-fitting problem would be significant.6. The. proposed use of the analysis  does not appear to be realistic. The authors need to calculate the percentage of subjects falling into the test positive and negative groups based on their analysis. Then, for each group, what percentage of subjects will eventually develop LC.7. Remove duplicated samples from the LC group.8. Calculate the relative risk for future cancer development for individuals with positive and negative test results.9. Instead of using sliding windows of fixed duration, use windows of increasing duration.10. Address the issue of over-fitting for the analysis from line 338 onwards.11. To address the issue of clinical utility raised in point 6 in the previous section. Essential revisions:1. The analysis and result of serum RNA sequencing were not clearly presented. RNA sequencing with an average of 18 million reads per sample was performed. How about the mapped rate? Since the samples are long-term stored and may be at different times of collection, it is important that the sequencing results should be consisent. How about the sample correlations? In addition, why many of the RNA candidates (1137 of 3306) are isomiRs? They are derived from miRs, or degradation?We have now improved the presentation of the analysis and results throughout the manuscript. The manuscript builds on previous publications showing feasibility and profiles of serum small RNA sequencing, confounding, long-term stability and differential expression in cancer in this biobank .laboratory processing and bioinformatic analyses” and included the information on mapping tools and rates. We created a separate section called “Bioinformatic analyses” (see page 4 and 5).However, we agree with the reviewer that more information about the characteristics of the RNA profiles would be helpful to the reader. To answer the specific questions: First, the average mapping rate was around 70%. We revised the materials methods section “Regarding long-term storage, we already showed that long-term stored samples are a viable source for small RNA analysis . Small RNAs (e.g. miRNA) are stable during long-term archiving . We showed that some traits such as sex, age of patients and blood donor group (BDg) affect RNA profiles. Therefore, these traits must be matched between cases and controls. BDg refers to a technical cofounder combining the effect of sample treatment at donation and storage time . In this revision, we further explained this and added these sentences into Materials and methods:“Therefore, we frequency matched LC samples and controls on sex, age at donation, and blood donor group (BDg). BDg is a technical cofounder combining the effect of sample treatment at donation and storage time .” (see page 4)In Umu et al., 2018, we also investigated whether RNAs were degradation products or not. We found that RNA fragments, such as tRNA derived fragments, had specific expression patterns, which can be explained by RNA biogenesis. We concluded that many RNA fragments in serum must be biological products, including tRFs and isomiRs. Therefore, in our submitted manuscript, we used the SeqCluster  tool and we regarded all isomiRs which passed the expression threshold as bona fide isomiRs. In this version, we also included mirTOP identifiers of candidate isomiRs.Our definition of expression threshold is:“any RNAs with more than 5 reads mapped in 80% of the samples”.This is a high threshold to detect robust signals of expression. We now cited SeqCluster and mirTOP and included our expression detection threshold in Materials and methods, which was missing in our initial submission. (see page 4)2. The authors showed that the candidate biomarkers were involved in the cancer-related pathways. This result is somewhat overestimated because these RNAs are small RNAs and the KEGG pathway enrichment analysis is based on the targets.Thank you for pointing this out. miRNAs can target multiple genes. Unfortunately there is no easy way to do pathway enrichment analysis for other important small RNAs (e.g. miscRNAs), which are underrepresented. We believe it is a plausible assumption to expect miRNAs will disrupt a large set of biological pathways including cancer related ones since they often regulate multiple mRNAs.While predicting miRNA targets, we used a high threshold to reduce the total number of false positives. We also performed a multiple testing correction to filter out less significant enrichments. Our literature search (as discussed in the results and discussion) also showed that many biomarker candidates, such as miRNAs and some other sncRNAs, were associated with cancer-related pathways.3. A strength of this study is the large sample size from prediagnostic cases and a large control group from cancer-free individuals from the same cohort. The authors obtained RNA-seq data from 1061 serum samples. However, only samples from a total of 645 individuals with smoking history were analyzed. It would be helpful to include the samples without smoking to investigate the specificity of the potential biomarkers.As reviewers suggested, we included non-smokers for AUC estimation and relative risk calculation. This also assessed the robustness and overfitting issue that was brought by the reviewers (please refer to question 7). Non-smokers were not used for training and only used to evaluate frequent biomarkers. We put non-smokers characteristics table into the supplementary document (Table S1).4. There are concerns regarding multiple samples from the same cancer subject. If samples from one subject go to both the training and testing set, there would be issues of overfitting. In the revision, the authors need to remove the duplicate samples from the analysis.To answer this question, we repeated the XGBoost modelling described in the Results section “ML algorithms can differentiate between prediagnostic cases and controls regardless of prediagnostic time” without multiple samples. We selected only one sample per patient. We used the identical modelling workflow. Our results showed similar performance in terms of AUCs. We therefore concluded that multiple samples do not cause overfitting or any bias .We discussed this result briefly in the main manuscript by writing:“Second, using more than one sample from the same individual can potentially cause overfitting. However, we did not detect any effect related to this issue .” (see page 18)5. Calculate the relative risk for future cancer development for individuals with positive and negative test results.We addressed this issue and calculated the relative risk (Table 3). We explained how we calculate the relative risk in Materials and methods. 6. Instead of using sliding windows of fixed duration, use windows of increasing duration.To address this question, we used a fixed duration approach as suggested by the reviewers. We used three different time windows 0 to 2 years, 0 to 5 years and 0 to 8 years before diagnosis. These points were selected because the results showed slight improvement in the model performance in some time windows. The performance is not as good as sliding window models.However, we included the results from fixed-duration models to the supplementary document as a separate section (Table S3) and briefly discussed the results in main manuscript by adding this paragraph:“As an alternative to sliding windows, we also performed a fixed window approach and trained models using samples from up to 2 years (0-2), up to 5 years (0-5) and up to 8 years (0-8) before diagnosis. The results showed slight improvement in model performance compared to full time models (see supplementary document and Table S1). However, sliding windows models performed better on specific time intervals.”.7. Address the issue of over-fitting for the analysis from line 338 onwards.We agree with the reviewer that overfitting might be an issue. We tried to overcome this by including an independent dataset that was never used for testing or training.Our machine learning workflow balances for case/control numbers and also matches samples in training and test based on “age”, “sex” and “BDg”. (Please see revision answer 1 and page 4 of the manuscript). Therefore, we previously discarded unmatchable samples for each strata and these samples were never not used. We have now utilized them to assess model performance and to assess overfitting by adding them into the testing group for section “Frequent features can create simple and accurate models”.Furthermore, we added non-smokers into the testing dataset. We measured evaluation metrics on test datasets which returned a better estimate on the model performance.In short, the training group contains 80% and the test group contains 20% plus smokers that have never been used for training (i.e. unmatchable samples) and non-smokers that we did not use in our original submission. We refer to these samples as independent leave-out dataset. We showed in Table 3 how many of the test set samples were never used. However, overfitting might still be an issue but we wanted to find and report a shortlist of the best biomarkers for future research.8. The proposed use of the analysis  does not appear to be realistic. The authors need to calculate the percentage of subjects falling into the test positive and negative groups based on their analysis. Then, for each group, what percentage of subjects will eventually develop LC.This is a good suggestion and to accommodate the reviewers comments, we have updated Figure 4 and the discussion. Please see the highlighted text (page 19).Reviewer #2:This study by Umu et al., investigated the biomarker potential of serum RNAs for the early detection of lung cancer (LC) in smokers at different prediagnostic time intervals and histological subtypes. They observed that smokers later diagnosed with LC can be robustly separated from healthy controls by using machine learning models with serum RNAs profiles. This work demonstrated that serum RNAs could be promising prediagnostic biomarkers in a LC screening setting. This finding is interesting but not convincing because only one sample cohort was evaluated in current study and the datasets generated for this manuscript are not available. Also there are some concerns as follows.Unfortunately the datasets generated for this article are not readily available because of the principles and conditions set out in articles 6 (1) (e) and 9 (2) (j) of the General Data Protection Regulation (GDPR). However, in a collaborative setting, data is available and can be shared in a secure environment.1.The analysis and result of serum RNA sequencing were not clearly presented. RNA sequencing with an average of 18 million reads per sample was performed. How about the mapped rate? Since the samples are long-term stored and may be at different times of collection, it is important that the sequencing results should be consisent. How about the sample correlations? In addition, why many of the RNA candidates (1137 of 3306) are isomiRs? They are derived from miRs, or degradation?Please see the essential revisions.2.The authors showed that the candidate biomarkers were invovled in the cancer-related pathways. This result is somewhat overestimated because these RNAs are small RNAs and the KEGG pathway enrichment analysis is based on the targets.Please see the essential revisions.3.A strength of this study is the large sample size from prediagnostic cases and a large control group from cancer-free individuals from the same cohort. The authors obtained RNA-seq data from 1061 serum samples. However, only samples from a total of 645 individuals with smoking history were analyzed. It would be helpful to include the samples without smoking to investigate the specificity of the potential biomarkers.Please see the essential revisions.Reviewer #3:The authors analyzed archived serum samples which were collected before the development of lung cancer (LC) for different species of RNA.They used machine learning to investigate if the combination of the RNA markers would be useful for the prediction of the future development of LC. In the machine learning analysis, they used 70% samples for training and 30% samples for testing. The process was repeated 5 times so that the samples can be assigned to both training and testing groups.The strength of the study is that they include a large cohort of samples collected years before the development of LC.The weaknesses of the study include1. the questionable biological basis of this approach. As the samples were collected years before cancer diagnosis, the amount of tumor-derived RNA is expected to be very low. In this regard, the aberrations in serum RNA, if any, are likely to reflect the background risk of the body as a whole.We agree with the reviewer, the origin of serum RNAs can be different tissues including immune cells or tumor cells. After diagnosis, the size of the tumor can still be too small to detect any originating RNAs. It is plausible that we detect RNAs originating from any tissues which signal cancer progression. We included a few sentences and citations that discuss the origin of circulating RNAs as well .As the reviewer suggested, we calculated the RR of a positive prediction.2. multiple samples from the same cancer subject. If samples from one subject go to both the training and testing set, there would be issues of overfitting. The authors would need to remove the duplicate samples from the analysis.As the reviewer suggested, we tested this effect and added the results into the Supplementary documents. Please also see the essential revisions answer 4.3. the low diagnostic performance. The sensitivity and specificity of the models were both around 70%. This would not be useful clinically. In fact, the use of relative risk for individuals with positive and negative test results would be more useful for predictive models.Please see the essential revisions.4. The authors used a sliding window of fixed duration for the analysis. This analysis is counter-intuitive. If a marker is useful for predicting cancer at 3-5 years before cancer, what is the reason why it cannot predict cancers that occur within 3 years. It would be better to use windows with flexible size e.g. prediction of cancer in the next 2, then next 3, then next 5 years, etc.Please see the essential revisions.5. For the analysis on lines 338 onwards, the authors further select features identified in the previous machine learning models for evaluation. What is the testing group in this analysis? Did they use all the samples without separating training and testing groups? If so, the over-fitting problem would be significant.Please see the essential revisions.6. The. proposed use of the analysis  does not appear to be realistic. The authors need to calculate the percentage of subjects falling into the test positive and negative groups based on their analysis. Then, for each group, what percentage of subjects will eventually develop LC.Please see the essential revisions.


With the advent of advanced high throughput biotechnologies such as next-generation sequencing and single-cell sequencing, there has been an increasing growth of complex multiple-omics data sets . These data are often heterogeneous, sparse, high dimension and high noise, which provide different levels of insightful information for disease. Integrative analysis of multiple-omics data can help the biomedical researchers to explore biological mechanisms and further assist in designing better diagnostic tools and therapies for the treatment of diseases. However, the development of effective methods for multiple-omics data analysis is very challenging as the complex characteristics of different kinds of data. Recently, machine learning methods especially graph embedding have shown powerful capability in analyzing multiple-omics data. Particularly, they are capable to represent data as low dimensional vectors while the data features are preserved.Graph Embedding Methods for Multiple-Omics Data Analysis.” This Research Topic presents 19 articles. We expect that these articles promote more advanced studies for multiple-omics data analysis.To provide a platform bridging graph embedding method and multiple-omics data analysis, we organized we organized a Research Topic on “Peng L. et al. identified potential antiviral drugs against SARS-CoV-2 by using regularized least squared classifier and bipartite local model. Ninety six virus-drug associations between 11 types of viruses similar to SARS-CoV-2 and 78 small molecular drugs were extracted in this study.Hou et al. proposed a method to capture potential functions in a microbial co-occurrence network. It integrated topological structures of microbial co-occurrence networks with k-mer compositions of operational taxonomy unit sequences and embedded them into a lower-dimensional continuous latent space.Pan et al. presented an embedding-based method for predicting the subcellular localization of proteins. The functional and network embeddings from GO terms and protein–protein network were combined as novel representations of protein locations for the construction of the final classification model.Gu et al. proposed a method incorporating feature engineering and feature selection algorithms to explore the common controlling genes and corresponding pathways among eight different organs' fibrosis. These results were helpful for understanding the molecular mechanisms of fibrosis diseases and finding new therapeutic indications of existing drugs.Zhang developed a feature selection algorithm for gene expression data classification by using approximate conditional entropy based on fuzzy information granule. The experimental results on large-scale gene datasets show that this algorithm not only greatly reduces the dimension of the gene datasets, but also is superior to the state-of-the-art algorithms in classification accuracy.Yuan and Yang proposed a deep learning method to identify circRNA-RBP interactions by using hybrid double embeddings for representing RNA sequences and a cross-branch attention neural network for classification. The experimental results on benchmark datasets show that their method outperforms the mainstream deep learning-based methods on not only prediction accuracy but also computational efficiency.He et al. adopted multiple kernel learning (MKL) to integrate somatic mutation to currently molecular data including gene expression, copy number variation (CNV), methylation, and protein expression data for the prediction of breast cancer survival. In addition, the maximum relevance minimum redundancy (mRMR) feature selection method was utilized to select features that present high relevance to survival and low redundancy among themselves for each type of data.Su et al. designed a multi-level model to improve both the quality and speed of large-scale PPIs prediction. The results showed that their model is promising for large-scale PPI prediction in both accuracy and efficiency, which is beneficial to other large-scale biomedical molecules interactions detection.Barbiero et al. tested the digital twin model on two simulated clinical case studies combining information at organ, tissue, and cellular level. The results show their approach is able to detect inflammatory cytokines which are known to have effects on blood pressure and have previously been associated with SARS-CoV-2 infection .Zhang et al. developed a computational method based on the Light Gradient Boosting Machine (LightGBM) to predict potential metabolite–disease associations. It extracted the features from statistical measures, graph theoretical measures. Three case studies confirmed that this method has obvious superiority in predicting metabolite–disease pairs and represents a powerful bioinformatics tool.Zhao et al. proposed a supervised gene selection method based on permutation and random forest classification. The experimental result on 10 datasets show that the gene selection performance of their method is better than other gene selection methods.Wang J. et al. presented a computational drug repositioning approach to discover potential drug-disease associations. The experimental results demonstrate that their approach outperforms recent state-of-the-art prediction models. In addition, the case studies further confirm the predictive ability of the proposed method.Wang, Dai, et al. introduced a pan-cancer classification method to identify a set of genes that can differentiate all tumor types accurately. Extensive experimental results on the public RNA-seq data sets with 33 different tumor types show that this method outperforms the other state-of-the-art classification methods.Wang, Cao, et al. proposed a computational method to predict and identify the m6A sites on mRNA by utilizing sequence-derived and graph embedding features. The comparison results show that the proposed method achieved the best performance compared with other predictors on four public datasets across three species.Wang Z. et al. explored the gene expression changes and its potential effects mediated by U11 snRNA in bladder cancer cell. This study show that U11 may be involved in the regulation of gene expression in bladder cancer cells, which may provide a potentially new biomarker for clinical diagnosis and treatment of bladder cancer.Feng et al. integrated transcriptomic, lipidomic, and metabolomic analyses to identify the differential lipids and metabolites between basal and luminal muscle invasive bladder cancer (MIBC) subtypes. The results suggest that free fatty acids (FFA) and sulfatides (SL), which are closely associated with immune and stromal cell types, have strong capacities to distinguish basal and luminal subtypes of MIBC tumors. Moreover, the results also show that the ratios of glycerophosphocholine (GCP)/imidazoles and nucleosides/imidazoles can accurately identify tumors of basal and luminal MIBC subtypes.Peng X. et al. presented a method to construct methylation haplotypes for homologous chromosomes in CpG dense regions. The proposed method not only can be applied to methylation analysis, but also can provide a clear explanation for the methylation difference at the resolution of methylation haplotypes.Liu and Zhang developed a computational model for the detection of copy number variation detection (CNV) of different lengths from whole genome sequencing data. It used a clustering algorithm to divide the read depth segment profile, and assigned an abnormal score to each read depth segment. The experimental results show that the performance of proposed model is better than those of several existing methods.Zheng and Wu proposed a method for predicting drug-target interactions based on heterogeneous network integration and cascade deep forest. The results show that their model outperforms the previously reported methods on the benchmark datasets.The article was written by WL and QC. Y-PC and WG have provided guidance to the manuscript preparation and have also reviewed and edited the paper. All authors have approved the final version of the editorial.This work was partially supported by the National Natural Science Foundation of China , the Natural Science Foundation of Guangxi (Nos. 2021GXNSFAA075041 and 2018GXNSFBA281193), the Science and Technology Base and Talent Special Project of Guangxi (No. AD20159044).The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.