We report a 98.13% test balanced success rate (BSR) of our model based on a sequestered test set of LCMS serum samples. The methodology employed is general and can be readily adapted to other LCMS, or metabolomics, data sets.We provide a pipeline for data preprocessing, biomarker selection, and classification of liquid chromatography–mass spectrometry (LCMS) serum samples to generate a prospective diagnostic test for Lyme disease. We utilize tools of machine learning (ML), e.g., sparse support vector machines (SSVM), iterative feature removal (IFR), and  Borrelia burgdorferi to a human host via an Ixodes tick. Typically a patient will develop an erythema migrans (EM) skin lesion and non-specific symptoms including fatigue, malaise, and joint and muscle pains1. Although an EM skin lesion is the most common manifestation of Lyme disease and is used for clinical diagnosis in endemic areas, not all patients develop or notice an EM skin lesion2. Additionally, southern tick-associated rash illness (STARI) also causes a characteristic EM-like rash and the geographic expansion of its associated vector Amblyomma americanum into Lyme disease endemic areas makes it difficult to accurately diagnose Lyme disease solely by the presence of the characteristic skin lesion4. The diagnosis of early Lyme disease is further confounded by the reliance of current diagnostics on a serological response that might not be fully developed early in infection and is not able to distinguish between current and past infection2. These pitfalls in the current Lyme disease diagnostics invite assessment of non-immune reliant diagnostic approaches.Early Lyme disease develops days to weeks following the transmission of 6. The classification tools developed were based largely on least absolute shrinkage and selection operator (LASSO) statistical modeling that worked well when the liquid chromatography–mass spectrometry (LCMS)) data of the training and tests sets were collected at the same time (i.e. during the same instrument run)7. However, we subsequently realized that the test accuracy faltered when a temporal difference existed for the collection of training and test sample data. This batch effect was in part hypothesized to be due to the sparsity parameters used for LASSO feature selection and in the normalization and imputation approaches used.Previously, we provided proof-of-concept studies for the use of metabolomics to identify host metabolic profiles that could be used as a diagnostic marker of early Lyme disease8. We begin with the hypothesis that feature vectors, or the vectors of metabolite peak areas, for patients with Lyme disease and their healthy counterparts are separated in space when restricted to some reduced set of discriminatory biomarkers. This is the base assumption of sparse, or minimal feature, models for feature selection. Uni-variate statistical tests, e.g. t-tests, identify individual biomarkers that may separate the data11. In contrast, the multi-variate methods employed here select sets of biomarkers that discriminate as a group by exploiting higher dimensional separation between different metabolic classes. Multivariate models in statistics and ML, such as partial least squares-discriminant analysis (PLS-DA), kernel support vector machines, deep learning networks, and decision trees, can over-fit when training on data sets with many features and relatively few samples14. This may be mitigated through hyperparameter tuning: controlling the balance between training and validation accuracy in a cross-validation experiment. Using a sparsity inducing penalty in the SSVM optimization problem reduces the number of parameters available to the model and serves to prevent over-fitting by regularizing the high-dimensional model.In this paper, we use sparse support vector machines (SSVM), a machine learning (ML) tool, to select an optimal set of metabolic biomarkers and then build a metabolite-based diagnostic for Lyme disease19. Evaluations of several ML methods across many different types of metabolomics data can be found20. In particular, SSVM and support vector machines with recursive feature elimination (SVM-RFE) have been successful in identifying important metabolic biomarkers for different cancers18. A review of the various predictive and ML models that have been used in metabolomics data can be found in Ghosh et al.21.ML for classification tasks in metabolomics has seen success for more than a decade. support vector machines (SVM), along with other ML models, have been applied on nuclear magnetic resonance (NMR), LCMS, and gas chromatography–mass spectrometry (GC–MS) metabolomics data, yielding high accuracy and low feature count models for potential metabolite-based diagnostics for conditions such stress, pneumonia, and cancer22. Using SSVM with iterative feature removal (IFR), we improve upon these previous methods, and show that our selected biomarkers and classification model yields greater than a 23.Previously, sparse linear statistical models, such as LASSO and elastic net, have been used to identify serum metabolite biomarkers and build classification models for distinguishing specific Lyme disease manifestations from healthy controls24. A total of 118 training and 118 test serum samples were included. The LCMS data acquired previously were processed using XCMS25. A list of 4851 features were detected in the training samples. After the untargeted selection in XCMS we checked for missingness in the data to identify features with missing values in more than 80% of training samples —none of features met this criterion. The abundance value for each feature was transformed by either the 26. Missing data were imputed using the k-nearest neighbors (KNN) algorithm. Uniform manifold approximation and projection (UMAP) was applied as a visualization tool for identifying possible batch effects in the data27. To bring together sample-batches of the same group, we utilized an IFR algorithm, Algorithm 1, paired with a SSVM classifier to identify and remove batch-discriminatory features23. This was performed for the data generated by each transformation scheme.Early Lyme disease and healthy control serum samples, previously analyzed by LCMS as two separate batches (discovery/training and test), were utilized in this studyk-fold feature selection (kFFS), using SSVM as the classifier. We obtained a selected feature set for each data transformation scheme and these features were then combined, and the raw LCMS and LCMS/MS data of each selected feature evaluated to determine appropriateness as a potential classifying feature . This resulted in a final biosignature of 42 high quality features. These were targeted in both the training and test samples’ data in the Skyline software to ensure accurate peak picking28. As a final step abundance data acquired via Skyline were log transformed, used to train an SSVM classifier with training samples’ data, and tested against the test samples’ data. The pipeline described is provided as python scripts contained in our github repository29—the repository contains all python libaries, scripts, and data necessary to reproduce the results of the paper. However, due to the random choice of partitions in the cross-validation scheme used in both IFR and kFFS small differences in the resulting feature sets may occur.Once sample-batch effect features were removed, feature selection for differentiation of Lyme disease vs healthy controls was performed with 30. This demonstrated that KNN imputation with log transformation on training samples provided the highest mean fivefold cross-validation accuracy  when an SSVM classifier was applied. Median imputation with log transformation performed similarly . Both standardization and median-fold change normalization obtained relatively high accuracy scores with low standard deviations when paired with KNN imputation. Thus, four transformation-imputation methods were moved forward for biosignature development. The complete results of this experiment can be found in the Supplementary_Data directory of our github repository29.Prior to the development of a differentiating biosignature and classifier, we evaluated 18 different combinations of transformation and imputation methods with the 4851 features found in training samples. This included median imputation, knn imputation, half-minimum imputation, standardization, log transformation, quantile normalization, and median-fold change normalizationThe structure of the training samples’ data generated with the four transformation/imputation schemes was visualized by UMAP. As exemplified in Fig. SSVM was applied with kFFS to select features that could populate an early Lyme disease versus healthy control biosignature. This process was performed using the features that remained after correcting the healthy control batch effect, see the “Here we present the results of our classification experiment on test samples. The test samples were comprised of a separate set of early Lyme disease patient samples obtained from NYMC and healthy control samples obtained from NYMC and Tufts. These test samples were also analysed using LCMS at a separate time than the training samples. For this experiment both training and test LCMS samples were targeted in Skyline at the 42 biomarkers derived from the 45 biomarkers described above; see the “Labeling positive as Lyme disease and negative as control, we recorded a BSR of 98.13%, a specificity (TNR) of 100.00%, and a sensitivity (TPR) of 96.25%. The confusion matrix can be viewed in Table Figure 31. These 17 features fell into the following metabolite super classes: organic acids and derivatives, organoheterocyclic compounds, alkaloids and derivatives, organic oxygen compounds, lipid and lipid-like molecules, and organic polymers.The biological relevance of the 42 biomarkers selected by SSVM using the training data were further investigated by LCMS/MS. Of the 42 biomarkers, MS/MS spectra could be obtained for 33 Table . Using tkFFS, but was not in the cut-off used to select the top 42 features. Thus, the m/z 317.4072 ion was replaced by m/z 317.2475 as a discriminatory feature.Manual inspection of the 45 biomarkers selected by SSVM revealed that monoisotopic peaks were not selected in the original list and thus the monoisotopic m/z values were used to replace the original m/z values as indicated in Table Additionally, there were three biomarkers that were removed from Table Our end-to-end pipeline starts with a large set of features detected through a non-targeted metabolomics experiment and produces an optimal set of targeted discriminatory features capable of identifying out-of-sample Lyme disease patients with high accuracy. This pipeline has significant potential for the development of additional ML based LCMS diagnostic tests.In particular, our SSVM classification model classified a sequestered batch of LCMS samples as healthy or having Lyme disease with a 98.13% balanced success rate, 96.25% sensitivity, and 100.00% specificity, see Table 5. Our results are strengthened by the fact that in Molins et. al. all of the test data participated in the same LC-MS runs as the training data—which was used to build their final LASSO model. Our test samples were completely sequestered from the training data which includes the step of being processed by LC-MS (3 month gap), so that none of test samples were run with any of the training samples. Our diagnostic greatly outperforms the models developed in Clarke et al. on 50 peripheral blood mononuclear cell (PBMC) RNA seq biomarkers32. The highest scoring logistic regression model on 50 biomarkers of Clarke et al. yielded approximately  and  as observed from their ROC and Precision-Recall curves; this is in contrast to our models  and . Clarke et al. has each of their batches represented in both training and test—yielding a significantly weaker model than ours, where training and test data are split into separate LCMS batches.Relative to the 44 LC-MS biomarkers discovered and LASSO diagnostic developed in Molins et al. our SSVM diagnostic shows an 8.35% increase in test sensitivity and a 5.00% increase in test specificity11. Our SSVM diagnostic outperforms their best results of 86% TPR and 86% TNR using their positive-ion mode biosignature  in a leave-one-out (LOO) experiment where training data and test data are run in the same batch24. As these pathways have come up multiple times, further investigation into the classification efficacy of all metabolites in these pathways may provide a more robust classifier for Lyme disease. Our null experiment, see the “Not only did our 42 features classify Lyme disease patients with high accuracy, our features were present in all samples upon manual inspection and they are tied to metabolic processes altered during Lyme disease. The included features belong to glycerophospholipid, eicosanoid, tryptophan and phenylalanine metabolic pathways previously shown to be altered during Lyme diseaseMore data needs to be acquired and further analysis needs to be performed to assess the efficacy of the classification model on health states outside of Lyme disease. For example, the model we built used only healthy controls, but it would valuable to see the classification results of patients infected with the common cold or influenza. In future work we propose to extend this test beyond distinguishing between suspected Lyme disease and actual Lyme disease to more specific disease identification.24. Detailed methods for metabolite extraction and LCMS analysis can be found in the cited publication.Serum sample LCMS data acquired previously was utilized24. These two independently processed batches formed our 118 training samples and 118 sequestered test samples respectively. Samples were categorized by the health state labels: EDL, ELL, healthy control non-endemic (HCN), HCE1, and healthy control endemic site 2 (HCE2). Training samples were partitioned as 30 EDL, 30 ELL, 28 HCN, and 30 HCE1. Test samples were partitioned as 40 EDL, 40 ELL, 30 HCE1, and 8 HCE2. We label a sample as Lyme disease if it belongs to either the ELL or EDL group, and label a sample as healthy if it belongs to the HCE1, HCN, or HCE2 group.Early Lyme disease and healthy control serum samples, previously analyzed by LCMS as two separate batches, were utilized in this study34. Peak detection was performed using the centWave algorithm35. Default parameters were used except for ppm = 30, peakwidth = c, and noise = 2000). Peak alignment by retention time was carried out using the obiwarp method with binSize = 0.6 and specifying the centerSample as the sample that was measured in middle of the LCMS run36. Quality control included manual inspection of plots of total ion counts and specified peaks by retention time. Peaks were grouped using the peak density method with default parameters except bw = 5 and minfrac = 0.425.For untargeted feature selection, raw data files were converted into mzML format files using MSConvert (Proteowizard) and then processed using XCMS (3.6.2) in R (3.6.1)kFFS were manually inspected to determine peak quality, whether the monoisotopic peak was chosen, any possible adducts, and feature presence in both runs. After manual evaluation, good quality features were targeted in both the training and test sets using Skyline with suggested settings37. Each peak was manually evaluated to ensure correct integration before exporting peak area values.Features selected by 38. KNN imputes missing data in a sample by finding its k-nearest neighbors, taking the mean of a feature with respect to its neighbors, and then imputing that value for the missing feature. Wahl et al. concludes that KNN imputation performs well across several evaluation schemes and computationally takes less resources39. Modified versions of the KNN imputation algorithm, such as normalized No-Skip KNN (NS-KNN), have been proposed and may even outperform the standard algorithm for real datasets when a significant portion of the missing data is Missing Not at Random (MNAR) type38. For this particular application we used As a first step, any metabolites which were missing in more than 80% of samples across each class of healthy or Lyme disease were removed. No features in our list met this criterion and so no features were removed. All samples with missing values were imputed by the KNN algorithm40. Standardization is defined as shifting and scaling each feature so that its mean is 0 and its variance is 1 across samples. These transformation schemes were chosen to be the best with respect to the classification accuracy of the SSVM model on the training data, amongst other transformation schemes such as quantile normalization26; see the Supplemental_Data directory in our github repository for our complete transformation/imputation experiment29.Once imputed, the samples were transformed by either the 41. Each sample n is the number of features/biomarkers/measurements. SVM classifies samples by first constructing a hyperplane ith row ith sample, C is a tuning parameter for balancing the misclassification rate against the sparsity, b is the scalar affine shift of the hyperplane 43. That is 45.We classify samples into two classes of healthy, entclass1pt{minimaentclass1pt{minimakFFS. First, we randomly partitioned training samples into k non-overlapping and equally-sized parts. We then chose k ways to choose k parts—therefore we obtained a k-fold experiment, known as k-fold cross validation . For each fold of the experiment we extracted features, ordered them by the absolute value of their weight in the SSVM model, grabbed the top k folds, see Fig. We selected features/biomarkers using a new method: 23. Specifically, we perform kFFS , while simultaneous yielding high accuracy and small variance, see Fig. Once we removed features for batch effects we restricted the training data to the remaining features, and we then either mentclass2pt{minimWe classified test samples by first restricting both the training data and test data to the selected features; found by the methods above. We restricted the samples by first targeting these features in Skyline. Once these new feature sets were obtained they were 47. MS/MS spectra were compared with available spectra in Metlin and NIST databases. The level of structural identification followed refined Metabolomics Standards Initiative guidelines proposed by Schymanski et al.31.Confirmation of the chemical structure of selected molecular features (MF) was performed by LCMS/MS. MS/MS spectra were manually evaluated using MassHunter Qualitative software (Agilent Technologies)Supplementary Figures.


Feature selection of multi-omics data analysis remains challenging owing to the size of omics datasets, comprising approximately KTD-based unsupervised FE was reformatted as the collection of kernelized tensors sharing common samples, which was applied to synthetic and real datasets.P values to features, which is rare for existing multi-omics–oriented methods.The proposed advanced KTD-based unsupervised FE method showed comparative performance to that of the previously proposed KTD method, as well as tensor decomposition-based unsupervised FE, but required reduced memory and central processing unit time. Moreover, this advanced KTD method, specifically designed for multi-omics analysis, attributes https://github.com/tagtag/MultiR/.The sample R code is available at  Since the raw P values must be heavily corrected for multiple comparisons when dealing with multi-omics datasets, P values become larger and inevitably less significant; thus, attributing significant P values to individual features, even after correction, is difficult. However, since the number of samples  is less than that of features , labels or values attributed to samples can be accurately predicted by any model .Feature selection with multi-omics datasets has been a long-standing challenge for bioinformatics. Among the numerous proposed methods adapted to multi-omics data analysis , 2, only problem , posing In multi-omics analysis, it is difficult to obtain large sample sizes since multiple observations, each of which corresponds to individual omics approaches, must be performed. In this sense, the required cost and time are multiplied in proportion to the number of omics approaches considered. This often results in a smaller number of samples to which multi-omics measurements are performed when only limited experimental resources are available.large p small n problem. Thus, these approaches are also suitable for feature selection in multi-omics analysis. Recently, the TD-based method was extended to kernel TD (KTD)-based unsupervised FE [P values to features, which is critical since this enables evaluation of the significance of the selected features, which is rarely possible using other methods applicable to multi-omics datasets [Reduction of required memory and long CPU timeIntegration of more than two types of omics datasimultaneously (see Discussions below). We here described a modification of the KTD-based unsupervised FE method to be more suitable for multi-omics data analysis. Although only a small modification was implemented, it nevertheless resulted in more flexibility for multi-omics data analysis, which was verified using synthetic and real data.Principal component analysis (PCA) and tensor decomposition (TD)-based unsupervised feature extraction (FE)  have beevised FE , which wmentclass2pt{minimdatasets . Althouglarge p small n problem. The former, feature selection, is more straightforward than the latter; it reduces the number of features less than the number of samples. On the other hand, feature extraction is more indirect, since it generates a limited number of new features from the original large number of features. In this study, we have employed a mixed strategy of these two. We first generated new features using feature extraction and selected features using the generated features.Before explaining the results, we briefly discuss the relationship between feature selection and feature extraction, both of which are employed when we are forced to deal with the K multi-omics datasets with sth experimental type. sth experimental type. Typical examples of m experimental conditions include human subjects, tissues, and time points. For example, if the measurements are performed for kth omics dataset. When K types of omics data are measured for each sample, Suppose that we have sth experiment type for k.  (HOSVD)  is then g in Eq. ,4\documeNext, the P values are attributed to x and P values are then computed by the pchisq function in R [Finally, ion in R .P values less than the established threshold  are selected. Correction by the BH criterion is performed by the p.adjust function in R with the option of method=“BH”.The obtained riterion  and the N is the number of variables, j is a variable that adds order to the jth sample, and k is the kth omics data. This synthetic dataset assumes that only the top N features have dependency on i independent of k. j, but in an omics(k)-dependent manner. The dependence on j is a linear increase upon j.A synthetic dataset was derived in the form of a tensor, N, M, and K, see the legend to Table k, P valuesi values associated with adjusted One hundred ensembles of mentclass2pt{minimk differ, distinct models were applied to each. After computing BH-corrected P values, the is associated with adjusted P values less than 0.01 were selected.To demonstrate the difficulty of this task, linear regression was used as an alternative methodd in Eq. . Since \M features, which is less than the number of features coincident with When the least absolute shrinkage and selection operator (lasso)  was applWhen random forest (rf)  was applThe real multi-omics HPV vaccine datasets were based on 75 samples measured in 15 individuals at five subsequent time points  after HBV vaccine treatment. Gene expression was measured using RNA-sequencing technology and methylation profiles were measured using microarray technology. The proteome was measured for whole blood cells (WBCs) as well as plasma. Since this multi-omics dataset is composed of four types of omics data measured for 15 individuals at five time points, it is thus formatted as a tensor.XXXXXXX_GRnn_Vm.count.txt.gz,” where XXXXXXX, nn, and m are integers; yyyyyyyy, where yyyyyyyy is an integer. Since the methylation profile was measured with microarray technology, the identifier can be annotated with the reference to the microarray annotation file, GPL6480-9577.txt.gz, which is available under GEO ID GPL6480. Since the other 75 column names are in the form of GSMXXXXXXX, the columns were reordered with reference to the columns of the data frame generated from the gene expression profiles as described above. The proteome dataset was obtained from ProteomeXchange [nn” and “Visit m,” respectively, the columns can also be reordered with reference to the column names of the data frame generated by gene expression profiles as described above.Gene expression and methylation profiles were retrieved from the gene expression omnibus (GEO) database using the GSE155198 and GSE161020 datasets, respectively. For gene expression profiles, the GSE155198_RAW.tar dataset was available in the Supplementary File section of GEO. Individual files that included count number of mapped reads toward genes were collected and integrated as a single file. Individual files are named according to the format “GSMkth feature type measured at the mentclass2pt{minimA linear kernel was employed as follows:As a result, The singular-value vectors of interest were as follows:entclass1pt{minimaentclass1pt{minimaj subjects  when gene expression profiles were considered Table , which wFinally, two set of proteins identified as DEGs in WBC and plasma profiles Table  were uplSince there are many enriched biological processes and pathways listed in Additional file hsa-mir-200c and hsa-mir-141 were selected from TCGA, and hsa-miR-141, hsa-miR-210, and hsa-miR-200c were selected from GEO. Thus, these miRNAs are highly coincident with each other, even more so than reported in previous works , 12 wherThe present implementation can reduce the required computational memory. As a result, required computational time can be reduced as well.The present implementation can integrate more than two omics data in a straight manner.Although the point was achieved in KTD-based unsupervised FE [ith or hth omics data of the jth sample), HOSVD is applied toith measurement of the kth omics data of the jth sample, where K is the total number of omics datasets. Since HOSVD is applied to There are several advantages in the proposed implementation of KTD-based unsupervised FE compared with previously proposed versions of KTD-based unsupervised FE , as wellvised FE , too, thvised FE  is applivised FE  is appliKernels cer data  Fig. 4)\documented Table . Thus, iP value of Although the list of enriched pathways in Additional file mentclass2pt{minimmentclass2pt{minimAlthough genes identified as DEGs in relation to HBV vaccination were also enriched in various neurodegenerative diseases other than hepatitis . For example, although MOFA [P values to individual features; thus, features cannot be selected based on statistical significance, and therefore, DIABLO was considered to be outside of the scope of this study. FSMKL [Not specific to the HTS architectureP values to individual features to evaluate statistical significanceAttributes Can handle more than or equal to three kinds of omics data simultaneouslylarge p small n (typically, Applicable to severe For example, for the large p small n problem, although Subramanian et al. [large p small n problem. As a result, the performance of the KTD-based unsupervised FE was not directly compared to other existing methods that satisfy all of the above conditions.The performance of the proposed method has not been compared to other existing multi-omics–oriented methods , 2 becauugh MOFA  is not sugh MOFA  is also y. FSMKL , which iP values to any features. This suggests that the proposed advanced KTD-based unsupervised FE method has superior ability to select features when applied to even particularly difficult multi-omics datasets.HBV vaccination data were selected to demonstrate the superior power of advanced KTD-based unsupervised FE because of the difficulty of the problem with this dataset. Since vaccination must be given to healthy people, side effects must be minimized ; in factlarge p small n problem. Since it is impossible to demonstrate the difficulty of using all the other methods, we consider two methods, class-specific mutual information variation for feature selection [large p small n problem. When k features were aimed to be selected, although the complexity of class-specific mutual information variation for feature selection was supposed to be kMN, it excluded the computational time needed for the computation of mutual information among N features, which is as large as large p small n problem associated with a large number of features, N. For example, in the synthetic example, we tried to compute mututal information among large p small n problem as well, since it can select at most M features. Multilabel feature selection with constrained latent structure shared term decomposes the matrix k features are selected. Nevertheless, in the large p small n problem, since M features. On the other hand, multilabel feature selection with constrained latent structure shared term was applied to the case where M. Thus, multilabel feature selection with constrained latent structure shared term cannot be used for the present synthetic example. Although these are only two examples, most of the popular feature selection methods are not suitable for the large p small n problem, as shown for these two methods.One might wonder why we did not employ more advanced feature selection methods other than lasso or rf. Generally, other methods are not fitted to the present situation, i.e., the election  and multelection  in orderment}N<M . In our large p small n problem. The proposed method was successfully applied to a synthetic dataset, as well as to two real datasets, and attributed significant P values to features with reduced CPU time and memory, even when applied to integrated analysis of more than two multi-omics datasets. Although the modification from the previously proposed KTD-based unsupervised FE was not significant, this slight modification was successful when applied to feature selection of multi-omics data analysis, which often poses a challenge in the case of a large p small n problem.In this paper, an advanced KTD-based unsupervised FE method was introduced, which was modified to be applied to feature selection in multi-omics data analysis that is often very difficult, mainly based on the Additional file 1. Tables S1 to S4. Supplementary Tables.Additional file 2: Data S1. List of selected features and enrichment analysis for methylation.Additional file 3: Data S2. List of selected features and enrichment analysis for gene expression.Additional file 4: Data S3. List of selected features and enrichment analysis for proteome of WBC.Additional file 5: Data S4. List of selected features and enrichment analysis for proteome of Plasma.


Advances in the expression quantitative trait loci (eQTL) studies have provided valuable insights into the mechanism of diseases and traits-associated genetic variants. However, it remains challenging to evaluate and control the quality of multi-source heterogeneous eQTL raw data for researchers with limited computational background. There is an urgent need to develop a powerful and user-friendly tool to automatically process the raw datasets in various formats and perform the eQTL mapping afterward.In this work, we present a pipeline for eQTL analysis, termed eQTLQC, featured with automated data preprocessing for both genotype data and gene expression data. Our pipeline provides a set of quality control and normalization approaches, and utilizes automated techniques to reduce manual intervention. We demonstrate the utility and robustness of this pipeline by performing eQTL case studies using multiple independent real-world datasets with RNA-seq data and whole genome sequencing (WGS) based genotype data.https://github.com/stormlovetao/eQTLQC.eQTLQC provides a reliable computational workflow for eQTL analysis. It provides standard quality control and normalization as well as eQTL mapping procedures for eQTL raw data in multiple formats. The source code, demo data, and instructions are freely available at  With the development of genome-wide assay of genetic variants, vast of complex traits associated variants have been detected by the genome-wide association studies (GWAS) . HoweverAs the decrease of cost in next-generation sequencing (NGS), RNA-seq technology has been widely used to measure the presence and quantity of transcriptome in eQTL studies . RNA-seqThe rapid development of tools and databases helps researchers to understand complex diseases –15. In rIn this study, we present an automated pipeline for eQTL analysis, termed eQTLQC, featured with a set of rigorous preprocessing approaches for both genotype and gene expression data. The main designs of eQTLQC have been used in data preprocessing and eQTLs identification in our recent work , 21. To As shown in Fig. RNA-seq based gene expression data is usually generated in the following four RNA-seq processing stages and in different data formats: the raw reads stage (FASTA/FASTQ formats), the read-aligned stage (SAM/BAM formats), feature-counted stage (read count format), or the standardized stage (RPKM/FPKM/TPM formats). The eQTL raw data could be at any of the four stages. And these data formats are heterogeneous and require different processing steps before eQTL mapping analysis.The eQTLQC integrates several standard tools and in-house scripts to transform the data formats in the first three stages into the format of standardized stage (i.e. TPM). For expression data in FASTA/FASTQ formats, eQTLQC provides Bowtie , Bowtie2To illustrate the following steps, we will perform an eQTL case study starting from the gene read count table from ROSMAP study , 23, repc reads in We first normalize read counts by gene length and sequencing library depth within-sample using TPM transformation. The TPM value of a gene reflects the relative transcription abundance of this gene in a sample by measuring how many RNA molecules are derived from this gene in every million RNA molecules. The TPM transformation has been widely used in eQTL studies, such as GTEx study . To be nNext, we identify samples with mismatched gender, which might be due to sample swap or mix-up. To achieve this purpose, we first predict the gender of the sample donator by measuring the expression profiles of two gender-specific genes, RPS4Y1 and XIST, which are specifically expressed in males and females respectively. Then, we compare the predicted gender and user-given gender to identify mismatched samples. Usually, the gender-mismatched samples are identified manually based on the scatter plot. To avoid artificial biases, we automatically identify the abnormal samples using the support vector machine (SVM) classifier. Under the hypothesis that only a few samples have incorrect gender labels, we first train the SVM model using the expression profiles of RPS4Y1 and XIST as features and the given gender labels. Then, we predict the sample labels based on the trained model. Samples will be excluded if the predicted labels are different from the user-given labels. In the ROSMAP dataset, we did not detect gender-mismatched samples, as shown in Fig. Samples with problematic gene expression profiles might be caused by sample contamination or failure of library preparation or RNA sequencing etc. In our pipeline, we use three approaches to measure the distance between samples and identify sample outliers, including Relative Log Expression (RLE) analysis, pair-wise correlation-based hierarchical clustering, and D-statistics analysis . A spikeG with genes in row and samples in column, the RLE analysis first subtracts the median expression levels of each gene from the original expression value of all samples in each row. For each sample, the residual expression values of all genes should be centered at zero under the above hypothesis. Then, the RLE analysis makes boxplots for all samples based on residual gene expression values and sorts those boxplots or samples by the interquartile range (IQR) in increasing order from left to right, as shown in Fig. The RLE analysis is under the hypothesis that in a sample only a few genes have extreme expression levels, while most genes have similar expression levels across samples. Given a gene expression matrix P and a distribution D in multi-dimensional space. In our context, we measure the Mahalanobis distances between each sample and the distribution of all samples. The chi-squared p value is calculated for each sample in each cluster. Clusters with p values p value  and RNA integrity number (RIN); and sample-relevant characteristics such as age, sex, and education length. Latent covariates such as environmental factors and population stratification are usually hard to access. In eQTLQC, we use SVA to adjust the known and hidden covariates. To be specified, we use the n in SVA  to adjusFigure Rigorous quality control of genotype data is also critical in eQTL analysis. eQTLQC accepts genotype data in VCF or PLINK formats, which are most widely used. Ten rigorous QC steps in both the SNP level and sample level are applied to improve data quality, as shown in Fig. Step 1: remove markers with excessive missing genotypes. The genotype missing rate reflects the data quality. Variants with systematically missing genotype values have no help toward the downstream analysis and might cause false-positive signals [ signals . In eQTLStep 2: exclude subjects with excessive missing genotypes. Similar to Step 1, the genotype missing rate in subject-level should also be checked. Subjects with a high genotype missing rate may be due to poor quality of DNA samples or library preparation. In the case that genotype data is generated and combined from different sequencing platforms or microarray chips, the missing genotypes are also very common in samples. In eQTLQC, subjects with genotype missing rate Step 3: identify gender-mismatched subjects. The gender of a subject can be inferred from genotypes of SNPs on chromosome X. In detail, for male samples, the homozygosity rate of SNPs on chromosome X is expected to be 1, since male only has one copy of X chromosome. However, the homozygosity rate is much lower for female samples. Comparing sex information in clinical records, the samples with discordant gender should be excluded, which might be due to sample contamination or plating errors [g errors . In eQTLStep 4: set the heterozygous haploid genotypes as missing. The variants on X chromosome in male samples should have haploid genotypes excluding the pseudo-autosomal region. The heterozygous haploid genotypes may exist in gender-mismatched samples, such as female sample labeled as male. After excluding gender-mismatched samples, described in Step 3, heterozygous haploid genotypes may be caused by sequencing or genotype calling errors, which should be removed. In this step, we set the heterozygous haploid genotypes as missing for all samples.Step 5: remove markers violating the principle of Hardy–Weinberg Equilibrium (HWE). SNPs that have genotyping errors may significantly derive from the HWE and should be excluded. Given the frequencies of SNP alleles, allele A and B for instance, in a cohort, the expected frequencies of genotypes AA, AB and BB could be estimated under the HWE hypothesis. If the observed genotype frequencies of the SNP are far from the expected genotype frequencies, which could be evaluated by the Chi-squared test, we filter the SNP violating the principle of HWE. In eQTLQC, markers with HWE test p value Step 6: remove markers with informative missingness. The failure of genotype calling may depend on genotypes, which can result in ’informative missingness’. For example, the successful calling rate of rare homozygous genotypes may be on average lower than heterozygous genotypes [p value threshold is set to enotypes . Bias wienotypes . In eQTLenotypes  to test Step 7: remove markers with low minor allele frequency (MAF). Due to factors like sample availability and cost, eQTL studies usually have sample sizes of dozens to thousands. In studies of limited sample size, variants with smaller MAF may not result in robust association signals. Typically, most eQTL studies apply MAF ectively , 37. In Step 8: exclude subjects with outlying heterozygosity rate. Heterozygosity rate is the proportion of heterozygous genotypes for an individual, which can reflect DNA sample qualities. Samples with excessive heterozygosity rates may be due to sample contamination, and samples with reduced heterozygosity rates may indicate sample inbreeding [breeding . The disStep 9: identify related and duplicated subjects.  Related and duplicated samples violate the prerequisites of eQTL mapping analysis where the linear regression model is usually used. Identification of related or duplicated subjects is based on two metrics: identity by state (IBS) and identity by descent (IBD). For each pair of subjects, the IBS score can be observed as 0, 1, and 2 at a given marker by counting the number of shared alleles. In other words, IBS0 represents two different alleles, and IBS1 and IBS2 represent 1 and 2 common allele(s) respectively. The shared allele(s) may be inherited from a recent common ancestor, and in this case, those shared alleles are called IBD. IBD can be estimated from genome-wide IBS scores [S scores . The expStep 10: identify and exclude subjects with divergent ancestry. Population stratification could bias the eQTL analysis as a major confounder, which needs to be adjusted by confounding adjustment approaches. Outlying subjects from divergent populations will enlarge the effect of population stratification and should be removed. To identify the population outlying samples, smartPCA [smartPCA  is emploGenotype imputation. The genotype profiles that passed rigorous quality controls could be used for eQTL analysis. However, before preceding to eQTL analysis, genotype imputation is recommended for microarray-based genotype data, which usually has a limited number of markers. Excellent hidden Markov model (HMM)-based genotype imputation tools have been developed such as minimac3 [minimac3 , minimacminimac3 , Beagle minimac3  and IMPUminimac3 . As the minimac3 . The MISminimac3 .334 subjects passed the quality controls in the preprocessing of both gene expression data and genotype data. 37.7% subjects are male, with an average age of 85.8, average RIN of 7.2 and average PMI of 7.5 h. 26,663 genes and 6,736,714 variants with MAF p value distributions were shown in Fig. The final eQTL mapping analysis is conducted using R Package MatrixEQTL . By defaTo evaluate the robustness of our pipeline, we conducted similar data preprocessing and eQTL mapping for the other three independent datasets: MayoRNAseq (Mayo) , MSBB 2 and Commp value less than 0.05. The upper side indicates the Pearson’s correlations of lower scatter plots, and the red star sign indicates the significance of correlation. We can see most Pearson’s r values are larger than 0.9, representing strong concordance. These results indicate that our pipeline could generate robust eQTL signals in different datasets.To further evaluate the concordance among independent datasets, we compared the effect sizes .Two main phases are included in the eQTLQC framework: the data preprocessing phase and eQTL mapping phase, as shown in Fig. In preprocessing of genotyping data, eQTLQC accepts widely used VCF and PLINK formats. Ten rigorous quality control steps are followed up to remove outlying variants and samples, including: (1) remove markers with excessive missing genotypes; (2) exclude subjects with excessive missing genotypes; (3) exclude subjects with gender-mismatch; (4) remove markers with heterozygous haploid genotypes; (5) remove markers violating Hardy-Weinberg Equilibrium (HWE); (6) remove markers with informative missingness; (7) remove markers with low minor allele frequency (MAF); (8) exclude subjects with abnormal heterozygosity rate; (9) exclude related subjects; and (10) identify and exclude individuals with divergent ancestry. After these rigorous quality control steps, clean genotyping data together with clean gene expression data will be reformatted to fit the requirement of MatrixEQTL, used in the eQTL mapping phase. In the following context, we will introduce the details of each step in the preprocessing of gene expression data and genotype data, and also the eQTL mapping phase based on real-world datasets generated by ROSMAP studies.www.synapse.org).The Religious Orders Study (ROS)  and Memo


MuLT performance was assessed by cross-validation experiments, in which it predicted treatment sensitivity with 68.70% of AUC. Finally, simulations showed numerical evidences that in average 17.07% of patients could get better response to a different treatment at the first line.Providing treatment sensitivity stratification at the time of cancer diagnosis allows better allocation of patients to alternative treatment options. Despite many clinical and biological risk markers having been associated with variable survival in cancer, assessing the interplay of these markers through Machine Learning (ML) algorithms still remains to be fully explored. Here, we present a Multi Learning Training approach (MuLT) combining supervised, unsupervised and self-supervised learning algorithms, to examine the predictive value of heterogeneous treatment outcomes for Multiple Myeloma (MM). We show that gene expression values improve the treatment sensitivity prediction and recapitulates genetic abnormalities detected by Fluorescence  Multiple Myeloma (MM) is a cancer of plasma cells, the second most common neoplasm. It is considered an incurable disease, with MM patients having a mean survival of five years , charactGiven the massive increase in data available to the medical research community, there are opportunities in the field of Machine Learning (ML) for new approaches regarding prognostic assessment. Hence, analysis of cancer high-throughput “omics” data in combination with clinical ones may lead to a more precise characterization of the disease and is likely to have higher clinical utility. Of note, TS prediction is one of the critical tasks and has the potential to benefit a subset of the patients who may be associated with serious side effects. However, proportional to the biological complexity associated to cancer, there are several computational challenges that should be addressed while creating models related to MM. High dimensional data with few samples require specific care about the modelling processes in order to guarantee that final patient classifiers are not overfitted . The unbCurrent approaches focus on generating combined clinical and molecular markers . DespiteDespite the significant advances promoted by STL, an approach considering the interplay of clinical markers, gene expression levels, and treatments into the same model still remains to be evaluated. Herein, we proposed the Multi Learning Training approach (MuLT) which aggregates clinical and molecular data to predict TS. This allow us to perform simulations to estimate which treatment could maximize sensitivity response of each patient.in situ hybridization (FISH) analysis. We further show that TS prediction accuracy increases by incorporating the gene expression levels. Next, we proposed the MuLT that combines supervised, unsupervised, and self-supervised methods to accurately predict TS. Finally, our simulations pointed out alternative first line treatment options for 17.07% of patients.We applied our approach on molecular and clinical data measured from 1,525 patients with newly diagnosed MM from the Multiple Myeloma Research Foundation (MMRF) CoMMpass study . We firsOur study started by assessing if the main markers employed in the current MM clinical decision-making process  could be20q13 marker had the worst performance with average AUC of 80.04%(±10.11%), while the predictor for the t—WHSC1 marker reached 100.00%(±0.00) on the same metric. Genes used as predictors for FISH markers and detailed metrics, including train and test AUC and loss, are listed at According to the Treatment Response (TR), patients were categorized into one of six classes: Progressive Disease (PD), Stable Disease (SD), Partial Response (PR), Very Good Partial Response (VGPR), Stringent Complete Response (SCR), and Complete Response (CR). TS is defined as a binary marker derived from TR, categorizing patients into either treatment sensitive or treatment non-sensitive classes. In our study, patients identified as SCR or CR compose the treatment sensitive class, and those in any of the remaining categories are in the non-sensitive one. The MMRF data set contains more than 700 patients data annotated with the TR outcome, and a relatively low prevalence (30%) of patients with the Days to Disease Progression (DDP) information. The reduced number of patients annotated with DDP in the MMRF cohort hampers the design of precise risk models. We overcome this by using TS as a surrogate for DDP. To further assess our approach, we apply it to stratify the DDP patients as shown in We used the SMLA  as input. Then we performed a second independent experiment, creating another TS predictor combining that single clinical marker to selected genes. All experiments were performed over 10-fold CV.m protein marker produced the largest gain of 25.76%, while the lgg marker had the smallest gain of 0.96%. The For each clinical marker, In an attempt to improve TS prediction accuracy, we proposed MuLT, that comprises the following integrative approaches: i) combining clinical makers, gene expression levels, and treatments to compose a more sophisticated patient description; and ii) creating new representation  of gene GP and GC steps were motivated by the fact that different genetic profiling and gene clusters could be associated with some transcription factors and known recurrent translocations that imply in better or worse overall survival . In addiThe Treatment Sensitivity Predictor Training (TSPT) step creates a TS predictor using a supervised learning algorithm. It inputs all features previously selected and created during Marker Selection and Extraction . As a leThe final classifier is based on the Light Gradient Boosting Machine (LightGBM)  algorithMuLT was evaluated via 10-fold CV. We split the data set into ten folds of similar sizes. To avoid bias, we randomly equalized the number of sensitive and non-sensitive patients and treatments per fold. For each experiment, we employed nine folds to compose the training data set used to apply MuLT, and one fold to compose the validation data set used to evaluate our proposed approach. MuLT performed with an average AUC of 68.70%, ranging from 59.49% to 74.70%, with standard deviation of 4.66%.−5 computed via t-test.We face the challenge of fairly comparing results to related work taking into account data set composition, available markers, and clinical outcome definition . To deal with this limitation, we carried out new experiments using SMLA with four different ML models, namely MLP , LightGBThe overall classification performance per treatment is shown in The MMRF data set contains a total of 55,103 genes and 26 clinical markers. The marker selection is based on Algorithm 1  and 5 clinical markers  were selected in common among all experiments. GIHCG belongs to a family of non-coding RNA (ncRNA) and has been associated with prognosis in hepatocellular carcinoma  and colop < 0.05) among the non-sensitive and sensitive groups , we computed the percentage of patients that received treatment a while simulations indicated treatment b . Simulations were performed over the 10-fold CV experiments. For each experiment and patient in the validation data set, we employed the trained TS predictor to estimate the TS scores associated with different treatments . Once this simulation is over, we have a data set of TS scores associated with patients and each possible treatment. For each patient, we chose simulated treatment as the one with the highest TS score. Finally, for each pair of treatments (Simulation results indicate that 17.07% of patients would have better sensitivity using a different first line treatment (see Treatment Simulation section for details). Our approach sheds light on underlying aspects of disease heterogeneity, which allows further understanding of the interplay between clinical and molecular data coupled with treatment. See (i) investigated how well gene expression levels can predict FISH markers, (ii) estimated accuracy gain by adding genes to TS predictors based on clinical markers, (iii) defined MuLT to advance the state-of-the-art in MM TS prediction, and (iv) simulated MM optimal treatment in a personalized manner by using TS predictors. High quality data sets containing clinical markers, gene expression levels, and treatments related to the same patient and associated to TR clinical outcome are a requirement to create MM models that are helpful for understanding why patients that are expected to be sensitive to a certain treatment do not and vice-versa. Moreover, the data sets must have a large number of samples to allow ML algorithms to capture the complex interplay among the clinical and molecular data. We next restricted analysis to the MMRF data set, which contains data from more than one hundred different sites. To the best of our knowledge, there is no other public MM data set with the required quality and quantity.Identifying which treatment could maximize patient survival is a possible way to improve current clinical decision-making processes. In this study, we Related work has performed analysis using different and independent data sets , 31, 32,Next steps are related to applying MuLT over different cancer data sets composed of clinical markers, gene expression levels, and treatment. This study was limited to binary classification, stratifying patients into either treatment sensitive or non-sensitive, but it could be generalized to perform regressions and multi-categorical classification. We are also interested in investigating more robust treatment representation. Techniques like Word Embedding  can be aWe extracted clinical markers, gene expression levels, FISH, treatments, and survival data from MMRF CoMMpass  . This study was directed to modeling TS. For that, we created the TS outcome, that is defined from TR. We choose the mapping between TR and TS classes using Cox proportional hazards regression models (CHMs) . Once TRMarker selection based on our marker selection algorithm (see Algorithm 1);Hyper-parameter optimization using the BO algorithm; andBinary classifier  training.We defined a baseline pipeline called SMLA in order to perform three different analyses: (i) to identify if it is possible or not to predict structural genetics markers carried by FISH technology from gene expression levels—they are particularly relevant once these markers are broadly used in the current clinical decision-making process in MM; (ii) to measure the accuracy gain obtained by combining clinical markers and gene expressions levels; and (iii) to assess MuLT classification performance against a benchmark. Hence, SMLA is composed of three steps:For the first and second analyses we employed the LightGBM classifier, and for the third one we performed three independent experiments employing LightGBM, MLP and SVM classifiers. l defined as:y represents expected values and yi and ith elements of their corresponding vectors, and N is the size of vectors y and f returns the log loss after training a predictor P, given hyper-parameter values H, and a data set D.We employed a BO algorithm that uses Gaussian Processes (GP)  to minimH values that correspond to the hyper-parameters described in f and creates a surrogate function that fits the hyper-parameters values (H) to their resulting log loss. Based on the surrogate function, BO estimates new hyper-parameter values that are expected to return the global minimum log loss. It then computes f again for these estimated values and updates the surrogate function. This procedure is repeated over 50. Finally, BO returns the hyper-parameter values associated with the global minimum log loss. This process is used in all of our experiments that employ BO, the detailed implementation is available at https://git.io/JGBsv.For each learning method, BO initially generates 10 random m of a marker M by n(m) = (m−minM)/(maxM−minM). Both maxM and minM are extracted from the training data set, and are also used to normalize the validation data set via min)). Minimum and maximum work for limiting output to the interval .MuLT aims to create TS predictors that estimate if a patient is sensitive to a particular treatment based on clinical markers, gene expression levels, and treatment. As stated previously, our experiments were carried out using CV, splitting data into 10-folds. We used nine folds to compose the training data set and one to compose the validation data set. All steps described below were applied on training data, while the validation data was used only for inference and result analysis. Our novel ML-based approach starts by normalizing each value F be a matrix composed of markers at columns and patients at rows, where each element mpM describes the value of a marker M associated with a patient p. Let c be a vector associating a treatment sensitivity class  to a patient p. Based on normalized training data set, Algorithm 1 inputs F and c, and splits the patients along sets (rows of F) based on the clinical outcome class described by c. For each marker M (column of F), the algorithm tests whether these two classes originate from the same distribution based on KS test [α, excluding markers with p-values higher than α. After that, the algorithm computes a pairwise linear correlation for remainder markers. For each marker β > 0.75 with any marker F′. This marker selection algorithm is applied over genes (GS step) and clinical markers (CMS step) independently, generating selected genes G′, and selected clinical markers C′.Let  KS test  for a siOur marker selection algorithm has two main objectives: (i) to identify which markers have different distributions when separated by TS classes; and (ii) to exclude markers that have their information encompassed by a more relevant marker. In order to implement the first objective, our algorithm applies a KS test between values of a marker among the TS classes. We also considered different approaches, testing the Maximal Information Coefficient (MIC)  with difAlgorithm 1 Pseudocode of the marker selection algorithm.function MarkerSelection 1: ExcludedMarkers ← {}2:  AnalysedMarkers ← {}3:  MarkerSet ← names(F)       ⊳ Get marker names from matrix F4:  for allM1 ∈ MarkerSetdo        ⊳ For each marker M1, do5:  AnalysedMarkers ← AnalysedMarkers ∪ {M1}    ⊳ Add M1 to AnalysedMarkers6:   7:   ifpM1 > αthen       ⊳ If p-value associated to marker M1 is greater than α8:   ExcludedGenes ← ExcludedGenes ∪ {M1}   ⊳ Add M1 to ExcludedMarkers set9:    else           ⊳ For each marker that was not analysed, do10:   for allM2 ∈ MarkerSet \ AnalysedMarkersdo11:    12:     ifpM2 > αthen13:     ExcludedMarkers ← ExcludedMarkers ∪ {M2}14:      else15:     16:      ifpc > βthen       ⊳ If linear correlation is greater than β, then17:      ifpM2 ≥ pM1then18:       ExcludedMarkers ← ExcludedMarkers ∪ {M2}19:        else20:       ExcludedMarkers ← ExcludedMarkers ∪ {M1}21:        S ← MarkerSet \ ExcludedMarkers22:  F′ ← FS*23:  return F′24:  G′, applies it to estimate patient genetic profiles by using the k-means algorithm [i to its Euclidean distances from a cluster centroid j, where m is the number of patients.GP inputs the selected genes lgorithm , and reto is defined by our Number of Clusters Selection (NCS) algorithm, which is based on the average Silhouette Coefficient (SC) [G′ and iterates over the number of clusters c = 2, 3, … to compute a Silhouette Coefficient-based Metric (SCM), defined byc clusters, σSC(c) is the standard deviation of SC for c clusters, and σNS(c) is the standard deviation of the number of samples in each cluster for c clusters. NCS stops after 10 iterations without getting a higher value for SCM, returning the number of clusters o = arg maxcSCM(c).Number of clusters ent (SC) . NCS inpGT′, estimates gene clusters using k-means algorithm, and returns a matrix i to the average expression level of each estimated gene cluster j, where k is the number of gene clusters defined by NCS algorithm.GC inputs the transpose of selected genes matrix G′ and returns a denoised representation Gd of selected genes. It is defined using a Deep Denoising Autoencoder (DDA) [i of input and output units equals the number of selected genes. Processing layers has ⌊0.5i⌋, ⌊0.4i⌋, ⌊0.3i⌋, ⌊0.4i⌋, and ⌊0.5i⌋ units, where ⌊⋅⌋ is the floor operation. A DDA model is trained by adding a noise to the input and then reducing the Mean Squared Error (MSE) between the raw input and the DDA output. We modified the input values adding a noise generated from a random variable GD inputs selected genes er (DDA) . DDAs arfunction  on procefunction  was emplG′ | C′ | L | E | Gd | T) and outputs a model able to predict an individual patient TS, where T is a matrix associating patients to its first line treatment. TSPT is defined by LightGBM [Finally, TSPT inputs the concatenation  is a trained predictor that returns a TS score.We performed simulations based on MuLT predictors in order to identify which treatment could maximize the TS score for a given patient tp is the actual treatment for a given patient p, Np is the number of patients in the cohort, ζ is a function that returns 1 if a and b are not equal and 0 otherwise.Next, the relative number of patients that had a simulated treatment different from the actual one was computed by the equation below:S1 Fig(PDF)Click here for additional data file.S2 Fig(PDF)Click here for additional data file.S3 Fig(PDF)Click here for additional data file.S1 File(ZIP)Click here for additional data file.S2 File(ZIP)Click here for additional data file.S3 File(ZIP)Click here for additional data file.S4 File(ZIP)Click here for additional data file.


Currently, a comprehensive method for exploration of transcriptional regulation has not been well established. We explored a novel pipeline to analyze transcriptional regulation using co-analysis of RNA sequencing (RNA-seq), assay for transposase-accessible chromatin using sequencing (ATAC-seq), and chromatin immunoprecipitation with high-throughput sequencing (ChIP-seq).The G protein-coupled receptors (GPCRs) possibly associated with macrophages were further filtered using a reduced-Cox regression model. ATAC-seq profiles were used to map the chromatin accessibility of the GPRC5B promoter region. Pearson analysis was performed to identify the transcription factor (TF) whose expression was correlated with open chromatin regions of GPRC5B promoter. ChIP-seq profiles were obtained to confirm the physical binding of GATA4 and its predicted binding regions. For verification, quantitative polymerase chain reaction (qPCR) and multidimensional database validations were performed.in vitro how its overexpression could regulate GPRC5B. GPRC5B was significantly downregulated in colon adenocarcinoma (COAD) as seen in 28 patient samples. The correlation between GPRC5B and macrophages in COAD was validated using multiple databases.The reduced-Cox regression model revealed the prognostic value of GPRC5B. A novel pipeline for TF exploration was proposed. With our novel pipeline, we first identified chr16:19884686-19885185 as a reproducible open chromatin region in the GPRC5B promoter. Thereafter, we confirmed the correlation between GATA4 expression and the accessibility of this region, confirmed its physical binding, and proved GPRC5B, correlated with macrophages, was a key GPCR affecting COAD prognosis. Further, with our novel pipeline, TF GATA4 was identified as a direct upstream of GPRC5B. This study proposed a novel pipeline for TF exploration and provided a theoretical basis for COAD therapy. This study provides a novel pipeline to explore transcription factors based on multi-omics data, which is described adequately enough to be repeated and taken further.With our pipline, GATA4 was identified to be the direct upstream transcription factor regulating GPRC5B.GPRC5B may affect COAD patient prognosis, possibly by interacting with macrophagesColon cancer, a type of malignant tumor, is the third leading cause of cancer deaths worldwide –3. ColonIn recent years, increasing evidence has revealed that immune infiltration might be an essential factor in COAD patient prognosis . Tumor-iG protein-coupled receptors (GPCRs), a group of cell surface signaling proteins, represent the most prominent superfamily of pharmacological targets , 9. It hCurrently, to explore the regulation of a target gene, various algorithms would be applied to transcriptome data for pathway quantization, followed by correlation analysis. If the target gene was statistically correlated with a quantized pathway, its involvement in the regulation of this pathway would be proposed. Besides analysis of transcriptome data from RNA sequencing (RNA-seq) profiles, our study further acquired epigenetic data to explore the target gene’s regulation. Assay for transposase-accessible chromatin using sequencing (ATAC-seq) used Tn5 transposase to determine the nucleosome position and map the open chromatin regions , 20. The(qPCR), chromatin immunoprecipitation with high-throughput sequencing (ChIP-seq), and multidimensional databases.Here, multi-omics bioinformatics was employed to discover the macrophage-correlated GPCRs that might play a key role in COAD. With transcriptome data from RNA-seq profiles, we explored the GPCRs that might be associated with tumor infiltrating macrophages. GPRC5B was eventually selected based on its clinical value. Moreover, epigenetic data from ATAC-seq profiles were also obtained to explore regulation mechanisms. The above-mentioned results were verified through quantitative polymerase chain reaction Our study was approved by Tongji Hospital, Shanghai, China (reference number 2018-LCYJ-005). Written consent was obtained from all participants/patients before the study.https://gdc.cancer.gov/about-data/publications/ATACseq-AWG). We acquired the Fragments Per Kilobase per Million mapped reads (FPKM) and htseq-count profiles of 514 samples from TCGA database (https://tcga-data.nci.nih.gov/), including 473 COAD samples and 41 solid normal tissue samples. Clinical demographic information of 426 COAD patients was also retrieved. The baseline information of all COAD patients is provided in The ATAC-seq profiles of 41 COAD samples were obtained from the NCI Genomic Data Commons  (The list of GPCRs was downloaded from the GPCR NaVa database (iacs.nl)  and genehttps://gdc.cancer.gov/about-data/publications/ATACseq-AWG).Publicly available datasets were analyzed in this study, including 41 paired ATAC-seq and RNA-seq profiles of the same COAD patients. The sources of these profiles have been described in data collection section. The upstream analysis of ATAC-seq data was completed following the pipeline proposed by M. Ryan Corces et al. And a total of 122872 reproducible peaks were observed in 41 ATAC-seq profiles of COAD patients. In this study, we directly downloaded these peaks from the supplemental data file “cancer type-specific count matrices in normalized counts” . We obtained the 45377 peaks which were annotated as “Promoter (<=1kb)” or “Promoter (1-2kb)”. 2) Getting the peaks located in the promoter region of target gene. We first used the R package “GenomicFeatures” to check the gene location of the target gene GPRC5B . Then, we search the above 45377 peaks, and we found the target peak (chr16:19884686-19885185) which had an overlap with the gene location of GPRC5B (chr16: 19856691-19886167). 3) Getting the mRNA expression of TFs. We downloaded the list of TFs from the Cistrome database (http://cistrome.org/). Based on the list, we can get the TF mRNA expression from RNA-seq profiles. 4) We used the R package “stats” (cor.test function) to perform the Pearson correlation analysis between the TF mRNA expression and the ATAC-seq peak accessibility of the target peak (chr16:19884686-19885185). The correlation threshold was set as an absolute value of r > 0.2, p < 0.01. In this study, the TF GATA4 expression was found to be most highly correlated with the target peak (chr16:19884686-19885185). 5) We used the Cistrome to browse the GATA4 ChIP-seq data , and we can check the overlap between the target peak (chr16:19884686-19885185) and the peaks of GATA4 ChIP-seq data. With the data above, we proposed a novel pipeline to explore TF regulation based on multi-omics data. Our pipeline included: 1) Peak annotation. We used the R package “ChIPseeker” to annotate peaks   , 28.https://www.ncbi.nlm.nih.gov/geo/), Timer 2.0 database (http://timer.comp-genomics.org/) (http://gepia.cancer-pku.cn/) (https://www.proteinatlas.org/) (https://linkedomics.org/)  , 30, GEPpku.cn/) , Human Pas.org/) , 33, andcs.org/) . We alsocs.org/) , XCELL (cs.org/) , TIMER (cs.org/) , and MCPcs.org/)  algorithcs.org/) .t-test was applied to screen the significant differences.We additionally validated the mRNA expression level of GPRC5B using fresh frozen tissue samples. COAD samples and paired normal tissues were obtained from 28 patients from Tongji Hospital, Shanghai, China. TRIzol reagent  was used for RNA extraction following the manufacturer’s instructions. After reverse transcription, qPCR was performed using cDNA, SyberGreen  and human GPRC5B primers . The amplification program was set as follows: 95°C for 210 sec; 40 cycles at 95°C for 210 sec, 58°C for 30 sec, 63°C for 20 sec; melting curve from 58°C to 95°C. Every sample in the qPCR experiment was repeated in triplicate. Additionally, human GAPDH was selected as the internal control . A paired https://CRAN.R-project.org/package=survminer). The differentially expressed genes were identified by R package edgeR (Version 3.28.1), and the statistical significance was set at adjusted p < 0.05. R  was the main analysis software.Statistical significance was set at p < 0.05. The correlation threshold was set as an absolute value of r > 0.2, p < 0.01 in Pearson or Spearman analysis. Variable normality was checked using Shapiro-Wilk normality test. For nonnormally distributed variables, the Wilcoxon rank-test was used for two independent group comparisons, while the Student’s t-test was used to compare normally distributed variables. Paired data which were normally distributed were analyzed by paired t-test. In Kaplan-Meier survival analysis, to prevent the bias caused by non-tumor-related death, we obtained results only from patients who had a follow-up time of more than 90 days. The optimal cutoffs for Kaplan-Meier survival analysis were determined by R package survminer , p < 0.05, First, we applied the R package EdgeR  to the h). For visualization, the pie plot indicated that the open chromatin regions were primarily located in the promoter regions  . GPRC5B was identified as a differentially expressed gene and found to be upregulated in the treatment group  was found to overlap with the GATA4 binding regions to a great extent. We also found that, with the upregulation and downregulation of GATA4 expression, GPRC5B would be regulated accordingly. Collectively, through this novel pipeline, we identified GATA4 as the direct upstream TF of GPRC5B.in vivo. Future studies should experimentally verify our findings.Several inevitable limitations need to be addressed. First, our pipeline required paired ATAC-seq and RNA-seq profiles of the same samples, the amount of which was relatively small in public databases. However, despite the limited data sources, we acquired 41 eligible paired profiles to prove the effectiveness of our pipeline. Second, although we have identified GPRC5B as a key molecule in COAD prognosis through bioinformatics, we have not put forward any proof Despite the limitations mentioned above, our study was the first to infer that GPRC5B, correlated with tumor-infiltrating macrophages, might be a key molecule affecting COAD prognosis. The interaction between GPRC5B and tumor-infiltrating macrophages could be a potential target for clinical therapy. We also proposed a novel pipeline, identifying GATA4 as a direct upstream TF of GPRC5B. This pipeline was based on the integration of multi-omics data, which was easy to apply and could be used to achieve a more convincing conclusion.GPRC5B, correlated with tumor-infiltrating macrophages, is a potential key molecule affecting COAD prognosis. Further, with our novel pipeline, TF GATA4 was identified as a direct upstream of GPRC5B. This study proposed a novel pipeline for TF exploration and provided a theoretical basis for COAD therapy.https://portal.gdc.cancer.gov/, and https://gdc.cancer.gov/about-data/publications/ATACseq-AWG.Publicly available datasets were analyzed in this study. This data can be found here: The studies involving human participants were reviewed and approved by Tongji Hospital, Shanghai, China (reference number 2018-LCYJ-005). The patients/participants provided their written informed consent to participate in this study.Conception/design: CH, ZN, BG, BD, and QH. Collection and/or assembly of data: CH, NZ, HX, ZN, XL, BL, BG, BD, and QH. Data analysis and interpretation: CH, HX, NZ, NW, ZN, BL, BD, and QH. Manuscript writing: CH, NW, ZC, and QH. All authors read and approved the final manuscript.This study was funded by Shanghai Science and Technology Innovation Action Plan (Grant No. 19441905700) and the Clinical Research and Cultivation Project of Shanghai Tongji Hospital (Grant No. ITJ (ZD) 1802, ITJ (ZD) 1804).The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


When identifying differentially expressed genes between two conditions using human population RNA-seq samples, we found a phenomenon by permutation analysis: two popular bioinformatics methods, DESeq2 and edgeR, have unexpectedly high false discovery rates. Expanding the analysis to limma-voom, NOISeq, dearseq, and Wilcoxon rank-sum test, we found that FDR control is often failed except for the Wilcoxon rank-sum test. Particularly, the actual FDRs of DESeq2 and edgeR sometimes exceed 20% when the target FDR is 5%. Based on these results, for population-level RNA-seq studies with large sample sizes, we recommend the Wilcoxon rank-sum test.The online version contains supplementary material available at 10.1186/s13059-022-02648-4. RNA-seq is a transcriptome profiling approach using deep-sequencing technologies –3. SinceTo evaluate the performance of DESeq2 and edgeR on identifying DEGs between two conditions, we applied the two methods to 13 population-level RNA-seq datasets with total sample sizes ranging from 100 to 1376  of the 109 RNA-seq samples in this immunotherapy dataset (Methods). Since any DEGs identified from these permuted datasets are considered as false positives, we used these permuted datasets to evaluate the FDRs of DESeq2 and edgeR. Surprisingly, DESeq2 and edgeR had 84.88% and 78.89% chances, respectively, to identity more DEGs from the permuted datasets than from the original dataset  were more likely to be identified as DEGs by the two methods from the permuted datasets Fig. : Fig. S2Out of curiosity and as a means of verification, we investigated the biological functions of the spurious DEGs identified by DESeq2 or edgeR from the permuted datasets. Unexpectedly, these spurious DEGs' top 5 enriched gene ontology (GO) terms included immune-related terms Fig. E. Hence,Then, a question followed: why did DESeq2 and edgeR make so many false-positive discoveries from this immunotherapy dataset? Our immediate hypothetical reason was the violation of the negative binomial model assumed by both DESeq2 and edgeR . To checMotivated by these findings, we further benchmarked DESeq2 and edgeR along with four other representative DEG identification methods on this immunotherapy dataset and the other 12 population-level RNA-seq datasets from two large consortia, the Genotype-Tissue Expression (GTEx) project  and the While the permutation analysis created true negatives (non-DEGs) to allow FDR evaluation, it did not allow the evaluation of DEG identification power, which would require true positives (DEGs) to be known. Hence, we generated 50  semi-synthetic datasets with known true DEGs and non-DEGs from each of the 12 GTEx and TCGA datasets. Then, we used these semi-synthetic datasets to evaluate the FDRs and power of the six DEG identification methods (Methods). In comparing 386 heart left ventricle samples and 372 atrial appendage samples in a GTEx dataset, only the Wilcoxon rank-sum test consistently controlled the FDR under a range of thresholds from 0.001 to 5% ; however, the central limit theorem only holds when the two samples are from distributions with finite second moments, excluding the possibility that samples may come from heavy-tailed distributions with infinite second moments [t test is not invariant to non-linear monotone transformations on the observations; for example, the two samples may have the same population mean on the original scale  but different population means on the log scale ; there is no consensus on what transformation is optimal for RNA-seq data. In contrast, the Wilcoxon rank-sum test is invariant to monotone transformations. Third, the t test only concerns the mean difference between the two samples’ distributions, but the mean parameter is not informative if a distribution is heavily skewed, and estimating the mean parameter is not robust to the existence of outliers. Unlike the t test, the Wilcoxon rank-sum test has no requirement on the data distributions. It concerns the null hypothesis that a random observation from one distribution has equal chances of being less or greater than a random observation from the other distribution, an informative null hypothesis to test against regardless of the skewness of distributions. Admittedly, the DE genes found by the Wilcoxon rank-sum test may have distributional differences other than the mean difference between the two conditions [We did not include the two-sample t test or the Welch’s correction as an alternative to the Wilcoxon rank-sum test for three reasons. First, if a gene’s two sets of observations under the two conditions  are from non-Gaussian distributions (which is the case for RNA-seq data), the  moments . Second,nditions , but we Finally, we note that, unlike DESeq2, edgeR, limma-voom, and dearseq, the Wilcoxon rank-sum test is a non-regression-based method, making it unable to adjust for confounders. Hence, to use the Wilcoxon rank-sum test for DEG identification, researchers can normalize RNA-seq samples to remove batch effects or use the probabilistic index models to adjust the Wilcoxon rank-sum test for covariates , 42.p-value-free FDR control solution for small-sample-size data [In conclusion, when the per-condition sample size is less than 8, parametric methods may be used because their power advantage may outweigh their possibly exaggerated false positives. However, if users are concerned about FDR control, our recent method Clipper provides a ize data ; for larWe selected the three parametric methods for identifying differentially expressed gene (DEGs) from RNA-seq data based on popularity: DESeq2 , edgeR , and limhttps://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE91061).For the immunotherapy study, we selected one dataset with a total sample size of 109, including 51 pre-nivolumab and 58 on-nivolumab anti-PD-1 therapy melanoma samples .For TCGA data, we selected RNA-seq datasets of six cancer types, which have paired normal tissues and sample sizes greater than 50 for both cancer and normal tissues. Then, we downloaded the gene read count matrices of these selected datasets from GDC Xena Hub .For GTEx data, we selected six pairs of tissues with sample sizes ranging from 126 to 706. Then we downloaded the gene read count matrices of these tissue samples from the GTEx Portal  including GTEx , TCGA 66, and anAdditional file DESeq function to perform differential analysis, followed by generating the results using the results function.For DESeq2 (v1.28.1), we used the filterByExpr function, followed by normalization using the trimmed mean of M values (TMM) method. Then, the quasi-likelihood F-test was used for differential analysis.For edgeR (v3.30.3), we first filtered out genes with very low counts using the lmFit and eBayes functions.For limma-voom (v3.44.3), we filtered genes and calculated the normalization factor in the same way as we did for edgeR. Then we applied the voom transformation to the normalized and filtered count matrix and performed the differential analysis using the noiseqbio function to identify DEGs.For NOISeq (v2.31.0), we used the dear_seq function with the asymptotic test to identify DEGs.For dearseq, the filtering and normalization steps were the same as those for edgeR. Then, we used the p-value calculation, we input each gene’s counts-per-million (CPM) values into the wilcox.test function in R (v4.0.2). Then, we set a p-value cutoff based on an FDR threshold using the Benjamini & Hochberg method.For the Wilcoxon rank-sum test, the filtering and normalization steps were the same as those for edgeR. For All six methods  took a read count matrix and a condition label vector as input. The parameters were set based on the user guides of these methods’ software packages.Specifically, DEGs were selected based on the corresponding FDR threshold for all the six methods .M to denote a gene-by-sample read count matrix (with genes as rows and samples as columns) and C to denote the vector of sample conditions labels (corresponding to the columns of M). Then, we generated a permuted dataset by randomly permuting all values in C and keeping the original order of samples in M. We repeated this permutation procedure for 1000 times to generate 1000 permuted datasets.From each original RNA-seq dataset, we generated permuted datasets between two conditions . We used true DEGs as the genes identified as DEGs by all six methods at a very small FDR threshold (0.0001%). We used X and Y to denote the read count matrices from the two conditions, and Xi and Yi to denote the read counts of gene i from the two conditions . Then, we generated semi-synthetic datasets X′ and Y′ in the following way: for each semi-synthetic datasets, we first randomly sampled k selected true DEGs from all true DEGs and preserved the read counts of these selected true DEGs; for each of the remaining genes, we randomly permuted its read counts between the two conditions. That is, i is a selected true DEG, i is not a selected true DEG, where σi is a random permutation of values in Xi and Yi. We repeated this procedure independently for 50 times to generate 50 semi-synthetic datasets. To generate down-sampled semi-synthetic datasets with a per-condition sample size of n, we randomly sampled n columns from X′ and Y′ each. The number of selected true DEGs k is 50% of the number of all true DEGs in most results. For results in first 4 rows of Additional file The semi-synthetic datasets were generated based on original RNA-seq samples from GTEx and TCGA. We first used all six DEG identification methods to identify DEGs from each original dataset containing two conditions. We then defined old 0.000%. We useold 0.000%. We useThe FDR is defined as the expectation of the false discovery proportion (FDP), the proportion of false positives among all the discoveries. The FDR cannot be directly observed, but the FDP can be calculated from benchmark datasets with known true positives and negatives. In our semi-synthetic data analysis, we defined true DEGs as true positives and the remaining genes as true negatives. First, we calculated the FDP of each DEG identification method  on each semi-synthetic dataset. Second, we calculated the method’s (approximate) FDR by taking the average of its FDPs on the 50 semi-synthetic datasets.The power of a DEG identification method is defined as the probability of identifying a gene as a DEG conditional on that the gene is a true DEG. It can also be considered as the expectation of the empirical power, which is the proportion of true DEGs being identified as DEGs. In our semi-synthetic datasets with true DEGs and true non-DEGs, we calculated the power of a method by taking the average of its empirical power on the 50 semi-synthetic datasets, similar to how we calculated the FDR.goodfit in the R package vcd as We used the goodness-of-fit test to evaluate how well a gene’s read counts under a condition can be fit by the negative binomial models estimated by DESeq2 and edgeR. To remove batch effects, we used the normalized read counts output by DESeq2 and edgeR. For each gene under each condition and each method (DESeq2 or edgeR), we conducted the goodness-of-fit test on the method’s normalized read counts, with the method’s estimated dispersion parameter of the negative binomial distribution. The goodness-of-fit test was implemented using the function summary, type = "nbinomial", par = list(size = 1/dispersion)))p-value. A smaller p-value indicates a poorer fit. Hence, we defined the poorness of fit as the negative log10 .which returns a Additional file 1. Figs. S1 to S30 and Table S1.Additional file 2. Summary of studies comparing DEG analysis methods.Additional file 3. Review history.