Liquid Chromatography Mass Spectrometry (LC-MS) is a powerful method for profiling complex biological samples. However, batch effects typically arise from differences in sample processing protocols, experimental conditions and data acquisition techniques, significantlyimpacting the interpretability of results. Correcting batch effects is crucial for the reproducibility of proteomics research, but current methods are not optimal for removal of batch effects without compressing the genuine biological variation under study. We propose a suite of Batch Effect Removal Neural Networks (BERNN) to remove batch effects in large LC-MS experiments, with the goal of maximizing sample classification performance between conditions. More importantly, these models must efficiently generalize in batches not seen during training. Comparison of batch effect correction methods across three diverse datasets demonstrated that BERNN models consistently showed the strongest sample classification performance. However, the model producing the greatest classification improvements did not always perform best in terms of batch effect removal. Finally, we show that overcorrection of batch effects resulted in the loss of some essential biological variability. These findings highlight the importance of balancing batch effect removal while preserving valuable biological diversity in large-scale LC-MS experiments. Despite the power of LC-MS, the utility of large-scale experiments remains compromised due to the omnipresence of confounding factors. Confounders can be divided into biological confounders, such as age or gender, and non-biological confounders, such as batch effects. The latter are practically unavoidable in large-scale studies due to limitations in instrumental availability and timeline of sample collection and ideally would be removed from the final biological quantification value 4. It can be difficult, even impossible, to completely remove batch effects without affecting the quality of the biological signal. By assessing the classification improvement of machine learning models, we can determine if the batch effect correction method successfully removes the technical variations and restores the underlying biological patterns 6. Furthermore, classification models enable personalized medicine by identifying patterns and biomarkers that can discriminate between different subjects. In highly heterogeneous disorders, such as Alzheimer’s Disease, it enables the discovery of biomarkers that are not common to every person affected by this disorder 7. Batch effects are a systematic variation that arises from experimental differences introduced unintentionally during data collection. It typically emerges from differences in sample processing protocols , experimental conditions , and data acquisition techniques  It is a critical problem in many high-throughput bioassays, such as microarrays 8, RNA-Seq9 or LC-MS-based proteomics and metabolomics 10, that involve processing samples in different batches or on different platforms, which can result in differences in data quality and analysis performance. Batch effects in proteomics occurs due to variability in sample preparation, instrument condition and performance, or environmental factors present during the sample preparation and analysis workflow. These technical variations may lead to false positive or false negative protein identifications, as well as inconsistencies in the quantification of protein abundances across batches, which can hamper the reproducibility and validity of the study 11.Liquid chromatography-mass spectrometry (LC-MS) has become an essential analytical technique in molecular biology because of its ability to accurately and simultaneously quantify thousands of compounds in biological samples. It is a powerful tool for large-scale screening of potential biomarkers, which enables the identification of specific measurable indicators that can aid in disease diagnosis 12), location-based methods  and matrix refactorization methods . These methods assume an accurate new representation can be obtained using a generalized linear model, which is not necessarily an accurate assumption. This issue can be addressed using Deep Neural Networks (DNN), which use a succession of nonlinear transformations 14. In the latter, batch information is incorporated into the DNN either as an input feature 16, as a regularization term in the objective function17 or vectorized and added to the encoded vector representing the input features, such as NormAE 14. However, because of their overparameterization, DNNs usually require large amounts of data to be generalizable. Given the high number of parameters to train, and the fact that there is a high number of possible architectures , DNNs can be resource-intensive and time-consuming to train.Common approaches to correct batch effect in LC-MS include Quality Control (QC) based methods , model-agnostic approaches (such as LIME25 and SHAP 26), all of which provide insight into how each feature is important for the model’s decision-making process. Not only do these methods can indicate the average importance of each feature for a complete classification task, but they can also provide interpretable classification explanations at the sample level 27.To determine whether a biological signal has been preserved after batch effect correction, one can utilize classification performance. The highest scores determine the effectiveness of the correction. There are several metrics available to evaluate the extent to which the batch effect has been removed, with the goal being to determine an optimal balance between preserving biological variation and removing the batch effect26 can be used for biomarker discovery. Finally, our method simultaneously corrects batch effects and performs sample classification, making the method an end-to-end solution which ensures that batch effect removal improves classification.In this context, we propose a suite of models to overcome these various batch correction problems, all based on autoencoders executable in parallel. Our approach to countering batch effects is different from most other solutions, as we do not rely on a single solution that we claim to be superior to all others. Instead, we acknowledge that not all problems require the same solution and propose multiple potential solutions to address batch effects. Thus, we aim to empower researchers to easily try multiple methods simultaneously and pick the optimal approach for their dataset and scientific question. Amongst this suite of models, we present the first use of Variational Autoencoders (VAE), Domain Adversarial Neural Networks (DANN) and Domain Inverse Triplet Loss (invTriplet) for batch correction in LC-MS. Additionally, in contrast to other batch correction methods, we do not recommend using the corrected output of the autoencoder for biomarker discovery through downstream analysis. Instead, we demonstrate how Shapley values 3.118, but we are using the Gradient Reversal Layer (GRL) to make the training more straightforward 28. To our knowledge, all methods in the literature using DNN to deal with batch effects use an autoencoder 18, but it is not mandatory . The label classifier is optional to get a representation free of batch effect, but is essential for model selection, as we define the objective of the model as to get the best classification scores in batches never seen during training. This objective ensures the maximum biological information is preserved and should therefore always be used to improve the reliability of downstream analysis. To ensure generalizability across batches, we always validate our results on batches that were not used during training.Our models, that we call Batch Effect Removal Neural Networks (BERNN), are composed of different modules, each with different objectives: the autoencoder, the batch classifier and the label classifier  see . With re30 does not provide the corrected features: it creates an integrated embedding. All our methods also create a new embedded representation, but can also return a vector of the corrected data, although the utility of these data is not guaranteed. The corrected data is obtained by reconstructing the original input from the new embedding using the decoder  as an example of how the features contributing the most to the decision of the models can be identified without the corrected original features.Most batch effect correction methods return the corrected expression data, but 3.231. Class imbalance was strongest in the Adenocarcinoma dataset, where 87.5% of the samples in the dataset were of the dominant class. As can be seen in Figure S3B, the best accuracy for this dataset is obtained using the raw data, because the model always predicts the dominant class. The main drawback of using MCC is its high sensitivity to misclassification of the samples in the minority class. When the imbalance is very high, misclassification of a single sample can have a big impact on the score. This can partly explain why the error bars for MCC were sometimes quite large. In comparison, error bars are smaller for accuracies .To get an accurate picture of the model performances benchmarked in this study, we used three datasets with different characteristics . They arTo evaluate the performance of batch effect correction, we used two main categories of metrics: batch mixing metrics and quality control (QC) metrics. To assess the batch mixing performance, we used Batch Entropy (BE), Adjusted Rand Index (ARI) and AMI see . QC metr3.3As explained in The first scenario was used only for the Alzheimer dataset, the two others used the second scenario. When the second scenario is used with the Alzheimer dataset, the overfitting problem is too important and only random predictions are made for the validation and test sets. However, because the labels classifier cannot influence the representation learned, it probably causes an underfitting of the data. It may be because of the data augmentation they provide that VAEs, particularly VAE-DANN, perform better on this dataset . On the 3.4We define the best model as the one with the best average classification performances over all repetitive holdout iterations see . BecauseMost batch correction methods improve (decrease) batch entropy (BE). AMI and ARI are also consistently improved  in datasets with the greatest initial batch effects, but not in the Alzheimer dataset, which had moderate batch effects in the raw dataset. For every dataset, the best validation MCC score always used a transformation that improved BE . AlthougIn both datasets that used replicate QC samples (Alzheimer and Adenocarcinoma), the networks with the best classification performances had better nMED than the uncorrected data, but the networks that did best on these metrics did not necessarily result in the best classification metrics. Note than in 3.5margin), which makes hyperparameter optimization more complex. It has been shown to have important consequences on optimization in scRNAseq where inverse triplet loss was also used to overcome batch effects 30. It is possible they might require more hyperparameters optimization to achieve even better performances. Nevertheless, one of the invTriplet models was either the top performer , which did not perform as well as the other models developed in this study, would perform best in other datasets . In all datasets, one of revTriplet or invTriplet was always part of the best of the BERNN models for batch correction according to the batch entropy . Combat or harmony were often the best according to the three batch mixing metrics, but always at the cost of very poor classification performances. The triplet losses require an additional hyperparameter . However, they either reduced the efficiency of the classification tasks or barely made any difference in performance. The classification was significantly reduced by some methods, such as harmony or combat (excluding pycombat), indicating that the biological signal crucial for distinguishing between conditions was completely eliminated.It is well known in machine learning that no single model can claim to be the best at solving any task. In this study, we propose a suite of deep learning architectures models to enable users to find the optimal solution for different problems. This suite introduces batch correction in LC-MS using VAE-based models, the use of GRL for implementing a DANN and triplet losses, all of which were part of the best model in at least one dataset.32, because it requires all losses to be minimized to function properly. In this study, some models require to simultaneously train multiple losses: the autoencoder reconstruction loss (1), the batch classification loss (2), the classification loss (3) and, for VAEs, the Kullback-Liebler loss (4). Each of these losses needs hyperparameters to lever their importance for the model to be optimal. Adversarial models, for example, are notoriously known to be difficult to train because of the fragile balance between the training of the discriminator and generator 33.The inverse triplet loss is particularly interesting because it is the only loss that is effectively minimized. The other losses that use GRL attempt to minimize the batch classification, but the loss increases until it reaches the loss corresponding to a random classification of the batches. This property is particularly useful in the context of multitask learning 26 or LIME 34 , that can identify the most useful features using methods.The bottleneck representation should be preferred to the reconstructed inputs, because the decoder cannot improve the representations. The reconstructions can only be as good as the bottleneck representations or more likely, worse. Reconstruction is usually chosen because it is easier to interpret for biologists, because the goal of the studies is to identify features that can be used as biomarkers. If the end goal is not the classification in itself, but to identify biomarkers that can be used to search for new therapeutical compounds, then it makes sense to focus on denoised representations. We argue, however, that the bottleneck should be used in combination with other methods, like SHAP 35, as 18 did not have its dataset available, but the dataset description suggest they are the same dataset).We found that the original NormAE available online had poor classification results, especially on the Alzheimer dataset (unpublished data). Instead of using the implemented version available online, it was implemented in our suite of models, as it is very similar to the DANN method, except without the GRL  samples were obtained according to standardized collection and processing protocols through the Mass General Institute for Neurodegenerative Disease (MIND) biorepository, following written informed consent for research biobanking (IRB: 2015P000221). This cohort represents a clinically complex cohort spanning a wide variety of neurological disorders, which closely aligns with a real-life diagnostic situation. The raw data for this dataset is available on ProteomeXchange with accession number PXD04321636 software (version 1.8.1) for protein identification and quantification. DIA-NN was used in two steps: 1) Library-free search on the GPF files using a Uniprot Reference Homo Sapiens database to generate a spectral library; 2) Library-based search on the sample and QC sample files using the spectral library generated in step one. The main report generated by DIA-NN was used with the DIA-NN R package 37 to get quantifications of proteins corresponding to unique genes. Common contaminants were removed from the dataset to avoid any bias in the classification or in any subsequent analysis. A complete list of removed protein IDs is available in the supplementary content (Table S1).CSF proteins were trypsin-digested prior to LC-MS/MS analysis on an Orbitrap Fusion Tribrid (Thermo Fischer Scientific) mass spectrometer operating in Data Independent Acquisition (DIA) mode. The 923 samples were injected in duplicate across 22 different batches. A QC sample was also generated by mixing a small aliquot of all CSF samples and analyzed in the same conditions at the beginning and at the end of each batch. This QC sample was also used to generate a Gas Phase Fractionation (GPF) library. The raw files were then processed with DIA-NN The dataset consists of a total of 923 samples, with 84 QC samples and 839 samples that were obtained from 408 subjects, with 22 subjects having more than one sample from repeat clinic visits. The samples were distributed into 22 batches with an average of approximately 41 samples per batch . The cohort was subdivided into 6 different disease classes. Cognitively unimpaired patients (CU), Alzheimer’s Disease with dementia (DEM-AD), Alzheimer’s Disease with mild cognitive impairment (MCI-AD), dementia with causes other than Alzheimer’s disease (DEM-other), mild cognitive impairment with causes other than Alzheimer’s disease (MCI-other) and patients with Normal Pressure Hydrocephalus (NPH). Importantly, CU patients are not healthy controls, but all had clinical indication for lumbar puncture, and span a variety of other non-dementia diagnoses5https://github.com/spell00/BERNN_MSMS.The batches are very heterogeneous with a similar total number of samples per batch, but different class composition across each batch . Classes are not fully balanced. The complete preprocessed dataset is available at 5.1.238, then preprocessed using the R package XCMS. After data processing, the final dataset has 6461 metabolite peaks. More details on this dataset in the original paper 35. The dataset is available at https://github.com/dengkuistat/WaveICA_2.0/tree/master/data .The dataset is composed of a total of 642 samples, with 74 QC samples and 568 patients, comprising 497 colorectal cancer and 71 chronic enteritis patients. There were 192, 192, and 184 subject samples with 25, 25, and 24 QCs in three batches, respectively. The raw MS file were converted to mzXML using ProteoWizard 5.1.3. The dataset is made of a total of 372 samples, of which 171 received a high fat diet and 201 has a chow diet. There were only 3 QC samples, all in the same batch, and thus these samples were discarded. The samples were distributed into 7 batches with an average of 53 samples per batch . Each sample has 17887 features that represent the peptides’ precursors. The raw data for the this AgingMice dataset is available with ProetomeXchange accession number PXD009160.This dataset was introduced by https://github.com/symbioticMe/batch_effects_workflow_code. Then, download the file http://ftp.pride.ebi.ac.uk/pride/data/archive/2021/11/PXD009160/E1801171630_feature_alignment_requant.tsv.gz and place it the folder AgingMice_study/data_AgingMice/1_original_data of the batch_effects_workflow_code repository. Then, run the scripts 1a_prepare_sample_annotation.R, 1b_prepare_raw_proteome.R and 4b_peptide_correlation_raw_data.R to get the matrix of log precursor values used in this study.To reconstruct the data matrix used in this study, first download to Github repository 5.2The tools we used to evaluate the presence of batch effect can be divided into 3 main categories: visual diagnostic using a dimensionality reduction technique, batch mixing metrics and quality control metrics. It is important to note that batch effects can be subtle and difficult to detect, and that different methods may identify different sources of variation in the data. Therefore, it is often recommended to use multiple methods and to carefully validate and interpret the results.5.2.140 or t-SNE 41. It is often how batch effect is first noticed. The presence of a visually observable batch effect means it is a high source of variance in the data. However, these visualizations can be incomplete, thus the absence of visually observable batch effect does not mean it is inexistant. For example, for the Alzheimer dataset, the batch effect is not as obvious as in other datasets. If only visual diagnostics is done, batch effects might go unnoticed.The first category for evaluating batch effect is visual diagnostic . This is5.2.2All batch mixing tests use a classifier trained to predict which batch each sample is from. We used a k-nearest neighbors’ classifier with 20 neighbors to calculate the probability of a sample belonging to each batch. The highest probabilities were used as predictions to calculate ARI and AMI. The probabilities are used to calculate the batch entropy.5.2.1.1We can say there is no batch effect when it is impossible to accurately predict from which batch a sample is drawn from. If the best prediction is random, there is no batch effect. To get the maximum batch entropy , so high batch mixing is represented by values close to 0 (the batch predictions of a KNN classifier are no better than a random prediction). We use the Adjusted Rand Index , so high batch mixing is represented by values close to 0 (the batch predictions of a KNN classifier are no better than a random prediction). The variables compared are the batch predictions and the batches’ true values.Where 5.2.318.Two of the datasets used in this study contain QC samples that were systematically analyzed with each batch of analyses. The features of that sample should always be the same, so we can calculate how much they diverge and use these metrics to measure the batch effect importance. These metrics were (to our knowledge) introduced by 5.2.1.4Because it is always exactly the same sample, we know that all QCs should theoretically be perfectly correlated, which means a perfect Pearson Correlation Coefficient (PCC) of 1. All samples are compared in pairs, so the final value is an average over all PCCs.5.2.1.518, the median is used because it is less affected by aberrant values. Unlike 18, we also normalize the value by dividing it by the median Euclidean distance of all non-QC samples. If a transformation makes the QC samples very close to each other, but non-QC samples are equally close to each other, then the transformation did not actually alleviate the batch effect. For this reason, nMED should be preferred to the average Euclidean distance proposed in 26 .If batch correction is efficient, all QC samples should be very close to each other. The Euclidean distance is used to measure how far each pair of samples are from one another. Instead of using the average, like in 5.3minmax_per_batch, standard_per_batch and robust_per_batch. The latter two have more potential to remove batch effect, transforming the values into z-scores, thus forcing each batch to have a mean of 0 and unit variance. We also used combat 8 and Harmony 13, as they are popular methods in microarrays/RNAseq and scRNAseq respectively. Combat is also used to remove batch effects from LC-MS datasets 42. We used two implementations of combat: a R version https://rdrr.io/bioc/sva/man/ComBat.html and a python version https://github.com/epigenelabs/pyComBat, which we named pycombat in this manuscript. Intriguingly, the two implementations had very different outcomes.We tried normalizing each dataset using three different methods: minmax, standard and robust standardization. The first methods to counter batch effects that we tried was applying the same three normalization methods, but to counter batch effects, they were applied individually to each batch individually, which we named 25 and NormAE 18, as they are state-of-the-art methods is batch effect correction of LC-MS data. We used our own implementation of NormAE, which has a slightly different architecture. We reduced the number of layers to a single hidden before and after the bottleneck to make the comparison with our own models. Unlike NormAE, we consider the number of neurons in each layer to be hyperparameters that are optimized. To give it a fair chance to outperform our own methods, we also use all the same hyperparameter optimization as our models.We also used WaveICA 5.4All the models in BERNN are implemented using PyTorch and are based on autoencoders. In short, autoencoders are composed of an encoder and a decoder. The encoder turns the inputs into embeddings , which are usually smaller than the inputs, but not necessarily. The objective of the autoencoder is to obtain new representations and reconstruct the original inputs from it the best it can. The embeddings should then contain as much information as possible from the inputs, without the unnecessary noise. To find the best performing model on a given dataset, a total of 10 models can be trained using BERNN. For a complete representation of all the possible models that can be trained using BERNN, see Fig S1. All the models were not represented in the main results to alleviate the reading, but the complete results are available in Figures S4-S6.5.4.118. It makes it possible to obtain better reconstruction loss by adding the batch information into the vector for the reconstruction. Because the batch ID is contained in this vector added to the embedding, the latter does not need to contain information about the batch. A similar method is used to get batch-free representations in scRNAseq, such as in 14. In this case, the batch ID is directly appended to the bottleneck representation of the variational autoencoder.The first method that is implemented to obtain batch-free representations is to add to the embedding of the autoencoder a vector of the same size representing the batch ID . This was implemented in NormAE, although not mentioned in the paper 5.4 228. In this case, we are defining batches to be from different domains. The original DANN was developed to adapt the learning from a single domain to another one, so our work is more akin to 43, which extends domain adaptation for multiple domains. Our AE-DANN model is represented in Fig S1. The loss of AE-DANN is the following:Domain Adaptation Neural Network (DANN) is a type of deep learning algorithm that enables a model trained on one domain to be adapted to another related domain with different characteristics, allowing it to perform better on the target domain. DANN achieves this by learning to extract domain-invariant features from the input data The NormAE is also similar in nature to a DANN, except it does not use the GRL. Using the GRL is advantageous, because the total loss is composed of losses that are all minimized and added together. When not using the GRL, like with NormAE, the adversarial loss is maximized, and it is subtracted to the other losses being optimized simultaneously.Using this definition, if the second term of the equation becomes too large, the loss could become negative, which should not be allowed to happen.5.4.3The inverse triplet loss is like the normal Triplet Loss , but the positive and negative samples are inversed; the negative samples take the place of the positive samples in the triplet loss equation, and vice-versa.Positive and Negative samples in the equation, which encourages batch-free representations. The samples from different batches get closer, while samples from the same batch are pushed further apart. The later objective is used to prevent all samples from collapsing. If the samples from the same batch are not pushed apart, the loss would be optimal if all samples were transformed into the exact same value, which is not the desired outcome. The distance minimized in this case is the Euclidean distance, but any distance could be used.5.4.444:The variational autoencoder (VAE) is a probabilistic generative model based on the variational Bayes approach. To train a VAE, we want to optimize the lower bound defined in Eq. 3 of Where 5.55.5.1Repetitive holdout is a method to evaluate the performance of a model on a dataset. It is similar to cross-validation, however each split is random, there is no limit to the number of times it can be done on a dataset and the test set is resampled for every holdout iteration. Resampling the test set is particularly important, because some batches can be much easier to classify than others, which can make the test set classification much better or much worse than the validation set. Using repetitive holdout makes classification on the valid and test sets comparable.When splitting the dataset, the samples from a given batch must all be contained in the same split. We do this in order to inform on the generalization abilities of a model to make predictions in a new batch.5.5.2WeightedRandomSampler is used to counter class imbalances in datasets by giving more weights to samples from minority classes during training.Class imbalance has a negative impact on machine learning models predictive abilities. If nothing is done about it, the model might learn to only predict the majority class. This is especially true if the imbalance is very large. It is also a concern if the problem is very hard to model. PyTorch’s 5.5.3optimize from the package ax-platform (https://pypi.org/project/ax-platform/) to perform a Bayesian optimization of the hyperparameters for each of the BERNN models . We used 20 combinations of hyperparameters to optimize each model. The hyperparameters optimized are the following: learning rate, weight decay, dropout, number of warmup epochs, layer1, layer2, label smoothing parameter, triplet loss margin (when it applies), beta , gamma . The batch size is set to 32 and the number of epochs after warmup is set to 1000, but the training is stopped if no improvements are made in the last 100 epochs. Models were trained on Nvidia RTX3090 GPUs.We used the function 5.5.4scikit-learn. For both, we used models from the python package scikit-learn. To deal with classes imbalance in some datasets, the parameter class_weights was set to “balanced” for both models. Automatically adjust weights inversely proportional to class frequencies. The hyperparameters optimized for the RFC were min_samples_split, min_samples_leaf, n_estimators, criterion, and oob_score. For the LinearSVM, we optimized tol, max_iter, penalty and C. The hyperparameters for the RFC and LinearSVM models were optimized using the package scikit-optimize (https://scikit-optimize.github.io/stable/) to execute a Bayesian Optimization.For the classification of raw data, or any batch corrections that do not involve a DNN, we used either a Random Forest Classifier (RFC) or a Linear Support Vector Machine (LinearSVM) from the python package 5.5.526 to produce an example analysis. The advantage of that approach is that it makes it possible to explain the decision made on individual samples and could be used for precision medicine. It would allow identification of complex patterns that apply only to a subset of the samples that could not be identified by differential analysis.For model explanation purposes and to identify the most important features for the classification, we used SHAP


Our findings indicated that all approaches led to similar taxonomic profiles (with P > 0.05 in PERMNAOVA and P <0.001 in the Mantel test), although the number of ASVs/OTUs and the alpha-diversity indices varied considerably. Despite considerable differences in disease-related markers identified, disease-related analysis showed that all methods could result in similar conclusions. Fusobacterium, Streptococcus, Peptostreptococcus, Parvimonas, Gemella, and Haemophilus were identified by all three methods as enriched in the CRC group, while Roseburia, Faecalibacterium, Butyricicoccus, and Blautia were identified by all three methods as enriched in the healthy group. In addition, disease-diagnostic models generated using machine learning algorithms based on the data from these different methods all achieved good diagnostic efficiency (AUC: 0.87–0.89), with the model based on DADA2 producing the highest AUC . However, there was no significant difference in performance between the models (P >0.05). In conclusion, this study demonstrates that DADA2, Deblur, and de novo OTU clustering display similar power levels in taxa assignment and can produce similar conclusions in the case of the CRC cohort.16S rRNA is the universal gene of microbes, and it is often used as a target gene to obtain profiles of microbial communities via next-generation sequencing (NGS) technology. Traditionally, sequences are clustered into operational taxonomic units (OTUs) at a 97% threshold based on the taxonomic standard using 16S rRNA, and methods for the reduction of sequencing errors are bypassed, which may lead to false classification units. Several denoising algorithms have been published to solve this problem, such as DADA2 and Deblur, which can correct sequencing errors at single-nucleotide resolution by generating amplicon sequence variants (ASVs). As high-resolution ASVs are becoming more popular than OTUs and only one analysis method is usually selected in a particular study, there is a need for a thorough comparison of OTU clustering and denoising pipelines. In this study, three of the most widely used 16S rRNA methods (two denoising algorithms, DADA2 and Deblur, along with  Colorectal cancer (CRC) is the third most common cancer globally, causing more than one million deaths annually  underwent two sequencing runs, as a larger fraction of ASVs could be recalled in the second sequencing run  technology  via the QIIME2 VSEARCH plugin with default settings. For DADA2-based methods, reads were truncated to lengths of 290 bp and 220 bp for the forward and reverse reads, respectively, to remove low-quality bases at the end of the reads, and the DADA2 plugin was run with default settings to construct the ASV feature table. For Deblur-based methods, the joined reads from VSEARCH were input into the Deblur plugin to construct the ASV feature table with default settings, and singletons were filtered. The OTUs and ASVs were then compared against the Silva Database    , and linear discriminant analysis (LDA score > 2) was used to assess the effect size associated with each feature.Typically, 16S rRNA data are examined at the genus level in further analysis, so genus profiles were entered into the following analyses. Bacterial taxonomic analyses were carried out, and comparisons between the three methods were conducted using the Wilcoxon rank sum test Bauer, . Linear In order to differentiate CRC samples from healthy samples, a random forest (RF) model  was conducted to analyze the variance of the data generated using different methods, and the Mantel test was used to analyze the associations between these data. Spearman correlation analysis was performed to analyze the correlations between microbiota features. Plots were constructed using the ggplot2 package (v3.3.3) in R.The Mann–Whitney The number of OTUs/ASVs obtained using the three methods varied considerably, with DADA2 obtaining the most variants, followed by Deblur and OTU clustering . At the Pfdr < 0.05) in abundance among the three methods (Pfdr < 0.05) , and only 71 genera showed significant differences  , S4. The < 0.05) . Finally < 0.05) , indicatPfdr < 0.05)  .P > 0.05 for PERMANOVA   . These rWe analyzed the differences between cases and controls to evaluate whether the different algorithms produced different disease-related outcomes. First, we found that although the Shannon index was lower for CRC patient samples than for healthy samples across all three methods, only DADA2 identified a significant difference between the groups . In contP-values of PERMANOVA were also significant, indicating that the different methods could produce the same conclusion.According to the results of PCoA analysis based on Bray–Curtis dissimilarity –D, the sFusobacterium, Gemella, Peptostreptococcus, and Streptococcus, which have been reported on widely in CRC research , followed by DADA2 (40 markers) and Deblur (39 markers) –C. A totP < 0.05)  . These rde novo OTU clustering, the identity standard was 97%, without no reliance on any database, and the number of OTUs was relatively low. The results under the DADA2 and Deblur methods indicated higher alpha diversity  to analyze data from a clinical cohort. First, we found that DADA2 and Deblur could identify more ASVs than OTUs in the community . The reaiversity , B. Compiversity , which miversity .de novo clustering algorithm is Uparse , we compared them with the reference database containing known taxa in order to obtain taxonomic information using the classify-sklearn algorithm, which is a form of naive Bayes classifier (a classification method based on machine learning)   could be identified using all three methods . Among t < 0.05) . OTU clu < 0.05) , possibl < 0.05) . TraditiP>0.05)   . The mic< 0.001) , confirmP < 0.05 in PERMANOVA  were found under all three methods to be enriched in the CRC group. Fusobacterium, especially Fusobacterium nucleatum, is an important marker of CRC  were obtained using the three methods, of which 37 markers (75.5%) could be reproduced using a different method; the enrichment trend was consistent for most markers –C. For eRoseburia, Faecalibacterium, [Eubacterium]_eligens_group, Bilophila, Phascolarctobacterium, Butyricicoccus, Blautia, and other genera  were found to be enriched in the healthy group. Butyricicoccus and Faecalibacterium are butyric acid producers, which play an important role in intestinal and host health and act as protectors against CRC  and plays various important roles in maintaining human health, such as enhancing gastrointestinal function, reducing inflammation levels, and influencing metabolic state and mood of the host  and to be found in greater abundance in CD  (P > 0.05)  models were constructed to evaluate diagnostic efficiency based on each of the three methods. For the training set, the highest AUC was obtained for the DADA2 model, at 89.4%, followed by 88.9% for the OTU clustering model and 86.8% for the Deblur model; the only significant difference in AUC was between DADA2 and Deblur ( < 0.05) , C, E. T > 0.05) , D, F. TIn conclusion, although there were differences in the number of OTUs/ASVs obtained using the three methods, the differences in the numbers of taxa were smaller, especially for the comparison between DADA2 and OTU clustering at the genus level. Moreover, the microbial profiles were strongly correlated. This indicates that the results obtained using the three methods are comparable. Case–control analysis also showed that the three methods could yield similar results, with mostly consistent identification of CRC-related markers. However, it should also be noted that the three methods were performed with the default parameters, and adjusting some of the parameters in the selected method could help users to obtain their desired results.The original contributions presented in the study are included in the article/JW and GL designed the research study. GL and TL performed bioinformatics analysis and wrote the manuscript. XZha and XZhu edited the manuscript. All authors read and approved the final manuscript.


The respiratory system’s complex cellular heterogeneity presents unique challenges to researchers in this field. Although bulk RNA sequencing and single-cell RNA sequencing (scRNA-seq) have provided insights into cell types and heterogeneity in the respiratory system, the relevant specific spatial localization and cellular interactions have not been clearly elucidated. Spatial transcriptomics (ST) has filled this gap and has been widely used in respiratory studies. This review focuses on the latest iterative technology of ST in recent years, summarizing how ST can be applied to the physiological and pathological processes of the respiratory system, with emphasis on the lungs. Finally, the current challenges and potential development directions are proposed, including high-throughput full-length transcriptome, integration of multi-omics, temporal and spatial omics, bioinformatics analysis, etc. These viewpoints are expected to advance the study of systematic mechanisms, including respiratory studies. The respiratory system comprises the respiratory tract and lungs and is one of the organs that directly interface with the external environment . The comTranscriptomics is a significant advance in combining high-throughput sequencing and bioinformatics to explore biological mechanisms . HoweverSince it was proposed in 2016, spatial transcriptomics (ST) has provided a new perspective to decipher physiological and pathological bases . While mThis review focuses on the latest iterative technology of ST in recent years and summarizes how these techniques can be applied to the physiological and pathological processes of the respiratory system, such as lung development, lung atlas, lung cancer, and lung injury. Finally, the current challenges and potential development directions are proposed, including high-throughput full-length transcriptome, multi-omics and spatiotemporal omics integration, bioinformatics analysis, etc. By presenting a unique combination of comprehensive disease coverage, in-depth exploration of disease mechanisms, emphasis on spatial heterogeneity, and future directions, this review can provide a distinct and valuable contribution to the field of ST application in respiratory diseases.Spatial transcriptome methods have developed and emerged rapidly and can be categorized into imaging-based and sequencing-based methods according to detection strategies. Previous reviews have elaborated on the principles and classification of ST , 16, 17.Image-based ST technologies include fluorescence in situ hybridization (ISH)- and in situ sequencing (ISS)-based methods. The advent of high-resolution microscopy and single-molecule fluorescence in situ hybridization (smFISH) has made it possible to quantify the subcellular resolution of transcripts in situ . A uniquISS-based methods are categorized into targeted and untargeted mRNA detection. One of the earliest targeted in situ sequencing techniques, ISS Fig. e, generaOn the other hand, spatially-resolved transcript amplicon readout mapping (STARmap) bypasses cDNA synthesis and uses SNAIL probes and sequencing with error-reduction by dynamic annealing and ligation (SEDAL) to identify gene identifiers. By combining hydrogel histochemistry, it analyses tissue samples in 3D tissues rather than a single 2D pattern . This yeFluorescent in situ sequencing (FISSEQ) is a representative method in non-targeted sequencing that can achieve unbiased coverage of the whole transcriptome. However, its random primers lead to low detection efficiency and involve complex enzymatic reactions  Fig. 1gg1g. ReleCombining the next-generation sequencing (NGS) platform and spatial information significantly improves the throughput of ST and unbiased retrieval of transcripts compared to image-based methods. A method based on laser capture microdissection (LCM) and scRNA-seq was developed to allow a spatially unbiased analysis of the transcriptome and classification and sequencing of regions of interest (ROI) under microscopic guidance Fig. a. In addGiven the low throughput and capture rate issues of imaging and microdissection technologies, researchers are gradually considering the in situ spatial indexing methods. ST was developed, mRNA location information and expression levels were mapped using spatial barcode and unique molecular identifier (UMI), which was acquired by 10 × Genomics (100 μm), and its capture efficiency  was further improved . Slide-sMicrofluidic channel-based approaches have also been integrated for spatial localization . DetermiST has been widely used in the study of lung development and respiratory disease mechanisms due to the great interest in the molecular structure of the respiratory system. This section summarizes the emerging applications of the respiratory system, such as lung development, lung atlas, lung cancer, and lung injury Fig.  63–69].–69.63–69Ljungberg et al.  using spThe Human Cell Atlas (HCA) aims to establish atlas data of different organs and tissues in healthy individuals at single-cell resolution , 72, incIn the past few centuries, tumors have been considered a highly organized “organ” instead of a simple aggregation of abnormally proliferating cells . Lung caCD27, CCL5, and ITGAM in tumors with high-level TAM infiltration might be potential targets for immunotherapy. Furthermore, applying multiplex immunohistochemistry (mIHC) and digital spatial profiler (DSP) has been observed for TME analysis in samples of NSCLC patients following immune checkpoint inhibitor (ICI) treatment [Immune checkpoint (ICP)-targeted therapy has shown considerable success in lung cancer, including non-small cell lung cancer (NSCLC), lung adenocarcinoma (LUAD), squamous and non-squamous carcinoma. However, studies have shown that the rate of positive responses in patients receiving targeted drugs remains low, with a possibly high degree of immune-related adverse events . In addireatment .Many studies based on scRNA-seq have revealed the heterogeneity of lung cancer cells –97. With+ cancer cell subpopulation, in which UBE2C mediates the proliferation and metastasis of tumor cells. As LUAD progresses from adenocarcinoma in situ (AIS) to invasive adenocarcinoma (IAC), the spatial distribution of cancer cells may be more important than their type. This finding compensates for the absence of spatial information for immunotyping LUAD by scRNA-seq. Furthermore, studies have reported the mechanisms of brain metastasis (BrMs) in NSCLC based on DSP. These findings highlight the highly immunosuppressive microenvironment associated with BrMs lesions, as compared to primary tumors, characterized by the reduced abundance of B and T cells and increased infiltration of neutrophils, providing a framework for the spatial heterogeneity of BMS in lung cancer and identifying the characteristic genes of metastasis to predict patients’ prognosis [Cancer development involves tumor cells’ adaptation to the environment and is the inevitable and continuous result of life . To daterognosis .CTNNB, HIF1A, and VEGFA genes. In addition, several signaling pathways related to tumorigenesis also positively correlated with bacterial content in lung cancer. We believe that spatial transcriptome and local interaction group analysis can predict individual tumor behavior and provide useful resources for understanding and reversing lung cancer progression.Lung cancers have a unique microbial composition . SeveralPulmonary fibrosis is a highly heterogeneous end-stage pathological change of the lung, whose pathogenesis has not been fully elucidated , 108. ThThe corona virus disease 2019 (COVID-19) pandemic, induced by severe acute respiratory syndrome coronavirus 2 (SARS-CoV2), has caused millions of cases of severe acute respiratory illness worldwide , 112, thIn situ sequencing and capture techniques have been applied to the mechanistic study and therapeutic target discovery of tuberculosis –127 and Although ST and related frontier bioinformatics analysis have entirely changed the research on complex organs and tissues and brought great hope for developing systems and disease mechanisms, several challenges should still be addressed to develop its potential beyond these Fig. .Fig. 4ThThe stability and availability of pre-sequencing samples might be a major obstacle limiting ST. Any intact tissue containing mRNA is suitable for the spatial transcriptome ; howeverA growing body of evidence suggests that ST can potentially aid in clinical diagnosis, understanding human disease, and making proper decisions concerning medicines . FormaliThe ST has solved the problem of losing spatial location information by cell dissociation in the scRNA-seq method. However, apart from spatial addressing, the transcription coverage and depth of these methods are still in their early development stages. Most ST methods can only retrieve single-ended transcripts rather than full-length ones . The resCurrently, most advanced space technologies are limited to displaying cell organization and gene expression on a 2D pattern rather than an actual 3D spatial atlas, which cannot summarize the highly complex spatial cell environment. Tomo-seq performs RNA-seq on different tissue slices to obtain spatial information . SimilarThe current ST methods cannot be applied to in vivo histological research on living cells because they investigate only cell snapshots at a certain time in fixed samples. The observed gene expression profile may only be the product of expression heterogeneity . These tST is progressing rapidly towards multi-omics data and achieving single-cell resolution. This advancement enables the retrieval of comprehensive information encompassing splicing variation, genetic and epigenetic changes, proteomics, and time-dimension data, all within a single experimental setup . This wiResearchers have significantly advanced in developing of various single-cell multi-omics technologies, as exemplified in Table On the other hand, gene expression in biological systems is highly dynamic , and anaIncreased spatial dimensionality, data volume, and complexity present formidable challenges during the data analysis phase. Numerous bioinformatics tools employed for scRNA-seq analysis can be readily adapted for spatial analysis, encompassing deconvolution, clustering, cell type annotation , 196, anWith the exception of commercial platforms like 10 × Visium, Stereo-seq, and GeoMx DSP, the majority of the aforementioned ST techniques are generally confined to laboratory settings. This limitation is expected, given the recent publication of these methods and the high costs associated with translating experiments into diagnostic applications. Standardizing experimental procedures and data analysis pipelines is anticipated to facilitate the commercialization and widespread accessibility of spatial omics analysis techniques. Moreover, ongoing efforts to integrate automated sample processing, 3D structure, deep section scanning, and time series data will further advance this field by revealing new cell structures and expanding our understanding of biological processes. Given that the future progress of ST necessitates the convergence of multiple disciplines, close collaboration is imperative between researchers in bioinformatics analytics, automation devices, clinical translational research, and biomedical fields .This review presented cutting-edge technologies in ST and their applications to organ/tissue physiological and pathological processes. As a newer iteration of scRNA-seq, the field of ST is expanding rapidly, significantly improving our understanding of developmental biology and pathogenesis and transforming our ability to diagnose, understand, and treat diseases. With this growing panoply and collaborative efforts of bioinformatics, engineering, and SM-omics, we expect to obtain high-throughput molecular information concerning full-length transcriptomes when the cost significantly decreases. These data will provide a more comprehensive view for clarifying interactions between cell biological mechanisms within tissue ecosystems.


Late-stage drug development failures are usually a consequence of ineffective targets. Thus, proper target identification is needed, which may be possible using computational approaches. The reason being, effective targets have disease-relevant biological functions, and omics data unveil the proteins involved in these functions. Also, properties that favor the existence of binding between drug and target are deducible from the protein’s amino acid sequence. In this work, we developed OncoRTT, a deep learning (DL)-based method for predicting novel therapeutic targets. OncoRTT is designed to reduce suboptimal target selection by identifying novel targets based on features of known effective targets using DL approaches. First, we created the “OncologyTT” datasets, which include genes/proteins associated with ten prevalent cancer types. Then, we generated three sets of features for all genes: omics features, the proteins’ amino-acid sequence BERT embeddings, and the integrated features to train and test the DL classifiers separately. The models achieved high prediction performances in terms of area under the curve (AUC), i.e., AUC greater than 0.88 for all cancer types, with a maximum of 0.95 for leukemia. Also, OncoRTT outperformed the state-of-the-art method using their data in five out of seven cancer types commonly assessed by both methods. Furthermore, OncoRTT predicts novel therapeutic targets using new test data related to the seven cancer types. We further corroborated these results with other validation evidence using the Open Targets Platform and a case study focused on the top-10 predicted therapeutic targets for lung cancer. The development of novel anticancer drugs is associated with high costs, poor safety profiles, and is a time-consuming process with significant failure rates . Thus, sIn this regard, Nidhi and colleagues  were amoin silico target identification. They developed an ML approach that prioritizes proteins based on similarity to targets of an approved drug. This was done by computing an extensive collection of protein features that the learning method scores based on the features’ ability to discriminate the approved target proteins from others. The model achieved high prediction performance (Area Under the Curve (AUC) of 0.89) based on an independent set of clinical trial targets. In the second approach, Bazaga and colleagues . Thus, our method, OncoRTT, is the first attempt to use DL-based models whose primary goal is to systematically predict potential cancer-specific therapeutic targets . The maiThis work focuses on ten globally prevalent cancer types based on the cancer burden  in 2020 . The canTo create our data, the Oncology Therapeutic Targets (OncologyTT) dataset, several steps were applied using multiple data sources, as shown in https://www.cancer.gov/about-cancer/treatment/drugs/cancer-type by July 2021. Second, for each anticancer drug, we obtained all synonyms and drug bank IDs from the DrugBank database  from the national cancer institute (NCI) for each of the ten cancer types database . Third, database . Since tdatabase  from then cancer . It alson cancer  portal fn cancer . The detn cancer , we refiwww.humanmine.org, a segment of the InterMine project  from this pool for each cancer type (without any overlap of the protein-coding genes between the cancer types), equal to the number of positive samples. That is, even if the negative set is chosen randomly, there may be some biases in the dataset, therefore we created random negative datasets per cancer type to ensure the potential biases do not affect all the tests. It is worth mentioning that we have generated the same number of negative samples as the positive samples to obtain a balanced dataset, which is important in the ML classification problem to give equal priority to each class and avoid poor predictive performance for the minority class or biased classification.We additionally generated a negative dataset  for our classification model. First, we obtained a pool of human genes by retrieving the gene name and protein name for every gene entry in HumanMine  project , that seAll the steps mentioned above allowed us to obtain the final dataset, “OncologyTT”, which includes the positive gene dataset  and the negative gene dataset  summarized in For all positive and negative samples  in our datasets, the amino-acid sequences were downloaded in August 2021 from the Uniprot database  using thCancer-Target datasets on 20 September 2021, from the GitHub link: https://github.com/storm-therapeutics/CancerTargetPrediction/tree/master/analyses_data/training_sets_genes. This dataset consists of target and non-target genes for nine common cancers. However, we excluded two cancer types not included in our study. We downloaded the It is worth mentioning, we considered all random negative samples initially generated from the pool of human genes as unlabeled data samples and then predicted the novel therapeutic targets. This justification is based on a study  that shoX = {x1, x2, …, xn} where n is the number of all data samples. Since our problem is supervised learning, we also provided all data samples with their class labels Y = {y1, y2, …, yn} by specifying if the cancer gene is a target  or if it is non-target  such as:This study describes the goal of identifying the therapeutic targets as a binary classification problem. As mentioned in the previous section, we generated all data samples  in our datasets that can be represented as vector We followed the same methodology for all cancer types. For each data sample (gene), we extracted different features from multiple resources, as explained later. The classification model aims to find the hidden patterns and associations between genes and their labels based on the feature vector (FV) and then predict the class labels .1. Generating data samples (consisting of target genes and non-target genes),2. Extracting and integrating features from amino-acid sequences by applying BERT-based embeddings and from omics features,3. Building several classifiers for target prediction,4. Retraining the best-performing DL model using the whole dataset,5. Utilizing new independent test data to predict novel therapeutic targets,6. Validating the novel therapeutic target using multiple sources.We provide a detailed explanation of each step below.The amino acid sequences are the fundamental building blocks of proteins, and the amino acids’ chemical and chemo-physical properties define the protein’s biological activity, specifically, the drugs that bind to it. Thus, a major Bioinformatics objective is to analyze amino-acid sequences of proteins and infer different structural and functional information . TherefoAdditionally, single and multi-omics data have been widely used to predict potential target proteins . Omics tBidirectional Encoder Representations from Transformer (BERT) is a well-known DL-based embedding method that has many benefits over conventional sequential models such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU). It has promising achievements in several natural language processing (NLP) tasks by extracting data patterns using unsupervised learning from massive unlabeled data. BERT-based models have been used in the biomedical domain . FurtherIn this work, we used one of the ProtTrans models called ProtBert-BFD . ProtBerThe ProtBert-BFD model that we used consists of several layers: 30, hidden layers size: 1,024, hidden layers intermediate size: 4096, number of neurons: 128, masking probability: 15%, optimizer: lamb , learninWe directly used the pre-trained ProtBert-BFD model in a transfer-learning fashion that can be used for different downstream ML tasks, predicting targets/non-targets in our case. We applied the ProtBert-BFD model on our dataset genes’ amino-acid sequences per cancer type to embed vector representations per sequence. Therefore, we automatically extracted the information learned by the protein LMs through embeddings  with a size equal to n*1,024, where n is the number of genes in each cancer type, and 1,024 is the dimension of the embeddings. We used these embeddings integrated with other features as input to the DL classifiers.For the omics data, we used TCGAbiolinks , a R/BioAfter we obtained the expression and mutation data for all the genes, we extracted features for each gene across several patient samples for matching cancer types. Since our data samples are the genes, not the patients, we aggregated gene expression values for each gene by finding the maximum, average, median, and minimum expression values over all patient samples for corresponding cancer. Therefore, each gene is represented by four features representative of gene expression level across several patient samples, which also capture whether or not the gene is always highly expressed or not. Furthermore, we calculated the gene mutation feature by counting how many times each gene is mutated across all patient samples used per cancer type. Finally, all features are combined and then normalized using min-max normalization.After completing the feature extraction process and obtaining a feature vector (FV) for all genes per cancer, we fed the three sets of FV into the classifiers, which include two ensemble ML classifiers, one classical ML classifier, and one DL classifier, to predict target/non-target genes, as illustrated in This section introduces the evaluation metrics we used to measure the accuracy of our prediction method and the experimental settings. For the experimental setting we implemented to evaluate the OncoRTT methods’ prediction performance and robustness, we independently applied stratified 10-fold cross-validation (CV) on each cancer dataset. Therefore, the data were randomly split into ten subsets in a stratified way where each subset must include the same percentage of the target and non-target genes . Then, we held one subset for testing and used the remaining nine subsets to train the model. This process was repeated ten times to have each subset of the data in the test data. Finally, we averaged the AUC that is calculated for each fold for all ten folds.Furthermore, another evaluation setting used by , was alsWe ran all experiments on a Linux Ubuntu 18.04.5 LTS Intel Xeon Platinum 8176 workstation, 64-bit OS, with 112 processors and two GPUs: Quadro and Titan, with CUDA version 11.0. For implementation, we mainly used Python version 3.8, and we used R version 4.1.1 for parts related to omics features preprocessing, and gene expression and mutation features.We systematically evaluated the OncoRTT method’s performance using the datasets we created. Next, we compared the performance of our method with the state-of-the-art method. Finally, we predicted the novel target genes using unseen independent test data.We aim to integrate the omics features extracted from the expression and mutations associated with each gene with the BERT embedding features automatically generated from the amino-acid sequences and use them for training, testing, and evaluating our method. However, to show the effectiveness of the feature integration process, we also obtained results using the omics features as stand-alone and the BERT embedding feature as stand-alone. Thus, we separately trained and tested our proposed DNN using three distinct sets of FVs: OMICS FV, BERT-Embeddings FV, and Integrated FV for each cancer type separately. In addition, we quantitatively evaluated three versions of OncoRTT models in terms of AUC calculated as the average performance of models on the test set during the 10-fold CV. The best results were obtained when we used the leukemia data (AUC = 0.95) for the integration model, and the results obtained for the other two models (Embeddings-based FV and OMICS-based FV) were also among the highest (88% and 77%) compared to the other cancers for each model. The reason may be the leukemia dataset is larger than the other datasets collected, and DL models generally perform better with more extensive data.Using the OMICS FVs alone achieved the worst performances for all the cancer types, which is expected as only a small number of features (only five features) were included. However, combining the OMICS FV with the BERT-Embeddings FVs across all cancer types significantly improved the OncoRTT prediction performances. That is, the result shows the contribution of the BERT-Embeddings FVs translates into a range of gain of about 13%  up to 29%  and 30% (colon) in the different cancer types. This result shows that the BERT-Embeddings FVs contributed substantially to the overall prediction achieved using the Integrated FVs.On the other hand, using the BERT-Embeddings FVs alone achieved much higher performances for all the cancer types, but the performances were still lower than the prediction performances achieved when using the Integrated FVs. Thus, despite this low (AUC <0.70) or acceptable (AUC >0.75) performance in some cases, when we only used the OMICS FVs, when we combined the BERT-Embeddings FVs with the OMICS FVs across all cancer types, the result shows the contribution of the OMICS FV that translates into a range of gain of about 1% (which is a very slight increase as is the case for bladder cancer) up to 7%  and 8% (lung cancer) for the different cancer types. The OMICS FVs enhanced the prediction performance because therapeutic targets are often the mutated or overexpressed genes underlying the cancers’ progression . OverallEven though the Integrated FV produced the best-performing models, the OncoRTT method achieved promising results for all cancer types when we fed only the BERT-Embeddings FVs to the DNN classifier (0.92 > AUC >=0.82). These results demonstrate the effectiveness of DL in two aspects: 1) The feature extracted by the ProtBert-BFD DL-based model revealed that the LM-feature representation  from unlabeled and complex biomedical data represented in the protein sequences captured critical biophysical and biological features of the protein. 2) The prediction, where the DL method can identify the hidden pattern from the embeddings and better distinguish the target genes from non-target genes by giving the features different weights based on their importance and using them in the prediction process. However, it is not an easy task to interpret these auto-generated features .Using the different features for several cancer types highlighted a few key aspects worth mentioning.versus several versions of our DL model trained using the shuffled datasets. Therefore, for each cancer type separately, we first trained the DL model using the original data  and obtained the results using the test data. After that, for 100 iterations, we fixed the FVs but scrambled the labels, trained the model over the new features-labels pairs, and acquired the new performances. The evaluations have been done using test data in terms of the R-squared (R2) evaluation metric, which is commonly used to measure the goodness of fit  < 0.05 for each cancer type as shown in p-values >= 0.05. Moreover, OncoRTT DL models obtained good R2 results for all cancer types compared to the 100 DL models trained using the shuffled datasets, obtaining low (<=0.50) or negative R2 results. Negative R2 implies that the model does not observe the data trend leading to a worse fit than the horizontal line , which means poor prediction performance as expected for models trained using shuffled data. Getting very low p-values provides evidence of a dependency between the features and the labels, and our DL model unveiled these correlations and patterns.Finally, to show the robustness of our DL models and verify that the high performance of our method in all cancer types was not random, we implemented the Y-Randomization test  . The Y-rs of fit . ConsequTo illustrate the OncoRTT method’s effectiveness, we compared it with the previous work , which iUsing this dataset , we repe• In our study, we utilized the state-of-the-art method that generates protein sequence embeddings called ProtTrans BERT-based embedding, but in the baseline method, they utilized PPI embeddings. ProtTrans trained two autoregressive models (Transformer-X and XLNet) and four auto-encoder models  using more than two thousand million protein sequences, making it the state-of-the-art method to encode the protein sequence with the highest quality embeddings. ProtTrans-BERT captures different biophysical properties of the amino acids, structure classes of proteins, domains of life and viruses, and protein functions in conserved motifs.• Using features from the amino-acid sequences strengthens our method because amino-acid sequences are available for any protein, while the protein interaction profile is unavailable for many proteins.• Similar to the baseline method, the second source of our features is omics data. However, instead of using the average of gene expression values over the patients, we aggregate the GE values using four different functions, giving a better and broader representation of the GE features and thus enhancing the prediction performance.• We used a DNN classifier for prediction, which can learn arbitrary boundaries and thus handle the binary classification better than decision trees . Since we have a high number of features, the DNN works better than ML classifiers.We believe that this result is a consequence of the leukemia dataset once again having a larger number of positive and negative samples and DL being more effective when dealing with more extensive data. Moreover, the result suggests that the main advantages/features that make OncoRTT more efficient and powerful are.Beyond that, we also provide the comprehensive datasets, OncologyTT, which is richer than the baseline datasets. That is, we increased the number of targets by collecting from two reliable sources (DrugBank and the TTD database).It is important to mention that in our datasets, OncologyTT, the negative samples in each fold of the 10-fold CV were the same negative genes, while in the baseline datasets, the negative samples are different in each fold with no overlap. Our method, OncoRTT, utilized supervised learning classifiers  and performed well by achieving high AUC using OncologyTT and the baseline datasets. This result demonstrated the capability of our method and the crucial distinct features that we extracted for each gene that helps the classifier differentiate between negative and positive samples . Moreover, when we evaluated the prediction performance, we calculated the AUC for each fold and then averaged the results over the 10-fold CV for both experiments, which mitigates possible bias and minimizes the variance in the classification performance.To further demonstrate OncoRTT’s use, we conducted several new experiments to predict new therapeutic targets for each cancer type separately. To perform these experiments, we utilized two datasets. The first is the OncologyTT dataset used to build and train our model, and the second is the Cancer-Target baseline dataset used as independent unseen test data. Firstly, we considered all the negative genes in this test data as unlabeled genes collected from a pool of human protein-coding genes. Thus, we focused on these unlabeled genes to predict the potential novel target genes. We implemented the following procedure per cancer type: first, we retrained the best model  using the whole OncologyTT dataset. Second, we used this model for prediction using the unseen test data. Third, to utilize the prediction results, we increased the number of epochs to 100 since the data samples are much higher. In this manner, all the negative genes in the test data predicted to be positive  are collected and ranked based on their prediction probability scores. We further analyzed the results of the top-10 ranked targets related to three cancer types: colon, kidney, and lung cancers, listed in Text mining: Evaluating the strength of the association between the target gene and specific cancer type using their co-occurrence in the published literature.1. Genetic associations: Phenome-wide associated loci prioritizing the target genes as a likely causal gene for specific cancers.2. Somatic mutation: A catalogue of somatic mutations that causally implicate the target genes in specific cancers.3. Drugs: Clinical candidates and (or) approved drugs pharmacologically targeting the target genes and indicated for the specific cancers-the source of approved interaction in the ChEMBL database.4. Pathways and system biology: Multiple pathway analysis tools providing enrichment based on genomic alterations associating the target gene with specific cancers.5. RNA expression: Transcriptomic analysis reports a significant differential expression of the target gene when comparing control samples with specific cancer samples.6. We used the Open Targets Platform to verify each cancer’s novel predicted therapeutic targets . This coADIPOQ, linked to colon, colorectal, and metastasis colorectal cancers  are linked explicitly to renal cell carcinoma, an aggressive kidney cancer originating in the lining of the proximal convoluted tubule (a part of the tiny tubes in the kidney) that primarily transport urine  related to the tumor-cell growth and survival . Therefop-values <0.05. p-values and the adjusted p-values (FDR) in p-values <0.05, which provides experimental support for our predicted therapeutic targets.First, we accessed “TCGA-LUAD” and “TCGA-LUSC” to collect around 58 TCGA tumor samples for lung cancer patients and 58 corresponding normal TCGA samples and obtained their relevant transcriptome profiling and gene expression quantification data. Then, we compared the normal and primary tumor samples using pair-wise tests to obtain the differential expression genes between these two groups. Finally, we filtered the DEGs output by determining a cutoff threshold from the EPHA5 and RPL10L were not classified as DEGs based on the normal and primary tumor samples using the cutoff mentioned above. There is no evidence linking RPL10L to lung cancer, although it is linked to other cancer types. However, EPHA5 is associated with lung cancer. We found clinical candidates, and an approved drug (VANDETANIB) targeting EPHA5 has been indicated for non-small cell lung carcinoma at different phases. However, these studies’ status is currently only defined as either “completed” or “active, not recruiting”   . Howeverhttp://www.mirdb.org/), an online database to predict functional microRNA targets  by applying BERT to the proteins’ amino-acid sequence per cancer type. We also extracted omics features using gene expression and gene mutation data. Finally, we combined these features and fed them to the DNN models for prediction. We additionally created datasets, OncologyTT, to build, train, and test our model. OncoRTT demonstrated its ability to differentiate between cancer-specific type target genes and non-target genes by achieving high AUC. Furthermore, OncoRTT achieved better prediction performance than the baseline method in most cancer types and, on average, across all the cancer types common to both studies. The obtained results indicate that the performance of DL classifiers exceeded the ML classifiers in most cases despite the DL model’s capabilities being limited by the small number of positive targets and more data needed to build and train DL models. The last limitation to highlight is the lack of interpretability of the BERT embeddings , preventing gaining insight into the critical features. Thus, as a future direction, the output of DL-based models should be made more interpretable and meaningful for bioinformaticians and experimental scientists.Applying other embedding techniques such as graph convolutional neural network (GCN) on PPI to generate latent feature representation of each gene.Integrating more omics features such as copy number variants.Utilizing different ML/DL classifiers.Making the DL models more interpretable in terms of feature extraction and classification.For further improvements to predict novel therapeutic targets, we suggest.Upgrade the OncologyTT dataset by including more data samples for the current cancers and incorporating more cancersValidating the novel predicted therapeutic targets.Our ongoing project aims to predict oncology-related DTIs for the newly identified targets provided by our method, OncoRTT, and for the existing targets using our DTi2Vec tools . In addiAlso, we plan to extend our work in several directions, including.To our knowledge, this is among the few studies to consolidate data from several resources per cancer type and then identify novel therapeutic targets per cancer using an ML/DL approach. Beyond that, our findings pinpoint some essential proteins per cancer type that could be possible therapeutic targets, for which we found several lines of evidence linking them to the specific cancer types. Non-etheless, follow-up experiments should be performed to validate these novel therapeutic targets.


We have entered the multi-omics era and can measure cells from different aspects. Hence, we can get a more comprehensive view by integrating or matching data from different spaces corresponding to the same object. However, it is particularly challenging in the single-cell multi-omics scenario because such data are very sparse with extremely high dimensions. Though some techniques can be used to measure scATAC-seq and scRNA-seq simultaneously, the data are usually highly noisy due to the limitations of the experimental environment.To promote single-cell multi-omics research, we overcome the above challenges, proposing a novel framework, contrastive cycle adversarial autoencoders, which can align and integrate single-cell RNA-seq data and single-cell ATAC-seq data. Con-AAE can efficiently map the above data with high sparsity and noise from different spaces to a coordinated subspace, where alignment and integration tasks can be easier. We demonstrate its advantages on several datasets.https://zenodo.org/badge/latestdoi/368779433. github: https://github.com/kakarotcq/Con-AAE.Zenodo link:  Single-cell multi-omics methods promise great opportunities to understand the cellular system more comprehensively. To achieve that, we should integrate multi-omics data, which describe cells from different perspective. Profiling multi-omics for the same set of single cells have become available , such asSome computational methods have been proposed to deal with these two crucial but challenging problems, aligning and integrating data from different omics. People usually integrate and align multi-omics data in the learned low-dimensional embedding space using dimension reduction techniques, such as principal component analysis (PCA)  and non-To promote the single-cell multi-omics data analysis, we propose a framework based on Contrastive cycle adversarial Autoencoders (Con-AAE), which can integrate multi-omics data .Con-AAE uses two autoencoders to map two modality data into two low-dimensional manifolds under the constrain of adversarial loss, trying to develop representations for each modality that are separated but cannot be identified by an adversarial network in a coordinated subspace . HoweverIn this section, we give our framework in details below with R as vectors of scRNA-seq domain and A as vectors of scATAC-seq domain. We would like to find two mappings f and h such that for any samples from R and A, f and h would maps the scRNA-seq profile and scATAC-seq profile to a coordinated subspace, where a discriminator f(R) and h(A). Due to the disparity between the original distributions of different omics data being complicated and non-linear, how to find a pair of f and h is a major problem.We denote g such that,In addition to map scRNA-seq data and scATAC-seq data to a coordinated subspace. The identity labels are available, and, therefore, it can be trained in a supervised way to maintain the cluster structure of cells. For Our main model is built upon the framework of Adversarial Auto-Encoders  specialid stands for indicated distance in the embedding space. Discriminator The usage of adversarial autoencoders aims to constrain embeddings for each modality in a coordinated subspace, where embeddings mapped from different omics are close to each other. Therefore, as shown in {Ei,Ej}  to map {Lrecon and Ladv are trained together with the same weights.The above losses The backbone framework enforces the embedding manifolds to align gradually. However, a critical problem underlying is that since scRNA-seq and scATAC-seq data are sparse in a high dimensional domain, the training procedure above only aligns and trains on those regions where the data exist.A in the embedding space around iD nor the encoder iE is trained on A, thus they would not compute in a “reverse” mapping way, and the result of For instance, if a region Therefore, we introduce a cycle-consistency loss shown in Lcyc aims to train the set of encoder–decoder on the domain where different omics data may not exist, which enforces the smoothness and consistency in those regions. In this way, we could compare the embedding of The above framework works in an unsupervised manner such that the embedded latent manifolds of multi-omics align properly.vice versa. We equally treat both modalities in contrastive training, which benefits the alignment task in the sense that multi-omics of the same single-cell data should obviously belong to the same cluster. We show that lowering the In-Batch distance indeed improves the alignment accuracy in the below ablation studies. On the other hand, contrastive training benefits integration by enabling the decision boundary to be smoother and more robust.We could further improve our work on both tasks by taking advantage of the ground-truth cell type labels. The cell type labels could refer to biological cell types or the labels of data batches collected from different times or platforms. Following the idea of contrastive learning , we emplaz as anchor vector in embedding space, we select pz such that nz as hard negative such that nz is defined as the closest vector from a different cluster. The objective immediately follows,In practice, we first encode data from two modalities to the embedding space. Define the embedding by α is the margin defined accordingly by us. Thus, by the contrastive loss, we tend to optimize,Above, C. The predicted result for an embedding kz is a vector of m is the number of cell types. Then, we have kc as the predicted probability of real cell types for kz. We try to minimize the loss Lclassifier as:We introduce a simple classifier in the coordinated subspace to further promote the framework’s performance. This simple classifier takes both embeddings encoded from scRNA-seq and scATAC-seq as input and predicts their labels for cell types. As Ladv and Algorithm 1:Training Procedurewhile numbers of training iterations do while k1 steps do     sample mini-batch    sample mini-batch az, pz for each   Search positives and negatives    Update end while  while k2 steps do     sample mini-batch    sample mini-batch    Update Discriminator end while  end while  In this way, the Discriminator In the above sections, we proposed several losses related to different objectives. Following the training procedure of Generative Adversarial Nets , we adopk closest scRNA-seq embeddings. We calculate k-nearest neighbours accuracy as:where n is the cell numbers in test set. R is the set of RNA-seq samples and A is denoted as the set of ATAC-seq samples. We denote ir as a sample from R, ia as a sample from A. ia and ir, respectively. k nearest k as 10, 20, 30, 40, and 50, evaluating alignment performance. We call this evaluation way We utilize two existing manners  to evaluInstead of assuming all datasets share the same underlying structure or specifying parts of hyperparameters like some traditional machine-learning methods , we obtak=10, 20, 30, 40, and 50, so there are 120(5 × 24) results for each method. We generate 24 sets of different sizes and SNRs. The generating process can be found in Supplementary Section B. The simulated datasets contain data of four different sizes, 1200, 2100, 3000, and 6000, each of which has six versions of SNR, 0, 5, 10, 15, 20, and 25, respectively. We implement all methods on the simulated datasets. The results in k is  scRNA-seq embedding. We conduct the transferring process on four real-world datasets test set with the help of Con-AAE and visualize embeddings by t-SNE . We can In this article, we propose a novel framework, Con-AAE, aiming at integrating and aligning the multi-omics data at the single-cell level. On the one hand, our proposed method can map different modalities into a coordinated subspace with the help of an adversarial loss and a novel cycle-consistency loss. On the other hand, we apply a novel self-supervised contrastive loss in the embedding space to improve the robustness and scalability of the entire framework. Comprehensive experimental results on the simulated and real datasets show that the proposed framework can outperform the other state-of-the-art methods for both alignment and integration tasks. Detailed ablation studies also dissect and demonstrate the effectiveness of each component in the framework. Our method will be helpful for both the single-cell multi-omics research and the general multi-modality learning tasks in computational biology.For future work, we aim to extend our work from a two-domain task to a multiple-domain study, allowing it to integrate and align multiple omics. Besides integration and alignment between sequence modalities, we intend to perform our method on different kinds of biological data, including but not limited to images, geometrical spatial structure, etc. Obviously, it is exciting to investigate the spatial transcriptomics data. We will also develop methods for translating modalities. By doing so, we hope to build a system that could benefit various downstream analyses in single-cell multi-omics and spatial multi-omics.btad162_Supplementary_DataClick here for additional data file.