Computational integrative analysis has become a significant approach in the data-driven exploration of biological problems. Many integration methods for cancer subtyping have been proposed, but evaluating these methods has become a complicated problem due to the lack of gold standards. Moreover, questions of practical importance remain to be addressed regarding the impact of selecting appropriate data types and combinations on the performance of integrative studies. Here, we constructed three classes of benchmarking datasets of nine cancers in TCGA by considering all the eleven combinations of four multi-omics data types. Using these datasets, we conducted a comprehensive evaluation of ten representative integration methods for cancer subtyping in terms of accuracy measured by combining both clustering accuracy and clinical significance, robustness, and computational efficiency. We subsequently investigated the influence of different omics data on cancer subtyping and the effectiveness of their combinations. Refuting the widely held intuition that incorporating more types of omics data always produces better results, our analyses showed that there are situations where integrating more omics data negatively impacts the performance of integration methods. Our analyses also suggested several effective combinations for most cancers under our studies, which may be of particular interest to researchers in omics data analysis. Cancer is one of the most heterogeneous diseases, characterized by diverse morphological, phenotypic, and genomic profiles between tumors and their subtypes. Identifying cancer subtypes can help patients receive precise treatments. With the development of high-throughput technologies, genomics, epigenomics, and transcriptomics data have been generated for large cancer patient cohorts. It is believed that the more omics data we use, the more accurate identification of cancer subtypes. To examine this assumption, we first constructed three classes of benchmarking datasets to conduct a comprehensive evaluation and comparison of ten representative multi-omics data integration methods for cancer subtyping by considering their accuracy, robustness, and computational efficiency. Then, we investigated the influence of different omics data and their various combinations on the effectiveness of cancer subtyping. Our analyses showed that there are situations where integrating more omics data negatively impacts the performance of integration methods. We hope that our work may help researchers choose a proper method and an effective data combination when identifying cancer subtypes using data integration methods. Cancer is a complex, heterogeneous, and serious disease extremely harmful to human health. Research on cancer has been going on for decades. With the continuous development and the declining cost of high-throughput technology, international collaborative projects, such as TCGA, ICGC, and CCLE, have generated and collected a large number of different omics data of the same cohort of cancer patients at different levels , includiApplying data integration methods to cancer analysis has three main goals: understanding the molecular mechanism of cancer, clustering disease samples, and predicting an outcome, such as survival or therapy efficacy . One of In recent years, many computational integration methods for cancer subtyping have been proposed. To choose among these methods, practitioners usually face two inevitable problems. The first problem is about how to compare the performance among these methods and the second problem is related to the selection of available data types to integrate in order to achieve the best possible results. The first problem is due to the lack of gold standards and consistent performic criteria , and theIn some recent studies on multi-omics integration methods, efforts have been given to understand the above two problems and most of the studies focused on the first problem on performance comparison and benchmarked the methods under only one performance criterion. Treating these subtyping methods as methods for unsupervised clustering, Chauvel et al.  evaluateTo the best of our knowledge, only two studies ,35 consiIn this study, we constructed three classes of benchmarking datasets by integrating all the possible combinations of four different types of multi-omics data of nine different cancers. Using these datasets, we evaluated and compared the performance of ten representative integration methods for cancer subtyping by taking into consideration their clustering accuracy as well as their clinical significance. Based on the experiment results, we further studied the influence of different omics data and their various combinations on the effectiveness of data integration methods for cancer subtyping.In this section, we briefly review existing integration methods and discuss in detail every process of our experiments, including data pre-processing, dataset construction, integration and subtyping, and performance evaluation.Researchers have been committed to developing computational multi-omics data integration methods for cancer subtyping using different strategies or techniques. Previous studies have classified the current integration methods into several categories based on different criteria, including clustering of clusters ,29, tranAiming at a systematic evaluation of integration methods and data selection issues, in this work, we adopted the simple methodology-based division proposed by Bersanelli , where iSNF constructs sample-sample similarity networks, one for each type of omics data, where the nodes representing the samples and weighted edges between nodes representing the similarity between the samples. A message-passing process is then used to update the weights of similarity among multiple similarity networks on the same set of samples iteratively to make these networks more and more similar. The spectral clustering [ustering  method iNEMO builds a similarity network between samples for each omics data and then modifies the similarity to relative similarity (RS) which is more comparable between omics. When integrating different omics data, NEMO simply averages RS in the different similarity networks for each pair of samples. Cancer subtypes are identified on the averaged similarity network using spectral clustering. NEMO can handle partial multi-omics datasets in the situation that each pair of samples has measurements in at least one common omics in which they are both measured.CIMLR uses different Gaussian kernels to measure patient-to-patient distance and then learns weights for these multiple kernels in each data type. After estimating the number of clusters using the gap statistic, it combines the multiple kernels into a resulting similarity matrix with a block structure. K-means clustering is used on the resulting similarity matrix to identify cancer subtypes.MultiNMF is a multi-view clustering algorithm based on nonnegative matrix factorization. It obtains a shared coefficient matrix as the common consensus data matrix by formulating a joint matrix factorization process with the constraint. The consensus data matrix is considered as a latent representation of the original data points reflecting the latent clustering structure shared by different views. K-means is used directly on this consensus matrix to obtain the cancer subtyping results.PFA exacts local information of each biological dataset using a dimension reduction strategy. Based on an adaptive optimization method, it aligns local sample patterns into a global sample-spectrum. The clustering method K-means [ K-means  is used LRAcluster uses a low-rank approximation based integrative probabilistic model to integrate different types of multi-omics data. The real-type, binary, and count-based data matrices are modeled as different distributions determined by a latent representation of the original data matrix. The objective function of LRAcluster is designed to be convex so that the global optimal can be found by the simple gradient-ascent algorithm. To clustering the samples, the K-means method is used on the latent representation matrix.moCluster is based on a multitable multivariate analysis. It finds latent variables using sparse consensus PCA and determines the number of latent variables using permutation and elbow test. Cancer subtypes are identified using conventional clustering methods such as hierarchical clustering and K-means clustering method.iClusterBayes uses different models for continuous, binary, and count data. It integrates different types of omics data by projecting them into a common low-dimensional integrated space, using the Bayesian latent variable regression model. K-means clustering is performed on the projected latent representation of the samples.PINS uses its own perturbation clustering method to cluster each omics datasets and constructs connectivity matrices which will be merged into a combined similarity matrix subsequently. It uses similarity-based clustering methods such as uses hierarchical clustering [ustering , PAM [99ustering  to identSubtype-GAN is a deep learning-based multi-omics data integration approach for discovering cancer subtypes. It extracts features from each omics data by relatively independent layers and integrates different omics by feeding the extracted information to the same shared layer simultaneously. Consensus GMM clustering is used to predict cancer subtyping results.The Cancer Genome Atlas Program (TCGA) has collected a large number of different types of omics data from more than 30 types of cancers. In this study, four types of omics data were chosen, including copy number variation in genome level, DNA methylation and miRNA expression in epigenome level, and mRNA expression in transcriptome level . To deciFor the mRNA and the miRNA expression data, we downloaded the level-three data of HTSeq-FPKM data and BCGSC miRNA Profiling miRNA-Seq data from TCGA. For the DNA methylation data, we chose Illumina Human Methylation 450 level-three data from TCGA. We selected the promoter-associated probes and mapped them to each gene as LRAcluster did. The promoter regions were based on Yang’s previous study ,102. WheFor each type of omic data, we first filtered the features that had more than 20% missing values across all patients and filtered the samples that had more than 20% missing values across all features as SNF did. Then, we selected the samples that belonged to the same patient cohort for four omics datasets. We used the K-nearest neighbor method to impute the missing values . Since bTo make a comprehensive evaluation and comparison, we constructed three different classes of benchmarking datasets using four types of omics data of nine cancers. Dataset group #1: Nine-cancer datasets. Dataset group #1 considers all four types of omics data for each of nine cancers. There are also two versions for each cancer dataset. The first version of each dataset contains samples with all features after the data processing step, which is denoted as the complete dataset. The second version of each dataset contains samples that only have significant features, which is called the significant dataset. We explain in the following how significant features are determined. For the mRNA expression and the DNA methylation data, we adopted the median absolute deviation (MAD) to select the top 2,000 most-variable features as iClusterPlus [sterPlus  did. ForsterPlus  identifiDataset group #2: Noise datasets. We used the significant datasets of BRCA and COAD in Dataset group #1 as the original datasets. Noise datasets were constructed by adding Gaussian noises with five different levels to each of the original datasets. The mean of the Gaussian noise is the same as the original dataset, and the variance is 0.5, 1, 2, 3, and 4 times of the original one, respectively.Dataset group #3: Gold standards datasets. There are two parts of Dataset group #3. The first part includes the BRCA and COAD datasets with true labels. Although there is no gold standard for cancer subtyping, subtypes of BRCA and COAD patients have been analyzed by the TCGA group in a previous study [us study , and we us study  that arePrevious studies on integrative cancer analysis have used different combinations of multi-omics data, among which mRNA expression + miRNA expression and mRNA expression + miRNA expression + DNA methylation are the two most used combinations. Because of the inconsistency of the data combinations used in previous studies and the lack of a common guideline, we enumerated all the possible combinations of the four types of omics data. As https://github.com/GaoLabXDU/CEPICS). Since the LRAcluster, PFA, and MultiNMF package only output an integrated sample-feature matrix rather than clustering (subtyping) results, we used the K-means clustering with 300 iterations on the sample-feature matrices to obtain stable subtyping results.We used the packages or codes provided by the authors of these ten representative integration methods to obtain the subtyping results on each data combination in the whole benchmarking datasets. Using the subtyping results we obtained, we evaluated the performance of the ten integration methods. To make a comprehensive and reliable comparison, three performance criteria were used: the accuracy, the robustness, and the computational efficiency.Accuracy. The accuracy is the most important factor for users to choose a method. Accurate results may help researchers find new cancer subtypes and make precise treatments for different cohort patients. However, evaluating the accuracy of integration methods for cancer subtyping is challenging because of the lack of gold standards and ground truth. We believe that in addition to the use of clustering-based metrics, subtyping results should also be evaluated using clinically-relevant metrics to make sure that good subtyping results capture significant differences in clinical features. Therefore, we evaluated and compared the accuracy of the ten methods by taking into consideration both of the clustering accuracy and the clinical significance. For unlabelled datasets, we used the log-rank test p-value and silhouette coefficient to evaluate the subtyping results. The p-value of the log-rank test is the most wildly used metric that represents the extent of the significance of different cohort patients on survival time from the clinical aspect [l aspect –111. Thel aspect  measuresRobustness. Due to the limitations of sequencing techniques and experiment conditions, different levels of noise exist in the omics data. When we integrate multi-omics data, the effects of noise may get worse and cause inaccurate subtyping results. It is, therefore, necessary and important to take into consideration the robustness of integration methods. We evaluated the robustness of each method by calculating NMI and ARI between the results of original datasets and the corresponding noise datasets disturbed by different levels of noise.Computational Efficiency. Typical biological datasets are known as small-sample-high-dimension datasets in that the number of samples might not be very huge, but the number of features associated with a sample is normally large. In most omics datasets, there are normally a few hundred samples, while each sample typically has thousands of features. For example, the original COAD methylation dataset has 291 samples, but a sample has over 450 thousand features. Datasets with such characteristics pose a great challenge for integration methods since the running time of these integrated methods may become too high.Using the experiment results on the accuracy of the integration methods we used on all possible combinations of the omics data types of the cancers, we further analyzed the influence of different omics data on the effectiveness of data integration in cancer subtyping. Moreover, to understand the inconsistency of the results reported in different previous studies of data integration methods for cancer subtyping, we studied the impact of different data combinations on the performance of the integration methods for eight of nine cancers and identified some data combinations that are effective for most of the eight cancers. We hope that our analysis and observations serve as useful guidance for the practice of using an integrated approach to cancer subtyping.We evaluated each method in terms of its accuracy, robustness, and computational efficiency on the constructed benchmarking datasets. A comprehensive comparison of the performance of the ten integration methods was conducted, taking into consideration both their clustering accuracy and their clinical significance. We further analyzed the influence of different types of omics data and their combinations on cancer subtyping.k) is difficult and still an open question. We notice that, in these ten selected methods, iClusterBayes, SNF, PINS, NEMO, moCluster, CIMLR, and Subtype-GAN have their own criteria to estimate the best k from the user-specified range. For SNF, NEMO, and CIMLR, they estimate the best k automatically according to eigen-gaps or rotation cost. For PINS, it automatically evaluates the instability of different connectivity matrices to determine the best k. For iClusterBayes and moCluster, they calculate different metrics  to generate a plot that is used to choose the best k by users manually. For Subtype-GAN, it uses Consensus GMM clustering to choose the best k. Therefore, we evaluate and compare the performance of all methods suggested and all possible number of clusters to make our experiments comprehensive. All ten integration methods were applied to each dataset in Dataset group #1 and #3, using each of the 11 possible combinations of the data types, which were summarized in As cancer subtyping results are generally obtained by unsupervised clustering, the determination of the number of cancer subtypes/clusters (denoted as k) was unknown, we first evaluated and compared the performance based on the number of clusters suggested by each method. As these methods require users to specify a maximum number of clusters (i.e. k-max), we set k-max to 8, which was the maximum number of subtypes commonly identified in cancer subtyping studies [k were listed in S1 Table in For Dataset group #1, as the true number of subtypes , iClusterBayes and LRAcluster performed the best  is the ranking of a method as evaluated with a specific metric and a specific dataset, while Item(∙) is the number of the datasets used for a specific metric. The rationale behind the above formula is that a good method should be accurate under both clustering-based metrics and clinical-based metrics. As shown in To get an overall ranking of these integration methods, we aggregated the performance measurements of them over all datasets and all performance metrics using the following formula:i, we obtained a set of subtyping results i,c,k,σR, where c = 1,2,⋯,11 is the index of the data combination used, k = 2,3,⋯,8 is the number of subtypes to calculate, and σ = 0,0.5,1,2,3,4 is the noise level. For each method i, we calculated the NMI and ARI between the result Ri,c,k,0  and the result i,c,k,σ′R obtained when different levels of noise are introduced into the data. As PINS could not cluster samples into a specific number of clusters, we calculated the NMI and ARI between Ri,c,0 and i,c,σ′R. The calculations of robustness were reported in We believe that robust methods are those that have a reasonable degree of resilience to noise in the data. Under this assumption, we used Dataset group #2 to study how quickly the subtyping results of these methods change as higher and higher levels of noise are introduced into the data. In this set of experiments, for each method k = 2,3,4, and even better than SNF when k = 2,3. However, when k>4, the NMI decreases sharply and becomes much lower than the NMI of SNF under the same condition. MultiNMF has the acceptable robustness performance at only k = 2,3. Using the BRCA noise datasets, we found that LRAcluster is the most robust method since it has the largest average NMI at all noise levels . When thThe computation time required to complete the tasks is an important factor to consider when practitioners choose among different integration methods, especially when the amount of data to be processed is huge. Sometimes, a user may prefer a less accurate method over a method that takes a long time to finish.Dataset group #1 and the Pan-cancer dataset in Dataset group #3 were used to evaluate the computational efficiency of the integration methods. The scales of the datasets used in the evaluation are listed in For each method and each combination of data types, the total running time is the sum of the time the method takes to complete the computation for all the settings for the number of clusters from 2 through 8. The calculations of computational efficiency were reported in The scale of a dataset is decided by two factors: the number of samples and the number of features per sample. To get a better picture about the computational efficiency of the integration methods, we analyzed the influence of the two factors on the running time. Using our experiment results on the accuracy of the integration methods over various data combinations and cancers, we explored the influence of different omics data and their combinations on the effectiveness of data integration for cancer subtyping. We focused on two issues. The first one is whether a specific type of omics data has similar discriminative power for subtyping on different cancers. The second one is whether there are combinations of data types that are effective for the majority of the integration methods and cancers.k in these ten methods are different. The performance based on the method-suggested k and all possible k are inconsistent for some methods. There could be two possible reasons: (1) the procedure of suggesting k in the methods may not work for our constructed benchmarking datasets; (2) the performance based on the method-suggested k are not optimal. Therefore, in order to avoid the bias introduced by the selection of k, we do not take the results using the method-suggested k into consideration. Moreover, as PINS cannot generate the subtyping results for a specific k, we do not consider the results of PINS in the following analysis.To quantify the relative influence of different types of omics data on the effectiveness of data integration for a specific cancer, we use the Weighted Average Z-Score (among the group of 4 omics data types) that takes into consideration the accuracy measures from our experiments on the integration methods under all accuracy metrics, including precision, NMI, ARI, F-measure, silhouette coefficient and transformed log-rank test p-value. We notice that the strategies of suggesting For a given cancer, the Weighted Average Z-Scores are calculated as follows. For each type of omics data and each integration method, we calculate the average accuracy measure  of the integration method applied to a specific data combination that contains this type of omics data. Since each omics data type belongs to exactly 7 of the 11 possible combinations, there are 7 measurements for each accuracy measure, and we take the mean of these 7 values as the Overall Accuracy Measure of the omics data type on a specific metric. We then calculate the z-scores (with respect to the particular integration method) among the four types of omics data, using their Overall Accuracy Measures. The weighted average is defined in a similar way as the weighted rank in the Section “Accuracy”. The Weighted Average Z-Score of a type of omics data is its average z-scores over all the integration methods. Regarding the use of different metrics, we mention that, as we had calculated the silhouette coefficient and the p-value of these methods under different settings for the number of clusters (from 2 to 8), we took the average of them. For the other four metrics, we used the value obtained in the experiments directly. When we evaluated the clinical significance of KIRP, we found that many zero p-values occurred which was abnormal, and could not do the log transformation. Therefore, we did not analyze KIRP in this work.To analyze the effectiveness of different data combinations, we calculated the weighted average z-scores among all possible combinations of data types for each pair of cancer and the integration method. For a given cancer and integration method, the weighted average z-scores are based on the accuracy measures of the method applied on the datasets of that cancer using different data combinations, and the strategies of choosing measurements and the calculation of weighted average z-scores are the same as mentioned above. The radar plots of these weighted average z-scores are shown in As these radar plots show, we find that there are no combinations in any cancer datasets that are effective for all nine methods. The situation becomes better when we consider combinations that are effective for five or more methods. For example, for the BRCA datasets, the collection of data combinations {“13”, “14”, “123”, “124”, “134”, and “234”} are all effective for at least five integration methods, and we consider these combinations as effective combinations of BRCA. In the following analysis, we summarize the effective combinations of each cancer to find the combinations which are effective for most cancers. It turns out that the combination “13” (i.e. mRNA expression and DNA methylation) is effective for seven cancers. The combinations “14” (i.e. mRNA expression and CNV), “123” , “124”  are effective for six cancers.To further verify the effectiveness of the combinations identified in the above analysis, we compared the performance of the integration methods using these combinations with the performance of three widely-used clustering methods  applied to individual omics data types. We used the silhouette coefficients and the transformed p-values calculated in the log-rank test for the comparison.Because of the characteristic of multi-omic data, high dimensions relative to the small samples, most of the existing methods can not properly separate the samples in the original space. For achieving better clustering results in multi-omics datasets integration, many effective integration methods project features of each omic data into a new integrated space and then cluster samples in this integrated space. In order to show the advantages of data integration, we used the silhouette coefficient results based on integrated space to do the comparison. As k. For each data combination and each cancer, we calculated the average p-value of the eight integration methods and the average p-value of three single-omic clustering methods, using the same number of clusters k. We then counted the total number of cases  where integration methods using the data combination have improved performance  over the single-omic-based clustering methods. The bottom plot in As there is no gold standard, a well-known challenging task to use unsupervised clustering methods is the selection of the parameter setting for the number of clusters. To further understand the power and clinical advantage of data integration in cancer subtyping, we compared the p-value of single-omic clustering-based subtyping methods and integration methods when they use the same (and fixed) number of clusters k after specifying a k-max, we must point out that PINS is the most friendly-used method because its automatic procedure of estimating k does not require users to do any additional work. In contrast, some methods (e.g. iClusterBayes and moCluster) require users to select k based on their generated curves, however, it could be a tough task for users because the breakpoint of the curve cannot be easily determined or inconsistent k could be recommended by two different types of plots. In summary, NEMO and SNF are recommended for general cancer subtyping tasks, however, users may also want to consider the other eight methods depending on their specific purpose or under certain circumstances.Many multi-omics integration methods have been proposed to improve our understanding of cancer. Cancer subtyping can be used in precision medicine to help patients receive more accurate and personalized treatments based on their response to different therapies and drugs. In this study, we have conducted a comprehensive evaluation and comparison of ten representative multi-omics integration methods in terms of their accuracy, robustness, and computational efficiency. Using our benchmarking datasets constructed from collections of real cancer data, we were able to evaluate the accuracy of these methods from the clinic perspective as well as the computational perspective. From our experiments and analyses, we observed that the methods NEMO and SNF perform very well in all three criteria, i.e. accuracy, robustness, and computation efficiency, which is mainly because the strategies that they adopt can capture both shared and specific information from different omics data and make the integrated similarity networks retain more information from every single similarity network with low-level noise. The other methods have certain limitations. For example, LRAcluster is the best in robustness, but has poor computational efficiency; iClusterBayes, while being good in accuracy and robustness, has the worst computational efficiency; PFA has the poor robustness and accuracy, but has better performance in running time. Among the seven methods that can suggest an optimal Using our results on the accuracy, we have also analyzed the influence of different omics data types and their combinations on cancer subtyping. Our analysis shows that the influence of these omics data types varies, and several commonly-used combinations of omics data types can indeed improve the accuracy of all the ten methods as measured by both clustering and clinical metrics. On the other hand, our analysis indicates that integrating more types of omics data may negatively impact the performance on cancer subtyping, refuting the widely held intuition that incorporating more types of omics data always helps produce better results . From ouIt is worthwhile to mention another surprising observation on the significance of DNA methylation data in the effectiveness of integration methods. Ramazzotti et al. proposed their method CIMLR  based onAs more types of omics data have being generated for cancer patients and more integration methods, especially deep learning-based methods, have being proposed, new data types such as proteomics and single-cell omics data as well as machine learning-based methods should be considered in the future development of the benchmarking framework for data integration.S1 FileS1 Fig. Silhouette coefficient comparison based on integrated space of Dataset group #1 Nine-cancer Datasets. We use “iCB”, “LRA”, “moC”, “CIM”, “MNMF” and “SGAN” to represent iClusterBayes, LRAcluster, moCluster, CIMLR, MultiNMF, and Subtype-GAN, respectively. (A) Silhouette coefficient based on the suggested k of methods. We set k-max as 8 and let each method suggest the best k. Each of the 11 data points in a box represents a silhouette coefficient of the subtyping results based on the method suggested k obtained by the corresponding method using one of the 11 possible combinations of data types. (B) Silhouette coefficient based on all the possible k. Each of the 11 data points in a box represents the average silhouette coefficient of the subtyping results from k = 2 to 8 obtained by the corresponding method using one of the 11 possible combinations of data types. k of each method using Dataset group #1 Nine-cancer Datasets.S1 Table. Suggested  Notations: B-BRCA, C-COAD, KC-KIRC, LA-LUAD, LS-LUSC, A-ACC, KP-KIRP, LI-LIHC, T-THYM. m-mRNA expression, mi-miRNA expression, me-DNA methylation, cnv-copy number variation. k of each method using Dataset group #3 Gold Standard Datasets.S2 Table. Suggested  Notations: m-mRNA expression, mi-miRNA expression, me-DNA methylation, cnv-copy number variation.Evaluation and comparison of multi-omics data integration methods for cancer subtyping. (DOCX)Click here for additional data file.S2 File(XLSX)Click here for additional data file.S3 File(XLSX)Click here for additional data file.S4 File(XLSX)Click here for additional data file.S5 File(XLSX)Click here for additional data file.S6 File(XLSX)Click here for additional data file.


However, sample mix-up is a pervasive problem in proteogenomics because of the complexity of sample processing. Here, we present a pipeline for Sample Matching in Proteogenomics (SMAP) to verify sample identity and ensure data integrity. SMAP infers sample-dependent protein-coding variants from quantitative mass spectrometry (MS), and aligns the MS-based proteomic samples with genomic samples by two discriminant scores. Theoretical analysis with simulated data indicates that SMAP is capable of uniquely matching proteomic and genomic samples when ≥20% genotypes of individual samples are available. When SMAP was applied to a large-scale dataset generated by the PsychENCODE BrainGVEX project, 54 samples (19%) were corrected. The correction was further confirmed by ribosome profiling and chromatin sequencing (ATAC-seq) data from the same set of samples. Our results demonstrate that SMAP is an effective tool for sample verification in a large-scale MS-based proteogenomics study. SMAP is publicly available at  Sample mix-up is a potential problem in large-scale omic studies due to the complexity of sample processing. Here, the authors present a pipeline for sample matching in proteogenomics to verify sample identity and ensure data integrity. These studies usually involve collecting single- or multi-layer omic data from a large number of subjects, followed by statistical significance tests. The power and effectiveness of the statistical tests rely on the accuracy of sample identity8. However, sample mix-up in large-scale multi-omics studies, as a result of specimen labeling errors, or data mismanagement, is a widespread problem9. Such error is often neglected, and it generally leads to irreproducible results, weakened statistical testing power, and false conclusions10. Therefore, to reduce the bias and increase the precision for subsequent analyses, verification of sample identity is one of the first steps in a large-scale omics project.With the development of high-throughput technologies in recent years, remarkable accomplishments have been made by well-conceived and large-scale projects carried out by large consortia, such as the Cancer Genome Atlas (TCGA) project11 that is complementary to other omics, such as genomics and transcriptomics. Due to recent technical advances, MS-based proteomics has undergone rapid development yielding numerous large-scale proteomic data5. Most of these data are generated by multiplexed isobaric labeling-based protein quantification methods, such as isobaric tags for relative or absolute quantitation (iTRAQ) and tandem mass tag (TMT). For example, the TMT-based strategy can now measure as many as 27 samples in one batch simultaneously12, and potentially allows for the measurement of 81 samples when combined with metabolic labeling by amino acids in cell culture (SILAC). These multiplexing strategies would further exacerbate the problem of sample mix-up, posing a significant challenge for sample verification and calibration.Mass spectrometry (MS)-based proteomics has emerged as an important molecular profiling technology13, correlation of mRNA and protein14, and correlation of variant fractions15. For example, we have also implemented a genotype-based method to address the sample mix-up problem from sequencing-based transcriptomic data16. Most of these methods exploit the genotypic information of a sample, which can be directly derived from sequencing data. Calling genotype from multiplexed isobaric labeling-based quantitative proteomic data remains a major challenge because multiple labeled samples are mixed during the experiment. While the proteogenomics approach has been widely used to detect variant peptides, it only calls variant peptides from label-free or isobaric labeled proteomic data19. As a result, there is no method available for verifying and calibrating sample identity in a multiplexed quantitative proteomic study.Several methods have been developed to verify sample identity in large-scale genomic and transcriptomic studies using genotype concordanceIn this study, we present a pipeline for Sample Matching in Proteogenomics (SMAP). SMAP first performs proteogenomics to detect variant peptides from multiplexed isobaric labeling-based quantitative proteomic data, and then infers genotypic information of each sample based on the expression level of the variant peptides. SMAP finally verifies and calibrates sample identity based on a combination of concordance and specificity scores using inferred genotypic information. The performance of SMAP is assessed by a simulation study and a large-scale proteomic dataset.19, which constructs a customized database using genomic variant files  or indirectly from RNAseq raw data in FASTQ format, in which JUMP performs preprocessing, tag generation, MS/MS pattern matching, and scoring as previously reported20. The identified variant peptides are further filtered with the target-decoy strategy to control the false discovery rate (FDR)22.SMAP consists of three main components Fig. : (i) ide2-scale, followed by a scale normalization using the following formula,SMAP infers sample genotypic information based on the relative expression level of variant peptides identified in proteomic data with multiplexed isobaric labeling-based quantification methods Fig. . For eacSMAP then uses the proteogenomics approach to detect variant peptides Fig. , followeOnce the genotypic information of each sample is inferred, SMAP verifies and calibrates the sample identity using a combination of two scores: concordance score  and specificity score  were quantified by 29 batches of 11-plex TMT-based proteomic technology. In addition to proteomic data, the PsychENCODE BrainGVEX project also generated other omic data with matched samples, including 285 samples with low-depth whole-genome sequencing (WGS), 426 samples with RNA sequencing (RNA-Seq), 295 samples with assay for transposase-accessible chromatin using sequencing (ATAC-Seq), and 197 samples with ribosome sequencing (Ribo-Seq). Previous analysis using the DRAMS program has identified ~19% mislabeled samples in both WGS and ATAC-seq, ~25% in Ribo-seq, and ~3% in RNA-seq data23 and 17,844,001 theoretical peptides translated from variants generated by WGS and RNA-seq data. SMAP identified a total of 5,065 unique variant peptides  and a signal distribution   can be used as a cutoff. This is consistent with the result found when evaluating the impact of the minimal intensity on the number of genotypes and the Cscore  ratio Fig. . For a vty) Fig. . The resore Fig. . The anaore Fig. . Followiore Fig. .After optimizing the parameters, we used 7628 filtered SNPs to infer genotypic information for 288 biological samples and 31 controls in the PsychENCODE BrainGVEX proteomic data  showed the same calibration in samples from all three platforms: MS-based proteomic, ATAC-seq, and Ribo-seq. In addition, a total of 9 samples had the same calibration between proteomics and ATAC-seq, 5 between proteomic and Ribo-seq, and 6 proteomic-specific calibrations  were reciprocally swapped Fig. , with ab14, a program recently developed to correct sample mislabeling in omic data through correlating mRNA and protein expression levels. A key step in the COSMO is feature selection by selecting highly correlated gene and protein pairs. With the default correlation coefficient cutoff of 0.5 in COSMO, only a few features were selected in our data probably due to about 20% of the samples being mislabeled based on the correction made by SMAP. To obtain an optimal number of features, we examined correlation coefficients ranging from 0.2 to 0.55, and found that the correlation coefficient of 0.35 produced the best performance when considering the number of selected features and matched samples   samples, including 218 samples matched to its original ID, and 52 mislabeled samples but corrected to the same identity by both methods. In the remaining 18 (6%) samples that showed inconsistent correction between COSMO and SMAP Sample mix-up is a pervasive problem in large-scale omic studies. In this study, we described a method, SMAP, for validating and correcting samples that are used for a large-scale TMT-based quantitative proteomic study. By applying SMAP to proteomic data from 288 biological samples in the pyschENcode BrainGVEX project, we verified and calibrated ~18% mislabeled samples. Based on the simulation study, SMAP has proven capable of validating the sample identity of as few as 20% of reliable genotypes.19, there is no study to infer genotypic information for each sample in multiplexed isobaric quantitative proteomic data. It is the first time that SMAP determines genotypic information of each sample based on the expression level of the variant peptides. With advances in next-generation sequencing and high-resolution mass spectrometry technologies, omic data from the same set of biological samples are now routinely collected in a project. Appropriate data quality controls at each step are required to ensure high data integrity. Therefore, SMAP complements existing approaches for sample verification and calibration that have been developed for genomic and transcriptomic data.While the proteogenomics approach is increasingly used to detect protein-coding variants and mutant splicing sitesCorrection of sample mislabeling is paramount to all large-scale omic studies. The SMAP program centers on sample correction for proteomic data by leveraging genotypic data. This concept of using genotypic data for sample correction can be applied to other omic data. The genotypes of each sample can be readily extracted from all massive sequencing data, ranging from whole-genome DNA sequencing, epigenome , to the transcriptome (bulk RNA and single-cell RNAseq). However, inferring genotypes from MS data is a much greater challenge than from sequencing data as we need to convert MS signals into amino acids and then convert amino acids into genotypes. More importantly, we proposed a scoring scheme in which a combined score is generated from two scores: (i) Cscore, which is a measure of the goodness of fit of genotypes inferred from proteomic data to non-mutant genotypes; (ii) 24.Although SMAP can infer sample-level genotypic information for sample verification, the inference certainty was still imperfect. First, the expression level of a variant peptide is quantitative, unlike variants called from genomic and transcriptomic data generated by sequencing technologies. Although we proposed to use three quartiles to determine the genotypes, the genotypes inferred by SMAP show uncertainties. In the analysis of the PsychENCODE BrainGVEX proteomic data, we found many mismatched genotypes were in the boundary of each range. Second, the homozygous mutant genotype (BB) and heterozygous genotype (AB) cannot be distinguished solely depending on the expression level if only AB genotypes are present in the samples. To mitigate this issue, SMAP examines the number of genotypes in the original genotypic data , and 29 individuals with bipolar disorder (BD). The samples include 282 Caucasians, 1 Hispanic, 1 African American, 3 Asian American, and 1 unknown. The brain frontal cortex samples were subjected to 29 batches of deep proteome profiling using TMT LC/LC-MS/MS technology. In addition to the proteomic data that we generated, five other omic data are also available at the psychENCODE knowledge portal (https://psychencode.synapse.org/Explore/Studies/DetailsPage?studyName=BrainGVEX), including RNA-Seq data from 426 samples , assay for transposase-accessible chromatin using sequencing (ATAC-Seq) data from 295 samples , and ribosome sequencing (Ribo-Seq) data from 197 samples . The genotypes were generated by the combination of three platforms, including Affymetrix chip data, whole-genome sequencing, and RNA-seq data16.A total of 288 well-characterized postmortem human brain samples  from the Stanley Medical Research Institute and Banner Sun Health Research Institute were used for this study19. In JUMPg program, the variant peptides were identified by the JUMP search engine, which performs preprocessing, tag generation, MS/MS pattern matching, and scoring20. JUMP was used to search MS/MS raw data against a composite target/decoy database22 to evaluate the false discovery rate (FDR). In JUMPg, a two-stage FDR method is used: in the first stage, the MS/MS data are searched against a reference protein database, and the confidently identified spectra are removed; in the second stage, the remaining spectra are searched against the variant protein database, and the FDR for variant peptides is calculated based on the second stage search results. The target database contains both core protein sequences downloaded from UniProt23 and all theoretical variant peptides generated from all nonsynonymous protein-coding variants. To generate theoretical variant peptides, all variants in the genotypes were re-annotated using the genome annotation tool ANNOVAR26 based on the human reference genome . We used the UCSC gene annotation model. The decoy database was generated by reversing protein sequences in the target database. Major parameters included precursor and product ion mass tolerance (±8 ppm), full trypticity, static mass shift for the TMT tags (+229.16293) and carbamidomethyl modification of 57.02146 on cysteine, dynamic mass shift for Met oxidation (+15.99491), maximal missed cleavage (n = 2), and maximal modification sites (n = 3). To filter variant peptides, PSMs were first filtered by user-specified parameters , then by precursor ion mass accuracy. The resulting PSMs were further grouped by precursor ion charge state and tryptic ends and then filtered by matching scores (Jscore and ΔJscore) to achieve a peptide FDR < 5%. If one peptide could be generated from multiple homologous proteins, the peptide was assigned to the protein with the highest PSM based on the rule of parsimony.JUMPg program was used to identify variant peptides by taking MS-based proteomic data and a customized database curated with genomic variant data27: (1) extracting TMT reporter ion intensities of each PSM; (2) correcting the raw intensities based on the isotopic distribution of each labeling reagent ; (3) removing sample loading bias by normalization with the trimmed median intensity of all PSMs; (4) calculating the mean-centered intensities across samples (e.g. relative intensities between each sample and the mean), (5) summarizing relative intensities of proteins or variant peptides by averaging related PSMs. The quantification values of variant peptides with and without missing values were extracted for inferring genotypic information for each sample by SMAP.The analysis was performed in the following stepsWe generated a simulated dataset to test the performance of SMAP by evaluating how many genotypes identified from MS-based proteomic data are needed for validating and correcting the sample identity. We first randomly extracted a subset of a matrix with 500 genotypes and 420 samples. We then randomly selected a sample from the matrix and permutated the genotypic information with replacement sampling at a certain probability level Further information on research design is available in the Supplementary InformationPeer Review FileDescription of Additional Supplementary FilesSupplementary Data 1Supplementary Data 2Reporting Summary


Genome-wide association studies (GWAS) have identified genetic variants associated with multiple complex diseases. We can leverage this phenomenon, known as pleiotropy, to integrate multiple data sources in a joint analysis. Often integrating additional information such as gene pathway knowledge can improve statistical efficiency and biological interpretation. In this article, we propose statistical methods which incorporate both gene pathway and pleiotropy knowledge to increase statistical power and identify important risk variants affecting multiple traits.We propose novel feature selection methods for the group variable selection in multi-task regression problem. We develop penalised likelihood methods exploiting different penalties to induce structured sparsity at a gene (or pathway) and SNP level across all studies. We implement an alternating direction method of multipliers (ADMM) algorithm for our penalised regression methods. The performance of our approaches are compared to a subset based meta analysis approach on simulated data sets. A bootstrap sampling strategy is provided to explore the stability of the penalised methods.Our methods are applied to identify potential pleiotropy in an application considering the joint analysis of thyroid and breast cancers. The methods were able to detect eleven potential pleiotropic SNPs and six pathways. A simulation study found that our method was able to detect more true signals than a popular competing method while retaining a similar false discovery rate.We developed feature selection methods for jointly analysing multiple logistic regression tasks where prior grouping knowledge is available. Our method performed well on both simulation studies and when applied to a real data analysis of multiple cancers. In recent years, genome-wide association studies (GWAS) have identified genetic variants associated to multiple traits. The phenomenon where one genetic loci affects multiple different phenotypes is called pleiotropy . A comprASSET [Amongst these methods one of the most popular for practitioners is a subset based meta-analysis ASSET . This meASSET , 11.Among existing methods, mixture model methods have received attention for modelling pleiotropic associations . These mIn this article, we propose novel methods which model pleiotropy for genomics data in the case of independent datasets. Our methods are developed to model pleiotropic correlation amongst jointly analysed traits and account for the gene structure information contained in the data. Integrating additional information such as gene pathway knowledge offers the potential to improve statistical efficiency. Our statistical approach exploits both gene (or pathway) and pleiotropy knowledge to increase the statistical power of identifying risk variants shared by multiple diseases. We conduct simulation studies to evaluate the performance of our method.K studies. In particular, we make note of Lounici et al. [Our method can be motivated in a multi-task framework . In our i et al.  who consi et al.  who propWe are motivated by an application of our methods to the analysis of pleiotropy between thyroid and breast cancers. Thyroid and breast cancers share some similarities: both are more frequent in women, are influenced by reproductive factors and are hormonally-mediated. Moreover, individuals diagnosed with breast cancer are more likely to develop thyroid cancer as a secondary malignancy than patient diagnosed with other cancer types . These aThe remainder of this article is organised as follows. In methods section, we describe the core model, algorithm and inference method. A stability exploration based on a bootstrap approach is provided. Our method is tested in a simulation study where we compare the joint penalised likelihood approach to state-of-the-art variable selection strategies to investigate pleiotropy. The results section present the results of the simulation study and the pleiotropy investigation on thyroid and breast cancers. The final section concludes with a discussion of the methods and potential extensions of the work.K independent datasets, n1,…,nK samples respectively. The response variable yik∈{0,1} is the binary phenotype of the ith individual of the kth study and p variables of the ith individual of the kth study. These data are assumed to come from a logistic regression model where Suppose we have data from k=1,…,K, where kth study. To simplify further notation, let K regression coefficients corresponding to the jth SNP over the K datasets. We let βjk denote the regression coefficient for the jth SNP of the kth study. We assume that the set of SNPs can be partitioned into G groups where each SNP belongs to a single unique group. Let πg,g=1,…,G denote the set of SNPs contained in the gth group and ng be the number of SNPs in group g. Finally, we let the matrix of all regression coefficients be denoted by B=. Since the K multiple studies were observed independently, the negative log likelihood for the combined datasets has the generic expression: for where Sparse Group Multi-Task (SGMT) approach is based on penalised likelihood maximisation. Using the likelihood form for independent datasets  by fixing α=0.Sparse multi-task penalised model (SMT) by fixing α=1.Sparse Grouped multi-task penalised model (SGMT) with 1>α>0.Equation  enables λ1=(1−α)λ and λ2=λα. The ADMM formulation of our optimisation problem is given by U with Lagrange multiplier ρ and is given by the following: We propose to fit this model Eq.  using thThe ADMM algorithm makes the following set of updates: l2 regularisation logistic regression, a convex optimisation problem and a dual variable update (respectively): l2 regularised logistic regression is solved using the efficient Limited-memory Brouden-Fletcher-Golfarb-Shanno (L-BFGS) algorithm implemented in the RcppNumerical package. Let A corresponding to the SNP indices in πg. Following [Zt+1) are fitted using tuning parameters chosen by repeated K-fold cross-validation. Genes (or pathways) are then detected and strategy . This nostrategy , 24. Boostrategy , 25 and strategy .selected genes (or pathways) and non selected genes (pathways) over the M bootstrap samples quantifies the stability of the selected variables. We report both the variables selected and the selection rates for the application. More details on the bootstrapping procedures are given in the Results and Applications sections. Specifically, we estimate the probability of selection for each variable (or group of variables) with a given set of tuning parameters  based on the proportion of times they are included on the bootstrapped fits. Commonly in GWAS data we are concerned with controlling the false discovery rate (FDR). One way to approach controlling this would be to specify a cut-off for the minimum probability of inclusion for each variable. In our simulation study we consider a strict control, only selecting variables that are included on every bootstrap. This conservative strategy would often be too drastic and for or application section we adopt another approach. This is to report the variables that had selection probabilities at least as high as the selection probabilities of the variables included on the full data fit. We state both the variables and their bootstrapped selection rates. More details on the bootstrapping procedures are given in the Results and Applications sections. An example demonstrating the Bootstrapping approaches is given in the supplementary material and can be reproduced from github.com/matt-sutton/SGMT.Resampling bootstrap is used in our approach where the different models are implemented on each bootstrap using tuning parameters selected from the original data. The frequency of the ℓ1 penalty of the lasso jth coefficient. Similar to the adaptive lasso we also allow a weighted version where the G2,1-norm and ℓ2,1-norm penalties are replaced by, G2,1-norm weights γg are taken to be the inverse of the G2,1-norm of the OLS coefficients for g=1,...,G. Similarly we take the weights for the ℓ2,1-norm to be κi where κi is chosen as the inverse of the ℓ2,1-norm applied to the OLS coefficients for i=1,...,p. That is, we set the weights to be: While penalised approaches allow for shrinkage of coefficients to zero, they come at the cost of possibly excessive shrinkage to non-zero coefficients. This has motivated a number of approaches that aim to reduce the effect of shrinkage on non-zero coefficients. One simple approach is the adaptive lasso . The adaterature , 29, theAlternative choices for the weighting function could also be considered and would be a topic of interesting further work. In addition one could also consider reducing the shrinkage effect by refitting the solution naively on the selected variables. While this topic has received some attention in penalised linear regression modelling  it has rTo assess the correctness and efficiency of our methods, we run simulations and compare the results with the well known frequentist approaches ASSET and GPA for detecting pleiotropic signal , 13. In K=2 datasets are generated. We look at monitoring the efficiency of our methods for detecting effects across the multiple studies. In particular, we considered the effect of grouping information on the efficiency of the methods. The simulations were set to range from a simulation setting in which the grouping information was not as useful, the groups were almost entirely set to zero, to one where grouping information entirely determined the selected variables, i.e every variable in the group was active. The true effect size for any active variable was set to 0.8, and active variables in study 1 were all set positive, while those in study 2 alternated .At each simulation, pa within groups consisting of 20 variables. This was set to pa=2,4,8 and 16 active variables out of the 20 variables within each group. Each variable occurs in both of the K=2 studies so the total number of effects to be estimated within a group is 40. To offset the effect of having more active variables when pa is larger, we increase the total number of variables in these simulations, considering p=80,160,320 and 640 variables . Under this design, simulations should naturally favour sparse methods such as ASSET or SMT initially and GMT as the group structure becomes more relevant . The simulation settings are given in more detail in Fig. n=100,200,400 and 800, keeping the ratio p/n constant, with half of the observations in each study. The ratio of number of active groups to total number of groups was kept constant across the simulations so that the difficulty of group selection was consistent.We considered four simulation settings where we increased the number of active variables ℓ1 error ℓ2 error K=3 is considered in the supplementary material. This additional simulation was taken as the simulation here for K=2 with an additional third study generated identically to study 1. We found that the performance of our methods only improved for this additional data, see the supplement for further details and results.Once data are generated, our novel methods are applied and compared to the R implementation of ASSET using default parameter settings . We wantnt (MCC)  are compWe used individual-level data from CECILE study , a FrencParticipants from CECILE study were genotyped using a customized microarray including variants from 28 candidate pathways (648 genes) selected from KEGG database and from a literature review. A total of 8,716 SNPs were selected to capture SNPs within 5 kb of each gene (pairwise approach with r2 > 0.8) with a minimum minor allele frequency (MAF) of 0.05 in the Caucasian population (CEU) genotyped by the HapMap Project  . In EPIT2 > 0.98). Then, only SNPs belonging to non overlapping groups (genes and pathways) were selected. At the end of the QC, the two datasets included the same panel of 3,766 SNPs within 331 genes and 10 pathways  has been calibrated using 5 repetitions of 5-fold cross-validation. Once the methods were fit to the data, we explored the stability of the penalised methods using a bootstrap sampling strategy. We evaluated the methods on 10,000 bootstrap samples of the data using the tuning parameters from the original fit to the full dataset. We evaluated the frequency of selected SNPs (or groups for GMT) on the 10,000 bootstrap samples. Finally, we selected only the variables with a higher bootstrap selection rate than the non-selected variables from the original fit to the full dataset. For details see the supplementary material.We applied the three proposed methods to the investigation of pleiotropy between breast and thyroid cancers. The GMT and SGMT methods were both applied twice in order to consider both gene and pathway as different group structures. First, the tuning parameters has been calibrated using 5-fold cross-validation procedures. For SMT and GMT, we then performed the analysis using the values of We also analysed these datasets using ASSET for an empirical comparison of the methods. We first performed GWAS analyses for breast and thyroid cancers separately in order to get summary statistics. As ASSET is based on p-values, we applied a FDR to correct for multiple testing. As we only have interest in identification of pleiotropic effects, we only considered SNPs detected in both datasets.Furthermore, we compared the results of our proposed methods with previously published results on the same data using Bayesian meta-analysis models called GCPBayes at gene-level  which arλ and α values. We defined a variable to be selected (active) only if it was selected in every bootstrapped run.Table This differs from the bootstrapping approach in the application which promotes a higher sensitivity as it is more suitable in genomic context in order to detect more potential signals. We report results using the bootstrapping from the applicaiton in the supplementary material. Our results here gives a more comparable control of the false discovery rate with the ASSET and GPA approaches, and thus a fairer comparison of the methods. Results comparing the methods using the bootstrapping selection approach from the application are given in the supplementary material. Table A consequence of using this strict FDR and bootstrapping procedure is that the true negative rate is almost consistently at 100% with low standard error. The MCC and TPR in comparison have higher variability and differ more amongst the methods. We note also that the variability of the methods appears to decrease for the larger simulations. This is because the smaller simulation settings have a small number of true variables and consequently there is more variance in the estimate of the true positive ratio. For example Sim 1 has only 2 active variables so there are 3 possible TPR values for any dataset.variable-level with SMT having slightly better performance in TPR and MCC. In simulation 4 where group information is most relevant SMT and ASSET suffer in terms of TPR, indicating that the method struggles to detect true effects. Meanwhile GMT has the best performance for simulation 4. However, for simulation 1 and 2, GMT has lower performances in term of MCC compared to SMT and SGMT, although GMT is still good in term of TPR. In simulation 1, 2 and 3, SGMT also shows the best performances in MCC overall. In simulation 4, SGMT is outperformed by GMT in MCC, but still shows better performances than SMT and ASSET. SGMT offers the best compromise overall at variable-level.The simulations results in Table At a group-level, the Multi-Task methods all had significantly better performance than the ASSET approach for TRP, TNR and MCC. Methods which incorporated grouping information were even better yet in their accuracy for recovering the active groups of variables. Moreover, we found that the SGMT method was able to outperform the GMT method for selection at a group-level when there was sparsity within the active groups.Finally, in Table We first run the analyses at SNP-level i.e. that do not take into account for group structure. As a results, no significant SNP was detected by ASSET after correction for multiple testing. Our proposed SMT method selected 11 SNPs from which 8 have been confirmed by the bootstrap sampling strategy. The results of the analyses with the proposed methods are shown in Table RORA  as pleiotropic. This gene located on the chromosome 15 is involved in the regulation of circadian rhythms. RORA was still selected after the bootstrap procedure. However, GMT does not perform variable selection for variables within a group. SGMT selected RORA and a further seven genes. After the bootstrap procedure, only 4 SNPs remained selected, each located in intron of a different tag gene: rs1482057 in RORA, rs1342862 in NEGR1 , rs17332991 in ERCC8 , and rs878156 in PARP2 (poly(ADP-ribose) polymerase 2). These SNPs were also selected by SMT. NEGR1 located in chromosome 1 is an obesity-related gene. PARP2 located in chromosome 14 encodes for a class of nuclear enzymes involved in the pathogenesis of diverse gynecologic tumors [We then performed gene-level analysis using GMT and SGMT. GMT selected the gene c tumors . The freobesity pathway and two single signals in the F _pub_he2010_4 and F _cell_cycle pathways.Analysis with pathway as group structure using GMT did not allow any pathway detection. However, SGMT with pathway as the grouping structure detected 13 consistently selected signals see Fig. , with onRORA and NEGR1. Again, the gene PARP2 appeared to a suggestive threshold as a gene with potential pleiotropic effect which could have been selected with a larger sample size. The fourth gene, ERCC8, was not selected by GCBPayes. Also, GCPBayes selected 3 different SNPs as specific effects in NEGR1 but not rs1342862, that is not in linkage disequilibrium with these 3 SNPs. However, rs1342862 has a D’=1 with rs12133119 and rs17588812, indicating these SNPs share co-inherited alleles. No specific SNP corresponding to RORA was selected by GCPBayes.It can be highlighted that 2 out of 4 selected genes by SGMT were selected by a previous analyse on the same data using GCPBayes Bayesian meta-analysis methods at gene-level : RORA anThe proposed methods can bring power to detect new shared genetic effects between multiple diseases by allowing to simultaneously analyse multiple variables and traits. This allow us to take into account for the correlations between variables and between traits in the analysis, in contrary to methods based on summary statistics from GWAS. Also, our methods allow for incorporating prior knowledge such as group structure corresponding to genes or pathways which can increase the statistical power to identify important risk variants. However, it should be noted that taking into account the group structure requires labeling the data without overlap between the groups, which may require re-partitioning the variables within groups if variables are linked to several groups.A simulation study showed excellent performances of our proposed methods. Even without incorporating grouping knowledge, our SMT method outperformed ASSET in almost all situations. GMT, which only considered variable selection at a group level, showed great performance. This was especially clear when the ratio of nonzero to zero variables within a group was high. We note that the correlation structure of genetic data that is the LD, should help GMT to perform reasonably well even when the ratio of true pleiotropic variables in a group is lower, what is likely in real data. More generally, higher correlation in the data would be in favour of multivariates methods such as our proposed methods. The SGMT method that allow selection at variable and group-level takes the best parts of both SMT and GMT. This method showed the highest performances in almost all simulations and was comparable to the best performance at both variable-level and group-levels.RORA was detected by GMT leading to strong evidence about implication of this gene in the mechanism of both cancers. Interestingly, RORA is part of the core circardian genes and variants in these gene were previously reported to be associated to several cancers, including breast, prostate and pancreatic cancers [The proposed approaches were applied to the investigation of the shared genetic effects between thyroid and breast cancers in candidate pathways. The application study have shown our proposed methods are capable of detecting new signals would not be detected by ASSET. All the multi-task methods were applied to both genes and pathways as group structure. The SGMT method allowed to detect more signals than SMT and GMT methods. SGMT detected 11 pleiotropics SNPs in 6 different pathways, from which 8 SNPs were also detected by SMT. The analysis with genes as group structure highlighted 4 out of these 11 selected variables located in 4 different genes, from which  cancers . This ge cancers .We present three novel feature selection methods at group and variable level adapted for pleiotropy detection in GWAS data using the multi-task regression framework. These methods use penalised likelihood methods, exploiting different penalties, to induce structured sparsity at a group and SNP level. Our methods are developed to model pleiotropic correlation amongst jointly analysed traits and account for the effect of linkage disequilibrium by incorporating known group structures such as gene or pathway. They take into account heterogeneity in the size and direction of the genetic effects across traits. An ADMM algorithm is used to solve the penalised regression problems. We have conducted simulation studies to evaluate the performance of our method compared to one of the most popular method adapted for pleiotropy for practitioners. We have applied our methods to the analysis of two datasets on breast and thyroid cancers.https://github.com/matt-sutton/SGMT.Future work could consider extending these methods to allow for groups with overlap with extensions to the ADMM optimisation or alternative efficient methods . Other e


Given the ever-increasing amount of high-dimensional and complex omics data becoming available, it is increasingly important to discover simple but effective methods of analysis. Divergence analysis transforms each entry of a high-dimensional omics profile into a digitized (binary or ternary) code based on the deviation of the entry from a given baseline population. This is a novel framework that is significantly different from existing omics data analysis methods: it allows digitization of continuous omics data at the univariate or multivariate level, facilitates sample level analysis, and is applicable on many different omics platforms. The divergence package, available on the R platform through the Bioconductor repository collection, provides easy-to-use functions for carrying out this transformation. Here we demonstrate how to use the package with data from the Cancer Genome Atlas. The technologies that provide us with high-dimensional omics data continue to advance at a rapid rate. Particularly in the last decade, the available modalities of omics data have considerably expanded and now include, among others, coding and noncoding RNA expression, micro RNA expression, protein expression, epigenetic profiling related to histones and CpG methylation, copy-number and mutation profiling. As a result, the analysis of multi-modal omics data has become indispensable in many domains of biological and medical research.Whereas the quantity and quality of available data has appreciably increased, the results of research based on such data are often not reliable, robust and replicable. A key challenge has been to quantify the level of variability and diversity in omics profiles in a given population, and to separate normal and technical variability from abnormal variability indicative of a biological property such as disease.Recently we have introduced divergence analysis as a method for simplifying high-dimensional omics data for bioinformatic analyses . This mehttps://www.bioconductor.org/packages/release/bioc/html/divergence.html) which provides functionality for divergence based computations.We have prepared the ‘divergence’ R package  offered Divergence coding can be used for a wide variety of types of data analysis: class comparison and prediction, feature selection, regression, combination of omics modalities, and many others. Here we present how to use this package and showcase the many different ways in which the divergence coding can be manipulated to perform interesting analyses. In the following we use breast normal and tumor samples from the TCGA project  for whicX = (Xj), j = 1..m, m being the dimensionality of the omics profiles. The following transformation is applied which converts the data to a normalized rank, with the minimum being zero.We may summarize the divergence method as follows. Before computing the baseline range (univariate) or support (multivariate) and the resulting divergence coding, a rank-transformation is applied to all data. Consider an omics sample represented by a vector Qj(X) ∈ . The zero minimum is particularly useful in preserving the zero valued mass usually observed in many omics data modalities, such as RNA-Seq.Then S—i.e. a subset of the given m features . We will denote the corresponding subset of a given omics profile X following the quantile transformation as Q(X)S. Suppose we have n such profiles that constitute the baseline group. We estimate the baseline support as follows: given a parameter γ ∈ , we compute l which is the floor of nγ. Then from each baseline sample k, if rk is the distance from Q(X)S to it’s lth nearest neighbor in the multivariate feature space. If we denote the sphere around Q(X)S of radius rk as Now consider a multivariate feature indexed by X, the divergence coding Z(X)S can be computed as:Following the baseline estimation in this manner, for any given omics profile Z(X)j ∈ {−1, 0, 1} with −1 indicating a value below the baseline interval for feature j, 1 indicating a value above the baseline interval, and 0 indicating no divergence.For the univariate scenario, the support is the union of a series of intervals. However, we apply a further simplification by replacing these with a single interval spanning the lowest end to the highest end of these intervals. Accordingly the divergence coding becomes ternary: α and β. The β parameter is used for allowing a certain number of outliers to be exempted from the baseline. Given the radii of the spheres r1..rn, let β)th percentile of these values. Then we select only the spheres with α is the average proportion of divergent features (multivariate or univariate) among the baseline cohort. As discussed in the following section, we specify α and β and then select the γ value that fits these specified parameters. However the functionality for estimating a baseline for a specified γ parameter is available in the package as well.In practice we use two more parameters, For a more detailed description of the method, see .To carry out an analysis based on divergence, the first step is to determine the case and control samples. The control cohort here will be used for computing the baseline interval for each feature, and we will refer to it as the baseline cohort or baseline group. Once the baseline is computed, the divergence values for the case cohort will be computed with reference to the baseline.What data should be used as the baseline cohort depends on the problem at hand and needs careful investigation based on the biological or clinical questions of interest that are being investigated. In many scenarios of disease based data, and in particular cancer, normal samples would be a good choice. The more normal samples available, the more robust the baseline will be.All samples, both in the case cohort and the baseline cohort should have the same features—for example genes or microarray probes, and be from the same platform (e.g. RNASeq or a specific microarray platform). Ideally the baseline cohort should be from the same experiment to avoid study-specific batch effects as much as possible. In general we suggest at least 20 samples be available to compute the baseline.The divergence algorithm applies a scaled rank transformation to all samples before the digitization of the data. The package provides functionality for the user to compute this rank transformation manually, or it can be set to compute internally when the divergence computation is requested by the user. No other normalization procedures are applied, and the user may apply any normalization procedures beforehand as necessary.α, β and γ. All parameters are in the  range. The β parameter adjusts the support to account for a certain percentage of outliers included in the baseline data, and the γ parameter provides a way to widen or tighten the support around each baseline sample. The closer γ is to 1, the further the support around each normal sample will extend. For a given support, the α value is simply the average number of divergent features per sample for the baseline cohort.There are three parameters involved in the divergence computation: α and β values and a range of possible γ values and let the package find the most appropriate γ value out of these for the given α and β values. By default, the package uses α = 0.01, β = 0.95, and γ ∈ {0.01, 0.02, …, 0.09, 0.1, 0.2, …, 0.9} as a list of candidate γ values. Thus if you use these default values, the package will consider 95% of the baseline cohort to be included in the support and find the smallest possible γ value from the given list that will provide the average number of divergent features per sample in the baseline cohort to be 1% or less. For more details, see  range indicating the proportion of methylation observed at a given CpG. As before, we use the methylation data for the TCGA normal samples as a baseline to compute divergence for the tumor samples. The mode of divergence used here is univariate divergence—i.e. we obtain a divergence coding for each sample at each CpG.k clusters; given i = 1..n samples, for cluster rk this iszi,j being the binary divergence coding for CpG j in the cluster and sample i.We can now use this data to look for differentially methylated regions. By arranging the CpG positions along the genome by chromosome and location, we may cluster the CpGs into regions of interest. Here we cluster CpGs in each chromosome into clusters such that that all adjacent CpGs in a cluster are no more than 300 bp apart. Then we simply compute the average sum of divergence observed over the tumor samples in each cluster. Suppose we have Further, we can assign a p-value to each region by computing a null distribution for the clusters through permutation of the CpG labels. The following CpG clusters appear to be the most highly divergent in breast tumor samples .We can compare these results to those obtained via bumphunter , a populHaving computed univariate divergence for two omics modalities—both methylation and gene expression—in this example we combine the two to discover what CpG—gene pairings show significant simultaneous divergence from both modalities. We are provided a table of CpGs mapped to nearby genes such that each CpG is mapped to a gene (with a single gene being mapped to one or more CpGs). We can compute the proportion of samples for which a given CpG—gene pair are both divergent.i (i = 1..n), suppose k and j. Then the proportion of samples for which there is aberrant activity simultaneously in both modalities is an indication of the expression and the methylation between gene j and CpG k being related and co-aberrant. For gene j and CpG k, this is simplyFor a given sample By considering the ternary form of divergence (i.e. −1/0/1), in addition to this proportion we can also compute the ‘concordant’ and ‘discordant’ proportions—i.e. the proportions of samples for which the sign of divergence are also in agreement. Thus samples for which both the gene and CpG are divergent with a + 1, or both with a −1—would be considered concordant, while other samples would be considered discordant.pj,k with the correlation for the gene—CpG pair between the gene expression and methylation values. pj,k values, indicating pairs with extremely positive or negative correlations are more likely to indicate high divergence. Some of these pairs are labeled with the associated gene. In this example, we compute and use co-divergence in the gene expression and methylation modalities in a different way. Using the gene—CpG annotation pairing previously mentioned, we use multivariate divergence to compute divergence for methylation data at the gene level. That is, sets of CpGs that are mapped to a single gene are treated as multivariate features of interest; multivariate divergence is computed for them using a normal methylation baseline. For genes mapped to a single CpG, binarized univariate divergence is computed. This provides us with a gene level divergence computed from the methylation data, which we can now combine with the gene level divergence computed previously with the gene expression data.The two divergence codings for the same genes and samples—one from the methylation data and one from the gene expression data—can now be ‘added’. The binary values are combined to indicate co-divergence in both modalities. Thus for a given sample and gene, a 1 indicates the sample being divergent by gene expression level at that gene, and also being divergent in the CpG methylation space annotated to that gene .The co-divergence probabilities from this resulting data, for any gene considered, is an indicator of how likely the gene is demonstrating aberrant activity in both the transcriptome and the methylome simultaneously. The gene divergence probabilities resulting from this data are similar to the gene—CpG pair proportions computed in the previous example.We now turn to look at relapse free survival (RFS) with this co-divergence data. We select tumor samples that are in stages I or II. This provides 23 samples with recorded relapses, and 260 censored samples. The selection criteria here is reflective of patients for whom existing genomic diagnostic tests such as MammaPrint , 16 are χ2-tests performed to select differentially divergent genes between the two groups. We select the resulting top 20 genes to fit to a cox proportional hazards survival model.To examine genes of interest with respect to relapse, we first select samples that have a censoring time of greater than 3 years among those with no relapse observed. Genes that have a divergence probability ≥0.3 in at least one group (i.e. between the relapsed or censored) are selected, followed by P ≤ 0.05 threshold. P < 10−5). Apolipoprotein D (APOD) is primarily down-divergent in the gene expression data, and low expression of it has been noted to be associated with poor prognosis in breast cancer [Genes are iteratively added to the model fit, keeping only genes that provide a significant coefficient estimate at a t cancer , 18. Regt cancer . BEX1 hat cancer –22. The t cancer , 24. ThiIn the above results we have showcased a variety of analyses that can be performed with divergence at the univariate and multivariate level. While we have used RNA-seq, microarray and methylation 450k data here, the software is applicable to many other modalities of high dimensional omics data, some of which we have showed in . While oOnce the data has been processed as necessary and the baseline cohort identified, the R package can be used to compute the divergence coding quite easily. A full outline of the functions that can be used and the workflows possible are provided in the package vignette . Here wehttps://github.com/wikum/divergenceApplications. (The divergence package is available through https://www.bioconductor.org/packages/release/bioc/html/divergence.html). The code provides an option for downloading the relevant TCGA data through the TCGAbiolinks package [All the analyses conducted in this mansucript are available as R code through the publicly available github repository  package .Divergence provides high utility with respect to omics data analysis. We note that it is a novel framework without any comparable prior methods: the ability to binarize omics data, applicability across many different platforms, enabling sample level scoring and analysis, and being applicable in both univariate and multivariate modes are highly desirable features that are not usually found together in other methods for omics data analysis. Given this fact, we believe that the divergence package will be highly useful to the computational biology research community in their pursuits.


Processed pseudogenes (PΨgs) are disabled gene copies that are transcribed and may affect expression of paralogous genes. Moreover, their insertion in the genome can disrupt the structure or the regulatory region of a gene, affecting its expression level. These events have been identified as occurring mutations during cancer development, thus being able to identify PΨgs and their location will improve their impact on diagnostic testing, not only in cancer but also in inherited disorders.SMAD4-PΨg within the last intron of the SCAI gene in one sample. While for the ubiquitous CBX3-PΨg, present in 82.6% of the samples, we found it reversed inserted in the second intron of the C15ORF57 gene.We have implemented PΨFinder (P-psy-finder), a tool that identifies PΨgs, annotates known ones and predicts their insertion site(s) in the genome. The tool screens alignment files and provides user-friendly summary reports and visualizations. To demonstrate its applicability, we scanned 218 DNA samples from patients screened for hereditary colorectal cancer. We detected 423 PΨgs distributed in 96% of the samples, comprising 7 different parent genes. Among these, we confirmed the well-known insertion site of the PΨFinder is a tool that can automatically identify novel PΨgs from DNA sequencing data and determine their location in the genome with high sensitivity (95.92%). It generates high quality figures and tables that facilitate the interpretation of the results and can guide the experimental validation. PΨFinder is a complementary analysis to any mutational screening in the identification of disease-causing mutations within cancer and other diseases.The online version contains supplementary material available at 10.1186/s12859-022-04583-4. Pseudogenes (Ψgs) are abundant and ubiquitous protein-coding gene copies that are originally derived from functional genes . These hDepending on their mechanism of origin, Ψgs can be classified in three major classes: unitary, unprocessed and processed. Unitary Ψgs are derived from an ancestral protein-coding gene that has lost its protein-coding potential due to spontaneous mutations , 12. WhiPΨgs, are the most abundant type of Ψgs in the human genome with an estimated amount between ~ 8000 and 14,112 –18. As fATP8A2-PΨg has been restricted to breast tumors with luminal histology showing a potential oncogenic nature [PTPN12-PΨg induces the removal of the MGA promoter, a likely tumor suppressor gene [POU5F1B, a PΨg adjacent to MYC, is a prognostic marker [KLKP1-PΨg and KJK4 gene may be a potential biomarker in routine screening [Next-generation sequencing has contributed to the discovery of a large number of PΨgs and further studies have confirmed their involvement in the development, progression and prognosis of certain diseases, including cancer. A comprehensive list of PΨgs participating in the pathogenesis of different diseases has been compiled by Chen et al. . Moreovec nature . In lungsor gene . In gastc marker , while icreening , 26.SMAD4-PΨg is a confounding element in quantitative results and increases erroneous variant calls. Besides, the integration of the SMAD4-PΨg in the SCAI gene has been corroborated in hereditary cancer-predisposition cases [SCAI is characterized to have suppressive effect on tumor cell invasiveness, it has not been determined whether SCAI expression is hindered by the SMAD4-PΨg [Several PΨg integrations have been also identified, however no clear function nor correlation to disease has been yet understood. For instance, the on cases , and whiMAD4-PΨg .A/KS), occurrence of polyadenine tail and the existence of frame disruptions, among others [Ψgs were often discovered as by-product of gene sequencing or PCR experiments. With the advent of whole genome sequencing projects, computational approaches have aided in their identification and annotation, relying on the specific features of the Ψgs, such as level of sequence homology and completeness relative to a parent gene, lack of introns, ratio of non-synonymous to synonymous substitution rates , a bioinformatics pipeline that rapidly screens alignments of DNA sequencing data to detect such events. It creates a simple table that can be sorted and filtered in any spreadsheet program, as well as graphical representations that besides providing a visual confirmation of the candidates, can be used to guide the experimental validation and the characterization of the genomic arrangement of such candidates. In addition, PΨFinder also provides information about known PΨgs found in the analyzed samples and can be used with any organism from whose genome is available.PΨFinder aims to detect PΨgs within DNA sequencing data and predict their insertion sites. PΨfinder is written in python (3.6)  and requThe overall workflow is shown in Fig. To demonstrate the use of PΨFinder, we scanned DNA sequencing data from 218 human blood samples. These samples were initially sequenced and analyzed (data not shown) using a custom-designed panel that included genes associated with hereditary colorectal cancer . The most common PΨgs detected were CBX3-PΨg and BMPR1A-PΨg, found in 180 and 155 samples, respectively. PTEN-PΨg and DHFR-PΨg were detected in 51 and 29 samples, respectively, while 6 samples contained HNRNPC-PΨg. POLE-PΨg and SMAD4-PΨg were found in one sample each.We detected a total of 423 PΨgs distributed across 209 samples. The predictions included PΨgs of only seven parent genes: CBX3-PΨg located within the second intron of C15ORF57 , (2) HNRNPC-PΨg detected in the intergenic region between LINC02541 and MARCKS , (3) BMPR1A-PΨg inserted in the intergenic region between PCGF5 and HECTD2  and (4) SMAD4-PΨg located within intron 18 of SCAI .The detection of the insertion sites of the novel PΨgs, relies on CPs and CRs, and while having both as evidence is not essential (one suffices to narrow down the insertion region), nevertheless, they do strengthen the accuracy of the predicted insertion site  may suggest that it might have an effect in the predisposition to colorectal cancer development. However, in previous studies the CBX3-C15ORF57 fusion was not only found in cancerous tissues, but also in normal or noncancerous samples [From RNA-seq data of lymphoblast tissue, C15ORF57 , 41. ThiC15ORF57  as well C15ORF57 . In thisder Fig. A. The exble Fig. B. The fa samples . AlthougSMAD4 have been shown to result in pancreatic cancer [SMAD4-PΨg has interfered with diagnostic analyses based on clinical sequencing applications, creating false-positives results in 0.24–0.26% of the cases [SMAD4 and SCAI as predicted with PΨFinder   AdditioA. ConsidTo investigate the overall performance of PΨFinder we simulated 117 samples, each with a different PΨg inserted in a random position within the genome : Linux, Mac OS.Programming language: Python (3.6), bash.Other requirements: STAR (2.7.7a), SAMtools (1.11), BEDTools (v2.30.0), R (4.0.3).License: GNU General Public License, version 3.0 (GPLv3).Any restrictions to use by non-academics: None (except the ones stated in GPLv3).Additional file 1: Fig. S1. PΨFinder summary report and visualization aids.Additional file 2. Supplementary methods, experimental validation of SMAD4-SCAI and CBX3-C15ORF57 including primers and gel pictures.Additional file 3: Tables S1–S5. Sequencing data summary statistics and output from PΨFinder of case samples, downsampled data and simulated data. Benchmarking results.