Researchers have previously developed a multitude of methods designed to identify biological pathways associated with specific clinical or experimental conditions of interest, with the aim of facilitating biological interpretation of high-throughput data. Before practically applying such pathway analysis (PA) methods, we must first evaluate their performance and reliability, using datasets where the pathways perturbed by the conditions of interest have been well characterized in advance. However, such ‘ground truths’ (or gold standards) are often unavailable. Furthermore, previous evaluation strategies that have focused on defining ‘true answers’ are unable to systematically and objectively assess PA methods under a wide range of conditions.In this work, we propose a novel strategy for evaluating PA methods independently of any gold standard, either established or assumed. The strategy involves the use of two mutually complementary metrics, recall and discrimination. Recall measures the consistency of the perturbed pathways identified by applying a particular analysis method to an original large dataset and those identified by the same method to a sub-dataset of the original dataset. In contrast, discrimination measures specificity—the degree to which the perturbed pathways identified by a particular method to a dataset from one experiment differ from those identifying by the same method to a dataset from a different experiment. We used these metrics and 24 datasets to evaluate six widely used PA methods. The results highlighted the common challenge in reliably identifying significant pathways from small datasets. Importantly, we confirmed the effectiveness of our proposed dual-metric strategy by showing that previous comparative studies corroborate the performance evaluations of the six methods obtained by our strategy.Unlike any previously proposed strategy for evaluating the performance of PA methods, our dual-metric strategy does not rely on any ground truth, either established or assumed, of the pathways perturbed by a specific clinical or experimental condition. As such, our strategy allows researchers to systematically and objectively evaluate pathway analysis methods by employing any number of datasets for a variety of conditions.The online version of this article (10.1186/s12859-017-1866-7) contains supplementary material, which is available to authorized users. Researchers commonly identify genes that are up- or down-regulated under biological conditions of interest to understand the molecular mechanisms underlying biological processes. However, genes almost always act cooperatively rather than independently. Consequently, methods that analyze the functional contributions of genes should properly take such interactions into account. Biological pathways—the “wiring diagrams” describing the interactions of gene products and other biomolecules as well as their regulatory relationships corresponding to certain biological processes—provide the means to characterize this cooperative nature of gene expression. Pathway analysis (PA) algorithms, combined with large molecular interaction databases, allow for the processing of high-throughput genomic (or proteomic) data, such as those from gene expression microarray or RNA-seq experiments, and the rank ordering of significantly perturbed (up- or down-regulated) pathways associated with an experimental  condition , 2. The Rapid technological advancements during the past decade in our ability to generate and collect high-throughput genomic data have spurred the development of a large number of PA methods. A review article in 2009 reported 68 such methods , and manTo address the lack of a gold standard, researchers have developed numerous strategies, many of which are centered on defining such ‘true answers.’ One approach utilizes simulated gene expression data to create a hypothetical gold standard –11, wherAnother strategy uses well-characterized, experimental gene expression data for a certain condition for which specific pathways  are known to be perturbed , 10, 11.Yet another approach identifies perturbed pathways via the consensus of multiple analysis methods applied to the same dataset, on the assumption that the likelihood of a pathway being a true positive is higher under such a consensus . This strecall and discrimination, each of which reflects a different aspect of what an effective PA method should satisfy without recourse to uncontrolled assumptions. Recall is a measure of consistency: an effective method should provide consistent results when applied to multiple datasets corresponding to the same experimental condition, and show increased consistency with increasing sample size. Discrimination is a measure of the difference in results obtained when a method is applied to datasets that originate from sufficiently different experimental conditions. Because the two metrics are based on comparing or contrasting PA results, calculating recall and discrimination does not require an unverified gold standard. The two metrics are mutually complementary. A trivial or highly biased method that preferentially identifies certain pathways regardless of a change in experimental condition could exhibit high recall, but by definition would also show low discrimination. In contrast, a method that identifies distinct pathways for two conditions regardless of whether they are the same or different would show high discrimination but low recall. A genuinely effective PA method would need to score sufficiently high on both metrics.In light of these drawbacks, an approach to assess the performance of PA methods without reference to unverified gold standards is desirable. In this report, we propose a new strategy that achieves this goal by introducing two metrics, Our strategy for calculating recall is to randomly resample one large dataset consisting of many samples to form a group of smaller sub-datasets that represent the same experimental condition. To calculate discrimination, we use two large datasets from different conditions, resampling each dataset as when calculating recall. The calculation of recall is in accord with the general procedure of biomedical studies, in which a hypothesis is first generated from an experiment by using a few samples, after which it is validated in a replication that involves a large number of samples. If we use a PA method with high recall to formulate a hypothesis, we expect that it will be validated in a subsequent large-scale experiment. In addition, we expect that a highly discriminating PA method is also more likely to extract information that is specific to a particular experimental condition.This resampling procedure allows us to compute recall and discrimination for sub-datasets of different sizes and to account for the effect of sample size, an issue not fully addressed in previous studies. This is important because most publicly available datasets have small sample sizes—a situation that presents an additional challenge for assessing pathway analyses. For example, a search in the Gene Expression Omnibus (GEO) database  revealedTo test the proposed strategy, we implemented the two metrics to assess the performance of six PA methods in identifying significantly perturbed pathways in 10 gene expression datasets. We also used 14 gene expression datasets to further evaluate our strategy. We considered three well-established methods—over-representation analysis (ORA) , gene seWe used our proposed metrics to evaluate PA methods, with respect to a number of gene expression datasets containing multiple samples from two cohorts (treatment and control). Below, we describe the preparation of these gene expression datasets, the pathway database, the PA methods evaluated, the definitions of recall and discrimination, and the computational evaluation of these measures.http://software.broadinstitute.org/gsea/msigdb], in January 2016. This database contained pathway information curated from multiple online databases, including the Kyoto Encyclopedia of Genes and Genomes (KEGG), Reactome, and BioCarta. By limiting the range of gene set sizes to 20–400, we obtained a database of 892 pathways containing 8,791 human genes in total.We downloaded the Molecular Signatures Database (MsigDB) C2 collection of gene sets , providehttp://www.ncbi.nlm.nih.gov/geo/) in February 2016 [studies of Alzheimer’s disease (AD) [15], Parkinson’s disease (PD) [https://www.broadinstitute.org/cmap/) [breast cancer cell line MCF7 and prostate cancer cell line PC3 after drug treatment [We selected publicly available human gene expression microarray datasets obtained under a variety of experimental conditions. Each dataset contained more than 60 samples, was extracted from multiple tissue types , and represented different types of diseases. Of the datasets meeting these criteria, we randomly selected 10 for our study (Table ase (PD) , Crohn’sase (PD) , influenase (PD) , breast ase (PD) , lung caase (PD) , 23, burase (PD) , and traase (PD) , and tworeatment . We normreatment .Table 1TIn addition to the 10 datasets above, we also downloaded 14 gene expression microarray datasets from the GEO repository in August 2017 to further evaluate our strategy  , gene set-test, where the differential expression is considered significant if the t-test P value is less than a cutoff. Subsequently, it counts the number of differentially expressed genes in each pathway and tests for over-representation of these genes in all pathways, using a hypergeometric test. It considers a pathway as significant if the over-representation P value is less than a cutoff.ORA identifies significant pathways in two steps. It first identifies genes that are differentially expressed between two cohorts , as determined by a statistical test such as Student’s GSA, GSEA, and AFC, instead of identifying genes differentially expressed between experimental and control treatments within pathways, first compute a score for each pathway by using the expression levels of all its constituent genes. To calculate this score, GSA uses a maxmean statistic that aggregates the signed scores of individual genes, where the sign and magnitude represent the direction of regulation (up/down) and the differential expression level, respectively, of a gene. For each pathway, it computes a mean positive score and a mean negative score, and selects the score with the larger absolute value as the pathway score. GSEA uses a weighted Kolmogorov-Smirnov test to calculate the pathway score, ranking all genes by their significance and testing the distribution of each pathway’s constituent genes along the rank-ordered gene list. It then assigns a high score to a pathway whose member genes along the list are significantly clustered.P value, defined as the probability that the pathway score for a random dataset is greater than the score from the actual data. We considered a pathway as significant if its P value was less than a designated cutoff . We performed the null hypothesis test by generating randomized datasets, either with permutations of gene labels or sample labels. We evaluated both approaches for GSEA and AFC. In GSA, the two permutation methods were combined into a single re-standardization procedure.In AFC, we calculated the mean fold-change for each gene, that is, the difference between the mean log-transformed gene expression values for samples in the treatment and control cohorts, and defined the pathway score as the average mean fold-change of all genes in the pathway. We then used the pathway scores of gene expression datasets to perform null hypothesis tests and estimated each pathway’s significance by its P-value cutoff. To eliminate variation in the number of significant pathways, we fixed the number for all methods. We used P values to rank-order the pathways for each method, and from each method we selected the top 20 and the top 50 most significantly up-regulated pathways, as well as the top 20 and the top 50 most significantly down-regulated pathways and used them to evaluate the performance of a method.Different PA methods may identify different numbers of significant pathways for the same D contains N samples from two cohorts . D is randomly resampled without replacement to obtain a small sub-dataset d of n samples (n < N). Because d is a subset of D and represents the same experimental condition, a PA method should ideally identify the same set of significant pathways for both datasets. Moreover, because D contains a larger number of samples than d, it should yield the most relevant set of significant pathways. We thus evaluate a PA method by applying it to datasets D and d and identifying two sets of significant pathways A and a, respectively. The recall, r, is then defined asA∩aǁ represents the number of pathways contained in both sets A and a, and ǁAǁ represents the number of pathways in set A. Thus, r represents the fraction of significant pathways identified from the smaller sub-dataset among all significant pathways identified from the original dataset.Figure n on the performance of each of the six methods, we randomly resampled D without replacement to generate 500 sub-datasets of size n  and calculated 500 recall values by using each method for each sub-dataset. The mean and distribution of these recall values were then compared across the six methods for each sample size.To examine the effect of sample size r, as defined by Equation .The recall Equation , could bs can be defined with respect to two datasets D and D', each representing a different experimental condition. From D and D', we create sub-datasets d and d', respectively, as described above. By using a PA method and our procedure, we obtain four sets of significant pathways: A and A' from the original datasets D and D', respectively, and a and a' from the corresponding sub-datasets d and d'. Because D and D' represent different experimental conditions, A and A', as well as a and a', are expected to contain both distinct and shared pathways. The differences between A and A' and between a and a' make it possible to correctly associate a with A and a' with A'. We do this by computing recall and associating a with A if r > r and a' with A' if r > r. Correctly associating a and a' with A and A′, respectively, becomes difficult when fewer distinct pathways are identified. Hence, we define discrimination s, which measures the ability of a method to identify distinct pathways, as the number of sets a and a' that are correctly associated with A and A', respectively, divided by the total number of sets a and a':ai and ai', with i = 1, 2, …, M, are computed for M randomly resampled sub-datasets di and di', respectively.The discrimination metric A and A' or a and a' are nearly identical, discrimination is close to zero. The concept of discrimination can also be extended to more than two datasets, each representing a different condition: r should be greater than any other recall value r, where B is a set of significant pathways for datasets other than D.Discrimination is close to 1 when a method identifies more distinct significant pathways for datasets representing different experimental conditions. Conversely, if sets Although discrimination depends on recall, the two metrics are complementary: discrimination considers only those pathways that are specific to particular experimental conditions and does not directly reflect the overall extent to which all significant pathways are sufficiently represented in each condition. In contrast, recall considers all significant pathways, both specific and non-specific. A trivial method that yields the same result for any dataset would show high recall but low discrimination. Taken together, recall and discrimination can provide a balanced and objective evaluation of PA methods.We computed recall for each of the six PA methods that were applied to each of the 10 gene expression datasets, and evaluated their performance in identifying a fixed number 20 or 50) of significantly up- or down-regulated pathways for resampled sub-datasets of different sizes . Figure 0 or 50 oFigure We computed the discrimination values of significantly up- or down-regulated pathways identified by the six PA methods. Figure We used 9 additional large datasets  and compared the results with those for five large datasets of the same disease  analyzing changes in recall and discrimination for multiple datasets comprising different conditions, 2) comparing our pathway performance evaluation results with previous comparative studies of PA methods, and 3) comparing overlapping significant pathways predicted for a pair of datasets of the same disease.We demonstrated our proposed strategy by assessing the performance of six PA methods. Because there is no verified gold standard to directly assess the exact performance of PA methods, we assessed our strategy by Our performance comparisons for the six PA methods highlighted the common challenge in reliably identifying significant pathways from small datasets. Decreasing the sample size steadily decreased both recall and discrimination. This trend was common for all six methods across all 10 analyzed datasets, with only a few exceptions observed for GSEA (but not GSEAs). Although each method attained different recall values for different datasets  changes in expression. By only counting significant genes, ORA risks neglecting the collective effects of many marginally associated genes. Sample sizes that are inadequate for effective application of Student’s t-test in determining significant genes may further deteriorate the performance of ORA in analyzing small datasets. In contrast, AFC (AFCs), GSEA (GSEAs), and GSA use measurements for all genes that show differential expression in a pathway. Consistent with this view, the recall- and discrimination-based evaluation indicated that ORA performed worse than AFC (and AFCs) in all comparisons, worse than GSEA in six out of eight comparisons of recall and three out of four comparisons of discrimination with respect to small datasets (size 6 and 12), and worse than GSA in seven out of eight comparisons of discrimination. For the remaining comparisons, the performance of ORA was comparable to that of GSA. Comparisons of ORA and GSEA in analyzing large datasets (size 24 and 48) were inconclusive.Neither GSEA nor GSA consistently outperformed ORA in any recall- and discrimination-based comparison. This suggests potential issues with the GSEA and GSA methods. For GSEA, one problem may be the weighted Kolmogorov-Smirnov test, which has been criticized as lacking sensitivity . As for Our strategy clearly showed differences between the two null hypothesis tests employed in AFC and GSEA, sample-label permutation and gene-label permutation, which have been extensively discussed , 31. SamOur comparisons of the proportion of significant pathways that one method predicted for a small dataset and those it predicted for a large dataset of the same disease showed that for some diseases , the ranks of the six PA methods ordered by this proportion were nearly identical to those ordered by recall  in the original dataset needs to be preserved in the sub-datasets. In general, this is achieved by generating a large enough number of re-sampled sub-datasets. In this case, the majority of the sub-datasets will then retain the same distribution of covariates as the original dataset. However, this may not be achieved when the size of a sub-dataset is too small or the incidence of a covariate in the original dataset is extremely low.We used microarray data in our demonstration because, while enormous amounts of such data have been accumulated, the data are considered noisy and thus present challenges for PA methods to produce meaningful results. We emphasize that our strategy can also use any other bio-molecular data suitable for pathway analysis  to evaluate PA methods.In summary, we have demonstrated a newly developed dual-metric strategy that evaluates the performance of PA methods. Our strategy allows us to employ any number of datasets for various conditions because we need not know the truly perturbed pathways associated with each condition. In addition, it is applicable not only to PA methods that do not take into account interactions between genes, but also to more advanced methods that consider such topological information .Additional file 1: Table S1.The 14 additional gene expression datasets used to further assess the performance of pathway analysis methods. (DOCX 41 kb)Additional file 2: Figure S1–S22.Additional comparison of recall values computed for the six pathway analysis methods using different datasets. (PPTX 6592 kb)


Understanding the aspects of the cell functionality that account for disease or drug action mechanisms is a main challenge for precision medicine. Here we propose a new method that models cell signaling using biological knowledge on signal transduction. The method recodes individual gene expression values (and/or gene mutations) into accurate measurements of changes in the activity of signaling circuits, which ultimately constitute high-throughput estimations of cell functionalities caused by gene activity within the pathway. Moreover, such estimations can be obtained either at cohort-level, in case/control comparisons, or personalized for individual patients. The accuracy of the method is demonstrated in an extensive analysis involving 5640 patients from 12 different cancer types. Circuit activity measurements not only have a high diagnostic value but also can be related to relevant disease outcomes such as survival, and can be used to assess therapeutic interventions. Despite most phenotypic traits (including disease and drug response) are multi-genic, the vast majority of biomarkers in use are based on unique gene alterations  Obviously, the determination of the status of a single gene is technically easier than multiple gene measurements. However, regardless of their extensive clinical utility, single gene biomarkers frequently lack any mechanistic link to the fundamental cellular processes responsible for disease progression or therapeutic response. Such processes are better understood as pathological alterations in the normal operation of functional modules caused by different combinations of gene perturbations (mutations or gene expression changes) rather than by alterations of a unique gene .c-Jun N-terminal kinase pathway, shows significantly higher association to neuroblastoma patients’ mortality than the activity of their constituent genes  [Of particular interest are signaling pathways, a type of functional module known to play a key role in cancer origin and progression, as well as in other diseases. Consequently, analysis of the activity of signaling pathways should provide a more informative insight of cellular function. Actually, the recent demonstration that the inferred activity of the omarker)  constituomarker) .a posteriori, usually by heuristic methods. Thus, pathway activity analysis (PAA) emerges as an alternative way of defining a new class of mechanistic biomarkers, whose activity is related to the molecular mechanisms that account for disease progression or drug response. However, capturing the aspects of the activity of the pathway that are really related to cell functionality is not trivial. This requires of an appropriate description of the elementary sub-pathways and an adequate computation of the individual contributions of gene activities to the actual activity of the sub-pathway. Different ways of computing activity scores for diverse sub-pathway definitions using gene expression values [However, conventional methods for pathway analysis, even the most sophisticated ones based on pathway topology, can only detect the existence of a significant level of gene activity within the pathway . Howevern values –8, or evn values , have beHere we propose a new method to estimate the activity within a pathway that uses biological knowledge on cell signaling to recode individual gene expression values (and/or gene mutations) into measurements that ultimately account for cell functionalities caused by the activity of the pathway. Specifically, we estimate the level of activity of stimulus-response sub-pathways  within signaling pathways, which ultimately trigger cell responses  The activity values of these canonical circuits connected to the activation/deactivation of cell functionalities can be considered multigenic mechanistic biomarkers that can easily be related to phenotypes and provide direct clues to understand disease mechanisms and drug mechanisms of action (MoA). Therefore, we designate this method as canonical circuit activity analysis (CCAA).https://tcga-data.nci.nih.gov/tcga/). In order to detect possible batch effects, principal component analysis (PCA) were calculated. The samples were plotted in the PCA representation by sequencing center, plate, cancer type and project. Only a clear batch effect by sequencing center and cancer was found  and the ratio of false positives was always very low, far below the conventional alpha value of 0.05 see .In order to obtain an estimation the true positive rate of the CCAA method, we compared cancer samples versus the corresponding healthy tissue in a series of contrasts with different sizes  from which we expect differences in cancer-associated pathways. Two different cancer types, KIRC and BRCA, were used to avoid biases derived from using only a specific type of cancer. We have used two definitions of cancer associated pathways, one of them taken from KEGG   data wasFirstly, 526 cancer samples were compared against the 72 available controls of normal kidney tissue adjacent to the primary tumors  and DNA replication , whose upregulation is significantly associated to bad prognostic .Another cancer hallmark is the activation of metastasis and invasion, favored when the Uniprot function Angiogenesis appears as significantly associated to survival in both Uniprot and GO annotations . Actually, the downregulation of the opposite term, Negative regulation of angiogenesis, is also associated to bad prognostic, as expected, although with marginal significance .The third classical cancer hallmark in solid tumors is the induction of angiogenesis. otations  and S7. gluconeogenesis , Sodium transport  and, the general term Transport   see .c-Jun N-terminal kinase pathway as a superior predictor of prognostic in neuroblastoma [Moreover, in some specific circuits triggering cancer hallmarks the association of the activity of the circuit to the mortality of the patient resulted to be higher than the individual association of any of the genes that form the circuit. Table blastoma  can be gDNA replication is an example of function that can easily be mapped to the sustained proliferative signaling cancer hallmark [−8). Three effector circuits belonging to the Cell cycle and the p53 pathways  to explain how their perturbations ultimately cause diseases or responses to treatments. Therefore, such perturbations  can be related to disease mechanisms or drug MoAs , 28.A unique feature of the CCAA method is that, if the analysis is made at the level of cell functionality, the changes in the activity detected can be traced back to the circuits in order to discover which ones are triggering the action and what genes are the ultimate causative agents of such functional activity changes. Therefore, the resulting models can be used to suggest and predict the effect of interventions  on specific genes in the circuits so as to find suitable clinical targets, predict side effects, speculate off-target activities, etc. Depending on the scenario studied, such interventions can be more general or more personalized.Another relevant feature missing in the rest of PAA methods Table  is the pSince clinical data are available at the TCGA repository, we were able to find significant associations of specific pathway activities to patient survival, proving thus the validity of PAA methodology to capture cell processes involved in disease outcome.Finally, it is worth mentioning that the integration of information on protein functionality in the model, if it is available, is straightforward. (See Methods for details). Other omic data  could also be easily introduced in the model providing they could be coded as proxies of presence and/or integrity of the protein.https://tcga-data.nci.nih.gov/tcga/) in which RNA-seq counts for healthy control samples were available in addition to the cancer samples: Bladder Urothelial Carcinoma (BLCA) [We used 12 cancer types from The Cancer Genome Atlas (TCGA) data portal (a (BLCA) , Breast a (BLCA) , Colon aa (BLCA) , Head ana (BLCA) , Kidney a (BLCA) , Kidney a (BLCA) , Liver ha (BLCA) , Lung sqa (BLCA) , Prostata (BLCA) , Thyroida (BLCA)  and Utera (BLCA)   associated to Genome Characterization Center (GCC) and plate ID must be removed from the RNA-seq data. The COMBAT method  was usedModelling of pathway activity requires initially of a formal description of the relationships between proteins within the pathway, which can be taken from different pathway repositories. Here KEGG pathways  are usedA total of 60 KEGG pathways  proteomic and chemoproteomic experiments , howeverPathways are represented by directed graphs, which connect input (receptor) nodes to output (effector) nodes. The signal arrives to an initial input node and is transmitted along the pathway following the direction of the interactions until it reaches an output node that triggers an action within the cell. Thus, from different input nodes the signal may follow different routes along the pathway to reach different output nodes. Within this modelling context, a canonical circuit is defined as any possible route the signal can traverse to be transmitted from a particular input to a specific output node  node , arriving to the current node from activation edges, I is the total number of inhibitory signals (is) arriving to the node from inhibition edges.Where The algorithm to compute the transmission of the signal along the network is a recursive method based on the Dijkstra algorithm . Each tiEffector nodes at the end of the circuits trigger specific functions in the cell. These functions are defined here based on the annotations of the proteins contained in the effector node. Gene Ontology  (GO) terThe signal intensity received by the effector node can be propagated to the functions triggered by them following the same rationale of signal propagation along the circuits. Figure per se, because it is considered to produce a non-functional protein. The HiPathia program enables the analysis of mutations found in standard variant files (VCF) from whole exome/genome sequencing experiments in combination with gene expression values.Finally, the integration of genomic and transcriptomic data in the proposed modeling framework of signaling pathways is straightforward. In order to transmit the signal a protein needs to be present (gene expressed) and to be functional (harboring no impairing mutations). Genomic data can be integrated with transcriptomic data to infer combined gene activity and integrity . In the simplest approach  the normg, an empirical distribution of gene expression values was derived from the patients of the KIRC dataset. Specifically, the mean μg and variance σ2g was inferred for each gene g taking into account the gene expression values measured for these gene in all the samples. Then, N individuals were generated by simulating the gene expression values for each gene g as random numbers sampled from a normal distribution N; iii) N individuals were generated by simulating their gene expression values as random numbers from a normal distribution N as above.To estimate the false positive rate, different groups of N identical individuals were generated and further divided into two datasets that were compared to each other for finding differentially activated circuits. This comparison was repeated 2000 times for different data sizes  in three different scenarios: i) N individuals were randomly sampled among KIRC patients; ii) For each gene Since the individuals involved in the comparison were taken either from the same type of samples or were generated in the same way, any differential activation found can be considered a false positive. The comparisons were carried out for both, circuits and effector proteins.To estimate the true positive rate, we tested a scenario in which biological differences are expected. For this purpose, we used the two 2 cancers in Table We compared the reliability of the CCAA method proposed here to other proposals for defining sub-pathways and for calculating an activity score for them. Among the methods listed in Table survdiff from the survival R package  for each signaling circuit, each effector circuit and each cell function (either Uniprot or GO definitions) with a significant difference of activity when cancers were compared to the corresponding controls. Specifically, the 10% of individuals presenting the highest (or lowest) activity were compared to the rest of them.The KIRC TCGA samples contain survival information among the clinical data available. These can be used to check whether the circuit or function activities estimated for each patient have a relationship with survival or not. Kaplan-Meier (K-M) curves  were esthttp://hipathia.babelomics.org.A user-friendly web server that runs the code for carrying out the CCAA method is freely available at https://github.com/babelomics/hipathia.The R code implementing the method is available at This work is supported by grants BIO2014-57291-R from the Spanish Ministry of Economy and Competitiveness and “Plataforma de Recursos Biomoleculares y Bioinformáticos” PT13/0001/0007 from the ISCIII, both co-funded with European Regional Development Funds (ERDF); PROMETEOII/2014/025 from the Generalitat Valenciana (GVA-FEDER); Fundació la Marató TV3 (ref. 20133134); and EU H2020-INFRADEV-1-2015-1 ELIXIR-EXCELERATE (ref. 676559) and EU FP7-People ITN Marie Curie Project (ref 316861).


Breast cancer is the second largest cause of cancer death among U.S. women and the leading cause of cancer death among women worldwide. Genome-wide association studies (GWAS) have identified several genetic variants associated with susceptibility to breast cancer, but these still explain less than half of the estimated genetic contribution to the disease. Combinations of variants (i.e. genetic interactions) may play an important role in breast cancer susceptibility. However, due to a lack of statistical power, the current tests for genetic interactions from GWAS data mainly leverage prior knowledge to focus on small sets of genes or SNPs that are known to have an association with breast cancer. Thus, many genetic interactions, particularly among novel variants, remain understudied. Reverse-genetic interaction screens in model organisms have shown that genetic interactions frequently cluster into highly structured motifs, where members of the same pathway share similar patterns of genetic interactions. Based on this key observation, we recently developed a method called BridGE to search for such structured motifs in genetic networks derived from GWAS studies and identify pathway-level genetic interactions in human populations. We applied BridGE to six independent breast cancer cohorts and identified significant pathway-level interactions in five cohorts. Joint analysis across all five cohorts revealed a high confidence consensus set of genetic interactions with support in multiple cohorts. The discovered interactions implicated the glutathione conjugation, vitamin D receptor, purine metabolism, mitotic prometaphase, and steroid hormone biosynthesis pathways as major modifiers of breast cancer risk. Notably, while many of the pathways identified by BridGE show clear relevance to breast cancer, variants in these pathways had not been previously discovered by traditional single variant association tests, or single pathway enrichment analysis that does not consider SNP-SNP interactions. Susceptibility to breast cancer is partially encoded in our genomes, but despite the development of new genomic technologies over the past decade, we are still not able to accurately predict disease susceptibility from genome sequences. One reason for this gap is that we lack methods for finding combinations of genome variants that lead to disease. Extensive studies in model organisms have experimentally constructed millions of double mutants to study genetic interactions and have defined the basic principles by which genes combine to cause phenotypes in an organism. One powerful outcome of these studies in model systems is that genetic interactions frequently form highly organized patterns that can be used as a basis for improved detection of them in humans. We developed a novel computational approach based on this principle for identifying pathway-level interactions that contribute to breast cancer disease risk. Applying this method to six different groups of breast cancer patients, we identified a core set of pathways, including glutathione conjugation, vitamin D receptor, purine metabolism, mitotic prometaphase, and steroid hormone biosynthesis. These pathways are well-supported across multiple cohorts and may contribute to breast cancer susceptibility. Cancer, like many common diseases, is influenced by a variety of genetic and environmental factors. With the rise of inexpensive genotyping technologies, the portion of risk due to inherited genetic variants has been measured with unprecedented resolution. A recent comprehensive study reported excess familial risk for 20 of 23 cancer types with an overall heritability estimate of 33% . This vaAs with other heritable diseases, there has been substantial interest in identifying specific genetic loci that increase or decrease an individual’s risk for specific cancers. Over the past decade, genome-wide association studies have been the primary strategy for discovering such loci, and indeed, have been successful at identifying a large number of single-nucleotide polymorphisms (SNPs) with statistically significant association to a variety of diseases including cancer –7. Howev11 for 500k SNPs) is a daunting computational task. While there have been efficient and scalable computational tools developed for this purpose .–42.38–42global, χ2local and pperm for BPM and WPM)  data . The detWPM) See  23]. Br. BrglobaWe first applied our BridGE approach to the BPC3 and CGEMS cohorts . These cohorts are both comprised of European Americans with genotypes measured using a common array platform  (−4 ≤ p ≤ 0.05) with breast cancer incidence , respectively, none of them would reach a standard level of genome-wide significance (p ≤ 10−8) suggesting that accounting for interaction between combinations of different common variants may contribute significantly to breast cancer heritability.Focusing first on identifying between-pathway model (BPM) interactions, we applied BridGE to the BPC3 cohort . At a fa × 10−5) . This BP × 10−5) . FurtherThe steroid hormone biosynthesis pathway represented a major hub among our discoveries for the BPC3 cohort, appearing in 8 of the 11 discovered BPMs . The step ≤ 0.05) by all three measures of significance. Another two BPMs, both involving interaction of the steroid hormone biosynthesis pathway with either Na+/Cl- dependent neurotransmitter transporters or Amine compound SLC transporters, were also nominally significant by 2 of the 3 measures in the CGEMS cohort (p = 0.0043).We next examined whether significant between-pathway interactions discovered in the BPC3 cohort replicated in the CGEMS cohort . A recent study showed that amino acid transporters (e.g. SLC6A14) were upregulated in tumors of epithelial origin, including breast cancer, and suggested them as a possible new target for cancer treatment . AnotherIn addition to between-pathway interactions, the BridGE approach can also be used to identify single pathways that are enriched for SNP-SNP interactions mapping to multiple genes within the same pathway  or pathways that are enriched for SNP-SNP interactions across the entire genome . Applying BridGE to the BPC3 and CGEMS cohorts, we identified 2 WPM and 3 PATH interactions from the BPC3 cohort and 2 WPM and 4 PATH interactions on the CGEMS cohorts  and SBCGS (CHN) . The JPNIndeed, we were able to find genetic interactions in three of the four additional cohorts, although the number of interactions identified varied across cohorts as did the corresponding model  . Notablyp ≤ 1 × 10−5, see p = 0.02, Most of the significant genetic interactions discovered across the five cohorts were unique to each cohort, suggesting that the strongest genetic interactions are distinct in each population and may reflect the broad set of ethnicities represented by these cohorts. However, we reasoned that there may also be common genetic interactions underlying breast cancer risk across diverse populations, and that if we performed joint discovery across these diverse cohorts, we may be able to detect such universal risk factors. We anticipated that pathway-level interactions with moderate significance in individual cohorts that were consistently identified across multiple populations would be highly significant when analyzed together. Applying this principle, we extended our BridGE approach to enable joint analysis of between-, within-, and hub-pathway interactions across multiple cohorts. Significance of BPMs, WPMs, or PATH interactions with support across multiple datasets was assessed through resampling of the pathway-level statistics from 10 sample permutations (case-control label permutations) we ran for all five cohorts see . For exaWe visualized the complete set of BPM interactions as a network to explore the relationship between the discovered interactions . SeveralGlutathione S-Transferases (GSTs) comprise a large and conserved family of enzymes that catalyze conjugation of reduced glutathione (GSH) to a variety of substrates . GST-medBased on their broad enzymatic and non-enzymatic functions, GSTs have been identified as important targets for anti-inflammatory and anti-tumor drug therapies . Indeed,FDR ≤ 0.25 (Given the discovery of the glutathione conjugation pathway as a pathway interaction hub (PATH) from our consensus analysis, we performed a full analysis of the three relevant cohorts to focus on discovering significant between-pathway interactions that specifically involved the glutathione conjugation pathway. By focusing on just this pathway, we further reduced the number of hypothesis tests to improve our power to discover specific pathways interacting with glutathione conjugation. This approach was successful for two of the three cohorts  and produced 17 and 77 interactions at R ≤ 0.25 , respectStrikingly, 3 of these BPMs were independently discovered in both cohorts: regulation of PGC-1a, toll-like receptor 9 cascade and response to E. coli infection . One of 2 (15d-PGJ2), a potent cyclopentanone [2 biosynthesis requires GST [2 activity by directly binding to both GSH-conjugated and unconjugated forms of 15d-PGJ2 and sequestering the ligand in the cytosol away from its nuclear target, PPARγ [Importantly, a mechanistic link between the PGC1a pathway and the glutathione conjugation pathway is well established, supporting the interactions we discovered and suggesting increased breast cancer risk in patients carrying variants in both of these pathways . Specifientanone . 15d-PGJires GST  and, in t, PPARγ , 81. Indt, PPARγ . Other st, PPARγ , 80, 82.Another interaction involving glutathione conjugation included the Toll-like receptor 9 (TLR9) pathway . TLR9 isWe described the application of our recently developed method, BridGE, to several breast cancer cohorts. We found significant discoveries across 5 of the breast cancer cohorts examined, suggesting that genetic interactions indeed play a role in determining breast cancer risk. Our approach leverages the key observation from reverse genetic screens in yeast, which observed that genetic interactions often form dense clusters in which they bridge across two pathways, or connect pairs of genes within the same pathway. This observation about local structure prevalent in the yeast genetic interaction network provides a powerful basis for discovering interactions in human populations. Our results here demonstrate that this approach can shed new light on risk factors for breast cancer.−5) from published GWAS. Mapping these variants to nearby genes and then to pathways reveals that only 9 of the 25 pathways involved in the genetic interactions reported here include a gene for which a SNP had previously been reported, suggesting that the remaining 16 of 25 have not been previously implicated through germline genetic analysis. Thus, despite their clear relevance to breast cancer, the majority of genetic interactions reported here represent novel mechanisms underlying genetic risk of breast cancer relative to previous studies of single variants. Interestingly, the BridGE-discovered pathways also cover many of the previously reported SNPs. Of the 172 unique GWAS SNPs mentioned above, 47 can be mapped to our collection of 833 pathways and 34 of these 47 (72%) map to at least one of the set of BridGE-discovered pathways (FDR ≤ 0.25), suggesting that most pathways linked to previously identified SNPs from single locus analysis are also involved in genetic interactions.We note that many of the pathways involved in genetic interactions reported here are novel and have never been implicated as genetic risk factors for cancer. For example, if we consider only BPM, WPM or PATH interactions passing a conservative cutoff of FDR < 0.05, we discovered a total of 25 pathways across the five cohorts . Based oThere were a large number of pathway-level interactions unique to individual cohorts we examined, suggesting that interactions can be contributed by a broad range of mechanisms and likely vary substantially across different human populations. We did, however, find evidence for a core set of interactions with support across multiple populations. Specifically, significant interactions involving glutathione conjugation, vitamin D receptor, purine metabolism, mitotic prometaphase, and steroid hormone biosynthesis pathways were discovered across different cohorts, suggesting these pathways may act as important general modifiers of breast cancer.There are several other interesting directions for future work based on the results presented here. First, one of the main inputs of the BridGE method is the definition of pathways, for which it then discovers genetic interactions. Of course, the quality and utility of the genetic interactions discovered depend on the quality of the input pathway definitions. We expect that there are several pathways highly relevant to breast cancer that are not yet well-understood or at least not well-captured by current pathway databases. As these pathway definitions improve, the BridGE approach will improve in terms of its power in discovering interactions. In the context of breast cancer, there is a wealth of functional genomic data (e.g. gene expression profiles) that could directly inform the definition and further refinement of pathways. Leveraging these unbiased data to improve the input pathways before running BridGE would be worthwhile.Another limitation of the BridGE approach is the resolution of the discovered interactions. The genetic interactions reported in this study were all discovered at the pathway level (i.e. between or within-pathways). The premise of the method, and indeed the reason we are even able to discover genetic interactions, is that while power to detect individual pairs of SNPs with disease association is low, these associations can be discovered at the pathway level. Because of this, it is typically difficult to pinpoint individual SNPs or combinations of SNPs for further investigation. For example, for the steroid hormone biosynthesis-AML gene set interaction, the BPM was discovered on the BPC3 cohor and replicated on the CGEMS cohort. However, the overlap in the individual SNP-SNP interactions supporting these BPMs in the different cohorts was relatively small. This likely reflects both the fact that our power for detecting the actual SNP-SNP interactions underlying the association is limited as well as the fact that the actual SNPs contributing interactions between these pathways can be highly heterogeneous. Our analysis of the glutathione conjugation pathway discovery provides some hints at how to approach this challenge. Once BridGE identified the glutathione conjugation pathway as a risk factor in several cohorts, we computed the density of SNP-SNP interactions connecting each gene in the pathway. This did highlight substantial differences in the SNP-SNP interaction density contributed by each gene, providing some clues as to which individual SNPs have the strongest contributions to the pathway-level trend . ConsensFinally, another direction worth further investigation is analysis of the clinical relevance of the discovered interactions. We expect that, at least in some cases, the genetic interactions predisposing individuals to breast cancer will influence the prevention, progression, and optimal treatment of the disease. Application of our method to large cohorts with the corresponding clinical information and development of predictive modeling approaches that leverage both pathway-level and SNP-level information from the discovered genetic interactions to model clinical features will be a focus of future work.The National Cancer Institute provided seven breast cancer GWAS datasets for the U4C Stimulating Innovation in Breast Cancer Genetic Epidemiology Challenge: phs000812.v1.p1, phs000147.v3.p1, phs000517.v3.p1, phs000799.v1.p1, phs000851.v1.p1, phs000912.v1.p1, phs000383.v1.p1. Among them, we focused our analysis on four datasets: phs000147.v3.p1, phs000812.v1.p1, phs000517.v3.p1, and phs000799.v1.p1. A brief summary of these four datasets can be found in We used Hapmap Phase III data  as our pThe details of the BridGE method are described in our separate paper , but we Between-pathway model (BPM): Between-pathway interactions occur when two pathways impinge on a common function required to maintain a healthy (non-disease) state; because the two pathways can functionally compensate for each other, the disease phenotype only occurs when genetic perturbations occur in both pathways in the same individual.Within-pathway model (WPM): Under the within-pathway model, a single genetic variant partially disables a pathway’s function and additional partial loss of function variants affecting the same pathway result in a complete loss of pathway function, leading to a disease state.Hub pathway model (PATH): Pathway hubs correspond to pathways with frequent modifier effects where the target loci are not necessarily functionally coherent as under the between-pathway model.Briefly, the BridGE approach involves the following five main components :global), the marginal interaction density of the two pathways , as well as a permutation test (pperm) conducted by randomly shuffling the SNP-pathway assignment (e.g. 100K~200K times). (5) Pathway-level statistics are assessed for significance after correction for multiple hypothesis testing. Each pathway-level interaction can be associated with either increased risk of disease  or decreased risk of disease . So, for example, for the between-pathway model (BPM) the number of hypothesis tests evaluated by BridGE is two times the number of all pair-wise pathway-pathway interactions. A sample permutation strategy (e.g. permutation of the case-control labels 10 times) is used to estimate the false discovery rate accounting for multiple hypotheses testing. Further details of our methods are described in [(1) Data processing consisting of sample quality control, adjustment for population structure between the cases and controls to avoid false discoveries due to population stratification, and control for linkage disequilibrium (LD) by pruning the full set of SNPs into an unlinked subset, as LD could otherwise result in spurious BPM or WPM substructures. (2) Construction of SNP-SNP interaction networks based on SNP pair-level genetic interactions scored under different disease model assumptions . (3) A low-confidence, high-coverage interaction network is derived by applying a lenient threshold to the SNP-SNP interaction network. (4) Pairs of pathways from predefined gene sets are tested for BPM or WPM enrichment of SNP-SNP pair interactions with a chi-squared test. The observed density is evaluated for significance based on comparisons to the global density  we used a standard PLINK  procedure to remove individuals with more than 5% missing values, and remove SNPs with more than 5% missing values, less than 5% minor allele frequency, or failed Hardy-Weinberg equilibrium test at 1.0E-6; (2) we checked relatedness among individuals, and for any pair of individuals that had a proportion IBD score greater than 0.2, one of them was removed from the study; (3) we removed subjects that were identified as population outliers based on multidimensional scaling (MDS) analysis after combining the study data with HapMap phase III data ; (4) we For the two breast cancer European cohorts, BPC3 (phs000812) and CGEMS (phs000147), since there is a partial overlap in cases and controls between these two cohorts, we first applied identity by descent (IBD) analysis to recognize these overlapping individuals and removed them from the BPC3 (phs000812) cohort, while keeping them in the CGEMS cohort (phs000147).The phs000517 dataset has three population groups: JPN, LTN and AA. We used MDS analysis as described above to split this cohort into three sub-cohorts, one for each ethnic group. Samples that did not cluster within each group based on MDS analysis were filtered from the cohort. These three sub-cohorts are also labeled as JPN517, LTN517 and AA517 respectively.To allow for direct comparisons with results from the phs000517 cohort, we used imputed SNPs from the phs000799 cohort, which was genotyped using the Affymetrix 6.0 platform, to match the SNPs measured on the Human660W-Quad_v1_A and Illumina Human 1M platforms, which were used to genotype the phs000517 cohort. We ran our BridGE approach on both the non-imputed dataset and imputed dataset and reported discoveries for both. These imputed and non-imputed versions of the phs000799 dataset are referred to as “CHN799 imputed” and “CHN799 non-imputed”.We applied the BridGE method to six different cohorts derived from four GWAS breast cancer studies . Specifically, we tested pathway-level interactions  for 6 different cohorts , and for the CHN799 cohort, we used the both imputed and non-imputed data, independently. Details of the procedure used are described in the sections that follow.For each dataset, we first ran pilot runs to find a proper set of parameters to be used for a full BridGE run. Specifically, we tested the four disease models  with different network thresholds by performing a small number of SNP permutations  (SNP-pathway assignment was randomly permuted), and estimated which combination of disease model and network density cutoff was the most sensitive for each dataset. Based on the pilot results, a recessive/dominant combined disease model was chosen for BPC3 (network density = 0.06), SBCGS (network density = 0.04) for both imputed and non-imputed version; a dominant model was chosen for CGEMS (network density = 0.04), MCS JPN cohort (network density = 0.04), and MCS LTN cohort (network density = 0.02). For the MCS AA cohort, the pilot run suggested that we were unlikely to discover pathway-level interactions, so we did not apply a full BridGE run on this dataset in order to focus our computational resources on analysis of other cohorts. For all BridGE runs, we used supercomputing resources provided by the Minnesota Supercomputing Institute.global and χ2local are chi-square tests to measure whether the observed SNP-SNP interaction density between two pathways, or within a pathway, is significantly higher than expected globally , and locally . Additionally, a permutation test in which SNP labels are randomly reassigned is used to derive a third measure of significance (pperm). These permutations are used to establish a null distribution for χ2global and χ2local for each between-/within- pathway interaction. Finally, a false discovery rate is estimated for the entire set of between- or within-pathway interactions based on sample permutations in which the entire process is repeated under permutations of the case-control labels  [As described in , for thed pperm) .For the hub pathway interactions (PATH), a one-tailed rank-sum test was used to test if the SNPs linked to each pathway show significantly more interactions than non-pathway SNPs, in terms of interaction degree. The sample permutation and SNP label permutation procedure is same as the between- and within-pathway interaction discovery.We reported all significant BPM, WPM or PATH interactions with FDR≤0.25. The summary table  shows thFor significant pathway-level interactions identified from any of the two European cohorts , we performed replication analysis. The disease models used for full interaction discovery in the BPC3 and CGEMS were different based on trends observed in the pilot runs (see details above), so for the replication analysis, we used the disease model with the discovery cohort and ran BridGE with 1000 SNP label permutations for all candidate pathway-level interaction. Ten sample permutations were also run for the replication cohort, just as in the discovery cohort.global, χ2local and pperm) on the replication cohort and tested whether they met a nominal significance criteria (p ≤ 0.05). Of the discoveries from the BPC3 cohort, one of the 18 significant pathway-pathway interactions, the BPM connecting the steroid hormone biosynthesis pathway to an AML gene set, was significant (p ≤ 0.05) by all three measures. Two additional BPMs were significant (p ≤ 0.05) by two of the three significance measures (amine compound SLC transporter gene set and Na(+)- and Cl(-)-dependent neurotransmitter transporters) when tested for replication in the CGEMS cohort , we measured all three significance scores  with breast cancer susceptibility. To measure the extent to which our approach produced new pathway-level insights about breast cancer susceptibility, we evaluated how many pathways in our collection were implicated basted on these 172 risk loci, and how many pathways discovered by BridGE analysis were novel relative to this set derived from traditional GWAS single variant analysis.Based on the NHGRI-EBI GWAS catalog, there aFDR ≤ 0.05), which yielded a total of 25 unique pathways either from significant BPM, WPM or PATH discoveries. Among these pathways, 9 were in common with the pathways already implicated by at least one known breast cancer risk locus. Thus, our analysis of genetic interactions by BridGE has implicated 16 new pathways (FDR ≤ 0.05) as playing a role in breast cancer susceptibility. We listed all unique pathways resulted from the less stringent FDR cutoff (FDR ≤ 0.25) in Of 172 SNPs linked to known breast cancer risk loci, 47 of these SNPs could be mapped to our collection of 833 pathways. Then we collected all pathways that were identified by BridGE in any of the breast cancer cohorts analyzed here with a conservative FDR cutoff  across all cohorts. We then selected pathway-level interactions that were nominally supported by at least two cohorts, for which we required that all test scores  be nominally significant (p ≤ 0.05). For each surviving pathway-level interaction, we computed the geometric mean of the p-values of all individual cohorts that met the nominal significance requirement. These criteria produced a total of 3930 consensus between-pathway interactions (BPM), 76 within-pathway interactions (WPM), and 59 hub pathway interactions (PATH), which were sorted based on the aggregate p-value (see −5), within-pathway interactions (geometric mean p ≤ 5.0 × 10−3), and hub pathway interactions (geometric mean p ≤ 5.0 × 10−3) are visualized in More specifically, we first ran the standard version of BridGE on 4 different GWAS datasets , and we summarized each pathway-level interaction based on its permutation p-values  and counted how many of interactions from the real consensus table met the cutoff relative to the permuted results to derived an empirical p-value for the BPM, WPM and PATH consensus observations independently. For BPMs, our analysis suggested the real consensus results were significantly larger than expected at the chosen cutoffs  (p < 0.02). For WPM and PATH, we tested geometric mean p-value cutoffs of . The WPM consensus interaction set was significantly larger than expected by a consensus p-value cutoff of 0.05 (p = 0.05). The PATH consensus interaction set was significantly larger than expected by chance (p ≤ 0.05) with consensus p-value cutoffs of . Detailed information for all consensus interactions is reported in  (5.0 × 10orted in .FDR ≤ 0.25) and was also nominally significance in MCS JPN (JPN517). Given this strong support across different cohorts, we further investigated this pathway.From the consensus interaction analysis, we identified the glutathione conjugation pathway as a major source of genetic interactions in multiple cohorts. As a PATH hub interaction, glutathione conjugation was deemed significant in MCS LTN (LTN517) and SBCGS (CHN799) , SBCGS (CHN799) and MCS JPN (JPN517) cohorts. We first calculated the interaction degree for all glutathione conjugation SNPs and then summarized them at the gene level. To enable comparison across cohorts, we measured a fold enrichment for each gene. More specifically, we first computed the interaction rate  and then divided it by the background interaction density in each dataset. This fold enrichment was averaged across all SNPs mapping to each gene in cases where there was more than one. This analysis showed that GSTM1, GSTM4, GSTP1, GSTA5’s interactions are 1.5-fold higher than background interactions in all three cohorts. These results are included in FDR ≤ 0.25), three of which were in common. All glutathione conjugation related between-pathway interactions that are supported by at least one cohort (FDR ≤ 0.25) are listed in Although the glutathione conjugation pathway was enriched for interactions across the genome in multiple cohorts (pathway hub model), BridGE was not able to discover between-pathway or within-pathway interactions associated with glutathione conjugation when run independently on each of the three cohorts. To identify which pathways were interacting with glutathione conjugation gene set, we re-ran BridGE on two of the cohorts (MCS LTN517 and SBCGS CHN799-imputed), but only focused on identifying interactions with glutathione conjugation gene set. Specifically, we limited our hypothesis tests to only BPMs that involved glutathione conjugation. This reduced the number of hypotheses test from more than 400k to less than 2k, substantially improving our power to detect interactions with glutathione conjugation. We detected 16 pathways interacting with glutathione conjugation in MCS LTN (LTN517) and 73 in SBCGS (CHN799-imputed) (S1 Table(A) List of between-pathway interactions discovered in this cohort; (B) List of within-pathway interactions discovered in this cohort; (C) List of pathway hub interactions discovered in this cohort; (D) Corresponding pathway names for (XLSX)Click here for additional data file.S2 Table(A) List of replicated between-pathway interactions (BPMs); (B) Replication statistics.(XLSX)Click here for additional data file.S3 Table(XLSX)Click here for additional data file.S4 Table(A) List of within-pathway interactions discovered in this cohort; (B) List of pathway hub interactions discovered in this cohort.(XLSX)Click here for additional data file.S5 Table(A) List of between-pathway interactions from the consensus analysis; (B) List of within-pathway interactions from the consensus analysis; (C) List of pathway hub interactions from the consensus analysis; (D) Statistical significance results for consensus analysis; (E) Corresponding pathway names for (XLS)Click here for additional data file.S6 Table(A) List of pathway hub interactions discovered in this cohort.(XLSX)Click here for additional data file.S7 Table(A) List of between-pathway interactions discovered in this cohort; (B) List of within-pathway interactions discovered in this cohort; (C) List of pathway hub interactions discovered in this cohort.(XLSX)Click here for additional data file.S8 Table(A) List of between-pathway interactions discovered in this cohort; (B) List of within-pathway interactions discovered in this cohort; (C) List of pathway hub interactions discovered in this cohort.(XLSX)Click here for additional data file.S9 Table(A) List of pathway hub interactions discovered in this cohort.(XLSX)Click here for additional data file.S10 Table(XLSX)Click here for additional data file.S11 TableList of pathways identified by BridGE across 5 different cohorts with FDR≤0.25 and their link to GWAS genes.(XLSX)Click here for additional data file.


Objectives: Network pharmacological methods were used to investigate the underlying molecular mechanisms of LianXia NingXin (LXNX) formula, a Chinese prescription, to treat coronary heart disease (CHD) and disease phenotypes (CHD related diseases and symptoms).Methods: The different seed gene lists associated with the herbs of LXNX formula, the CHD co-morbid diseases and symptoms which were relieved by the LXNX formula (co-morbid diseases and symptoms) were curated manually from biomedical databases and published biomedical literatures. Module enrichment analysis was used to identify CHD-related disease modules in the protein–protein interaction (PPI) network which were also associated to the targets of LXNX formula (LXNX formula’s CHD modules). The molecular characteristics of LXNX formula’s CHD modules were investigated via functional enrichment analysis in terms of gene ontology and pathways. We performed shortest path analysis to explore the interactions between the drug targets of LXNX formula and CHD related disease phenotypes .Results: We identified two significant CHD related disease modules , which were targeted by the herbs of LXNX formula. Pathway and GO term functional analysis results indicated that G-protein coupled receptor signaling pathways (GPCR) of M146 and cellular protein metabolic process of M203 are important functional pathways for the respective module. This is further confirmed by the shortest path analysis between the drug targets of LXNX formula and the aforementioned disease modules. In addition, corticotropin releasing hormone (CRH) and natriuretic peptide precursor A (NPPA) are the only two LXNX formula target proteins with the low shortest path length (on average shorter than 3) to their respective CHD module and co-morbid disease and symptom gene groups.Conclusion: G-protein coupled receptor signaling pathway and cellular protein metabolic process are the key LXNX formula’s pathways to treat CHD disease phenotypes, in which CRH and NPPA are the two key drug targets of LXNX formula. Further evidences from Chinese herb pharmacological databases indicate that Pinellia ternata (Banxia) has relatively strong adjustive functions on the two key targets. Jin Gui Yao Lue) (Coronary heart disease (CHD) is a cardiac disease caused by coronary artery arteriosclerosis leading to cardiac vascular stenosis or obstruction, and/or by functional change (or vasospasm) of coronary artery which leads to myocardial ischemia or necrosis . AccordiYao Lue) . Even toYao Lue) .LianXia NingXin formula (LXNX formula) was created by Professor Ping Li, a Chinese medicine expert in our research team, on the basis of the TCM theory of the heart controlling the blood vessels, nerves and spirit . It alsoCoptis chinensis (Huang Lian), Pinellia Tuber (Ban Xia), Orange peels (Chen Pi), Porio coco (Fu Ling), etc. These nine Chinese herbs have more than one hundred drug ingredients which can affect more than one thousand human biological proteins. It will be extremely difficult for us to identify LXNX formula’s drug targets one by one from thousands of proteins through traditional method. In recent years, it is well recognized that human body is a complex system. A drug that only regulates one protein target may not be able to affect the whole human protein network  database2 and Malacards3 databases. Gene information for co-morbid symptoms was gathered from the Malacards, HPO4 and DiseaseConnect5 databases.We used the disease terms to filter the associated genes for LXNX formula treated phenotypes from several well-established phenotype-genotype association databases. In particular, genes associated with the co-morbid diseases were identified via the OMIM6, TCMID7 databases, and the biomedical literatures in PubMed8 and CNKI9 databases. The data was then organized into an herbs-ingredients-targets relationship dataset for LXNX formula.The drug ingredients and protein targets information associated to nine herbs which constitute the LXNX formula were curated from HITP < 0.05) of these enrichment results.Module enrichment analysis was used between the eight CHD modules determined from the previous preliminary experiment and the following seed gene groups: each individual herb, each co-morbid disease, each co-morbid symptom, set of all herbs, set of all co-morbid diseases, and set of all co-morbid symptoms. Fisher’s exact test was used to calculate the enrichment degree between each seed gene group and the CHD modules. And then, the significant CHD modules related to LXNX formula were identified on the basis of the positive results  between CHD modules related to LXNX formula was calculated by Fisher’s exact test. Gene ontology term analysis was based on Gene Ontology database, and was carried out by the plugin BiNGO 2.44 in the software Cytoscape 2.8.2. Pathway enrichment analysis was based on Reactome database and was carried out by online software KOBAS2.072. P-value was calculated in these two enrichment analyses, and P < 0.05 means the enrichment degree had statistically significance and the pathway results would be important functional mechanisms of disease modules. Secondly, the drug protein targets of LXNX formula in CHD modules related to LXNX formula were identified through shortest path analysis. The first step of the shortest path analysis is to calculate the shortest path between any node pair in the PPI network; subsequent calculations are then done to find the average shortest path length of a fixed source node to given node set. The shorter the average length is, the more important this source node is in connecting the entire node set. The Dijkstra algorithm  enrichment analysis based on the David platform . On the lgorithm , which iFigure 1 showed the flowchart of the pathway analysis and target identification of LXNX formula to treat CHD.Similarly, we identified the drug targets of LXNX formula in regards to gene sets of co-morbid diseases and symptoms using shortest path analysis, respectively. Based on these drug targets and the targets for LXNX formula’ s CHD modules, key candidate drugs targets of LXNX formula to treat both CHD and its co-morbid diseases and symptoms were identified. Table 1).The results of previous pre-clinical studies  on LXNX Table 1 and Supplementary Tables In accordance with the searching results from gene databases, there were 3230 genes related to LXNX formula co-morbid diseases and symptoms. These genes had 4605 interactions with co-morbid diseases and symptoms. For the symptoms of chest tightness, bitter taste in mouth, and dream-disturbed sleep, no related genes were found  (see Supplementary Table p < 0.05). Specifically, M146 had 12 positive results, while M203 had 8 positive results (see Table 2).Module enrichment analysis between the preliminary eight CHD modules and various seed gene groups for co-morbid diseases, co-morbid symptoms, and Chinese herbs showed that M146 and M203 had statistically significant results (P < 0.05) . Besides CRH and NPPA, there were no other proteins which served as key drug targets simultaneously in LXNX formula’s CHD modules and all co-morbid diseases and symptoms.Thirty proteins in LXNX formula were closely related to M146 genes  and pathways (those bold lines between key targets) of LXNX formula to treat CHD based on the analysis results of the enriched functional pathway and the shortest path.p < 0.05) indicated that the biological mechanism of the disease module was related to the functional pathway of the disease, symptom, or herb seed gene groups. Disease module analysis results showed that the functional pathways in M146 and M203 were strongly associated with these aforementioned seed gene groups, both individually and collectively. Other CHD modules did not have such lots of positive results. Therefore, these results indicate that the treatment effect of LXNX formula for CHD and its co-morbid diseases and symptoms is mainly accomplished by modulating M146 and M203 functional pathways.Presently, the incompleteness of knowledge regarding human protein interaction and genes related to diseases resulted in the hindering of research into complex systemic diseases such as CHD. Recently, it has been recognized that proteins associated with diseases did not exist randomly in a scattered network, but rather were connected into functional subgroups called disease modules. Thus, disease module analysis can now be used to identify previously unknown genes and mechanisms associated with complex diseases . For thiBased on the results of pathway and GO term enrichment analysis and of the shortest path analysis, the signaling pathways of G-protein coupled receptors (GPCRs) related to adrenaline, adenylate cyclase, dopamine, and calcium ion are significant in M146, and the drug targets of LXNX formula in M146 are all related to these pathways; the pathways of specific cellular protein metabolic process related to the apolipoprotein (APO) and natriuretic peptide are important in M203, and the drug targets in M203 are all related to these pathways. GPCRs signaling transduction pathway constitute a large protein family of cellular surface receptors which are responsible for various ligand responses, signal transduction from extracellular space to intracellular space, and activating cellular responses. GPCRs are largely expressed in human body, and play a key role in regulation. For example, β-adrenergic receptor (β-AR) is a typical ligand activated GPCR. Afterβ-AR is activated, adenylate cyclase is then activated, and the concentration of cyclic adenosine monophosphate (cAMP) is subsequently increased, inducing a series of biological effects. β-AR is widely distributed in the heart and thus plays an important role in regulating cardiac functions . The patThe proteins CRH and NPPA are critical drug targets for LXNX formula’s CHD module and all co-morbid diseases and symptoms; no other protein has shown such simultaneous results. CRH is synthesized in the hypothalamus and acts upon the hypothalamus-pituitary-adrenal gland axis (HPA axis). HPA axis is a complex system which includes direct effects and negative feedback mechanisms. When stressed , the hypothalamus secretes CRH , which pNatriuretic peptide precursor A is the precursor to atrial natriuretic peptide (ANP) before synthesis; NPPA polymorphism may induce myocardial necrosis, hypertension, atrial fibrillation, water and sodium disequilibrium, and heart remodeling . After AIn summary, CRH and NPPA have strong relationship with LXNX formula for the reasons that they are both the most important drug targets of LXNX formula in LXNX formula’s CHD modules, respectively, co-morbid diseases and symptoms, they are both relevant to the important pathways, respectively, of LXNX formula, and they both can have influence on CHD and CHD co-morbid diseases which LXNX formula can treat. Therefore, CRH and NPPA are key drug targets of LXNX formula to treat CHD and its co-morbid diseases and symptoms.Pinellia ternata (Ban Xia) has a direct influence on CRH; adenine in Porio coco (Fu Ling), undecanal in dried orange peels (Chen Pi), guanosine and L-arginine (L-Arg) in Pinellia ternata (Ban Xia) have a strong impact on NPPA. Evidently, Pinellia ternata (Ban Xia), Porio coco (Fu Ling), and dried orange peels (Chen Pi) are important herbs in LXNX formula. GABA is an important inhibitory neurotransmitter in the brain. By combining with GABA receptors, it inhibits CRH neuron excitability to reduce the release of CRH (L-Arg is the precursor of EDRF (Endothelium derived relaxing factors)/NO. Shortage of EDRF/NO is essential to the occurrence of cardiovascular diseases including CHD  found in e of CRH . L-Arg iding CHD . On the renaline .in vitro and in vivo experiments on mice with myocardial ischemia.Based on the analysis mentioned above about previous documentation, GPCRs signaling transduction pathway related to CRH and cellular protein metabolic process pathway related to NPPA both can influence immune and nervous systems to treat CHD, this character is accordance with the mechanism of LXNX formula to treat CHD. Thus, the forecast results of network medicine concerning LXNX formula’s mechanism of action for treating CHD is reliable. In future research, further experimentation can be done to test the influence of LXNX on CRH and NPPA pathway by L-arginine and gamma-aminobutyric acid, etc. in Pinellia Tuber, Poria coco, affect CRH in GPCRs signaling pathway and NPPA in specific cellular protein metabolic process.In accordance with this research, the important pharmacological mechanisms of LXNX formula to treat CHD disease phenotypes are that the ingredients such as PL, XZ, YY, and KY conceived and designed the study. YY, TH, and GZ extracted the data. KY analyzed the data. PL, XZ, YY, KY, and RL wrote and revised the manuscript. All authors reviewed and approved the submitted version of the manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


We describe the BioCyc update notifications service, a new mechanism to keep researchers informed of the latest developments in their areas of interest. BioCyc.org combines databases for 9,300 sequenced organisms that integrate genome, metabolic pathway, and regulatory information with extensive bioinformatics tools. Users of the BioCyc website can register their specific areas of interest online by specifying a set of genes, pathways and/or Gene Ontology terms. Then, when significant new information becomes available in a BioCyc database in a user’s interest areas , an email notification is sent to the user. The BioCyc ontology is leveraged to identify changes that are both relevant to a user’s specified interests and worthy of notification. Every effort is made to ensure that the resulting email text is both concise and informative, with links to relevant BioCyc pages.Database URL:https://BioCyc.org, https://EcoCyc.org, https://MetaCyc.org The BioCyc collection of databases  forms a Updated versions of the BioCyc website and component databases are released three times per year. At those times, a general announcement is sent by email to all registered users of the site. Although this announcement typically outlines significant areas of new or updated curation, the description is by necessity general and incomplete. If a researcher wanted to know if new curation was available for specific entities of interest (e.g. genes), until recently they would have no recourse but to visit the web pages for each of those objects and try to manually discern what information, if any, is new.Other curated model organism databases (MODs) face similar issues. They typically issue news announcements when notable or significant data updates are available, but these announcements are not targeted to individual user interests. MODs that use the InterMine platform  enable uOur solution to the alerting problem is enabling the registered users of the BioCyc, EcoCyc and MetaCyc websites to subscribe to specific areas of interest in a given organism database. Then, shortly after every new release, newly curated data that matches a user’s interests results in an email that is sent informing the user of the relevant changes. However, these days, users are deluged with email on a daily basis, and a notification email is not useful unless the user actually reads it. The main challenge we faced was how to tailor the email notifications to ensure that the user receives all relevant information without being inundated by a mass of less relevant details. This challenge breaks down roughly into three components: (i) deciding on an appropriate level of granularity at which to specify areas of interest, and making it simple and convenient for users to register for and manage their notifications; (ii) determining what types of database updates are significant enough to warrant notification; and (iii) generating email content that is sufficiently informative yet not overwhelming to the recipient.Users can define their interest areas by registering for notifications about specific genes or pathways, about Gene Ontology (GO) terms, and about pathway classes from the MetaCyc pathway ontology .trpA, in order to decide whether changes have been made to that gene, the notification software must check not just the gene object but also the gene product; complexes the gene product may belong to; enzymatic or regulatory activities of the gene product or complexes; pathways containing the product or complexes; and transcriptional, translational and substrate-level regulation. When a user registers for a specific pathway, we consider not just the pathway object but also its reactions, metabolites, enzymes, genes and regulatory interactions.Each designation of a gene or a pathway actually encompasses a whole collection of functionally related database objects. When a user registers for a specific gene such as Bacillus subtilis, will only be notified of changes to the pathway and its enzymes in that database, not to the corresponding pathway in EcoCyc, MetaCyc or any other organism).GO terms and pathway classes define sets of genes and pathways, respectively, and can be quite useful for specifying biological interest areas of appropriate specificity. When users subscribe to a GO term such as GO:0001906 (cell killing) or GO:0022610 , they have two choices: (i) they can choose to be notified only when new genes are annotated to that term, or (ii) they can be notified of changes (using the above criteria for changes associated with a gene) to all genes currently annotated to that term. An analogous choice applies when a user registers for a pathway class such as secondary metabolites biosynthesis, chlorinated compounds degradation, or arsenate detoxification. All object designations are specific to a single database  where they can either enter (with autocomplete) the objects they wish to subscribe to or cancel any object notifications they are no longer interested in. If users have a large list of specific genes or pathways that they wish to subscribe to , they can create a SmartTable  containiFrom one release to the next, a database object can undergo changes for a variety of reasons, and to any number of different fields. We do not want to trigger update notifications for all of these changes. A curator may update an enzyme to correct a typo or erroneous data value, to add a synonym or minor clarification, or to reflect data from a newly curated publication. All of these would be considered curated changes, yet it is our judgment that most users would prefer to be notified only of the latter, but not of any of the former. Databases also undergo periodic programmatic updates. Often these updates represent only minor changes in schema or representation that are unworthy of notification; but sometimes, they involve integration of new data sources that are likely to be of significant interest. Other types of changes that we consider unworthy of notification  include when redundant objects are merged, obsolete objects or GO annotations are deleted, and various fixes are applied to maintain database consistency, even though all these operations are likely to involve multiple changes to existing database objects.Although we can programmatically compare any object in a new DB release with the same object in the previous version and determine what changes have been made, determining from that comparison the reason for each change, and whether or not it reflects a significant change about which users should be notified, is not straightforward.Fortunately, we can leverage the fact that BioCyc curators never make significant changes to an object without adding a relevant citation (either to the changed attribute itself or to the object as a whole). Similarly, the importation of a significant source of new data can also be expected to be accompanied by a new citation. Thus, regardless of the number and type of changes, a gene or pathway is considered to have undergone significant updates that warrant notification only if at least one new citation (not present in the previous release) is associated with it or any of its associated objects or attributes. Once we have determined via this filter that a particular object meets the threshold for notification, we collect all differences between the old and new versions (since we often cannot know which specific field changes are associated with the new citations) to be summarized in the notification email.The purpose of the notification email is not to reproduce the contents of any BioCyc web page. Rather, as compactly and concisely as possible, we want to provide users with links to updated pages in their areas of interest, and enough information about the types of changes that they can both decide if clicking any given link is worthwhile, and, if so, where on the page they should look for the changes. For example, if a user is subscribed to a pathway, and there are changes to two enzymes in the pathway, the email would contain a link to the pathway page and then list the changed enzymes with links to those enzyme pages. For each changed enzyme, a listing of the changes, such as changes to transcriptional or translational regulation, changes to the textual summary, GO terms, etc. would be given. Finally, we include the list of new publications. We have developed a software tool for notifying users of significant updates to curated BioCyc databases. Rather than specifying interest areas using keywords, as with previous notification systems, interest areas are specified using ontologies (Gene Ontology or the MetaCyc pathway ontology) and database objects to provide much higher precision. Notification emails are triggered when a new citation is added to an object within a user’s defined interest area. Notification emails are both comprehensive and concise to appeal to busy biologists. It is our hope that many BioCyc users will avail themselves of this opportunity to stay up-to-date with the latest developments in their fields of interest.