Thus computational approaches for detecting protein complexes from protein interaction data are useful complements to the limited experimental methods. They can be used together with the experimental methods for mapping the interactions of proteins to understand how different proteins are organized into higher-level substructures to perform various cellular functions.Most proteins form macromolecular complexes to perform their biological functions. However, experimentally determined protein complex data, especially of those involving more than two protein partners, are relatively limited in the current state-of-the-art high-throughput experimental techniques. Nevertheless, many techniques (such as yeast-two-hybrid) have enabled systematic screening of pairwise protein-protein interactions Given the abundance of pairwise protein interaction data from high-throughput genome-wide experimental screenings, a protein interaction network can be constructed from protein interaction data by considering individual proteins as the nodes, and the existence of a physical interaction between a pair of proteins as a link. This binary protein interaction graph can then be used for detecting protein complexes using graph clustering techniques. In this paper, we review and evaluate the state-of-the-art techniques for computational detection of protein complexes, and discuss some promising research directions in this field.Experimental results with yeast protein interaction data show that the interaction subgraphs discovered by various computational methods matched well with actual protein complexes. In addition, the computational approaches have also improved in performance over the years. Further improvements could be achieved if the quality of the underlying protein interaction data can be considered adequately to minimize the undesirable effects from the irrelevant and noisy sources, and the various biological evidences can be better incorporated into the detection process to maximize the exploitation of the increasing wealth of biological knowledge available. Most proteins form complexes to accomplish their biological functions -3. In faet al. [in vitro purification of whole-cell lysates [While there are a number of ways to detect protein complexes experimentally, Tandem Affnity Purification (TAP) with mass spectrometry  is the pet al.  have sho lysates . This meen masse have enabled the construction of PPI networks on a genomic scale. A graphical map of an entire organism's interactome can be constructed from such experiments by considering individual proteins as the nodes, and the existence of a physical interaction between a pair of proteins as a link between two corresponding nodes. Given that protein complexes are molecular groups of proteins that work together as "protein machines" for common biological functions, we may expect the protein complexes to be functionally and structurally cohesive substructures in the binary PPI networks [Recently, high-throughput methods  for networks . Researcnetworks , suggestnetworks . HoweverBefore we review the current computational approaches for protein complex detection, let us make a principled distinction between two biological concepts, namely, protein complexes and functional modules . A proteLet us now introduce some terminologies which are widely used in protein complex mining. Then, we will present the use of traditional graph clustering techniques for complex mining followed by some new emerging techniques for this task.G = , where V is the set of nodes (proteins) and E = {|u, v ∈V } is the set of edges (protein interactions). A graph G1 =  is a subgraph of G if and only if V1⊆V and E1⊆E. For a node v ⊆V , the set of v 's direct neighbors is denoted as vN where vN = {u |u ⊆V,  ⊆ E }. v 's degree in G, deg (v ), is the cardinality of vN, i.e., |vN|. Density. A PPI network is often modeled as an undirected graph G, denoted as density (G ), is defined to quantify the richness of edges within G as shown in equation (1) [density (G ) ≤ 1. If density (G ) = 1, then G is the fully connected graph or a clique, which has the maximum number of edges, i.e., every pair of distinct vertices is connected by an edge.The density of the graph tion (1) . BasicalClustering Coeffcient. The clustering coeffcient of a node v is the density of the subgraph formed by vN and their corresponding edges, which quantifies how close v 's neighbors are to being a clique (complete graph) [e graph) .Local Neighborhood. Given a node u ∈V, its local neighborhood graph uG is the subgraph formed by u and all its immediate neighbors with the corresponding interactions in G . It can be formally defined as uG= , where uV= {u } ∪u N, and uE= {| ∈j , vk E, v ∈u V}If the edges in the PPI network are weighted , thew (e ) is the weight of the edge e .where v is the weighted density of the subgraph formed by Nv and their corresponding edges.Similarly, the weighted clustering coeffcient of the node First, we review the conventional graph clustering approaches for protein complex mining. These methods mine for cliques or densely connected subgraphs in PPI networks which could correspond to protein complexes. While the methods mainly use the PPI networks for mining, additional information, such as gene expression data ,24, funcIn this section, let us describe the graph clustering approaches that use PPI networks as the sole underlying dataset for the mining task.MCODE.The MCODE algorithm proposed by Bader et al. [k-core in node 5's neighborhood graph with k=2 [d = 5/6 . Thus, node 5 has an initial weight w (5) = k × d = 2 × 5/6 = 1.67. Next, the node with the highest weight  is selected as an initial cluster. Node 2, as node 1's neighbor, satisfies the weight constraint to be included into the cluster because w (2) = 3 ≥ (1 – wT) × w (1). Here wT is a threshold for cluster formation that is set as 0.2 by default. Similarly, nodes 3 and 8 are also added into this cluster and finally MCODE predicts {1, 2, 3, 8} as a protein complex. Subsequently, {4, 5, 6, 7} is detected as another putative protein complex from this sample PPI graph.r et al.  is one owith k=2 ,29 and dThe experimental results of MCODE method showed that the number of predicted complexes is generally small and the size of many predicted complexes is often too large.Clique.					 Spirin and Mirny [nd Mirny  proposedMCL. Markov Clustering (MCL) [ng (MCL) ,31 can ang (MCL) , MCL is ng (MCL) -36.LCMA.					 Instead of adopting the over-constraining full cliques as the basis for protein complexes, Li et al. [i et al.  devised i et al.  describeDPClus. Amin et al. [n et al.  proposedPCP. Chua et al. [a et al.  proposeda et al.  to evalua et al.  and a paHub Duplication.						 Ucar et al. [r et al.  developer et al. ) are firCFinder. Adamcsek et al. [k-clique percolation clusters as functional modules using a Clique Percolation Method [k-clique is a clique with k nodes and two k-cliques are adjacent if they share (k – 1) common nodes. A k-clique percolation cluster is then constructed by linking all the adjacent k-cliques as a bigger subgraph.k et al.  providedn Method . In partSCAN.						Mete et al. [e et al.  proposede et al. . SCAN fiGS. Navlakha et al. [a et al.  applied a et al.  to clustCMC. Liu et al. [u et al.  recentlyu et al. . CMC theu et al. . TherefoProteins which interact with each other can be expected to be activated and repressed under the same conditions. In other words, interacting proteins are likely to exhibit similar gene-expression profiles. In fact, gene expression data has been widely exploited to annotate protein functions (guilt by association) and predict novel protein-protein interactions -52. We dGFA. Feng et al. [G = , two different density definitions as shown in equation (3) are used:g et al.  proposedw (v ) is the weight of the protein v weighted by –expressionev )  is the log fold change of v's gene-expression profile. GFA first applies DSA (Densest Subgraph Algorithm) [where gorithm)  to find DMSP. Maraziotis et al. [s, its neighbors and even its indirect neighbors are iteratively included based on different criteria to form the module.s et al.  developeMATISSE. Ulitsky et al. [k neighbors with the highest weights are picked to form a set of (k + 1) seeds. Last, after selecting the seeds, Jointly Active Connected Subnetworks (JACS) are obtained by several operations . Two small JACSs are merged to form a new one if they are closely connected.y et al.  also proFunctional information can also be incorporated to accurately detect protein complexes. Since proteins within the same protein complex are generally aggregated to perform a common function, the functional enrichment of a cluster can be used to indicate its tendency to be a real complex. The reliability of interactions, evaluated by the consistency of functional similarity between two proteins, can also help to provide cleaner PPI data for protein complex detection.RNSC. King et al. [g et al.  proposedDECAFF. Li et al. [i et al.  proposedi et al.  are procSWEMODE. Lubovac et al. [c et al.  presentec et al. -62. Secoc et al. .STM. Cho et al. [o et al.  extendedo et al.  to identet al. [The techniques discussed above have used pairwise physical interactions detected by high-throughput experiments such as Y2H as the PPI dataset for detecting protein complexes. More recently, there are some researchers who attempt to detect protein complexes from interaction data obtained solely from TAP experiments. Unlike Y2H method which detects direct physical interactions, using TAP data requires careful weighing of the detected links as TAP also detects indirect interactions in protein complexes. Krogan et al.  were oneet al. ,31 is thet al. [Caroline et al.  proposedet al. ,31 is apet al. [et al. [Pu et al.  also appet al. . Similar [et al.  used altet al. [G =  where U and V represent the sets of baits and preys respectively and E describes the bait-prey relationships detected in the experiments as shown in Figure Recently, Geva et al.  proposedIn this section, we review a number of emerging techniques for protein complex detection that are different from the application of traditional graph clustering described in the previous section.The previous graph clustering methods described above are unsupervised and are more or less based on the basic assumption that dense subgraphs in PPI networks are likely to be protein complexes. The protein complexes detected by many of these methods must be either cliques or defective cliques ,67 or aret al. [Qi et al.  proposedTo obtain the training data, they collect available known protein complexes and also generate some random subgraphs as non-complexes. Topological and biological properties of these training graphs are then summarized as features. A probabilistic Bayesian network (BN) is then applied to integrate all these features and the parameters of this BN model are learned from the training data.Given a graph candidate and its corresponding features, a log likelihood ratio score can thus be calculated by the BN model to show whether it is qualified to form a complex. A simulated annealing search  is furthIn the previous section, we have mentioned that some researchers have explored the use of TAP data instead of Y2H data for complex detection. However, as TAP does not detect direct pairwise protein-protein interactions (unlike Y2H), the PPI networks constructed using TAP data are not ideal for detecting protein complexes. Recently, several techniques are proposed to directly detect protein complexes from the TAP data without constructing the PPI networks.et al. [Rungsarityotin et al.  applied et al. [Z, where each entry ijz indicates that the thi  protein is in the complex j . The prior distribution of Z is learned from an infinite latent feature model. By considering the pairwise similarity between proteins obtained by a graph diffusion kernel [Z is further inferred to indicate the protein complex memberships by the Gibbs sampling.Chu et al.  used a Bn kernel , the posTwo adjacent interactions (those with a common protein) may be mutually exclusive ,74 due tet al. [Jung et al.  recentlyet al.  and LCMAet al.  to generet al. [Jin et al.  exploiteWith the increasing availability of PPI data for most species , it has become feasible to use cross-species analysis to derive insights into the evolution of the PPI networks for complex detection.et al. recently proposed a series of methods for comparative analysis in two or more species. They used these methods for conserved pathway detection [Sharan etection , proteinetection  and consetection -80. Basiu, v ) in the orthology graph is weighted by the sequence similarity between the protein pair u and v . An edge , ) is associated with a pair of weights , w ), where w  is the weight of the interaction . Two models, the protein-complex model and null model, are proposed to learn the weights of interactions and detect protein complexes in each species. In [In , each nocies. In , the rescies. In . A subgrcies. In .QNet was developed for queries in PPI networks. The similarity between two graphs is defined based on the node and edge similarity and the penalty scores for node deletion and insertion. QNet then performs tree queries and bounded-treewidth graph queries by the color coding algorithm [In , a tool lgorithm . Conservet al. [Another group of researchers, Dutkowski et al.  also proet al. [In the genome-wide screen for protein complexes using affnity purification and mass spectrometry  reportedet al. , they obet al. .et al. [Zhang et al.  proposedet al.  and someet al. [Recently, Leung et al.  proposedet al. [To provide insights into the organization of protein complexes, Wu et al.  presentsThe ability to detect overlapping cores is essential to understand how different cores are organized into the higher-level structures in PPI networks and how these cores communicate with each other to perform cellular functions. It also facilitates better detection of protein complexes from PPI networks, which will be shown in the evaluation results in the next section.Before we present the results of our comparative experiments, let us first introduce the various evaluation metrics that have been used to evaluate their computational methods for complex detection. We will then present the experimental results of comparing different state-of-the-art techniques using these evaluation metrics.Overall, there are three types of evaluation metrics used to evaluate the quality of the predicted complexes and compute the overall precision of the prediction methods.NA  between a predicted complex p =  and a real complex b =  in the benchmark complex set, as defined in equation (4) below, can be used to determine whether they match with each other. If NA  ≥ ω , they are considered to be matching . Let P and B denote the sets of complexes predicted by a computational method and real ones in the benchmark, respectively. Let cpN be the number of predicted complexes which match at least one real complex and cbN be the number of real complexes that match at least one predicted complex. Precision and Recall are then defined as follows: [Precision, recall and F-measure are commonly-used evaluation metrics in information retrieval and machine learning. For evaluating protein complex prediction, we need to define how well a predicted complex which consists of a set of protein members, matches an actual complex, which is another set of protein members. The neighborhood affnity score follows: ,66,87 :, positive predictive value (PPV ) and accuracy (Acc ) have also been proposed to evaluate the accuracy of the prediction methods [n benchmark complexes and m predicted complexes, let ijT denote the number of proteins in common between thi  benchmark complex and thj  predicted complex. Sn and PPV are then defined as follows:Recently, sensitivity  functional annotations. The statistical significance of the occurrence of a protein cluster (predicted protein complex) with respect to a given functional annotation can be computed by the following hypergeometric distribution in equation (9) ,89:(9)C contains k proteins in the functional group F and the whole PPI network contains |V | proteins. The functional homogeneity of a predicted complex is the smallest p-value over all the possible functional groups. A predicted complex with a low functional homogeneity indicates it is enriched by proteins from the same function group and it is thus likely to be true protein complex. By setting a common threshold which specifies the acceptable level of statistical significance, the numbers of predicted complexes with functional homogeneity under this threshold for the various methods can then be used for evaluating their respective overall performance.where a predicted complex Sn ), positive predictive value (PPV ) and Accuracy (Acc ). For sensitivity (Sn ), if a method predicts a giant complex which covers many proteins in the known real complex set, then this method will get a very high Sn score. As for PPV value (PPV ), it does not evaluate overlapping clusters properly. Here is a case in point: if the known gold standard MIPS complex set (with proteins that belong to multiple complexes) [PPV value is 0.772 instead of 1 (indicating an imperfect match) while the Precision and Recall are both correctly 1. As such, the Accuracy (Acc ) score, as the geometric average of Sn and PPV , will also not make good sense. In addition, all the evaluation metrics described above assumed that a complete set of true protein complexes is available, where in reality we are far from it. If a method predicts an unknown but real protein complex (which is not similar with any of the known complexes), all of these evaluation metrics will regard it as a false positive. Furthermore, for P-values, since its calculation relies on the availability of the proteins' functional information, its applicability would be limited in the less studied genomes. As such, so far it has mainly been used in the model organism yeast for which rich molecular functional information is available.It is important to realize that the evaluation metrics described above can only provide us some sense of how well the current graph mining techniques can be used to detect the protein complexes from protein interaction data. These metrics are by no means  absolute measures — they all have their own limitations, especially for sensitivity (mplexes)  is takenRelatively speaking, the Precision, Recall, F-measure and P-values are thus more acceptable for evaluating the performance of current techniques. Still, we need to treat the current evaluation metrics with caution, as more research is needed to come up with a robust evaluation metric for the protein complex prediction task.For this review, we have performed extensive experiments to compare the existing techniques for which we are able to obtain the software implementations — either source codes or binary executable systems. Those existing techniques that do not provide available software are not included in the comparison exercise. Fortunately, we have a good representative collection of implemented algorithms for comparison: MCODE , RNSC 5, MCL 3131, DPCluω is set as 0.20 to evaluate if a predicted complex matches with a gold standard protein complex (see equations (4) and (5)).In order to evaluate the predicted complexes, the set of real complexes from  was seleWe have compared these techniques over two publicly available benchmark yeast PPI datasets, namely DIP data  and KrogcpN) and the number of real complexes that match at least one predicted complex cbN. Taking MCODE on DIP data as an example, it has predicted 50 complexes, of which 44 match 21 real complexes. These 50 predicted complexes cover 844 proteins out of 4930 proteins in DIP. As shown in these two tables, MCL and RNSC assigned every protein (4930) into its predicted complexes as long as they are present in PPI networks  while all the other methods only assigned those highly interactive proteins (or the proteins that occurred in the dense subgraphs) into the predicted complexes. In fact, both MCL and RNSC basically partitioned the PPI network simultaneously into non-overlapping clusters while the remaining approaches are more sensible by generating clusters in a one-by-one manner and allowing overlaps in the clusters/complexes. We also noticed that for DECAFF algorithm the number of predicted complexes that matches at least one real complex (cpN) is significantly higher than the other methods — this is mainly because it is designed to search many dense and possibly overlapping complexes from the PPI networks.Tables Figures http://db.yeastgenome.org/cgi-bin/GO/goTermFinder.pl). The complexes with only one protein are discarded because calculating P-values for those complexes makes no sense according to the equation (9). We considered a predicted complex with a corrected P-value ≤ 0.01 to be statistically significant. The results showed that MCODE was able to obtain the highest proportion of significant complexes.Figure Unfortunately, this was an artefact of its predicting very few complexes as compared to all the other methods. Ignoring MCODE, then COACH and DECAFF have both achieved decent proportions of their predicted complexes as significant. As for DPClus, MCL and RNSC, because they predicted many protein complexes with extremely small size  which resulted in large P-values since they could occur by chance. For CORE, it generated many protein-complex cores with only one protein. Given such a core with one protein, CORE can only form a protein complex by including all the interacting partners of the protein as attachment. These protein complexes have low statistical significance, leading to the low performance of CORE.en masse have already become routine in the laboratories for generating large datasets of protein interaction data , while the high-throughput technologies for detecting protein complexes remained relatively immature. Hence computational approaches for detecting protein complexes are needed to help fill up the relatively empty map for the protein "complexome".Identifying protein complexes is important for biological knowledge discovery since many important biological processes in the cell are carried out through the formation of protein complexes. However, there is currently a wide gap between data on protein complexes and (pairwise) protein-protein interactions. High throughput technologies for detecting pairwise protein-protein interactions In this paper, we have reviewed current computational approaches that have been proposed to exploit the abundant protein interaction data to bridge the data gap for protein complexes. Protein interaction graph mining algorithms that identify graphical subcomponents in the protein-protein interaction networks can be used for predicting protein complexes. We have surveyed the state-of-the-art algorithms by describing the traditional graph clustering methods as well as the recent emerging techniques for computational detection of protein complexes from PPI and other data sources. Table On the other hand, our results also show that more further research is needed to improve these methods. In the following discussions, we describe three key challenges for further improvements .The computational methods are highly dependent on the quality of the underlying interaction data. Unfortunately, despite the abundance of PPI data, the data quality of these data still leaves much to be desired. For example, the experimental conditions in which the PPI detection methods are carried out may cause a bias towards detecting interactions that do not occur under physiological conditions, resulting in false positive detection rates that could be alarmingly high . At the PPI networks are very large graphs with thousands of vertices and tens of thousands of edges, even for a simple organism such as yeast. For the more complicated species such as the human being, the scale and complexity of the PPI networks are clearly overwhelming. Graph mining on the PPI networks is certain to test the limits of any computational methods. The fact that many graph-based problems, such as subgraph isomorphisms ,100 and Through integrating various independently obtained biological evidences, we can assess/weight the protein interactions by using appropriate confidence measures. For example, we can employ metrics from biological evidences such as reproducibility of the interactions from multiple experimental methods, support from such other non-interaction data as co-expression, co-localization and shared functions, as well as the conservation of the protein interactions across other genomes, etc to address the limitations in the current quality of PPI data. Computational methods, such as Bayesian network models  and kernFrom this review, it is clear that researchers have been tireless in devising new computational approaches for detecting protein complexes. It is indeed heartening that our evaluation results have showed that the proposed methods have generally improved in performance over the years. In time, we will be able to fill up the currently rather empty map of the complexome with combined efforts  from biologists as well as computational scientists computational scientists.The authors declare that they do not have any competing interests.XL and MW drafted the manuscript together. MW was responsible for performing experiments to compare the existing techniques. CKK and SKN participated in discussion and conceptualization as well as revising the draft. All authors have read and approved the manuscript.


Deciphering physical protein-protein interactions is fundamental to elucidating both the functions of proteins and biological processes. The development of high-throughput experimental technologies such as the yeast two-hybrid screening has produced an explosion in data relating to interactions. Since manual curation is intensive in terms of time and cost, there is an urgent need for text-mining tools to facilitate the extraction of such information. The BioCreative  challenge evaluation provided common standards and shared evaluation criteria to enable comparisons among different approaches.1 score of28.5%. In extracting protein interaction pairs, our profile-based method was competitive on the SwissProt-only subset  and on the entire dataset . From the biologist's point of view, however, these findings are far from satisfactory. The error analysis presented in this report provides insight into how performance could be improved: three-quarters of false negatives were due to protein normalization problems (532/698), and about one-quarter were due to problems with correctly extracting interactions for this system.During the benchmark evaluation of BioCreative 2006, all of our results ranked in the top three places. In the task of filtering articles irrelevant to physical protein interactions, our method contributes a precision of 75.07%, a recall of 81.07%, and an AUC (area under the receiver operating characteristic curve) of 0.847. In the task of identifying protein mentions and normalizing mentions to molecule identifiers, our method is competitive among runs submitted, with a precision of 34.83%, a recall of 24.10%, and an FWe present a text-mining framework to extract physical protein-protein interactions from the literature. Three key issues are addressed, namely filtering irrelevant articles, identifying protein names and normalizing them to molecule identifiers, and extracting protein-protein interactions. Our system is among the top three performers in the benchmark evaluation of BioCreative 2006. The tool will be helpful for manual interaction curation and can greatly facilitate the process of extracting protein-protein interactions. An important step in functional systems biology is to elucidate the relationships between biomolecules. Interactions between proteins define biological pathways, and knowledge of the processes in which proteins are involved is essential to gaining a fundamental understanding of the cellular machinery. The study of protein interactions is a pressing biological imperative, and thus characterizing protein interaction partners is crucial to our understanding of both the function of individual proteins and the organization of entire biological processes ,2.More and more interaction data are being published in the literature as a result of the development of high-throughput experimental technologies, such as the yeast two-hybrid screening and affinity purification coupled with mass spectroscopy. These experimental techniques make it possible to study protein interactions on a much larger scale, although they suffer at times from poor resolution. To provide reliable protein interaction data for biologists, interaction databases such as Molecular Interactions Database (MINT)  and IntAIn addition to the rate at which interaction data are being produced, there are other challenges for manual interaction curation. Experimental methods are not equally reliable, and when they extract protein interactions curators must place emphasis on thorough description of the experimental evidence. Furthermore, many authors continue to use ambiguous gene or protein names in their reports, or they fail to provide the organism or tissue from which the genes or proteins originate. Difficulty in mapping gene and protein names to SwissProt/UniProt ,6 identiBecause of the accumulation of interaction data in the biomedical literature and the challenges that present for manual curation, there is an urgent need for text-mining tools to facilitate the extraction of such information. In particular, the extraction of physical protein-protein interactions, defined as the co-localization or direct interaction between protein molecules, is becoming extremely important because physical interactions are the most reliable data produced in high-throughput experiments. The development of effective text-mining tools could aid the mapping of proteins to SwissProt/UniProt identifiers, as well as the discovery of experimental evidence for interactions and the discrimination of physical interactions from other types of interactions. In comparison with previous studies on bio-text mining, BioCreative 2006 ,9 addresThe protein-protein interaction (PPI) task of BioCreative 2006 is comprised of several subtasks: the interaction article subtask (IAS) to determine whether an article (abstract only) is relevant to some physical interactions; the interaction pair subtask (IPS), in which interacting protein pairs should be extracted from full-text articles; the interaction sentence subtask, in which participants are required to submit a list of summary sentences for each interaction; and the interaction method subtask, in which the experimental detection methods should be given for each interaction.Here, we present the methods and results from our participation in the PPI task of BioCreative 2006 . By usin1 score on the SwissProt-only subset, and in second place on the entire dataset. However, these results clearly need improvement if they are to be useful for any real task.In addition, we propose a named entity recognition framework that utilizes the information on the organism in articles. We present a quantitative analysis of how the extraction of physical interactions is influenced by the errors caused by the named entity recognition module. We point out that the framework is extremely important because, in the interaction curation task, protein names must be normalized to molecule identifiers so that molecular properties such as sequence can easily be identified. Finally, a profile-based method is proposed for the IPS. The goal of the task is to extract the protein pairs that have experimentally verified evidence, which requires the curator to collect information from multiple sentences in the article. This goal is different from the general aim of PPI extraction systems, most of which extract PPIs at the sentence level. Inspired by experience in curation, we constructed a profile vector for each candidate interaction from the whole article. By integrating evidence from the whole article, a better prediction is achieved for robust interaction curation. Our results were ranked in first place in terms of FAutomatically filtering out articles that are irrelevant to interaction curation will be useful to database curators. According to the reports from database curation projects, it takes 2 to 3 hours to process a paper even for highly qualified curators . The IAS1 score), which was also observed by ) between two distributions to measure the divergence between distributions , the probability of a feature x occurring in the relevant articles, there was no remarkable difference between the term distributions estimated from the leave-out dataset, the remaining training dataset, or official test dataset. In other words, the three different datasets have almost the same term distribution. However, significant differences were observed for Pr(x|c-), the probability that a feature x appeared in the irrelevant articles, whose distributions are illustrated in Figure For Pr, and some are completely different and can easily be filtered out. It is difficult to provide a good set of representative irrelevant articles, and so these irrelevant articles introduce more uncertainty and bias into the learning machines.In an attempt to overcome the problem, we first took strings as features based on the above analysis. Furthermore, we propose a new scheme (defined by Equation 5 [see below]) to diminish the divergence between the training data and the test data. The new scheme takes into account the probability of a feature being observed in both relevant articles and in irrelevant ones, instead of simply using TF*IDF (term frequency × inverse document frequency). Alternatively, we tried to incorporate more high-level semantic features such as the named entity features and template features. The entities were recognized using ABNER  , includiWe first studied how the features influence performance in terms of classification using a support vector machine (SVM) with a linear kernel as the classification model Table . The strSecond, we studied how this issue is influenced by different classification models, each model analyzing the data from a different point of view. SVM learns to separate data by a decision hyperplane, whereas the naïve Bayes classifier and multinomial classifier estimate probability distributions and try to interpret data from the probability perspective. The linear kernel SVM requires the data to be represented as feature vectors, whereas the p-spectrum kernel SVM simply views an example as a string. The different description powers can be combined by AdaBoost  and the t-test experiments to determine whether the observed improvements were statistically significant. More details are presented in our paper published elsewhere [Readers should note that the results presented in Table lsewhere .It is extremely useful to normalize protein names with molecule identifiers, which will largely ease the process of interaction curation. However, the task is challenging because inconsistent naming terminologies are used. It is common in reported research to cite just a few, nonstandard abbreviations, or to mention proteins without specifying species or organisms, or without specifying isoforms. This problem can be exemplified as follows.1. Common terms, such as p53, are not easily normalized without any contextual information.P42337), human (P42336), and cow (P32871), whose genes have the same term PIK3CA.3. The same term is used to name different isoforms of molecules. For example, in mouse PI3K refers to both Q8BTI9 (the β isoform of the protein) and O35904 (the δ isoform). There are two important steps in normalizing names to SwissProt identifiers. First, the terms of database entries must be curated to canonical forms, and the new terms used to detect the mention of proteins. Second, the ambiguities of multiple mapping of protein mentions to molecule identifiers should be removed by using organism and contextual information. The following rules are used to curate database entries.2. The same term is used to name different molecules from the same or related genes but different organisms. For example, PI3K may refer to different molecules in mouse . Also, our method outperforms traditional template-based methods, considering that ONBIRES  represents the state-of-the-art performance [The results shown in Table formance .There are reasons for the success of our profile-based method in this particular task. First, by integrating evidence from the whole article, the method is more robust when extracting physical interactions. For example, the sentence 'A interacts with B' will definitely be taken as a positive example by template-based methods, although it may describe a genetic interaction. In the profile-based method other evidence is required to make such a claim. Second, abundant features such as term features, entity features, template features, and position features are all integrated into the method. Here, we analyze the errors in detail in order to identify the problems hindering overall performance. There are 798 manually annotated interaction pairs in the 358 test articles, and although 339 protein pairs were extracted, only 100 of these are true positive pairs. There were 8,172 pairs that co-exist, and many of these include incorrectly recognized names -P37231 (PPAR gamma), and although we correctly extracted the interaction, the names were unfortunately normalized to Q6P9Z1-O19052.1. Twenty-two errors were due to incorrectly normalized names. For example, in the sentence 'BAF60c interacts directly with PPAR gamma 2. Twelve errors were due to false-positive names in sentences where protein A and B physically interacted, and a false positive recognized protein C coupled to A or B.3. Sixteen errors were due to the classifier, which included classifying nonphysical interaction pairs as exhibiting a physical interaction and other problems. This problem is partly due to the classification model and partly to the incompleteness of the training set, which does not provide evidence of samples that truly interacted.Among the 698 false negative errors (areas II + V), the majority were caused by the identification and normalization of protein mentions (532 errors), whereas 166 were due to the interaction extraction model. In the 166 pairs that co-existed, we found that 37 were negatively classified because the sentences did not contain sufficient evidence. Examples of such sentences include 'A activates B' or 'Camptothecin-induced nuclear export of A does not require B'. This problem is also due to the fact that the evidence of physical interactions is not confined to a single sentence. The remaining 129 errors are believed to be caused by our classifier.From these analysis, we conclude that the difficulty of protein name normalization leads to the majority of errors, producing about 64% (34/50) false-positive errors and 76.2% (532/698) false-negative errors. The second problem is that of incomplete annotation. Because the annotation only specifies the interacted protein ID in the article without the passages providing evidence or the location of these molecules, it makes the training process untraceable and the process of error analysis extremely difficult. Currently, a major limitation of our method is the requirement of protein coincidence within a sentence. This is not always the case in practical interaction curation, where curators often find evidence from contextual sentences, each of which may contain only one protein of the interacting pair. For example, the sentence 'the two proteins are co-purified together' may describe a physical interaction, even though both proteins are mentioned in the preceding sentences instead of in this sentence itself.In this report we discuss three key issues related to practical interaction curation. Specifically, we deal with filtering articles irrelevant to physical PPIs, identifying the mention of proteins and normalizing them to molecule identifiers, and extracting experimentally verified interactions. Different levels of features, including the string, term, named entity, and template features, are exploited to study the problem of distribution divergence between the training and test data. An AdaBoost-based information fusion technique is studied to integrate the various powers of description of different classifiers. Through these improvements, high-performance article filtering produces a system that may facilitate the process of interaction curation. Although the current state of protein name identification and normalization leaves much to be desired, our method utilizes the organism information to reduce ambiguity and, with further improvements, it will aid biologists. The profile-based interaction extraction method combines evidence from sentences in the whole document, thereby providing some improvement in the ability to predict physical interactions. In comparison with traditional methods that extract interactions at the sentence level, our method utilizes information from the whole article.There are still many difficulties and challenges in extracting biologically meaningful knowledge, for example recognizing biological molecules with widely accepted identifiers and mining physical interactions with experimentally verified evidence. This report provides methods to help resolve these problems both from the perspective of the feature and that of the classifier.We first present the architecture of our method . All ofd denotes an article, c+/c- denotes relevant/irrelevant articles, wi indicates a feature, and RNB(d) is the output score indicating the degree of relevance. The multinomial model implies a different distribution:Where xi denotes the number of times that a feature wi appears in the document d. In these two models, we must estimate the probability of each feature appearing in both relevant and irrelevant articles. This can be easily implemented using the equation below:Where V is the total number of features, POS is the set of relevant documents, and TF is the frequency that the feature wi is observed in the document dj.Where These two models are called probabilistic models, because they interpret the data by estimating a probability distribution. As mentioned previously, to analyze the quantitative difference of two distributions, we define the average Kullback Leibler divergence as follows:p and q are two distributions over the random variable x. If the two distributions are identical, then the AKL is 0; otherwise it is positive.Where The SVM is a discriminative model, which constructs a hyperplane in the feature space to separate the data into categories. The classification decision is made by calculating the distance of a sample from the hyperplane and in this module; we investigated two types of SVM models. The first is a traditional SVM with a linear kernel and in which each sample is represented as a feature vector. Instead of using TF*IDF, we proposed a new computational scheme to overcome the issue of divergence between the training set and test set, as follows:TF is the frequency of the feature wi observed in the document dj. The computational scheme performs much better than TF*IDF in the benchmark evaluation. The decision variable in the SVM model is as follows:Where SV means the support vectors. The kernel function provides an alternative mechanism to represent data in a composite manner in addition to the feature-vector representation. For instance, the p-spectrum kernel computes the number of common substrings shared by two input samples [Where  samples :x and y are two strings (or documents) defined in the alphabet Θ, and Θp indicates all possible substrings of length p. In our method, we take p = 7, which is about 1.5 times the average length of unigrams. An example of how to construct string features is shown in Table Where Identifying protein mentions and normalizing them to molecule identifiers is a necessary step toward the extraction of protein interactions. In contrast to traditional named entity recognition tasks, this task requires the submitted protein pairs be mapped to unique SwissProt identifiers rather than presenting the original names in the text. We not only must identify named entities but also we must map them to unique molecule identifiers. As shown in Figure After curation, there are in total 230,000 protein identifiers, and more than 1 million terms. Obviously, it is not feasible to use all of the terms during the dictionary-based matching process. Moreover, the same terms, particularly abbreviations, may correspond to many protein identifiers. This is common when the same gene products only differ in organisms, and thus the organism context is crucial to remove such ambiguities. We first detect the organism information in an article, and then this information is used to rule out irrelevant database entries and to remove ambiguities when the terms are mapped to multiple protein identifiers. Our assumption here is that physical interactions described in one paper should occur only within a few organisms. The organism database used here is the NCBI  taxonomy . DictionProfile-based PPI extraction moduleTo extract experimentally verified physical interactions in practical interaction curation, curators usually collect evidence from multiple sentences. Previous methods to extract protein interactions are all at the sentence level, where each sentence is processed independently, and thus they fail to synthesize information from multiple sentences. Our profile-based method is able to exploit profile features from multiple sources throughout the whole document, and for each candidate interacting protein pair a profile vector is constructed from multiple sentences. In comparison with traditional methods, the profile-based method is more robust for the IPS of BioCreative 2006 and gained the best and the second best results on the SwissProt-only subset and the entire dataset, respectively.Every protein pair coincident in a sentence is viewed as an interaction candidate. For each pair, profile features are calculated from all the sentences in which the pair coincides. The corresponding bit is set to 1 if the feature is found in these sentences Figure . Throughχ2 test, and 68 are taken manually from the branches of the physical interaction and detection method in the Molecular Interaction (MI) ontology [1. One hundred and sixty-eight unigram/bigram features. One hundred of these features are selected using the ontology .1 * bind to * Protein2', where * means that any word can be omitted. Some template examples are listed in Table 2. Ninety-one template features. These features are generated in a semi-supervised manner  and thei3. Two position features. One of which is whether the two proteins coincide in the title and the other is whether they coincide in the abstract.1 binds to Protein2', does not necessarily indicate the existence of a physical interaction. However, if there is other evidence, such as 'The binding of Protein1 to Protein2 is determined by Y2H', then the interaction is more trustworthy. Clearly, more evidence will strengthen confidence in the interaction. In addition, our algorithm is more robust when the performance of the molecule recognition module is far from satisfactory. For example, in the sentence 'The Y2H experiment proved the interaction between Protein1 and Protein2, CGA ...', CGA, whichis the sequence of Protein2, will be recognized as chromogranin A precursor, and then it will coincide with Protein1 and Protein2. The previous methods will fail, and although these false pairs are less statistically significant across the whole document, our method is able to resolve the problem by incorporating evidence from multiple sentences.Our method is more robust than the traditional methods because a single description, such as 'ProteinAUC, area under the receiving operator characteristic curve; GN, gene normalization; IAS, interaction article subtask; IMS, interaction method subtask; IPS, interaction pair subtask; MI, Molecular Interaction; PPI, protein-protein interaction; SVM, support vector machine; TF*IDF, term frequency × inverse document frequency weighting.MH designed the framework and wrote the manuscript. SD implemented the named entity recognition and interaction extraction modules. HW finished the algorithms and methods for the article filtering. XZ made many valuable suggestions and provided much support throughout this work.


Researchers interested in analysing the expression patterns of functionally related genes usually hope to improve the accuracy of their results beyond the boundaries of currently available experimental data. Gene ontology (GO) data provides a novel way to measure the functional relationship between gene products. Many approaches have been reported for calculating the similarities between two GO terms, known as semantic similarities. However, biologists are more interested in the relationship between gene products than in the scores linking the GO terms. To highlight the relationships among genes, recent studies have focused on functional similarities.S. cerevisiae. The receiver operating characteristics (ROC) and correlation coefficient analysis of these methods showed that the maximum method outperformed the other methods. Statistical comparison of multiple- and single-term annotated proteins in biological process ontology indicated that genes with multiple GO terms may be more reliable for separating true positives from noise.In this study, we evaluated five functional similarity methods using both protein-protein interaction (PPI) and expression data of This study demonstrated the reliability of current approaches that elevate the similarity of GO terms to the similarity of proteins. Suggestions for further improvements in functional similarity analysis are also provided. Gene ontology (GO)  describeet al. [In semantic-based applications, it is necessary to compute the similarity among GO terms before investigating the similarity between gene products. Fortunately, the method for calculating two terms in the well-known semantic tree WordNet  has beenet al.  found thet al. ,5 to infet al. ,6. Similet al. . There het al. -12 in faet al.  and Lin et al.  since thet al. [S. cerevisiae. The five above-mentioned methods were compared, and we hope that the results would provide a basis for further GO-based similarity studies.Biologists are more interested in the relationship of gene products than in the scores of GO terms. Various functional similarity approaches have been attempted. In this paper, these methods are referred to as 'Max' ,9,15, 'Aet al.  have evaet al. . Unfortuet al.  and geneet al.  of S. ceS. cerevisiae protein interactions were retrieved from the Database of Interacting Proteins (DIP) [The functional similarity methods introduced above and abbreviated as Max, Ave, Tao, Schlicker and Wang were tested by the receiver operating characteristics (ROC) analysis. ROC grades the performance of classifiers and rankers as a trade-off between specificity and sensitivity. The area under the ROC curve (AUC) is often taken as a measure of the prediction performance. An area of 0.5 represents random forecasts, while an area of 1 reflects perfect forecasts. A total of 6,459 ns (DIP)  and filtUnexpectedly, the Max method consistently showed the best performance in spite of the fact that the performances of all measures were barely distinguishable in MF ontology Fig. . The AUCTo highlight the contribution of multiple-term annotations, we compared the ROC curves derived from BP and observed that all methods performed better when single-term annotations were abolished Fig. . HoweverThe PPIs from the Munich Information Center for Protein Sequences (MIPS) Comprehensive Yeast Genome Database CYGD  . A plausible reason for this is that 79 expression profiles in Eisen's data may not be sufficiently sensitive for detecting the interacting proteins; therefore, more expression data would be required. On the other hand, false positives are unavoidable in the PPI dataset. Another possibility is that these genes are simply not related at the expression level. Therefore, the use of multiple standards (PPI and expression data) to evaluate the performance of functional similarity approaches has the advantage of both coverage and reliability.In this study, we used the expression and PPI datasets of reported ,6,25-28.CDC20 and CDC26 also have unique annotations such as 'anaphase-promoting complex activation during mitotic cell cycle' and 'protein ubiquitination', which led to some low semantic scores, and the Ave method was tuned to obtain a score of 1.9. Moreover, two interacting proteins SEC23 and BOS1 (described in the DIP dataset) have several annotations in which 'ER to Golgi vesicle-mediated transport' is their shared term. According to an earlier report [In all tests, the Max method consistently showed the best performance. Shared annotations or closely related GO terms, which lead to high semantic scores, was probably one of the reasons. There were 2,622 gene pairs  that had semantic scores of more than 6, and 2,278  were contributed by shared GO terms in the multiple annotation dataset of BP ontology. To obtain biological details, two case studies are presented, one of which was supported by expression data and the other by PPI data. First, we considered two cell division control proteins CDC20 and CDC26 that have similar expression profiles. Although there is no PPI information on these two genes, their significant coexpression correlation value of 0.77 suggested functional relationships, which was also indicated by their shared GO annotations  terms, 7,879 molecular functions (MF) terms and 2,019 cellular component (CC) terms. These vocabularies possessed a spindle distribution along 16 levels of depth, where the 8th level contained most of the terms. Genes inferred from electronic annotation (IEA) were eliminated from further analysis due to the lack of reliability.S. cerevisiae from DIP (release date: 7 October 2007). Considering that genes annotated with terms from the top levels of the directed acyclic graph (DAG) structure of GO would create noise, only those terms starting from the 3rd level and below (3rd–16th levels) were retrieved, resulting in 5,946 protein pairs  from BP, 4,267 pairs  from MF, and 6,121 pairs  from CC ontology. These were used as positive datasets for ROC curve analysis. For ALL (root ontology), 4,088 protein pairs that designated 1,850 proteins were used. Note that the ALL dataset contained the least number of proteins because the genes should simultaneously have the 3rd to 16th levels of annotations in BP, MF and CC ontologies.We retrieved 6,459 distinct PPIs  in et al. that contained 79 gene expression profiles of S. cerevisiae. Since most of the proteins in the Eisen dataset were well annotated in GO, 2,461 non-IEA annotated genes were obtained.The expression dataset was taken from the study of Eisen ROC grades the performance of classifiers and rankers as a trade-off between specificity and sensitivity. The positive datasets are described above. The negative datasets with the same number of protein pairs were generated by randomly choosing proteins from the non-positive genes located in the GO annotation files. To distinguish the reliability of multiple GO annotations and single GO annotations, positive and negative datasets of PPI containing 2,414 protein pairs annotated by multiple GO terms were built for BP ontology. The ROC and ROCR libraries in the R programming language were employed to calculate the AUCs and draw the graphs .Millions of gene pairs were derived from the Eisen  dataset TX, YZ and LD jointly designed the study, TX participated in the evaluation analysis, TX and YZ drafted the manuscript. All authors read and approved the final manuscript.Improvement of ROC curves after elimination of single annotations from the DIP dataset in the MF and CC ontologies. The four methods  are plotted in separate colours. The performances of the methods after eliminating single annotations are shown by the solid lines. Results based on the original dataset are shown by the dashed lines. The Schlicker method was not shown because it requires more ontologies than BP.Click here for fileROC curves of PPI evaluations. ROC evaluations of functional similarity measures based on the S. cerevisiae PPI dataset derived from CYGD are shown. The evaluation was done in (a) biological process (BP), (b) molecular function (MF), (c) cellular component (CC) and (d) ALL (root ontology). Since the Schlicker method requires all three ontologies, it is only suitable in ALL.Click here for fileImprovement of ROC curves after elimination of single annotations from the CYGD dataset in the BP, MF and CC ontologies. The four methods  are plotted in separate colours. The performances of the methods after eliminating single annotations are shown by the solid lines. Results based on the original dataset are shown by the dashed lines. The Schlicker method was not shown because it requires more ontologies than BP.Click here for file


As experimental techniques for detection and validation of protein interactions are time consuming, there is a need for computational methods for this task. Protein interactions appear to form a network with a relatively high degree of local clustering. In this paper we exploit this clustering by suggesting a score based on triplets of observed protein interactions. The score utilises both protein characteristics and network properties. Our score based on triplets is shown to complement existing techniques for predicting protein interactions, outperforming them on data sets which display a high degree of clustering. The predicted interactions score highly against test measures for accuracy. Compared to a similar score derived from pairwise interactions only, the triplet score displays higher sensitivity and specificity. By looking at specific examples, we show how an experimental set of interactions can be enriched and validated. As part of this work we also examine the effect of different prior databases upon the accuracy of prediction and find that the interactions from the same kingdom give better results than from across kingdoms, suggesting that there may be fundamental differences between the networks. These results all emphasize that network structure is important and helps in the accurate prediction of protein interactions. The protein interaction data set and the program used in our analysis, and a list of predictions and validations, are available at  For understanding the complex activities within an organism, a complete and error-free network of protein interactions which occur in the organism would be a significant step forward. The large amount of experimentally derived data now available has provided us with a chance to study the complicated behaviour of protein interactions. The power of such studies, however, has been limited due to the high false positive and false negative rates in the datasets. We propose a network-based method, taking advantage of the tendency of clustering in protein interaction networks, to validate experimental data and to predict unknown interactions. The integration of multiple protein characteristics  allows our predictive method to significantly outperform two other approaches based on homology and protein-domain relationships on datasets which contain a large amount of interactions, but not much detailed information on the proteins involved in the interactions. In addition, our predictive score based on triadic interaction patterns improves over a pair-wise approach, suggesting the importance of network structure. Moreover, using pooled interactions as prior information, we find evidence for fundamental differences in protein interaction networks between eukaryotes and prokaryotes. For understanding the complex activities within an organism, a complete and error-free network of protein interactions which occur in the organism would be a significant step forward. Experimentally, protein interactions can be detected by a number of techniques, and the data is publicly available from several databases such as DIP, Database of Interacting Proteins Computational methods have already been developed for these purposes. For interaction validation, these have mainly centered on the use of expression data For prediction of protein interactions in contrast, many methods have been suggested. The majority of these generate lists of proteins with a functional relationship rather than physical interactions In terms of physical interaction prediction the available methods can be typified by the two approaches of Deng et al. S.cerevisiae. However, the limited number of known domains may well not be enough to describe the variety of protein interactions. This approach has had further extensions, such as an improved scoring for domain interactions S.cerevisiae, two other organisms, C.elegans, D.melanogaster, are included.In Deng et al.'s method, a domain interaction based approach, a protein interaction is inferred on the basis of domain contacts. If a domain pair is frequently found in observed protein interactions, it is likely that other protein pairs containing this domain pair might also interact. From the observed protein interaction network, the probabilities of domain-domain interactions are estimated. The expectation-maximum algorithm is employed to compute maximum likelihood estimates, assuming that protein interactions occur independently of each other. This likelihood is then used to construct a probability score for a protein pair to interact, it is inferred based on the estimated probabilities of domain interactions within the protein pair. Deng et al.'s prediction is based on a total of 5,719 interactions from The second type of approach, as used by Jonsson et al. In principle, statistical clustering algorithms such as Neither Deng et al.'s method nor Jonsson et al.'s method make use of network structure beyond pairwise interactions; interactions are considered as isolated pairs. However these pairs could and should be considered as a network, where the proteins are nodes and their interactions are links x with y, where both x and y interact with a third protein z. Therefore in our approach we particularly focus on two simple three node network structures, triangles and lines. A triangle is a subnet formed by an interacting protein pair with a common neighbour. A line, by contrast, is a subnet formed by an non-interacting protein pair with a common neighbour. We will show that these network structures and the protein characteristics within them help to predict protein interactions.Our goal is to predict (undirected) protein interactions of the type S.cerevisiae interaction network from the DIP database. During the validation we assume that function and structure are known for all proteins (fully annotated) and that the protein interaction network is known for all but one interaction. With triadic interacting patterns, we predict the interaction status of those protein pairs with at least one common neighbour and compare our results with those from three other published scores. We go on to demonstrate that the requirement to have fully annotated proteins can be relaxed to include partially annotated proteins, with a slight drop in the accuracy. The prediction is also compared with simulated networks where all proteins are shuffled while the network structure is maintained, in order to examine whether the specific network structure, triangles and lines, keep useful information in forming protein interaction networks.We apply our method to the To measure the true positive rate in a set of protein pairs, Deane et al Our predictive model uses a prior interaction database and for this we use three prior databases, pooling protein interactions collected from prokaryotes, eukaryotes and all interactions. The results from using different prior databases show that the use of interactions from within the same kingdom rather than across kingdoms significantly improves the results, indicating as in Comparing our method to three other standard approaches, namely the domain-based approach by Deng et al. and an extension by Liu et al., and a homology-based approach by Jonsson et al., we find that our method outperforms the above approaches on the subset of interactions in the DIP Yeast data set which contains enough annotation and connectivity to be included in our analysis. Our method complements the methods by Deng et al. and Liu et al., as their approaches apply to a rather different subset of potential interactions yielded from the DIP Yeast data set.S.cerevisiae, excluding self-interactions, are obtained from DIP (DIP Yeast). Self-interactions  are excluded, implying that all triangles and lines are constructed of three different proteins. Three different prior data bases are constructed by pooling interactions considering eukaryotes , prokaryotes (E.coli and H.pylori), or all interactions; the interaction we would like to predict or to validate is always excluded.Experimental protein interactions of The proteins in our dataset are classified into the seven SCOP classes We use the 24 functional groups from the secondary level of Molecular Function in the Gene Ontology A, B, C and D denote protein characteristics, whereas different shapes indicate different proteins. A protein may possess more than one characteristic. Our triplets are triangles and lines of three characteristic vectors according to their interacting patterns. A characteristic line is a specific pattern constructed by three vectors with two vector interactions among them. A characteristic triangle is formed by three vectors interacting with each other.The protein interaction network is used to build an upcast set of triplets of characteristic vectors as in Here we abuse the English language; while it would be clearer to say “pair of characteristics” and “triangle of characteristics” we prefer the shorter version “characteristic pair” and “characteristic triangle” for easier reading.x, its characteristic vector cS (x) contains all its characteristics of a certain type , and S(x) denotes the set of vectors formed using different characteristics. In the case of two protein characteristics, S1 (x) and S2 (x) are the two respective vectors, and S(x) is the setTo assess our method we also compare it with a score based on characteristic pairs only. In a similar manner to the upcast set of characteristic triplets, we construct an upcast set of characteristic pairs. Here we grasp the opportunity to introduce some notation. For a protein S; this set may contain a vector av multiple times.We shall denote the set of all characteristic vectors for all proteins by characteristic pair is constructed by two characteristic vectors from two interacting proteins. If two proteins x and y interact, for each pair {aν, bν} with aν ∈ S(x), bν ∈ S(y), we write aν∼bν. If two protein do not interact, the relation between two vectors is denoted by aυ≁bυ. The upcast set of characteristic pairs is then the collection of all characteristic pairs extracted from the protein interaction network, which may stem from one or from multiple organisms.A For our upcast sets to be informative for a protein interaction, an eligible protein pair has to satisfy two conditions: Firstly, the proteins need to have at least one common interacting neighbour; and secondly, the query protein pair and the neighbours have to be at least partially annotated.Among 4,931 proteins in the observed interaction network, 2,416 (49%) proteins are fully annotated with both characteristics (structure and function) and 3,808 (77%) are annotated with at least one characteristic.triangle rate score from the upcast sets of characteristic triplets. This score thus includes information not only from the query protein pair but also from its neighbours. Therefore, it is a network-based score which goes beyond pairwise interactions.We derive our xyt be the total frequency of all characteristic triangles around the query protein pair {x,y}; denoting by z ∈ B the set of all common neighbours of x and y in the protein interaction network,f (aν∼cν∼bν∼aν) is the frequency of triangle {aν∼cν∼bν∼aν} among all characteristic triangles in the prior data base. Similarly, xyl is the total frequency of all characteristic lines around the query protein pair {x, y}. We define the triangle rate score, tri for the protein pair {x, y}. as the odds of observing triangles versus lines among triangles and lines in its neighbourhood,Within the triplet interactions, we assess the odds to observe triangles versus lines around the query protein pair. More formally, let When multiple characteristics are simultaneously included, the triangle rate score defined above requires the query protein pair and the common neighbour to be fully annotated with multiple characteristics. However, there are many partially annotated proteins in the neighbourhood which may provide useful information. These proteins are particularly important when only a few fully annotated ones are available. In Supporting Information , C, an epair-based score. The details are as follows.To assess whether the triangle rate score significantly improves prediction and validation, we also construct a similar score based on pairwise interactions only, which we call the Based on the pairwise interactions, we also provide an odds ratio-based score, see also The number of proteins in each type of characteristic vector is multinomially distributed.aν, bν} is binomially distributed, with the probability of success abπ being the probability of interaction between the proteins in the pair, and these binomial random variables are independent.Given the total number of characteristic pairs which can be derived from the frequency of characteristic vectors, the number of actual interactions for each type of characteristic pair {aν, bν}, under the multinomial-binomial model above the maximum likelihood estimate for abπ is given byabo is the number of times an interaction has been observed for the characteristic pair {aν, bν}, and abn is the number of times that no interaction was observed for the pair {aν, bν}.Given a specific characteristic pair {pair-based score for a query protein pair {x, y} aspair is the average of the estimated probabilities πˆab for all characteristic pairs generated by the query protein pair in the prior data base. Heuristically, the higher the score, the more likely it should be that the two query proteins interact. An extended version of the score is able to cover protein pairs which are only partially annotated, see Supporting Information  and the specificity is the ratio of true negatives over . To assess our scores we first use a Receiver Operating Characteristic (ROC) curve, which is a useful technique for examining the performance of a classifier In order to put our scores to work we choose a threshold; all pairs with scores above that threshold would be classified as interacting, while all pairs below that threshold would be classified as non-interacting. The choice of threshold depends on the desired sensitivity and specificity; recall that the While we are never completely certain that a prediction is correct, we assume that a positive prediction is correct if it is contained in our gold-standard positive (GSP) set, and that a negative prediction is correct if it is contained in our gold-standard negative (GSN) set. The GSP set is based on 8,250 hand-curated interactions in MIPS complexes catalog The set of gold-standard negatives (GSN) are random protein pairs which neither share protein localisation, nor expression nor homologous interaction data We have many more gold-standard negatives than positives. The unequal sizes of gold-standard sets may affect the ROC curve; when the cutoff is high, too many gold-standard negatives would cause a rapid increase in true negatives, which would result in artificially high specificity. To avoid this bias, we collect 300 samples of randomly selected pairs from the extensive GSN. Each sample is the same size as our GSP set. Predictions are verified against these 300 reference sets obtained by combining the GSP set and the sample from the GSN set.z-test for differences. Since the AUC is limited by a unit square, its value will be between 0 and 1.0. While there is a possibility for a correlation between the AUC of two samples, randomly generating 1,000 samples of two sets of 30 random samples from the set of 300 AUC values, no significant covariance was detected for any of the scores under consideration. Hence assuming that our 300 samples are approximately independent, from the Central Limit Theorem the average AUC should be approximately normally distributed. Therefore here we used a z-test to compare the mean difference between the 300 AUC from two scores. If the difference between two mean AUC is too large then we reject the null hypothesis that two AUC are equal and conclude that there is evidence that one ROC curve is significantly better than the other one. Here we not only use tests at 5% significance level; but we also give the p-values of the tests. For details of the z-test see Supporting Information (In order to differentiate the ROC curves of the different predictors we have developed a method to compare the areas under two curves (AUC) ormation , D.precision is the ratio of true positives over , whereas the recall is the ratio of true positives over , i.e. the sensitivity. The P-ROC curve plots recall against precision. While there is a tendency for recall and precision to be inversely related, Precision-Recall curves are not necessarily decreasing. An increasing P-ROC curve is an indication for perverse retrieval, in which there is a strong tendency that first the negative interactions are retrieved; only when there are so few of those left that it is almost unavoidable to retrieve positive interactions, these are also covered; see for example When evaluating performance for a classifier when the test data is unbalanced, such as when there is a disproportionate number of negative versus positive cases, instead of choosing subsamples of the same size as for our tests between two ROC curves, the Precision-Recall Operating Characteristic (P-ROC) curve provides an alternative. The Initially we compare our method to the methods suggested by Deng et al. We compare our triangle rate score with three other methods, the two by Deng et al. E. coli, H. pylori, C. elegans, D. melanogaster, M. musculus and H.sapiens from DIP for the search of similar sequences), and additionally using orthologs and paralogs  and homology-based scores, see p-values. Here the comparison with the domain-based methods has to be taken with a pinch of salt, as the amount of overlap between the eligible pairs for those methods and our method is very small.The comparison of scores are shown in The P-ROC curve in The number of predictions which overlap with the MIPS-GSP  is also an indicator of coverage. Our triangle rate score is able to predict 928 of them, which is the largest number of predictions from any of the four sets. Deng et al. and Liu et al.'s scores, based on protein-domain relationships, can only predict 85 and 174 interactions in GSP respectively. Their methods cannot predict protein pairs without domain information, limiting their coverage. Liu et al.'s score, when including information from other organisms, improves the coverage over Deng et al.'s score, but not the overall performance in terms of AUC. Jonsson et al.'s score covers more interactions in GSP (390 interactions) than the domain interaction based approaches, however, it appears to perform worse in terms of AUC, though not significantly. Jonsson et al.'s method is still limited in coverage, however, because only sequences with very high similarity are useful for transferring interactions, and often qualified homologs are not available, see We also compare our triangle rate score to the pair-based score, thus allowing us to ascertain the effect of network structure on our scoring method. The ROC curves in We have also employed a logistic regression model to include pair- and triplet-based statistics, see Supporting Information , E for dThe triangle rate score can be used to validate experimentally derived interactions. It is estimated that the false positive rates for high-throughput experiments vary from 35 to 83% dependent on source At a cut-off score value of 0.09, our prediction reaches 0.83 for both sensitivity and the specificity. Of the 2,896 DIP Yeast interactions tested by the triangle rate score, 1,732 (60%) are validated at the score cut off of 0.09. This gives an estimated false positive rate of around 40%, close to that given by EPR We also calculate the EPR index (% correct) for subsets of our predictions. The EPR index estimates the biologically relevant fraction of protein interactions detected in a high throughput screen. As the EPR index is between 70–80% for DIP CORE, we cannot hope for a correct prediction rate (fraction of true predictions over true positives) higher than 70–80%. Indeed this upper limit is reflected by a sharp drop-off in the ROC curve  for  that are not observed in DIP Yeast. Among these pairs, about 80% are co-localised and 60% share the same function. Indeed, half of them share both function and subcellular location. These findings indicate that such highly scoring protein pairs are likely interactors.Among five randomly chosen pairs, two were confirmed by manually checking other public protein databases such as BioGRID The two proteins “ATP synthase subunit 4” (YPL078C) and “ATP synthase subunit epsilon” (YPL271W) are both in the DIP database, but DIP does not record an interaction between them. The pair receives a very high triangle rate score, suggesting a possible interaction. From BioGRID and Pubmed, we find that their interaction is confirmed in a co-purification experiment and in the literature Our second example is the pair “Transcription initiation factor TFIID subunit 1” (YGR274C) and “Transcription initiation factor IIA small subunit” (YKL058W), which also has a high triangle rate score. Both share transcriptional activation as on of their functions; their positive interaction can be verified in the literature in We can also consider the converse, using the triangle rate score to validate a stated interaction, with the aim to identify potentially false positives. We examined our lowest scoring 5% ; 49 of which are found in DIP Yeast. Among these 49 pairs, 42 do not share the same function. There are 11 pairs that share neither function nor subcellular location. One example is the interaction between “Protein TEM1” (TEM1) and “Long-chain-fatty-acid–CoA ligase 4” (FAA4). The database entry is based on Yeast two-hybrid experiments, a particularly error-prone experimental technique. While TEM1 is located in cytoskeleton, endoplasmic reticulum, or punctate composite, FAA4 is in cytoplasm. In terms of functional categories, TEM1 involves in nucleotide binding and in hydrolase activity, and FAA4 is in long-chain-fatty-acid-CoA ligase activity. These two proteins are located differently and share no common function, raising a question mark on whether they indeed interact. False positive interactions could arise from several reasons, such as autoactivation of reporter transcription by the bait protein alone. We suggest that a small-scale experiment should be carried out on this specific protein pair.A list of the high scoring protein pairs which are not in DIP and a list of low scoring pairs which are in DIP are provided in Supplementary Information The triangle rate score can be extended to gather information from partially annotated proteins; see Supporting Information , C. The The accuracy is the fraction of correct prediction out of all predictions against each of the 300 reference sets. Again, the 300 reference sets are employed to avoid the bias raised from too many negative pairs, i.e. a high accuracy may arise simply from making no positive prediction.E.coli and H.pylori, and eukaryotes, including C.elegans, S.cerevisiae, D.melanogaster, M.musculus and H.sapiens, and a final global pooled dataset including all interactions. As a random background, we also generate a simulated interaction network by shuffling the annotation of proteins in the Yeast protein interaction network. Based on the five prior data bases - Yeast, eukaryotes, prokaryotes, all interactions, and a shuffled protein network, we predict protein interactions using the triangle rate score. The AUC for all curves are calculated and tested for differences, see To explore how different priors affect the prediction, we group protein interactions into prokaryotes, including The ROC curves show that the prior from Yeast itself gives the best prediction, followed by that from eukaryotes before third, all interactions; see χ2 test of homogeneity for triangles and lines in the two prior data bases. We compare characteristic triangles and lines that are annotated with structure, function and both, and group patterns with counts of at least 5. All 6 tests suggest a significant difference between eukaryotes and prokaryotes. This difference might arise from evolution and suggests that only priors from close organisms (within same kingdom) are helpful. It is not always beneficial to construct a large data base without taking the difference among organisms into account.The clearly different ROC curves from the eukaryotes prior and the prokaryotes prior suggest that their networks are very different, in terms of the interaction patterns of protein characteristics. We perform a The ROC curves for predicting interactions from shuffled protein network are close to diagonal, as is expected. Without the information from protein structure and function and the interacting patterns, the prediction is random. The different trends between using real data and simulated data show that the interacting patterns of protein structure and function play important roles in protein interactions.The P-ROC curve in The different performance of prokaryotic and eukaryotic priors relates to their networks being rather different with respect to their distributions of protein structure and also of protein function. The most striking difference relates to small proteins. While 15% of eukaryote proteins are small proteins, less than 1% of prokaryote proteins are small proteins. Among the 10 most frequently observed structure category interactions, in eukaryote 3 of them  involve small proteins, while in the list of top 10 structure category interactions in prokaryotes small protein related interactions do not appear. Another considerable difference concerns the distributions of the two functions “RNA polymerase II transcription factor activity” and “GTPase regulator activity”. While 4% of the eukaryotic proteins possess one of these two functions, they are not found in the prokaryotic proteins. In addition, in the list of top 10 most frequently observed function category interactions, in eukaryotic networks we observe many function category interactions with “protein binding” proteins, while they do not appear on the list of prokaryotes networks.With the triangle rate score we provide a novel statistical tool for prediction and validation of protein interactions. Our method uses triadic-level statistics, in addition to the traditional dyadic-level statistics arising pairwise interactions. This network-based method is shown to complement the existing domain-based approach, and to outperform the homology-based methods as well as a comparable pair-based method.As our method requires annotated proteins occurring interacting with at least two other proteins, currently the only data set which is large enough to warrant application is that of Yeast, see G in Combining our method with priors from other organisms allows us to compare protein interaction behaviour among kingdoms, from the viewpoint of comparative interactomics. The significant difference in protein interactions networks between eukaryotes and prokaryotes serves not only as a caution to integrate interaction information from only close organisms, but also as encouragement for further, micro-level study between the two upcast sets, hoping for more insight into the biological difference between two kingdoms.Text S1Supporting Information Text(1.13 MB DOC)Click here for additional data file.Table S1Estimates of r-bar from characteristic triplets(0.04 MB DOC)Click here for additional data file.Table S2Observed patterns vs. all possible patterns†(0.06 MB DOC)Click here for additional data file.Table S3List of 7 classes in SCOP(0.03 MB DOC)Click here for additional data file.Table S4List of 24 main functional groups(0.04 MB DOC)Click here for additional data file.Table S5Z-tests for AUC comparison from the triangle rate score with different priors(0.03 MB DOC)Click here for additional data file.Table S6Numbers of protein interactions(0.05 MB DOC)Click here for additional data file.Dataset S1The list of 652 protein pairs, with score among our top 1%, but not in DIP(0.01 MB TXT)Click here for additional data file.Dataset S2The list of 49 protein pairs, with score among our bottom 5%, but which are found in DIP(0.00 MB TXT)Click here for additional data file.


Semantic similarity measures are useful to assess the physiological relevance of protein-protein interactions (PPIs). They quantify similarity between proteins based on their function using annotation systems like the Gene Ontology (GO). Proteins that interact in the cell are likely to be in similar locations or involved in similar biological processes compared to proteins that do not interact. Thus the more semantically similar the gene function annotations are among the interacting proteins, more likely the interaction is physiologically relevant. However, most semantic similarity measures used for PPI confidence assessment do not consider the unequal depth of term hierarchies in different classes of cellular location, molecular function, and biological process ontologies of GO and thus may over-or under-estimate similarity.We describe an improved algorithm, Topological Clustering Semantic Similarity (TCSS), to compute semantic similarity between GO terms annotated to proteins in interaction datasets. Our algorithm, considers unequal depth of biological knowledge representation in different branches of the GO graph. The central idea is to divide the GO graph into sub-graphs and score PPIs higher if participating proteins belong to the same sub-graph as compared to if they belong to different sub-graphs.F1 score over Resnik, the next best method, on our Saccharomyces cerevisiae PPI dataset and 2 times on our Homo sapiens PPI dataset using cellular component, biological process and molecular function GO annotations.The TCSS algorithm performs better than other semantic similarity measurement techniques that we evaluated in terms of their performance on distinguishing true from false protein interactions, and correlation with gene expression and protein families. We show an average improvement of 4.6 times the  Gene Ontology (GO)  is a useSaccharomyces cerevisiae proteins Rpl10p  and Sqt1p (annotated to GO cellular component term 'ribosome') physically interact with each other but do not share a GO term. Often, a sub-set of GO terms or a reduced version of GO, like GO slim  by dividing with the maxi-mum information content in a sub-graph or meta-graph. For a term th isub-graph i tis defined as shown in equation (8).where mG, the information content (ICM) of i tis calculated as shown in equation (9).For a term A and B may be annotated to more than one GO term. Let, S and T be the sets of GO terms annotated to gene products A and B respectively. Then the semantic similarity between gene products A and B is defined by the maximum approach, as shown in equation (10).Gene products LCA is the lowest common ancestor (or the common ancestor with maximum information content) of the terms i sand j t. If both the terms is and tj belong to the same sub-graph then their lowest common ancestor will be in that sub-graph, otherwise it will belong to the meta-graph.where et al. (Wang) [et al. (simRel method) (Schlicker) [et al. (SimGIC) [F1 measures. ROC grades the performance of classifiers as a trade-off between true positive rate (TPR) and false positive rate (FPR). We also used the F1 measure, which is the harmonic mean of precision  and recall (the proportion of relevant information that is retrieved) and indicates the classifier's ability to retrieve relevant information. The evaluation was done separately for cellular component (CC), biological process (BP), and molecular function (MF) ontologies.In the previous section, we presented a new algorithm, Topological Clustering Semantic Similarity (TCSS), to compute semantic similarity between GO terms annotated to proteins that normalizes GO DAG branch depth. We compared the performance of TCSS with other semantic similarity measures given by Resnik , Lin 1111, Wang . (Wang) , Schlickhlicker) , Jiang &hlicker) , Pesquit(SimGIC)  on the pS. cerevisiae positive and negative protein interaction sets (see Methods) were used to evaluate the above mentioned semantic similarity measures for their ability to distinguish positives from negatives. TCSS, Resnik, Lin, Jiang and Schlicker were tested using both the maximum (MAX) and best-match average (BMA) (see Methods) approach of combining multiple GO gene annotations and Wang was tested using only the BMA approach, as only BMA was used in the original Wang publication and is the only option available in the author's implementation. BMA averages scores when multiple combinations of GO terms are possible (for gene products annotated with multiple terms). SimGIC considers multiple GO annotations while calculating semantic similarity scores, thus MAX and BMA approaches are not relevant for it. We focused initial tests on manually annotated GO annotations ("without" annotations with IEA evidence codes (IEA-)), but also tested with all annotations, including electronic annotations ("with" annotations with IEA evidence codes (IEA+)).F1 scores at different semantic similarity cutoffs for all the three ontologies  were used to evaluate TCSS, Resnik, Lin, Jiang, Schlicker and SimGIC methods. The evaluation was done using BMA and MAX approaches for combining multiple GO annotations on IEA+/-datasets  were evaluated against the above mentioned semantic similarity methods. The evaluation was performed as above using the BMA/MAX approaches of combining multiple GO annotations on IEA+ dataset. TCSS showed the best correlation between gene expression and semantic similarity with all three GO ontologies  website was developed by Pesquita et al.  to evaluF1 scores over Resnik, the next best method, on our S. cerevisiae PPI test and 2 times on our H. sapiens PPI test. This clearly indicates the advantage of using TCSS to retrieve positive protein interactions and hold back negative interactions over Resnik's method. We compared TCSS using both the BMA and MAX approaches for combining multiple GO annotations, and found that MAX generally works best for PPI datasets. The use of the MAX function to score PPIs, instead of an 'average' function, makes sense because proteins in PPIs only need to be in close proximity (similar cellular component terms) or in a similar biological process once, among all possible combinations annotation terms, to be biologically relevant. Therefore, the MAX approach is unlikely to overestimate true PPIs. However, there may be application scenarios  where the MAX approach could lead to over-estimation and BMA would be a better choice. In these cases, TCSS can be modified to use the BMA method instead of MAX (see Methods). For example, TCSS shows worse correlation with Pfam similarity than SimGIC on the biological process ontology test, but becomes better when using BMA  for calculating semantic similarity and tested its performance against other methods. TCSS shows an average improvement of 4.6 times in Scatter plots of the semantic similarity scores obtained by TCSS (MAX) and Resnik (MAX) methods clearly indicate that a significant number of positive protein interactions are under-scored by Resnik Figure  in all tCellular component: Rpl10p is a S. cerevisiae protein responsible for joining of the 40 S and 60 S ribosomal subunits [• subunits . It has subunits -38 with subunits  using afBiological process: The Nth1p-Dcs1p protein-protein interaction was experimentally shown by Uetz et al (2000) and Yu et al (2008) [• l (2008) ,40 usingMolecular function: Mft1p and Hpr1p are the subunits of the nuclear THO complex, which is involved in transcription elongation, mitotic recombination and telomere maintenance [• ntenance . The Mftntenance ,41-43. BFuture directions for TCSS development include testing if the GO graph edge type  can provide additional information that will lead to improved performance and also testing the method more rigorously with other data sets.S. cerevisiae and H. sapiens. We also tested the correlation between multiple semantic similarity scoring methods with gene expression, protein sequence, EC, and Pfam similarity. Performance tests were generally in favor of TCSS in all three GO ontologies: cellular component, biological process and molecular function.We present a new semantic similarity algorithm, Topological Clustering Semantic Similarity, designed to use the GO for PPI confidence assessment. It partitions the GO DAG into non-overlapping sub-graphs, using a topological clustering method, and computes semantic similarity normalized within each sub-graph. We evaluated TCSS against other methods for measuring semantic similarity between GO terms annotated to proteins involved in protein-protein interactions from This new method will be useful as an evidence source in PPI prediction or in confidence assessment of PPI datasets.Ontology data: Ontology data was down-loaded from the Gene Ontology database [• database  (dated MGO Annotation data: Gene annotations for GO terms were downloaded from the Gene Ontology database for S. cerevisiae (dated February 2010) [H. Sapiens (dated August 2010) [• ry 2010)  and H. Sst 2010) . ElectroInteraction dataset: To evaluate the performance of TCSS against other semantic similarity measures on the problem of scoring PPI confidence, we created positive and negative interaction datasets for S. cerevisiae and H. sapiens.• S. cerevisiae: We retrieved 4,598 unique pairwise S. cerevisiae PPIs from the core set of Database of Interacting Proteins (DIP) (dated December 2009) [-er 2009) . The DIPer 2009) . The poser 2009) .H. sapiens: We retrieved 2; 077 unique pairwise PPIs (with three or more publications) for H. sapiens from DIP (dated June 2010). The positive dataset for CC, BP, and MF ontologies comprised interactions with both proteins annotated to terms (other than root) in their respective ontologies . This was done independently for CC, BP, and MF annotations of GO (including IEA annotations).• eneMANIA   is an online tool for the automated evaluation of GO-based semantic similarity measures in terms of performance against sequence, Pfam (protein family) and EC (enzyme commission number) similarity [• milarity . ProteinF1measures) on a given benchmark/test. The relationship between AUC and topology cutoff follows a U - shaped curve with a global maximum for all three ontologies. Average F-score shows a general upward trend with topology cutoffs  is defined as (12),where et al. (simRel method) (Schlicker) [et al. (SimGIC) [et al. (Wang) [Semantic similarity measurement methods given by Resnik , Lin 1111, Schlihlicker) , Jiang &hlicker) , and Pes(SimGIC)  were imp(SimGIC)  implemen. (Wang) .Different measures used for analyzing the performance of our algorithm are as follows:• True positive rate (TPR), also known as Recall:• False positive rate (FPR):• Precision (P):1 measure (F):• F1 score is calculated as the average improvement at different semantic similarity cutoffs.• Improvement in F• Area under curve (AUC) was calculated using the trapezoidal rule.TP, FP, TN, FN are true positive, false positive, true negative, and false negative, respectively.where S. cerevisiae positive and negative interaction dataset using Fisher's z transformation [Average gene expression Pearson correlation was calculated for the ormation .n ris the Pearson correlation between two genes for the nth experiment. Then, an appropriate estimate of the true mean is calculated as,where N is the total number of experiments. Then, by inversion, average correlation is calculated as,where TCSS, Schlicker, Jiang, SimGIC and Resnik methods were used to find the semantic similarity between protein pairs provided by CESSM. Correlation between semantic similarity scores and sequence, Pfam, EC similarity for these methods was calculated using the CESSM online tool. Wang was not used here due to the difficulty in modifying the R implementation to use the datasets provided by the CESSM website.• Project name: Topological Clustering Semantic Similarity (TCSS)http://baderlab.org/Software/TCSS• Home page: • Operating system(s): Unix/Linux (recommended)• Programming language: Python• Other requirements: Python 2.6 or higher• License: GNU LGPL• Any restrictions to use by non-academics: noSJ conceived the study and carried out all programming and analysis as a Ph.D. student in the lab of GB. GB supervised and provided input on all aspects of the study. All authors read and approved the final manuscript.Supplementary figures and tables. The file contains supplementary figures S1 - S17 and tables S1 & S2.Click here for file