The ever-increasing flow of gene expression and protein-protein interaction (PPI) data has assisted in understanding the dynamics of the cell. The detection of functional modules is the first step in deciphering the apparent modularity of biological networks. However, most network-partitioning algorithms consider only the topological aspects and ignore the underlying functional relationships.In the current study we integrate proteomics and microarray data of yeast, in the form of a weighted PPI graph. We partition the enriched PPI network with the novel DetMod algorithm and we identify 335 modules. One of the main advantages of DetMod is that it manages to capture the inter-module cross-talk by allowing a controlled degree of overlap among the detected modules. The obtained modules are densely connected in terms of protein interactions, while their members share up to a high degree similar biological process GO terms.Moreover, known protein complexes are largely incorporated in the assessed modules. Finally, we display the prevalence of our method against modules resulting from other computational approaches.The successful integration of heterogeneous data and the concept of the proposed algorithm provide confident functional modules. We also proved that our approach is superior to methods restricted to PPI data only. One of the key issues left at the hands of bioinformatics to be solved, is the deciphering of the complex organization of biological networks. In recent years, many studies have focused on determining small-scale subnetworks with distinct functional role, called functional modules ,2. TowarFollowing this concept all state-of-the-art approaches have elaborated on protein-protein interaction (PPI) networks, where the in-between interactions reflect the direct collaboration of proteins to achieve a certain task. Nevertheless, this data is flooded with many false interactions, thus it is already established that functional modules descending solely from this data are often considered as unconfident and misleading. In literature several are the studies that concentrated on an unweighted PPI graph, despite the disadvantages posed by topology. The study of Rives and Galitski applied a hierarchical clustering algorithm based on shortest-distance as a metric to unravel the modular organization of yeast network . Spirin Fewer were the attempts to enrich the PPI topology with gene expression data in the form of a weighted graph. The underlying concept is that genes with similar expression profiles are under the same transcriptional control and functionally associated . NeverthRecently, the majority of contemporary studies integrate PPI and gene expression profiles to detect biologically meaningful clusters or modules ,14,22. HIn current study we propose a method for determining functional modules based on the integration of PPI and gene expression profiles. The extraction of functional modules is performed by a novel graph clustering algorithm named DetMod (Detect Modules) that overcomes all the drawbacks mentioned above. Firstly DetMod algorithm identifies valid modules and subsequently allows the modules to merge, in cases where the merging procedure leads to better results.One of the main characteristics of DetMod is that the extracted modules may display a controlled degree of overlap concerning their members, thus the inter-module crosstalk is preserved and a more realistic estimation of the protein network is acquired. In the literature there are other algorithmic approaches that can produce overlapping modules ,25. HoweWe validate the functional modules extracted from DetMod through biological and topological criteria, and by comparing our method with other PPI module detecting approaches and graph clustering algorithms .Saccharomyces cerevisiae that our method provides modules with functional and topological consistency and prevails over similar studies in literature.We prove based on data of We realized the concept of our approach based on yeast proteomics and microarray datasets. Firstly we integrated these datasets in the form of a weighted PPI graph. Next the enriched PPI graph was partitioned according to DetMod algorithm. The proposed algorithm performs clustering on the weighted graph structure determining functional modules with controlled overlap as described in detail at the Methods section. The 335 identified functional modules were tested concerning their connectivity density, their coverage in protein complexes and their functional enrichment in biological process GO terms. At the same time we compare, by means of these validation criteria, our modules with artificial modules and modules resulting after applying MCL algorithm  to our PPI dataset ,30.Although, during the last years several algorithms -26 have Saccharomyces cerevisiae. An important issue is reliability, when dealing with PPI data from high-throughput techniques [We tested the performance of our method based on data of chniques ,32,5. Wechniques ,34, whicThe gene expression data of the corresponding proteins is derived from a study that contains cell cycle related profiles using cdc15 synchronization over three cell cycles . The exp[The initial stage of our approach involved the clustering of expression profiles by SSFKCN algorithm . SSFKCN . This toNext we weighted the PPI graph according to the procedure described in the Methods section. Lastly, DetMod algorithm was applied in the enriched large component of the graph and 335 modules were determined. In these 335 modules, 2384 proteins (85.2%) of the large component are contained. In regard to the overlap displayed between modules, we set such parameters to the algorithm so that the allowed overlap was limited to 35%. After examining all 335 modules, we observed that 181 (54.2%) modules had no common members with any other modules.The first criterion applied to our 335 determined modules is connectivity density. This metric is the ratio of the total in-module degrees of the vertices to the total number of their connections and depicts how well connected are the members of the module. Many studies have used this topological metric, which ranges between 0 and 1, and it has already been established that functional modules should have connectivity density between 0.5 and 1 to fit this definition . As the Apart from just analysing our modules in terms of this metric we compared their connectivity density with artificial modules. We achieved this, through a randomization procedure, where we replaced 25% of the members of the modules with others that connect to the members of the modules but do not belong to the original ones. This randomized replacement was realized iteratively 10 times for each one of the 335 modules, and the average connectivity density was estimated.Additionally, we calculated the connectivity density of another set of modules that descended after applying the MCL algorithm to our PPI dataset. In Figure The second criterion involved the GO (Gene Ontology) annotation scheme. To gain insights into the underlying biological processes of the modules, we used the SGD GO term Finder . This toSpecifically, we examined DetMod modules and control  modules to see in which case the p-values were better. It is apparent from Figure A significant validation criterion of our approach was to analyze modules in terms of protein complexes they contain. It is known that protein complexes are by definition very close to the concept of functional modules, since they represent assemblies of proteins that interact up to a great degree and carry out distinct biological activities . Thus, iAdditionally it was of great importance to prove that the modules of our integrated approach were biologically more meaningful than the modules extracted from other methods. In order to have a more objective comparison we have evaluated our approach and DetMod algorithm, against not only the PPI method but also with other clustering algorithms that could provide overlapping modules as well as against a method that could benefit from the enriched graph structure we have created.A characteristic example of an algorithm that can produce overlapping clusters is the case of MCODE . One majThe plot of Figure Saccharomyces cerevisiae in the form of a weighted PPI graph. Then, we applied the novel DetMod partition algorithm at the main component of the graph and 335 functional modules were identified in the yeast protein-protein interaction network. We validated these modules by examining their connectivity density, their functional enrichment in biological process GO terms and their coverage in protein complexes.In this study, we integrated gene expression and protein-protein interaction data of The challenging task was to unify the two types of data in a manner that would lead to more valid functional modules. Our network in particular is very complex since it consists of 3250 proteins with 10750 interactions. We chose to weight every interaction via the corresponding gene expression profiles to lessen the burden of false interactions and acquire better functional modules. Interestingly, the weight serves in many cases as a savior for interactions that are not favored topologically but are experimentally verified. After all it is already established that domain knowledge over the strength of connections can promote network analysis ,41. ThenThe fast DetMod algorithm we propose was proved successful in the detection of functional modules. Its concept surpasses many drawbacks of current graph clustering algorithms, such as the input number of clusters . It is aSpecifically, the first validation criterion is connectivity density, a topological metric that shows how densely connected are the modules. We proved that DetMod modules have by far improved connectivity density from both control methods. An example is the module (12 members) characterized by the GO terms mRNA processing (GO: 0006397) and mRNA polyadenylation (GO: 0006378) with p-value smaller than e-15. The second term refers to the enzymatic addition of a sequence of 40–200 adenylyl residues at the 3' end of a eukaryotic mRNA primary transcript . This moGO: 0006626) or its children. The second term is a process that is carried out at the cellular level resulting in the formation, arrangement of constituent parts, or disassembly of any organelle within a cell. It is obvious that the two terms are correlated with the second having a broader meaning.The second criterion involves the characterization of functional modules in terms of biological process GO terms. It is essential for the modules to comprise proteins that carry out a certain task and this estimation is quantified using p-value. An example is a 12 member module characterized by the terms 'mitochondrial translocation' (GO: 0006628) and 'organelle organisation and biogenesis' (GO: 0006996) with p-value smaller than e-15 in both cases. The first term refers to the translocation of proteins across the mitochondrial membrane. In the presence of a translocating chain, the outer membrane import machinery (MOM complex) and the inner membrane import machinery (MIM complex) form translocation contact sites as a part of the membrane preprotein import machinery . HoweverAnother interesting example is two modules 16 and 23 members) that have 3 common members and both are characterized with the GO term 'ubiquitin-dependent protein catabolic process' (GO: 0006511). It involves all the chemical reactions and pathways responsible for the breakdown of a protein or peptide by hydrolysis of its peptide bonds, starting from the covalent attachment of an ubiquitin moiety, or multiple ubiquitin moieties to the protein. One would expect that these two modules should be merged into one module but this split occurred for various reasons. Firstly the PPI data used in our study is adequate but in some cases can be proved insufficient. The two modules are topologically separated due to sparse connections between them. Additionally, these few connections had quite large weight, thus during the constructing procedure they were considered as barrier between two densely connected modules. Nevertheless, this aspect should not be regarded as a defect of our method, because the introduced merging of modules bridges the gap mentioned above. Figure 3 membersSEC31 (YDL195w) and it is worth mentioning that DetMod provided a module restricted to this complexes only, leaving no space for other proteins. Responsible for this is partially DMSP algorithm embedded in DetMod algorithm. This algorithm constructs modules around a seed protein and it is noteworthy that the size of module remained the same even after the DMSP stage. DetMod algorithm builds biologically concise modules, in contrast to other well-known graph clustering algorithms [Finally, we substantiated our functional modules by quantifying the overlap of our modules to well established protein complexes. There are several studies that exploit the unique nature of protein complexes to validate functional modules ,45,14. Agorithms ,47.Another representative example is the module (7 members) characterized by the eIF3 (7 members) complex, which is responsible among with other eIFs for the initiation of protein synthesis in eukaryotic cells by stimulating the binding of mRNA and methionyl-initiator tRNA (tRNAi-Met) to 40S ribosomes to form the 48S pre-initiation complex [Lastly an issue worth discussing is the overlap displayed between modules. Most of the graph partitioning algorithms neglect the inter-module crosstalk, which is crucial for the stability of the whole network, and provide completely separated clusters. In the path of extracting functional modules from a network, one should bear in mind that the identification and preservation of links is of equal importance. Toward this goal DetMod succeeded in detecting modules with varying overlap in terms of proteins and interactions. Besides it is already widely accepted that proteins can by nature take part in many distinct tasks, or to be members of more than one functional module. Thus, we chose to embody this network property into our integrated approach and our modules as provided are consistent with this concept.The post genomic era demands the consolidation of different types of data, which all depict the dynamics of the living cell through different perspective. This study is a proposal toward the identification of functional modules in biological networks. We successfully integrated proteomics and microarray datasets, with the first having the leading role and the second acting as reinforcement.Saccharomyces cerevisiae that the novel algorithm DetMod detected highly confident functional modules onto the PPI network. Then, we examined these modules by validation criteria, which in turn substantiate that our subnetworks deserve the characterization 'functional module'. Specifically, we checked them by measuring their connectivity density, their enrichment in biological process GO terms and their coverage in protein complexes. These parameters were also estimated in two control methods, i.e. artificial modules and modules descending from a method restricted to PPI data. We observed that the obtained by DetMod modules surpass the control methods in all criteria and the difference is not random.We have proved based on data of the model organism At this section we analyze in detail the basic concepts of our method. Firstly we elucidate the integration of PPI and gene expression data and the reasons why this procedure can lead to biologically more meaningful functional modules. Then we describe in detail the graph clustering algorithm DetMod, which is responsible for the determination of functional modules on the PPI network. The proposed algorithm identifies functional modules on a PPI graph, which is weighted with the gene expression information. The first step of DetMod algorithm involves the construction of modules starting from a 'seed' protein. Next DetMod algorithm merges modules by examining a score we compute for each one of the extracted modules. The procedure of merging is preferred in cases, where the score of the merged cluster is better than the score of the forerunning clusters. However if the merged cluster does not significantly overlap in respect to its members with one of the forerunning clusters, then both the merged as well as the old cluster are preserved.In our work we chose to unify the above types of data for various reasons. Firstly PPI data descending from high-throughput techniques suffers from many false interactions . Also prSpecifically we used highly confident PPI data in the form of a graph G , where vertices represent proteins and edges represent interactions. Then we applied a clustering algorithm at the respective gene expression profiles. The number of clusters was appointed both by the algorithm itself as well as by the functional enrichment of clusters in GO (Gene Ontology) terms. Next we weighted the interaction between two proteins according to the weight function:x and Ky symbolize the centroids of the clusters that genes x and y respectively, belongs to. The constants n1 and n2 add an extra confidence score to the factors of the weight function. They can have the same or different values according to which member (if any) of the function we want to enhance. We chose n2 > n1  because we consider the distance between centroids more significant comparing to the distance of each gene from its centroid. This selection was motivated by the noise (outliers) of gene expression profiles. Based on several runs of the algorithm and the corresponding results, in the current study we have set the values of the two variables as n2 = 0.7, n1 = 0.3, but in general we systematically found better results when the value of n2 was larger than n1.||·|| stands for the distance metric and there are many metrics for measuring it, in this study we have used Euclidean distance. KThe outcome of our integration method is a weighted PPI graph, at which the proposed algorithm will be applied in order to detect functional modules that are supported by both types of data.1 of a larger graph G we define the internal degree |NG1INT| as the number of edges connecting x with other vertices belonging to G1 and external degree as the number of nodes with which x is connected and exist in G but do not belong to G1.As we have already mentioned, in the approach we have followed, we combine gene expression profiles and PPI data, in the form of a weighted graph, G. By N(x) we denote the neighbours of a node x, or in other words the set of nodes that are connected to x. Then, the degree of x is equivalent to the number of neighbours of x |N(x)|. For a given subgraph G1 over |NG1INT|:The above concepts can be expanded to the weighted graphs easily. Weighted degree of a node is the sum of weights of the edges between x and its neighbours divided by |N(x)|. Weighted internal degree of a node x is the sum of weights of the edges between x and its neighbours within GCorrespondingly we define the term of weighted external degree.w(G), is the sum of the weights of actual edges over the number of possible edges among all nodes in G:The density of a graph G is generally measured by the proportion of the number of edges in the graph to the number of all possible edges, which is equal to |V|(|V|-1) for an undirected graph. Weighted density of a graph or subgraph DDetMod incorporates in its first phase the application of another algorithm called Detect Module from Seed Protein (DMSP)  which opDMSP algorithm initiates its function by selecting only a certain number of the neighbours of the 'seed' protein (named hereafter s). These adjacent nodes are sorted in descending degree of significance and this subset of nodes – proteins is named kernel.s is equal to all the neighbours of s. Then for each one of the neighbours ui belonging to Kernel(s) we find the NINT(Ks), NEXT(Ks), as well as the βINT and βEXT. The objective for selecting the kernel of the seed node is two-fold. Firstly we check so that the number of edges of a kernel node within the rest of the kernel is larger or at least equal to the number of the edges that a node has outside the group. We accomplish this by requesting for the internal and external degrees of each node:The two criteria by which the original kernel is selected are the density of the kernel and the weighted internal and external degrees of it. Initially, the kernel K1 to have value over 45%. At the same time and after we have confirmed that a selected node fulfils the first condition, we request that the same node has smaller weighted internal degree than its corresponding weighted external degree. Nodes that fail to pass the above criteria are discarded, while those that do, are sorted based on the level that each one of them manages to do so.In this study we selected pw(Ks) as:This original subset of proteins is further distilled, in order to acquire an even more coherent kernel. This can be achieved by minimizing DIn this step, DMSP removes one at a time, each one of the sorted per significance nodes starting from the most insignificant until it reaches a minimum value of weighted density.nd and 3rd level or not. The first criterion the algorithm checks is the same as the first one of the initial stage of DMSP described by (2). After this criterion has been checked then we select a node to be added to the module, if it satisfies the following:After the creation of the initial kernel, DMSP iteratively adds adjacent nodes based again on certain criteria. The depth of the neighbours  checked by DMSP vary per specific problem and data set, meaning that as long as the criteria we will mention are true the algorithm can go beyond the 2s), we select the constant p2 to be anywhere between 0.9 and 1.0 . Ideally the value of p3 should be equal to 1.0 but given that we work on a real and very complex biological problem we allow the value to range down to 0.9. The experiments we have conducted showed that a lower value could create artifacts in the final determination of modules. Relation (4) states that in order for an adjacent node ui of some kernel node v, to become member of the module, its weight must be less or equal to a specific percentage of the weighted internal degree of node v.G is the final module that is built from the initial kernel . The distinction of this value depends on whether the current node is a direct neighbour of the kernel or not. In this way we have a two-layer scheme where we retain a looser criterion for immediate neighbours and a stricter one for the remote neighbours of the initial kernel  to 0.75 when DMSP checks for members in the remote neighbours of the initial kernel).At this point we should emphasize, that DMSP, uses two values describing the relation of internal and external neighbours (2)  G' = G;2) Modules_List = Empty3) While G' != emptyI) Retrieve randomly a node v from G'II) Apply DMSP to create a new functional module M with v as seed, M = DMSPIII) For all modules in Modules_Listi) Check if there is a module with more than p% overlap with Mii) If there is, find = true then breakIV) End_for4) If find! = trueI) Keep M in Modules_ListII) Keep v in Nodes_List5) End_If6) Delete v from G'7) End whileAs we have seen DetMod allows every node to be part of more than one module. In this way DetMod manages to compromise between the complexity of genes or their products and their tendency to participate in different groups towards achieving different goals. For this reason we insert the term of node score (easily extended to module score). This metric has dual purpose, it checks if the majority of the immediate neighbors of a respective node are in the same cluster as well as the repetitive appearance of a node and its immediate neighbors in many different clusters.Node score is an expansion of the node degree term, and is related with the connectivity of a node in regard to its neighbors in every module.To compute the score of a node  For i = 1:N-12) For j = i+1:Ni and Modj are connectedI) If Modnew = mergei) Modnew) > Score(Modi) AND Score(Modj)ii) If Score(Modnew has no overlap conflict save in Merge_Modules(a) If Mod(b) Delete the other two modulesnew) > Score(Modi) OR Score(Modj)iii) If Score(Modnew has no overlap conflict save in Merge_Modules(a) If Mod(b) Delete the module with the worst scoreII) End_If3) End_For4) End_ForIAM designed, implemented the DetMod algorithm and the data integration method, and prepared the data sets. KD was responsible for writing the main body of the text and especially for the biological aspects of the study. All authors contributed in the preparation of the manuscript and approved its final form.


We have developed a method that predicts Protein-Protein Interactions (PPIs) based on the similarity of the context in which proteins appear in literature. This method outperforms previously developed PPI prediction algorithms that rely on the conjunction of two protein names in MEDLINE abstracts. We show significant increases in coverage (76% versus 32%) and sensitivity (66% versus 41% at a specificity of 95%) for the prediction of PPIs currently archived in 6 PPI databases. A retrospective analysis shows that PPIs can efficiently be predicted before they enter PPI databases and before their interaction is explicitly described in the literature. The practical value of the method for discovery of novel PPIs is illustrated by the experimental confirmation of the inferred physical interaction between CAPN3 and PARVB, which was based on frequent co-occurrence of both proteins with concepts like Z-disc, dysferlin, and alpha-actinin. The relationships between proteins predicted by our method are broader than PPIs, and include proteins in the same complex or pathway. Dependent on the type of relationships deemed useful, the precision of our method can be as high as 90%. The full set of predicted interactions is available in a downloadable matrix and through the webtool Nermal, which lists the most likely interaction partners for a given protein. Our framework can be used for prioritizing potential interaction partners, hitherto undiscovered, for follow-up studies and to aid the generation of accurate protein interaction maps. Protein-protein interactions (PPIs), which we define as proteins that physically interact, are crucial in most complex biological processes. Experimental high-throughput methods such as yeast two-hybrid screens have been used to make large inventories of PPIs and to create protein interaction maps et al. were the first to demonstrate that text mining can lead to the discovery of new knowledge (e.g. the treatment of Raynaud's disease by fish oil). Other studies in the biomedical domain verified the importance of implicit information for knowledge discovery i.e. synonyms) are mapped to a single concept identifier, and ambiguous terms, e.g., identical terms used to indicate different concepts (i.e. homonyms) are resolved by a disambiguation algorithm. Such an approach is essential given the wide diversity and many ambiguities in gene and protein nomenclature However, text mining can go one step further; by combining known associations, previously unknown PPIs can be inferred. Because most text mining research, including this study, limits itself to MEDLINE abstracts, these ‘previously unknown’ interactions also include interactions that are effectively known, but not explicit in MEDLINE as they are only mentioned in a full text article. Swanson concept profilesIn order to predict PPIs, we summarize the typical context in which each protein appears into In this article, we first demonstrate the added value of a concept-based approach over a traditional term-based approach in detecting explicitly described relations. We proceed to show the added value of the concept profile-based approach over classical direct relation extraction, including the text-mining techniques used in the STRING database. Subsequently, we show the predictive power of our method by doing a retrospective study; we demonstrate that we can employ the literature available in 2005 to predict 52% of the PPIs newly described in Swiss-Prot in 2007 at a specificity level of 95%. We show that in addition, some of the PPIs that we predicted but are not yet recorded in any database represent indirect protein interactions and have biological relevance. Finally, we confirm one of the many predicted PPIs in three wet lab experiments, supporting our claim that the concept profiling method is capable of previously unknown PPI prediction from current literature.These predictions will be useful for (i) the ranking of potential PPIs for more specific experimental analysis, and (ii) complementing other types of data such as co-expression and yeast two-hybrid data when using an integrative systems biology approach.We compared the performance of different PPI prediction approaches in detecting known human PPIs in MEDLINE. The online human-curated databases Biogrid, DIP, HPRD, MINT, Reactome, and UniProt/Swiss-Prot were used to establish a set of 61,807 known human PPIs. A set of probable Non-Interacting Protein Pairs (NIPPs) was generated from all pairs of proteins that do not occur in the above databases nor in the IntAct Word-based direct relation. This approach uses direct PubMed queries (words) to detect if proteins co-occur in the same abstract. This is the simplest approach and represents how biologists might use PubMed to search for information.Concept-based direct relation. This approach uses concept-recognition software to find PPIs, taking synonyms into account, and resolving homonyms. Here two concepts (in our case two proteins) are detected if they co-occur in the same abstract.STRINGConcept profile-based relation. This approach uses the similarity in literature context. Here two proteins (concepts) can also be indirectly related via the concepts in their profiles. More detail on concept profiles and their construction can be found in the The word-based and concept-based direct relation methods could find at least one abstract containing both proteins for respectively 33% and 32% of the pairs in the PPI set. A text mining score from STRING could be obtained for 30% of the PPIs, in line with the co-occurrence based approach used to create STRING. Thus, a majority of the known PPIs cannot be found explicitly in MEDLINE. For the concept profile-based approach, we could create concept profiles and calculate a similarity score for 76% of the PPI set.Similar to STRING, the other three approaches can also be used to calculate a continuous score that indicates the strength of the relation between two proteins. The results show that, using concept profiles, we can detect 43% of the known PPIs, with a specificity of 99%, and 66% of all known PPIs with a specificity of only 95%. In contrast, the direct relations methods and STRING show much lower scores .e.g. they could be involved in the same pathway or be part of the same protein complex, but do not physically interact. To determine whether this occurs, we also tested both concept-based approaches on the detection of known connections via one intermediate protein. For instance, if the protein pairs A-B and B-C are recorded as PPIs in databases, we form the additional protein pair A-C. In total we were able to create 1,028,265 of such pairs to serve as an independent test set. When the pairs are filtered on coverage by all methods the remaining set contains 790,245 pairs. At a specificity level of 99% and 95% the sensitivity level of the different methods was determined for those pairs. The results are given in The results reported in the previous section indicate that not all proteins with high similarity scores are known to interact according to the combined protein databases. One possible explanation for this is that the proteins are related in another way, Most researchers will not be interested in all PPIs, but only in those interactions involving a (set of) protein(s) of interest. Therefore, for each protein we created a top 10, top 100, and top 1,000 best matching proteins according to the concept-based direct relation, the concept profile method, and STRING. In these lists, we calculated the number of PPIs that are either (i) part of the PPI set, or (ii) described in the IntAct database, or else (iii) part of the pairs that are connected through intermediate proteins as described in the previous section. We limited our analyses to the 10,812 proteins that were detected in at least five MEDLINE abstracts (covered by the concept profiles method). The averages of these performance measures in terms of precision and recall are shown in Protein annotation databases are struggling to stay up-to-date with the literature, and there is often a substantial time lag between the first publication of a finding, and the time the PPI is entered in a database. It could therefore be postulated that many of the unknown PPIs predicted today are in fact correct, but may not be entered in a database for several years. We have performed a retrospective study to answer the question: how many of the PPIs that would have been predicted by the different methods in 2005 were confirmed in 2007?Both direct relation and concept profile method-based PPI prediction scores were created using a MEDLINE corpus with publication dates up to February 2005. We ranked the PPIs according to the scores, and set a cut-off value at the 95% and 99% specificity levels based on PPIs present in Swiss-Prot 2005 (this is the only database for which historic versions are available). We subsequently evaluated how many of the 3,295 PPIs that were added to Swiss-Prot between 2005 and 2007 were above these cut-off values in 2005. These are the sensitivity values reported in The prediction performance is much better for concept profiles (52% versus 38% for a specificity level of 95%). This indicates that the majority of currently known PPIs were not yet explicitly described in MEDLINE at our testing point, but would have been predicted at a specificity rate of 95%. We postulate that this finding is indicative for the assumption that based on the full current literature a meaningful percentage of the ‘unknowns’ that pass the prediction threshold will be actual pairs worth studying in more detail.The next logical step was therefore to investigate whether this method can only predict PPIs that are ‘known’ but not explicit in the literature corpus used, or whether it would also be able to effectively predict unknown, but real PPIs. We investigated this in two case studies. We generated predicted interactions for proteins with two proteins that are intensively investigated in our group: (i) Dystrophin (DMD), a structural protein causing Duchenne muscular dystrophy when defective, and (ii) Calpain 3 (CAPN3), a protease when mutated causing Limb-girdle muscular dystrophy (LGMD).−10). There are three proteins in the predicted set with at least indirect evidence in the literature for a physical interaction with DMD . One protein (SLMAP) may well interact given its distribution and localization but this needs experimental testing. Ten proteins in the list are found in the same protein complex as DMD but do not interact directly as far as known. Four proteins in the list were found wrongly associated with DMD due to homonym problems during literature indexing.We presented the list of predicted interacting proteins with DMD ordered by descending association scores, to two experts for evaluation. At a specificity of 99%, there are 196 proteins predicted to interact with DMD. This list was too long to manually evaluate and we therefore restricted the human curation analysis to the 99.8% specificity level  but are not likely to physically interact. If we only allow direct physical interaction pairs as true positives (11 proteins) the estimated precision is 26%. If predictions of protein pairs in a complex also are counted as true positives , the estimated precision would be 50%. Since also conceptually-related proteins that do not physically interact may be of interest to the biologist, the overall precision of our prediction method may be as high as 90%.−5). Interestingly, the concept profiling method correctly predicted the interaction between myosin light chain 1 (MYL1) and CAPN3 on the basis of conceptual overlap in MEDLINE abstracts (specificity >99%), although this interaction was only described in a full text paper For CAPN3, an evaluation of the precision is more difficult since there is, compared to an intensively studied protein such as DMD, not enough established knowledge about its regulatory partners and substrates. Apart from literature based rediscovery of known interactions, we also set out to actually find new interaction partners for CAPN3. We selected predicted interaction partners that have not been entered in PPI databases so far and that do not have a direct co-occurrence in MEDLINE. The top ranked conceptual match is with Sarcoglycan-epsilon (SGCE), which is the smooth muscle counterpart of SGCA. Like for CAPN3, mutations in SGCA cause LGMD, but as far as we know, the protein is not expressed in skeletal muscle.The second highest ranking protein was deemed to be an interesting candidate by the experts: Parvalbumin B (PARVB). The concept profiling method yielded a high association score because both proteins are described to have a physical interaction with dysferlin (DYSF) First, it was shown that immobilized GST-fused PARVB could pull down recombinant T7-CAPN3 from bacterial lysates. Second, immobilized GST-PARVB could pull down endogenous CAPN3 from IM2 mouse myoblasts, and vice versa .CAPN3 is hypothesized to act as a cytoskeleton remodeler and has been shown to interact with other focal adhesion proteins like Talin and Vinexin This showcase is just one example of a correct and meaningful PPI prediction using concept profiles. This exemplary case study can not be seen proof that many of the other high ranking predictions will also be true physical and biologically relevant interactions. However none of the other consulted applications  predicted this pair of interacting proteins. As the predictions using concept profiling are based on conceptual relatedness rather than an explicit co-occurrence in MEDLINE, this case study is indicative of the power of concept profiles to discover new, implicitly related pairs of interacting proteins. The statistics presented in this paper support the conclusion that predicted PPIs using our method, especially the subset that remains after expert analysis of the top ranking list are likely to be very significantly enriched for proteins that are worthwhile studying in wet lab experiments.Scientists in general and scientific annotators in particular derive their knowledge on PPIs not directly discovered by their own experiments from the literature. However, as we show here, only 32% of the known PPIs covered by curated PPI databases can be found in MEDLINE abstracts , the resi.e. how many of the predicted proteins are true interaction partners) are still seemingly quite moderate. A first consideration is that we are intrinsically unable to determine an accurate ‘true false positive rate’ for the predicted PPIs, due to the fact that many PPIs have simply not been discovered and described yet. This unavoidable complication most certainly will lead to an underestimation of precision levels. The case study of CAPN3 and PARVB signifies this point; initially this pair would have been classified as a ‘false positive’.The specificity and sensitivity levels achieved by our novel prediction method appear to be very promising. However, when we predict interaction partners for a specific protein, the estimated precision levels . In Nermal, researchers can enter the UniProt identifier of a protein of interest, and the tool will return a ranked list of proteins that are most likely to interact with the query protein, in combination with information on whether the PPI has already been described explicitly in MEDLINE and/or in one of the protein databases.For a realistic estimation of the precision of our prediction method, effectively each predicted protein pair should be validated in a wet lab experiment, which is out of the realistic scope of this study. For this reason we developed Nermal. (A second complicating factor is the size of the ‘negative’ set (>50 million) compared to the ‘positive’ set . This aspect is illustrated by the average prediction performance for each protein in To allow researchers to incorporate conceptual overlap data into their own analyses, we have made the concept profile similarity scores publicly available in two forms; first, a table containing similarity scores between all human proteins can be downloaded from our website; second, the previous mentioned web tool dubbed Nermal.We conclude that concept profile similarity is a significantly better literature based predictor of PPIs than co-occurrence based methods. These improved predictions can be used to increase the biological interpretation and accuracy of interaction maps generated by high-throughput experiments, or can be used to prioritize proteins for further testing. In further studies, we will evaluate whether the use of concept profiles can also be applied in the prediction of other types of relations, for instance between drugs and diseases, and between genes and diseases.Direct relations are typically extracted from literature based on co-occurrence λ of concepts i and j.in, jn, and ijn are the number of documents containing i, j, and both i and j, respectively. j occurs in an abstract irrespective of i, j occurs in an abstract containing i, j occurs in a document not containing i, and We used the likelihood ratio H(X) is the entropy for X and H(X|Y) is the entropy for X given Y. X and Y can be any concept known in the ontology, e.g. drugs, proteins, diseases, disorders, chemicals, etc. The uncertainty coefficient is an information-theoretical measure that takes the a priori probability of direct relations into account. It gives extra weight to those concepts that are very specific for the set of documents belonging to the protein for which the concept profile is constructed. For a detailed description of concept profiles we refer to Jelier et al. To calculate the similarity of the contexts in which proteins appear in literature, we summarize the context of each protein in a concept profile. This profile contains all concepts that have a direct relation with a protein as found using the direct relation method described above. We evaluated two possible ways of applying this method: (i) using co-occurrences within a sentence, and (ii) using co-occurrences within an abstract. As shown in supplement S6  the thk uncertainty coefficient in the profile and N the total number of concepts the two profiles have in common. The inner product increases with increasing overlap in concept profiles. If two proteins co-occur, the inner product of their concept profiles is in general high. This is shown in supplement S4  they are not in the positive PPI set, (ii) not the same protein, e.g. proteins that interact with themselves are not taken into account, (iii) the protein pair is not already in the NIPP set, e.g. protein pairs can only occur once in a set. The random sample is actually quite small compared to the total NIPP set, however the ROC curve analysis is set size independent if the sample size is sufficiently large.One last remark is that the positive set is incomplete. Therefore the creation of the NIPP set will introduce false negatives (PPIs that should have been in the positive set and recorded in a curated database). However the bias introduced by false negatives is negligible since the ratio of expected PPIs in human compared to the total set of formable protein pairs (∼60 million) is very small A copy of the STRING database, version 7.1, was downloaded from the STRING website. STRING is a pre-calculated database in PostgreSQL format. Only the text mining score table was used in our analysis.In information retrieval terms like the sensitivity, specificity and precision are frequently used. The definitions are:When both set sizes are equal (#NIPP = #PPI) the precision equals the sensitivity. The specificity is sometimes confused with the precision. The distinction is critical when the classes are different sizes. A test with very high specificity can have a very low precision if there are far more true negatives than true positives, and vice versa.http://biosemantics.org/nermal/. The full set of all protein pair match scores for human proteins can be downloaded at this link as well as the PPI and NIPP set used in the study.Nermal is a web tool that prioritizes proteins that are most likely to be related with the protein you study. Given a query protein, the similarity scores are calculated between this protein and all other proteins in the ontology. The proteins are ranked on the similarity scores and presented in a table. Each row shows the similarity score between the two proteins, the databases in which the protein pair is known, and the sensitivity and (1-specificity) for that similarity score. These two rates should be interpreted as follows: given a similarity score between two proteins, (1-specificity) is the probability that a protein pair passing that score is a false positive. The sensitivity is the probability that you will miss a true PPI at that same score. Nermal can be found on aatatggatcctcctccgcgccaccacggt, rv atattctcgagctccacatccttgtacttgg. CAPN3 was similarly amplified with primers fw atgccaactgttattagtc, and rv ctaggcatacatggtaagc, and cloned into pET28aGST using fw tattacggatccatgccaactgttattagtc, and rv gtaatactcgagctaggcatacatggtaagc. The exon 6 deletion that does not autolyse was used for this experiment.PARVB was amplified from proliferating IM2 myoblast cDNA with the following UTR primers: fw cgcactcgcttatgtcctc, rv ctccacatccttgtacttggtg. The ORF was amplified with a nested PCR introducing restriction sites for cloning into pET28aGST (modified pET28a vector with GST tag instead of T7 E. coli cells (Stratagene) for protein production.CAPN3c129s in pET28c was described previously BL21 cells transformed with pET28aGST, pET28aGST-PARVB, pET28aGST-CAPN3 or pET28cCAPN3c129s were grown to log phase and stimulated with 1 mM IPTG (Fermentas), and left to grow for 3 h at 37°C. Next cells were spun down at 3,000 g and 4°C for 15 min. Pellets were dissolved in lysis buffer A ), and sonicated on ice. Lysate was cleared by centrifugation at 13,000 g, and 4°C for 30 min.IM2 cells were grown at 33°C and 10% CO2 in DMEM 60196  supplemented with 20% FCS, INFγ, glucose, pen/strep, glutamine and chick embryo extract. 15 cm plates (2x) were grown 75% confluent, washed 1x with PBS (37°C) and lysed on ice with 1 ml lysis buffer B . Lysate was spun down at 13,000 g and 4°C for 30 min.GST sepharose beads  were washed with PBS (2x) and pre-equilibrated with lysis buffer (2x), and added to the cleared GST fusion lysates. Lysates were incubated at 4°C and tumbling for 2 h. Next the lysates were spun down at 500 g, 4°C for 5 min, and washed 3x with lysis buffer A. Separately, IM2 lysates were treated with washed and pre-equilibrated GST sepharose beads (buffer B). An aliquot of the GST fusion proteins was loaded on SDS-PAGE gel and Coomassie stained to confirm equal loading.IM2 lysate, or T7-CAPN3c129s lysate, was added to the bait, and incubated O/N at 4°C and tumbling. GST sepharose beads were spun down and the sup was stored as non-bound fraction. The beads were washed 5x with ice cold lysisbuffer . All remaining sup was removed with an insulin syringe and proteins were eluted with 2x Laemmli sample buffer and boiled 5 min. An aliquot of the non-bound fraction was similarly prepared.Samples were loaded onto SDS-PAGE gels, separated and blotted to PVDF membrane. Blots were blocked in 4% skimmed milk PBS (Marvel) and incubated with primary antibody O/N at 4°C. Next morning blots were washed with 0.05% Tween in PBS, and incubated with secondary antibody for 1 h. Blots were washed again and scanned with an Odyssey scanner (Licor) or incubated with ECL plus (Amersham) and exposed to a Kodak XAR film. The following antibodies were used for Western detection: GaGST  MaCAPN3 , GaPARVB , GaMouseIRDye680 , DaGIRDye800 , RaMouseHRP , DaGoatHRP .Figure S1Histogram of the distributions of similarity scores of the concept profile-based method for the PPI and NIPP sets. A log transformation is applied to the similarity scores for better visualization.(1.35 MB TIF)Click here for additional data file.Figure S2CAPN3 and PARVB can directly interact. A: Immobilized GST-fused PARVB can pull down recombinant CAPN3 from a bacterial T7-tagged CAPN3 lysate (Lane 2 vs 1), where unfused GST cannot (Lane 4 vs 3). As CAPN3 is an unstable protein that outside skeletal muscle rapidly autolyses we used the active site mutant C129S48. All fractions were resolved on SDS-PAGE gel and analyzed by immunoblotting with anti-CAPN3. The lanes represent: GST-PARVB non-bound fraction (1), GST-PARVB bound fraction (2), GST non-bound fraction (3), GST bound fraction (4). B: Equal loading was confirmed with anti-GST . C: GST-fused PARVB can pull down endogenous full-length CAPN3 from an IM2 lysate (Lane 1 vs 2), contrary to unfused GST (Lane 3 vs 4). Lane 1 GST-PARVB bound fraction, Lane 2 non-bound fraction, Lane 3 GST bound fraction, Lane 4 non bound fraction. D: Likewise, GST-CAPN3 can pull down endogenous PARVB (Lane 1), contrary to GST (Lane 2). Both PARVB translation products bind. Here we used the Δ6 variant of Capn3 that does not autolyse yet retains function30, 49, and is expressed in the proliferating IM2 myoblasts. The arrows indicate the detected proteins and in all panels a molecular marker is depicted on the left.(1.67 MB TIF)Click here for additional data file.Table S1Performance of different PPI prediction approaches on detecting known PPIs in MEDLINE. CDR stands for Concept-based Direct Relation method.(0.03 MB DOC)Click here for additional data file.Table S2Performance on predicting proteins that are connected via an intermediate protein.(0.03 MB DOC)Click here for additional data file.Table S3Analysis of the top 10, 100, and 1,000 returned by the Concept Profile (CP) method, the Concept-based Direct Relation (CDR) method, and by STRING. The analysis shows the precision and recall of protein pairs that are in the PPI set, of additional pairs that are found in IntAct, and of additional pairs that are in the set of protein pairs that are connected via an intermediate protein. In the field of information retrieval the term recall is more often used instead of sensitivity.(0.04 MB DOC)Click here for additional data file.Table S4Results of the retrospective prediction of PPIs added to Swiss-Prot between 2005 and 2007. PPIs are ranked based on MEDLINE up to 2005, and specificity levels are based on Swiss-Prot 2005.The sensitivity is determined on Swiss-Prot 2007.(0.03 MB DOC)Click here for additional data file.Table S5Top 42 ranked proteins with DMD. In total 10,812 proteins were matched against DMD. 7 proteins as known to interact with DMD. Only 4 proteins are real false positives due to homonyms problem resulting in a precision over 0.9.(0.09 MB DOC)Click here for additional data file.Table S6List of proteins known to interact with Calpain-3. In total 10,812 proteins known to have a concept profile are matched against Calpain-3.(0.05 MB DOC)Click here for additional data file.Supporting Information File S1Supplementary data.(0.59 MB DOC)Click here for additional data file.


We participated in three of the protein-protein interaction subtasks of the Second BioCreative Challenge: classification of abstracts relevant for protein-protein interaction (interaction article subtask [IAS]), discovery of protein pairs (interaction pair subtask [IPS]), and identification of text passages characterizing protein interaction (interaction sentences subtask [ISS]) in full-text documents. We approached the abstract classification task with a novel, lightweight linear model inspired by spam detection techniques, as well as an uncertainty-based integration scheme. We also used a support vector machine and singular value decomposition on the same features for comparison purposes. Our approach to the full-text subtasks (protein pair and passage identification) includes a feature expansion method based on word proximity networks.Our approach to the abstract classification task (IAS) was among the top submissions for this task in terms of measures of performance used in the challenge evaluation . We also report on a web tool that we produced using our approach: the Protein Interaction Abstract Relevance Evaluator (PIARE). Our approach to the full-text tasks resulted in one of the highest recall rates as well as mean reciprocal rank of correct passages.Our approach to abstract classification shows that a simple linear model, using relatively few features, can generalize and uncover the conceptual nature of protein-protein interactions from the bibliome. Because the novel approach is based on a rather lightweight linear model, it can easily be ported and applied to similar problems. In full-text problems, the expansion of word features with word proximity networks is shown to be useful, although the need for some improvements is discussed. Much of the research presently conducted in genome biology relies on the induction of correlations and interactions from data. Indeed, during the past decade, fueled by the production of large biomedical databases (particularly those containing genomic data) as well as the widespread use of high-throughput technology, we have witnessed the emergence of a more data-driven paradigm for biological research, which in turn has created new analysis challenges. Because we ultimately want to increase our knowledge of the biochemical and functional roles of genes and proteins in organisms, there is a clear need to integrate the associations and interactions among biological entities that have been reported and accumulate in the literature and databases. Such integration can provide a comprehensive perspective on presently accumulated experimental knowledge, and may even uncover new relationships and interactions induced from global information but unreported in individual experiments.Literature mining ,2 is expWe participated in three subtasks of the second Biocreative challenge , which f1. Protein interaction article (interaction article subtask [IAS]): classify PubMed abstracts as being relevant or irrelevant to PPI. Relevance implies that articles 'are useful to derive protein interaction annotations according to the curation standards used by the' IntAct  and Mole2. Protein interaction pairs (interaction pair subtask [IPS]): identify interacting protein pairs from full-text articles. Pairs of interactors are classified as such according to the curation standards of IntAct and MINT - that is, only for colocalizations and physical interactions.3. Protein interaction sentences (interaction sentences subtask [ISS]): identify the sentences most relevant to characterize the interaction pairs identified in the IPS task.In most text-mining projects in biomedicine, one must first collect a set of relevant documents, typically from abstract information. Such a binary classification, between relevant and irrelevant documents for PPI, is precisely what the IAS subtask in BioCreative II aimed to evaluate. Naturally, tools developed for IAS have great potential to be applied in many other text-mining projects beyond PPI. For that reason, we opted to produce a very general and lightweight system that can easily be applied to other domains and ported to different computer infrastructure. This design criteria lead us to a novel linear model inspired by spam-detection techniques. For comparison purposes, we also used a support vector machine (SVM) and singular value decomposition (SVD) enhanced with an uncertainty-based integration scheme.As for the IPS and ISS subtasks, our approach is centered on a feature expansion method, using word proximity networks, which we introduced in the first Biocreative challenge . Below, + and abstract co-occurrence word pairs . Even with this error, run 2 was still among the best performing submissions in the challenge; only 7 out of 51 other submissions (from 4 out of 19 groups) reported higher values of all three main performance measures used + and absAs can be seen in Table 1 measure (or F-score) is defined as F = 2 × precision × recall/, where precision is the proportion of abstracts returned that are relevant (positive), and recall is the proportion of relevant abstracts that are retrieved. In this case, again VTT yielded the best result (0.78), followed by our SVM (0.74) and SVD-UI (0.73) methods. Notice that the F-score of VTT was very close to the mean plus one standard deviation for all submissions to the challenge. In fact, only 3 submissions (from 2 groups) out of 51 to the challenge reported a higher F-score. The AUC measure can be understood as the probability that for a randomly picked positive abstract and a randomly picked negative abstract, the positive abstract is ranked above the negative one for protein interaction relevance. We obtained very good results with this measure for both VTT and SVM runs (0.8), and slightly above the mean for SVD-UI (0.75). This means that the probability of finding a false positive closer to the top of the abstract rank  produced by the SVD-UI method is considerably larger than in the ranking produced by the VTT and SVM methods. Furthermore, note that the AUC of both the VTT and SVM runs was close to the mean plus one standard deviation for all submissions to the challenge. Only 10 submissions (from 6 groups) out of 51 to the challenge reported a higher value of AUC than these two runs.Let us now look in more detail at the performance according to these three measures. Accuracy gives us the ratio of correct predictions for both positive and negative abstracts (relevant and nonrelevant for protein interaction). In this case, the VTT method yielded the best result (0.75), followed by our SVD-UI (0.71) and SVM (0.71) methods. Thus, the VTT method produced a more balanced prediction for both positive and negative abstracts, leading to the lowest error rate (0.25). We should also notice that the accuracy of VTT was well above the mean plus one standard deviation for all submissions to the challenge. In fact, only 2 submissions (from the same group) out of 51 to the challenge reported a higher value of accuracy. The FAs we discuss in the Materials and methods section (below), the SVD vector model alone produced the same classification of the test abstracts as SVD-UI, except that different rankings of abstracts were attained. Therefore, the values of accuracy and F-score are identical for the SVD vector model alone and SVD-UI. However, the AUC of the SVD method alone was much lower (0.68) than that of the SVD-UI method (0.75). We can thus say that the integration method improved the AUC of the SVD method alone. On the other hand, its performance according to accuracy, F-score, and AUC was worse than the other constituent methods employed in the uncertainty integration, such as VTT as submitted in run 2. Thus, uncertainty integration did not improve the VTT alone. The fairly lackluster performance of this uncertainty integration method is possibly due to computing Shannon's entropy for the only two classes of this problem: positives and negatives. The method was originally developed to classiA comparison of all our methods in the accuracy/F-score, accuracy/AUC, and F-score/AUC planes is depicted in Figures 1. Accuracy: all three runs are above the mean and median values of accuracy for all teams. Run 2' (VTT) yielded an accuracy one standard deviation above the mean accuracy.2. F-score: all three runs are above the mean and median values of F-score for all teams. Run 2' (VTT) yielded an F-score nearly one standard deviation above the mean accuracy.3. AUC: all three runs are above the mean and median values of AUC for all teams. Runs 1 and 2' (SVM and VTT) yielded nearly one standard deviation above the mean.4. Balance across all performance measures: the VTT method (run 2') was the only one that was quite above the mean and median for all measures tested .One of the problems encountered by all methods, but particularly so for our SVM and SVD methods, was the significant difference between the training and the test IAS data in BioCreative II. It is clear that the abstracts in the training data are distinct from those in the test data. To quantify this distinction, after the challenge we trained a SVM model to classify labeled and unlabeled data - that is, between training and test data, regardless of them being relevant (positive) or irrelevant (negative) for protein interaction. If the two sets of abstracts were sampled from the same coherent semantic classes, then the AUC should be about 0.5, indicating that training and test data were indistinguishable. Instead, we obtained an AUC of 0.69. We also obtained a very high precision (0.88) and F-score (0.92). Thus, we can say that there are quite a few labeled data points (training abstracts) that are not close to anything in the unlabeled dataset (test abstracts). Because it appears possible to 'recognize' whether an abstract is from the training or the test set, one could improve classification using this information. Clearly, such an exercise would not lead to a useful system architecture for real world applications, where we do not have a 'test set'. Nonetheless, it is interesting to note that classification in the challenge can be improved because of the evident distinction between training and test data. We chose not to do this in the official submission , but we Training our SVM classifier  on the bootstrapped training set  resulted in an improved performance . This was the best AUC that we obtained, although the accuracy and F-score of the VTT method , trained on regular and not bootstrapped data, were still superior. This highlights our observation that our novel VTT method seems to suffer less from over-fitting to training data than both the SVM and SVD methods. Indeed, the performance of both the SVM and SVD on the training data was superior to that of the VTT. The mean values of F-score and accuracy on the eight training and eight additional data partitions  for the SVD and SVM were around 0.92 . In contrast, the VTT achieved values only around 0.88 for these same measures. Because the VTT surpassed both the SVM and SVD on the test data, we conclude that it generalized the 'concept' of protein interaction better.λ0 = 1.25 and β = 13 . If we had submitted this run, then the accuracy would have been the same as our submission (0.75), but both F-score and AUC would have been considerably worse, at 0.75 (from 0.78) and 0.75 (from 0.8), respectively. Thus, we conclude that the inclusion of additional data was useful in achieving a generalization of the 'concept' of protein interaction in the bibliome. Figure It is important to note that our inclusion of additional data to train our algorithms (in particular the VTT) was very useful in achieving this generalization. Indeed, if we had trained exclusively on the supplied training data, then the VTT parameters would have been different: + was made based on the rank product of performance measures on training and additional data . However, the performance of bigrams+ on training and additional data was just marginally inferior to co-occurrence pairs. When it came to the test data, however, bigrams+ led to a considerably lower AUC (0.77 from 0.8 obtained by co-occurrence pairs) and slightly lower F-score and accuracy. This seems to indicate that word co-occurrence word pairs in abstracts are also able to generalize relevance to protein interaction better than bigrams+. The bigram+ version of VTT (VTT-bi) was included in our uncertainty integration scheme and its performance can also be appreciated in Figures We should also note that our final VTT submission used abstract co-occurrence word pairs as features. The decision to submit a run with these features instead of bigramsC depends on the number of proteins identified in the abstract np(a), and constants λ0 and β. Currently, the tool arbitrarily assumes that if C ≤ 0.1, then the confidence is considered low; if it is >50% (C ≥ 0.5) then it is high; otherwise it is medium. In future work we intend to develop a more statistically sound measure of confidence based on random tests and user feedback.We produced a web tool based on the VTT method, in order to classify abstracts according to protein interaction relevance: PIARE. The prototype of this tool is available on the internet . A finalThe results for the IPS and ISS tasks, except for the recall measure of performance in IPS and mean reciprocal rank in ISS, were somewhat disappointing, although in line with the central tendency of all submissions. Our three submitted runs to IPS were barely distinguishable (within 1%). Runs 1 and 2 yielded the exact same results, which shows that using either the features extracted from IAS abstracts or those extracted from the IPS/ISS training sentences leads to the same results . On the other hand, run 3 yielded an improvement of about 1% over the two other runs, which shows that using both types of features simultaneously is somewhat advantageous. For all of our three runs, the precision was below the mean and median of all submissions but still well within one standard deviation of the mean. On the other hand, recall was above the mean and median of all submissions; very close to being above the mean plus one standard deviation for all articles; and above it for the subset of articles containing exclusively SwissProt IDs. There were 6 submissions (from 4 groups) out of 45 with higher recall for the set of all articles, and 7 for the case of articles with SwissProt IDs only  in addition to the expanded word-pair features is advantageous - gaining another 1%. Our matches (387) and unique matches (156) to previously selected passages were above the average of all submissions  - there were only 3 out of 26 submissions (from 3 groups) with higher number of matches, and 8 (5 groups) out of 26 with higher number of unique matches . One shoFor IAS, we set out to test a novel lightweight classification method for protein interaction relevance, by contrasting it with two of the known best-performing classification methods: SVM and SVD. The lightweight VTT method uses a small number of original features (650) and a linear decision surface, but it outperformed our runs with SVM and SVD, as measured by accuracy, F-score and AUC, when using the same set of features. The VTT method also performed quite well when compared with all submissions to the BioCreative II challenge. The submissions from other groups that surpassed or were similar to VTT all used SVMs, but with more sophisticated features including semantic features from relevant biological ontologies and nomenclatures. In contrast, the VTT uses only the off-the-shelf named entity recognition package ABNER  , and sinWe also tested an uncertainty-based integration method that we had previously developed for protein family prediction . This inRegarding the IPS task, although obtaining a good recall measure, our system could improve precision by considering additional biologically relevant information. In particular, because our system, for the same protein mention, outputs different Uniprot IDs for each of the organism MeSH terms of the document at stake contain more than one organism, it could be improved by identifying organism information in the text. Using a classifier (such as our VTT or an SVM) to preclassify documents and passages according to different organisms could result in increased precision. We should also do more about removing genetic interactions. These were not considered in BioCreative, but we did not remove gene mentions from ABNER as features in our system. These and other improvements will be tested in the future.Regarding the ISS task, our system was good at recalling and ranking predicted passages, but the feature expansion method based on word proximity networks could improve its precision. We used solely protein names to extract additional features from proximity networks; unfortunately, protein names as recognized by ABNER were often broken up by stemming and preprocessing of documents, appearing in different forms in the proximity networks and thus failing to be recognized by the feature expansion method. Because, even so, the expansion of feature words was modestly beneficial, and because it is clear from the manual observation of proximity networks that they do capture the contextual relationships of individual documents, we plan to use the method to find additional words related to general features, not just protein names. We also plan to use these document-specific networks to predict and classify documents according to organism, which should help improve protein pair prediction for tasks such as IPS and ISS.In general, our participation in three subtasks of the BioCreative II challenge, with such a large set of members, was very useful in validating our approaches as well as learning from other groups. It also led us to a position where we are more easily able to extend the methods to biomedical applications other than protein interaction.As mentioned under Background (above), most text-mining applications in biomedicine start with the collection of relevant documents, typically from abstract information. In the case of the IAS, the goal is to classify 'articles which contain relevant information relative to protein interactions' . The gropTP(w) that a word w appears in a positive abstract, as the ratio of the number of positive abstracts containing w, over the total number of positive abstracts. Similarly, we computed the probability pTN(w) that a word w appears in a negative abstract, as the ratio of the number of negative abstracts containing w, over the total number of negative abstracts. After stemming with the Porter algorithm, filtering out short words with two or fewer letters, and removing common stop words except the word 'with', we ranked all words according to the score: S(w) = |pTP(w) - pTN(w)|.  The words with the highest score S tend to be associated either with positive or negative abstracts. Therefore, such words are assumed to be good features for classification. In order to keep the system very lightweight, we used only the top 650 stemmed abstract words with largest S as our continuent features; the top 15 words are listed in Table S per ranked words becomes essentially flat (null derivative). Furthermore, more aggressive pruning of features (fewer features) resulted in worse performance on training data (as described below), and more conservative pruning of features (more features) resulted in no improvement.All three runs submitted use 650 word features extracted from the training data using a method broadly based on the spam filtering system SpamHunting . First, 2 = 422,500 possible word pairs, although not all occur. First, we removed all words not in the first feature set from the abstracts. Then, from these filtered abstracts (vectors), we obtained the second and third feature sets, which are comprised of 66,301 pairs of words immediately adjacent in the filtered abstract vectors (bigrams+) and 48,006 unique pairs of words that co-occur in these vectors, respectively. We also computed the probability that such word pairs  appear in a positive or negative abstract: pTP and pTN, respectively. Figure Sab = |pTP - pTN|, plotted on a plane where the horizontal axis is the value of pTP and the vertical axis is the value of pTN; we refer to this as the pTP/pTN plane. Table Sab. Plots and additional data for all feature sets are included in supplemental materials that are available online  and β ∈ , in intervals of Δλ = 0.25 and Δβ = 2. For each  combination, we computed the mean and variance of F-score and accuracy measures for the eight k-fold tests and eight additional data tests. We computed four ranks for the classifiers tested in the parameter range: S, bigrams+, and abstract co-occurrence pairs. Finally, we submitted a run with VTT using the parameters that yielded the smallest value of R, which was 640. The parameters were λ0 = 1 and β = 15, using the third feature set (abstract co-occurrence words). Thus, this combination optimized F-score and accuracy on the training and additional data. It is worth mentioning that the parameter set that optimizes these measures on both training and additional data is not the same one that optimizes results on training data alone . Figures The final values of nd β ∈ 1,0, in intTo compare the performance of the lightweight VTT system above, we tested it against machine learning methods that typically yield the best performance in classifications tasks: SVMs  and the S. We applied additional dimensionality reduction and then trained classification models to discriminate between positive and negative data. Dimensionality reduction involved a two-step process. First, a feature selection filter based on the t-test was used, in which all features with the p value below a prespecified threshold tf were retained. Then, we applied the principal component analysis ). Thus, although SVD and SVD-UI classified the abstracts in the same manner, they led to different rankings. This indicates that using Shannon's measure of entropy on the compound vector space yields a better ranking than distance from the SVD decision surface alone.Whereas the IAS subtask in the BioCreative II challenge focused on detecting relevant articles for extracting protein interaction information from collections of abstracts, the IPS and ISS focused on discovering and extracting the actual protein interaction information from individual, full-text documents. IPS focused on identifying pairs of proteins, described by their UniProt IDs , whose iwi, wj), from the third feature set. Because the purpose of these tasks is to identify portions of text in which PPI information appears, we do not need to worry about features indicative of negative PPI information. Thus, these features were chosen and ranked according to the highest values of the following:From the features extracted from abstracts in the IAS subtask, we collected 1,000 abstract co-occurrence word-pair features,  are not equally ranked. Using Equation 6, when PTN = 0, p = pTP, rather than the 1 we would obtain if we did not use the additional pTP factor. We refer to this set of 1000 stemmed word pairs, as the 'word pair feature set'.Where fppi(w). Then, we calculated the frequency of stemmed words of the entire training corpus of 740 full-text articles: fc(w). Finally, similarly to the word pair features above, we selected as sentence features the top 200 stemmed words which maximize the following score .B Largest number of protein mentions in paragraph. As in the IAS subtask, we also used ABNER to collect protein mentions in the full-text documents provided for these two subtasks. Paragraphs without protein mentions are thrown out (rank 0).C Largest number of sentence features in the paragraph. Each feature that occurs in a paragraph adds 1 to the count. Paragraphs without feature matches are thrown out (rank 0).From these three distinct paragraph preference orderings, for each document we produced another three rankings that aim to integrate this information in different ways. For each document, we rank paragraphs according to the following criteria.1. Rank product of preference orderings from A (word pair features) and B (protein mentions) above.2. Rank product of preference orderings from B (protein mentions) and C (sentence features) above.3. Rank product of preference orderings from A, B, and C above.Because paragraphs thrown out in A, B and C are rank 0, at this step only paragraphs with feature matches and protein mentions remain. The resulting three rankings constitute the paragraph rankings used in the three runs submitted for the IPS subtask: 1, 2, and 3, respectively.To obtain the actual PPI pairs contained in the paragraphs of ranks 1, 2 and 3 described in the previous subsection, we had to convert the textual mentions obtained with ABNER to UniProt IDs. Protein and gene references identified using the ABNER system were mapped to UniProt IDs through exact matching with either a gene or a protein name occurring in SwissProt - considering both primary names and synonyms. UniProt version 8.2 was used for the mapping; this is not the most current version and could have resulted in missing relevant mappings. These mappings were then filtered using the reduced UniProt subset provided by BioCreative II. This process typically resulted in many UniProt IDs for the same ABNER protein mention, mostly because the same protein name maps to different UniProt IDs for different organisms. We therefore filtered the protein mention to include only UniProt ID mappings associated with organisms in the set of MeSH terms of a given article. Unfortunately, many of the articles listed several organisms in their MeSH terms, and so in that case our system returns several UniProt IDs for the same protein mention, resulting in the high recall observed (see Results [above]).Finally, for the IPS task we returned all the combinations of protein pairs (UniProt accession numbers) occurring in the same sentence - for sentences included in the paragraphs of ranks 1, 2, and 3 above. For a given document (PMID), the rank of each PPI pair is the rank of the highest ranked paragraph in which the pair occurs in a sentence. We submitted three distinct rankings of PPI pairs according to the three ranks 1, 2, and 3 above. Because only paragraphs with feature matches and protein mentions remain after computing ranks 1, 2, and 3, we return a ranked list of all PPI pairs identified in every paragraph still in these three ranks.For the ISS subtask, we used only the PPI pairs that were obtained via the IPS rank 1 above.  We used a version of a method we previously employed in the first BioCreative competition to obtain additional, contextualized features associated with protein mentions . The assdi, j ∈ {0.1} is an element of the document's paragraph × words relation D: P × W; P is the set of all m paragraphs in a document, and W is the set of all n stemmed words also from that document. This yields a proximity network for each document, where the nodes are words wi, and the edges are the wpp proximity weights.Where Figures Next, for every PPI pair (obtained by IPS rank 1) occurring in a given document, we obtain the words closest to the protein labels in the document's proximity network. Notice that these protein labels are words identified by ABNER for the given PPI pair, and they should appear on the proximity network as regular nodes - unless stemming or other processing breaks them. For each protein pair we selected the five stemmed words (nodes) in the proximity network with largest minimum proximity to both protein names. These additional stemmed words were then added to the list of general features obtained from the training data, but only for the respective document. Therefore, each document contains general word features extracted from the entire corpus, plus five specific word features near to each PPI pair in the proximity network. The assumption is that these additional word features endow our method with additional context sensitivity.From ranked paragraphs, we selected passages (sets of three sentences) containing a given PPI pair. Finally, we submitted three runs to the ISS subtask.1. Passages ranked by largest number of occurring word pair features.2. Passages ranked by largest number of occurring word pair features, expanded with the extra, context-specific words extracted from the document's proximity network as described in the previous subsection.3. Same as 2, with the addition of the following factor (100/[rank 1 from IPS submission]) to the number of features found in the passage.ABNER, A Biomedical Named Entity Recognizer; AUC, area under the receiver operating characteristic curve; IAS, interaction article subtask; IDF, inverse document frequency; IPS, interaction pair subtask; ISS, interaction sentences subtask; MINT, Molecular Interactions Database; PIARE, Protein Interaction Abstract Relevance Evaluator; PPI, protein-protein interaction; SVD, singular value decomposition; SVD-UI, SVD with uncertainty integration; SVM, support vector machine; TN, true negative; TP, true positive; VTT, variable trigonometric threshold.The authors declare that they have no competing interests.AA-H produced all code necessary for preprocessing abstracts, computing training data partitions, and conducting the tests of the IAS task; he also participated in the development of the VTT algorithm and feature extraction in the IPS and ISS tasks. JK produced sentence features as well as general programming and analysis in the IPS and ISS tasks. AM programmed, tested, and participated in the development of the uncertainty integration (SVD-UI) method as well as ran ABNER for the IAS task; she also produced the proximity networks and programmed the feature expansion method used in the ISS task. PR conducted all portions of this work related to our SVM run as well as the SVM postchallenge analysis of data for the IAS task; he also extracted additional protein-relevant abstracts. AR computed and performed the SVD analysis in the IAS task. KV linked ABNER output to Uniprot data in the IPS task; she also participated in using ABNER in the IAS task. ZW participated in extracting features from full-text articles as well as general programming necessary for the IPS and ISS tasks. LMR was responsible for integrating the team and designing the experimental set up for all tasks; he also participated in the development of the VTT and SVD-UI methods in IAS, and developed the methods used in the IPS and ISS tasks, including the proximity network feature expansion.