A major application of exome sequencing is the identification of genetic variations likely to cause Mendelian diseases. This requires processing large amounts of sequence information and therefore computational approaches that can accurately and efficiently identify the subset of disease-associated variations are needed. The accuracy and high false positive rates of existing computational tools leave much room for improvement. Here, we develop a boosted tree regression machine-learning approach to predict human disease-associated amino acid variations by utilizing a comprehensive combination of protein sequence and structure features. On comparing our method, ENTPRISE, to the state-of-the-art methods SIFT, PolyPhen-2, MUTATIONASSESSOR, MUTATIONTASTER, FATHMM, ENTPRISE exhibits significant improvement. In particular, on a testing dataset consisting of only proteins with balanced disease-associated and neutral variations defined as having the ratio of neutral/disease-associated variations between 0.3 and 3, the Mathews Correlation Coefficient by ENTPRISE is 0.493 as compared to 0.432 by PPH2-HumVar, 0.406 by SIFT, 0.403 by MUTATIONASSESSOR, 0.402 by PPH2-HumDiv, 0.305 by MUTATIONTASTER, and 0.181 by FATHMM. ENTPRISE is then applied to nucleic acid binding proteins in the human proteome. Disease-associated predictions are shown to be highly correlated with the number of protein-protein interactions. Both these predictions and the ENTPRISE server are freely available for academic users as a web service at  Knowledge of an organism’s DNA sequence is not only indispensable for fundamental biological research, but is also of practical use in medical applications. With the rapid advance of next-generation sequencing technologies, it is now possible to obtain sequenced human exomes at affordable cost for use in personalized treatment and preventive diagnosis . Since gDespite the good performance of each method as assessed by their respective authors, their accuracy has much room for improvement, especially for distinguishing disease-associated variations from neutral ones on the same protein/gene. For example, a recent study  shows thIn this work, we address these issues, especially the issue of discriminating disease-associated from neutral variations in the same protein/gene. We present a computational method that takes advantages of evolutionary conservation using protein sequence entropy and the local environment of a given variation using predicted protein three-dimensional 3D) structures. The combination of this information is realized through the boosted tree regression machine-learning algorithm, 26. SeqD structuIn addition to testing on a conventional test dataset, to test the ability of methods to discriminate disease-associated variations from neutral variations on the same gene/protein, we construct a new balanced test dataset where each gene/protein has both neutral and disease-associated variations such that the ratio of neutral/disease-associated variations is between 0.3 and 3. This is in contrast to conventional test datasets  that will favor methods classifying all variations in the selected protein as either all neutral or all disease-associated. In other words, a protein is classified as being either disease neutral or disease associated.ENTPRISE—a comprehensive combination of sequence ENTropy and PRedIcted protein StructurE for predicting human disease-associated amino acid variations—shows consistent, significant improvement over widely used methods, e.g., SIFT[We have also carried out tests on large-scale neutral variations from the 1000 Genomes project to test .g., SIFT, PolyPhe.g., SIFT for all We then apply ENTPRISE to predictions of disease-associated variations in nucleic acid binding proteins. Systematic analysis shows the interesting result that some proteins are “hot” in that most mutated positions are disease-associated while others are not. A possible explanation of “hot” proteins is the involvement of protein-protein interactions. The fraction of positions having disease-associated variations is highly correlated with number of protein-protein interactions the mutated protein experiences. It is also highly correlated with the probability of a protein being disease-associated. Thus, these predictions provide guidance for future experimental investigations.fl is the normalized 3590 frequency profile. <> stands for averaging over the entire protein sequence.Evolutionary sequence conservation information is utilized by all existing methods. Variations in a sequence conserved position are expected to be more prone to cause disease than non-conserved positions. Here, sequence conservation is represented by the relative sequence entropy defined as the difference of a given position’s entropy from the average entropy of all positions in a given protein:The 3590 frequency profile is adopted from the PROSPECTOR threading algorithm for protein structure prediction , where 3The aforementioned relative entropy is one feature used in the boosted tree regression method, 26. OthVMT [VMT-lite[VMT method[p-value of 3.4 x 10−5 [comb for ligand virtual screening reaches its best performance as long as protein target model quality has a predicted TM-score ≥ 0.4. Thus a model with predicted TM-score ≥ 0.4 or 0.5 is not only statistically significant, but also practically meaningful. The list of predicted protein structures is available at http://cssb.biology.gatech.edu/repurpose/.Protein structures were predicted by the latest variant of one of the best protein structure prediction methods TASSERVMT –34. To mVMT . SpecifiVMT , the parVMT . After t[VMT-lite—a fast vMT method to the lMT method is the pMT method, 40 ≥ 0.4 x 10−5 ). The pr4 x 10−5  and is d4 x 10−5  shows th4 x 10−5 , we founα distance from the mutated Cα position found in the predicted or experimental 3D structures. The 12 Å cutoff is adopted from a machine learning approach for predicting protein-ligand binding affinity[Structure related features that will be used for boosted tree regression, 26 cons affinity and is n affinity.A summary of all features used in this work is given in Tm is a decision tree, ε is the shrinkage factor or learning rate, and Ntree is the number of trees. Two function values are adopted: 1 for disease-associated variations, and 0 for neutral variations. A total of 81 variables are used for regression: 1 from the relative entropy  and tes in Ref. ) that ar in Ref.  trained  in Ref. . We map  in Ref. . In factENTPRISE-TR and ENTPRISE-TE sets for training and testing, respectively. ENTPRISE-TR has 9,728 proteins with 14,497 disease-associated variations on 1,458 proteins & 32,163 neutral variations on 9,618 proteins, of which 22,424 variations are on 8,270 proteins without any disease-associated variations. ENTPRISE-TE has 9,725 proteins with 14,443 disease-associated variations on 1,461 proteins & 32,131 neutral variations on 9,609 proteins, of which 20,669 variations are on 8,264 proteins without any disease-associated variations.We then randomly partition the above set at the variation level into two approximately equal-sized sets called ENTPRISE-balance set. ENTPRISE-balance is not an independent set and is a subset of the above ENTPRISE-TE set having balanced disease-associated and neutral variations on the same protein. In this set, the ratio of neutral/disease-associated variations of proteins having disease-associated variations is between 0.3 and 3.0. It has 761 proteins with 4,501 disease-associated and 4,406 neutral variations. ENTPRISE-balance was created for testing methods’ performance on discriminating variations in the same protein. All three data sets along with their associated protein sequences are available at http://cssb.biology.gatech.edu/entprise/.A summary of the above three sets as well as the duplicate-sampled ENTPRISE-TR described below in training protocol is given in 1k-Genome set consists of 156,799 variations from 17,298 proteins. Of these, 60,026 variations (38.3%) are from 7,039 (40.7%) proteins known to be disease-associated according to the GeneCards database (www.genecards.org) [1k-Genome set are neutral variations, we assume that its false negative rate is negligibly small. In practice, in the Results section, we estimate its false positive rate to be <2% and note that no current method can achieve this level. To further confirm our assumption, we also test methods on the expert filtered neutral variation set VariSNP [For the false positive test, a large set selected from the 1000 Genomes data (~1/3) that have no overlap with the training data is employed. This set called the rds.org) . While t VariSNP .ftp://ftp.ncbi.nih.gov/genomes/H_sapiens/protein/[The wild type Human protein sequences are downloaded from /protein/.Our training set, ENTPRISE-TR represents the natural distribution of real world observed data. However, as shown in There are two basic ways of reducing imbalance in training data: under-sampling and over-sampling . Under-stp, tn, fp, fn are the numbers of true positives, true negatives, false positives and false negatives for disease-associated variations, respectively.For ENTPRISE-TE and ENTPRISE-balance benchmarking data sets, we evaluate methods using the following criteria:A list of experimentally verified or putative nucleic acid binding proteins in the human proteome was obtained from UNIPROT 1/7/2015 release) . A total release https://sites.google.com/site/jpopgen/dbNSFP [The results on the test set ENTPRISE-TE are shown in n/dbNSFP . To be fα distance cutoff for defining the contacting composition, as ENTPRISE_CUT8 is almost indistinguishable from ENTPRISE. The most important feature is the domain composition. Without it (ENTPRISE_NODOM), ENTPRISE will have MCC of 0.464 (though it is still better than many other methods). Using the domain composition features alone  will result in MCC of 0.691 that is even better than that of full ENTPRISE. However, it will have very poor discrimination ability for variations on the same protein as tested next on the ENTPRISE-balance set. Similar considerations remove ENTPRISE_NOCNT as the method of choice. Other features seem to have very little effect on this test set.By examining the performance of various alternative implementations of ENTPRISE, we find that ENTPRISE is not sensitive to CAnother challenging test that so far has not been performed by existing methods is on a large set comprised of entirely neutral variations, e.g., those from the 1000 Genomes project . Even thThe false positive rates (FPR) of six methods  on the two sets have a Pearson’s correlation coefficient of 0.999 with the fitting: FPR(1kgenome) = FPR(varisnp)*1.075+0.015±0.005. A perfect method should have FPR(varisnp) = 0 and will have FPR(1kgenome) = 0.015±0.005. This extrapolated value corresponds to the apparent true positive or false negative rate of the 1k-Genome set, which is < 2%.By examining the performance of alternative ENTPRISE methods and FATHMM on this set, we find that the domain composition has the most important contribution to lowering the false positive rate of ENTPRISE. Thus, domain composition has a favorable contribution to all three test sets .A ten fold cross-validation of ENTPRISE at PFAM family lTo test how robust ENTPRISE is to varying fraction of disease-associated variations used in the learning procedure, we carried out a new training procedure that uses only observed variations. This training set differs from our standard training set, i.e., the balanced set, where duplicated variations are introduced to balance the proteins having both disease-associated variations and neutral variations in the training data . We trained ENTPRISE on observed variation data (i.e. ENTPRISE-TR without duplicating variations on proteins in the training set). In practice, this set mainly consists of proteins with only neutral variations. In We perform a test of ENTPRISE for predicting cancer driver missense mutations using the benchmark sets from Ref.. The tesWe applied ENTPRISE to 3,041 nucleic acid binding proteins, most of which are DNA or RNA binding proteins involved in transcription. We analyze their “hot” spots defined as positions predicted to be disease-associated for all possible variations regardless of variation type. The normalized distribution of proteins versus the fraction of “hot” spots is shown in To understand why some genes like TP53 are so fragile to variations (viz. most of their positions are “hot”), we hypothesize that these proteins have, on average, more protein-protein interactions than others so that more of their surface residues are involved; in addition, buried residues are important for monomer stability. There ahttp://cssb.biology.gatech.edu/entprise/.We next examine the relationship between a protein being disease-associated and the fraction of “hot” spots in the protein of interest. viz. 4–5 times that of ENTPRISE. Thus, due to its low false positive rate  of “hot” spots that are disease-associated. The fraction of “hot” spots is highly correlated with the protein being disease-associated, which in turn is highly correlated with the number of protein-protein interactions it experiences.A slight disadvantage of ENTPRISE as with the FATHMM method is that it does not provide 100% coverage of a given exome. A few large proteins  cannot be modeled. Some proteins also have regions that cannot be reliably modeled (on average 8.5% of the residues in a protein lie in regions that we cannot model at the moment). Thus, its expected coverage is around 90%. Nevertheless, this small reduction in coverage should not impair its utility in practical applications. This is especially true in real world exome applications, where the number of putative disease-associated proteins is roughly a factor of 5 less than either SIFT or PPH2 . Thus, it is a useful tool for assisting in the examination of clinical exomes.It should also be noted that even though ENTPRISE can predict disease associated/cancer driver variations, it does not necessarily mean that it can predict loss or gain of biochemical function . Moreover, even if it could, there is no guarantee that such a functional change will cause the disease of interest. Only changes in proteins that play an essential physiological role underlying a disease are contributory to the disease. In our assumption that the 1000 Genome data to be neutral, the word “neutral” should not be taken as functionally neutral, but disease-associated neutral.S1 Fig(TIF)Click here for additional data file.S2 FigShown in figure are the combined predictions of the 10 fold cross-validation.(TIF)Click here for additional data file.S3 Fig(TIF)Click here for additional data file.S1 File(ZIP)Click here for additional data file.S2 File(PDF)Click here for additional data file.S1 Table(DOCX)Click here for additional data file.S2 Table(DOCX)Click here for additional data file.S3 Table(DOCX)Click here for additional data file.S4 Table(DOCX)Click here for additional data file.S5 Table(DOCX)Click here for additional data file.


Since protein–protein interactions present certain constraints on the functional divergence of macromolecular assemblies during evolution, it is possible to predict complexes based on orthology data. Here, we show that incorporating interaction information through network alignment significantly increases the precision of orthology-based complex prediction. Moreover, we performed a large-scale in silico screen for protein complexes in human, yeast and fly, through the alignment of hundreds of known complexes to whole organism interactomes. Systematic comparison of the resulting network alignments to all complexes currently known in those species revealed many conserved complexes, as well as several novel complex components. In addition to validating our predictions using orthogonal data, we were able to assign specific functional roles to the predicted complexes. In several cases, the incorporation of interaction data through network alignment allowed to distinguish real complex components from other orthologous proteins. Our analyses indicate that current knowledge of yeast protein complexes exceeds that in other organisms and that predicting complexes in fly based on human and yeast data is complementary rather than redundant. Lastly, assessing the conservation of protein complexes of the human pathogen Mycoplasma pneumoniae, we discovered that its complexes repertoire is different from that of eukaryotes, suggesting new points of therapeutic intervention, whereas targeting the pathogen’s Restriction enzyme complex might lead to adverse effects due to its similarity to ATP-dependent metalloproteases in the human host.Macromolecular assemblies play an important role in almost all cellular processes. However, despite several large-scale studies, our current knowledge about protein complexes is still quite limited, thus advocating the use of  Almost every major process in a cell, such as replication, transcription, translation and degradation, is carried out not by single proteins, but by macromolecular complexes, regulated through intricate networks of protein–protein interactions. To understand cellular behaviour on a systemic level, we thus need a comprehensive knowledge of the protein complexes present in the respective organism.Saccharomyces cerevisiae , which we previously defined . We evaluated both precision  and recall  in the same way as in our previous study . We found that incorporating interaction data through network alignment significantly increases the precision of orthology-based complex prediction . When using NetAligner with default parameters there is a significant increase in precision from 34.6% to 54.1% , which is also present when calibrating NetAligner for the alignment of human complexes to the yeast interactome  or vice versa  to whole species interactomes using NetAligner  for each query complex.To predict complexes in yeast, human and fly, which are the species with the most interaction data available, and to identify novel complex components, we aligned the non-redundant sets of human, yeast and mycoplasma protein complexes . Here, we did not use the complexes identified in large-scale studies to ensure that our predictions are based only on curated data sources. This yielded 257 non-redundant significant complex predictions in yeast and 89 in human . We found 307 non-redundant novel components across 181 yeast complexes and 175 non-redundant novel components in 65 human complexes  separately for each Gene Ontology (GO) category  , the majority of the predicted complexes in yeast (70%) and human (73%) are indeed functionally homogeneous. The fact that the fraction of human query complexes that are homogeneous is smaller than the fraction of predicted yeast complexes (63% vs. 70%) might indicate that our predictions are, on average, of similar quality as the manually-curated human complexes stored in the CORUM database . We found that 233 of our complex predictions in yeast (91%) and 87 in human (98%) were significantly functionally enriched with respect to the given species interactome, indeed suggesting specific cellular roles for those complexes and further validating our predictions. For the complete list of predicted complexes, see Materials & Methods). This resulted in HC sets of 105 yeast and 61 human complex predictions  localise to the mitochondrial matrix, while the other two (QCR1 and QCR2) localise to the mitochondrial inner membrane as core components of the Cytochrome bc1 complex. Although it might be that the MPP and QCR subunits form two separate complexes in vivo, a combined MPP/QCR complex might also exist, since the two subcellular localizations are adjacent, and it was observed that in plants, the MPP complex is actually integrated into the Cytochrome bc1 complex, with QCR1 and QCR2 being identical to MPPB and MPPA, respectively  , and commponent)  and 3B. database . When usdatabase  and 3B. database . In addidictions  and 2C, dictions  and 2D. dictions . All foudictions . NetAligdictions , but no ectively . Anotherectively . The yeaectively . The DNAectively , and theectively . In addiectively . TogetheFinally, comparing the distributions of the total number of predicted  and 2C aDrosophila melanogaster is an important model organism. However, there does not yet exist a dedicated database of protein complexes in fly, the first large-scale screen, reporting 556 complexes, has only recently been completed . This resulted in 66 non-redundant significant complex predictions originating from yeast and 219 from human , our complex predictions contained 252 non-redundant novel components across 56 complexes based on yeast data and 454 non-redundant novel components in 200 complexes originating from human data  revealed that the majority of the predicted fly complexes are functionally homogeneous, independent of whether they originated from yeast (70%) or human (68%) query complexes , indicating specific cellular roles for those complexes and further validating our predictions. For the complete list of all predicted fly complexes, see Materials & Methods and in silico validation experiments revealed that all six predicted components are involved in DNA replication and have ATP-binding capability, important for performing the loading of the DNA sliding clamp . This analysis revealed a rather complicated machinery of almost 200 protein complexes, of which the majority were yet unknown . The complex to interactome alignments led to only 11, 9 and 6 non-redundant significant predictions in those species, respectively , our predictions contained 86 non-redundant novel components across 6 yeast complexes, 68 across 9 fly complexes and 30 in 5 human complexes , only 55%, 44% and 33% of all predicted yeast, fly and human complexes, respectively, are functionally homogeneous. If the mycoplasma interactome (on which the query complexes are based) contained many false positive interactions, one potential reason for the observed low functional homogeneity could be that these complex predictions are less reliable. Other possible reasons include inherent differences in the complexes repertoires of those species or aspects of biology that are less well studied. In contrast, the fact that only 39% of all query mycoplasma complexes are homogeneous is likely due to a lack of functional annotations in that organism. unknown . To predectively . Compareomplexes . Based oomplexes , we can Overall, since we used only the high-confidence subset of mycoplasma interactions , we think the very low numbers of significant complex predictions (independent of the target species) and their low functional homogeneity suggest that the mycoplasma interactome (at least the part currently known) is very different from the interactomes of yeast, fly and human. This indicates that there probably exist protein complexes that are unique to the pathogen and might thus be targeted by drugs without causing adverse effects in the human host.Materials & Methods and All complex predictions, except for one in yeast, were significantly functionally enriched with respect to the given species interactome, suggesting that they perform specific biological roles. For the complete list of predicted yeast, fly and human complexes, based on mycoplasma data, see Comparing the different subsets of mycoplasma complexes that led to significant predictions in the other species , one comin silico validation experiments that we performed, our NetAligner-based complex predictions had about the same functional homogeneity as the known complexes we used for querying, and we were able to assign specific functional roles to almost all complexes. In contrast, predictions based on simple orthology mapping often resulted in reduced functional homogeneity. Finally, aligning the protein complexes of the human pathogen M. pneumoniae  using NetAligner , our receumoniae  to the ieumoniae . NeverthHomo sapiens), fly (Drosophila melanogaster), yeast (Saccharomyces cerevisiae) and mycoplasma (Mycoplasma pneumoniae) from the UniProt Knowledgebase release 15.8  between two complexes c1 and c2 was defined as: We collected non-redundant sets of protein complexes in human, yeast and mycoplasma. For this, we first extracted all human complexes from the CORUM database , the setTo identify novel components in our complex predictions, we compared them with the set of known complexes of the respective species. We took all 1826 known human complexes from the CORUM database , all 402h(c) of each complex c per GO category, defined as the maximum fraction of protein components p(c) that share the same GO term t , we determined the statistical significance of their enrichments in the complex with respect to the given species interactome, using a one-sided Fisher’s exact test with Bonferroni multiple testing correction and a strict We post-processed all sets of predicted complexes to define high-confidence (HC) subsets. A predicted complex has to fulfil the following criteria to be present in the given HC set: it has to (i) be part of the non-redundant subset of significant predictions, (ii) originate from a functionally homogeneous query complex, (iii) be homogeneous itself and (iv) share at least one homogeneous GO term with the given query complex. Query complexes were defined as high-confidence if they were functionally homogeneous.10.7717/peerj.413/supp-1Table S1HC yeast predictions - Prediction of yeast complexes by aligning human complexes to the yeast interactome (high-confidence predictions). All yeast predictions —Prediction of yeast complexes by aligning human complexes to the yeast interactome . HC human predictions - Prediction of human complexes by aligning yeast complexes to the human interactome (high-confidence predictions). All human predictions—Prediction of human complexes by aligning yeast complexes to the human interactome . # QCC/# TCC: number of query/target complex components; only significant GO terms are shown; those GO terms that also belong to the set of most abundant significant GO terms of the given query complex are highlighted in capitals; complexes are sorted by homogeneity, followed by the fraction and number of new components compared to all known complexes in the target species; new components are shown in lower case.This table contains four sheets with the following contents: Click here for additional data file.10.7717/peerj.413/supp-2Table S2HC predictions from human—Prediction of fly complexes by aligning human complexes to the fly interactome (high-confidence predictions). All predictions from human—Prediction of fly complexes by aligning human complexes to the fly interactome . HC predictions from yeast—Prediction of fly complexes by aligning yeast complexes to the fly interactome (high-confidence predictions). All predictions from yeast—Prediction of fly complexes by aligning yeast complexes to the fly interactome . #QCC/#TCC: number of query/target complex components; only significant GO terms are shown; those GO terms that also belong to the set of most abundant significant GO terms of the given query complex are highlighted in capitals; complexes are sorted by homogeneity, followed by the fraction and number of new components compared to all known fly complexes; new components are shown in lower case.This table contains four sheets with the following contents: Click here for additional data file.10.7717/peerj.413/supp-3Figure S1Example of using NetAligner  to prediClick here for additional data file.10.7717/peerj.413/supp-4Table S3HC predictions in human—Prediction of human complexes by aligning mycoplasma complexes to the human interactome (high-confidence predictions). All predictions in human—Prediction of human complexes by aligning mycoplasma complexes to the human interactome . HC predictions in fly—Prediction of fly complexes by aligning mycoplasma complexes to the fly interactome (high-confidence predictions). All predictions in fly—Prediction of fly complexes by aligning mycoplasma complexes to the fly interactome . HC predictions in yeast—Prediction of yeast complexes by aligning mycoplasma complexes to the yeast interactome (high-confidence predictions). All predictions in yeast—Prediction of yeast complexes by aligning mycoplasma complexes to the yeast interactome . #QCC/#TCC: number of query/target complex components; only significant GO terms are shown; those GO terms that also belong to the set of most abundant significant GO terms of the given query complex are highlighted in capitals; complexes are sorted by homogeneity, followed by the fraction and number of new components compared to all known complexes in the target species; new components are shown in lower case.This table contains six sheets with the following contents: Click here for additional data file.10.7717/peerj.413/supp-5Table S4Human interactome—List of binary protein–protein interactions in human  or interaction detection method(s)). Fly interactome—List of binary protein–protein interactions in fly  or interaction detection method(s)). Yeast interactome—List of binary protein–protein interactions in yeast  or interaction detection method(s)). Mycoplasma interactome—List of binary protein–protein interactions in mycoplasma . Interaction reliabilities were estimated based on the number of publications supporting the given interaction .This table contains four sheets with the following contents: Click here for additional data file.


Many protein complexes are densely packed, so proteins within complexes often interact with several other proteins in the complex. Steric constraints prevent most proteins from simultaneously binding more than a handful of other proteins, regardless of the number of proteins in the complex. Because of this, as complex size increases, several measures of the complex decrease within protein-protein interaction networks. However, k-connectivity, the number of vertices or edges that need to be removed in order to disconnect a graph, may be consistently high for protein complexes. The property of k-connectivity has been little used previously in the investigation of protein-protein interactions. To understand the discriminative power of k-connectivity and other topological measures for identifying unknown protein complexes, we characterized these properties in known Saccharomyces cerevisiae protein complexes in networks generated both from highly accurate X-ray crystallography experiments which give an accurate model of each complex, and also as the complexes appear in high-throughput yeast 2-hybrid studies in which new complexes may be discovered. We also computed these properties for appropriate random subgraphs.We found that clustering coefficient, mutual clustering coefficient, and k-connectivity are better indicators of known protein complexes than edge density, degree, or betweenness. This suggests new directions for future protein complex-finding algorithms. Some of these complexes, such as the proteasome, are well-characterized, but others are not. In addition, it is hypothesized that there are many protein complexes in the cell that have not yet been identified. Complexes play an important role in the function of the cell, and by discovering new complexes and learning more about their composition and structure, we can gain insights into cellular biology.Proteins are a critical unit in biology. Rather than performing their function alone, many proteins formEver since high-throughput protein-protein interaction data sets have become widely available, scientists have used the interaction data to create graphs called protein-protein interaction (PPI) networks. The vertices in PPI networks represent proteins, and there is an edge between two vertices if the corresponding proteins interact. These graphs are not perfect models of protein interaction in an organism since the experiments that produced the edges are error-prone and contain both false positives and false negatives. Despite these errors, however, they are useful tools for studying the proteome of an organism.7. Most edge density algorithms search for subgraphs with edge density above a certain threshold4. Other properties that have been used include clustering coefficient7, degree statistics8, maximum flow9, and path length11. Biological networks have also been examined using the properties of mutual clustering coefficient13 and betweenness centrality17.One use for PPI networks is to predict unknown protein complexes from the interaction data. Previous algorithms have used several different properties to find complexes. By far the most common property has been edge density, the fraction of pairs of nodes (possible edges) that have an edge connecting themk-connectivity of a graph is a measure of how many distinct paths exist between any pair of vertices. A graph or subgraph isk-connected if there arek disjoint paths between every pair of nodes, or equivalently, if the removal of at leastk vertices or edges from the graph are required in order to disconnect it. We believe that a highk-connectivity may be more indicative of a protein complex than other measures, and can serve to identify protein complexes even with low edge density. If each protein in the complex binds to some number of adjacent proteins, then as the number of proteins in the complex increases, the edge density will decrease because the maximum number of proteins that a single protein can bind to is limited by steric constraints. Thek-connectivity, however, will stay roughly constant as long as each protein remains bound to roughly the same number of neighbors. Also,k-connectivity implies a certain degree of stability, and a complex with a highk-connectivity might be able to retain its structure and even partial function in the event of a mutation that caused an interaction to be lost or for a certain protein to be missing altogether.Thek-connectivity has only rarely been used in connection with finding protein complexes. Habibiet al.18 found that, in mass spectrometry data,k-connectivity was a better indicator of protein complexes than edge density. Hartuv and Shamir19 looked for connected subgraphs of n proteins that aren/2-connected; however, because their stopping condition is a function of the number of proteins in the subgraph, this is closer to a measure of edge density thank-connectivity.k-connectivity is a useful indicator of complexes in pairwise interaction data, we examined known complexes in the iPFam20 and MIPS21 databases. For each of these known complexes, we computedk-connectivity as well as various other topological properties, with a particular focus on those used in previous complex-finding algorithms: edge density, degree statistics, clustering coefficient, mutual clustering coefficient, number of triangles and 4-cycles, and betweenness centrality. We calculated these statistics in protein interaction graphs representing complexes. For each complex we used interactions determined by low-throughput X-ray crystallography data, where available, as well as high throughput yeast 2-hybrid (Y2H) studies. Finally, in addition to surveying these topological measures in complexes, we compared them to those of random complex-like subgraphs, which we callpseudocomplexes, pulled from the PPI network. This allows us to assess the utility of each of these statistics for discovering unknown protein complexes.In order to test the hypothesis thatet al.18 study in several ways. First, we used low throughput X-ray crystallography for data on complexes where it was available in order to obtain a “ground truth” about interactions in complexes. The information from this ground truth data, while only available on a limited number of complexes, can give evidence thatk-connected subgraphs are an important property of complexes independent of the data set in which we are looking at them. Second, we used Y2H studies for our high throughput data rather than mass spectrometry studies used by Habibiet al. Our use of a different type of high throughput interaction data can offer evidence that the ability ofk-connectivity to find complexes can be applied more generally rather than being a particular property of mass spectrometry data. Y2H data is also better suited for network studies due to the fact that it is binary; Y2H assays reveal the presence of an interaction between exactly two protein, unlike mass spectrometry studies which involve interactions between large sets of proteins and can be difficult to translate into the binary interactions required by networks22. Third, we examine a wider variety of statistics than the earlier study, which focused exclusively onk-connectivity and edge density, which gives a more extensive look at various statistics that could be useful in complex-finding, and howk-connectivity and edge density both rank among these. Finally, in addition to surveying these topological measures in complexes, we compared them to those of random complex-like subgraphs, which we call pseudocomplexes, pulled from the PPI network. This allows us to assess the utility of each of these statistics for discovering unknown protein complexes.Our study compliments the HabibiSaccharomyces cerevisiae from two different sources. The first source was iPFam, where we were able to obtain data about protein complexes as well as which proteins interact within the complex20. These interactions were determined via X-ray crystallography, which, while not perfectly accurate, should be considered highly reliable. Unfortunately, only 13 complexes with at least three distinct proteins were included in this database. The second source of data on known complexes was the MIPS database21. The MIPS database is far more extensive, but only contains the proteins present in the complex, not the interactions that occur within the complex.We obtained details about protein complexes in23. We did not include data from high-throughput affinity purification-mass spectrometry experiments, as did Habibiet al.18, because these experiments are biased towards protein complex interactions, and we sought to understand the properties of protein complexes and how these differ from a random background. To discover new protein complexes, it is appropriate to use all available data, as in the Habibiet al. study, but this was not our purpose. In addition, we wished to avoid complications from representing mass spectrometry interactions, which are not intrinsically binary, in a binary graph. We used high-throughput Y2H interactions exclusively because they are intrinsically binary, and do not suffer from known biases towards interactions within protein complexes. For similar reasons, we did not use the PCA binary interactions from Tarassov24 because that study used known complexes to filter the results and therefore would be biased in favor of known complexes.We obtained pairwise Y2H interaction data from Biogrid and created an interaction graph using a composite of all Y2H studies in yeast available on Biogrid25, a subset of the interactions with higher confidence, but there aren’t enough interactions in this data set between proteins in the same complex to give us meaningful results; only 25 of the 154 complexes in MIPS induced a connected graph, and of those 25, only 4 had more than 3 proteins in the data. This was not enough data to give a meaningful picture of complexes, so we decided it was better to accept the lower quality but higher number of interactions from the composite data set. It is worthwhile to discover metrics that would allow us to find protein complexes in the abundantly available data. We therefore decided to accept a lower specificity and a higher number of false positives in order to increase the sensitivity.The high-throughput Y2H data set, however, has a high error rate and includes both false positives (proteins that don’t interact but have been reported to interact in one or more studies) and false negatives (proteins that do interact but whose interaction has not been reported in a Y2H study). We considered using the Y2H Union subset of interactionsIn order to avoid confusion, for the remainder of the paper, we will refer to the entire collection of proteins and interactions determined by Y2H interactions as the “network”. The collection of proteins and interactions in a complex will be a “graph” while a subset of those interactions will be a “subgraph”.https://github.com/suzanneg/complex-stats.For the complexes from iPFam, we looked at both the interactions determined by the X-ray crystallography on isolated complexes and also the graph induced in the Y2H network by the proteins determined to be in the complex and all Y2H edges amongst these proteins. SeeWe assessed the following graph measures:k-connectivity.Edge and vertex Measures of the number of distinct paths between any pair of vertices. A graph or subgraph isk-edge-connected (k-vertex-connected) if between every pair of nodes there are at leastk edge-disjoint (intermediate vertex disjoint) paths. Equivalently, anyk-1 edges (vertices) can be removed from the graph without disconnecting it. In the remainder of this paper,k-connectivity refers to vertexk-connectivity.Edge density. The number of interactions (edges) divided by the number of possible interactions (pairs of vertices).Degree statistics. The maximum, minimum, and mean degrees for each graph, along with the standard deviation of the mean. In order to compare these statistics between complexes with differing numbers of proteins, we normalize by dividing the degree statistics by the number of vertices in the graph.Clustering Coefficient (CC). A measure of how many of a vertex’s neighbors are neighbors of each other. Over a graph or subgraph, clustering coefficient is defined as 3 times the number of triangles divided by the number of length 2 paths.Mutual Clustering Coefficient (MCC). For a pair of vertices, the percentage of their neighbors that they share. There are several different ways of defining the mutual clustering coefficient between two vertices, but for our purposes, we define it as the number of shared neighbors divided by the minimum degree (number of neighbors) of the two vertices. This method was the best of the ratio methods from Goldberg and Roth13 for assessing confidence in PPI networks. We calculate the MCC between all pairs of vertices in a complex, and as with degree, we report the maximum, minimum, mean, and standard deviation.Motifs. Particular subgraphs in each complex. We were interested in the number of triangles and 4-cycles.Betweenness centrality. For a vertex, the number of shortest paths between all other pairs of vertices that contain that vertex. Again, we report the maximum, minimum, mean, and standard deviation. As with the degree statistics, we normalize by dividing by the number of vertices in the graph. Because complexes are expected to be well-connected, we expect betweenness values to be small.7); and 3) the subgraph that isk-connected for the highest value ofk, which we call the most highly connected subgraph (MHCS).For each graph of a complex, we looked at three subgraphs: 1) the original graph, which includes vertices representing all proteins in the complex; 2) a “haircut” subgraph, where we recursively eliminate all vertices of degree 1 or less, ensuring the subgraph has a minimum degree of 2  and , removing those edges, and replacing them with edges  and . We use the method recommended by Miloet al.26: for a network withn vertices, the process is repeated 100n times to ensure proper mixing. The end result is a random network with the same degree distribution as the original network27. This process is repeated 10 times, giving us 10 random networks for comparison.In order to assess the significance of properties in the complexes and the Y2H network as a whole, we used two different methods of generating random graphs. For the Y2H network, we generated networks with the same number of vertices and the same edge distribution by “switching”. Switching works by choosing two random edges with different endpoints,  complexes contained at least one triangle. Although this bias may make pseudocomplexes more likely to contain a triangle than real complexes are, we believed it was better to be overly conservative in this respect. We considered only complexes with at least 4 proteins because fewer nodes in a connected subgraph require some measures to be unreasonably high, and this would skew our comparisons. We calculated the same measures for pseudocomplexes as we did for the complex graphs, and compared our results with the real complexes.A somewhat different method was used to assess the significance of the properties of the complexes. Switching would only allow us to compare a protein complex graph with another graph of the same degree distribution, when what we really want is to compare it to other graphs from the Y2H network. Our question is “how likely are we to see this result in the actual network where there is not a complex?” so we seek graphs that are similar to our complexes. For each complex with at least 4 proteins, we found a “matched” graph that we call aThere were 35 studies in iPFam that involved complexes with at least 3 proteins. Some of these studies were of the same or similar complexes; we grouped studies together if they produced the exact same graph, i.e. the same proteins with the same set of interactions. This grouping gave us 13 distinct graphs. All graphs are illustrated inIn all except two cases, the interactions from the X-ray crystallography produced connected graphs. Most complexes were only 1-connected due to the presence of a small number of degree 1 vertices; in all cases except one, the haircut subgraphs were at least 2-connected. About half the complexes had a subgraph that was at least 3-connected. In general, the edge density could be closely correlated with the number of vertices in the complex; complexes with only 3 proteins produced cliques while those with 12 or more tended to have edge densities closer to 1/3. Clustering coefficients had a similar pattern to edge density in that the value was closely correlated with the number of vertices in the complex. Mutual clustering coefficients were more scattered, but also tended to decrease as the number of vertices increased.When we look at the iPFam complexes in the Y2H data, we see that 9 of the 13 have all of their proteins present, 3 have slightly more than 60 percent, and 1 has only 1 out of 4 proteins present. Only in one, a complex with 3 proteins, were all of the interactions from the X-ray crystallography present in the Y2H data. With the exception of that complex, none of the complexes induced connected graphs, and they all had edge densities of less than 0.1. In all except two cases, the haircut produced an empty subgraph. Only two complexes had a subgraph that was at least 2-connected. Most graphs had clustering coefficients of 0. Average mutual clustering coefficients were higher, between 0.14 and 0.63. Comparing these results with the results obtained using the X-ray crystallography data set gives an indication of how many interactions have not been detected using a Y2H assay and how these false negatives make it difficult to detect complexes. Note that, ink-connectivity, edge density, clustering coefficient, and mutual clustering coefficient are summarized inResults onFull results from our survey of MIPS database complexes including all of the statistics. There is a separate excel sheet for results for the full graph , the haircut subgraph (Haircut), and the most highly connected subgraph (Max Connect). The final sheet (No Interaction) contains a list of all complexes where there were no interactions between any of the component proteins in the Y2H data. In each spreadsheet, complexes that induced a connected subgraph and complexes that did not induce a connected subgraph are listed separately. Further instructions for how to interpret these tables are in the README spreadsheet.Click here for additional data file.Copyright: © 2015 Gallagher SR and Goldberg DS2015Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).k-connectivity are shown at the top ofk-connectivity in the full complex graphs, the haircut graphs, the MHCS, and all connected components of complexes. From this we can observe that most complexes are at most 1-connected, but when degree 1 vertices are removed, all complexes not destroyed by this operation  are 2-connected. Many complexes also had a subgraph with even higher connectivity.The results onk-connectivity in real complexes and pseudocomplexes are shown on the top right ofk-connectivity of the MHCS rather than the entire complex or pseudocomplex. This was done because most complexes and pseudocomplexes had ak-connectivity of 1. It was only looking at the MHCS that the differences between complexes and pseudocomplexes became apparent. While roughly the same number of complexes and pseudocomplexes had a 2-connected subgraph, a far higher percentage of complexes had more highly connected subgraphs. Note that pseudocomplexes were designed to have, with high probability, a triangle (a 2-connected subgraph).Comparisons betweenk-connectivity due to the high standard deviation of the pseudocomplexes: the point where the maximum difference between known complexes and pseudocomplexes can be seen , 0.5, was well within a standard deviation of the average for pseudocomplexes. The obvious cut-off point fork-connectivity, 3-connected, by contrast was more than a standard deviation away from the average of the pseudocomplexes. The number of complexes with high clustering coefficients was also quite small, but clustering coefficients had a far more dramatic contrast with pseudocomplexes, especially for lower thresholds. Again, however, there was a fairly high deviation among pseudocomplexes. Mutual clustering coefficients have higher raw values but much less of a contrast with pseudocomplexes. When the deviation of pseudocomplexes is considered, mutual clustering coefficient does not differentiate from complexes as well ask-connectivity.The remainder of28. This pattern does not, however, appear to hold completely true for all PPI networks; specifically, inDrosophila melanogaster, triangles appear to be more overrepresented than 4-cycles29.There are a few further things to note about clustering coefficients and mutual clustering coefficients. Clustering coefficients were quite high in haircut graphs, but this is somewhat misleading. The haircut can remove length 2 paths from the graph but cannot remove any triangles; therefore, we would expect to increase clustering coefficient, but this increase would not necessarily help us in finding complexes. Average mutual clustering coefficient is much higher than clustering coefficient. The reason for this is that there are many more 4-cycles than triangles. While triangles are overrepresented in the Y2H network as compared to a random network of the same degree distribution produced by switching , 4-cycles are also overrepresented . The frequencies of triangles and 4-cycles relative to random networks has been calculated for a previous yeast PPI network, also with the result that both were overrepresented, with 4-cycles also overrepresented by a higher margin, though this was not stated explicitlyThis pattern also seems to hold in the complex graphs. Neither triangles nor 4-cycles were particularly prevalent in complexes relative to pseudocomplexes (which were each seeded within a triangle), but 4-cycles were more prevalent than triangles. In 50% of complexes, there were more 4-cycles as compared to matching pseudocomplexes. However, only 29% of complexes had more triangles than their matching pseudocomplexes.8. However, there are some problems with using this idea to search for complexes in the data. The first is that we did not notice a strong correlation between proteins with high degree and proteins that appear in known complexes; roughly 30% of proteins of degree 3 or higher in our data set appeared in at least one complex, and this number remained roughly constant as we increased the degree threshold until it eventually started decreasing due to the limited number of proteins with degrees above 20. The second problem is that if we look at the protein in a complex with the most interactions with other proteins in that complex, the majority of its interactions in the Y2H data are not within the complex. Therefore, the strategy of looking for a protein of high degree and taking it and all of its neighbors as a complex seems unlikely to produce meaningful results for finding protein complexes in Y2H data.The normalized results for maximum degree and comparisons with pseudocomplexes are inNormalized maximum betweenness is also shown inWe used a PPI network whose interactions were determined solely by high-throughput Y2H assays. Other binary interaction data sets, such as small-scale experimental data and literature curated interactions, were not used in this study due to the fear that they would be biased in favor of interactions in known complexes. While these interactions would be included in the data set used by an algorithm looking for unknown complexes, they should not be included in an attempt to learn the properties of complexes and what differentiates them from random.Similarly, we chose not to use non-binary data such as affinity purification data in this study. While these data again might be used in a complex-finding algorithm, the correct way to translate the data from these non-binary experiments into the binary interactions required by graphs is not completely obvious. The two commonly used methods (clique and spoke) produce very different topological properties, and neither captures well the underlying biology. Therefore, we decided to sidestep the issue by using only binary data. Future studies may include finding a way to use these data.30.As we carried out this analysis, we were always aware of the fact that our data are error prone. We must keep in mind that the absence of an edge does not mean that there is no interaction. In order to see that we have false negatives, we need only look at the complexes with their interactions determined by X-ray crystallography and compare them to the interactions of those same proteins in the Y2H data . PresumaWhile false positives may cause problems in complex-finding algorithms, our survey suggests that false positives may be less of a problem than false negatives. If we had used a cleaner data set, we would have had fewer false positives but also fewer true positives, and we would have had even more difficulty discerning complexes. Even in the data set we used, complexes often did not stand out when compared to pseudocomplexes.While the errors in the Y2H data are noteworthy, we do not feel that they represent a weakness in our study. To the contrary, a complex-finding algorithm would also be working in this same error-prone data. While it would be interesting to know how a complex would appear in a completely correct network, it is more useful to know how it appears in the data we have.Another point about our data worth noting involves the pseudocomplexes used for comparison to represent “background” areas of the graph. Because the generating algorithm was trying to find “complex-like” subgraphs, some of our “pseudocomplexes” may in fact be unknown protein complexes. This would skew our results somewhat, but generally gives a conservative comparison; some unique features of true complexes may not be discovered, but it is less likely that noted differences between true complexes and the set of “pseudocomplexes” are spurious.et al.31 looks for complexes with an edge density of at least 0.7 with a minimum number of proteins. If this algorithm were applied to Y2H binary interaction data , our research suggests that such a technique would find all of the proteins involved in a complex for just over a tenth of known complexes with 3 or more distinct proteins. An edge density threshold of 0.7 would find the MHCS of about 60% of known complexes, thus finding at least part of the complex, but this still leaves more than a third of complexes undetected. Also, on average, the edge densities in complexes were only slightly higher than the edge densities in the pseudocomplexes, which suggests that edge density may produce many false positives as well. Therefore, while edge density has a role in complex-finding algorithms, we would be skeptical of methods that purport to find complexes in Y2H data based solely on edge density.We found that edge density may have been overrated as a property of complexes. We found that in Y2H data, the complexes were not particularly clique-like and edge densities were nowhere near as high as most complex-finding algorithms assumed. For example, the algorithm used by KingClustering coefficient has not been as popular a parameter for complex-finding algorithms as edge density, but it has long been one of the standard tools used to study the PPI network and its subgraphs. We found that clustering coefficients in real complexes were higher than those from equivalent pseudocomplexes.Mutual clustering coefficient is another statistic that has not been used extensively in complex-finding algorithms, but we believe shows promise. Many complexes have high average mutual clustering coefficients as seen inLooking at maximum degree, we can see that many complexes have at least one protein with interactions with a high percentage of the other proteins in the complex. At the high end, this differentiated complexes from pseudocomplexes. However, we were not able to correlate proteins of high degree with proteins present in known complexes. Also, even among high-degree proteins that were present in complexes, the majority of the neighbors of those proteins were not co-complexed. For these reasons, we are hesitant to recommend degree as an important part of a complex finding algorithm.Betweenness was one of the statistics that performed the most unexpectedly. Vertices of high betweeenness are usually believed to be vertices that exist between different biological modules. Under that assumption, we would expect all vertices in a complex to have low betweenness. However, when we looked at complexes under this assumption, we found that most complexes had at least one vertex with a higher betweenness than their pseudocomplex counterparts. Therefore, any algorithm that partitioned the network by looking for high betweenness vertices would run the risk of dividing complexes. It is possible that betweenness could still be used in a complex finding algorithm, but likely not in the way that it has been used traditionally.k-connectivity of complexes, on the other hand, stood out versus thek-connectivities of the pseudocomplexes. Our results were mixed but promising. Most complexes were only 1-connected, but this was due to a small number of degree 1 vertices. When these vertices were removed by the haircut, a 2-connected subgraph usually remained, and many complexes had 3-connected or 4-connected subgraphs. The presence of 3-connected and 4-connected subgraphs is significant; because of the way we generated our pseudocomplexes, they were biased towards including a 2-connected subgraph , but very few had a 3-connected subgraph. Almost none of the pseudocomplexes that were designed to mimic the connected complexes had a 4-connected subgraph.Thek-connectivity is that, while some of the haircut graphs were empty, none of the others had ak-connectivity of 1. Eliminating vertices of degree 1 is not by itself enough to guarantee that a non-empty graph will be at least 2-connected, so this result is significant. It indicates that removing all degree 1 vertices from complexes also eliminates all articulation points, vertices whose removal disconnects the graph, leaving behind a graph where no one vertex can be removed to disconnect the graph. It should also be noted that while our results onk-connectivity in the error-prone data were promising, our results in the more accurate X-ray crystallography data were even more so. In the X-ray crystallography data, all complexes had at least a 2-connected subgraph, and the majority of complexes had a 3-connected or 4-connected subgraph. This suggests that as our data become more complete and accurate, highly connected subgraphs will play an even stronger role in searching for complexes.Another feature that is noteworthy aboutet al.18. The fact thatk-connectivity was shown to be an important indicator of protein complexes in a different type of experimental data than the one used by Habibi et al suggests that the importance ofk-connectivity is real and not just an artifact of one type of data.Our analysis confirms the connection between highly connected subgraphs and protein complexes first suggested by Habibiet al.18 present an algorithm for finding complexes based onk-connectivity. We are somewhat skeptical of using vertex connectivity alone as the basis of a complex finding algorithm in Y2H data, however, because subgraphs with these connectivities are too common; it is easy to find 2- or 3-connected graphs of almost any size in the PPI network. Starting with a triangle, it is possible by adding one vertex at a time to build a 2-connected subgraph of any size up to 1689 vertices. Starting with a 4-clique, it is possible to build a 3-connected graph of any size up to 913 vertices. Nevertheless, we feel these vertex connectivity results are significant. The MHCS of graphs representing real complexes were much more highly connected than those of pseudocomplexes, despite our method of generating pseudocomplexes being (perhaps unfairly) biased towards higherk-connectivity, and less biased towards higher edge density. The presence of a highly connected MHCS was one of the statistics that most differentiated real complexes from pseudocomplexes, suggesting thatk-connectivity has a role in complex-finding algorithms. The absence of articulation points and the presence of highly connected subgraphs indicates something about the structure of complexes.In their paper, Habibik-connectivity should be used in conjunction with other properties in a complex-finding algorithm. Several other properties examined in this survey, most notably clustering coefficient and mutual clustering coefficient, were also highly correlated with complexes. A complex-finding algorithm based on these data could try to build a 3- or 4-connected subgraph that also had high clustering coefficients and mutual clustering coefficients. Several existing complex-finding algorithms use multiple criteria, such as MCODE 7, the algorithm of Kinget al. (clustering and edge density)31, and the Bayesian network of Qiet al. 32. Connectivity could also be used to evaluate candidate subgraphs produced by other complex-finding algorithms. Subgraphs found by other methods could be examined to find their most highly connected subgraph, with higher confidence scores being given to those with higherk-connectivity values for their most highly connected subgraphs. Finally, we hypothesize that the most highly connected subgraph of a complex graph may correspond to the “core” of a protein-complex as described by Dezsoet al.33 and Gavinet al.34. If true, this would imply thatk-connectivity could be used in improvements to algorithms that use the core-attachment model36.We believek-connectivity, edge density, maximum normalized degree, clustering coefficient, mutual clustering coefficient, triangle (3-cycle) count, 4-cycle count, and betweenness centrality in various graphs representing known protein complexes in the high-throughput Y2H data available for new protein complex discovery. For each known protein complex, we computed these properties in the graph induced by proteins contained in the complex in the Y2H network as well as in the haircut and MHCS subgraphs of these, which are more likely to be discoverable by an automated method. To measure the significance of our results we computed the same properties as we did for the complexes on random “complex-like” graphs from the Y2H network.Before designing a new algorithm to find unknown protein complexes in protein interaction data, we must understand the topological properties of known protein complexes. We conducted a principled and comprehensive survey of the topological properties of known protein complexes. We computed vertexk-connectivity, clustering coefficient, and mutual clustering coefficient appear to be the most effective measures for differentiating protein complexes from background pseudocomplexes in the pairwise Y2H interaction data. Importantly, our analysis suggests thatk-connectivity, a graph metric which has rarely been used in the study of protein networks, would improve algorithms designed to find protein complexes in protein-protein interaction data.Although the property of edge density has been the most commonly used measure when searching for complexes in the PPI network, we found that it may not be the best graph measure for protein complex discovery. Instead, we found thatThe data referenced by this article are under copyright with the following copyright statement: Copyright: © 2015 Gallagher SR and Goldberg DSData associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).Figshare: Characterization of known protein complexes using k-connectivity and other topological measures. doi:10.6084/m9.figshare.72908637 This manuscript provides an analysis of different features that could potentially be used as input for methods that identify protein complexes from pairwise interaction data from yeast two-hybrid screens. The manuscript is strong in using X-ray structures as a gold standard for real interactions and in performing parallel analysis of real complexes and pseudo-complexes drawn from reasonably realistic randomized networks. It motivates use of metrics based on clustering and connectivity rather than edge density or centrality.There are two areas where additional work could increase the value of this manuscript. First would be to examine which metrics perform better for the task of identifying complexes. A reasonable approach would be to use the different features to rank a mixture of real complexes and pseudo-complexes and to see which features perform better at ranking the real complexes above the pseudo-complexes. The second area would be to investigate properties of complexes that affect the performance of different features. I suspect that one of the explanations might be that some methods perform better for smaller vs. larger complexes. It would probably be helpful to look at performance for complexes grouped into size classes.Suggestions:et al.(2008) Park and Bader (2011).In the Background paragraph summarizing other methods, which starts “One use for PPI networks”, I think it would be useful to add “patterns of connectivity as represented by stochastic block models” as a feature used for identifying complexes. I think these are the best-performing methods. Two references are ClausetIn the Background, the statement “Y2H assays reveal the presence of an interaction between exactly two protein” should be “proteins”. Also, it might be too strong. It is possible that a third protein that forms a complex with the bait and prey and is co-localized to the cell compartment of the Y2H assay could bridge the bait and prey resulting in detection of an indirect interaction. Given access to the x-ray structures, they authors might be able to discover whether any of the published Y2H pairwise interactions are likely to be indirect rather than direct.et al. (2006); Huanget al. (2007); Gentleman and Huber (2007); and possibly Cusicket al. (2009).In several locations, false positives and false negatives are mentioned. It could be helpful to provide citations to estimates of false-positive and false-negative rates. Examples are HartMethods, Assessment: It could be helpful to mention that many of the Y2H assays were asymmetric, with a smaller number of baits used as queries against a larger library of preys. Some randomization methods have attempted to keep the distinction between baits and preys as part of the edge swapping.Results: Are some of the metrics more sensitive to the size of a complex? Can the authors provide some analysis as a function of protein complex size? Spatial constraints, for example, limit the edge density in real complexes. Clustering coefficients may have a better inherent normalization for cluster size.Results: It would be helpful to see how the different methods perform in identifying complexes. For example, suppose that the real complexes and pseudo-complexes are grouped together, and then the different features are used to rank-order the mixed collection. The ROC/AUC and PR/F-score could be used to give a rough idea of how well the individual features perform in distinguishing real from randomized complexes.Figure 2: It would be better to have axes that represent percentages to stop at 100% instead of 120%.Minor points:Abstract, “Saccharomyces cerevisiae” should be Italic font.Abstract, “subgraphs.We found” insert space after period.Background, “Proteins are a critical unit” change to “Proteins are critical units”Background, “contain both false positives and false negatives”. Might be better to say “subject to” rather than “contain” because the false negatives aren’t there, not really contained.Methods, “The code used for calculating the statistics of protein complexes can be found at https://github.com/suzanneg/ complex-stats.” It would also be helpful to provide the code and data sets as supplementary information to ensure continued availability of a snapshot of the material at the time of publication.I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. The idea of using k-connected subgraphs to predict protein complexes has already been investigated by other researchers. The authors should therefore clearly point out what is new about their concept and how it differs from earlier published methods.et al. (2007) Sharan the functional similarity of protein pairs will decrease as the distance of the path connecting them increases. Hence, a large k value results in a less functional correlation among proteins. Specifically, for k>= 3, functional similarity significantly decreases and for k<3, or k=1 or 2, the k-connected graph is dense. As this has already been investigated by others the authors need to clearly state what makes their method significant. According toThis paper is a survey of the topological properties of known protein complexes and is mostly focused on k-connected subgraphs as a good candidate for predicting protein complexes in PPI networks. K-connected subgraphs have been previously proposed as a good alternative for predicting less dense protein complexes. I have the following comments about this paper:I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. This is a promising paper. I think the idea is sound, has some novelty, and can potentially improve previous results. The explanations of the figures are of particular note. In addition, the authors could try a larger benchmark complex dataset (CYC2008 category).I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.


Saccharomyces cerevisiae in the Protein Data Bank (PDB) database. First, we build linear descriptions of interacting and non-interacting sequence segment pairs based on their inter-residue distances. Secondly, we train machine learning classifiers to predict binary segment interactions for any two short sequence fragments. The final prediction of the protein–protein interaction is done using the 2D matrix representation of all-against-all possible interacting sequence segments of both analysed proteins. The level-I predictor achieves 0.88 AUC for micro-scale, i.e., residue-level prediction. The level-II predictor improves the results further by a more complex learning paradigm. We perform 30-fold macro-scale, i.e., protein-level cross-validation experiment. The level-II predictor using PSIPRED-predicted secondary structure reaches 0.70 precision, 0.68 recall, and 0.70 AUC, whereas other popular methods provide results below 0.6 threshold . Our results demonstrate that multi-scale sequence features aggregation procedure is able to improve the machine learning results by more than 10% as compared to other sequence representations. Prepared datasets and source code for our experimental pipeline are freely available for download from: http://zubekj.github.io/mlppi/ .Accurate identification of protein–protein interactions (PPI) is the key step in understanding proteins’ biological functions, which are typically context-dependent. Many existing PPI predictors rely on aggregated features from protein sequences, however only a few methods exploit local information about specific residue contacts. In this work we present a two-stage machine learning approach for prediction of protein–protein interactions. We start with the carefully filtered data on protein complexes available for  Systems biology and bioinformatics study interactions between various biocomponents of living cells that spans across multiple spatial and temporal scales. The goal is to understand how the complex phenomena arise given the properties of building blocks. Specifically, proteins are characterised in multiple scales: first in the microscale, by their local post-translational modifications; second, by the interactions with metabolites and small chemical molecules (inhibitors); third in the mesoscale, by the three-dimensional structure of active sites, or interaction interfaces; fourth in the macroscale, by the global 3D structure that comprises the macromolecular complexes; and finally in the time-scale, by their dynamical properties related to the changes of their structure, or physico-chemical properties upon participating in the given biophysical process. Such variety of scales, each linked with different biological function is rooted in their complex and spatio-temporal network of interactions with other smaller biomolecules , comparable size proteins, RNA molecules, and much larger DNA macrochains. Starting from reliable information on single, binary interactions it is possible to reconstruct the whole interaction network, therefore providing further insight into proteins’ biological functions on the whole-cell level.In this paper, we focus on protein–protein interactions. We develop ensemble learning method for identification of proteins binary interactions by predicting residue-residue interactions. Moreover, we demonstrate how to integrate sequence information from the lower scales into the higher scale machine learning predictor. In our approach, binary interaction between two proteins is predicted by considering all possible interactions between their sequence segments using level-I predictor. An output of this phase is the matrix of scores with the size corresponding to the proteins’ lengths. Given the threshold on the likelihoods, this represent the whole network of all possible residue contacts that could be made during the complex formation. Later, we transform the scores matrix into a fixed length input vector suitable for further statistical data analysis , and we identify the network properties  using interaction graph. This data is used by the level-II predictor, which integrates information similarly to a human expert. Recently, several machine learning algorithms were applied to predicting protein interactions. Our study takes similar route to Our method differs from the previous approaches in several ways. First, we treat residue level predictions only as the input for identification of protein level interactions. The information flows only bottom-up: from the residue level to the protein level. In We compare our results with PPI predictors targeted at yeast which exploit global sequence properties . One such methods was developed by Another sequence-based method was proposed by Other popular protein sequence representations used for predicting protein–protein interactions include Pseudo Amino Acid Composition , 2-gramsMany other algorithms solving the same problem, but operating on different principles, were developed. One of the possible examples is PIPE predictor Here we would like to stress one of the most important outcomes of our study. Namely, the performance results reported in various publications are generally not comparable directly. They tend to differ in collected data methodology, definition of positives and negatives and evaluation procedures. In our work, we focused first on preparing the collection of high quality interactions data and removing any bias from the further evaluation procedure. To obtain a meaningful comparison between different methods, we reimplemented several schemas for aggregating features of protein sequences  structures of all yeast protein hetero-complexes from Protein Data Bank (PDB) , which wOn the residue level, from 3D structures it was possible to extract all interacting pairs of residues. We considered any two residues from two distinct proteins as interacting if they were located in Euclidean space within a distance threshold of 4 Å from each other. On the protein level, we identified two proteins as an interacting pair if there was at least one residue level microscopic interaction between them. We were interested only in heterodimers, i.e., interactions occurring between two different proteins. While predicting homodimers is also valuable, we decided to leave out homodimers and focus on heterodimers for two reasons: (a) hetero-interactions can contribute a lot to our understanding and the reconstruction of the true protein interaction network (PIN), (b) in our data homodimers occurred much more frequently and there was a risk that they will dominate heterodimers completely.The first step of our procedure was to build a training dataset for level-I predictor. We employed the sliding window technique to extract fragments of protein sequence. In this work we refer to it as extraction window.Positive examples in the training set were formed from pairs of fragments in which central residues interact, and additionally certain number of other residues within a specified distance from the central one interact. We refer to the required number of interacting residues as interaction threshold and to the maximal distance from the central residue as the maximal neighbour interaction distance. By introducing this restriction we deliberately focused on strong interactions, filtering out the weaker ones, which could be just noise in the crystallisation process.In the data preparation phase, we fixed the maximal neighbour interaction distance at 10 residues. We have chosen this value following the studies of Let us observe that the values of the maximal neighbour interaction distance and extraction window size are not necessary the same. One can imagine identifying residue level positives using larger interaction distance, and then encoding features of only a few central residues using small extraction window, and vice versa. Indeed, in our work we tested different sizes of extraction window ranging from 3 to 31 independent of the fixed interaction distance.In our data extracted from PDB we had no natural source of negative residue interactions. Pairing sequence fragments fully randomly could result in a lot of noise and false negatives. Therefore, we decided to extract non-interacting pairs of fragments only from interacting protein pairs. Fragment pairs without any interacting residues or fragment pairs in which one fragment has some interactions but the other has none were considered negatives. This way it was guaranteed that at least one of the fragments did not come from the interface region. The number of potential negatives was much larger than the number of positives, therefore we decided to keep the imbalance ratio at 3:1. The required amount of negatives was therefore sampled at random. This is a common practice in machine learning since most of the algorithms perform poorly on datasets with large class imbalance , …, Deg(nv)] be a vector containing degrees of vertices from V. Let G2 be a graph of negative interactions. At first it has vertices identical to G1 and no edges.Let •v such that Deg(v) > 0:While there exist 1.v with the largest Deg(v).Find vertex 2.u if exist such that:Find vertex (a)v, u) in G1.There is no edge ((b)u has as large Deg(v) as possible.(c)d in G1 is as large as possible.Distance 3.u exist:If (a)u, v) to G2.Add edge ((b)Deg(v)←Deg(v) − 1(c)Deg(u)←Deg(u) − 14.Deg(v)←0else: Such schema of constructing negative protein pairs set is unbiased, i.e., the protein composition of the positives and the negatives remains identical. Every single protein has the same number of positive and negative interactions. This forces the trained classifier to predict meaningful biophysical interactions rather than predicting general reactivity (the relative number of interactions) of a single protein. Otherwise, the best results would be achieved by a predictor, which predicts that the two proteins interact if each of them has a lot of interactions in general, regardless of their actual compatibility. It’s also important that our algorithm favours protein pairs which are remote to each other in the interaction network which reduces the risk of introducing false negatives.The last step of data generation procedure is splitting samples between training and testing sets. In order to truly evaluate our method in a realistic setup, we split the benchmarking dataset at the protein level, not at the residue level. This made our goal more difficult as compared to previous works that often used residue-level splitting of benchmarking dataset. The schematic depiction of the train-test split is given by Level-I predictor was trained to recognise interacting pairs of fragments. This should be the equivalent of detecting compatible protein patches on the surface of a protein. During prediction, for each possible pair of fragments from two different proteins a prediction is made, and the likelihood estimation for all against all pairs of fragments are stored in the interaction matrix. The Level-II predictor uses the output of level-I predictor, predicting binary interactions between two proteins using the aggregated features, i.e., the complementarity between their surface patches.We trained the level-I predictor on interacting sequence fragments of proteins from the training set and tested it on the testing set. Input for level-I predictor consisted of pairs of sequence fragments of the length of extraction window.The following sets of features were considered:•Raw sequence—raw sequence of amino acids encoded numerically.•HQI8—sequence of amino acids encoded with High Quality Indices . These f•β-sheet, H—α-helix, C—coil.DSSP structure—secondary structure of the protein extracted from PDB complex with DSSP software. It was limited to the three basic symbols: E—•β-sheet, H—α-helix, C—coil.PSIPRED structure—secondary structure of the protein predicted from sequence with PSIPRED software. It was limited to the three basic symbols: E—As the core classifier, we evaluated two popular machine learning methods: Random Forest and Support Vector Machine. Both algorithms are commonly used in bioinformatics and are considered the best off-the-shelf classifiers . Their pTo infer a binary interaction between two proteins, we consider all possible interactions between their sequence segments as predicted by level-I predictor. An output of this phase is a matrix of likelihoods with the dimension equal to the multiplied proteins’ lengths. Each prediction score is a real number between 0 and 1. Sample matrices of scores for a positive and negative case are presented in To transform the 2D matrix into an input vector suitable for level-II predictor, we extracted the following features :•the mean and variance of values over the matrix (2),•the sums of values in 10 best rows and 10 best columns (20),•the sums of values in 5 best diagonals of the original and the transposed matrix (10),•the sum of values on intersections of 10 best rows and 10 best columns (1),•the histogram of scores distributed over 10 bins (10),•graph features: fraction of nodes in the 3 largest connected components (3).The graph features require further explanation. Predicted contacts between residues were represented as a bipartite graph. Nodes in the graph represented residues and edges represented predicted contact. To make the graph more realistic biologically, for each node we left only 3 strongest outgoing edges. We set the value of this threshold (3) following the observation that in our PDB structures the mean number of interactions of a single interacting residue is between 2 and 3. In the trimmed graph we calculated fractions of nodes contained in 3 largest connected components. Those values were also appended to the feature vector.The performance of the level-II predictor was evaluated through a variant of stratified 30-fold cross-validation performed on the protein level. Each fold contained 1.Let 2.F ⊂ O:For each fold (a)P composed of all proteins occurring in F: Build a set (b)A ⊂ O composed only of pairs consisted of proteins occurring in P: Build a set (c)B ⊂ O composed only of pairs consisted of protein not occurring in F but occurring in the testing set: Build a set (d)B set, and test it on A.Train the classifier on 3.A-sets, and calculate performance metrics.Collect all the predictions for The above described procedure differs from the standard cross-validation, since the number of observations in constructed test sets vary slightly, but this variance is small, and does not influence the estimated performance. Such evaluation schema does not allow for any information leak: the datasets are always balanced, and the classifier is tested on previously unseen proteins.As classification methods for level-II predictor we used Random Forest and Support Vector Machine with parameters tuned through a grid search.We compared our ensemble method with various sequence feature aggregation schemas that are commonly applied in machine learning predictors of proteins interactions. To make the benchmarking results comparable between different algorithms, we used the same classification method (Random Forest) and evaluation procedure  as for level-II predictor. We benchmarked the following feature aggregation schemas:1.AAC—Amino Acid Composition . Feature2.k-th tier correlation factors added. We calculate those correlations on HQI8 indices.PseAAC—Pseudo Amino Acid Composition . Feature3.2-grams . Feature4.d combined values of this property d for a given amino acid pair are summed up for all the pair’s occurrences over the full protein sequence. Occurrences for pairs of residues separated from each other by 0, 1, 2…m residues. In effect, one obtains dQRC vectors of length 400 × m. In this model we also use HQI8 indices.QRC—Quasiresidue Couples . A set o5.ijx is the value of descriptor j for amino acid at position i in sequence P, n is the length of protein sequence P, and d is the distance between residues in the sequence. For the purpose of the comparison, we tested this method with the original 7 amino acid indices used by Liu, as well as with HQI8 features. We tested different values of L from 5 to 30 in a quick cross-validation experiment on our data and chose L = 9 as yielding the best results.Variation of Liu’s protein pair features . The metWe evaluated carefully all subsequent steps of our method to choose optimal features and parameter values. Then we compared performance of level-II predictor with popular sequence encoding schemas. In our experiments we stuck to the rule that during classifier training we can use all the information available in PDB complexes, but in the evaluation phase only information derived from the sequence is allowed. This was to demonstrate that our method can be employed successfully in a situation when only protein sequences are known.The first task was to decide on the set of optimal features for the level-I predictor. We have chosen ROC AUC as the performance metric. To have a complete picture, we measured both the performance of level-I predictor and the performance of level-II based on the trained level-I predictor. We tested two sources of secondary structure information—DSSP and PSIPRED—separately, but the evaluation was done using PSIPRED secondary structure (results on DSSP are given in parentheses for completeness). Initial experimentation was done with Random Forest, then we tested whether it is possible to gain anything by replacing it with a carefully tuned SVM.Evaluation results of the predictor on different features for 5 interaction thresholds are presented in The final decision was selecting the optimal extraction window size. After fixing the parameters, we wanted to choose classification algorithm for level-II predictor and compare performance of our multi-level representation with representations based on the aggregated protein sequence. Results are presented in Our results stress the link between secondary structure and residue contacts. Compatibility of structural motives is important for contact forming. To get more insights from our results, we extracted feature importances from the trained Random Forest model of level-I predictor. Relative importance of an attribute in a decision tree is expected fraction of examples split by nodes based on this attribute. In Random Forest this value is averaged over all trees. Relative feature importances for the input vector constituted by secondary structure annotation are given by To understand what is happening on these positions, we calculated frequency of particular secondary structure patterns occurring between the residues of interacting and non-interacting fragments. Calculated frequencies are presented in From our experiments it is clear that using the real secondary structure for classifier training is preferable over the predicted secondary structure, even if later only predicted secondary structure will be available. This limits the noise introduced during the training phase allowing the predictor to focus on the important patterns.What is surprising at first glance is that encoding both secondary structure and protein sequence in a single feature vector improves the prediction on level-I but leads to a worse prediction quality on level-II. It is true that protein sequence contains more information than just secondary structure, but, on the other hand, including more attributes increases the risk of overfitting. This is the case of so-called bias–variance trade-off, where increasing the model complexity potentially decreases classifier bias but at the same time increases classifier variance . Here, tWe may speculate that this phenomena comes directly from the hierarchical nature of biological systems. Biological processes have to be robust and predictable, harnessing the dynamic of physical molecules in a constructive way. It is easier to build the upper layers of a complex system using constrained and standardized building blocks. Secondary structure motives are such stable building blocks, more constrained than the protein sequence. In that sense secondary structure may be seen as an useful data compression optimized evolutionary to perform certain functions. Our classifier performs better with a more compact, specialized representation than with the full sequence information, which requires too many free parameters to estimate.In our work we focused on predicting binary protein interaction and the constructed residue-level predictor served only to generate data for level-II predictor. We optimised level-II prediction quality and demonstrated that optimising level-I only could lead to overfitting on level-II. In fact, for the threshold values greater than 0, our level-I predictor was not trained to identify single residue contacts anymore; it focused on discovering only strongly interacting fragments important for inter-protein interactions. To assess the importance of this difference in task formulation, we calculated ROC AUC score for level-I predictor not on our filtered segment pairs data but on real contact matrices of the test proteins. The predictor obtained 0.65 AUC—a score lower than on the filtered data and lower than level-II predictor score. This means that level-I predictor produces useful information for predicting protein–protein interactions but is not necessarily a good predictor of specific residue contacts.After examining predicted contact maps visually, we realised that they consist of characteristic horizontal and vertical lines. This means that certain sequence fragments are predicted to be very active and likely to interact while the others are classified as inactive. We considered the possibility that our predictor focused on predicting solvent exposure of the fragments. To verify this hypothesis we calculated relative solvent accessibility for each residue . We chosIt is possible that our level-I predictor reflects more specific protein properties such as the location of active sites. Improving our method would require an in-depth analysis of these matters. Since solvent accessibility is not correlated with the current predictions perhaps it is possible to include this information to further improve accuracy.We draw the reader’s attention to the fact that the performance of popular protein representation strategies evaluated on our data was generally much lower than results reported in the literature. One of the reasons may be a relatively small size of our dataset—it might not contain enough examples for a classifier to learn complex patterns. The other explanation is the way we constructed positives and negatives. In our case every protein occurred in the same number of positive and negative pairs. Moreover, we performed cross-validation with splits on the protein level, which means that no single protein occurred simultaneously in training and testing set. In such conditions any method which makes a good prediction of general proteins’ reactivity but does not consider their actual compatibility performs poorly. This observation is consistent with results obtained by Even though our evaluation procedure was carefully designed to reduce certain biases, we still do not expect it to reflect the reality of protein–protein interaction prediction perfectly. The most important issue is the proportion of positives to negatives in our datasets. On the protein level, we used balanced sets with the same number of positives and negatives. In reality the number of negatives, i.e., pairs of non-interacting proteins is much larger than the number of positives. d = 762. Let us examine the definitions of the specific performance metrics: Classifier performance is calculated from a confusion matrix consisting of the following terms: TP—true positives (positives classified as positives), TN—true negatives (negatives classified as negatives), FP—false positives (negatives classified as positives), FN—false negatives (positives classified as negatives). The correction involves multiplying terms based on negative examples in the data set—TN and FP—by constant in vitro experiments—choosing protein pairs for experimental verification in a more informed way would make the procedure more effective. Another possibility is using the predictor as a filter to improve quality of the interaction data collected from the existing databases. This is also an important issue because the quality of results from high-throughput experiments is still limited.In the light of these results, we believe that current methods for predicting protein–protein interactions from sequences, including ours, are not mature enough for large-scale application. The goal of reconstructing full protein interaction network is right now beyond reach. However, the developed predictors could still be applied in a variety of contexts. One of the possible applications is the initial screening of candidates for more costly In this work we presented a method for constructing a multi-level classifier for protein–protein interactions. We demonstrated that the information present at the lower level can be successfully propagated to the upper level to make reasonable predictions. No additional features other than protein secondary structure predicted from sequence were required.Our goal, predicting actual compatibility between two proteins regardless of their relative reactivity, forced us to collect high quality data and develop a rigorous evaluation procedure. We have taken into account properties of protein interaction network to construct balanced negatives. During the evaluation we carefully separated training and testing proteins to avoid information leak. We demonstrated that our method is working under such conditions better than popular sequence feature aggregation schemas.There is still much room for further improvements regarding classification accuracy. We plan to include additional features both at the residue level and at the protein level to see if our model can benefit from them. Another direction that we want to explore is expanding the model to include proteins from organism other than yeast, and evaluating it on bigger datasets.We hope that our work will inspire further discussion regarding evaluation strategies for protein interaction predictors. We believe that deeper understanding of those matters would allow the comparison of different methods in a more systematic manner which would be beneficial for the research done in this area.