Geometric deep learning has recently achieved great success in non-Euclidean domains, and learning on 3D structures of large biomolecules is emerging as a distinct research area. However, its efficacy is largely constrained due to the limited quantity of structural data. Meanwhile, protein language models trained on substantial 1D sequences have shown burgeoning capabilities with scale in a broad range of applications. Several preceding studies consider combining these different protein modalities to promote the representation power of geometric neural networks but fail to present a comprehensive understanding of their benefits. In this work, we integrate the knowledge learned by well-trained protein language models into several state-of-the-art geometric networks and evaluate a variety of protein representation learning benchmarks, including protein-protein interface prediction, model quality assessment, protein-protein rigid-body docking, and binding affinity prediction. Our findings show an overall improvement of 20% over baselines. Strong evidence indicates that the incorporation of protein language models’ knowledge enhances geometric networks’ capacity by a significant margin and can be generalized to complex tasks. Integrating knowledge from protein language models into geometric networks improves performance in tasks such as protein-protein interface prediction, model quality assessment, rigid-body docking, and binding affinity prediction. Consequently, tremendous efforts in structural biology have been devoted to deriving insights from their conformations5.Macromolecules  are essential to biophysical processes. While they can be represented using lower-dimensional representations such as linear sequences (1D) or chemical bond graphs (2D), a more intrinsic and informative form is the three-dimensional geometry9 have been proposed, and simultaneously achieve several crucial properties of Euclidean geometry such as E(3) or SE(3) equivariance and symmetry. Notably, they are essential constituents of geometric deep learning (GDL), an umbrella term that generalizes networks to Euclidean or non-Euclidean domains10.With the rapid advances of deep learning (DL) techniques, it has been an attractive challenge to represent and reason about macromolecules’ structures in the 3D space. In particular, different sorts of 3D information, including bond lengths and dihedral angles, play an essential role. In order to encode them, a number of 3D geometric graph neural networks (GGNNs) or CNNs13 and ProtTrans14. These protein language models can capture information about secondary and tertiary structures and can be generalized across a broad range of downstream applications. To be explicit, they have recently been demonstrated with strong capabilities in uncovering protein structures12, predicting the effect of sequence variation on function11, learning inverse folding15 and many other general purposes13.Meanwhile, the anticipated growth of sequencing promises unprecedented data on natural sequence diversity. The abundance of 1D amino acid sequences has spurred increasing interest in developing protein language models at the scale of evolution, such as the series of ESM17. This is nontrivial because compared to sequence learning, 3D structures are much harder to obtain and thus less prevalent. Consequently, learning about the structure of proteins leads to a reduced amount of training data. For example, the SAbDab database18 merely has 3K antibody-antigen structures without duplicate. The SCOPe database19 has 226K annotated structures, and the SIFTS database20 comprises around 220K annotated enzyme structures. These numbers are orders of magnitude lower than the data set sizes that can inspire major breakthroughs in the deep learning community. In contrast, while the Protein Data Bank (PDB)21 possesses approximately 182K macromolecule structures, databases like Pfam22 and UniParc23 contains more than 47M and 250M protein sequences respectively.With the fruitful progress in protein language models, more and more studies have considered enhancing GGNNs’ ability by leveraging the knowledge of those protein language models24. The unobserved variables that decide a protein’s fitness, including structure, function, and stability, leave a record in the distribution of observed natural sequences25. Those protein language models use self-supervision to unlock the information encoded in protein sequence variations, which is also beneficial for GGNNs. Accordingly, in this paper, we comprehensively investigate the promotion of GGNNs’ capability with the knowledge learned by protein language models  aims to select the best structural model of a protein from a large pool of candidate structures and is an essential step in structure prediction26. For a number of recently solved but unreleased structures, structure generation programs produce a large number of candidate structures. MQA approaches are evaluated by their capability of predicting the global distance test (GDT-TS score) of a candidate structure compared to the experimentally solved structure of that target. Its database is composed of all structural models submitted to the Critical Assessment of Structure Prediction (CASP)27 over the last 18 years. The data is split temporally by competition year. MQA is similar to the Protein Structure Ranking (PSR) task introduced by Townshend et al.2.Protein-protein Rigid-body Docking (PPRD) computationally predicts the 3D structure of a protein-protein complex from the individual unbound structures. It assumes that no conformation change within the proteins happens during binding. We leverage Docking Benchmark 5.5 (DB5.5)28 as the database. It is a gold standard dataset in terms of data quality and contains 253 structures.Protein-protein Interface (PPI) investigates whether two amino acids will contact when their respective proteins bind. It is an important problem in understanding how proteins interact with each other, e.g., antibody proteins recognize diseases by binding to antigens. We use the Database of Interacting Protein Structures (DIPS), a comprehensive dataset of protein complexes mined from the PDB29, and randomly select 15K samples for evaluation.Ligand Binding Affinity (LBA) is an essential task for drug discovery applications. It predicts the strength of a candidate drug molecule’s interaction with a target protein. Specifically, we aim to forecast K is the binding affinity in Molar units. We use the PDBbind database31, a curated database containing protein-ligand complexes from the PDB and their corresponding binding strengths. The protein-ligand complexes are split such that no protein in the test dataset has more than 30% or 60% sequence identity with any protein in the training dataset.32 and PyG33 on four standard protein benchmarks. For MQA, PPI, and LBA, we use GVP-GNN, EGNN, and Molformer as backbones. For PPRD, we utilize a deep learning model, EquiDock34, as the backbone. It approximates the binding pockets and obtains the docking poses using keypoint matching and alignment. For more experimental details, please refer to Supplementary Note We evaluate our proposed framework on the instances of several state-of-the-art geometric networks, using PytorchRS), Pearson’s correlation (RP), and Kendall rank correlation (KR) in Table RS, of 34.66% and 58.75% in global and mean RP, and of 43.21% and 63.20% in global and mean KR respectively. With the aid of language models, GVP-GNN achieves the optimal global RS, global RP, and KR of 84.92%, 85.44%, and 67.98% separately.For MQA, we document First Rank Loss, Spearman correlation  and the state-of-the-art pretraining results set by Jing et al.43 if it is enhanced by the protein language model.Apart from that, we provide a full comparison with all existing approaches in Table For PPRD, we report three items as measurements: the complex root mean squared deviation (RMSD), the ligand RMSD, and the interface RMSD in Table For PPI, we record AUROC as the metric in Fig. RS, RP, and KR in Table RP for the 30% and 60% identity, an average increment of 66.60% and 8.90% in RS for the 30% and 60% identity, and an average increment of 68.52% and 6.70% in KR for the 30% and 60% identity. It can be seen that the improvements in the 30% sequence identity is higher than that in the less restrictive 60% sequence identity. This confirms that protein language models benefit GGNNs more when the unseen samples belong to different protein domains. Moreover, contrasting PPRD or PPI, LBA studies how proteins interact with small molecules. Our outcome demonstrates that rich protein representations encoded by protein language models can also contribute to the analysis of protein’s reaction to other non-protein drug-like molecules. The result of a different data split has been placed in Supplementary Table For LBA, we compare RMSD, 44, Cormorant45, LSTM46, TAPE47, ProtTrans14, 3DCNN2, GNN2, MaSIF48, DGAT49, DGIN49, DGAT-GCN49, HoloProt50, and GBPNet42 as the baseline. It is clear that even if EGNN is a median-level architecture, it can achieve the best RMSD and the best Pearson’s correlation when enhanced by protein language models, beating a group of strong baselines including HoloProt50 and GBPNet42.In addition, we compare thoroughly with existing approaches for LBA in Table 12. Here we conduct an ablation study to investigate the effect of protein language models’ sizes on GGNNs. Specifically, we explore different ESM-2 with the parameter numbers of 8M, 35M, 150M, 650M, and 3B and plot results in Fig. It has been observed that as the size of the language model increases, there are consistent improvements in tasks like structure predictionDespite our successful confirmation that PLMs can promote geometric deep learning, there are several limitations and extensions of our framework left open for future investigation. For instance, our 3D protein graphs are residue-level. We believe atom-level protein graphs also benefit from our approach, but its increase in performance needs further exploration.In this study, we investigate a problem that has been long ignored by existing geometric deep learning methods for proteins. That is, how to employ the abundant protein sequence data for 3D geometric representation learning. To answer this question, we propose to leverage the knowledge learned by existing advanced pre-trained protein language models and use their amino acid representations as the initial features. We conduct a variety of experiments such as protein-protein docking and model quality assessment to demonstrate the efficacy of our approach. Our work provides a simple but effective mechanism to bridge the gap between 1D sequential models and 3D geometric neural networks, and hope to throw light on how to combine information encoded in different protein modalities.51 and RosettaFold52 has made a huge step in alleviating the limitation brought by the number of available experimentally determined protein structures, neither AF nor its successors such as Alphafold-Multimer53, IgFold54, and HelixFold55 are a panacea. Their predicted structures can be severely inaccurate when the protein is orphan and lacks multiple sequence alignment (MSA) as the template. Consequently, it is hard to conclude that protein sequences can be perfectly transformed to the structure modality by current tools and be used as extra training resources for GGNNs.It is commonly acknowledged that protein structures maintain much more information than their corresponding amino acid sequences. And for decades long, it has been an open challenge for computational biologists to predict protein structure from its amino acid sequence. Though the advancement of Alphafold (AF)56, angles40, torsion, and terms of other orders57. The position index hidden in protein sequences, however, is usually neglected when constructing 3D graphs for GGNNs. Therefore, in this section, we design a toy trial to examine whether GGNNs can succeed in recovering this kind of positional information.Moreover, we argue that even if conformation is a higher-dimensional representation, the prevailing learning paradigm may forbid GGNNs from capturing the knowledge that is uniquely preserved in protein sequences. Recall that GGNNs are mainly diverse in their patterns to employ 3D geometries, the input features include distanceN nodes and M edges respectively. Nodes have their 3D coordinates ψh-dimension roto-translational invariant features r-ball graphs, fully-connected (FC) graphs, and K-nearest neighbors (KNN) graphs. In our setting, nodes are linked to K = 10 nearest neighbors for KNN graphs, and edges include all atom pairs within a distance cutoff of 8Å for r-ball graphs.Here the structure of a protein can be represented as an atom-level or residue-level graph Since most prior studies choose to establish 3D protein graphs based on purely geometric information and ignore their sequential identities, it provokes the following position identity question:Can existing GGNNs identify the sequential position order only from geometric structures of proteins?N, the residue number of each protein. This task adopts accuracy as the metric and expects models to discriminate the absolute position of the amino acid within the whole protein sequence. We compute the distribution of the protein sequence lengths in Supplementary Fig. To answer this question, we formulate two categories of toy tasks see Fig. . The firIn addition to that, we propose the second task named relative position estimation (RPE) to focus on the relative position of each residue. Models are required to predict the minimum distance of residue to the two sides of the given protein and the root mean squared error (RMSE) is used as the metric. This task aims to examine the capability of GGNNs to distinguish which segment the amino acid belongs to .43 extends standard dense layers to operate on collections of Euclidean vectors, performing both geometric and relational reasoning on efficient representations of macromolecules. EGNN58 is a translation, rotation, reflection, and permutation equivariant GNN without expensive spherical harmonics. Molformer9 employs the self-attention mechanism for 3D point clouds while guarantees SE(3)-equivariance.We adopt three technically distinct and broadly accepted architectures of GGNNs for empirical verification. To be specific, GVP-GNNWe exploit a small non-redundant subset of high-resolution structures from the PDB. To be specific, we use only X-ray structures with resolution < 3.0Å, and enforce a 60% sequence identity threshold. This results in a total of 2643, 330, and 330 PDB structures for the train, validation, and test sets, respectively. Experimental details, the summary of the database, and the description of these GGNNs are elaborated in Supplementary Notes ↑/↓ if higher/lower is better, respectively. It can be found that all GGNNs fail to recognize either the absolute or the relative positional information encoded in the protein sequences with an accuracy lower than 1% and an extremely high RMSE.Table 59, social networks60, knowledge graphs61, molecules do not have explicitly defined edges or adjacency. On the one hand, r-ball graphs utilize a cut-off distance, which is usually set as a hyperparameter, to determine the particle connections. But it is hard to guarantee a cut-off to properly include all crucial node interactions for complicated and large molecules. On the other hand, FC graphs that consider all pairwise distances will cause severe redundancies, dramatically increasing the computational complexity especially when proteins consist of thousands of residues. Besides, GGNNs also easily get confused by excessive noise, leading to unsatisfactory performance. As a remedy, KNN becomes a more popular choice to establish graph connectivity for proteins63. However, all of them take no account of the sequential information and require GGNNs to learn this original sequential order during training.This phenomenon stems from the conventional ways to build graph connectivity, which usually excludes sequential information. To be specific, unlike common applications of GNNs such as citation networks64 when they are located at different segments of the protein. Thus, GGNNs may be incapable of differentiating two residues with the same 1-hop local structures. This restriction has already been distinguished by several works65, but none of them make a strict and thorough investigation. Admittedly, sequential order may only be necessary for certain tasks. But this toy experiment strongly indicates that the knowledge monopolized by amino acid sequences can be lost if GGNNs only learn from protein structures.The lack of sequential information can yield several problems. To begin with, residues are unaware of their relative positions in the proteins. For instance, two residues can be close in the 3D space but distant in the sequence, which can mislead models to find the correct backbone chain. Secondly, according to the characteristics of the MP mechanism, two residues in a protein with the same neighborhood are expected to share similar representations. Nevertheless, the role of those two residues can be significantly separate66. On the contrary, comparing the number of protein sequences in the UniProt database67 to the number of known structures in the PDB, there are over 1700 times more sequences than structures. More importantly, the availability of new protein sequence data continues to far outpace the availability of experimental protein structure data, only increasing the need for accurate protein modeling tools.As discussed before, learning about 3D structures cannot directly benefit from large amounts of sequential data. Subsequently, the model sizes of GGNNs are limited, or instead, overfitting may occur12 is adopted in our case, and extract the per-residue representations, denoted as ψPLM = 1280. Then h. For residue-level graphs, h as the input node features.Therefore, we introduce a straightforward approach to assist GGNNs with pretrained protein language models. To this end, we feed amino acid sequences into those protein language models, where ESM-268. They, therefore, do not perfectly match the corresponding sequences . There are two choices to address this mismatch. On the one hand, we can simply use the fragmentary sequence as the substitute for the integral amino acid sequence and forward it into the protein language models. On the other hand, we can leverage a dynamic programming algorithm provided by Biopython69 to implement pairwise sequence alignment and abandon residues that do not exist in the PDB structures. It is empirically discovered that no big difference exists between them, so we adopt the former processing mechanism for simplicity.Notably, incompatibility exists between the experimental structure and its original amino acid sequence. That is, structures stored in the PDB files are usually incomplete and some strings of residues are missing due to inevitable realistic issuesFurther information on research design is available in the Supplemental InformationDescription of Additional Supplementary FilesSupplementary DataReporting Summary


In recent years, high-throughput sequencing technologies have made large-scale protein sequences accessible. However, their functional annotations usually rely on low-throughput and pricey experimental studies. Computational prediction models offer a promising alternative to accelerate this process. Graph neural networks have shown significant progress in protein research, but capturing long-distance structural correlations and identifying key residues in protein graphs remains challenging.In the present study, we propose a novel deep learning model named Hierarchical graph transformEr with contrAstive Learning (HEAL) for protein function prediction. The core feature of HEAL is its ability to capture structural semantics using a hierarchical graph Transformer, which introduces a range of super-nodes mimicking functional motifs to interact with nodes in the protein graph. These semantic-aware super-node embeddings are then aggregated with varying emphasis to produce a graph representation. To optimize the network, we utilized graph contrastive learning as a regularization technique to maximize the similarity between different views of the graph representation. Evaluation of the PDBch test set shows that HEAL-PDB, trained on fewer data, achieves comparable performance to the recent state-of-the-art methods, such as DeepFRI. Moreover, HEAL, with the added benefit of unresolved protein structures predicted by AlphaFold2, outperforms DeepFRI by a significant margin on Fmax, AUPR, and Smin metrics on PDBch test set. Additionally, when there are no experimentally resolved structures available for the proteins of interest, HEAL can still achieve better performance on AFch test set than DeepFRI and DeepGOPlus by taking advantage of AlphaFold2 predicted structures. Finally, HEAL is capable of finding functional sites through class activation mapping.https://github.com/ZhonghuiGu/HEAL.Implementations of our HEAL can be found at  Recent development in high-throughput sequencing has resulted in a great increase in the number of protein sequences in benchmark databases such as . HoweverRecent advances in deep learning have led to the development of various effective techniques for protein function prediction. Sequence-based approaches  followinIn spite of recent advances in protein function prediction using GNN-based approaches, the following limitations remain to be solved: (i) Long-distance structural correlations are difficult to be included. Due to the major oversmoothing problem, existing methods usually adopt shallow GNNs. The restriction on network depths makes it hard to explore long-distance spatial patterns in 3D protein structures. (ii) It is hard to capture residues that are crucial for protein function. Protein representations are often generated using a simple graph pooling procedure that takes the average or maximum of all residue representations, treating all residues equally despite the fact that protein functionality often depends on specific residues. Therefore, an effective procedure needs to be envisaged to generate protein representations in an adaptive manner.To tackle the aforementioned limitations, we propose a novel method, Hierarchical graph transformEr with contrAstive Learning (HEAL) for protein function prediction in this study. Our approach involves constructing a graph input based on both sequential features and the contact map, followed by the collection of short-distance information using message passing neural networks. We then introduce a hierarchical graph Transformer to explore long-distance correlations and aggregate the node representation in a self-adaptive manner. To learn topological semantics, we first introduce a set of super-nodes that interact with nodes in the protein graph. We then use the self-attention mechanism to aggregate the semantic super-node representations into a graph representation. Additionally, we incorporate graph contrastive learning  One-hot residue encoder: each sequence is encoded by one of amino acid symbols. (ii) ESM-1b protein language model  loss objective for downstream multilabel classification as follows:where mly and l-th function of the m-th sample, respectively. The final loss function is derived by combining both supervised loss and regularization loss as:Overall, our models comprise of four graph convolutional layers, one hierarchical graph Transformer layer and one MLP module. We train the proposed HEAL using the Adam optimizer  with a lhttps://github.com/flatironinstitute/DeepFRI. The dataset comprises of 36 641 protein structures from PDB database and 244 775 protein structures from SWISS-MODEL repository .We further tested whether recent development in protein structure prediction improves data augmentation. We selected 44 137 proteins with low-frequency GO terms (proteins with IC>10 in the PDBch training set) and retrieved their structures predicted by AlphaFold2 (AF2) from AlphaFold Protein Structure Database  to clustBlast  is trained solely on the PDBch training set. We assess the performance of the models on three gene ontology domains  separately. The results, as presented in When the SMch dataset was included for training, the resulting model HEAL-SW performs better than both HEAL-PDB and DeepFRI. As recently developed deep learning-based methods for protein structure prediction have become more reliable, we improved our model further by incorporating the AFch training set and the resulting model is referred as HEAL. HEAL achieves Fmax of 0.747, 0.595, 0.687, Smin of 0.342, 0.509, 0.458, and AUPR of 0.691, 0.337, 0.467 on MF, BP, CC tasks, which surpasses the state-of-the-art open-source GCN method DeepFRI by a significant margin, and even goes beyond HEAL-SW despite using much fewer proteins for training. These results indicate that HEAL leads the way in protein function prediction.To investigate how different components of HEAL contribute to its performance, we conduct the ablation experiments on the PDBch test set. We introduce three variants as below: (i) HEAL w/o CL: it removes the contrastive learning objective. (ii) HEAL w MP: it utilizes the max pooling to replace HGT (iii) HEAL w/o EE: it removes ESM-1b embeddings from the node attributes. The results are summarized in In order to evaluate the generalizability of HEAL, we evaluate its performance on PDBch test sequences with varying homology to the combined PDBch and SMch training sets, which are the training set for DeepFRI and DeepGO models. The five sequence identity thresholds are 30%, 40%, 50%, 70%, and 95%. Given that both HEAL and HEAL-PDB were trained on subsets of the PDBch and SMch training sets, this comparison is valid. Notably, deep learning methods significantly outperform sequence alignment-based methods. Therefore, DeepFRI, DeepGO, and HEAL-PDB are also included in the comparison. Fmax, AUPR, and Smin are computed using 10 bootstrap iterations across all test proteins. As is depicted in We spilt proteins in the PDBch test set into three groups based on the IC of each term belonging to MF, BP, and CC tasks . Among all the three ranges, as depicted from A more realistic usage scenario for our methods is to predict biological functions for proteins with neither experimentally resolved structure nor annotated similar sequences. In this scenario, it is more appropriate to compare our methods with robust methods that rely on both sequence and homology information. To accomplish this, we retrained DeepGOPlus  on the sAs is shown from L denotes the length of the protein and D denotes the hidden dimension. Then we take the derivative of the protein function ly with respect to F as the gradient weight To localize the prediction result into each residue, we apply the gradient-weighted Class Activation Map . As is shown in For BP-GO terms, we offer two examples. The first example is 4TMK, a thymidylate kinase involved in the nucleoside triphosphate metabolic process (GO:0009141). As shown in We extracted the binding sites of the four proteins from the BioLiP database , and useAs the HEAL models were trained on both experimentally solved and AF2 predicted structures, it can also predict the key binding sites from AF2 predicted protein structures. A0A3P7DWR6 is another DNA repair protein RAD51 homolog, and its structures are unavailable from PDB. Our model can recall functions of A0A3P7DWR6 confidently with its AF2 predicted structures as input. The conservativeness of DNA binding sites between A0A3P7DWR6 and 5H1C suggests that they share the same binding mode , and theIn this study, we have proposed the contrastive learning assisting GCN model HEAL, and introduced the hierarchical graph Transformer to conduct node aggregation and graph pooling. By integrating protein structure and sequence language embeddings, HEAL provides a powerful tool for protein function prediction, which significantly outperformed the state-of-art model DeepFRI. In addition, HEAL demonstrates better generalizability to sequences that are dissimilar from those in the training set, as well as superior specificity-resistance for infrequent function prediction. Owing to the close relationship between structures and functions, addressing the issue from a structural perspective circumvents the complex challenge of discerning long-term correlations in sequence data. Furthermore, protein language models inherently acquire evolutionary information through self-supervised learning. This intrinsic characteristic significantly enhances the capacity of neural network models to effectively capture and comprehend the evolutionary patterns underlying functional motifs (On the AFch test set, which includes AF2 predicted structures with low sequence similarity to the training set and no experimentally resolved structures, HEAL exhibits remarkable robustness and outperforms other state-of-the-art methods. This result suggests that HEAL has great potential for application in real-world scenarios.By introducing the grad-CAM method, we find that our model can identify functional residues that correspond well with experimentally confirmed residues. When the AF2 predicted protein structures are utilized, our model still exhibits good interpretability.To infer protein functions using HEAL, either experimentally solved structures or AF2 predicted structures are required as input, which adds an additional step compared to sequence-based methods. However, the recent breakthrough in protein structure prediction using large protein language models (btad410_Supplementary_DataClick here for additional data file.


In this paper we discuss the advantages and problems of two alternatives for ab initio protein structure prediction. On one hand, recent approaches based on deep learning, which have significantly improved prediction results for a wide variety of proteins, are discussed. On the other hand, methods based on protein conformational energy minimization and with different search strategies are analyzed. In this latter case, our methods based on a memetic combination between differential evolution and the fragment replacement technique are included, incorporating also the possibility of niching in the evolutionary search. Different proteins have been used to analyze the pros and cons in both approaches, proposing possibilities of integration of both alternatives. For this purpose, traditional laboratory methods, such as X-ray crystallography, Nuclear Magnetic Resonance (NMR) and electron cryo-microscopy, are expensive and time-consuming. As an alternative, computational Protein Structure Prediction (PSP) methods attempt to close the gap between the number of proteins with known sequence (on the order of millions) and the number of resolved proteins with known structure . For example, PSP methods of homology modeling are based on finding a PDB-resolved protein with a homologous amino acid sequence, since with high homology the structures are the same. PSP threading is another possibility if there are no resolved proteins with a homologous sequence. In the latter case, for a target protein and a library of possible templates (folds), threading methods search for the fold in which the target sequence best fits.In the most difficult and challenging alternative of PSP, called ab initio, only the primary sequence information of the protein (its amino acid sequence) is used. This ab initio prediction is based on Anfinsen’s dogma Anfinsen , which shttp://www.rosettacommons.org) is one of the leading methods for PSP with these energy-based approaches. Rosetta ab initio protocol uses small protein fragments (from resolved proteins) and the classical Metropolis criterion  . In addition, in a partnership between DeepMind and the European Molecular Biology Laboratory (EMBL), AlphaFold2 was used to publicly provide structure predictions .Nevertheless, a more recent approach in PSP is to predict the contact map or the interdistance map between amino acids, which is a simpler representation of the protein three-dimensional structure. This prediction typically uses the information from Multiple Sequence Alignment (MSA) of the target protein sequence as input to deep learning schemes, such as the initial approaches of RoseTTAFold . Moreover, RoseTTAFold allows the prediction of accurate protein-protein complex models only from sequence information, as a new avenue to short-circuit the traditional docking approach.Likewise, AlphaFold2 , extending the previous work presented at conference IWINAC 2022 , whereas the side chains are modeled with a pseudo-atom located at their center of mass Fig. .Fig. 1Rohttp://www.rosettacommons.org), with the low-resolution protein representation, employs a search technique in which a Monte Carlo procedure decides whether the dihedral angles of small protein fragments can replace the original ones . The Rosetta score function named score3 integrates all of the energy components.In Rosetta, the energy of a protein conformation is defined as a weighted linear combination of different energy terms that model the molecular forces acting between the amino acid atoms. For example, steric overlap between backbone and side-chain atoms is penalized, while other Rosetta’s energy terms correspond to van der Waals interactions, electrostatics effects and solvation, hydrogen bonding, repulsion and scores related to protein secondary structure . The detailed definition of the energy terms can be found, for example, in Rohl et al. , and theIn the search for protein conformations with minimum energy, the stochastic Metropolis Monte Carlo procedure is run thousands of times. For this, the Rosetta ab initio protocol is divided into four stages, which use different score functions (progressively incorporating new energy terms) and number of fragment insertion attempts. Detailed information about these four stages can be found, for example in Rohl et al.  and VareHybridDE memetic approach). The genetic population encodes possible protein conformations using the Rosetta’s coarse-grained representation . The hybrid combination integrates the advantage of the global search of the evolutionary algorithm with the local search provided by the replacement of fragments. The fragment insertion technique is used in DE to refine the population solutions and to refine the DE candidate or trial solutions. The memetic search follows a three-stage evolutionary process, as the fitness of the encoded conformations corresponds to different Rosetta score functions in each stage, while the first stage of Rosetta is used to define the initial population . The memetic version is detailed in Varela and Santos   and SARS-CoV-2 protein orf8 as target. In each evolutionary phase, the fitness is associated to the energy score that Rosetta uses in its corresponding phase and, consequently, it has different ranges in each evolutionary stage. Note that, using Rosetta nomenclature in Stage 3, score5 was the final energy (with energy terms focused on compactness and secondary structure), although score3 in the final Stage 4 is the energy score incorporating all energy components. This evolutionary process with different stages, integrating in each of them more detailed energy functions, allows a progressive structural refinement.Figure HybridDE version, thus forcing the search algorithm to obtain optimized conformations in different niches  . This occurs when the conformation with the minimum energy does not correspond to the conformation closest to the real native structure. Consequently, one strategy is to obtain a set of optimized conformations  with structural diversity. For this purpose, we introduced niching methods such as crowding, fitness sharing and speciation into the d Santos , 2022.FiAlphaFold2Jumper et al. .The recent deep learning-based methods of r et al.  and Rosek et al.  for PSP i in a homologous sequence (with respect to the target sequence), and a correlated mutation appears at position j in the same homologous sequence, then it is likely that residues i and j are in contact in the tertiary structure, since the correlated mutation tends to maintain the protein structure unchanged.The idea behind the use of MSA as input information is that correlated mutations between residues indicate their spatial physical interaction. That is, if an amino acid mutates at position AlphaFold2’s architecture is detailed in Jumper et al. . Both representations exchange information, as updates in the MSA embedding provide new information to change the structural hypothesis in the pair representation, and vice versa.One of the key modules of the internal architecture of AlphaFold2 to progressively refine the final and predicted model of the protein.Evoformer is followed by a NN module or Structure Module that maps the embedding or abstract representation of the Evoformer stack to concrete 3D coordinates of all atoms (as well as the per-residue confidence commented below). In this module, NN attention mechanisms (with invariance to rotations and translations of the protein conformation in space) are used to progressively refine the structure (which includes the side-chain atoms). Finally, the predicted structure information is returned to the Evoformer blocks. Consequently, these two steps  are repeated several times in AlphaFold2 provides two confidence measures of the predicted structures. The first is the predicted local-Distance Difference Test (plDDT), a per-residue measure of local confidence . The local-Distance Difference Test (lDDT) is a superposition-free score that evaluates local distance differences of all atoms in a model  reports the expected position error at residue x, when the predicted and real structures are aligned on residue y. Consequently, it provides a level of confidence about the relative positions of the amino acids (and different domains) of the protein.RoseTTAFold ). RoseTTAFold also provides an estimate of the per-residue precision, based on the estimated k et al. , with rek et al. .AlphaFold2 (https://colab.research.google.com/github/sokrypton/ColabFold/blob/main/AlphaFold2.ipynb) and RoseTTAFold (https://colab.research.google.com/github/sokrypton/ColabFold/blob/main/RoseTTAFold.ipynb) ColabFold servers were used in the predictions. Coupled with Google Colaboratory, ColabFold .The PSP approaches discussed in the Methods section are used with different proteins. In the case of Rosetta ab initio PSP protocol, taking into account its stochasticity, the protocol is run 1,000 times to generate 1,000 decoys (candidate conformations). Rosetta parameter l et al.  and Varel et al. ), as recHybridDE and CrowdingDE), the same setup used in Varela and Santos . These DE-based energy minimization approaches will use the same number of fitness/energy evaluations as Rosetta ab initio for generating the 1,000 decoys. It should be noted that energy evaluations are synonymous with fragment insertion attempts, since an insertion attempt involves the energy evaluation of the resulting conformation. The fair comparison is obtained since, in HybridDE and CrowdingDE, parameter increase_cycles is set to 0.1 (100 times less than in the case of Rosetta ab initio protocol), because the evolutionary approaches refine 1,000 conformational solutions with the same Rosetta approach based on fragment insertions, but over 100 generations.With the same purpose of generating 1,000 decoy conformations, AlphaFold2 and RoseTTAFold), the default configuration provided by the ColabFold servers was used, that is, using the MSA information as input, while the servers provide five candidate models, those with the highest prediction confidence.For the deep learning-based approaches  were selected in order to test the behavior of deep learning-based approaches.1r69 . Figure score3, which includes all individual energy terms).A first example is selected in which, even with the scarce information provided by MSA, deep learning methods show excellent prediction results. The example corresponds with protein It should be noted that the RMSD distance can only be calculated with proteins whose native structure is known a priori, such as those considered here. That is, with these proteins acting as a benchmark since their native structure is known, we can analyze the quality of the predictions in terms of their structural distance with respect to the native reference structure.AlphaFold2 and RoseTTAFold predicted solutions are very accurate, with very low values of RMSD with respect to the native structure deposited in PDB. On the contrary, the energy-based approaches  present solutions with a large variety of RMSD values, with few decoys with values lower than 2 Ȧ (angstroms) and several solutions with large distances (in RMSD terms) to the native structure.Figure AlphaFold2 and RoseTTAFold solutions, some atoms have collisions in the side chains, a problem that can be improved by further refinement of the structure. Considering the energy minimization-based approaches, HybridDE and CrowdingDE obtain solutions with better energy with respect to Rosetta ab initio, showing the better ability of the evolutionary approaches to sample the conformational space under the same number of energy evaluations. However, HybridDE obtains the best solution in energy terms, but it corresponds to a solution that is farther away from the native structure with respect to other solutions provided by HybridDE and Rosetta. This shows the inaccuracies of the Rosetta energy model, since the solutions with the best energy do not have to correspond to those closest to the native structure, defining a clear deceptive energy landscape for the search algorithms, in which the best approaches in energy terms are those that provide the worst solutions in RMSD terms.Nevertheless, in terms of energy of the predicted solutions, the comparison is totally different, since the deep learning-based approaches present solutions with higher energies. This is due to the fact that, in the CrowdingDE), allows us to obtain a wider distribution of optimized solutions, as clearly shown by the distributions in the violins in Fig. HybridDE. Nevertheless, the goal of CrowdingDE is precisely this, to present a set of optimized and structurally different proteins, as shown by the distribution. However, even with the inclusion of crowding, in this protein, CrowdingDE does not present solutions closer to the native structure with respect to Rosetta ab initio.To address the problem of deceptiveness in the energy landscape, the inclusion of crowding in the memetic evolutionary algorithm  of the 5 returned solutions . The confidence results of the AlphaFold2 solutions . Typical computing times are 45 min for each of the parallelized 10 independent runs of HybridDE and CrowdingDE (protein 1r69 as target). Rosetta Ab initio requires an average of 24.64 h to generate 1,000 solutions. The experiments were run in the Supercomputing Center of Galicia (www.cesga.es), with Intel Xeon E5-2680 v3 processors at 2.50GHz and 1GB of RAM. The computing time for the deep learning approaches is variable, since these were run in the ColabFold servers . It can vary from 4 min to 30 min .Regarding computing times, it should be noted that Rosetta ab initio is not parallelized while 1ha8 . As shown in Fig. 1ha8, now the energy minimization approaches return solutions closer to the native structure with respect to the deep learning approaches. In the case of the former approaches, HybridDE and CrowdingDE obtain solutions with better energy with respect to Rosetta ab initio, again showing the enhanced ability of evolutionary approaches to sample the conformational space. Once again, HybridDE obtains the best optimized solution in energy terms, a solution that is farther away from the native structure with respect to many other solutions, which again shows the deceptiveness of the Rosetta energy landscape also with this protein. As with the previous example, the inclusion of crowding (CrowdingDE), allows us to obtain a wider distribution of optimized solutions. In fact, now CrowdingDE presents the closest solution to the native structure.The second example corresponds with protein AlphaFold2 and RoseTTAFold present worse solutions in terms of RMSD distance from the native structure. As shown in Fig. 1ha8, again the MSA does not present high sequence identity between the target protein and the homologous ones and, in this case, it does not include such a large number of homologous sequences as in the previous protein. This information is not sufficient for fairly high confidence in the predictions. This is shown by the poor confidence in the plDDT per residue. Similarly, the PAE graphs for the five best-rated prediction models show the low prediction confidence. The analysis with the RoseTTAFold solutions is similar, as Fig. 2ksw . Figure HybridDE and CrowdingDE provide better average energy of the optimized solutions, although Rosetta ab initio also discovers solutions with low energy. As in the previous example, the AlphaFold2 and RoseTTAFold solutions present high RMSD values from the native structure and these are worse with respect to many solutions of the energy minimization approaches. The reason is that, in this case, the MSA map is simple, since there are no homologous sequences found in the genetic databases (except for the sequence itself which was found 3 times). That is, it is an example with no information of proteins with similar sequence. Consequently, the AlphaFold2 models present very low confidence at all positions in the protein chain, as can be seen in Fig. The third example corresponds to protein RoseTTAFold solution is also poor, with estimated AlphaFold2 solutions and the RoseTTAFold solution ). This protein has no homologous proteins in the PDB database [20]. Even with the search for homologous sequences in genetic databases, the MSA coverage is poor  with l et al. .However, deep learning-based methods cannot provide different predicted structures for proteins mutated in a few amino acids, as discussed in Buel and Walters ; Callawa


Many proteins form multimers for their activity, and over 50% are estimated to naturally form homomultimers. Experimental quaternary state determination can be challenging and require extensive work. To complement these efforts, a number of computational tools have been developed for quaternary state prediction, often utilizing experimentally validated structural information. Recently, dramatic advances have been made in the field of deep learning for predicting protein structure and other characteristics. Protein language models, such as ESM-2, that apply computational natural-language models to proteins successfully capture secondary structure, protein cell localization and other characteristics, from a single sequence. Here we hypothesize that information about the protein quaternary state may be contained within protein sequences as well, allowing us to benefit from these novel approaches in the context of quaternary state prediction.Determining a protein’s quaternary state, We generated ESM-2 embeddings for a large dataset of proteins with quaternary state labels from the curated QSbio dataset. We trained a model for quaternary state classification and assessed it on a non-overlapping set of distinct folds (ECOD family level). Our model, named QUEEN (QUaternary state prediction using dEEp learNing), performs worse than approaches that include information from solved crystal structures. However, it successfully learned to distinguish multimers from monomers, and predicts the specific quaternary state with moderate success, better than simple sequence similarity-based annotation transfer. Our results demonstrate that complex, quaternary state related information is included in such embeddings.https://colab.research.google.com/github/Furman-Lab/QUEEN/blob/main/QUEEN_prediction_notebook.ipynb.QUEEN is the first to investigate the power of embeddings for the prediction of the quaternary state of proteins. As such, it lays out strengths as well as limitations of a sequence-based protein language model approach, compared to structure-based approaches. Since it does not require any structural information and is fast, we anticipate that it will be of wide use both for in-depth investigation of specific systems, as well as for studies of large sets of protein sequences. A simple colab implementation is available at: The online version contains supplementary material available at 10.1186/s12859-023-05549-w. Many proteins form oligomers to carry out their molecular assignments activation = ['adaptive']learning_rate = learning_rate_init = solver = Max_iter = 0.001, 0.01, 0.1, 1]tol = hidden_layer_sizes = batch_size = [10, 50, 100, 150, 200, scipy.stats.ranksums, with alternative = 'greater', the correct predictions passed as x and the wrong predictions passed as y.The statistical analysis was carried out using a one-sided (“greater”) Wilcoxon test, implemented through Python3.7.All the scripts were written using Torch. The ESM-2 model used is EsmForSequenceClassification, with the specific version of esm2_t33_650M_UR50D [Embedding the sequences was carried out using _UR50D . We initprot_bert_bfd [Embedding with Protbert was done using BertModel and the specific version bert_bfd .MMseqs version ad5837b3444728411e6c90f8c6ba9370f665c443 was used, installed locally.UMAP version 0.5 was used.pandas version 1.3.5 [For analyses and data handling we used on 1.3.5 .matplotlib and seaborn, versions 3.5.3 and 0.12.0 [Graphs and plots were generated with d 0.12.0 , 40.networkx version 3.0.The network plot was generated using PyMol version 2.2.0 Open-Source.Protein structure visualization was done using Additional file 1: Supplementary Table SI:  Embeddings.Additional file 2: Supplementary Table SII:  Model Results. Additional file 3: Supplementary Material. Contains Supplementary Figures S1-S6 and Supplementary Tables SIII-SV.


Inhibition of PPIsby drug-like compounds is a common basis for therapeutic approaches.In many cases the flat interface of PP complexes prevents discoveryof specific compound binding to cavities on one partner and PPI inhibition.However, frequently new pockets are formed at the PP interface thatallow accommodation of stabilizers which is often as desirable asinhibition but a much less explored alternative strategy. Herein,we employ molecular dynamics simulations and pocket detection to investigate18 known stabilizers and associated PP complexes. For most cases,we find that a dual-binding mechanism, a similar stabilizer interactionstrength with each protein partner, is an important prerequisite foreffective stabilization. A few stabilizers follow an allosteric mechanismby stabilizing the protein bound structure and/or increase the PPIindirectly. On 226 protein–protein complexes, we find in >75%of the cases interface cavities suitable for binding of drug-likecompounds. We propose a computational compound identification workflowthat exploits new PP interface cavities and optimizes the dual-bindingmechanism and apply it to 5 PP complexes. Our study demonstrates agreat potential for  Simulation studies reveal potential drug binding cavitiesat several protein−protein interfaces. Principles for identifyingspecific stabilizers are elucidated, and a design work flow is developed. PPIscan be inhibited by allosteric effects (involving binding of compoundsnot necessarily at the interface) or by targeting the protein–proteininterface directly such that protein partners can not form a complex.However, the design of PPI inhibitors suffers from the general featureof a typically flat and large buried interface area  and also often hydrophobic13 surface, which is frequently considered as “undruggable”for small molecules.15 Larger peptide-based inhibitors16 can be adopted as a promising strategy withlow toxicity, and indeed several peptides have been approved as clinicaldrugs.17 In particular, cyclic peptideswith predefined conformation and good cell permeability have receivedincreasing attention in PPI inhibitor design,20 and using in silico “hot-spots” matching with cyclicpeptides has been proposed for rapid rational design.21Complex formation of proteins plays afundamental role in the majorityof biological processes such as cell fate,11 The former mechanism describes the allosteric regulation of oneprotein partner upon stabilizer binding, which increases the bindingaffinity to another binding partner, while the latter mechanism describesthe direct binding of the stabilizer at the PPI interface with a potentiallydruggable cavity for small molecules.23 With severaladvantages compared to PPI inhibitors, stabilization of PPIs has becomean emerging field in recent years. Zarzycka et al.24 and Andrei et al.10 surveyedthe available protein–protein-stabilizer complexes and discusseddifferent groups of stabilizers according to the stabilized PPI typeand the origin of the stabilizers, respectively.For many diseases, promotion or stabilization canbe more helpfulthan the inhibition of PPIs. Such stabilization can be achieved bysmall organic molecules through an allosteric (not at the interface)or a direct interface binding mechanism.25 the tumor suppressorprotein p53,26 oncogenic transcriptionfactors TAZ,26 and estrogen receptor-derivedpeptide.27 Sijbesma et al. performed structure-basedvirtual screening over nearly 6 million compounds from the Molportdatabase and tested 13 compounds in vitro, amongwhich 2 are validated as potent stabilizers.28 Tang et al. combined molecular dynamics (MD) and a molecular dockingtechnique to investigate 22 14-3-3 protein-peptide-stabilizer complexesand concluded that a simulation time of 20–50 ns combined withmolecular mechanics Poisson–Boltzmann surface area (MM/PBSA)and generalized-Born surface area (MM/GBSA) methods are useful toidentify stabilizers among decoys.29So far, onlya few drug design efforts have been directed towardstabilizer design. For example, fragment-based drug discovery wasused for the stabilizer design of the adapter protein 14-3-3 withp65-derived protein,31 protein–protein docking,34 and protein complexstructure prediction,35 the predictionof potential PPI stabilizers from merely the knowledge of just proteinsequences can be a promising route at relatively low cost. To achievesuch a goal, it is of fundamental importance to understand the functionalmechanisms of the existing stabilizers.With the rapid advance in protein structure prediction,In the first part ofthis study, we discuss the theory of PPI stabilizationand emphasize the importance of a dual-binding mechanism. Next, weanalyze existing protein–protein (PP) complexes with boundstabilizers using MD simulations combined with the calculation ofinteraction free energies between protein partners and the stabilizer.We use the term stabilizer-induced PPI for complexes that do not formwithout a stabilizer 1A, wherein silico by direct computationalprobing of the PP complex (without stabilizer) crystal structure.Those pockets hidden in the PP complex structure can be revealed byrunning short MD simulations and subsequent pocket detection. We alsodemonstrate that the dual-binding mechanism can be useful to identifypotential PPI stabilizers. Based on the model, we propose a protocolfor PPI stabilizer discovery combining pocket probing, molecular docking,and MD simulation. We also check a large set of known PP complexesfor druggable pockets and find, for 75% of the cases, interface cavitiesuseful for stabilizer binding and apply the suggested protocol toa subset of 5 PP complexes. It indicates promising results that couldbe implemented in the structure-based drug design of PPI stabilizers.Importantly, we show that 80% of the stabilizer-bindingpocketscan be detected 36 ΔΔ, where KD is the dissociationequilibrium constant and ΔΔG is the Gibbsbinding free energy, kB is Boltzmann constant,and T is the temperature, we can study how [RLSopt] changes with ΔΔGRS and ΔΔGLS according to GRS < ΔΔGLS, e.g., the stabilizer binds more strongly to the receptor,enhancing both ΔΔGRS and ΔΔGLS by a small amount improves the effectivedissociation equilibrium constant KRL,eff = [R0][L0]/[RLSopt] much more efficiently than enhancing ΔΔGRS alone. This indicates that the protein thatbinds weaker to the stabilizer, namely, max {ΔΔGRS, ΔΔGLS, plays a more decisive role in determining the stabilization efficiency andby Pyrrolidone 1 .42 In example A2, the dimerization of the weaklydimerizing protein immunologic regulators programmed death 1 (PD-1)ligand  is enhanced by small molecules BMS-202  and BMS-8 .43 Similarly,the dimerization of bromodomain-containing protein 4 (BRD4) can bealso induced by small triazolopyridazine-containing molecules compound6  and compound6 44 in example A3. Notably, although different binding posesof the stabilizers in A1 and A2 are observed, the RL complexes adoptsimilar conformations upon stabilizer binding, as indicated by thehigh TM-scores in A stabilized PPI contains a receptor protein (R),a ligand protein, and a stabilizer (S) andis abbreviated as an RLS complex. Similarly, a PPI without a stabilizeris abbreviated as an RL complex. The 15 PPIs with available three-dimensional(3D) structures are split into 3 data sets. Set A contains 3 stabilizer-inducedPPIs where the stabilization effect of two stabilizers on the sameR-L pair are experimentally measured and cocrystallized. These wereselectedto investigate how the experimentally measured stabilization efficiencyis correlated with the structural, dynamical, and thermodynamic propertiesof the RLS complexes. Set B is formed by an additional 7 stabilizer-inducedPPIs (structures known in complex with one stabilizer) and 5 stabilizer-enhancedPPIs from set C . In example A1, the complex between protein 14-3-3 andthe C-terminal domain of PMA2 can be induced by the dipeptide Epibestatinand S S 3A using ΔΔG(RL)S . Indeed,GLS less than −5 kcal/mol , which is theminimum binding free energy required to induce a PPI complex with KRL,eff at a μM level at [R0] = [L0] = 1 μM, accordingto the theoretical model  and without stabilizers (R-only). We found that the interfaceresidues of the receptors, e.g., the residues in contact with theligand protein in the RLS complexes, remain stable only in the presenceof stabilizers but become much more mobile when the stabilizers areremoved, indicated by increased interface root-mean-square deviationiRMSD, 3C–D.HenGRL) the stabilizer . Interestingly, the unfavorableRL contacts in the stabilizer-induced PPI complexes are often formedby the residues responsible for stabilizer binding .We next calculated the directinteraction between receptor andligand with the stabilizer izer see . Figure Figure S3, and SI Appendix, Figure S4). In contrast, the interfaces of the stabilizer-enhancedPPIs remained stable without stabilizers . The binding of these stabilizers involves many favorableRL contacts  and is thereforemore likely to disturb the direct RL interactions.Thus, in addition tothe dual-binding mechanism, stabilizers canalso contribute to more favorable binding by shielding the unfavorableRL interactions. This may explain why the conformations of receptorand ligand in set A tend to form a similar conformation upon stabilizerbinding where favorable RL contacts and druggable pockets near theunfavorable RL contacts can be formed simultaneously 1. When s55 to probe druggable pockets of the natural RLcomplexes and detected at least 10 pockets in each RL complex . Importantly, the stabilizer-bindingpocket hidden in the RL crystal structure of the C4 complex occasionallyexpands to a volume large enough to accommodate the stabilizer witha rapid fluctuation in pocket volume . Upon removal of the stabilizer from the RLS complexes,the volume of the stabilizer-binding pocket showed in a few caseslittle changes during simulations. However, in many cases, a reducedpocket size distribution sampling frequently pocket volumes smallerthan the ligand volume was observed . Note, interface distortion and RL complex dissociation are expectedin all stabilizer-induced PPI if simulations were extended to muchlonger  time scales due to the very weak interactionin these cases.In order to discover potential PPI stabilizers in the scheme of structure-baseddrug design (SBDD), it is crucial to identify useful potential ligand-bindingpockets. Structural comparison has shown that PPI stabilizers canbind into an interface pocket of an RL complex without deforming theconformation 1, offeri complex 4A. Altho complex . Besides complex 4B. Furth complex reveals  complex S7. Impor complex S7. ImporOur systematicstudy of 18 PPI stabilizers reveals PPI stabilization mechanisms andthe dynamics of the stabilizer-binding pockets. In particular, thedual-binding mechanism derived from the starting theoretical modeland supported by experimental results shows that the binding affinitybetween the stabilizer and the weakly interacting protein partnerplays a decisive role in the RLS complexation process. Hence, it mayprovide a better ranking than the conventional structure-based drugdesign (SBDD) workflow, which usually only considers the total calculatedinteraction energy between the compound and the target of interest.28 through a virtual screening approach . We performed molecular docking,MD simulations, and MM/GBSA calculations to check the dual-bindingmechanism of these compounds . For all compounds, a stronger calculated interactionwith the receptor compared with the ligand was observed  have at leastone pocket consisting of 5 atoms from each side of the interface,173 complexes (76.5%) have at least 10 atoms from each side of theinterface, and 57 complexes (25.2%) have at least 20 atoms from eachside of the interface . We further extracted the best-scored interface pockets composedof at least 5 atoms from each protein, and several pockets were identified,some with pocket scores and drug scores higher than the known stabilizer-bindingpockets discussed in the previous sections .As a demonstration, we implemented the protocolto a subset ofprotein docking benchmark set 5.0,1AK4, 1Y64, 2GAF, 3H11, and 4FZA, for virtual screening(57 (see 3H11), all pockets can accommodate over 75%of the screening compounds with a docking score better than −6kcal/mol, indicating good druggability preselected with drug scores. As demonstrated in the present study, an ideal PPI stabilizerusually possesses a strong binding affinity to the weaker stabilizer-bindingpartner, indicated by a low max{ΔΔGRS, ΔΔGLS}, and a similarbinding affinity to both proteins, indicated by a low |ΔΔGRS – ΔΔGLS|. Among the 50 selected compounds, 34 compounds (68%)possess a max{ΔΔGRS, ΔΔGLS} lower than −15 kcal/mol, and a |ΔΔGRS – ΔΔGLS| smaller than 10 kcal/mol (Table S5) all exhibiteda max{ΔΔGRS, ΔΔGLS} lower than −24 kcal/mol, and |ΔΔGRS – ΔΔGLS| smaller than 4 kcal/mol, and are expected to be promisingstabilizers. Together with the dual-binding mechanism of stabilizers,our results provide not only a prediction of potential PPI stabilizersbut also a guide for further lead optimization, for example, by improvingthe RS interaction of compound 3 in PDBID 1AK4 or by improving the LS interaction ofcompound 4 in PDBID 3H11 .Among these pockets, we chose 5 interfacepockets with high drugscores, including PDBID creening5C. We pe structure.in silico.Indeed, our analysis of the stabilizer-binding pocketsin experimentalstructures without stabilizers demonstrated that most stabilizer-bindingpockets can be readily detected. Those pockets hidden in the RL complexescan show up in short MD simulations. Furthermore, interface pocketsthat can accommodate ligands, allowing stabilization by the dual-bindingmechanism, could be detected in the majority of cases in a large setof known PP complex structures. Based on our analysis, we demonstratedthat the dual-binding mechanism can be useful to identify potent stabilizersand propose a protocol for PPI stabilizer discovery. We applied theprotocol to 5 test cases and can suggest potential PPI stabilizersfor five protein–protein complexes. Combining the pocket-scoringtechnique and dual-binding criteria, our SBDD workflow can efficientlyidentify druggable interface pockets and estimate the stabilizingefficiency 35 one can design molecules that induce the complexation of selectedproteins. For a given interaction geometry, our protocol could alsobe useful for the identification of such compounds that induce a PPIor support a very weak association. Our work is the first computationalstudy systematically characterizing the binding behavior across differentkinds of PPI stabilizers. We anticipate that our study will facilitatefuture developments for the discovery of PPI stabilizers.Ideally, with advancesin structural prediction and protein–proteindocking techniques,58 was used with the wild-type sequences listedin 3bbr, T686A in PDBID 3b6q, L22Q, T51D, V52Iin the ligand of PDBID 3o98, S13A, T30D, V31Iin the ligand of PDBID 3m50 and 3m51, C6S in PDBID 3u15, and L9E and 3vbg. The RL complexes of sets A and B are generated by stripping thestabilizers from the corresponding RLS complex. Because of the highstructural similarity, A1-a, A2-a, and A3-a were taken to representthe RL complexes of A1, A2, and A3, respectively. Besides the 14-3-3protein and its ligands, where the terminal residues are as well interactingwith each other and the stabilizers, the N-terminus and C-terminusare capped with the neutral capping group ACE and NME with Ambertools20,59 respectively. To maintain the conformationsof the complex, two calcium ions are kept in the S100A4 homodimerand CaM/CaMBD2a complex, four glutamic acids are kept in the iGluR2homodimer, and a zinc ion is kept in the CK1α/CRL4. Also, disulfidebonds in PD1L/PD1L and lambda-6A/lambda-6A complexes are preserved.In this study, in total, 23 protein complexstructures were usedfrom the RCSB PDB database, including 18 RLS complexes and 5 RL complexes1. Due to60 at pH = 7.4. The atomicinteractions are described by the ff19SB force field61 for protein, OPC force field62 for water, the TIP4PEW63 force fieldfor ions, and GAFF264 for the stabilizers.The same procedure was also implemented to prepare the MD simulationof the 5 example complexes from protein docking benchmark data set5.0. Five replicas of 50 ns trajectories of each RL and RLS complexand 5 replicas of 200 ns long trajectories of each RS complex andreceptor-only system of PDBID 3m51, 2o98, 4mdk, and 1kkq are generated with MD simulation withthe CUDA-accelerated version of PMEMD from AMBER20 package with randomlyassigned initial velocities. Each simulation box underwent the minimizationprocess of maximal 35,000 steps and was equilibrated for 75 ps underconstant volume and 225 ps under constant pressure. Gradually decreasingpositional restraints on protein atoms are applied during the minimizationand equilibrium processes. The temperature was kept at 310 K usinga Langevin thermostat, and the pressure was kept at 1 bar with a Berendsenbarostat. The nonbonded cutoff distance was set to 9 Å, and thesimulation was integrated with a time step of 2 fs with the SHAKEalgorithm. Interface residues are defined by the residues within 5Å of the partner protein calculated using VMD-PYTHON. RMSD andiRMSD are calculated with the starting structure as the reference.RMSF, RMSD, and iRMSD are calculated using CPPTRAJ. The last 30 ns,in a total of 75 frames, of each 50 ns simulation in the RL and RLSsimulation are taken to calculate the interaction free energies betweenreceptor, ligand, and stabilizer using molecular mechanics coupledwith the generalized-Born surface area (MM/GBSA) method with an internaldielectric constant of 1 and external dielectric constant of 80. Theeffective Born radii are calculated with the GBOBCII model (igb =5) in ref  in ref . Pocket28 was generated with Open Babel60 and docked with Autodock vina.66 The docking procedure was performed on the 3D structurefrom PDB 5F74. The preparationof 5*50 ns trajectories and the binding energy calculation were performedwith the same procedure as mentioned above.Re-evaluation of the 13 compounds performed by Sijbesmaet al.Figure S19), we fetched 122,847,475 compoundsfulfilling such molecular properties from the ZINC20 database57 with the standard highest reactivity. We furthereliminated compounds with complex structures, which might cause difficultiesfor force field parametrization, by filtering the compounds with BertzCTcomplexity67 > 700 using RDkit,68 leading to 15,072.167, from which we randomlyselected 1 million compounds to perform molecular docking with AutoDock-GPU.69Considering manystabilizers in the present study possess molecularweights between 375 and 425 Da, and LogP between 2 and 4.5 (SI Appendix,


The 534 protein kinases encoded in the human genome constitute a large druggable class of proteins that include both well-studied and understudied “dark” members. Accurate prediction of dark kinase functions is a major bioinformatics challenge. Here, we employ a graph mining approach that uses the evolutionary and functional context encoded in knowledge graphs (KGs) to predict protein and pathway associations for understudied kinases. We propose a new scalable graph embedding approach, RegPattern2Vec, which employs regular pattern constrained random walks to sample diverse aspects of node context within a KG flexibly. RegPattern2Vec learns functional representations of kinases, interacting partners, post-translational modifications, pathways, cellular localization, and chemical interactions from a kinase-centric KG that integrates and conceptualizes data from curated heterogeneous data resources. By contextualizing information relevant to prediction, RegPattern2Vec improves accuracy and efficiency in comparison to other random walk-based graph embedding approaches. We show that the predictions produced by our model overlap with pathway enrichment data produced using experimentally validated Protein-Protein Interaction (PPI) data from both publicly available databases and experimental datasets not used in training. Our model also has the advantage of using the collected random walks as biological context to interpret the predicted protein-pathway associations. We provide high-confidence pathway predictions for 34 dark kinases and present three case studies in which analysis of meta-paths associated with the prediction enables biological interpretation. Overall, RegPattern2Vec efficiently samples multiple node types for link prediction on biological knowledge graphs and the predicted associations between understudied kinases, pseudokinases, and known pathways serve as a conceptual starting point for hypothesis generation and testing. Protein kinases play a fundamental role in cell signaling by phosphorylating peptides and proteins on serine, threonine, and tyrosine residues. The 534 protein kinases encoded in the human genome regulate protein activity in diverse pathways including those involved in DNA repair, cell cycle control, and metabolism . BecauseThe compendium of genomic, proteomic and interactome data now available through the efforts of the IDG consortium and numerous investigator-initiated efforts allows for the possibility of inferring the functions and pathways for dark kinases through integrative mining of the known patterns and relationships in existing data. In particular, the development of network-based approaches, such as knowledge graphs (KGs), that relate and link diverse types of protein kinase data in the forms of networks (composed of nodes and edges) enables the prediction of dark kinase functions or pathways through network context . Indeed,Drug–targets–Protein and Protein–isAssociatedWith–DiseaseState, where Drug and Protein are node types and targets and isAssociatedWith are edge types (or relationships). Because heterogenous networks offer a broader range of knowledge representation over homogenous networks, they are suitable for biological representations that require data integration and conceptualization from multiple data sources. However, models that can fully leverage heterogenous graphs often suffer from scalability issues, limiting their predictive power and making the construction of the KG difficult (The working premise of these network-based approaches is that proteins (kinases in this case) that share similar network neighbors  with well-studied kinases are likely to be involved in common pathways. Several predictive ML methods utilize this premise, however many of these methods are designed for use with homogenous networks . Homogenifficult . Due to Embedding-based approaches have been proposed to overcome the limitations of sparse data within KGs, as they have the capability to represent the graph in dense and low dimensional feature space . HoweverRelation-preserving methods attempt to capture complex relationships by matching semantic similarities of entities and relations and learn “context” from nodes within the KG. Many base-line relation-preserving methods are still widely used for link prediction tasks with biological data . This deOne of the most well-known relation-preserving methods that utilizes random walks is metpath2vec. metapath2vec preserves the relationships in the graph during the sampling process by utilizing schemas (ordered sets of specific node or edge types). By specifying what node/edge types should be sampled, integration of different data types can be fully leveraged for predictions, thus accurately capturing the structure of heterogenous graphs. However, creating such meta-paths for complex heterogeneous KGs, often with no well-defined schema, is challenging and time-consuming with the performance of the model being highly dependent on the chosen meta-path (series of nodes).Several previous attempts to automatically generate meta-paths have been reported in the literature  but theyTo address this, we employ RegPattern2Vec , a novelhttps://github.com/gravelCompBio/RegPattern2Vec).Overall, we successfully employed RegPattern2Vec to capture the node/edge context more accurately within our KG. After sampling, the ML technique (link prediction) is used to predict new associations between dark kinases and characterized pathways. We place 34 dark kinases for which we have consistent, high-confidence predictions produced through replicate runs (with differing hyperparameters) in a pathway context and based on an analysis of the meta-paths navigated for three selected dark kinases, we provide biological interpretations of the predicted pathway associations. The RegPattern2Vec pipeline is available at the GitHub repository (ϕ:V → A and an edge type mapping function ψ:E → R (v ∈ V belongs to one particular node type in the node type set e ∈ E belongs to a particular edge type in the edge type set Subject-predicate-Object (KGs are very similar to Heterogeneous Information Networks. An Information Network is a directed graph  ψ:E → R . Each nork (RDF) , a notate-Object  represenCDK 13 (cyclin dependent kinase 13) can be connected by an edge (relationship) labeled hasPathway to a node representing a pathway NeutrophilDegranulation, which represents the knowledge that CDK 13 participates in the pathway NeutrophilDegranulation. In such a KG, CDK 13 may be connected to other nodes using different labels (hasPathway), such as CDK13–hasMolelcularFunction—CyclinBinding, CDK13–hasBiologicalProcess—GranulocyteActivation and CDK13–hasCellularComponent—GolgiApparatus. Other protein kinases can additionally be included in the neighborhood context through shared nodes, such as the shared molecular function of CyclinBinding between dark kinase CDK13 and light kinase CDK6 (cyclin-dependent kinase 6). Here, edges have multiple labels, and destination nodes are of different types, which indicates that it is a heterogeneous KG.For example, in ϕ:V → A and its edge (relation) type mapping ψ:E → R.S is a directed graph defined over node types A, with edges as relations from R. Similarly, the schema of a KG represented in RDF is represented in RDFS . Also, given a KG schema, we can create a KG conforming to the schema containing many individuals (nodes).Given a Heterogeneous Information Network F Schema ). For exDisease . Information on these data types was extracted from the following databases: ProKinO  associations were also included within our KG under node type PTM and were retrieved from iPTMnet v5  (GOTerm node type) and Reactome v76 (Pathway node type), respectively.The node types  ProKinO , Dark Ki ProKinO , UniProt ProKinO , PDBe-KB ProKinO , Pfam v3 ProKinO , and STR ProKinO . Of noteTMnet v5 . The CheDatabase  (proteinDatabase . The GenDatabase  database (KLIFS)  and Phar (KLIFS) , respect (KLIFS)  (GOTerm tome v76  (Pathwayhubs). Similar filtering was done with Protein-GO term associations retrieved from Gene Ontology-GO terms with more than 5000 associations were removed. Additionally, Protein-GO terms were assigned separate node and edge types in the KG according to the MolecularFunction, CellularComponent, and BiologicalProcess terms.The retrieved datasets were further curated prior to KG generation and population. All protein-pathway associations obtained from Reactome were split into manually curated associations (evidence = TAS) and predicted associations (evidence = IEA), only manually curated associations were included in our KG. Additionally, due to the hierarchical nature of the data taken from Reactome , we eliminated high level pathways as they were deemed too general for our experiments. For example, we filtered out the pathways beneath “Disease” (R-HSA-1643685) but not beneath “Infectious disease” (R-HSA-5663205). This was done not only to reduce redundancy in the data but also to ensure the vector embeddings created from our KG for link prediction were not overfitted to highly connected nodes  directly involving a kinase and with an experimental data score of more than 700 were included in our KG, and only PTMs (iPTMnet) with a confidence score of more than 1.0 were included in the graph. Additionally, Pfam entries with the protein kinase domains (“Pkinase” and “Pkinase_Tyr”) were removed.Protein, GO Terms , Disease, and other types. The nodes collected during a hypothetical random walk are shown in Protein, any node not Protein and not Pathway, Protein, Pathway] (Protein type includes the understudied kinases (referred to as DarkKinases), well-studied kinases (referred to as LightKinases), and other proteins. This distinction is essential, as it gives us the ability to make predictions for a subset of proteins. The edge types (edges in the schema) are not labeled here, as we do not consider them in our method described here. Instead, we only rely on the types of source and target nodes. The loop edge returning to the Protein type indicates the Protein–Protein Interaction relation, which is included in our knowledge graph. Finally, The KG used in our link prediction experiments described here contains several node types such as Pathway] . In addiIn the experiments and results presented in this paper, we used RegPattern2Vec. It is used as the first step in our link prediction process to produce vector representations for the nodes in the graph and formulate the link prediction as a classification problem. A ML model was trained using the combined vectors of existing pairs of nodes connected by an edge (link) of interest. The outline of the method is illustrated in H[∧T]+ HT for link prediction tasks, i.e., edges (links) between nodes of type H and T. In the pattern, H is a source node type and T is a target node type of the link of interest, respectively. Additionally, [∧T] denotes any node type different from T (in regular expressions terminology it is known as the complement operator), and [∧T]+ denotes a repetition (sequence) of one or more node types that are different from T. The overall pattern requires that any random walk must match the specified sequence of node types, in that a walk must begin with a node of type H then it must be followed by one or more nodes of type different than T, then a node of type H, which must then be immediately followed by a node of type T.Given a Heterogenous Information Network (as defined above), we have used regular patterns of the form Protein [∧Pathway] + Protein Pathway, and each random walk instance must match it. Each walk starts from a Protein node. Selecting the next nodes on a walk is based on the existing graph nodes and matching the neighbor’s type in the regular pattern. When a random walk reaches a Pathway node, it follows the pattern in reverse order until a certain number of steps (nodes) in the walk are reached, or when it reaches a termination node, i.e., when there is no neighbor to match the pattern. We designed this specific aspect of the model due to the challenge of data sparsity for dark kinases. By allowing the reverse pattern to be followed we allow for additional relevant neighborhood nodes to be collected- when walking backwards once the pathway node is reached, the walk travels in a random direction until the hyperparameters are satisfied, capturing potentially relevant neighbors for our prediction task.Simply put, the regular pattern is defined as a schema subgraph that should include only the relevant node types for a specific problem. An example regular pattern subgraph is shown in walk length” hyperparameter. Having the undirected heterogeneous network and a selected regular pattern, the random walk can be started from each instance of a starting node type in the pattern. As we want all the nodes to appear in our walks, iterating over them would be desirable. It is obvious that if we repeat the walk from each node , we will discover more walks as the node might link to multiple nodes of the same type. We call this hyperparameter the “number of walks” (referred to as NW in Results ‘Benchmarking RegPattern2Vec’s predictive performance with metapath2vec and other Graph Embedding Approaches’–Investigating the impact of hyperparameters on regpattern2vec’s predictive variability among replicate runs’). We will discuss how to choose the hyperparameter and the analysis of their impact in the Results ‘Benchmarking RegPattern2Vec’s predictive performance with metapath2vec and other Graph Embedding Approaches’–Investigating the impact of hyperparameters on regpattern2vec’s predictive variability among replicate runs’. The next step for each node is to select a node from the adjacent nodes based on the established regular pattern. This might result in multiple choices, and this is where randomization comes to play. RegPattern2Vec can utilize random distribution created by a user-defined function to generate the same probability for all the nodes or use an arbitrary distribution. To implement the random walk collection process, a regular pattern is converted to a Deterministic Finite Automaton (DFA), denoted by M. Each possible walk step is mapped to the DFA. The DFA M is used to check if transitions are allowed (an edge between two nodes); hence, a disallowed change gets a zero probability and is not used in the random walks.As this work considered undirected graphs, enumerating a given graph’s path is computationally infeasible. The solution is to sample a subset of paths from all possible paths from a random distribution. The random walk constrained by a regular pattern is selected to generate walks of arbitrary length controlled by the “hubs). Because such high degree nodes can dominate random walks and, consequently, representation learning, one popular way to address this issue is to bias the walks by the inverse of degrees of nodes  type and then use the probability distribution by the inverse of node degrees to select the next node in the walk, but only among the nodes connected by the edge type chosen.In the formula, RegPattern2Vec uses a modified skip-gram model presented in  to generPathway data type. To generate negative examples, we must first work under the closed-world assumption. This assumes that all information needed is provided, so for a statement to be true it must be explicitly stated to be so. Thus, a lack of a statement denotes that it is false (this contrasts with the open-world assumption in which statements can be true without them being explicitly known to be so). For our relation of interest (predicted link), we randomly select a head and tail node not connected by an edge in the graph. The embeddings of such nodes are then combined to produce negative examples. This process is repeated until the number of negative examples matches the positive examples.For each pair of protein-pathway associations, we combined their vector embeddings utilizing a widely used Hadamard product . The resOur logistic regression method utilizes the one-versus-rest strategy, in which a multi-class classification is split into one binary classification problem per class, meaning a classifier will be trained for each task, and therefore all the other data points in the split data are used as negative examples with the methods discussed above (under the closed-world assumption). Thus, the number of negative examples is dependent on the number of positive examples for each classification task. Although our dataset is large, this strategy is employed due to the inherent data imbalance between well-studied and dark kinases.In this work, we propose a new guided random walk approach for KGs, known as RegPattern2Vec, used to predict pathway associations for dark kinases. To generate protein-pathway predictions, we first constructed a protein kinase KG by integrating curated data from various resources populating the graph with nodes on molecular function, disease association, protein–protein associations and more . The resulting KG consisted of 1,064,097 nodes (entities) and 6, 279, 373 edges (relationships) , Fig. 1CAfter sampling, the model learned vector embeddings for each node in the collected paths, utilizing a Natural Language Processing (NLP) model similar to metapath2vec . Link prProtein”, “Pathway”, and “Others” when learning representation for the nodes. The separation of nodes in PCA shows our vector representation captures the node types and their network context. Several key differences exist between protein nodes and other nodes in our graph that may explain why they cluster separately from all other nodes. Protein nodes (including those annotated as dark/light kinases) are the default starting node in the schema, and many edge types describe proteins and connect them to other nodes in our graph . This may allow protein nodes to occupy a unique space within the embedding reflected in the PCA.As previously mentioned in more detail,  RegPattern2Vec uses a modified skip-gram model presented in  to gener‘Protein’, ‘FunctionalDomain’, ‘Protein’, ‘Pathway’, ‘Protein’) showed the best performance, based on 10-fold cross-validation, with an f1-score of 0.87 and AUC ROC of 0.94 for protein-pathway prediction. In contrast, RegPattern2Vec achieved an f1-score of 0.90 and AUC ROC of 0.96 on the same training and testing datasets . To compare our method to metapath2vec, we produced AUC ROC curves for both models. Multiple curves were generated for the metapath2vec model as multiple meta-paths were used resulting in variability in the model’s performance . Metapatdatasets . We alsodatasets , MAGNN  and compared these results. The percentage of protein-pathway predictions that overlap  was used as a metric for our model’s robustness. This analysis also revealed the variability in the overlap of the protein-pathway predictions on a per-kinase basis (for both light and dark kinases). This variability may be due to imbalances in the data available for each kinase. Notably, greater variability was seen amongst dark kinases, with the highest percentage overlap being 87.5% and the lowest percentage overlap being 0% (for the hyperparameter 40 NW with 95% confidence cutoff). A supplementary table ”, which specifies the number of times a node should be sampled as the starting node during the random walk process . Repeating the walk from a specified node allows for a more complete characterization of neighbor nodes, often allowing for more node/edge types to be captured and considered in downstream analysis which is especially important when working with sparse data.One source of variability within our model is directly controlled by the hyperparameter “a To find the optimal NW, the same starting node was used for generating paths in the random walk anywhere from 10 to 80 times. This process was repeated, resulting in a total of three replicates for each variation of the NW hyperparameter. Overlap between the protein-pathway predictions obtained through replicate hyperparameter conditions was then compared. From this analysis, we sought to identify the smallest value for the hyperparameter NW that can accurately capture local structure and node context within the graph to make consistent predictions (as determined by replicate overlap). This analysis revealed that changing the hyperparameter NW did not drastically change the number of overlapping protein-pathway predictions between replicate datasets . Of noteAfter finding the optimal value for the hyperparameter NW , we deciDAG/IP3Signaling). A portion of the subgraph highlighting the common nodes sampled between different paths are shown in The dark protein kinase cAMP-activated catalytic subunit beta (PRKACB) was amongst the kinases identified with the highest overlap of protein-pathway predictions . PRKACB,PRKACG , PRKAR2A   and both A-kinase anchor protein 28 (AKA28), and cGMP-specific 3′, 5′-cyclic phosphodiesterase (PDE5A). Both proteins have biologically relevant functions supporting the protein-pathway prediction for PRKACB’s involvement with DAG/IP3 signaling. AKA28 has been shown to bind to type II regulatory subunits of PKA (related to our dark kinase) and anchors/targets PKA to discrete locations within the cell   and  data for dark kinases not included in our training data. PPI data was obtained from the Dark Kinase Knowledgebase . Of noteCDK19) and the pathway node of interest TGF-betaReceptorSignaling (R-HSA-170834) (CCNC (cyclin-c) and CDK8 (cyclin dependent kinase 8). Recent total mRNA sequencing has demonstrated that loss of CCNC could activate the transforming growth factor (TGF)-beta signaling pathway , we were able to isolate nodes commonly traversed for the protein node of interest (-170834) . Again, -170834)  displays pathway . CDK8 ha pathway . Through pathway . This co pathway . Taken tProtein-pathway predictions were additionally produced by our model for an understudied protein kinase PSKH2  which isPSKH2) and the prediction node of interest (CiliumAssembly R-HSA- 5617833) (UNC119B. unc-119 lipid binding chaperone B (UNC119B) is a protein that plays a key role in the localization of ciliary proteins by binding to N-myristoylated proteins and masking the hydrophobic lipid from hydrophilic cytosol, therefore facilitating trafficking to the primary cilium or the immunological synapse (Comparing the overlap in protein-pathway predictions (40 NW dataset) with the Reactome enrichment, 18 pathways were shown to overlap . The pre5617833) . Through synapse . Many ofOverall, through our analysis of the paths utilized by the model as context for the link prediction process, we demonstrated that our modified random walk approach, RegPattern2Vec, can successfully sample multiple types of nodes and edges. These findings support our notion that our model preserves both local and latent structure through appropriate neighbor sampling and therefore provides the node context needed to make predictions. The predictions made by our model regarding dark kinases allow us to explore dark kinases of interest in a pathway context, giving us an idea of how these kinases may function and relate to other proteins.In this work, we present a network approach to predict pathway associations for understudied dark kinases by leveraging the network context of well-studied kinases. We predict pathway associations for dark kinases and propose a new graph embedding approach, RegPattern2Vec, which has significant advantages over the previously proposed metapath2vec approach for KG embedding. First, unlike metapath2vec, RegPattern2Vec does not require pre-defined knowledge of KG schema to define the paths. Second, because RegPattern2Vec does not use fixed length meta-paths, it captures the structure of the graph and the characteristics of both local and latent representations more effectively, as indicated by improved performance measures . Third, i.e., two proteins interacting or a protein having a certain association). To address this limitation, we are currently working on a KG that includes subcellular localization information of proteins, and tissue/cell specificity to create a more informed negative set that better reflects the natural boundaries placed by biology rather than the boundaries of our current knowledge. Additional future directions include fully incorporating edge information into the embeddings used for our model’s predictions. The use of edge labels could allow for the use of additional Natural Language Processing (NLP) based models such as Transformers  of the KG using labeled semantics (regular patterns) to not only learn the local similarity between nodes and edges but also more complex and non-trivial associations which lead to novel protein-pathway predictions for dark kinases. Our method performs better in terms of scalability for large graphs compared to other network-based methods by sampling the graph more accurately while preserving node/edge context and structure.10.7717/peerj.15815/supp-1Supplemental Information 1https://github.com/gravelCompBio/RegPattern2Vec).AUC ROC curves were generated by excluding 50% of the known associations from the training set, link prediction was then carried out using a logistic regression algorithm type as the binary classification model. Both RegPattern2Vec and metapath2vec (with the best performing schema) perform well despite the imbalanced nature of the data within the graph which causes both data sparsity and scarcity. BoxE (fully expressive proximity-preserving method), RHINE (method that uses two major meta-labels for preserving graph structure), and MAGNN  appear to have greater difficulty learning on our knowledge graph. All models were used with default settings, the only parameters changed were embedding size  to allow for use of the embedding with our link prediction method. The configuration files which show the parameters each model was run with are additionally included in our github  are displayed. The Click here for additional data file.10.7717/peerj.15815/supp-3Supplemental Information 3Comparison of the overlap between protein-pathway predictions produced by the model when changing the hyperparameter for Number of Walks (NWs). The NW hyperparameter, indicating how many times the starting node should be sampled, was tested from 10 to 80 with each variation of the changed NW hyperparameter for the random walk process replicated three times. (A–B) Box plot of the average percentage overlap for protein-pathway predictions between the three replicates produced for all variations of the hyperparameter used (10–80). Overlap of the predictions produced by the model was compared on a per dark kinase basis and then averaged overall for all dark kinases (A) filtered for 0.7 confidence or (B) filtered for 0.9 confidence.Click here for additional data file.10.7717/peerj.15815/supp-4Supplemental Information 4Table depicting all node type abundance and average degree (number of edges) for node types within our KG.Click here for additional data file.10.7717/peerj.15815/supp-5Supplemental Information 5Table depicting the average overlap between three replicates for all hyper-parameters tested (NW from 10–80). Columns include “Number of predictions that overlap across 3 replicates” and “Total number of unique pathway predictions”. The “Percent of pathway predictions that overlap” column is made by dividing the values of the other columns.Click here for additional data file.10.7717/peerj.15815/supp-6Supplemental Information 6This file can be used to generate the subgraph highlighted in Click here for additional data file.10.7717/peerj.15815/supp-7Supplemental Information 7Table depicting all protein-pathway predictions produced through our model that overlapped with the Reactome enrichment results produced through protein–protein interactions from the Dark Kinase Knowledgebase. Pathway associations for dark kinases.Click here for additional data file.10.7717/peerj.15815/supp-8Supplemental Information 8Table depicting all protein pathway predictions that overlapped with the protein–protein interactions from experimentally validated mass-spectrometry data not used in training. Pathway associations that were connected to dark kinases from RegPattern2Vec’s predictions were linked to their associated pathway according to Reactome.Click here for additional data file.10.7717/peerj.15815/supp-9Supplemental Information 9This file can be used to generate the subgraph highlighted in Click here for additional data file.10.7717/peerj.15815/supp-10Supplemental Information 10This file can be used to generate the subgraph highlighted in Click here for additional data file.10.7717/peerj.15815/supp-11Supplemental Information 11This file can be used to generate the subgraph highlighted in Click here for additional data file.