Once bound to their target proteins, Fe(iii)-induced catalytic oxidation occurred in proximity of the antigen's epitope. Oxidative proteomic analysis was then used to determine the degree of oxidation, the site of oxidation within the targeted antigen, and the interacting proteins that were in close proximity to the targeted antigen. An AAPL score was generated for each protein yielding the specificity of the oxidation and proximity of the interacting protein to the target antigen. As a final demonstration of its utility, the AAPL approach was applied to map the interactors of liver–intestine-cadherin (CDH17) in colon cancer cells.Antibodies targeting specific antigens are widely utilized in biological research to investigate protein interactions or to quantify target antigens. Here, we introduce antigen–antibody proximity labeling (AAPL), a novel method to map the antigen interaction sites as well as interactors of antibody-targeted proteins. As a proof of concept, AAPL was demonstrated using sodium/potassium transporting ATPase (ATP1A1) and epidermal growth factor receptor 2 (ERBB2)-specific antibodies that were modified with an Fe( Modified catalytic antibodies targeting specific antigens are employed to investigate protein interactions and antigen interaction sites. Biochemical engineering approaches such as phage display libraries have been applied to screen the potential antigens of a targeted antibody based on their interactions.2,3 With computational tools, potential protein interactions (PPI) can also be simulated based on properties of amino acids and their post-translational modifications.4,5 Mass spectrometry (MS)-based approaches, such as affinity purification mass spectrometry (AP-MS) and cross-linking mass spectrometry (XL-MS) have recently emerged for characterizing protein–protein interactions. In AP-MS, enrichment of targeted proteins (baits) is realized through the overexpression of epitope-tagged bait protein, with interactors of the bait being enriched simultaneously. The mass spectrometry analysis enables the identification and quantification of interactors, which reveals the alterations of interaction under different conditions.6 This approach has been applied widely to map PPI networks of distinct targeted proteins.7–9 However, weak and transient interactions are difficult to be captured by this technique. To capture both strong and weak interactors in XL-MS, interacting amino acids were crosslinked by a linker in situ, and the crosslinked peptides were analyzed by high resolution LC-MS/MS.10,11 Recent advances in novel linkers as well as in instrumentation to yield higher-order fragmentation, and annotation software allowed XL-MS to be applied widely to mapping protein–protein interactions.12,13 In one study, Wheat et al. developed a XL-MS platform using a multifunctional cross-linker with advanced separation and MS instrumentation for in vivo mapping of PPI thereby generating a comprehensive PPI network in vivo in HEK293 cells.14Cell membrane protein interactions are essential in signal transduction and other fundamental biological pathways.15–17 With proximity labeling proteomics, interactors with both strong or weak interactions of the target are identified. Several studies have employed this approach to characterize protein interactions. Lobingier et al.18 have developed a proximity labeling method combining with the engineered ascorbic acid peroxidase (APEX) to investigate the interaction network of the protein β adrenergic receptor (B2AR). In another study, a biotinylation reaction was used for the proximity labeling of cell membrane proteins.19 However, biotinylation-based approaches require more complicated sample preparation and streptavidin enrichment does not provide information about the site of the interaction. Hydroxyl radical protein footprinting (HRPF) is commonly used for the determination of protein structure and protein–ligand interactions, where the interaction sites are characterized by the site-specific oxidation of proteins and mass spectrometry.20,21 Combining proximity labeling and hydroxyl radical protein footprinting, Li and coworkers developed a labeling approach by immobilizing a catalytic Fe(iii) probe on cell membrane sialic acids to map the proteins interacting with sialylated glycans through site-specific oxidation.22 The enrichment of interacting proteins was not required with this approach, which assess the extent of the protein–protein interactions by quantifying the degree of oxidation at interaction sites.Mass spectrometry-based proximity labeling coupled with proteomic techniques is another widely used approach for probing protein–protein interactions. By fusing the targeted proteins with specific enzymes or catalytic probes, proteins localized near the interaction site are labeled by catalytic reactions that can then be characterized by quantitative proteomics.N-glycans of the antibodies were modified by changing terminal galactoses  to N-azidoacetylgalactosamines , followed by the conjugation of a synthesized clickable probe, dibenzocyclooctyne-functionalized bromoacetamidobenzyl-EDTA iron(iii) chelate (DBCO-FeBABE) to the GalNAz groups. In another approach, the antibody was directly modified with the iron(iii) chelate of bromoacetamidobenzyl-EDTA (FeBABE) molecule at cysteine residues via their free thiol group. Modified antibodies directed against sodium/potassium transporting ATPase (Na+/K+-ATPase) and human epidermal growth factor receptor 2 (ERBB2) were used as prototypical examples, with the goal of labeling the targeted cell membrane proteins and their interactors in the immortalized prostate epithelium cell line, PNT2, and the human breast cancer cell line, SKBR3, respectively. The oxidations of targeted antigens and their associated proteins were monitored through the quantitative oxidative proteomic analysis to optimize the reaction conditions for specific labeling. The AAPL method with optimized condition was then applied to study the interactors of liver–intestine cadherin (LI-cadherin/CDH17), which is a cadherin-like protein expressed mainly in liver and intestine, as well as the involvement of CDH17 glycosylation playing in its interactome.In the current study, we developed antigen–antibody proximity labeling (AAPL) as a technique to map the epitopes of a targeted antigen as well as the interactors of the antigen by proximity-dependent oxidation. Two separate approaches were employed to modify the antigen-specific targeting antibody to create a molecule with catalytic function. In one method, the iii)-induced catalytic oxidation reactions to their target antigens. Specifically, two methods were used. In the first method, antibody Fc region glycans were covalently modified with addition of DBCO-FeBABE (FeDBAb). The second method directly coupled FeBABE to cysteine residues of the antibody's amino acid backbone (FeAb). Each of the resulting antigen-specific oxidizing antibody probes targeted a unique type of cell membrane protein: ERBB2, a cell membrane receptor; Na+/K+-ATPase, a membrane pump; and LI-cadherin, an adhesin molecule. The modified antibodies were characterized for their glycan modification and binding properties through glycomic analysis and immunofluorescence imaging. The glycomic analysis of modified antibody glycans showed that azido group-containing glycans accounted for around 50% of all types of glycans. Moreover, immunofluorescent imaging revealed that the binding properties of modified antibodies with ether FeDBAb or FeAb were comparable to non-modified antibodies. More details of antibody characterization are included in the ESI.†Monoclonal antibodies specific to human epidermal growth factor receptor 2 (ERBB2) (Herceptin), sodium/potassium transporting ATPase subunit α1 (ATP1A1), and liver–intestine-cadherin (CDH17) were engineered  to direc2O2) was added to the culture medium to produce hydroxyl radicals in the proximity of the attached antibody , cell line treated with unmodified antibody (C2), and cell line treated with H2O2 (C3). For each of the engineered antigen-specific oxidizing probes, treatment conditions were optimized to obtain the optimal oxidation conditions. For FeAb, various ratios of antibody to FeBABE were tested to optimize Fe(iii) conjugation , were tested in parallel using cell lines expressing their respective antigens. After antibody binding, hydrogen peroxide . The antigen together with these co-oxidized proteins can be referred to as an “interaction network”. For the oxidative proteomic analysis, two values were calculated for the total protein oxidation: spectral counts of the oxidized protein and the extent of protein oxidation (EPO), as measured by ion abundances. The combination of spectral counts and ion abundances provided a more consistent method for measuring protein oxidation at the protein-specific level.23 In TopS, the spectral counts of each bait-enriched prey protein are used to calculate a TopS score to evaluate the extent of interaction of the prey protein and the bait protein. Similarly, in AAPL the extent of interaction was evaluated based on an AAPL score calculated using the spectral count of each oxidized protein together with its EPO. Detailed equations for the calculations are included in the Method section of the ESI.A method for calculating topological scores (TopS) of protein interactions in affinity purification mass spectrometry (AP-MS) was adapted here to generate an AAPL score.To generate a more stringent selection criteria for the antigen interactors, we first calculated the average number of oxidation sites for all proteins with AAPL values above 0 and below 1 after ATP1A1- and ERBB2-targeted oxidation. Results differed depending on the targeted antigen. For ATP1A1, a score of 0 corresponded to 2.8 sites, while a score of 1 corresponded to 4.8 oxidation sites. In ERBB2-targeted proximity labeling proteins with AAPL score above 0 had on average 2 oxidation sites, while those with AAPL score above 1 had on average 2.4 oxidation sites. The number of average oxidation sites suggest that proteins with AAPL scores above 1 are more oxidized, indicating a stronger interaction with the target protein.To validate the AAPL scores, the classification of interactors was compared to STRING, which provides scores based on the extent of interaction between proteins using derived literature data. The oxidized proteins were examined with STRING. Oxidized proteins that were known to interact directly with the target protein were classified as Type 1 (T1) and Type 2 (T2) with T1 defined as high confidence interactors, having STRING scores above 0.7, and T2 as medium level interactors, having STRING scores in the range of 0.4–0.7. Secondary interactors, those interacting with T1 and T2 proteins and not directly with ATP1A1 or ERBB2, were classified as T3 interactors. The remaining oxidized proteins were further classified as T4 interactors. The numbers of each type of interactors for ATP1A1 and ERBB2 based on the STRING analysis are summarized in The distributions of AAPL scores for T1 and T2 interactors were then examined. For ATP1A1, more than 50% of the interactors (T1 and T2 combined) had AAPL scores larger than 1. Nearly 15% had AAPL scores in the range of 0–1, and the remainder were below 0 . For ERBWe therefore classified the oxidized proteins based on the AAPL scores with the following criteria. Proteins with AAPL scores above 1 were classified as Level 1 (L1) interactors, while those with AAPL scores in the range of 0–1 and EPO FC above 1.5 were classified as Level 2 (L2) interactors. With this classification criteria, several of STRING classified T3 and T4 oxidized proteins were also found as L1 interactors of ATP1A1 and ERBB2 in PNT2 and SKBR3 cell lines, respectively; while not all the T1 and T2 proteins were primary interactors. We should note interactors identified by the AAPL scores may be dependent on the specific cell lines (or tissue) and culture conditions used in the analysis. In total, four T3 proteins and thirteen T4 proteins were found as AAPL L1 interactors of ATP1A1. Twelve T3 proteins and three T4 proteins were found as AAPL L1 interactors of ERBB2. ATP1A1 and ERBB2 L1 and L2 interactors and corresponding STRING-type associations are summarized in Comprehensive quantitative networks of AAPL interactors were obtained from STRING . The net24,25 All of enriched molecular function terms had adjusted p-value below 0.01, indicating a significant level of enrichment .34 These inhibitors are known to dramatically alter cell glycosylation. For example, Kif treatment of Caco-2 cells drastically increases high-mannose N-glycan surface expression. In contrast, SI extensively decreased the relative abundances of sialylated glycans. To investigate the effects of glycans on the protein interaction networks, differentiated Caco-2 cells were treated with the different glycosylation inhibitors, followed by incubation with the LI-cadherin-specific oxidizing probe and proximity labeling with H2O2. The oxidized sequences were then quantified by nanoLC-MS/MS.Cell surface glycosylation was modified with various inhibitors, including kifunensine (Kif), an α-mannosidase inhibitor,In total, there were 30 oxidized proteins that were also glycosylated based on the glycoproteomics analysis. Among these proteins, six of them were identified as interactors of LI-cadherin, with three classified as L1 interactors including CEAM1, ATLA3, and STT3A, and three as L2 interactors including 1A02, VTNC, and NOMO2. To examine whether the extents of the interactions were altered by the inhibitors, we measured the extent of protein oxidation of the native and the glycan modified cells and used the variations to determine extent of interaction. It was found that the oxidation of CEAM1, ATLA3, 1AO2, and VTNC were all decreased when cells were pretreated with kifunensine, suggesting that these proteins interacted less with LI-cadherin. However, there were little to no decrease in the extent of oxidation for STT3A and NOMO2. An identical behavior was observed when the sialic acid expression was decreased by SI . Importain situ. Antibodies labeled with DBCO-FeBABE were found to oxidize their respective protein antigens under optimized experimental conditions. The sites of oxidization on the target proteins were found to be higher in some regions, and it is worth noticing that some of them are located at extracellular regions  provides a new method for determining the interactors of targeted antigen. The approach for modifying antibodies with DBCO-FeBABE was developed and successfully employed to label the cell membrane antigen and its environment. The reaction conditions of AAPL were optimized for the optimal specific oxidations of targeted cell membrane proteins including ATP1A1 and ERBB2 and their associated proteins in cell lines PNT2 and SKBR3, respectively. With the optimal reaction conditions, the AAPL method enabled the specific oxidation of ATP1A1 and ERBB2 with their corresponding antibodies. The approach also yielded the identification of several known interactors of ATP1A1 and ERBB2, as well as novel potential interactors under the certain cellular environment. Further application of AAPL to the LI-cadherin in both undifferentiated and differentiated Caco-2 cell lines revealed several novel interactors of LI-cadherin. Finally, modifications on cell membrane glycosylation in differentiated Caco-2 cells resulted in decreased oxidation of a subset of AAPL interactors, indicating the essential roles that glycosylation plays in the interaction between these proteins with LI-cadherin.via the PRIDE35 partner repository with the dataset identifier PXD033466 and 10.6019/PXD033466.The mass spectrometry proteomics data have been deposited to the ProteomeXchange Consortium Q. L. designed and performed experiments, analyzed data, created the figures and wrote the manuscript. X. Y. and R. R. designed and performed experiments. E. M. wrote the manuscript. C. B. L. conceived the idea, supervised the study, and co-wrote the manuscript.There are no conflicts to declare.SC-013-D1SC06898A-s001SC-013-D1SC06898A-s002SC-013-D1SC06898A-s003SC-013-D1SC06898A-s004SC-013-D1SC06898A-s005SC-013-D1SC06898A-s006SC-013-D1SC06898A-s007SC-013-D1SC06898A-s008


For the multi-atom ligand, sulfate ion (SO42−), the predicted binding sites have a median RMSD of 3.82 Å to the true binding sites. The ligand-specific models built in this study can be used in solvent mapping studies and the RL framework can be readily scaled up to larger and more diverse sets of ligands.Protein ligand docking is an indispensable tool for computational prediction of protein functions and screening drug candidates. Despite significant progress over the past two decades, it is still a challenging problem, characterized by the still limited understanding of the energetics between proteins and ligands, and the vast conformational space that has to be searched to find a satisfactory solution. In this project, we developed a novel reinforcement learning (RL) approach, the asynchronous advantage actor-critic model (A3C), to address the protein ligand docking problem. The overall framework consists of two models. During the search process, the agent takes an action selected by the actor model based on the current location. The critic model then evaluates this action and predict the distance between the current location and true binding site. Experimental results showed that in both single- and multi-atom cases, our model improves binding site prediction substantially compared to a naïve model. For the single-atom ligand, copper ion (Cu Protein–ligand docking is a molecular modeling technique that predicts the binding and binding affinity between a target protein and a ligand , 2. As pThe protein–ligand docking (PLD) procedure can be described as a combination of searching algorithm and scoring function. The searching algorithm searches the space of the binding conformations between the ligand and the protein during the docking procedure, and the scoring function evaluates the goodness of a given binding conformation. Many methods have been developed in the past with various searching algorithms and scoring methods to address the docking problem. In general, there are three types of docking: rigid docking, flexible-rigid docking, and flexible docking . FirstlyOver the past decade, machine learning methods have been successfully applied to PLD and achieved state-of-the-art performances. Specifically, RF-Score  used theactor-critic (AC) [advantage actor-critic (A2C) [asynchronous advantage actor-critic (A3C) [This manuscript proposes a new approach to address the protein–ligand docking problem by applying deep reinforcement learning (RL). RL is a type of machine learning method which deals with sequential decision-making. It is straightforward to combine the RL method with deep learning, and the resulting method is known as deep RL (DRL). The key elements in RL are the agent, environment, and reward where the agent takes actions in the environment to make changes to the environment, which provides feedback in the form of a reward based on the action. The tic (AC)  is an RLic (A2C)  is an upic (A3C)  is an adDesigning an effective RL framework for protein–ligand docking that addresses both the sampling algorithm and scoring function is more challenging than applying deep learning methods separately to the two problems. To the best of our knowledge, there has been very few RL-based deep learning model  on proteThe rest part of this paper is organized as follows. Section 2 describes the details about the experimental data, including the source of the data and the preprocessing procedure. We also introduce the A3C algorithms for the protein–ligand docking problem and describe the training and testing procedures. In Sect. 3, we evaluate the performance of the proposed method on real protein–ligand binding data. In particular, we will focus on the effect of hyper-parameter settings, model architectures, and sample sizes to the model performance during the training and test processes. We conclude the paper with a discussion and summarization in Sect. Figure In this study, we generated several three-dimensional (3D) cubic gridded boxes to capture the local protein structure. Each box contains the whole true ligand position and the protein environment surrounding it. During the training process, the ligand of interest is placed at a randomly selected location in the box, and then it will move based on the guide from the model. Therefore, it is analogous to the maze problem in reinforcement learning: we need a model which is reasonable enough to guide the object in the environment to the goal and adjust the ligand pose to bind with the protein properly.2+, SO42−) from the Protein Data Bank (PDB) [https://www.rcsb.org/. All these structures are determined by X-ray crystallography with a resolution better than 2.0 Å and do not have any DNA/RNA/UNK molecules.We select protein–ligand complexes with some specific ligands (Cunk (PDB) , 26 with2+) ligand is there. The black points are atoms in the protein environment. The red point is the starting ligand site, which is randomly selected.The cubic gridded boxes are generated from the protein–ligand complex data. Given the protein structure and the ligand on the true ligand position , we at first randomly rotate the whole structure, and then generate a point as the center of the 18 Å × 18 Å × 18 Å cubic box. This box is gridded with 1 Å unit size voxel; most voxels contain no more than one heavy atom. The whole ligand and part of the protein structure should be contained in the box. Finally, we randomly rotate the ligand and move it to a random place in the box as the starting ligand site. According to this data generation process, each protein–ligand complex can generate multiple cubic gridded boxes. The right plot of Fig. In this paper, different types of atoms will be put in different channels, so the dimension of the input data is Asynchronous Advantage Actor-Critic (A3C) algorithm [Our reinforcement-learning-based protein–ligand docking method is constructed by using the lgorithm . We prop42−), respectively. We here simplify the problem by ignoring the atom collisions and bonds between atoms. The data setups and model structures will be discussed in the following sections.Thousands of chemical substances can form complexes with proteins and serve biological purposes. A ligand can be formed by either one or multiple atoms. This manuscript will show the natural protein–ligand complex experimental settings and results with ligand Copper ion  are selected from PDB. The protein sequence identity is lower than 30%. The cubic gridded boxes are generated based on the description in Sect. 2.1, and the number of proteins and boxes in the training and test dataset are presented in Table Input layer: Dimension is A parallel 3D convolutional layer which consists of three independent 3D convolutional layers: 4 filters with size Max pooling layer: Flatten layer.Dense layer with unit number 256 and ReLU activation.Output layer with unit number 6 and Softmax activation.We refer to the parallel Convolutional Neural Network (CNN) structure from ProDCoNN , 2 for aThe 42−-ligand dataset) is also selected from PDB with protein sequence identity lower than 30%. The numbers of proteins and boxes of the training dataset and test dataset are presented in Table Same as the Cu-ligand dataset, the Sulfate ligand dataset (SO42−-ligand dataset experiment is the method to compute the distance between two ligands. The shape of an SO42− ligand is very close to a regular tetrahedron with a Sulfur (S) atom in the middle and four Oxygen (O) atoms on the vertices, respectively. These four Oxygen atoms are symmetrical so that there are 24 (= 4!) possible ways to match two SO42− ligands in total. Therefore, for the SO42− ligand, we use permutation-distance as the metric to measure the distance between two ligands. We compute the root-mean-square deviations (RMSDs) of all the 24 possible matches, and then choose the smallest RMSD as the permutation-distance between two ligands. Another distance metric is the center-distance, which is the Euclidean distance between the S atom of the current ligand and the true ligand position. We will use both distances to evaluate the actor model performance for multi-atom ligand condition.One special feature in the SO42−-ligand dataset experiment is different from the structure in the Cu-ligand dataset experiment. Figure 42− ligand is composed of 5 atoms, which is much larger than the Cu ligand. Larger filters can better capture the features of the SO42− ligand. Second, there is a large space between the SO42− ligand and protein structure in many protein–ligand complexes so that larger filters can detect this property and improve the model performance. It is necessary to consider the rotation of the ligand in the SO42−-ligand dataset experiment. In addition to the six possible directions for the translation, we need to output extra six possible directions for the rotation: rotate clockwise and counterclockwise on each of X, Y, and Z axes. The sequential structure will be divided into two parallel branches at the end of the structure, and two 6-dimensional vectors will be output to predict the translation and rotation simultaneously. The critic model also has six layers, with the same first four layers as the actor model. In contrast, there is only one fully connected layer as the fifth layer with unit size 256 and activation ReLU. It is followed by the output layer with 1 unit and activation “tanh”.The actor model structure in the SO42−-ligand dataset.In this section, we will provide the training and test results of the Cu-ligand dataset and SOWe created 20,000 training samples for If we use the critic model prediction This formula describes the distance between the current state and the true ligand position predicted by the critic model, called critic distance. Four examples of the critic distances and real distances between the current and true sites are shown in Fig. 2 distance between the true and predicted location. The last two columns in Table The test process follows Algorithm 2 using 4000 samples. Figure 42−-ligand dataset experimental results are not as good as that for Cu-ligand. We can see the average improvement rate for center-distance in SO42−-ligand test data is 48.08%. On the contrary, the average improvement rate of Cu-ligand test case is 62.13%. One possible reason is the structural complexity of Sulfate ligand. Protein–ligand complexes with the SO42− ligand are more complex, and the models are more difficult to capture sufficient information. Another reason is that a large space is found around the Sulfate ligand true ligand position, whereas the actor model prefers to sites around atoms. However, we emphasize that the improvement rate in the SO42− ligand dataset still indicates the success of the proposed framework. The effect of the rotation is hard to observe since there is still a large gap between the predicted ligand position and the true ligand position. In addition, it is still applicable to use the critic distance as a stopping criterion because it shows the same tendency as the real distance curve.Table 42− data. Hence, we restrict our samples to the copper-ligand. We then perform the same prediction steps as described for the overall data, and the result in the test data is given in Table Considering the fact that both copper and sulfate ions can be found at nonspecific sites on a protein surface with no known biological function, we split out test samples into specific and nonspecific binding group by comparing the shortest distance between the ion and specific atoms in the protein and vdW bond . This spIn the 42− demonstrated the potential ability of generalization on more complicated ligands in the future study. In addition, when PDB structures show protein–ligand binding, it can be difficult to distinguish whether it is real binding or artifacts when the binding sites are not functional sites of the protein. In our study, we did not make that distinction. As a result, the performance was probably negative affected. If the artifacts cases can be identified and removed from the dataset, the performance may be improved.In this study, we have proposed a novel framework based on the A3C model for protein–ligand docking (PLD) and tested its feasibility. Using single-atom ligands and a small multi-atom ligand, we showed that the algorithm could move the ligands to locations very close to their true binding sites when starting from random starting locations far away from the proteins. In particular, we point out that our promising results on SOThe models built in this study are ligand-specific because we want to start from a simple scenario to see whether the RL would work in this simple case. Another reason for developing ligand-specific models first is because the general model would require a large number of ligands and ligand–protein complexes. Developing such a model would be well beyond the goal of this study, which is to demonstrate the feasibility of applying RL to PLD problem. In fact, ligand-specific models can be quite useful for solvent mapping , which hIn this study, we used a carefully designed Convolutional Neural Network (CNN) architecture to learn both the actor and critic models. More sophisticated architectures can also be used such as DenseNet, ResNet, and various attention mechanisms , Kimber One of the future studies would be to combine all the ligands together and train a general model for protein ligand docking. Such model would require the model to learn more specific interactions among different atoms on both proteins and the ligands. To extend out framework to more general applications, we first need to add more atom types for ligands (Sciortino et al. , 34, andWe emphasize that the proposed RL framework is still preliminary. Apart from what has been covered, alternative methods, more datasets and more types of proteins, and detailed comparison with the state-of-the-art would be explored in the future. In particular, we could collect multiple history steps in a set, called the replay buffer , and dra


The World Health Organisation reported in 2020 that six of the top 10 sources of death in low-income countries are parasites. Parasites are microorganisms in a relationship with a larger organism, the host. They acquire all benefits at the host’s expense. A disease develops if the parasitic infection disrupts normal functioning of the host. This disruption can range from mild to severe, including death. Humans and livestock continue to be challenged by established and emerging infectious disease threats. Vaccination is the most efficient tool for preventing current and future threats. Immunogenic proteins sourced from the disease-causing parasite are worthwhile vaccine components (subunits) due to reliable safety and manufacturing capacity. Publications with ‘subunit vaccine’ in their title have accumulated to thousands over the last three decades. However, there are possibly thousands more reporting immunogenicity results without mentioning ‘subunit’ and/or ‘vaccine’. The exact number is unclear given the non-standardised keywords in publications. The study aim is to identify parasite proteins that induce a protective response in an animal model as reported in the scientific literature within the last 30 years using machine learning and natural language processing. Source code to fulfil this aim and the vaccine candidate list obtained is made available. These symbiotic relationships encompass a spectrum governed by the benefits or detriments experienced by the microorganism and host. At the extreme end of this spectrum is parasitism, in which the microorganism acquires all the benefits at the host’s expense2. Parasitism is thought to be the most common mode of life on this planet3. Parasitic relationships can vary between asymptomatic infections to one that kills the host4. Organisms known to display parasitic relationships by living on or in a host include bacteria, viruses, fungi, protozoa, and helminths. In this study, the term parasite refers only to protozoa, helminths and ectoparasites ; and the hosts of interest are humans and livestock.Microorganisms exhibit diverse complex relationships with larger forms of life that harbour them5. Despite the global decline, the World Health Organisation (WHO) reported in December 2020 that six of the top 10 causes of death in low-income countries are infectious diseases. These countries are mainly in tropical regions, with marginalized populations living in impoverished conditions6. Malaria tops the list of parasitic induced diseases that cause the greatest burden. WHO in 2019, estimated that there were 229 million cases of malaria worldwide with 409,000 deaths. The list of burdensome parasites and diseases is not static7. Rapid population growth in areas with weak health systems, urbanization, globalization, climate change, civil conflict, antimicrobial resistance, and the changing nature of pathogen transmission between human and animal populations8 entails that the world will continue to be challenged by established, unknown, neglected tropical diseases (NTDs), emerging, and re-emerging infectious disease threats. Infectious diseases also cost the livestock industries billions of dollars annually, aside from substantial animal suffering10.Humans and livestock have evolved over millennia in a constant balance between their immune-system defences and parasite virulence. Infection occurs when the balance shifts in favour of parasites as they multiply within or on the host’s body. A disease develops if the infection disrupts the normal functioning of the host. This disruption can range from mild to severe. Table 11. The millions of lives saved due to vaccines against polio, smallpox, measles, diphtheria, tetanus, and rabies12 is testament to their effectiveness and future potential. An effective vaccine induces a pathogen-specific immune response providing long-lasting protection against infection13. The most effective type of vaccine so far has been live but attenuated whole-organisms with reduced virulence14. However, live attenuated vaccines have the potential to cause disease in immunosuppressed individuals; and/or are impracticable to grow in culture; and/or contain components likely to trigger detrimental side-effects, allergenic and/or reactogenic responses15; and/or present the possibility of reversion to a virulent form16.Vaccination is considered the most efficient tool for preventing current and future infectious disease threats17, such as specific proteins and/or polysaccharides. Although these non-living components on their own have generally proven to be less immunogenic than attenuated organisms, their safety superiority and easier manufacturing capacity without having to culture the pathogen16 makes them worthwhile endeavours for vaccine developers. Moreover, subunit components supported by appropriate vectors and adjuvants have the potential to enhance the efficacy18. Subunit vaccines can be further categorized into protein-based ; polysaccharide ; and conjugate (e.g. Haemophilus influenza type b vaccine)7. In this study, the subunit components of interest are proteins that possess immunogenic properties, which are expected to be proteins accessible to the immune system19.Subunit vaccines contain only antigenic components sourced from the disease-causing organism20. Previous studies detail the in silico vaccine discovery approach inspired by reverse vaccinology for parasites22, and for bacteria23. The following summarises this approach. Protein sequences, at the heart of the in silico approach, have been shown to encode signals that provide informative characteristics about the source protein, such as its destined subcellular location, the presence of transmembrane domains and epitopes, or whether it is anchored to a membrane. As to date however, there has been no detection of a sequence-derived characteristic of a parasitic protein indicating protective immunity in a host. Some predicted characteristics such as epitopes suggest immunogenicity, although direct methods of predicting epitopes recognised by T-cells and B-cells remain problematic25 (indirect prediction methods focusing on peptide binding to major histocompatibility complex (MHC) molecules have proved more computationally practicable but requires training data containing a sufficiently large set of MHC-peptide binding affinities that are experimentally validated and specific to the host of the target pathogen26). The in silico strategy, as a compromise, is based on the premise that immunogenic proteins will have a different profile of characteristics to non-immunogenic proteins, and immunogenic proteins are more likely to provide protective immunity. This profile difference is not distinguishable to an observer and requires a trained binary classifier implemented via supervised machine learning (ML). Training data are the initial information used to teach supervised ML algorithms in the process of developing a model, from which the model creates and refines its approaches required for classification. Consequently, quantity and quality of training data are paramount to the ML algorithm’s performance when given an unseen profile to classify. Ideal training data for the in silico strategy would comprise two labelled datasets of characteristic profiles: one set derived from proteins shown to induce a protective immune response in a host, and an opposing set derived from known non-immunogenic proteins. This ideal is currently not readily forthcoming for parasites and raises a fundamental cyclic conundrum that currently limits the in silico vaccine discovery potential. That is, a sufficient number of proteins verified to provide protective immunity are required in the training data to predict proteins likely to provide protection. The only known repository that contains protective antigens associated with parasites is a web database created in 2011 called Protegen27. It currently contains 167 unique parasitic antigens manually curated from selected peer-reviewed publications.The traditional approach to identifying subunit vaccine components involves first cultivating and dissecting the pathogen in the laboratory; followed by the identification of each isolated component. In 2000, reverse vaccinology was first proposed as a revolutionary idea to identify protein antigens in silico28 was in 1966. Over the following decades an increasing trickle of publications has accumulated to 1112 with ‘subunit vaccine’ or ‘subunit vaccines’ in their title. It is difficult to ascertain from PubMed search terms and keywords, how many of these publications are specific to ‘protein-based’ subunit vaccines. Furthermore, it is estimated here that there are possibly hundreds of published studies over the past several decades without ‘subunit vaccine’ in their title, but still report immunogenicity results of parasitic proteins. The exact number is unclear, however, given the non-standardised keywords in publications. Despite this uncertainty in exact numbers, PubMed searches suggest there is a potential pool of immunogenic proteins that could fulfil the elusive training data requirement for the in silico vaccine discovery approach.The PubMed database maintained by the National Center for Biotechnology Information (NCBI) at the U.S. National Library of Medicine (NLM) indicates that the first published investigation of a subunit vaccineThe aim of the current study is to ‘automatically’ identify all parasite proteins that induce a protective response in an animal model as reported in the scientific literature within the last 30 years. To achieve this aim, we have developed a computational pipeline that first classifies published abstracts using ML, and then extracts protein and/or gene names from the classified ‘abstracts of interest’ using natural language processing (NLP). Source code for the pipeline is provided via GitHub (a source code repository hosting service). The pipeline extracted 606 parasitic proteins from four phyla . All these proteins are reported in highly cited publications; and 485 of the 606 have evidence supporting their accessibility to the immune system. We judge them to have vaccine candidacy merit, and therefore relevant for ML training data and/or further investigation. Furthermore, protein characteristics of the candidates were extracted from existing resources or predicted from their sequences. A comparative analysis of these characteristics from different phyla is presented via tables and graphs. Unresolved limitations remain with the pipeline and in particular, it has a programmed inclination to identify more popular well-reported candidates to reduce the numbers of false positives. However, we believe this is the first reported attempt to ‘automatically’ generate a vaccine candidacy list from the scientific literature as a starting point for investigation, and is a superior time-saving alternative to a manual gathering process.Figure All publications over the last 30 years that contained either the word ‘parasite’, ‘vaccine’, ‘vaccinated’, or ‘vaccination’ in its title or abstract text were downloaded from PubMed—332,627 publications met this selection criteria. Each title and associated abstract was assessed to determine whether it contained matching words in keyword files . For instance, an abstract was classified as one of importance (a positive) if it contained defining words for the following: a parasite species, protective immunity, an animal model, a parasitic disease, and a gene or protein name associated with parasites. Supplementary Table and has a relevant protein name. This manual assessment suggests the rule-based approach has a 42.5% accuracy of selecting an ‘abstract of interest' but increases to 85% when the classifying threshold is greater than one protective immunity keyword. This accuracy, however, reduces to 80% in its capacity to fully identify the relevant protein name. These manually assessed abstracts are referred to henceforth as the ‘Verified positive’ and ‘Verified negative’ abstracts, and used later to evaluate the ML algorithm’s performance.Two sets of 100 abstracts were randomly selected from the 2744 classified abstracts. One set contained only abstracts with more than one protective immunity keyword (2308 met this criteria); whilst the other set contained only one keyword, excluding those with ‘vaccine’ or ‘vaccination’ (381 met this criteria). These abstracts were manually verified. Supplementary Table As a further independent test, the title and abstract were taken from 50 publications known to contain vaccine candidates . The rule-based criteria were applied to the 50 ‘title+abstracts’ (referred to henceforth as the ‘Evidence’ abstracts). The results are shown in Supplementary Table An in-house ML pipeline was created that classifies an unseen abstract as either one of interest i.e., one potentially containing a vaccine candidate (a positive) or one that is not of interest (a negative). Materials and methods—Abstract classification using machine learning—describes the pipeline. The pipeline accuracy as determined from tenfold cross-validation of the ML training data (1556 positives and 1556 negatives) was 99.6% . Abstracts of interest have steadily increased from 420 publications in 1991 to 4619 in 2020 (except years 2002 and 2016 showed declines from the year before). Supplementary Data https://spacy.io/—last viewed September 2021). It provides named entity recognition (NER) functionality as one of its options. NER, a sub-task of information extraction, finds and classifies named entities in text into pre-defined categories such as names of persons, organizations, and countries. In this study, our entities of interest are protein and gene names. We investigated three types of NER trained models: SpaCy default models (https://spacy.io/models), scispaCy, and a SpaCy custom model trained for this study . scispaCy is a Python package containing spaCy models trained for processing biomedical, scientific or clinical text29. As an illustration, Supplementary Data SpaCy is an open-source library for advanced NLP in Python (The Evidence and the Verified abstracts were processed with the custom NER model and a rule-based approach (see later ‘Protein name extraction using a rule-based approach’). An important point is that entities identified by both approaches are checked for a match in an in-house compiled protein and gene database . The accuracies in identifying ‘database verified’ protein or gene names were 88% and 91% (custom NER model), and 95% and 97% (rule-based) given Evidence and Verified abstracts, respectively. The rule-based accuracy is higher entirely because of the database verification e.g., a greater percentage of the entities checked for database verification are not valid protein names, whereas a substantially smaller percentage are invalid for the custom NER model. This equates to the rule-based approach generating more false positives and less false negatives; and conversely, the custom NER model generating less false positives and more false negatives. Using a consensus of the ‘rule-based+custom NER model’ entities, the accuracies reduce to 86% and 90% given Evidence and Verified abstracts, respectively.Supplementary Table (UniProt) database30 e.g., the assumed misspelt ‘Apcial membrane antigen I’ has the UniProt ID Q26162. Several names were incorrectly extracted from abstracts due to the following two reasons: (1) an incorrect link of a ‘gene name from an abstract’ to a ‘protein in a database’ e.g., some abstracts contain the words ‘circumsporozoite (CS) protein’. CS is the gene name for both Chorismate synthase and Citrate synthase. Protein names extracted from such abstracts included Circumsporozoite protein, Chorismate synthase, and Citrate synthase; (2) an incorrect name extraction from a larger name e.g., for abstracts containing ‘Calcium-dependent protein kinase’, ‘Apical membrane antigen 1’, and ‘heat shock protein 70’, the names ‘protein kinase’, ‘Apical membrane antigen’, and ‘heat shock protein’ were also extracted because they are valid names associated with unique Uniprot IDs.The protein names extracted from the 1776 classified abstracts were compiled into one list of 403 unique names , there are 3281  and 26  proteins with the name ‘Circumsporozoite protein’. Given only abstracts, it was not possible to determine which of these proteins was used in the related study’s vaccine candidacy evaluation. We did, nonetheless, narrow down the number of protein possibilities by using only proteins from the species specified in the abstract e.g., 31 analysis was performed on the 3731 sequences. CD-HIT provides functionality to cluster protein sequences that meet a sequence similarity identity threshold. The threshold chosen here was 90%. The CD-HIT analysis created 1099 clusters  were removed leaving 3731 sequences. Second, a CD-HIT (cluster database at high identity with tolerance)Leishmania infantum and Leishmania donovani, and HSP70 from Schistosoma japonicum. Figure Supplementary Table and if the publication was more than 5 years old—114 of the 438 unique proteins were questionable with respect to vaccine candidacy. We therefore propose that 324 proteins are worthy candidates for further investigation.The lists of ‘representative names’ and ‘unique names’ in Supplementary Table To provide a guide to the reliability of the proposed candidates, a random selection of publications associated with the candidates were manually read. Table 32. Focusing on only proteins naturally exposed to the immune system greatly narrows down the candidates for investigation. It must be acknowledged, however, that this strategy excludes potential candidates from the interior. For example, peptides from interior proteins can be presented to the immune system via MHC Class II molecules on professional antigen presenting cells (APCs) that have engulfed and digested protozoan parasites.Parasites are composed of thousands of proteins. Theoretically, any protein from a parasite irrespective of its normal location could potentially trigger a response when presented to the immune system. This is because all parasite proteins are ultimately foreign material to the host. It is currently impractical to investigate all proteins for their vaccine candidacy potential. A compromised strategy is therefore to identify those proteins more ‘likely’ to induce an immune response and consequently more worthy of laboratory investigation. A protein that is either external to or located on, or in, the membrane of a pathogen are strong indicators that it is more likely to be accessible to surveillance by the immune system than a protein within the interior of a pathogenCharacteristics of the 1099 representative proteins were extracted from existing resources or predicted from their sequences using freely available bioinformatic programs. Supplementary Table The characteristics collection also provided a resource for comparative analysis between the representative proteins —see Fig. The 1099 representative proteins were further filtered on accessibility score and h-index thresholds to create two vaccine candidate groups: exposed and non-exposed , Toxoplasma gondii (strain ME49), and Babesia bovis (strain T2Bo) were processed through Vacceed to predict their probability of being exposed to the immune system. All three species are apicomplexans. Plasmodium causes malaria ; T. gondii causes toxoplasmosis (responsible for birth defects in humans) and considered the model organism for Apicomplexa34; and B. bovis causes babesiosis, a significant economic disease burden to livestock industries36.Every known protein for Plasmodium list has 354 unique names associated with 704 proteins having a greater than 90% probability of being correctly classified as naturally exposed to the immune system. Note that 582 of these 704 proteins are predicted, 102 are inferred from homology, 19 have evidence at the protein level, and 150 have ‘putative’ in the name. Notable in the 354 names are those reported to have been explored as vaccine candidates, namely: apical membrane antigen 137, merozoite surface protein 138, merozoite surface protein 339, circumsporozoite protein40, glutamate-rich protein GLURP41, serine repeat antigen 542, erythrocyte binding antigen-17543, gpi-anchored micronemal antigen44, sporozoite invasion-associated protein 245, thrombospondin-related anonymous protein46, cell traversal protein for ookinetes and sporozoites47, liver stage antigen 346, liver stage antigen 148, rhoptry neck protein 249, cytoadherence linked asexual protein 3.145, 6-cysteine protein P12/P3850, and duffy binding-like merozoite surface protein45. Notable names explored as candidates but classified here with a less than 90% probability were sporozoite threonine and asparagine-rich protein (82.7%)51, merozoite surface protein 2 (80%)52, rhoptry-associated leucine zipper-like protein 1 (75.3%)53, reticulocyte binding protein homologue 5 (45.5%)54, ring-infected erythrocyte surface antigen (60.3% and 38.8%)55, and erythrocyte membrane protein 1 (PfEMP1)56. There are 61 PfEMP1 proteins and all were less than 70% (25 less than 50%). PfEMP1 proteins do not contain signal peptides and are exported by distinct non-classical secretion pathways56, which may explain the low Vacceed scores. Of the 704 proteins, 144 are named ‘conserved protein, unknown function’. These proteins of unknown function are considered promising candidates worth exploring because they have characteristics encoded in their sequences similar to those proteins previously explored.Supplementary Table Toxoplasma list has 393 unique names associated with 1022 proteins having a greater than 90% probability of being correctly classified as naturally exposed to the immune system . Notable names previously explored as vaccine candidates are dense granule proteins GRA2 to GRA8, GRA14 (GRA6 an exception with 85.9%)61; Microneme protein MIC262; MIC363; Protein disulfide-isomerase64; MIC12, SAG-related sequence SRS13, and Rhoptry protein ROP665. Notable names explored as candidates but classified here with a less than 90% probability are SAG-related sequence SRS29A (89.9%)65, MIC1 (88.8%)66, MAG1 (87.7%)67, Rhoptry protein ROP18 (77.4%)64, ROP1 (40.4%)68, Profilin (1.3%)69. Of the 1022 proteins, 474 are named ‘Uncharacterized protein’.The Babesia list has 106 unique names associated with 435 proteins having a greater than 90% probability of being correctly classified as naturally exposed . Notable names previously explored are 12D3 antigen70, merozoite surface antigen (MSA) 1 and 272, thrombospondin-related anonymous protein73, spherical body protein 2 (SBP2)74. A notable name explored as candidate but classified here with a less than 90% probability is rhoptry-associated protein 1 (RAP-1) (77.4%)75. Proteins that are exported from Babesia into the host’s red blood cell (RBC) cytoplasm and/or the RBC membrane are considered worthy vaccine candidates76. Variant erythrocyte surface antigen (VESA) proteins are the main known B. bovis RBC surface-exposed proteins77. There are 133 VESA proteins, nine have a greater than 90%, 79 between 50 and 90%, and 45 less than 50% probability of being naturally exposed. Small open reading frame (smORF) proteins are proposed to play a role in sequestration together with VESA proteins78. Sequestration is the attachment of parasitised RBCs to ligands on endothelial cells of blood vessels. There are 44 smORF proteins, 41 have a greater than 90% probability of being exposed . Of the 435 proteins, 39 are named ‘Uncharacterized protein’, 126 ‘conserved hypothetical protein’, and 107 ‘membrane protein, putative’.The Only laboratory testing can verify that the proteins with predicted high probabilities are truly commendable for continued investigation. Given the thousands of proteins constituting a pathogen, the purpose of the lists is to highlight the more worthy ones for laboratory testing. The expectation is that a reduced list will ultimately save time and money over the traditional identification approach.23 have shown that reverse vaccinology can be improved with a machine learning approach. Crucial requisites to this approach are quantity and quality training data consisting of positives and negatives. Ideally, positives are sourced from hundreds of proteins shown to induce a protective immune response in a host. The catalyst for this study was a need for positives to conduct an in silico vaccine discovery approach against Babesia bovis. A search for single or multiple public repositories that might contain a sufficient quantity of B. bovis positives was not forthcoming. The only repository distinguishing proteins with immunogenic potential is Protegen with currently 167 unique parasitic antigens, including five for B. bovis. Given there are hundreds of parasite species causing infectious diseases  to be false positives.Apcial membrane antigen I’; (4) proteins that are possibly the same have slight name variations in the abstract and/or databases e.g., Rhoptry associated protein-1 and Rhoptry-associated protein 1 are recorded as separate proteins in UniProt; and (5) all protein names extracted from abstracts are not necessarily immunogenic proteins.The second pipeline stage, protein extraction, presented the most challenges. To reiterate, the extraction goal was to determine whether any word or ‘series of words’ in the classified abstract matched a known gene or protein in UniProt and/or NCBI databases. The challenges were because (1) abstracts rarely contain a protein’s unique identifier , which would unambiguously provide the link to the protein associated with the abstract’s study. Protein or gene names in UniProt and NCBI are not unique (hence the reason for IDs); (2) many proteins and genes are poorly annotated with single names like ‘protein’, ’antigen’, ‘putative’, ‘raw’, ’dead’, ‘sand’, ‘impact’ and a multitude of other names with an English language meaning; (3) names in abstracts and databases have inconsistency in spelling and/or capitalising names e.g., ‘In an attempt to overcome the protein extraction challenges, we used spaCy to build a custom NER model, trained specifically to extract gene and protein name entities. The NER model was trained with protein names from the training positives used to build the abstract classifier. Testing the custom NER model with the task of protein extraction on verified abstracts showed that it identifies expected names with less false positives but more false negatives than a rule-based approach. Notably, every protein and gene was correctly identified as an entity in an abstract if they also appeared in the training data, whereas other valid proteins and genes in the abstract were sometimes missed if not used in the training data. We propose that the number of false negatives can be greatly reduced by including more protein names in the NER training data. Ideally, the training data should contain a BIO tag for every parasite protein and gene name in the UniProt and NCBI databases. These databases represent the entire real world domain for protein name extraction, which means it would be impossible to overtrain the NER model given the ideal training data.The final output from the entire pipeline given 332,627 was 1099 protein names and their associated sequences. An important point is that this output number is variable due to the user-defined thresholds governing the final output e.g., separate thresholds can be applied to the ML abstract classification probability, CD-HIT identity, number of publications, number of animal models, and start and end year of publications. We acknowledge that it is debatable whether the most appropriate thresholds were used, but our aim was to obtain the optimum acceptable level of false predictions i.e., to find the optimal sensitivity and specificity. For example, both sensitivity and specificity would both be 100% in a perfect system. Changing thresholds in our real world system increased sensitivity whilst decreasing specificity, or conversely, decreased sensitivity while increasing specificity. Our considered optimum was the maximum obtainable sensitivity and specificity but with a higher sensitivity than specificity i.e., more false positives than negatives for further evaluation.The 1099 proteins were further filtered into two vaccine candidate groups, exposed and non-exposed, based on a protein’s normal location in the context of the entire parasite. The ‘exposed’ candidates are those proteins naturally exposed to the immune system i.e., membrane-associated or secreted, and ‘non-exposed’ are hidden within the cell interior. The groups were determined from circumstantial evidence of a protein’s accessibility to the immune system i.e., collected or predicted protein characteristics indicating a protein’s potential to be in view of a host’s immune system surveillance. Furthermore, proteins calculated here with high rather than low h-index were considered more favourably. Taken together, accessibility indicators and h-indexes, the final list of proposed exposed candidates contained 485 proteins . Although four arthropod proteins are in the 1099 representative names, all four had poor accessibility indicators and h-indexes and were consequently filtered from the candidate list. The unexposed group comprised 121 proteins with no accessibility indicators but high h-indexes .P. falciparum, T. gondii, and B. bovis) with the expectation 10% are incorrectly predicted given the training data tenfold cross-validation metrics; and the caveat that only laboratory testing can truly verify the predictions. The proteins with the highest probabilities are judged to be the optimum within the current quality constraints of protein sequences. Reverse vaccinology has the potential to rapidly advance vaccine development against parasites, but its potential has been hindered since its inception by the lack of verified protein sequences. Most parasite sequences are predicted from poor quality genomes. For example, a recent study79 reveals misassembly, karyotype differences, and chromosomal rearrangements of the T. gondii and Neospora caninum genomes following a re-evaluation. These genomes were originally sequenced using conventional Sanger sequencing technology80 i.e., first generation sequencing. Next-generation sequencing (NGS) has mostly superseded Sanger sequencing for genome research but has limitations caused by inadequacies of short-read outputs with repetitive regions scattered across the genome81. These limitations have caused assembly artefacts that are currently widely distributed in genome and proteome databases82. NGS is now regarded as second-generation sequencing. Third-generation sequencing (TGS), still under active development, has the capability to produce substantially longer reads than second generation sequencing. Reads that are longer than a repetitive region provides a solution to assembling long contigs spanning an entire genome. TGS potentially provides this solution but has a higher sequencing error rate and systematic error than NGS83. Nevertheless, the recent combination of NGS and TGS data with the advances in assembly technologies has resulted in greatly improved genome sequence quality, which is epitomised by the new TGS-derived T. gondii and N. caninum genomes79. An increase in quality parasite genomes ultimately equates to reverse vaccinology progressing towards its full potential.A vaccine is in effect a simulated infection to deceive the immune system into making memory B or T cell responses specific to an antigen. These memory cells then help protect against future infections from parasites possessing the same antigen i.e., provide a protective immune response. In the current study, we focused on naturally exposed immunogens under the premise that if they induced memory cells as a vaccine component, these memory cells are more likely to encounter the same immunogen during a real infection. We acknowledge that the benefits of reducing the number of candidates for investigation by maintaining this focus, comes at the cost of potentially missing important immunogenic peptides residing on interior proteins. This focus was further illustrated with a reverse vaccinology inspired program called Vacceed, which is designed to predict those proteins naturally exposed to the immune system. Probabilities of exposure were predicted for every protein from three important apicomplexan species  with an 86–90% accuracy based on independent testing. It is unclear how many candidates were missed because the percentage of publications reporting immunogenic parasitic proteins tested on animal models is unknown. A key closing point is that the current alternative to the pipeline for identifying pertinent parasitic proteins for further investigation is a manually, time-consuming approach using judicious PubMed keyword searching and colossal amounts of reading.All experiments and data generation was performed on a high performance computing (HPC) cluster node with 64 bit kernel, 32 MB memory, and 8 cores. The pipelines were designed for a Linux operating system and have only been tested on Red Hat Enterprise Linux 7.9, but are expected to work on most Linux distributions. Python version used was 3.6.8.https://github.com/goodswen/abstract_classification.Source codes indicated in this article as being provided are available via GitHub: https://www.ncbi.nlm.nih.gov/books/NBK25497—last viewed September 2021). The utilities provide an interface into the Entrez query. Entrez is a data retrieval system that provides users access to NCBI's databases such as PubMed. An in-house Python script (source code provided: ‘get_abstract.py’) used the module ‘Entrez’ from Biopython (version 1.79)84 to implement the E-utility interface.PubMed was the source for all abstracts. There is a limit of 10,000 abstracts that can be downloaded from the PubMed webpage. The limitation was overcome using the Entrez Programming Utilities  OR (vaccine[Title/Abstract]) OR (vaccinated[Title/Abstract]) OR (vaccination [Title/Abstract])) AND (("1991"[Date-Publication] : "2022"[Date-Publication])).The ‘Entrez.efetch’ return mode parameter (retmode) defined ‘text’ as the format of the PubMed retrieved data (XML is another option). The retrieved data contained more information than required e.g., author information. An in-house Python script (source code provided: ‘process_abstract.py’) created a more workable file that included only PubMed ID, Title, and Abstract per row for each publication (referred to henceforth as the Abstracts).https://www.cdc.gov/parasites/), Protegen database , and Meat and Livestock Australia (MLA) website on ticks  and 12,688,175 (unreviewed) invertebrate-related proteins were read, and 1,058 (reviewed) and 1,462,608 (reviewed) proteins linked to 174 Taxonomy IDs were recorded in a file referred to henceforth as ‘parasite_proteins’.The files ‘uniprot_sprot_invertebrates.dat.gz’  and ‘uniprot_trembl_invertebrates.dat.gz’  were downloaded from: https://ftp.ncbi.nlm.nih.gov/gene/DATA/gene_info.gz. An in-house Python script was used to parse the uncompressed files to generate one file with one gene per row under the main column headings: Gene ID, Gene Name, and Taxonomy ID. Only genes with a Taxonomy ID in the taxonomy_ disease_ link were extracted. In summary: 26,379,168 genes were read, and 596,571 genes linked to 174 Taxonomy IDs were recorded in a file referred to henceforth as ‘parasite_genes’.The file ‘gene_info.gz  was downloaded from: An in-house Python script (source code provided: ‘score_abstract.py’) classified an abstract based on whether it contained matching words to the following four criteria words: (1) genus or species names from taxonomy_ disease_ link; (2) Protection_keywords; (3) Animal_keywords, and (4) Disease_keywords. Abstracts with at least one count for each of the four criteria were classified as ‘abstracts of interest’. These abstracts were further processed to check if they contained words matching to protein or gene names in parasite_proteins and/or parasite_genes (see the following method).An in-house pipeline consisting of Linux and Python scripts were used to extract parasite protein or gene names from the abstracts of interest (pipeline source code provided: rule_based_protein_extraction). More specifically, the aim is to associate the extracted name with a UniProt ID or NCBI Gene ID.The pipeline uses both parasite_proteins and parasite_genes for searches. parasite_proteins contains the following columns: UniProt ID, Protein name, Alternative name, NCBI Protein ID, NCBI Gene_ID, Gene name, Taxonomy ID; and parasite_genes contains: Taxonomy NCBI Gene_ID, Symbol, LocusTag, Gene Name. Note that ‘Protein name’ and ‘Gene name’ can consist of one or more words.The pipeline incorporates two methods to obtain UniProt ID or NCBI Gene IDs. Steps in Method #1 for each abstract: (1) punctuation ?!:-) are removed; (2) genus or species names from taxonomy_ disease_ link are checked for matches; (3) every ‘Protein name’ from parasite_proteins that is specific to the previously matched genus or species is checked for a match. An exception is that single word protein names matching to common words with English meaning in the abstract are ignored. For example, poorly annotated single names such as ‘protein’, ’antigen’, ‘putative’, ‘raw’, ’dead’, ‘sand’. A Python script included the module ‘enchant’, which provides functionality to check the spelling of words via in-built dictionaries of different languages. Here, when checking for single word protein name, any name in the enchant US dictionary or in ‘Ignored_names  a UniProt ID is recorded for all matches; (5) as per step #3, but each remaining word is checked for a match with NCBI Gene_ID, Symbol, LocusTag in parasite_genes and a NCBI Gene_ID recorded for matches.Steps in Method #2 for each abstract: (1) genus or species names from taxonomy_ disease_ link are checked for matches; (2) punctuation ?!:-) at start and end of words, numbers, words  contained in Ignored_words  was disregarded when determining a match. The pipeline combines the output from both methods to create a single file containing a list of unique UniProt IDs for each PubMed ID (where matched NCBI Gene IDs are mapped to UniProt IDs).The 2,744 abstracts of interest generated by the rule-based approach were further filtered. An abstract was selected if it contained a parasite protein name or gene that is mentioned in more than five other publications and contained more than two ‘protective immunity’ keywords. An in-house Python script applied the thresholds (source code provided: ‘get_training_data.py’). Abstracts that exceeded the > 5 publications and > 2 keyword thresholds formed the positives training data .The negatives training data was obtained by randomly selecting 1556 from 15,358 abstracts  that had a zero matching count to all words contained in the five keyword files . Supplementary Table 85. In brief, the technique starts with a list of words (the BoW) that are considered important. The BoW in this instance contained all common words (minus stop words) from the positives’ training abstracts (1011 words). Then, given the ‘title+abstract’ text as input, the output for the training data was a numerical vector consisting of the frequency of each word from the BoW that occurred in the input e.g., for each ‘title+abstract’ input, the training data had 1011 values (features)+a 0 or 1 representing a negative or positive target. The BoW is listed in Supplementary Table As part of the ML feature extraction process, the title+abstracts associated with the training data PubMed IDs were converted to a vectorised format  using the text vectorization technique, Bag of Words (BoW)86. SVM is a popular ML algorithm for text classification when the amounts of training data are limited87. This algorithm was implemented via the ksvm R function, which is contained in the kernlab package. The function used two arguments: a data frame of numeric variables  and a numerical class vector, i.e., a vector representing the target label, which had two classes: 1 (positive) and 0 (negative). Given input data (a vectorised ‘title+abstract from a publication), the SVM algorithm generates a probability for each class e.g., class positive = 0.75, class negative = 0.25. This means that if the abstract was classified as a positive, then the classification is considered to be 75% likely to be correct. An important caveat, however, is that a class positive equal to 1.0 does not necessarily mean the associated abstract is unquestionably an abstract of interest. Parameters were tuned empirically with the training and validation datasets. All default parameters for the SVM algorithm were used except for kernel = "rbfdot" and type = "C-svc".A pipeline comprising Linux and R scripts performed a binary classification of abstracts i.e., two classes: abstracts of interest  and abstracts to be ignored. The ML algorithm used to build the predictive models was support vector machines (SVM)An additional pipeline using the same ksvm R function was created to perform tenfold cross validation. A threshold of 0.5 was applied to the predicted probabilities, whereby a true positive is a probability >  = 0.5 and a false positive is a probability < 0.5 for expected positives, and similarly, a true negative is <  = 0.5 and a false negative is > 0.5 for expected negatives. The measures used to evaluated the SVM model’s predictive performance were accuracy, error rate (misclassification), true positive rate (sensitivity), false positive rate, true negative rate (specificity), precision ; where the tags are protein and gene names extracted from the training data using the rule-based approach (source code: ‘get_bio_words.py’), and the sentences for tagging are those only containing a protein or gene name (obtained with source code: ‘get_sentences.py’).The program spaCy 3.1 was used to build a custom named entity recognition (NER) model. The first step was to convert the abstracts previously used for the ML training positives into an input format suitable for spaCy. This format comprises tagged sentences and the tagging style used was BIO  e.g., three labels  are used to define the required entity boundaries. For this study, the required entities are protein and gene names e.g., The [O] protein [O] name [O] is [O] The next step was to convert the BIO formatted sentences (‘bio_format.iob’) into a spaCy binary format: python -m spacy convert -c iob bio_format.iob/train (use ‘python -m spacy convert –help’ for details on syntax). To create a trained NER model: python -m spacy train acc_config.cfg—output/train/output (use ‘python -m spacy train—help’ for details). Note that the computational time to build the model is multiple hours e.g., over 24 h to model 5099 lines of BIO formatted sentences using the study’s computer platform. The built NER model was named ‘custom_NER_model’ and is provided.The final step is a Python script (source code provided: ‘NER_protein_extract.py’) that takes ‘title+abstract’ per PubMed ID as input. The script uses the module ‘spacy’ to load the custom NER model, which determines the entities within the input text. Each entity is checked for a match in parasite_proteins and parasite_genes. The output file contains a list of matched protein and gene names for each PubMed ID.88. In this study, the protein is the equivalent of the author. For example, a protein name is extracted from seven publications. The number of citations for each of the seven publications in descending order is 130, 85, 24, 15, 9, 4, and 1. In this example, the protein has an h-index of 5, meaning there are five publications with more than 5 citations i.e., the protein does not have six publications with 6 or more citations. The h-index is insensitive to the highly cited work89 e.g.; the h-index does not increase if the top five cited publications in the previous example continue to be cited. The ‘g-index’ allocates more weight to highly cited papers. The g-index is defined as ‘the largest number such that the top ‘g’ publications received together at least g2 citations90 e.g., the g-index for previous example would be 7, since the sum of the seven citations (268) is more than 49 (i.e. 72).The h-index  is a leading author-level research metric, where ‘h’ denotes the number of papers having at least an ‘h’ number of citationsThe column heading names of the 96 characteristics collected or predicted are listed in the ReadMe file of Supplementary Table 91; [24] TMs predicted by TMHMM (2.0)92; [25–27] TM domains from UniProtKB; [28] SP predicted by Phobius (1.0); [29] SP predicted by TargetP (2.0)93;  SP and cleavage site predicted by signalP (5.0)94; [32] SP from UniProtKB; [33] Vacceed33; [34–36] GO terms from UniProtKB;  PredGPI (version 1.0)95. Note GPI FPrate < 0.001 is highly probable, < 0.005 is probable, < 0.01 is weakly probable, and >  = 0.01 is not GPI-anchored; [39] Subcellular location from UniProtKB; [40–44] Immune Epitope Database and Analysis Resource (IEDB)96; [45–96] all taken from UniProtKB.[1] UniProt ID from UniProtKB; [2] Entry_name from UniProtKB; [3] Cluster # from study’s CD-HIT analysis; [4] Representative Protein Name is the majority protein name in Cluster #; [5] Warning (0 or 1) where ‘1’ indicates a questionable protein were the h-index is less than or equal to one and its associated publication is more than 5 years old; [6] accessibility score calculated in-house  and 8322 T. gondii (strain ME49) proteins were downloaded in a FASTA format from PlasmoDB (release 47) and ToxoDB (release 47), respectively, which are database members of Eukaryotic Pathogen Databases (EuPathDB)97. All 3706 currently available protein sequences for B. bovis T2Bo were downloaded in a FASTA format from PiroPlasmaDB (release 47), which is also a database member of EuPathDB.All 5460 https://github.com/goodswen/vacceed/releases. The original training data was replaced with positives formed from the 485 proteins identified in this study as potential candidates, and negatives formed from 485 proteins with known and/or predicted cell interior locations such as the cytoplasm or nucleus—see Supplementary Table species_name/pipeline/evidence/training_files, where ‘species_name’ is a user-defined name for the target species such as plasmodium, toxoplasma, or babesia. The new ‘train_profiles’ is provided via GitHub: https://github.com/goodswen/abstract_classification.Vacceed was downloaded from Note that many of the training proteins were also proteins in the three apicomplexan species. In such cases, their probabilities were independently determined by running Vacceed with a training dataset that excluded the training protein for which the exposure probability was sought e.g., Vacceed in effect was executed with only one input training protein sequence at a time whereby the same protein was removed from the training data.https://www.uniprot.org/help/annotation_score: last viewed October 2021). UniProtKB assign an annotation score from one to five to every protein, where five is considered the best-annotated entry . With an understanding UniProtKB annotation scores are only a guideline of annotation quality, we checked scores for all 5460 P. falciparum (strain 3D7), 8322 T. gondii (strain ME49), and 3706 B. bovis T2Bo proteins . However, this is not an indication of the accuracy or correctness of the protein sequence. Considering all P. falciparum proteins: 72.4% predicted, 22.5% inferred from homology, 0.1% with experimental evidence at transcript level, and 4.9% with experimental evidence at protein level; T. gondii: 78.5% predicted, 21.3% inferred from homology, 0.02% with experimental evidence at transcript level, and 0.13% with experimental evidence at protein level; and B. bovis: 57.0% predicted, 20.4% inferred from homology, 22.4% with experimental evidence at transcript level, and 0.13% with experimental evidence at protein level (see Supplementary Table UniProtKB also indicates the type of evidence that supports the existence of the protein (see Supplementary Information 1.Supplementary Information 2.Supplementary Information 3.Supplementary Information 4.Supplementary Information 5.Supplementary Information 6.Supplementary Information 7.Supplementary Information 8.Supplementary Information 9.Supplementary Information 10.Supplementary Information 11.


Protein aggregation is associated with many human disorders and constitutes a major bottleneck for producing therapeutic proteins. Our knowledge of the human protein structures repertoire has dramatically increased with the recent development of the AlphaFold (AF) deep-learning method. This structural information can be used to understand better protein aggregation properties and the rational design of protein solubility. This article uses the Aggrescan3D (A3D) tool to compute the structure-based aggregation predictions for the human proteome and make the predictions available in a database form. In the A3D database, we analyze the AF-predicted human protein structures (for over 20.5 thousand unique Uniprot IDs) in terms of their aggregation properties using the A3D tool. Each entry of the A3D database provides a detailed analysis of the structure-based aggregation propensity computed with A3D. The A3D database implements simple but useful graphical tools for visualizing and interpreting protein structure datasets. It also enables testing the influence of user-selected mutations on protein solubility and stability, all integrated into a user-friendly interface.http://biocomp.chem.uw.edu.pl/A3D2/hproteome. The data underlying this article are available in the article and in its online supplementary material.A3D database is freely available at: Bioinformatics online. In July 2021, a database of highly accurate structure predictions for the human proteome was published  Database by computing the aggregation propensity of the human protein models from the AF database. The A3D is a structure-based predictor of surface-exposed aggregation-prone regions. The A3D algorithm exploits the information of 3D atomic models to compute the structurally corrected aggregation values (A3D score) for each amino acid , to predThe A3D database integrates A3D analysis for 23391 predicted structures of the human proteome from the AF database. The content of the A3D database can be queried by UniProt ID, Gene or protein name  [PID2019-105017RB-I00 to S.V.], by ICREA, ICREA-Academia 2019, and by EU [PhasAge/H2020-WIDESPREAD-2020-5 to S.V.]; the Spanish Ministry of Science and Innovation with a Juan de la Cierva Incorporacion [IJC2019-041039-I to J.G.-P.]; S.K. acknowledges funding from the National Science Centre, Poland [2020/39/B/NZ2/01301].Conflict of Interest: none declared.btac215_Supplementary_DataClick here for additional data file.