The spectacular recent advances in protein and protein complex structure prediction hold promise for reconstructing interactomes at large-scale and residue resolution. Beyond determining the 3D arrangement of interacting partners, modeling approaches should be able to unravel the impact of sequence variations on the strength of the association.In this work, we report on Deep Local Analysis, a novel and efficient deep learning framework that relies on a strikingly simple deconstruction of protein interfaces into small locally oriented residue-centered cubes and on 3D convolutions recognizing patterns within cubes. Merely based on the two cubes associated with the wild-type and the mutant residues, DLA accurately estimates the binding affinity change for the associated complexes. It achieves a Pearson correlation coefficient of 0.735 on about 400 mutations on unseen complexes. Its generalization capability on blind datasets of complexes is higher than the state-of-the-art methods. We show that taking into account the evolutionary constraints on residues contributes to predictions. We also discuss the influence of conformational variability on performance. Beyond the predictive power on the effects of mutations, DLA is a general framework for transferring the knowledge gained from the available non-redundant set of complex protein structures to various tasks. For instance, given a single partially masked cube, it recovers the identity and physicochemical class of the central residue. Given an ensemble of cubes representing an interface, it predicts the function of the complex.http://gitlab.lcqb.upmc.fr/DLA/DLA.git.Source code and models are available at  The ever-growing number of sequenced individual genomes and the possibility of obtaining high-resolution 3D structural coverage of the corresponding proteomes  open up omplexes  -mutation’, the first deep learning architecture estimating mutation-induced ervision . It reliervision . In thiservision  and combormation . DLA-mutoperties . DLA-mutWe represent a protein–protein interface as a set of locally oriented cubic volumetric maps centered around each interfacial residue . The locproteins   and different choices for the atom . Its ability to recover the identity of the central residue in the input cube can inform us about the extent to which an interfacial residue’s 3D environment is specific to its amino acid type or physicochemical properties. To investigate this possibility, we analyzed the probability vectors computed by ssDLA when given a partially masked cube as input .To avoid any amino acid-specific bias, we masked a volume of constant shape and size, namely a sphere of radius 5 Å, in all training samples  descriptors. We performed an ablation study to assess the contribution of these descriptors and of the pre-training step . We found that the wild-type residue’s buriedness (CV) and interface propensity (PC) contributed very little to the accuracy of the predictions (TJET and SR-GEMME). Removing them is essentially harmless. In contrast, evolutionary information does significantly contribute to the model’s performance, as attested by the rather low PCC (0.648) obtained when using only SR ΔΔGBind pMPI v1.0 . We founMPI v1.0 .In absolute terms, DLA-mutation’s performance is moderate in these two experiments (PCC below 0.5). In the first case, we may interpret the difficulty faced for generalizing from an older to a more recent version of SKEMPI as reflecting differences in the way these two versions were compiled and in the quality of their data (see Section 5). The balance of mutation types in the train and test sets may also play an important role. While the substitutions to alanine represent less than half of the mutations in the train set (SKEMPI v1.0), they amount to about 75% in the test set (SKEMPI v2.0). The substitutions to alanine also represent a large proportion (83 out of 112) of the mutations in the second experiment’s test set.The location of a mutation in a protein interface might be a relevant indicator for the confidence in the estimation. We investigated this issue by describing an interface as three concentric layers of residues, the support , the core (the second layer), and the rim  . DLA-mutish them . Howeverish them . DLA-mutomplexes . Finallyomplexes .To evaluate the embedding vectors computed by the pre-trained ssDLA, we tested whether they could be mapped to per-residue and per-interaction physicochemical and functional properties. To do so, we added a two-layer fully connected network on top of ssDLA’s architecture , and we Knowledge acquisition and transfer from protein–protein interfaces with deep learning approaches are useful to address the fundamental questions about protein–protein interactions. Our approach leverages the non-redundant set of experimentally resolved protein complex structures to assess the impact of mutations on protein–protein binding affinity, among other applications. Compared with other state-of-the-art predictors, DLA-mutation generalizes better to unseen complexes.Despite the improvement over the state-of-the-art, the DLA-mutation generalization capability from the first to the second version of SKEMPI remains limited. This result likely reflects differences in the protocols employed to produce, collect and manually curate the data between the older version, released in 2012, and the new one, released 7 years later. The experimental methods used for measuring the binding affinities are not reported in the first version; therefore the reliable entries cannot be selected. Moreover, various strict checks with up-to-date references were applied for the second version to ensure its quality. In general, chniques .Future work will more thoroughly investigate the contribution of conformational sampling and the quality of the btad231_Supplementary_DataClick here for additional data file.


Pseudomonas putida (P. putida) as a method of bioremediation to mitigate the deleterious effects of pollution attributable to Pb and Cd. The use of various computational approaches, such as physicochemical assessments, structural modeling, molecular docking, and protein–protein interaction investigations, has enabled us to successfully identify the exceptional metal-binding properties that metallothionein displays in P. putida. The identification of specific amino acid residues, namely GLU30 and GLN21, is crucial in understanding their pivotal role in facilitating the coordination of lead and cadmium. In addition, post-translational modifications present opportunities for augmenting the capacity to bind metals, thereby creating possibilities for focused engineering. The intricate web of interactions among proteins serves to emphasize the protein’s participation in essential cellular mechanisms, thereby emphasizing its potential contributions to detoxification pathways. The present study establishes a strong basis for forthcoming experimental inquiries, offering potential novel approaches in bioremediation to tackle the issue of heavy metal contamination. Metallothionein from P. putida presents a highly encouraging potential as a viable remedy for environmental remediation, as it is capable of proficiently alleviating the detrimental consequences related to heavy metal pollution.Heavy metals, specifically cadmium (Cd) and lead (Pb), contaminating water bodies of Madinah (Saudi Arabia), is a significant environmental concern that necessitates prompt action. Madinah is exposed to toxic metals from multiple sources, such as tobacco, fresh and canned foods, and industrial activities. This influx of toxic metals presents potential hazards to both human health and the surrounding environment. The aim of this study is to explore the viability of utilizing metallothionein from  P. putida. These safeguarding measures could potentially play an important part in limiting the adverse impacts of pollution caused by heavy metals [The contamination of heavy metals, specifically Cd and Pb, in water bodies is posing significant challenges for the environment in Madinah, Saudi Arabia, . Cd, a wy metals ,7,8. Heay metals . Toxic my metals . Childrey metals . The expy metals ,13. Aquay metals ,15. Longy metals ,17.P. putida is a rod-shaped, Gram-negative bacterium that is frequently found in contaminated water bodies, vegetation, and soil. The adaptability of P. putida is clear in its opportunistic utilization of a varied range of nutrients, its speedy rate of growth, and its capacity to endure oxidative stress and exposure to harmful substances. The analysis of P. putida and its potential to degrade xenobiotics commenced in the 1960s and has subsequently been a voyage to expose its hereditary, biochemical, and physiological traits [P. putida’s demonstrated metal and antibiotic resistance properties make it an intriguing candidate for further exploration, particularly in the context of bioremediation [P. putida, enabling systems-level profiling and targeted genetic and genomic modifications. These advances have broadened the prospects for industrial applications, leveraging the bacterium’s biochemical capabilities and the expanding knowledge and technology surrounding it. This study intends to illuminate the genetic and metabolic components of P. putida’s bioremediation abilities and create a path for focused efforts to alleviate the detrimental impacts of lead and cadmium contamination.l traits . P. putiediation ,21. ThesP. putida is a bacterium that is very frequently employed and has been extensively researched due to its ability to break down and eliminate different pollutants, making it an exceptionally auspicious candidate for removing heavy metals [The deployment of efficient and sustainable techniques for the detoxification of heavy metals remains crucial in addressing the detrimental effects associated with lead and cadmium . P. putiy metals ,24,25.P. putida to effectively eliminate these hazardous metals from polluted environments. By means of computational modeling and bioinformatics tools, our aim is to reveal the subtle interactions between P. putida and heavy metals, identify critical pathways and enzymes involved in the bioremediation process, and ultimately contribute to the creation of sustainable strategies for heavy metal detoxification. This study makes a valuable contribution to the expanding domain of in silico bioremediation and offers potential for the advancement of ecologically sustainable strategies to address the issue of heavy metal contamination.This research provides a significant contribution to the expanding field of in silico bioremediation and has the potential to advance ecologically sustainable approaches to address the issue of heavy metal contamination. Through the utilization of in silico methodologies, we scrutinize the underlying mechanisms that enable P. putida strain 15420352, accession ID: QKL07673, retrieved from the National Center for Biotechnology Information , was subjected to analysis using the EXpasy ProtParam tool . This computational tool computes a range of physicochemical parameters that provide insights into the structure and functional characteristics of the protein. The factors considered in this study encompassed the molecular weight, theoretical isoelectric point (pI), the proportion of negatively and positively charged residues, estimated half-life, instability index (II), aliphatic index, and grand average of hydropathicity (GRAVY) value [The protein sequence of metallothionein from Y) value .P. putida. The VicmPred  algorithm employs pattern-based methodologies for the purpose of predicting functional classifications that are linked to a given protein. Our objective was to elucidate the potential functional roles of metallothionein in relation to its binding capabilities with lead and cadmium by inputting the protein sequence [The functional prediction tool, VicmPred, was utilized to perform a comprehensive study of the metallothionein sequence derived from sequence .P. putida was subjected to structural classification using the Superfamily 1.75 tool (https://supfam.org/SUPERFAMILY/ accessed on 2 February 2023). This tool identifies conserved domains and classifies proteins into structural classes, folds, and superfamilies based on their sequence features [The protein sequence of metallothionein from features .P. putida was analyzed for post-translational modifications (PTMs) using the MusiteDeep tool (https://www.musite.net/ accessed on 2 February 2023). The bioinformatics tool shown herein utilizes sequence characteristics to identify potential post-translational modification (PTM) sites and assign corresponding scores. This tool provides valuable insights into the potential regulatory alterations that may impact protein functionality [The protein sequence of metallothionein from ionality .P. putida (UniProt ID: Q88HU1) was submitted to computational analysis utilizing the EggNOG 6.0 database. EggNOG (http://eggnog6.embl.de/ accessed on 3 February 2023) offers valuable insights pertaining to the functional annotations, domains, and evolutionary relationships of proteins [In this study, the protein sequence of metallothionein derived from proteins .P. putida was subjected to computational analysis utilizing the STRING database (https://string-db.org/ accessed on 6 February 2023). The STRING database offers significant insights into protein–protein interactions (PPIs), molecular activities, biological processes, and linked protein domains [The metallothionein sequence derived from  domains .https://predictprotein.org/ accessed on 7 February 2023), a reliable platform for performing computational protein analysis. This complete tool enables the anticipation of secondary structure, solvent accessibility, topology, disordered regions, relative B-value dynamics, binding interactions, disulfide bond formation, and conservation scores. The methodology employed in this study consisted of loading the protein sequence into the PredictProtein software, and afterward, analyzing and interpreting the predictions given by the program [In order to assess the binding capacity of the metallothionein derived from Pseudomonas putida, we utilized the PredictProtein tool , Robetta (http://robetta.bakerlab.org/ accessed on 9 February 2023), ModWeb (https://modbase.compbio.ucsf.edu/modweb/ accessed on 9 February 2023), and SwissModel (https://swissmodel.expasy.org/ accessed on 9 February 2023), were utilized for this purpose. First, the primary amino acid sequence of P. putida metallothionein was determined. This sequence was then submitted to the aforementioned four homology modeling tools. The PHYRE2 Protein Fold Recognition Server is a web-based application that predicts protein structure using a combination of profile alignment and secondary structure prediction [https://proq3.bioinfo.se/pred/ accessed on 15 February 2023) [https://montelionelab.chem.rpi.edu/PSVS/psvs2/ accessed on 20 February 2023). These tools were employed to assess the quality and reliability of the generated protein structures.In this study, we intended to conduct homology modeling of the ediction . Robettaediction . The Modediction . SwissMoediction . The modry 2023)  and the For the purpose of investigating the binding interactions between heavy metals and the metallothionein (MT) modeled structure, molecular docking was conducted using CDocker, a module available within the Discovery Studio software suite . The procedure involved the preparation of the protein structure, ligands, and the definition of a binding sphere around the active site. The ligands employed for the simulations were lead and cadmium, utilizing their respective crystallographic structures. Prior to docking, the ligand structures were optimized through the addition of hydrogen atoms and the adjustment of charges using the tools provided by Discovery Studio. Using the CDocker module of Discovery Studio, the docking of heavy metals and the modeled metallothionein structure was executed. The procedure consisted of several essential steps. The binding site was initially defined using the “Input Site Sphere” parameter, with center coordinates of  and a radius of 32.941 Å. This specified the region within which the calculations for docking would be performed. For the simulations of docking, a set of parameters is configured. The “Top Hits” parameter was set to 10, indicating that the top 10 binding poses for the heavy metal–ligand would be taken into account. To facilitate analysis, a “Pose Cluster Radius” of 0.1 was applied to group similar poses together. To facilitate the investigation of ligand binding modes, “Random Conformations” were generated for the ligand, with ten conformations considered. The “Dynamics Steps” parameter was subsequently set to 1000 and the “Dynamics Target Temperature” parameter was set to 1000 K to enable dynamic movements during the docking procedure. Enabling “Include Electrostatic Interactions” accounted for electrostatic interactions. This ensured that electrostatic forces played a role in the interactions between the ligand and the receptor during docking. Using the “Orientations to Refine” parameter, ten different orientations of the ligand were refined, while the “Maximum Bad Orientations” parameter set a maximum of 800 orientations that did not meet the energy criteria defined by the “Orientation vdW Energy Threshold” of 300 kcal/mol. This refined the selection process for orientation. During docking, the “Simulated Annealing” method was utilized, with “Heating Steps” set to 2000 and “Heating Target Temperature” set to 700 K to facilitate a gradual increase in temperature. To stabilize the binding poses, “Cooling Steps” were set to 5000, and the “Cooling Target Temperature” was set to 300 K. Advanced settings included the use of the CHARMM forcefield and selection of the “Ligand Partial Charge Method” as Momany-Rone. To control the amount of potential energy used during the calculations, the “Use Full Potential” parameter was set to False. Following the docking simulations, the resulting ligand–receptor complexes were subjected to final energy minimization using the full potential with a gradient tolerance of 0 for precision. A grid extension of 8 Å was applied around the binding site of the receptor to account for the flexibility of the ligand binding. The docked complexes were evaluated for close intramolecular interactions.P. putida yielded a value of approximately 7907.74 Da. The compact dimensions of this size offer notable benefits in the context of bioremediation applications [The determination of the molecular weight of metallothionein from ications . The isoications . The MT ications . Negativications ,41. The ications . The proications . The proications .P. putida. These scores reflect the protein’s potential roles in various biological functions, shedding light on its multifaceted nature. Cellular Process (Score: 0.92678171): The high score suggests the involvement of metallothionein in critical cellular processes. Information Molecule (Score: −0.13593004): The negative score implies a lower association with information-related functions. Metabolism (Score: 0.99999996): The nearly perfect score highlights the potential participation of metallothionein in metabolic processes. Virulence Factors (Score: −0.57314452): The negative score suggests a relatively lower likelihood of the metallothionein’s involvement in virulence-related functions. This aligns with its potential role in detoxification rather than pathogenicity.The VicmPred analysis revealed distinctive functional class scores for the metallothionein from The protein is categorized as a member of the “Small proteins” class, which denotes its somewhat condensed size. This property is consistent with the concept that metallothionein often possess a compact size, which facilitates their effective coordination of metal ions within their constrained structural environment . The proP. putida. Position 2 (N): Asparagine at position 2 showed a low N-linked glycosylation score of 0.034, which may influence the protein’s structure and function [The in silico PTM analysis using MusiteDeep unveiled various modifications in the metallothionein from function . Positiofunction ,49. Posifunction . Positiofunction . PositioP. putida were determined by the EggNOG analysis. The existence of conserved domains associated with metal binding provides more evidence for its involvement in the processes of metal detoxification and bioremediation, as presented in The categorization and putative functional annotations within the protein family of metallothionein from P. putida yielded significant findings pertaining to the protein’s interactions, functions, and putative molecular pathways linked to its involvement in metal binding and bioremediation. The study yielded a total of 11 nodes, which represent discrete proteins or protein components that may be involved in interactions with metallothionein. The mentioned nodes serve as representations of prospective partners or molecules that have the capacity to participate in activities linked with metallothionein. The network displays a total of 15 edges, which symbolize the various connections or interactions among the nodes. The edges depicted in the diagram represent the plausible physical or functional connections between metallothionein and other proteins, within the framework of its biological functions. The mean node degree of 2.73 signifies that, on average, every node (representing a protein) inside the network is linked to around 2.73 additional nodes. This metric offers valuable insights into the degree of interaction complexity and potential functional associations between metallothionein and other proteins. The metallothionein network displays a significant degree of clustering, which is substantiated by its mean local clustering coefficient of 0.889. This suggests that proteins affiliated with metallothionein are more prone to forming interconnected relationships with one another. This observation indicates the existence of functionally interconnected protein clusters inside the network. The anticipated count of edges, approximated to be 10, functions as a benchmark for evaluating the network’s relevance. When the count of edges observed in a network far exceeds the anticipated quantity, it signifies the existence of significant interactions within the system. The p-value of PPI enrichment, which stands at 0.0968, serves as a quantitative measure of the statistical significance that is linked to the protein–protein interactions that have been observed. The molecular function of DNA binding implies that metallothionein may have a role in DNA interaction, potentially contributing to gene regulation or other activities related to DNA. The identified protein domains and features imply that metallothionein possesses several distinct structural and functional characteristics: MerR Transcriptional Regulator, MerR HTH Domain Type, and Superfamily of Putative DNA-Binding Domains, as shown in The utilization of the STRING database for the in silico examination of the metallothionein sequence derived from The insightful PredictProtein analysis brought to light the occurrence of not one but two sublime alpha-helices , and a solitary beta-strand (Residues: 27) adorning the metallothionein sequence. These secondary structure elements contribute to the protein’s overall fold and potential metal-binding sites. The analysis revealed that numerous residues, specifically residues 6, 8, 11, 13, 28, 29, 32, 33, 47, 58, and 62, are evidently exposed to the solvent. This suggests that these residues could potentially be involved in various interactions, such as metal binding. Residues with high B-values  suggest increased flexibility in these regions. This flexibility might be crucial for accommodating bound metal ions and undergoing conformational changes upon metal binding. Several significant regions that are conserved have been detected, which are located at residues 1, 6–8, 11, 13, 27–29, 31–33, 35, 36, 39, 42, 47, 49, 58, 59, 61, 64, 65, and 67–73, as shown in P. putida was modeled using four distinct homology modeling techniques. The validity of the predicted structures derived from PHYRE2, Robetta, ModWeb, and SwissModel was evaluated and compared. The degree of structural similarity among the models produced by PHYRE2, Robetta, ModWeb, and SwissModel varied. The overall fold and secondary structure elements of the models were generally consistent. However, there were variations in the loop regions and local structural characteristics, as shown in The metallothionein sequence from The protein structures generated by Phyre2 and Robetta covered the complete range of residues, spanning 1 to 74. On the other hand, both ModWeb and SwissModel generated structures comprising residues 1 to 73. The ProQ3 scores, which reflect the predicted global model quality, reveal differences among the computational methods. Robetta achieved the highest ProQ3 score of 0.467, indicating greater confidence in its structural predictions. Phyre2 followed with a score of 0.383, while ModWeb recorded a ProQ3 score of 0.000, raising concerns about the reliability of its model. The Ramachandran plot analysis evaluated the backbone torsion angles’ conformations, highlighting their agreement with preferred regions of the plot. Phyre2: 84.1% of residues fall within the most favored region, with 12.7% in additionally allowed regions and 3.2% in generously allowed regions. Robetta outperformed other methods, with 95.4% of residues in the most favored region, showcasing its superior prediction accuracy. ModWeb and SwissModel attained 87.1% and 75.8% in the most favored region, respectively. Close contacts and deviations from the ideal geometry provide insights into potential steric clashes and structural distortions. All methods demonstrated no close contacts within a 2.2 Å distance, suggesting the absence of severe steric clashes. Robetta and SwissModel displayed the lowest RMS deviations for bond angles and bond lengths , indicating better agreement with the ideal geometries. The global quality scores offer a comprehensive assessment of various aspects of the protein models. The Verify3D scores were lowest for Robetta (−5.78) and highest for ModWeb (−6.90), suggesting reasonable agreement between the models and the expected 3D protein structures. The ProsaII scores were negative for all methods, with Robetta, Phyre2, and SwissModel exhibiting relatively close values, while ModWeb had a positive score, possibly indicating deviations from the native-like conformations. Procheck analysis highlighted variations in the phi-psi angles and overall structure quality. Robetta displayed favorable results, while ModWeb and SwissModel exhibited negative scores, suggesting potential issues with their phi-psi angles. The MolProbity Clash-score indicates the presence of steric clashes, with lower scores indicating fewer clashes. Phyre2 displayed a substantially negative Clash-score (−19.42), suggesting minimal steric clashes. Robetta, ModWeb, and SwissModel all had positive Clash-scores, implying varying degrees of steric clashes .The molecular docking simulations provided insights into the binding characteristics of heavy metals lead and cadmium within the metallothionein (MT) modeled structure’s active site. In the case of lead binding, the docking revealed a CDocker energy of −0.10587, indicating a favorable binding affinity. This was corroborated by a corresponding CDocker Interaction energy of −0.10587, highlighting the stability of the lead–MT complex. Notably, the lead established two significant interactions: an electrostatic interaction with GLU30:OE2, situated at a distance of 4.76631 Å, and a metal–acceptor interaction with GLU30:O, characterized by a proximity of 2.13637 Å. Cadmium’s docking within the binding sphere exhibited a more substantial CDocker energy of −30.735, denoting a robust binding interaction. The CDocker Interaction energy was calculated as −17.013, indicating a substantial interaction strength. Cadmium’s interaction was marked by a single metal–acceptor interaction with GLN21:O, occurring at an impressively close distance of 1.59521 Å, as shown in P. putida and assess its physicochemical characteristics, with the intent of gaining comprehension of its ability to bind lead and cadmium. The physicochemical features of metallothionein from P. putida were thoroughly analyzed, revealing its considerable potential for effectively binding metals and its use in bioremediation. The protein’s low molecular weight [P. putida, have been reported, and therefore, it is crucial to find its virulence [The matter of metal contamination, particularly with respect to the existence of lead and cadmium, is a vital ecological apprehension that warrants expeditious consideration. Metallothioneins (MTs) pertain to a classification of condensed proteins that are discernible by their elevated concentration of cysteine. This particular feature endows them with the ability to proficiently bind and sequester metal ions. These proteins play an indispensable role in facilitating the process of metal ion binding and sequestration. The distinctive characteristics of MTs render them highly favorable instruments for the implementation of bioremediation approaches. The principal aim of this inquiry was to scrutinize the metallothionein that was extracted from r weight , presencr weight ,41, and r weight . Metal airulence ,53. VicmP. putida for the purpose of binding lead and cadmium in bioremediation endeavors. The metallothionein from P. putida was analyzed using the Superfamily 1.75 tool, yielding notable results in terms of its structural classification. The designation of this molecule as a “Small protein” is based on its compact configuration, which confers benefits in terms of effective metal coordination. The categories of the “Metallothionein” fold and “Metallothionein” superfamily correspond to the principal role of the protein in binding metal ions, particularly heavy metals, such as lead and cadmium [P. putida has undergone structural optimization to enhance its metal binding capabilities, hence indicating its potential utility in bioremediation endeavors. By leveraging its metal-binding properties, this protein exhibits promising potential for employment in the sequestration and immobilization of harmful metals in contaminated habitats, thereby rendering a valuable contribution to the pursuit of environmental remediation [P. putida via computational investigation presents a fresh perspective on the plausible regulatory mechanisms that govern its capacity to bind metals. The identification of various PTMs, including phosphorylation, glycosylation, and palmitoylation, implies the existence of a complex regulatory network that may affect the protein’s interactions with lead and cadmium ions [P. putida. The conserved domains associated with metal binding reaffirm its capacity to interact with heavy metals, such as Pb and Cd. The presented insights serve as a fundamental basis for conducting additional experimental investigations, with the goal of substantiating its metal-binding proficiency and investigating its potential for bioremediation implementations. The network’s observed PPIs underscore the intricate interdependence between proteins and pathways linked to the metallothionein from P. putida. The presence of 11 nodes and 15 edges suggests a multifaceted network in which the metallothionein might collaborate with other proteins to execute essential biological functions. While the PPI enrichment p-value of 0.0968 falls just outside the conventional significance threshold of 0.05, the trend toward enriched interactions implies a biologically relevant network that warrants further investigation [The present in silico investigation offers significant insights regarding the appropriateness of metallothionein derived from  cadmium . The afoediation ,56. The ium ions . The posium ions  and the tigation . The caltigation . The biotigation  motif frtigation . The “DNtigation . The in tigation .P. putida. The variations among the models generated by PHYRE2, Robetta, ModWeb, and SwissModel can be attributed to disparities in the algorithms and databases employed by each tool. Additionally, the availability and quality of template structures may influence the content of the models [P. putida.Homology modeling is an indispensable technique for forecasting protein structure in the dearth of empirical evidence . In thise models . The dise models . These mThe validation results reveal notable differences among the computational methods in predicting the protein structure’s accuracy and reliability. Robetta stands out with the highest ProQ3 score and the most residues in the most favored region of the Ramachandran plot, indicating its superior accuracy and better agreement with known protein structures. The absence of close contacts and lower RMS deviations for bond angles and lengths further support the robustness of Robetta’s predictions. Phyre2, while demonstrating good overall performance, lags slightly behind Robetta, especially in terms of Ramachandran plot analysis and RMS deviations. However, it possesses the lowest steric clashes. ModWeb’s ProQ3 score of 0.000 raises concerns about the quality of its generated structure, potentially indicating inaccuracies or structural distortions. The positive ProsaII score and unfavorable Procheck results suggest deviations from native-like conformations. SwissModel’s performance, despite generating relatively accurate structures, falls short in certain aspects, such as the Ramachandran plot and Phi-Psi angles’ assessment. The MolProbity Clashs-core aligns with other findings, indicating the presence of steric clashes in ModWeb, SwissModel, and, to a lesser extent, Robetta. Although Robetta has exceptional performance in multiple validation criteria, Phyre2 surpasses it in terms of ProQ3 scores, Ramachandran plot conformation, and a notably lower MolProbity Clash-score. These factors establish Phyre2 as a strong competitor for the top model. The capacity of Phyre2 to produce dependable protein structures is demonstrated by its precise modeling of the complete length of residues and its effective avoidance of steric conflicts.P. putida, as revealed by Superfamily 1.75 analysis, exhibits a compact size that is consistent with its categorization as a “Small protein”. This size confers unique advantages, allowing the protein to efficiently coordinate metal ions within its confined structural environment. Additionally, the categorization of the protein under the “Metallothionein” fold and “Metallothionein” superfamily underscores its central role in binding and sequestering metal ions [The metallothionein from tal ions . This sttal ions . The docP. putida in combatting the critical issue of heavy metal contamination. Through an extensive analysis that encompassed physicochemical exploration, structural evaluation, molecular docking, and scrutiny of protein–protein interactions, a robust basis has been established to support its application in bioremediation. The protein’s compact structure, enriched cysteine composition, and strategic arrangement of negatively charged residues synergistically equip it with a remarkable capacity to bind and sequester heavy metal ions, with particular emphasis on lead and cadmium. Key interacting residues come to the forefront in molecular docking simulations: the electrostatic association involving GLU30:OE2 at 4.76631 Å, along with the metal–acceptor interaction featuring GLU30:O at 2.13637 Å, distinctly underscore the centrality of GLU30 in steering lead binding. Similarly, the cadmium docking interactions highlight the indispensable contribution of GLN21:O at an impressive proximity of 1.59521 Å, thus underscoring the pivotal role of this residue in cadmium coordination. Furthermore, insight into potential avenues for enhancing metal-binding capacity is offered by post-translational modification analysis, opening doors for targeted engineering. The intricate network of protein–protein interactions accentuate its involvement in critical cellular processes, amplifying its potential roles in detoxification pathways. This study bridges theoretical analyses with practical applications, paving the way for future research, experimental endeavors, and innovative bioremediation techniques. Metallothionein derived from P. putida arises as a promising candidate for environmental remediation, poised to exert a significant influence in mitigating the negative effects of heavy metal pollution.In conclusion, this study illuminates the promising potential of metallothionein derived from


Human coronaviruses like MERS CoV are known to utilize dipeptidyl peptidase 4 (DPP4), apart from angiotensin-converting enzyme 2(ACE2) as a potential co-receptor for viral cell entry. DPP4, the ubiquitous membrane-bound aminopeptidase, is closely associated with elevation of disease severity in comorbidities. In SARS-CoV-2, there is inadequate evidence for combination of spike protein variants with DPP4, and underlying adversity in COVID-19. To elucidate this mechanistic basis, we have investigated interaction of spike protein variants with DPP4 through molecular docking and simulation studies. The possible binding interactions between the receptor binding domain (RBD) of different spike variants of SARS-CoV-2 and DPP4 have been compared with interactions observed in the experimentally determined structure of the complex of MERS-CoV with DPP4. Comparative binding affinity confers that Delta-CoV-2: DPP4 shows close proximity with MERS-CoV:DPP4, as depicted from accessible surface area, radius of gyration and number of hydrogen bonding in the interface. Mutations in the delta variant, L452R and T478K directly participate in DPP4 interaction, enhancing DPP4 binding. E484K in alpha and gamma variants of spike protein is also found to interact with DPP4. Hence, DPP4 interaction with spike protein becomes more suitable due to mutation, especially due to L452R, T478K and E484K. Furthermore, perturbation in the nearby residues Y495, Q474 and Y489 is evident due to L452R, T478K and E484K, respectively. Virulent strains of spike protein are more susceptible to DPP4 interaction and are prone to be victimized in patients due to comorbidities. Our results will aid the rational optimization of DPP4 as a potential therapeutic target to manage COVID-19 disease severity. DPP4 is a serine peptidase of highly conserved type II transmembrane glycoprotein, which comprises of a 6-residue N–terminal cytoplasmic tail, a 22-amino-acid transmembrane, and extracellular domain . DPP4 isIt is interesting that several transmembrane proteins in the target cell apart from the primary receptor are invariably essential for the viral cell entry ,10,11. TDPP4 is a notable receptor of MERS CoV . The potThe most prevalent comorbidities in SARS CoV-2-infected patients are hypertension and diabetes, followed by cardiovascular and respiratory diseases . InteresStudies on the cell-surface co-factors facilitating the attachment and entry of SARS-CoV-2 are less known. There are few computational studies based on the interaction of the spike protein of SARS CoV-2 with DPP4 through molecular docking studies. This work is an extension of the work by Li et al. 2020 , where telec, Evdw, and EACS) and their average buried surface area. The resulting docked structures are sorted by minimum energy criteria and root mean square deviation (RMSD) clustering. The interface of the docked complex is analysed based on the Cα distance (5Å) between binding partner residues.Docking between the spike protein of SARS CoV-2 in its native form  and human DPP4  has been performed in HADDOCK . The docThe crystal structure of MERS CoV spike protein complexed with human DPP4 (PDB id: 4L72)  is utili+ and Cl−) are added to make the system electro-neutral. The GROMACS 2018.6 program with GROMOS96 53a6 force field [We perform molecular dynamics (MD) simulations of all the systems in explicit water. The Spc216 (Simple Point Charge) water molecule  is used ce field  has beence field . The totce field .S) of wild-type and mutant structures.The DYNAMUT tool  has beenS between two conformations  in terms of their respective sets of eigenvalues is given by Δh represents the Boltzmann constant, where nth normal mode.alpha-Calpha) distance from the perturb site. The ΔQ value is the magnitude of perturbation experienced by each residue and Cα-Cα is the distance from the perturbed site. The perturbation effect can be analysed as a distance connecting the perturb residue to its nearby residues or on the interaction network strength. “Coupling distance” (dc) refers to a measure of the degree of coupling between two residues in a protein structure. This measurement is based on the interaction energy between residues and can provide insight into how changes in a protein’s structure can affect its function. The coupling distance can be used to predict the stability and functionality of a protein.PRS has been performed using the pPerturb server ,44. It aTo predict the specific binding potential of SARS CoV-2 with DPP4, a molecular docking approach with HADDOCK has been followed. HADDOCK uses a combination of experimental data and physical principles to generate models that are both accurate and informative. It has been used to predict the structure of the SARS-CoV-2 spike protein in complex with its receptor, ACE2 , to studA 500 ns long MD simulation is carried out on DPP4 and spike protein variants from MERS CoV and SARS CoV-2 complexes. The equilibrations of the simulated structures are assessed from the RMSD. RMSD is a quantitative measure of the variation of protein complex considering all the non-hydrogen atoms concerning the initial conformation of protein complex along the time of simulation, shown in the We have used DSSP to assign the secondary structural elements to the residues. The secondary structural elements play a vital role in mediating interactions in the binding interface. Studies on the structural analysis and comparison of protein–protein interfaces depict regular secondary structures (helices and strands), which are the main components of the protein homodimer (obligate interface), whereas non-regular structures  frequently mediate interactions in the heterodimeric protein–protein interfaces . SecondaWe characterize the comparative binding affinity of the complex through its radius of gyration (Rg). Rg of a complex measures its compactness. It is calculated as the average distance of the C-alpha atoms from their centre of mass. Rg is computed for every ensemble and the mean value is shown in The contact surface at the binding interface depicted by the solvent-accessible surface area (SASA) is used as a descriptor to measure the relative DPP4 binding strength to various spike proteins. The less the surface area of a biomolecule in a complex that is accessible to a solvent, the larger the binding interface. The mean value of SASA for each of the studied systems is given in H) due to DPP4-spike interactions for each system is tabulated in H, H(nH) is unimodal. An increase in the number of hydrogen bonds between delta SARS CoV-2 and DPP4 denotes stronger interaction between them. A similar observation is found in MERS CoV: DPP4  analysis. RMSF considers the overall magnitude of the fluctuation of each Cα-atom. We have calculated RMSF separately for DPP4 and spike variants. RMSF for DPP4 in all the complexes is almost equal A. OveralPrincipal component analysis (PCA) utilizes a covariance matrix that includes information on the anisotropy of such fluctuations and their correlations between sites. PCA in MD simulation identifies only the essential motion eliminating other rotational and translational movements. PCA analysis for DPP4 is similar in all the ensembles C. Howeve−1 K−1 and −0.728 kcal mol−1 K−1, respectively) leading to a decrease in molecular flexibility. It is found that due to the L452R mutation conventional hydrogen bond between V350 and L452 breaks, and the hydrophobic bond between L452 and L492 also breaks. In the case of T478K, intra-molecular interaction remains the same before and after mutation. On the other hand, E484K mutation for Gamma and Alpha variants elicits a destabilizing effect  on the protein and the vibrational entropy energy values  attributed towards the increase in molecule flexibility. It appears that the E484K mutation causes the formation of one carbon–hydrogen bond between K484 and C488.From the MD simulation study, we can observe that spike protein residues having mutations L452R and T478K in the Delta variant, and E484K in the Gamma and Alpha variants are found to directly participate in DPP4 interaction. Hence, we have chosen such an interacting interface to analyse the effect of mutations on protein structures by calculating the change in vibrational entropy (ΔΔS) and change in folding free energy (ΔΔG) as a thermodynamic state function. We have applied normal mode analysis (NMA) utilizing harmonic motions in a system to provide insights into its dynamics and accessible conformations due to such mutations in the spike protein variants. NMA is a valuable tool for studying protein dynamics and interactions. It is performed by calculating the eigenvalues and eigenvectors of the Hessian matrix of the molecule. The natural frequencies and vibrational modes of the molecule represent the different ways in which it can move ,52. NMA c) of 4.2 ± 0.4 Å, and perturbation analysis of K484 (in case of the Gamma and Alpha variants) reveals a dc of 2.5 ± 0.1 Å , Gamma (E484K) and Alpha (E484K) has been conducted. In the case of the Delta variant, perturbation analysis of R452 results in a coupling distance of 3.3 ± 0.2 Å and K478 results in a coupling distance (d ± 0.1 Å . If a mu ± 0.1 Å  as the m ± 0.1 Å . The maxThe COVID-19 pandemic has become more dangerous as new SARS-CoV-2 variants have appeared in each new wave of infection. The primary functional receptor for SARS-CoV-2 is ACE2, DPP4, has also been suggested to be its potential co-receptor for viral cell entry ,55. To pThe current study elicits that spike protein residues showing mutations like L452R and T478K in the Delta variant, and E484K in the Gamma and Alpha variants directly engages in DPP4 interaction. Bioinformatics techniques based on the protein crystal structures have speculated that the MERS-CoV receptor DDP4 might act as a candidate-binding target of SARS-CoV-2 as well ,7,8. It Overall, our study compared DPP4 interaction with MERS, Wildtype and all five variants of SARS-CoV-2, and identified unique RBD residues crucial for the interaction with the DPP4 receptor. By comparing the predicted interactions between all SARS-CoV-2 variants and DPP4 with those observed in MERS-CoV, such a study seeks to clarify the potential role of DPP4 as a co-receptor for SARS-CoV-2 and provide a basis for further investigation. In a study to predict human proteins likely to interact with SARS-CoV-2 spike protein based on the available crystal structures by interolog mapping, domain–domain interaction, and machine learning approaches ,57,58,59


Two artificial intelligence (AI)-based methods for protein structure prediction, AlphaFold 2 and RoseTTAFold, increase dramatically the quality of structural modeling from sequence, nearing experimental accuracy.Protein language models encode the written language of proteins, allowing for more accurate annotations and predictions than homology-based methods.Most model organisms, neglected disease pathogens, and proteins with curated annotations have models available with varying quality, aiding wet-laboratory experiments targeting single-question issues.Ultrafast alignment tools can traverse the protein space by both sequence and structure to identify remote evolutionary relations previously precluded to older and slower methods.Preliminary analyses of predicted AlphaFold 2 3D-models from 21 model organisms suggest that the majority (>90%) of globular domains in proteins can be assigned to currently characterized domain evolutionary superfamilies. Breakthrough methods in machine learning (ML), protein structure prediction, and novel ultrafast structural aligners are revolutionizing structural biology. Obtaining accurate models of proteins and annotating their functions on a large scale is no longer limited by time and resources. The most recent method to be top ranked by the Critical Assessment of Structure Prediction (CASP) assessment, AlphaFold 2 (AF2), is capable of building structural models with an accuracy comparable to that of experimental structures. Annotations of 3D models are keeping pace with the deposition of the structures due to advancements in protein language models (pLMs) and structural aligners that help validate these transferred annotations. In this review we describe how recent developments in ML for protein science are making large-scale structural bioinformatics available to the general scientific community. The number of experimentally determined, high-resolution structures deposited in the Protein Data BankMost proteins comprise two or more structural domains , that isVarious resources emerged to classify domain structures in evolutionary families and fold groups  ≥ 70 [Homo sapiens), mouse (Mus musculus), rat (Rattus norvegicus), and thale cress  [In this review, we consider recent developments in deep learning, a branch of ML see  operatinDT) ≥ 70 ) for 21 haliana) . We revihomology-based inference (HBI), that is, the transfer of annotations from labeled to sequence-similar yet unlabeled proteins [de novo protein function and structure prediction methods, which have ranked highly for decades in independent evaluations [Comparing a query protein sequence against the growing sequence databases can reveal a goldmine of evolutionary information encoded in related (similar) sequences . Closelyproteins ,37. Beyoluations , 40.. Holuations ,13,41, iluations ,42,43. Wluations ,44, the luations  growth oOne alternative to direct evolutionary information extraction is leveraging deep learning teaching machines to encode information contained in billions of known protein sequences by adapting so-called language models (LMs) from natural language processing (NLP) to learn aspects of the ‘grammar’ of the language of life as encoded in protein sequences , 51.. WhTechnically, this can be achieved by extracting the hidden states from the pLM referred to as embeddings. One key advantage of pLMs over evolutionary information is that the computation-heavy information extraction (learning the pLM) needs to be done only once during model training on efficient, high-performance computing centers. The extraction and use of the embeddings, by contrast, is done efficiently on consumer-grade hardware such as modern personal computers or even laptops.embedding-based annotation transfer (EAT) captures more information through comparing proteins in embedding, not sequence space [Since the introduction of the first general-purpose pLMs around 3 years ago ,48,50,56ce space ,60. Withce space . Adding ce space . The powevolutionary couplings (EV) [Considered 2021’s method of the year , AF2 2222 combinngs (EV)  extractengs (EV) ,68. Althngs (EV) .structure-based annotation transfer (SAT) emerged. SAT more reliably captures distantly related proteins (in silico predicted structures is available through the AF Protein Structure Database (AFDB)As alternatives to HBI or EAT, proteins . With thDespite efforts to improve the speed and sensitivity of structural aligners, traditional approaches , 75. areOne way to compress structural information is to break structures into fixed-size fragments. Geometricus  represenAnother category of tools represents tertiary structure as discretized volumes and compares these. BioZernike , for exaThe fastest category of structural aligners represents structures as sequences of a discrete structural alphabet. Most of these alphabets discretize the backbone angles of the structure , 83., hoSequence-based structural alignment tools are well equipped to handle the upcoming avalanche of predicted protein structures. Efficient storage of structure information and queries against these makes searches against hundreds of millions of structures feasible. Representing structures as sequences allows us to also adapt fast clustering algorithms like Linclust  to compaEmbeddings from pLMs in combination with fast structural aligners (Foldseek) could be orthogonal in covering, classifying, and validating assignments in large swaths of protein fold space, as shown in Following AFDB , structuLeishmania infantum to 76% for Escherichia coli).We recently analyzed the proportion of predicted AF2 structural domains in the 21 model organisms that could be assigned to known superfamilies in CATH . Only goBy classifying good-quality AF2 models into CATH, we can expand the number of structurally characterized domains by ~67%, and our knowledge of fold groups in superfamilies  increases by ~36% to 14 859 . As withOur analyses identified 2367 putative novel families . HoweverThe availability of off-the-shelf solutions based on AF2 – both as a tool , compleThanks to the increase in high-quality structure predictions spawned by AF2, there is an increasing need to readily leverage 3D information by prediction methods. Instead of using representations that first map 3D structures to 2D  or 1D  before feeding them to a predictor, such networks directly operate on 3D representations of macromolecules to make predictions about their properties. Using so-called ‘inductive bias’ when designing a network  avoids information loss during abstraction and enables the network to directly learn useful information from the raw 3D data itself. Recently, geometric deep-learning research , which fAF2 will help in eliminating an underlying bias in structural biology that has tended to focus more on drug discovery for human diseases, model organisms, or structures involved in pathogens. With its cheap footprint and cost, compared to traditional experimental means to characterize protein 3D structure, AF2 is neither constrained by access to expensive experimental instruments, nor to beam-time usually prioritized for large consortia. This will enable groups across the world to work on their proteins of interest without geographical or economical limitations. In a similar fashion to nanopore sequencing, model building could be done in real time for issues that are time- or location-sensitive , or limited to an individual, potentially allowing to transition from whole-genome sequencing to whole-proteome modeling, drug binding, and efficacy profiling.in silico, saving energy, time, and resources, requiring in vitro and in vivo experiments only at the most exciting stages of the experimental discovery process. Whilst not all designs fold, and caution is needed, an approach similar to spell correction in NLP but trained on millions of protein sequences allowed researchers to evolve and optimize existing antibodies to better perform a desired activity [Computationally predicting protein properties with increasing accuracy using deep learning ,53,57,58activity . Additioactivity . By seleWith booming putative structure databases, we see the emergence of analytical approaches leveraging a model similar to how UniProt’s mix of curated (SwissProt) and putative (TrEMBL)  sequenceWithin the realm of ‘traditional’ pLMs, that solely learn information from large unlabeled protein sequence databases, there is still room for improvement, as highlighted by recent advances in NLP. For example, there are approaches optimizing the efficiency of LMs, especially, on long sequences either by modifying the existing attention mechanism  or by prOutstanding questionsThe best structural models rely on quality and availability of protein structures found in nature. How can we further improve these deep-learning methods without relying on previous structural knowledge?How much structural novelty is hidden in metagenomes? Are we close to discovering all ways nature can shape a protein?Could protein modeling entirely replace experimentally derived structures?How can we use these methods to engineer a highly efficient enzyme function never encountered in living organisms?Speed, accuracy, and coverage of structure predictions and methods are skyrocketing. How can these tools be improved to better probe the dark universe of uncharacterized proteins?Could future methods build complexes of protein–protein interactions and models quickly and accurately enough to be used for precision medicine?Can we use embeddings-based methods to predict evolution in sequences?Can today’s deep-learning structure-prediction method capture dynamic changes in structure?Alt-text: Outstanding questionsUltimately, we see that a plethora of effective and efficient ML tools operating on different modalities, each with unique strengths and weaknesses, become available at researchers’ fingertips. Further developments in structure- and sequence-based approaches are inevitably needed see , yet, ev


The recent breakthroughs of Large Language Models (LLMs) in the context of natural language processing have opened the way to significant advances in protein research. Indeed, the relationships between human natural language and the “language of proteins” invite the application and adaptation of LLMs to protein modelling and design. Considering the impressive results of GPT-4 and other recently developed LLMs in processing, generating and translating human languages, we anticipate analogous results with the language of proteins. Indeed, protein language models have been already trained to accurately predict protein properties, generate novel functionally characterized proteins, achieving state-of-the-art results. In this paper we discuss the promises and the open challenges raised by this novel and exciting research area, and we propose our perspective on how LLMs will affect protein modeling and design. Machine Learning (ML) methods have a long-standing history in natural language processing (NLP), and considering the similarities between natural and protein languages , NLP metRecently two main breakthroughs in NLP research led to the so called “foundation models” , a.k.a. The first breakthrough is represented by the “attention mechanism” proposed in the Bengio’s seminal paper  by whichde novo generation of functionally characterized proteins . One possibility is to use the secondary structure for splitting the sentences into words or to exploit sub-word segmentation that does not require any predefined knowledge of words in the protein language. However, the tokenization process would require exploiting the tertiary structure with more intensive calculations. The overall understanding of the protein language is limited, requiring extensive experimental tests to identify its functionalities. Indeed, even if different corpora exist to train protein language models, the correct interpretation of the produced sequences remains a challenge. Protein evolution differs from language evolution, containing irregularities due to randomness and environmental pressure, and with a grammar that unavoidably will contain many irregularities. Finally, we have to remark on the size of the language of proteins that needs to cover millions of species on Earth, which necessitates studying the general properties of proteins rather than studying the proteins of a particular species.Proteins and natural languages also present differences that need to be taken into account in their processing. In human languages, the alphabet contains many symbols (like uniform punctuation and stop words) . In contWhile the dissimilarities between human and protein languages present significant challenges for applying NLP to protein design, the apparent connections between the two fields offer a new perspective in protein research, opening the way to the adaptation of NLP models to protein modeling and design.In this section we first discuss the main characteristics of the transformer  and thenThe Transformer is a deep neural network composed of two main components: an Encoder and a Decoder. Both the Encoder and Decoder possess a modular architecture, including a stack of repeated blocks, in which the output of each module is the input of the subsequent one .a to t. However, by changing only the last (top) layers of the network we can construct text classifiers, named entity recognizers, automatic summarizers, and more in general solve a large range of different prediction tasks. Here we introduce the main characteristics of this model. More details are available in the Supplementary Information.Basically, the Transformer can be applied to translate a text • Self-supervised learning: The Transformer learns in a supervised way, but without using explicit labels . This is• Multi-task and transfer Learning: The Transformer can learn multiple-tasks at a time  and can • Attention mechanism: This component  enables • Self-Attention: While the attention mechanism, originally proposed in the Bengio’s neural translation machine, leverages the relationships between the input words to learn the correspondences between the output words of the translation, the Transformer exploits a similar mechanism to find the semantic relationships between the words of the input sequence, in order to compute a representation of the sequence itself. In this way we can efficiently compute long-range dependencies between the elements of a sequence. See Supplementary Information for more details.• Multi-head attention: Self-Attention is computed multiple times in parallel using “multiple heads,” in order to capture the different syntactic and semantic relationships among the elements of the sequence.• Interpretability: A side-effect of Self-Attention is the interpretability of the model. Indeed each attention head can capture different types of syntactic and semantic relationships between the elements of the sequence .• Parallel computation: Instead of processing the elements of a sequence one at a time as in a RNN, Transformers are able to proceed in parallel, thus achieving a substantial speed-up in computation, fully exploiting the parallel computational capabilities of GPUs and TPUs.The Transformer is based upon the following main concepts:Each Encoder block is composed of two stacked sub-modules: 1) the Self-Attention layer and 2) a feed forward neural network (FFNN) with one hidden layer . ResiduaThe Decoder basically predicts step by step the translated sentence, receiving as input both the output of the last Encoder layer and the previously predicted words of the Decoder . During Other LLMs have been developed that leverage or extend the Transformer architecture, but the two most successful have likely been Bidirectional Encoder Representations from Transformers (BERT) , based o<MASK> tag) and predicted at the output of the Encoder. In this way, the model is trained to predict the masked word on the basis of its joint left and right context , while the standard Transformer and GPT learn only from the “left” context. This is because BERT basically learns a representation of the text, while GPT, that is essentially a generative model, will predict the next word on the basis of the previous “left” words. At the same time BERT is trained to learn the next sentence (NSP), given the previous one. Indeed, BERT may have in input either one or two sentences (separated by the <SEP> token in the latter case), and the final hidden embedding is used to predict whether the second sentence follows the first one , we can factorize the probability p(x) of observing a sequence of tokens x = {x1, …, xn} using the chain rule, thus decomposing the sequence prediction problem into next-word prediction  Self-supervised pre-training and 2) Supervised fine-tuning. During self-supervised training the model leverages linguistic information from unlabeled data by learning to predict the next token given the preceding tokens. In the second step, the general-purpose knowledge acquired in the first step is exploited and only a limited set of labeled examples is necessary to fine-tune the model, by adding a task-specific layer to perform prediction in a specialized context . Using sOpenAI released successive enhancements of GPT, namely, GPT-2 , GPT-3  and receThe success of LLMs for NLP and the similarity between natural language and “protein language” motivated the design of Protein Language Models (PLM), in which, rather than modeling the distribution of words/texts, amino acid and proteins are modeled instead . Indeed,The first proposed PLMs adopted an Encoder-only Transformer architecture, since their aim was to obtain embedded representations of proteins in a vector space for downstream tasks. For instance, TAPE (Task Assessing Protein Embedding) has been pre-trained to obtain embeddings, which have subsequently been processed via different supervised models in order to solve several downstream tasks  . Rives eOther models modified the original BERT Encoder-Transformer to better represent the protein world. For instance, ProteinBERT obtains “functionally aware” protein representations by simultaneously learning the protein sequences in the “local” Encoder stacked modules and their GO annotations in the “global” Encoder stacked modules. The local and global modules are trained in parallel and the former representations affect the latter ones through a global Attention module, while global representations influence the local ones through fully connected dense layers. The pre-trained models are then fine-tuned on several downstream tasks, ranging from secondary structure prediction, to remote homology, fold classes and signal peptide predictions, as well as post-translation and biophysical properties prediction, using only a fully connected dense layer on top of the Encoders .Another model that significantly modifies the original BERT Transformer is represented by Regularized Latent Space Optimization (ReLSO) . Its EncDecoders are generative models which learn to predict the next amino acid, given the previous ones in the sequence by using masked Self-Attention layers . In thisde novo generate protein sequences similar to natural ones, but can also explore protein regions unexplored by natural evolution. The new sequences, despite their relative sequence diversity compared to naturally occurring proteins, show structural similarity, predicted stability and several common properties with proteins sampled by natural selection  is a equences . The autequences ). Note tequences .f representing, e.g., a GO biological process, a molecular function or a protein family, or whatever property of the protein, thus decomposing the sequence prediction problem into next-amino acid prediction problem, instead of next-word prediction  has been developed by training on about half a billion of antibody heavy- and light-chain variable sequences, and conditioning on species of origin and chain type, thus opening the way to PLM-based engineering of specific antibodies .vice versa, given a ligand we could generate its corresponding protein binder. This is the approach proposed by in silico de novo engineering of protein drugs that bind to specific molecular targets. Moreover, by extending and adapting to the protein world recent IBM research on Attention-based neural networks for mapping the chemical space  and are able to do this by using self-supervised learning, i.e., by exploiting the pure unannotated protein sequences massively available in public repositories. From this standpoint, they are “general-purpose learners” in the sense that having learnt the protein distribution (if sufficient data are available and the model is sufficiently large), they can make predictions on tasks for which they have not been specifically trained or can be secondarily trained on specific tasks using only limited supervised fine tuning [as in the Lysozyme protein family prediction with ProGen ]. Such fvice versa, The main achievements and possible future outcomes of PLMs are schematically summarized in Indeed, an open problem posed by PLMs and more in general by LLMs is their explanation and interpretability. Given the increasing and widespread usage of LLM to solve problems involving high stakes decisions, we need to generate both global explanations, to provide hints about the generalized rules inferred by the model and its behavior as a whole, and local explanations, to interpret specific predictions .post hoc methods  have be methods , or modi methods  to bettefaithfulness and plausibility  in another GPT model . Though Another open issue is represented by the complexity of PLMs, that often requires costly special purpose hardware resources to train, or even query, a LLM/PLM. A possible solution could be the adoption of neural network compression techniques, such as pruning, quantization or distillation, to obtain thinner models once a LLM has been trained. Some experiments show the viability of these approaches, that in some case can attain a reduction of two orders of magnitude for the model size, at the price of a 1% drop in accuracy . Howeverde novo design of proteins, and more in general to solve complex problems in pharmacology, medicine, and environmental health. At this stage (September 2023), these possible outcomes mostly represent an attractive promise, but it is also true that in many fields, including biomolecular biology and medicine, the development and results of novel AI models exceeded any previous forecasting (In perspective, PLMs could generate synthetic libraries of functionally characterized proteins that can be used to discover, e.g., novel enzymes for industrial applications, or novel candidate drugs conditioned on specific functional characteristics. In particular, conditional Transformers, by using multiple functional tags, can expand the space of protein sequences beyond those sampled by the natural evolution. For instance we can condition protein generation on a functional tag for a specific enzymatic reaction and at the same time on another tag for a specific binding domain, thus generating proteins able to drive a specific biochemical reaction in a specific micro-environment. The capability of processing multi-modal data, i.e., not only sequence or functional tags, but also three-dimensional structures, images and bio-medical text could lead to multi-modal PLMs for precise ecasting .