TITLE: Leak Proof PDBBind: A Reorganized Dataset of Protein-Ligand Complexes for More Generalizable Binding Affinity Prediction 

ABSTRACT: Many physics-based and machine-learned scoring functions (SFs) used to
predict protein-ligand binding free energies have been trained on the PDBBind
dataset. However, it is controversial as to whether new SFs are actually
improving since the general, refined, and core datasets of PDBBind are
cross-contaminated with proteins and ligands with high similarity, and hence
they may not perform comparably well in binding prediction of new
protein-ligand complexes. In this work we have carefully prepared a cleaned
PDBBind data set of non-covalent binders that are split into training,
validation, and test datasets to control for data leakage. The resulting
leak-proof (LP)-PDBBind data is used to retrain four popular SFs: AutoDock
vina, Random Forest (RF)-Score, InteractionGraphNet (IGN), and DeepDTA, to
better test their capabilities when applied to new protein-ligand complexes. In
particular we have formulated a new independent data set, BDB2020+, by matching
high quality binding free energies from BindingDB with co-crystalized
ligand-protein complexes from the PDB that have been deposited since 2020.
Based on all the benchmark results, the retrained models using LP-PDBBind that
rely on 3D information perform consistently among the best, with IGN especially
being recommended for scoring and ranking applications for new protein-ligand
systems.
Many physics-based and machine-learned scoring functions (SFs) used to
predict protein-ligand binding free energies have been trained on the PDBBind
dataset. However, it is controversial as to whether new SFs are actually
improving since the general, refined, and core datasets of PDBBind are
cross-contaminated with proteins and ligands with high similarity, and hence
they may not perform comparably well in binding prediction of new
protein-ligand complexes. In this work we have carefully prepared a cleaned
PDBBind data set of non-covalent binders that are split into training,
validation, and test datasets to control for data leakage. The resulting
leak-proof (LP)-PDBBind data is used to retrain four popular SFs: AutoDock
vina, Random Forest (RF)-Score, InteractionGraphNet (IGN), and DeepDTA, to
better test their capabilities when applied to new protein-ligand complexes. In
particular we have formulated a new independent data set, BDB2020+, by matching
high quality binding free energies from BindingDB with co-crystalized
ligand-protein complexes from the PDB that have been deposited since 2020.
Based on all the benchmark results, the retrained models using LP-PDBBind that
rely on 3D information perform consistently among the best, with IGN especially
being recommended for scoring and ranking applications for new protein-ligand
systems.
Leak Proof PDBBind: A Reorganized Dataset of Protein-Ligand Complexes for More Generalizable Binding Affinity Prediction
http://arxiv.org/abs/2308.09639v1