In this light, reinforcement learning (RL) has been increasingly used for goal-directed molecular design, thanks to its exploration ability.
Previous models have been assessed with relatively simple objectives, such as cLogP and QED, or estimated bioactivity scores predicted by auxiliary models [1]}, [2]}, [3]}, [4]}.
However, high scores in those simple properties of molecules guarantee neither drug-likeness nor therapeutic potential, emphasizing the necessity of more relevant design objectives in generative tasks [5]}, [6]}.
The molecular docking method is a physical simulation that estimates the protein-small molecule binding affinity, a key measure of drug efficacy. As docking simulations are a more straightforward proxy of therapeutic potential, coupling RL with docking simulations would be a promising strategy.
While the simplistic scores (e.g., cLogP, QED) are computed by a sum of local fragments' scores and are not a function of global molecular structure, making the optimization tasks relatively simple, docking score optimization is a more demanding exploration problem for RL agents.
A change in docking score is nonlinear with the molecule's local structural changes, and a significant variance exists among the structures of high-scoring molecules, meaning that there exists many local optima [5]}.
