We turn to proximal policy optimization (PPO) [1]} for the gradient-based reinforcement learning algorithm used to try and find a policy, \(\pi _{\theta }\) , that produces the desired trajectories. PPO aims to maximize the policy gradient cost function,
\(L^{\rm PG}_{t}\left(\theta \right) = \hat{\mathbb {E}}\left[\log \pi _{\theta }\left(a_{t} | s_{t}\right)\hat{A}_{t} \right],\) 
