Another important algorithm is the Proximal Policy Optimization method [1]}. As in DDPG, it also utilizes actor-critic architecture to solve the task with continuous observation and action spaces. In this algorithm, critic estimates value function (eq. REF ) instead of Q-function. In PPO, the actor produces not a single action but a distribution of actions and trains to maximize the probability of actions that lead to a better outcome.
