Distributed Proximal Policy Optimization, DPPO [1]}, is a distributed reinforcement learning framework based on Proximal Policy Optimization, PPO, and improves over A3C in continuous problems. The applied architecture is similar to A2C and pretty straightforward. The architecture consists of a central chief and multiple workers. On every iteration, the chief sends the current parameters to the workers and anticipates the gradients to be calculated by the workers, with the PPO method. When the number of received gradient updates reaches a minimum, the chief applies the gradients' average to the parameter networks for value and policy. Hence, the distributed framework could be considered equivalent to A2C.
