All are trained using Proximal Policy Optimization [1]} with continuous state and action spaces (with Gaussian sampling), using the implementation in the RLLib library [2]}. The static dilemmas are trained for 1M environment steps (e.g., 8 agents acting simultaneously is 1 step), and the dynamic games are trained for 10M environment steps.
