Students' observations are as described in [1]}: They observe their own location, orientation and velocity and the location, orientation and velocities of objects and agents around them. Objects and agents that are hidden are masked out for the policy, but visible to the value function (the mask is only used in Hide and Seek, not in the maze environment). In addition, they have access to a 360-degree LIDAR-type observation that informs them about the distance to the nearest object. In addition to the observations in [1]} agents also observed episode time, which we find improves the value function fit for longer episode lengths. We utilize decentralized execution and centralized training for multi-agent environments.
