The teacher observed a 30x30 occupancy grid of walls and objects that were sampled using the stationary distribution \(X \sim p(X)\) . The teacher observation was processed through the 18-layer residual convolutional IMPALA network [1]}, followed by a fully connected layer of size 256, a layer norm and a factorial action head that specified the x and y grid position of each object the teacher controls. The value function used a separate network of identical architecture.
