[1]} (Figure REF a) is a multi-agent 2d environment simulating train traffic on the rail network. Each agent has its own starting point and destination and the goal of the environment is to guide agents to their destinations without collisions. The difficulty of the environment can be flexibly adjusted as the number of agents as well as the size of the map could be easily configured. Note that maps are generated procedurally for each trajectory, so the world model must be able to generalize well in order to achieve good performance. Each agent receives positive reward on arrival to its destination and negative reward if a collision occurs. We additionally use more dense variant of the reward provided by the winning solution of Flatland 2020 RL challenge [2]}. Agent receives additional positive reward if its action shortens the approximated distance to its destination. This reward allows us to alleviate credit assignment issue and facilitates exploration for better performance. In this environment, the density of the traffic can be quite sparse, allowing us to test locality of our approach. During the execution phase, the agents can send messages only to their neighbours, which are defined by their proximity on the railroad.
