We adopt low data training regime from [1]} to highlight sample efficiency of the current state-of-the-art MFRL methods. We chose MAPPO [2]}, COMA [3]}, QMIX [4]}, CommNet [5]} and G2A [6]} as the baselines for SMAC environments and winning solution of [7]} as the baseline for Flatland environment. There are three levels of difficulty both in SMAC and Flatland. In order to make up for this difference, we allocate more samples from the environment depending on its difficulty. Specifically, we use 100k, 200k and 500k samples for SMAC and 300k, 1kk and 1.5kk for Flatland. All hyperparameters, ablation study and details about the environments can be found in Appendix. We use this implementationhttps://github.com/starry-sky6688/StarCraft for all baselines in SMAC except MAPPO, which implementation is provided by authorshttps://github.com/marlbenchmark/on-policy. We use authors implementation of the winning Flatland 2020 solution that can be found herehttps://github.com/jbr-ai-labs/NeurIPS2020-Flatland-Competition-Solution.
<FIGURE>