Baselines and scenarios. We list all baselines in table REF , including the corresponding research questions to be answered. We implement our method on PyMARL [1]} and use 10 random seeds to train each method on all environments. We do not use macro-action methods [2]}, [3]} as the baseline because it is hard to make a fair comparison between macro-actions methods and our method. As discussed in Sec. , macro-actions rely on manually designed macro-actions, i.e., designing the macro-actions by utilizing the simulator settings and domain knowledge, which is different from learning options [4]}, [5]}. Designing macro-actions is not feasible in scenarios where domain knowledge and simulator settings are unknown, such as the OBMAS scenarios. In OBMAS, the agent has no idea of the durations of other agents’ actions, which is challenging for designing macro-actions. We conduct experiments on Stag-Hunter Game, Quarry Game, Afforestation Game (Fig. REF ) and StarCraft II micromanagement tasks [1]} where off-beat action are introduced.
