One of the major paradigms in Multi-Agent Reinforcement Learning is Centralized Training with Decentralized Execution (CTDE) [1]}, [2]}, [3]}, which holds to the premise that we can use additional information during training as long as execution remains decentralized. This paradigm is well-motivated when we want full autonomy for the agents [4]}, for example, when the task is not fully cooperative because agents might not want to share their information with others [5]}, [6]}, [7]}. Furthermore, in many real world scenarios, such as autonomous driving [8]} or drone management [9]}, agents must collaborate with each other to achieve their goals and they can also share information for better coordination. This setup is usually defined as a communication technique [10]}, [11]}, as the agents no longer have full autonomy yet they can leverage the messages from others to make their decisions. As a part of CTDE, communication methods usually restrict the message channel either by limiting bandwidth or receiving messages only from the neighbours in the environment [12]}. Other CTDE techniques can still be applied together with the communication technique to stabilize learning or solve the credit assignment issue. Note that if we decide to centralize execution even more, reducing the multi-agent problem to single-agent, this solution will suffer from scalability issues as the number of states and actions will grow exponentially with the number of agents. Thus, communication methods strike a balance between autonomy of the agents and efficiently using available information to make decisions in collaborative environments. However, previous studies do not fully leverage this additional shared information during the training phase.
