Learning communication protocols to solve cooperative tasks is one of the desired emergent behaviours of agent interactions and has become an active area in multi-agent reinforcement
learning [1]}, [2]}. Several algorithms have been put forward to help agents share information through differential protocols in recent years,
and we classify them into three categories. The first category focuses on effectively sharing observation information to solve partially observable problems. RIAL and DIAL [3]} use discrete
communication channels to demonstrate end-to-end learning of protocols. By contrast, CommNet [4]} uses continuous channels, averaging
message vectors and sending them to each agent. IC3Net [5]} controls when to communicate by a gate and can be applied to both cooperative and
competitive scenarios. ATOC [6]} proposes a communication model based on the hard attention mechanism to learn when to communicate and integrate information from others.
TarMAC [7]} applies a soft attention matrix to learn the relationship and aggregate messages based on the attention weights. VBC [8]} enables the agents to send
communication requests and reply to others adaptively based on their confidence about local decisions, reducing unnecessary information exchange. SchedNet [9]} schedules the agents by a
weight generator, and only a few agents are allowed to broadcast messages. NDQ [10]} uses message entropy and mutual information to shorten the message and reduce the uncertainty of
the receiver's state-action value.
