The literature predominantly considers two types of communication: (i) communication through a cheap talk channel [1]}, [2]} and (2) communication through environmental actions [3]}. In cheap talk, agents have a designated communication channel where they share messages at every timestep—the messages are free to the agents and agents do not reason about whether to communicate. Many authors find that selfish agents do not learn to effectively use an ungrounded cheap talk communication channel [4]}, [5]}. However, one can incentivise communication in a cheap talk channel by rewarding an intrinsic motivation to influence others [6]} or by introducing a pro-social reward [7]}, [8]}, as we do in our work. On the other hand, when communicating through actions, agents' environmental actions are made visible to other agents, enabling the environmental actions to be used for communication. Note that environmental actions are binding—they impact the state of the environment and the resulting reward, so agents do not have an inconsequential way to exchange information as with a cheap talk channel. In these scenarios, cooperation has a high success rate [4]}, [10]}, [11]}, but the communication is inflexible and directly connected with the task—there is mixed opinion whether agents are actually communicating [12]}. In this work, we evaluate a third type of communication that we refer to as situated communication as introduced by Wagner et al. (2003) and used e.g. by Lowe et al. (2019) in matrix games, such as the Prisoner's Dilemma. When using situated communication, the listener has to actively choose between communicative and environmental actions. While only environmental actions directly affect the environment and allow the agent to obtain a reward, the agent can choose to forgo an environmental action to communicate.
