However, none of these existing methods has a resemblance to how communication emerges in nature. Specifically, considering how human languages emerge, no central modules or communication-specific supervised signals were needed [1]}. In other words, beings in nature learn to communicate resembles more a fully decentralized setting. Yet, existing decentralized approaches to communication in multi-agent reinforcement learning (MARL) systems are known to perform poorly even in simple tasks [2]}. The main challenge lies in a lack of common grounding in communication, making it difficult for agents to communicate meaningfully [3]}. Recent work like [4]} and [3]} propose novel methods to ground communication in this setting. The former introduces biases to the loss function encourage positive listening and positive signaling, which was subsequently shown to be ineffective in sensory-rich observations [3]}. [3]} propose using autoencoder to ground communicative messages by having each agent reconstruct their observations. However, existing works have not considered the relationships among messages sent by different agents as a direction for grounding, which is arguably essential given how we would like to have all the agents produce messages under a common language.
