While cooperative MAS have obvious advantages over single-agent systems due to shared information, an important limitation is the presence of restrictions over the communication between different agents which, in some cases, can even harm the system's performance [1]}. Such constraints can result from various reasons, such as thermal noise [2]}, quantization error [3]}, latency [4]}, communication failure [5]} etc. While some theoretical results for cooperative MAS disregard such restrictions, these appear to be not only an important, but also a necessary consideration, as it has been argued that multi-agent systems with perfect communication are analogous to a single agent system with multiple effectors [6]}. In order to resolve this issue, many works assume a sparse or local communication setting, such as gossip algorithms [7]} and neighbors-only communication [8]}, in the hope of helping to reduce the burden over the communication network [9]}. However, the effects of sparse communication on the performance of RL algorithms and the relation to the communication noise strength has not been investigated thoroughly theoretically, and a fundamental question is in which cases sparse communication is indeed advantageous to solving the task.
