Multi-agent reinforcement learning (MARL) is the study of the emergent behaviour in systems of interacting learning agents, with stochastic games serving as the most popular framework for modelling such systems [1]}, [2]}. In recent years, there has been a considerable amount of research in MARL that has aimed to produce algorithms with desirable system-wide performance and convergence properties. While these efforts have lead to a number of empirically successful algorithms, such as [3]}, there are comparatively fewer works that offer formal convergence analyses of their algorithms, and the bulk of existing work is suitable only for systems with a relatively small number of agents.
