Given that decentralization and learning are the primary focuses of this study, we are interested in algorithms that have minimal coordination between agents after play begins. In particular, we wish to avoid forcing agents to follow the same policy during learning, which (as we discuss later) is a standard assumption in the literature on learning in MFGs. We also avoid the popular paradigm of centralized training with decentralized execution, wherein a global action-value function is learned during training and players select policies that can be implemented in a decentralized manner informed by this global quantity. Some examples of studies using this paradigm include [1]}, [2]} and [3]}.
