In this section, we study the convergence of learning iterates obtained when an independent learning agent, player \(i \in \mathcal {N}\)  in our mean-field game \(\mathcal {G}\) , naively runs single-agent RL algorithms that treats its observation variable \(\lbrace  y^i_t \rbrace _{t \ge 0}\)  as though it were the state variable of a MDP. This learning process is formalized in Algorithm , below, where we have fixed the policies of all players to be stationary. By fixing the policies of the remaining players \(-i\)  to be stationary, this section focuses on the effect that decentralized information has on independent learners, leaving aside the well-known challenge of non-stationary [1]}.
