In fact, decentralised policies need not be learnt in a decentralized fashion.
For safety and efficiency reasons [1]}, MARL training frequently takes place centrally in a laboratory or in simulation,
allowing agents access to each other's observations during training, as well as otherwise unobservable extra state information.
Centralized training allows training of a single joint policy for all agents that conditions on the joint observations and extra state information.
While centralized joint learning reduces or removes issues surrounding partial observability and environment non-stationarity,
it must cope with joint action spaces that grow exponentially with respect to the number of agents, as well as a variety of learning pathologies that can result in suboptimal policies [2]}.
Importantly, vanilla joint policies are not inherently decentralisable and naive policy distillation approaches are often ineffective [3]}.
Joint learning does not immediately address the multi-agent credit assignment problem either.
