As in Dreamer, we use learned transition function \(p_\phi (\hat{z}^i_t \mid h^i_t)\)  to predict the next stochastic state based on agents' actions and current states of the model. Starting from initial states sampled from the replay buffer, we generate rollouts based on current policies of the agents in order to deploy on-policy learning algorithms in the latent space. We set the length of the imaginary rollouts to \(H=15\)  to account for increased complexity of the multi-agent environments and possible compounding error [1]} of the world model.
<FIGURE><FIGURE>