

which is calculated in expectation over \(q_\phi (\textbf {z}_{1:T} \mid \textbf {a}_{1:T}, \textbf {o}_{1:T}) \)  and summed over all \(n\)  agents and rollout length \(T\) . As in Dreamer, all losses are optimized jointly using the Adam optimizer [1]}. Note that we do not use an additional \(\beta \)  weight for KL divergence [2]} to simplify algorithm complexity. As in DreamerV2, we use KL balancing to facilitate learning of prior \(p_\phi (\hat{z}^i_t \mid h^i_t)\)  by assigning a larger weight to cross entropy and a lower weight to posterior entropy in the KL term. The schematic representation of the training phase is in Figure REF .
