During the execution and imaginary rollout stages, agents are allowed to transmit only their stochastic states \(z^i_t\) . As in DreamerV2, stochastic state has a discrete nature and consists of 32 categorical distributions with 32 classes each. Straight-through gradients [1]} are used to learn these discrete representations. By design, stochastic state is the only message that can be broadcasted by the agent, which amounts only to \(32\log {32}=160\)  bits of information per message. This is largely because of the sparsity of the \(z^i_t\)  compared to the other possible distributions. We can also use a language-based interpretation [2]}, [3]} of \(z^i_t\)  as we restrict agents to a very limited vocabulary to express their current states. We further elaborate on this point in the next section.
