We use \(\lambda \) -target [1]} for value function as in Dreamer, which gives us more unbiased estimates than TD-error [2]}. We augment critic by giving it as an input all current latent states \(\hat{\textbf {z}}_t\) , which are processed using Attention mechanism. Unlike other CTDE approaches [3]}, [4]}, we cannot use additional information beyond the states predicted by the model as we train agents in the latent space. However, several studies [5]}, [6]} suggested that such additional information can cause centralized-decentralized mismatch, which can hinder the learning process due to the large variance of the policy gradient updates. Furthermore, in real world scenarios we would not have such additional information as the global environment may be too large.
