We follow [1]} and [2]} to perform \(h\) -step rollouts branching from the offline dataset \({\mathcal {D}}_{\mathrm {env}}\) , using the learned dynamic \(\widehat{P}, \hat{r}\) , and the current policy \(\pi _{\mathbf {\phi }}\) .
The generated data is added to a separate replay buffer \({\mathcal {D}}_{\mathrm {model}}\) .
