Offline MBRL.    
Most of the existing offline MBRL works focus on policy learning under a given model trained by MLE.
These works typically constrain the learned policy to avoid visiting regions where the discrepancy between the true and the learned dynamic is large, thus reducing the policy evaluation error of using the learned dynamic [1]}, [2]}, [3]}, [4]}.
Besides, some recent works [5]}, [6]} also adopt a GAN-style stochastic policy for its flexibility.
Rather than using a fixed MLE-trained model, we derive an objective that trains both the policy and the dynamic model toward maximizing a lower bound of true expected return
(simultaneously minimizing the policy evaluation error \(|J(\pi , P^{*}) - J(\pi , \widehat{P})|\) ).
Several recent works also propose to enhance model training, such as training a reverse dynamic model to encourage conservative model-based imaginations [7]},
learning a balanced state-action representation for the model to mitigate the distributional discrepancy [8]}, [9]}, and using advanced dynamic-model architecture of the GPT-Transformer [10]}, [11]}, [12]}, [13]} to achieve accurate predictions [14]}.
These methods are orthogonal to ours and may further improve the performance.
