Offline model-based RL.     In offline model-based RL algorithms, the true environmental dynamic \(P^*\)  is typically approximated by a parametric function \(\widehat{P}\left(s^{\prime } \,|\,s, a\right)\)  in some function class \(\mathcal {P}\) .
With the offline dataset \({\mathcal {D}}_{\mathrm {env}}\) , \(\widehat{P}\)  is trained via the MLE [1]}, [2]}, [3]} as
\(\textstyle \arg \max _{\widehat{P} \in \mathcal {P}} \mathbb {E}_{\left(s,a,s^{\prime }\right) \sim {\mathcal {D}}_{\mathrm {env}}}\left[\log \widehat{P}\left(s^{\prime } \,|\,s, a\right)\right].\) 
