Table REF  shows the shared hyperparameters for all datasets in our experiments.
Many of these hyperparameters follow the literature without further tuning.
For example, we use \(\eta _{\mathbf {\phi }}= \eta _{\mathbf {\psi }}= 2 \times 10^{-4}\)  as in [1]}, \(\eta _{\mathbf {\theta }}= 3 \times 10^{-4}\)  and \(N_{\mathrm {warm}} = 40\)  as in [2]}, \(c=0.75\)  as in [3]}, policy frequency \(k=2\)  as in [4]}, \(f=0.5\)  as in [5]} and model learning rate \(\eta _{\widehat{P}} = 0.001\)  as in [6]}.
Unless specified, the shared and the non-shared hyperparameters are used throughout the main results and the ablation study.
<TABLE>