Environments. Main results are presented with respect to three sets of environments: continuous control tasks, discrete-action games, and tasks with stochastic transitions. The continuous control tasks include the classic Mountain Car environment [1]}, the Mujoco-based Half-Cheetah environment [2]}, and the Ant Maze environment used in [3]}. The discrete-action games include 8 video games from the Atari Learning Environment (ALE; [4]}) and the Super Mario Bros. game, which is a popular NES platform game. The stochastic tasks include an image-prediction task with stochastic dynamics and two stochastic variants of Mountain Car, including a NoisyTV-like component.
