The majority of our Atari experiments use the regular ALE Atari environment [1]}, using DQN's default
preprocessing, random noop-starts and action-repeats [2]}, as well as using the reduced action-set (i.e. each game
exposing the subset of Atari's total 18 actions which are relevant to this game). For the `Sticky actions' experiment,
we use the OpenAI Gym variant of Atari [3]} enhanced with sticky actions [4]}.
