We implement DQN with \(F\) -value estimation on four environments from Gym with increasing difficulty and complexity [1]}: CartPole, Acrobot, MountainCar, and LunarLander.
We aggregate results over 150 runs using varying hyperparameters (see Appendix ).
<FIGURE>