We further extend this hypothesis: in the context of deep reinforcement learning
(i.e. with non-linear function approximation),
an inadequate data distribution can drive over-generalization [1]},
making such erroneous extrapolation likely.
While the tandem effect can show up as learning inefficiency even in the tabular case [2]}, it proves
especially pernicious in the case of non-linear function approximation,
where erroneous extrapolation can lead to errors not just on completely unseen,
but also rarely seen data, and can persist in the infinite-sample limit.
