Two kind of scenarios can arise in Reinforcement Learning. Either our model has to learn sequentially several environment (i.e. space invader, then quest, then pacman, etc.) [1]}, [2]} or tasks [3]}, [4]}, (incremental learning) either the current environment change while learning a single tasks (lifelong learning).
From the perspective of the algorithm, the data distribution always change as the policy change. Therefore, it is not easy for it to tell apart changes due to the policy update or due to a new task or a new environment.
Nevertheless, Continual RL is more associated to case were the training data is not i.i.d and where their is several type of agent, environment or tasks.
