(2) Reusable policy/value functions:
This approach aims to pre-train a policy or value function that can be further fine-tuned in downstream task learning. Methods include assuming task distribution priors (meta-RL [1]}), assuming environmental distribution priors (generalizable RL [2]} , robust RL [3]}), and contextual MDP based ones [4]}. Recently, representing a policy using generative sequential models has gained increasing popularity (Gato [5]}, Decision Transformers [6]}). However, their capability of downstream task learning is commonly restricted to the training setup, and handling large changes when reused remains challenging [5]}.
