Can we pre-train a generally reusable backbone to accelerate downstream task reinforcement learning (RL)? While such backbones are popular in computer vision and natural language processing (e.g., pre-trained ResNet [1]} and pre-trained GPT3 [2]}, pre-trained decision-making backbones that support general downstream task learning are a few. The challenges largely attribute to the vast complexity and variety of decision-making tasks, that the pre-trained model will suffer from the out-of-distribution nightmare when reused to learn a new task. For example, a downstream task's environment dynamics or task objectives may be unseen during pre-training. Or we may need to add an extra sensor modality (e.g., language) to the agent to best fit the task requirement (e.g., understanding human commands). How can we pre-train a backbone that handles the above-unseen changes in downstream task learning, without making any assumptions on the task similarity [3]}, [4]} of environment similarity [5]}, [6]}?
