Reusing the pre-trained model under the above heterogeneous settings is difficult due to out-of-distribution concerns. Traditional methods will not work. For example, in methods reusing the pre-trained representation [1]}, [2]}, changes in the environment will cause non-stationarity in the observation space that directly plugs in the learned representation will make the learning even worse than learning from scratch. For another example, in methods reusing the pre-trained successor features to fast compute the value functions [3]}, changes in task objective or the environment will break the linear reward feature assumption.
