Note that, learning a cross-domain transferable successor feature representation will not solve the out-of-distribution (O.O.D.) problem when reusing it in downstream tasks with unseen changes, so that directly plugin the pre-trained successor features, as a common approach in ([1]}, [2]}), will not make our pre-trained model generally reusable. We propose two techniques to tackle this problem, (1) embodied set construction that discretizes the successor features into prototype sets (Sec. 3.2); (2) feature projection and projected Bellman updates to enable learning stability-plasticity (Sec. 3.3). Combined together, we make the pre-trained agent-environment interaction model generally reusable.
