To this end, we propose Fine-Tuning with Lossy Affordance Planner (FLAP), a framework that leverages diverse offline data for learning representations, goal-conditioned policies, and affordance models that enable rapid fine-tuning to new tasks in target scenes.
As shown in Fig. REF , our lossy representations are acquired during the offline pre-training process for the goal-conditioned policy using a variational information bottleneck (VIB) [1]}.
Intuitively, this representation learns to capture the minimal information necessary to determine if and how a particular goal can be reached from a particular state, making it ideally suited for planning over subgoals and providing reward shaping during fine-tuning.
When a new task is specified to the robot with a goal image, we use an affordance model that predicts reachable states in this learned lossy representation space to plan subgoals for prospectively accomplishing this goal.
During the fine-tuning process, the goal-conditioned policy is fine-tuned on each subgoal given the informative reward signal computed from the learned representations.
Both the offline and online stage in this process operates entirely from images and does not require any hand-designed reward functions beyond those extracted automatically from the learned representation.
Building the components based on the prior work [2]}, [3]}, we demonstrate that the particular combination of lossy representation learning, goal-conditioned policies, and planning-driven fine-tuning can enable performance that significantly exceeds that of prior methods.
We evaluate our method in both real-world and simulated environments using previously collected offline data [4]}, [3]} for learning novel robotic manipulation tasks.
Compared to baselines, the proposed method achieves higher success rates with fewer online fine-tuning iterations.
