We propose a modular baseline and demonstrate that embodied (physical) commonsense extracted from large language models (LLMs) [1]}, [2]} serves as an effective planner for Housekeep.
Specifically, we find that finetuning LLM embeddings on a subset of human preferences generalizes well, and helps to reason about correct rearrangements for novel objects never seen during training.
We integrate this LLM-based planning module into a hierarchical policy that coordinates navigation, exploration, and planning as a baseline approach to Housekeep. Our hierarchical approach also generalizes to unseen objects and scenes in Housekeep achieving an object success rate of 0.23 for unseen (versus 0.30 on seen objects).
We also qualitatively analyze different failure cases of our baseline to highlight venues for further progress.
