In this section, we introduce open-vocabulary ObjectNav, a new task propelled by the vast diversity of objects that appear in Objaverse. Here, an agent is placed at a random starting location inside of a home and tasked to navigate to a target object provided from a text description (“Raspberry Pi Pico”). To facilitate this task, we procedurally generate 10K new homes in ProcTHOR [1]} fully populated with objects from Objaverse-LVIS. Until now, ObjectNav tasks have focused on training agents to navigate to 20 or so target objects provided their category label [2]}, [3]}, [1]}, and existing interactive embodied AI simulations, including ProcTHOR, only include around 2K total objects across around 100 object types [5]}, [1]}, [7]}. In this work, we take a large step to massively scale the number of target objects used in ObjectNav (20 \(\rightarrow \)  OpenVocab), the number of objects available in simulation (2K \(\rightarrow \)  36K), and the number of object types of the objects (100 \(\rightarrow \)  1.1K).
