The agent's embodiment is a simulated LoCoBot [1]}. The action space consists of six actions: MoveAhead, RotateLeft, RotateRight, End, LookUp, and LookDown. Given the excellent exploration capabilities of EmbCLIP [2]}, [3]}, we opt to keep the same overall architecture, just replacing the learned embedding for target types in prior work by a linear projection of the text branch output of CLIP for the target description, as shown in Fig. REF . Additionally, in order to provide more information about the target and the current visual input, we increase the respective internal representations for each modality from the original 32-D to 256-D. Note that our model does not employ the alternative zero-shot design described in [2]}, where the target description is not observed by the agent's RNN. Given the scale of Objaverse-LVIS, we can train agents with good generalization following a more standard design.
<FIGURE>