We propose the potential function network, a convolutional encoder-decoder model that estimates the potential function from a partially filled semantic map. Critically, we propose to train it interaction-free using a dataset of top-down semantic maps obtained from 3D semantic annotations [1]}, [2]} (see fig:intro, center). This is unlike prior work on RL which interactively learns ObjectNav policies by designing reward functions using the same semantic annotations [3]}, [4]}, [5]}. Specifically, our network predicts two complementary potential functions by leveraging geometric and semantic cues in the semantic map (e.g., the environment layout, room-object, and object-object relationships). The area potential function captures the unexplored areas in the map for efficient exploration, while the object potential function is a geodesic-distance based function that helps decide how to reach the object. Once trained, we deploy the potential function network in a modular framework for ObjectNav, where we combine the area and object potential predictions to decide where to look for a goal object.
