Visual Pre-training for Navigation: What Can We Learn from Noise?
A powerful paradigm for sensorimotor control is to predict actions from
observations directly. Training such an end-to-end system allows
representations that are useful for the downstream tasks to emerge
automatically. In visual navigation, an agent can learn to navigate without any
manual designs by correlating how its views change with the actions being
taken. However, the lack of inductive bias makes this system data-inefficient
and impractical in scenarios like search and rescue, where interacting with the
environment to collect data is costly. We hypothesize a sufficient
representation of the current view and the goal view for a navigation policy
can be learned by predicting the location and size of a crop of the current
view that corresponds to the goal. We further show that training such random
crop prediction in a self-supervised fashion purely on random noise images
transfers well to natural home images. The learned representation can then be
bootstrapped to learn a navigation policy efficiently with little interaction
data. Code is available at https://github.com/yanweiw/noise2ptz.