We introduce a new memory architecture for navigation in previously unseen environments, inspired by landmark-based navigation in animals. The proposed semi-parametric topological memory (SPTM) consists of a (non-parametric) graph with nodes corresponding to locations in the environment and a (parametric) deep network capable of retrieving nodes from the graph based on observations. The graph stores no metric information, only connectivity of locations corresponding to the nodes. We use SPTM as a planning module in a navigation system. Given only 5 minutes of footage of a previously unseen maze, an SPTM-based navigation agent can build a topological map of the environment and use it to confidently navigate towards goals. The average success rate of the SPTM agent in goal-directed navigation across test environments is higher than the best-performing baseline by a factor of three. Deep learning (DL) has recently been used as an efficient approach to learning navigation in complex three-dimensional environments. DL-based approaches to navigation can be broadly divided into three classes: purely reactive BID49 , based on unstructured general-purpose memory such as LSTM BID33 BID31 , and employing a navigation-specific memory structure based on a metric map BID36 .However , extensive evidence from psychology suggests that when traversing environments, animals do not rely strongly on metric representations BID16 BID47 BID13 . Rather , animals employ a range of specialized navigation strategies of increasing complexity. According to BID13 , one such strategy is landmark navigation -"the ability to orient with respect to a known object". Another is route-based navigation that "involves remembering specific sequences of positions". Finally, map-based navigation assumes a "survey knowledge of the environmental layout", but the map need not be metric and in fact it is typically not: "[. . .] humans do not integrate experience on specific routes into a metric cognitive map for navigation [. . .] Rather, they primarily depend on a landmark-based navigation strategy, which can be supported by qualitative topological knowledge of the environment."In this paper, we propose semi-parametric topological memory (SPTM) -a deep-learning-based memory architecture for navigation, inspired by landmark-based navigation in animals. SPTM consists of two components: a non-parametric memory graph G where each node corresponds to a location in the environment, and a parametric deep network R capable of retrieving nodes from the graph based on observations. The graph contains no metric relations between the nodes, only connectivity information. While exploring the environment , the agent builds the graph by appending observations to it and adding shortcut connections based on detected visual similarities. The network R is trained to retrieve nodes from the graph based on an observation of the environment. This allows the agent to localize itself in the graph. Finally, we build a complete SPTM-based navigation agent by complementing the memory with a locomotion network L, which allows the agent to move between nodes in the graph. The R and L networks are trained in self-supervised fashion, without any manual labeling or reward signal.We evaluate the proposed system and relevant baselines on the task of goal-directed maze navigation in simulated three-dimensional environments. The agent is instantiated in a previously unseen maze and given a recording of a walk through the maze (images only, no information about actions taken or ego-motion). Then the agent is initialized at a new location in the maze and has to reach a goal location in the maze, given an image of that goal. To be successful at this task, the agent must represent the maze based on the footage it has seen, and effectively utilize this representation for navigation.The proposed system outperforms baseline approaches by a large margin. Given 5 minutes of maze walkthrough footage, the system is able to build an internal representation of the environment and use it to confidently navigate to various goals within the maze. The average success rate of the SPTM agent in goal-directed navigation across test environments is higher than the best-performing baseline by a factor of three. Qualitative results and an implementation of the method are available at https://sites.google.com/view/SPTM. We have proposed semi-parametric topological memory (SPTM), a memory architecture that consists of a non-parametric component -a topological graph, and a parametric component -a deep network capable of retrieving nodes from the graph given observations from the environment. We have shown that SPTM can act as a planning module in a navigation system. This navigation agent can efficiently reach goals in a previously unseen environment after being presented with only 5 minutes of footage. We see several avenues for future work. First, improving the performance of the networks R and L will directly improve the overall quality of the system. Second, while the current system explicitly avoids using ego-motion information, findings from experimental psychology suggest that noisy ego-motion estimation and path integration are useful for navigation. Incorporating these into our model can further improve robustness. Third, in our current system the size of the memory grows linearly with the duration of the exploration period. This may become problematic when navigating in very large environments, or in lifelong learning scenarios. A possible solution is adaptive subsampling, by only retaining the most informative or discriminative observations in memory. Finally, it would be interesting to integrate SPTM into a system that is trainable end-to-end.SUPPLEMENTARY MATERIAL S1 METHOD DETAILS S1.1 NETWORK ARCHITECTURESThe retrieval network R and the locomotion network L are both based on ResNet-18 BID19 . Both take 160×120 pixel images as inputs. The networks are initialized as proposed by BID19 . We used an open ResNet implementation: https://github.com/raghakot/ keras-resnet/blob/master/resnet.py.The network R admits two observations as input. Each of these is processed by a convolutional ResNet-18 encoder. Each of the encoders produces a 512-dimensional embedding vector. These are concatenated and fed through a fully-connected network with 4 hidden layers with 512 units each and ReLU nonlinearities.The network L also admits two observations, but in contrast with the network R it processes them jointly, after concatenating them together. A convolutional ResNet-18 encoder is followed by a single fully-connected layer with 7 outputs and a softmax. The 7 outputs correspond to all available actions: do nothing, move forward, move backward, move left, move right, turn left, and turn right.
We introduce a new memory architecture for navigation in previously unseen environments, inspired by landmark-based navigation in animals.
{"src": "scitldr-AIC_train", "paper_id": "SygwwGbRW"}