Inspired by neurophysiological discoveries of navigation cells in the mammalian
 brain, we introduce the first deep neural network architecture for modeling Egocentric
 Spatial Memory (ESM). It learns to estimate the pose of the agent and
 progressively construct top-down 2D global maps from egocentric views in a spatially
 extended environment. During the exploration, our proposed ESM network
 model updates belief of the global map based on local observations using a recurrent
 neural network. It also augments the local mapping with a novel external
 memory to encode and store latent representations of the visited places based on
 their corresponding locations in the egocentric coordinate. This enables the agents
 to perform loop closure and mapping correction. This work contributes in the
 following aspects: first, our proposed ESM network provides an accurate mapping
 ability which is vitally important for embodied agents to navigate to goal locations.
 In the experiments, we demonstrate the functionalities of the ESM network in
 random walks in complicated 3D mazes by comparing with several competitive
 baselines and state-of-the-art Simultaneous Localization and Mapping (SLAM)
 algorithms. Secondly, we faithfully hypothesize the functionality and the working
 mechanism of navigation cells in the brain. Comprehensive analysis of our model
 suggests the essential role of individual modules in our proposed architecture and
 demonstrates efficiency of communications among these modules. We hope this
 work would advance research in the collaboration and communications over both
 fields of computer science and computational neuroscience. Egocentric spatial memory (ESM) refers to a memory system that encodes, stores, recognizes and recalls the spatial information about the environment from an egocentric perspective BID24 . Such information is vitally important for embodied agents to construct spatial maps and reach goal locations in navigation tasks.For the past decades, a wealth of neurophysiological results have shed lights on the underlying neural mechanisms of ESM in mammalian brains. Mostly through single-cell electrophysiological recordings studies in mammals BID23 , there are four types of cells identified as specialized for processing spatial information: head-direction cells (HDC), border and boundary vector cells (BVC), place cells (PC) and grid cells (GC). Their functionalities are: (1) According to BID38 , HDC, together with view cells BID5 , fires whenever the mammal's head orients in certain directions. (2) The firing behavior of BVC depends on the proximity to environmental boundaries BID22 and directions relative to the mammals' heads BID1 . (3) PC resides in hippocampus and increases firing rates when the animal is in specific locations independent of head orientations BID1 . (4) GC, as a metric of space BID35 , are regularly distributed in a grid across the environment BID11 . They are updated based on animal's speed and orientation BID1 . The corporation of these cell types enables mammals to navigate and reach goal locations in complex environments; hence, we are motivated to endow artificial agents with the similar memory capability but a computational architecture for such ESM is still absent.Inspired by neurophysiological discoveries, we propose the first computational architecture, named as the Egocentric Spatial Memory Network (ESMN), for modeling ESM using a deep neural network. ESMN unifies functionalities of different navigation cells within one end-to-end trainable framework and accurately constructs top-down 2D global maps from egocentric views. To our best knowledge, we are the first to encapsulate the four cell types respectively with functionally similar neural networkbased modules within one integrated architecture. In navigation tasks, the agent with the ESMN takes one egomotion from a discrete set of macro-actions. ESMN fuses the observations from the agent over time and produces a top-down 2D local map using a recurrent neural network. In order to align the spatial information at the current step with all the past predicted local maps, ESMN estimates the agent's egomotion and transforms all the past information using a spatial transformer neural network. ESMN also augments the local mapping module with a novel spatial memory capable of integrating local maps into global maps and storing the discriminative representations of the visited places. The loop closure component will then detect whether the current place was visited by comparing its observation with the representations in the external memory which subsequently contributes to global map correction.Neuroscience-inspired AI is an emerging research field BID12 . Our novel deep learning architecture to model ESMN in the mammalian navigation system attempts to narrow the gap between computer science (CS) and computational neuroscience (CN) and bring interests to both communities. On one hand, our novel ESMN outperforms several competitive baselines and the state-of-the-art monocular visual SLAMs. Our outstanding performance in map construction brings great advancements in robotics and CS. It could also have many potential engineering applications, such as path planning for robots. (2) In CN, the neuroplausible navigation system with four types of cells integrated is still under development. In our work, we put forward bold hypothesis about how these navigation cells may cooperate and perform integrated navigation functions. We also faithfully propose several possible communication links among them in the form of deep architectures.We evaluate ESMN in eight 3D maze environments where they feature complex geometry, varieties of textures, and variant lighting conditions. In the experiments, we demonstrate the acquired skills of ESMN in terms of positional inference, free space prediction, loop closure classification and map correction which play important roles in navigation. We provide detailed analysis of each module in ESMN as well as their functional mappings with the four cell types. Lastly, we conduct ablation studies, compare with state-of-the-art Simultaneous Localization And Mapping (SLAM) algorithms and show the efficacy of our integrated framework on unifying the four modules. We get inspirations from neurophysiological discoveries and propose the first deep neural network architecture for modeling ESM which unifies the functionalities of the four navigation cell types: head-direction cells, border cells and boundary vector cells, place cells and grid cells. Our learnt model demonstrates the capacity of estimating the pose of the agent and constructing a top-down 2D spatial representations of the physical environments in the egocentric coordinate which could have many potential applications, such as path planning for robot agents. Our ESMN accumulates the belief about the free space by integrating egocentric views. To eliminate errors during mapping, ESMN also augments the local mapping module with an external spatial memory to keep track of the discriminative representations of the visited places for loop closure detection. We conduct exhaustive evaluation experiments by comparing our model with some competitive baselines and state-of-the-art SLAM algorithms. The experimental results demonstrate that our model surpasses all these methods. The comprehensive ablation study suggests the essential role of individual modules in our proposed architecture and the efficiency of communications among these modules.
first deep neural network for modeling Egocentric Spatial Memory inspired by neurophysiological discoveries of navigation cells in mammalian brain
{"src": "scitldr-AIC_train", "paper_id": "SkmM6M_pW"}