MuPNet: Multi modal Predictive Coding Network for Place Recognition by Unsupervised Learning of Joint Visuo Tactile Latent Representations
Extracting and binding salient information from different sensory modalities to determine common features in the environment is a significant challenge in robotics. Here we present MuPNet (Multi modal Predictive Coding Network) a biologically plausible network architecture for extracting joint latent features from visuo tactile sensory data gathered from a biomimetic mobile robot. In this study we evaluate MuPNet applied to place recognition as a simulated biomimetic robot platform explores visually aliased environments. The F1 scores demonstrate that its performance over prior hand crafted sensory feature extraction techniques is equivalent under controlled conditions, with significant improvement when operating in novel environments.
ArXiv