We introduce a general self-supervised approach to predict the future outputs of a short-range sensor (such as a proximity sensor) given the current outputs of a long-range sensor (such as a camera). We assume that the former is directly related to some piece of information to be perceived (such as the presence of an obstacle in a given position), whereas the latter is information rich but hard to interpret directly. We instantiate and implement the approach on a small mobile robot to detect obstacles at various distances using the video stream of the robot's forward-pointing camera, by training a convolutional neural network on automatically-acquired datasets. We quantitatively evaluate the quality of the predictions on unseen scenarios, qualitatively evaluate robustness to different operating conditions, and demonstrate usage as the sole input of an obstacle-avoidance controller. We additionally instantiate the approach on a different simulated scenario with complementary characteristics, to exemplify the generality of our contribution.
In future planetary exploration missions, improvements in autonomous rover mobility have the potential to increase scientific data return by providing safe access to geologically interesting sites that lie in rugged terrain, far from landing areas. To improve rover-based terrain sensing, this paper proposes a self-supervised learning framework that will enable a robotic system to learn to predict mechanical properties of distant terrain, based on measurements of mechanical properties of similar terrain that has been traversed previously. In this framework, a proprioceptive terrain classifier is used to distinguish terrain classes based on features derived from rover–terrain interaction, and labels from this classifier are used to train an exteroceptive (i.e., vision-based) terrain classifier. Once trained, the vision-based classifier is able to recognize similar terrain classes in stereo imagery. This paper presents two distinct proprioceptive classifiers—a novel approach based on optimization of a traction force model and a previously described approach based on wheel vibration—as well as a vision-based terrain classification approach suitable for environments with unexpected appearances. The high accuracy of the self-supervised learning framework and its supporting algorithms is demonstrated using experimental data from a four-wheeled robot in an outdoor Mars-analogue environment. © 2012 Wiley Periodicals, Inc. © 2012 Wiley Periodicals, Inc. A key feature for an autonomous mobile robot navigating in off-road unknown areas is environment sensing. Extraction of meaningful information from sensor data allows a good characterization of the near to far terrains, and thus, the ability for the vehicle to achieve its tasks with easiness. We present an image feature extraction scheme to predict mobile platform motion information. For a sequence of run, several images of terrains and vibrations endured by the mobile robot are acquired using a camera and an acceleration sensor. Texture information extracted by the Segmentation-based Fractal Texture Analysis descriptor (SFTA) was used to find correlations with acceleration features quantified using different time analysis parameters. Experimental results showed that texture information is a good candidate to predict running information.