Data-driven approaches have emerged to consider learning 6-DoF poses directly from visual and inertial measurements without human intervention or calibration.
VINet [1]} is the first work that formulated visual-inertial odometry as a sequential learning problem, and proposed a deep neural network framework to achieve VIO in an end-to-end manner.
VINet uses a ConvNet based visual encoder to extract visual features from two consecutive RGB images, and an inertial encoder to extract inertial features from a sequence of IMU data with a long short-term memory (LSTM) network. Here, the LSTM aims to model the temporal state evolution of inertial data. The visual and inertial features are concatenated together, and taken as the input into a further LSTM module to predict relative poses, conditioned on the history of system states.
This learning approach has the advantage of being more robust to calibration and relative timing offset errors.
However, VINet has not fully addressed the problem of learning a meaningful sensor fusion strategy.
