Existing works explored to combine learning modules into a classical SLAM system at different levels - in the front-end, DNNs produce predictions as priors, followed by incorporating these deep predictions into the back-end for next step optimization and refinement. One good example is CNN-SLAM [1]}, which utilizes the learned per-pixel depths into LSD-SLAM [2]}, a full SLAM system to support loop closing and graph optimization. Camera poses and scene representations are jointly optimized with depth maps to produce consistent scale metrics.
In DeepTAM [3]}, both the depth and pose predictions from deep neural networks are introduced into a classical DTAM system [4]}, that is optimized globally by the back-end to achieve more accurate scene reconstruction and camera motion tracking. A similar work can be found on integrating unsupervised VO with a graph optimization back-end [5]}.
DeepFactors [6]} vice versa integrates the learned optimizable scene representation (their so-called code representation) into a different style of back-end - probabilistic factor graph for global optimization. The advantage of the factor-graph based formulation is its flexibility to include sensor measurements, state estimates, and constraints. In a factor graph bach-end, it is quite easy and convenient to add new sensor modalities, pairwise constraints and system states into the graph for optimization.
However, these back-end optimizers are not yet differentiable.
