To this end, LS-Net [1]} tackled this problem via a learning based optimizer by integrating the analytical solvers into its learning process. It learns a data-driven prior, followed by refining the DNN predictions with an analytical optimizer to ensure photometric consistency.
BA-Net [2]} integrates a differentiable second-order optimizer (LM algorithm) into a deep neural network to achieve end-to-end learning. Instead of minimizing geometric or photometric error, BA-Net is performed on feature space to optimize the consistency loss of features from multiview images extracted by ConvNets.
This feature-level optimizer can mitigate the fundamental problems of geometric or photometric solution, i.e. some information may be lost in the geometric optimization, while environmental dynamics and lighting changes may impact the photometric optimization). These learning based optimizers provide an alternative to solve bundle adjustment problem.
<TABLE>