The photometric consistency constraint assumes that the entire scenario consists only of rigid static structures, e.g. buildings and lanes. However, in real-world applications, environmental dynamics (e.g. pedestrians and vehicles), will distort the photometric projection and degrade the accuracy of pose estimation. To address this concern, GeoNet [1]} divides its learning process into two sub-tasks by estimating static scene structures and motion dynamics separately through a rigid structure reconstructor and a non-rigid motion localizer. In addition, GeoNet enforces a geometric consistency loss to mitigate the issues caused by camera occlusions and non-Lambertian surfaces. [2]} adds a 2D flow generator along with a depth network to generate 3D flow. Benefiting from better 3D understanding of environment, their framework is able to produce more accurate camera pose, along with a point cloud map.
GANVO [3]} employs a generative adversarial learning paradigm for depth generation, and introduces a temporal recurrent module for pose regression.
Li et al. [4]} also utilized a generative adversarial network (GAN) to generate more realistic depth maps and poses, and further encourage more accurate synthetic images in the target frame. Instead of a hand-crafted metric, a discriminator is adopted to evaluate the quality of synthetic images generation. In doing so, the generative adversarial setup facilitates the generated depth maps to be more texture-rich and crisper. In this way, high-level scene perception and representation are accurately captured and environmental dynamics are implicitly tolerated.
