Different from match-based methods that establish 2D-3D correspondences before calculating pose,
scene coordinate regression approaches estimate the 3D coordinates of each pixel from the query image within the world coordinate system, i.e. the scene coordinates. It can be viewed as learning a transformation from the query image to the global coordinates of the scene. DSAC [1]} utilizes a ConvNet model to regress scene coordinates, followed by a novel differentiable RANSAC to allow end-to-end training of the whole pipeline. Such common pipeline was then improved by introducing the reprojection loss [2]}, [3]}, [4]} or multi-view geometric constraints [5]} to enable unsupervised learning, jointly learning the observation confidences [6]}, [7]} to enhance the sampling efficiency and accuracy, exploiting Mixture of Experts (MoE) strategy [8]} or hierarchical coarse-to-fine [9]} to eliminate environment ambiguities. Different from these, KFNet [10]} extends the scene coordinate regression problem to the time domain and thus bridges the existing performance gap between temporal and one-shot relocalization approaches. However, they still trained for a specific scene and cannot be generalized to unseen scenes without retraining. To build a scene agnostic method, SANet [11]} regress the scene coordinate map of the query by interpolating the 3D points associated with retrieved scene images. Unlike aforementioned methods trained in a patch-based manner, Dense SCR [12]} propose to perform the scene coordinate regression in a full-frame manner to make the computation efficient at test time and, more importantly, to add more global context to the regression process to improve the robustness.
