We propose two sets of metrics to overcome this challenge without the use of camera teleportation. The first metric enables evaluating only on pixels that were seen in the input sequence by computing the co-visibility of every test pixel.
The proposed co-visibility mask can be used to compute
masked image metrics (PSNR, SSIM [1]} and LPIPS [2]}).
While the masked image metrics measure the quality of rendering, they do not directly measure the quality of the inferred scene deformation.
Thus, we also propose a second metric that evaluates the quality of established point correspondences by the percentage of correctly transferred keypoints (PCK-T) [3]}.
The correspondences may be evaluated between the input and test frames or even within the input frames, which enable evaluation on sequences that are captured with only a single camera.
