When computing this metric, we assume that (1) the object moves at roughly constant speed, (2) the camera always fixates on the object, and (3) the distance between the camera and the object remains approximately the same over time.
All sequences from existing works as well as ours meet these assumptions, except those from the NSFF [1]} and NV-DYN [2]} datasets.
In their case, the cameras are always facing forward, breaking the assumption (2).
However, we find that even though cameras are not fixated on the object since they are static, we can still compute the look-at point \(\mathbf {a}\)  by considering the center of mass of the foreground visible surfaces in 3D.
Both datasets provide accurate foreground segmentations and MVS depth, which we use to identify and back-project foreground pixels into 3D space.
The final look-at point is computed as the average foreground points over all frames.
