Note that both Nerfies [1]} and HyperNeRF [2]} use background regularization which requires a point cloud of the background static scene.
We first extract the object mask over time by MTTR, an off-the-shelf video segmentation network [3]}, which takes a text prompt of the foreground object as input.
Since our foreground objects are quite diverse (e.g., backpack and block), the segmentation results are usually noisy.
Thus we apply TSDF Fusion [4]} to the background point clouds over the whole sequence to get a completed background point cloud.
We find that this point cloud can be noisy when segmentation fails, and that it is necessary to manually filter the background point cloud to make sure that it does not include any foreground regions.
We consider this manual process a weakness of the previous background regularization [1]}.
