Our experiments use three indoor-scene datasets, which have RGB-D video frames with ground truth depths and known camera poses. ScanNet [1]} is
used in training and testing, and 7scenes [2]} and RGB-D Scenes V2 [3]} are evaluated for zero-shot generalization. (1) ScanNet. Our network is trained from scratch on ScanNet [1]} using the official training split. Following the frame selection heuristic in [5]}, considering appropriate view frustum overlap and sufficient baselines, we sample 279,494 training samples and 20,000 validation ones. Each sample contains 3 frames, with one as reference frame and the others as source frames. For testing, we use ScanNet's official test split (with 100 sequences from scene707 to scene806) and sample every 10 frames following [6]}, resulting in 20,668 samples for quantitative evaluation. Each sample contains N frames, with the middle one as reference, and the surroundings as source frames. Here we set \(N\) =3 or 5, and compare the performances with different number of views. We set the valid depth range as \(d_{min}\) =0.25 and \(d_{max}\) =20 meters, for networking training and metric calculation. ScanNet has images in 640\(\times \) 480 resolution. In training, they are resized to 256\(\times \) 256 with cropping following [5]}. For inference, the input images are resized to 320\(\times \) 256 without cropping. The predicted depth maps are upsampled with nearest neighbor interpolation to the original resolution 640\(\times \) 480 before calculating the quantitative metrics. (2) 7-Scenes. We select 13 sequences from 7-Scenes for zero-shot generalization. The valid depth range is set the same as that on ScanNet. We generate a test set with 1,610 samples (each with 5 frames, at 640\(\times \) 480 resolution) by sampling the sequences every 10 frames. (3) RGB-D Scenes V2. It contains indoor scenes, including chair, sofa, table, bowls, caps, cereal boxes, coffee mugs, and soda cans, etc. We select 8 sequences for testing. Similarly, we sample the video sequence every 10 frames to generate 610 testing samples (each with 5 frames). See more details about train/val/test splits in the supplement.
