Depth completion is a related problem where the aim is to densify or inpaint an incomplete depth map. Diebel and Thrun [1]} is one of the first studies to tackle this problem using Markov random fields. Hawe et al. [2]} estimate disparity using wavelet analysis. The problem gained popularity as commodity depth sensors and laser scanners (or LiDARs) become more available. Uhrig et al. [3]} proposed sparse convolution to train a sparse invariant network. Jaritz et al. [4]} leveraged semantics to train the network at varying sparsity levels. Ma et al. [5]} concatenated the sparse depth map to an RGB image, and used this RGBD volume for training. Xu et al. [6]} filled in the missing depth values using the depth normal constraint. Imran et al. [7]} addressed the depth completion problem using depth coefficients as a representation. Qiu et al. [8]} suggested depth and normal fusion using learned attention maps. Methods based on a spatial propagation network (SPN) iterative optimize the dense depth map either in local [9]}, [10]} or non-local [11]} affinity. Chen et al. [12]} suggested fusing features from an image and 3D points to produce the dense depth. However, these depth completion methods usually aim for outdoor environments and street views where the points come from a LiDAR.
<FIGURE>