Depth Map Super Resolution by Deep Multi Scale Guidance
Depth boundaries often lose sharpness when upsampling from low resolution (LR) depth maps especially at large upscaling factors. We present a new method to address the problem of depth map super resolution in which a high resolution (HR) depth map is inferred from a LR depth map and an additional HR intensity image of the same scene. We propose a Multi Scale Guided convolutional network (MSG Net) for depth map super resolution. MSG Net complements LR depth features with HR intensity features using a multi scale fusion strategy. Such a multi scale guidance allows the network to better adapt for upsampling of both fine and large scale structures. Specifically, the rich hierarchical HR intensity features at different levels progressively resolve ambiguity in depth map upsampling. Moreover, we employ a high frequency domain training method to not only reduce training time but also facilitate the fusion of depth and intensity features. With the multi scale guidance, MSG Net achieves state of art performance for depth map upsampling.
ECCV