@inproceedings{
wang2024digging,
title={Digging into contrastive learning for robust depth estimation with diffusion models},
author={JiYuan Wang and Chunyu Lin and Lang Nie and Kang Liao and Shuwei Shao and Yao Zhao},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=eyYcMjUtJ4}
}
Recently, diffusion-based depth estimation methods have drawn widespread attention due to their elegant denoising patterns and promising performance. However, they are typically unreliable under adverse conditions prevalent in real-world scenarios, such as rainy, snowy, etc. In this paper, we propose a novel robust depth estimation method with a customed contrastive learning mode for diffusion models, named D4RD, to resist performance degradation in complex environments. Concretely, we integrate the strength of knowledge distillation into contrastive learning, building the `trinity' contrastive scheme. It takes the sampled noise of the forward diffusion process as a natural reference and guides the predicted noise in different scenes to gather towards a more stable and precise optima. Meanwhile, we further extend noise-level trinity to more generic feature and image levels, building a multi-level contrast to distribute the burden of robust perception across the overall network. Moreover, before handling complex scenarios, we enhance the stability of the baseline diffusion model with three simple but effective improvements, which facilitate convergence and remove depth outliers.
Extensive experiments show that D4RD achieves superior performance to existing state-of-the-art (SoTA) solutions on both synthetic corruption datasets and real-world weather conditions. The code will be available.
Digging into contrastive learning for robust depth estimation with diffusion models
JiYuan Wang Chunyu Lin Lang Nie Kang Liao Shuwei Shao Yao Zhao