@inproceedings{
cen2023sad,
title={{SAD}: Segment Any {RGBD}},
author={Jun CEN and Yizheng Wu and Kewei Wang and Xingyi Li and Jingkang Yang and Yixuan Pei and Lingdong Kong and Ziwei Liu and Qifeng Chen},
booktitle={R0-FoMo:Robustness of Few-shot and Zero-shot Learning in Large Foundation Models},
year={2023},
url={https://openreview.net/forum?id=fzLpelzcXl}
}
The Segment Anything Model (SAM) has demonstrated its effectiveness in segmenting any part of 2D RGB images. A lot of SAM-based applications have shown amazing performance. However, SAM exhibits a stronger emphasis on texture information while paying less attention to geometry information when segmenting RGB images. To address this limitation, we propose the Segment Any RGBD (SAD) model, which is specifically designed to extract geometry information directly from images. Inspired by the natural ability of humans to identify objects through the visualization of depth maps, SAD utilizes SAM to segment the rendered depth map, thus providing cues with enhanced geometry information and mitigating the issue of over-segmentation. Compared to other SAM-based projects, we are the first to use SAM to segment non-RGB images. We further include the open-vocabulary semantic segmentation in our framework to provide the semantic labels of each segment.
SAD: Segment Any RGBD
Jun CEN Yizheng Wu Kewei Wang Xingyi Li Jingkang Yang Yixuan Pei Lingdong Kong Ziwei Liu Qifeng Chen