SUN RGB D: A RGB D scene understanding benchmark suite
Although RGB D sensors have enabled major break throughs for several vision tasks, such as 3D reconstruction, we have not attained the same level of success in high level scene understanding. Perhaps one of the main reasons is the lack of a large scale benchmark with 3D annotations and 3D evaluation metrics. In this paper, we introduce an RGB D benchmark suite for the goal of advancing the state of the arts in all major scene understanding tasks. Our dataset is captured by four different sensors and contains 10,335 RGB D images, at a similar scale as PASCAL VOC. The whole dataset is densely annotated and includes 146,617 2D polygons and 64,595 3D bounding boxes with accurate object orientations, as well as a 3D room layout and scene category for each image. This dataset enables us to train data hungry algorithms for scene understanding tasks, evaluate them using meaningful 3D metrics, avoid overfitting to a small testing set, and study cross sensor bias.
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)