Learning Depth Guided Convolutions for Monocular 3D Object Detection
3D object detection from a single image without LiDAR is a challenging task due to the lack of accurate depth information. Conventional 2D convolutions are unsuitable for this task because they fail to capture local object and its scale information, which are vital for 3D object detection. To better represent 3D structure, prior arts typically transform depth maps estimated from 2D images into a pseudo LiDAR representation, and then apply existing 3D point cloud based object detectors. However, their results depend heavily on the accuracy of the estimated depth maps, resulting in suboptimal performance. In this work, instead of using pseudo LiDAR representation, we improve the fundamental 2D fully convolutions by proposing a new local convolutional network (LCN) termed Depth guided Dynamic Depthwise Dilated LCN (D4LCN) where the filters and their receptive fields can be automatically learned from image based depth maps, making different pixels of different images have different filters. D4LCN overcomes the limitation of conventional 2D convolutions and narrows the gap between image representation and 3D point cloud representation. Extensive experiments show that D4LCN outperforms existing works by large margins. For example, the relative improvement of D4LCN against the state of the art on KITTI is 9.1% in the moderate setting. D4LCN ranks 1st on KITTI monocular 3D object detection benchmark at the time of submission (car, December 2019) The code is available at https:/github.com/dingmyu/D4LCN.
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)