Correlated and Individual Multi Modal Deep Learning for RGB D Object Recognition
In this paper, we propose a new correlated and individual multi modal deep learning (CIMDL) method for RGB D object recognition. Unlike most conventional RGB D object recognition methods which extract features from the RGB and depth channels individually, our CIMDL jointly learns feature representations from raw RGB D data with a pair of deep neural networks, so that the sharable and modal specific information can be simultaneously exploited. Specifically, we construct a pair of deep convolutional neural networks (CNNs) for the RGB and depth data, and concatenate them at the top layer of the network with a loss function which learns a new feature space where both correlated part and the individual part of the RGB D information are well modelled. The parameters of the whole networks are updated by using the back propagation criterion. Experimental results on two widely used RGB D object image benchmark datasets clearly show that our method outperforms state of the arts.
ArXiv