1. A computer-implemented method for performing object recognition comprising: receiving image data; extracting a depth image and a color image from the image data; creating a mask image by segmenting the image data into a plurality of components; identifying objects within the plurality of components of the mask image; determining a first likelihood score from the depth image and the mask image using a layered classifier; determining a second likelihood score from the color image and the mask image by generating an object image by copying pixels from the first image of the components in the mask image and classifying the object image using the deep convolutional neural network (CNN); and determining a class for at least a portion of the image data based on the first likelihood score and the second likelihood score.