1. A method implemented by an information handling system that includes a memory and a processor, the method comprising: training a classifier on a first set of images, wherein the classifier comprises a convolutional neural network that includes a plurality of layers; processing a second set of images using the trained classifier, wherein the processing comprises: extracting a plurality of feature sets from a selected layer included in the plurality of layers, wherein the plurality of layers includes a set of convolutional layers and a subsequent set of fully connected non-convolutional layers, and wherein the selected layer is a first layer in the subsequent set of fully-connected non-convolutional layers; and generating a plurality of tags based upon metadata corresponding to the second set of images; and assigning at least one of the plurality of tags to a new image based upon evaluating a new image feature set against one or more of the plurality of feature sets, wherein the new image feature set is extracted from the selected layer during processing of the new image.