1. A color feature extracting device, comprising: a main color extracting unit determining an object area of an input image corresponding to a size and a location of an object, dividing the object area and a non-object area of the input image into a plurality of rectangles, generating a color histogram for each color channel, and extracting a main color of an object rectangle from the color histogram; a weight calculating unit calculating a minimum distance between a color histogram bin of the object rectangle and a color histogram bin of a non-object rectangle with respect to the main color, a bin weight of the color histogram of the object rectangle, a color channel weight of the object rectangle, and a weight of the object rectangle to extract a discriminative color feature of the object; and a final color model generating unit generating a final color model of the object based on a result of re-weighting the color histogram bins of the object rectangle.