1. A method of identifying potential phishing abuse images, comprising: generating a plurality of base color palettes, each representing a subset of color values within a different base image of a plurality of base images; generating a target color palette that represents a subset of color values within a target image; for each base color palette of the plurality of base color palettes: comparing the target color palette with said each base color palette to generate a score, associating the score with the base image that corresponds to said each base color palette, and determining whether the score exceeds a threshold; comparing the target image with a base image only if the score associated with the base image exceeds the threshold; wherein the method is performed by one or more computing devices.