1. A method for interactively ranking image search results by using a color layout relevance, implemented at least in part by a computing device, the method comprising: extracting color layouts for images after the images are crawled from the internet; receiving a text query for an image search; evaluating a similarity between an interest color layout and a color layout of any image in an image search results, wherein the similarity is based on a color appearance compatibility, a spatial distribution consistency, and a contextual layout similarity, wherein the color appearance compatibility, the spatial distribution consistency, and the contextual layout similarity are based on a color similarity and a layout component-wise similarity; and reordering the image search results based on evaluated similarities between color layouts of images and the interest color layout from the user according to a selection of a color selection palette and a color layout specification scheme.