1. A computer-implemented method, comprising: receiving a plurality of image search results that are responsive to a search query, each image search result referencing an image that has been identified as responsive to the search query based on a relevance score for the image search result, the relevance score being determined independent of a visual similarity of the image to other images referenced by the image search results; for a particular image search result: receiving, in a data processing apparatus, image similarity data for the particular image search result, the image similarity data being indicative of a visual similarity between the image referenced by the particular image search result and other images that are referenced by the plurality of image search results; generating, in a data processing apparatus, an adjustment factor based on the relevance score for the particular image search result and the image similarity data for the particular image search result, the adjustment factor specifying a quality measure for the particular image search result relative to the search query; determining that the particular image search result is a co-relevant result, the co-relevant result being a responsive image search result for which: the relevance score meets a specified relevance score threshold; and the image similarity data indicates that the image referenced by the responsive image search result has a least a threshold level of visual similarity to a threshold number of the images referenced by the image search results; and scaling the adjustment factor for the particular image search result by an amplification factor in response to determining that the particular image search result is a co-relevant result.