1. A method implemented by data processing apparatus, the method comprising: obtaining landmark location data for an image, the landmark location data specifying one or more landmark locations derived from content depicted in the image and, for each landmark location, a landmark confidence score that indicates a measure of confidence that the image depicts content associated with the landmark location; obtaining caption location data for the image, the caption location data specifying one or more caption locations derived from user input and, for each caption location, a caption confidence score that indicates a measure of confidence that the image depicts content associated with the caption location; obtaining metadata location data for the image, the metadata location data specifying a metadata location derived from data provided by an image capturing device that captured the image; identifying location pairs from the landmark location data, caption location data, and metadata location data, each location pair including two of a landmark location, caption location, and metadata location; generating, for each location pair, a geographic consistency score that indicates a measure of consistency between locations specified in the location pair; ranking each location pair according to a predetermined hierarchy of location types, the predetermined hierarchy specifying a first priority for landmark locations, a second priority for caption locations, and third priority for metadata locations, wherein the ranking of each location air is based on a highest priority of the locations specified in the location pair; selecting a location pair based at least in part on the geographic consistency scores and location pair rankings; selecting an image location for the image from the locations specified in the selected location pair; determining an image location score for the image location based on a confidence score for one of the locations specified in the selected location pair; and associating the image location and image location score with the image.