1. A method comprising: accessing a set of images which are associated with geographic locations proximate to a traffic intersection; for each particular image of the set of images, identifying, by one or more computing devices, brightly-colored red, yellow, and green objects within the particular image; generating, by the one or more computers, a set of traffic signal labels based on the identified brightly-colored red, yellow and green objects; filtering, by the one or more computers, the set of traffic signal labels to remove labels for brightly-colored red, yellow, or green objects that correspond to objects other than a traffic signal light; and determining, by the one or more computers, a three-dimensional location of a traffic signal based on the filtered set of traffic signal labels and the geographic locations of the images of the set of images.