1. A method for maintaining associations between profiles and objects in an object recognition system, wherein a profile comprises a set of identifying information extracted from an image set associated with the profile, comprising the computer-implemented steps of: receiving a first image set; generating a stored profile based on the first image set; causing to be displayed, to a user, first data that identifies one or more objects that are automatically determined by the object recognition system to be possible matches to the stored profile; wherein the one or more objects includes a particular object that is associated with a particular stored profile; after the first data is displayed to the user, receiving, from the user, feedback that indicates that the particular object does not match the stored profile; in response to receiving the feedback from the user, updating a running score that is (a) maintained for the particular stored profile and (b) based on how often the object recognition system identified the particular object as a possible match but a user indicated the particular object was not a match; receiving a second image set; generating a second stored profile based on the second image set; determining a confidence score in a subsequent possible match that involves the particular stored profile and the second stored profile; weighting the confidence score based, at least in part, on the running score; causing to be displayed, to a user, second data that identifies one or more second objects that are automatically determined by the object recognition system to be possible matches to the second stored profile, wherein the object recognition system determines whether to include the particular object in the one or more second objects based on the weighted confidence score; wherein the method is performed by one or more computing devices.