1. A system, comprising: an interface circuit that, during operation, communicates, via a network, with another electronic device; a processor, and memory, coupled to the processor, which stores a program module that, during operation, is executed by the processor, the program module including: instructions for receiving input data including an image; instructions for identifying, using a configurable detector associated with labeled data, a feature in the image and an associated candidate tag based on user-defined items of interest, and determining an associated accuracy metric based on the feature and the candidate tag, wherein the user-defined items of interest include one or more features and one or more associated tags; instructions for selectively obtaining, using a scheduler, feedback, having a feedback accuracy, about the candidate tag from a set of individuals based on the accuracy metric, a cost of requesting the feedback and a feedback threshold, wherein the scheduler dynamically selects a given individual in the set of individuals based on a given cost of requesting the feedback from the given individual; instructions for generating a revised tag based on the feedback when the feedback indicates the candidate tag is incorrect; instructions for presenting a result to the other electronic device, wherein the result includes one of: the feature and the candidate tag when the accuracy metric exceeds a detection threshold; and the feature and the revised tag when the feedback accuracy exceeds the detection threshold; and instructions for updating, based on a quality metric, the labeled data to include the input data, the feature and one of the candidate tag and the revised tag, wherein the updated labeled data are to be used to retrain the configurable detector.