1. In a system where an annotation guide is used to label utterances in speech data with a call type, a method for monitoring a labeler of the speech data, the method comprising: presenting, via a processor of a computing device, a test utterance to the labeler; generating a determination indicating whether the labeler correctly labeled the test utterance as one call type from a list of call types; and based on the determination, performing at least one of revising the annotation guide, retraining the labeler, and altering the test utterance.