1. In a system where an annotation guide is used to label utterances in speech data with a call type, a method for monitoring labelers of the speech data, the method comprising: presenting via a processor a test utterance to a labeler; receiving input from the labeler that selects a particular call type from a list of call types; determining via the processor if the labeler labeled the test utterance correctly; and based on the determining step, performing at least one of: revising the annotation guide, retraining the labeler or altering the test utterance.