1. A computer-implemented method for selecting a subset of a set of comments associated with a group of documents, the method comprising: accessing, at memory locations, the set of comments and a predetermined annotation probability distribution of annotations for another set of comments associated with another group of documents, wherein the annotation probability distribution specifies biases in the annotations for the other set of comments; and using a computer processor that is coupled to the memory location and programmed to select the subset: selecting the subset based on how informative expected annotations for the comments in the subset are for a classifier and probabilities of occurrence of the expected annotations based on the predetermined annotation probability distribution, wherein the classifier predicts how likely the expected annotations are accurate for the comments in the subset.