1. A method implemented by one or more processing devices, comprising: determining, for an image, a first set of labels, each label being determined to be indicative of subject matter of the image based on content feature values of the image; for each label in the first set of labels: querying, using the label in the first set of labels as a query, an association of labels that are associated with each other based on a degree of semantic relatedness; determining a second set of labels that are responsive to the label in the first set of labels, each label in the second set of labels determined to be semantically related to the label in the first set of labels based on a degree of semantic relatedness and having a confidence score, wherein: the confidence score indicates a confidence that the label in the second set of labels is semantically related to the label in the first set of labels; and the confidence score is independent of the label in the second set of labels and the label in the first set of labels being returned for a same image; assigning a score to each label in the second sets of labels based on the confidence score of the respective label in the second set of labels; and based on the scores assigned to each label in the second sets of labels, assigning one or more of the labels in the second sets of labels to the image.