We chose the AffectNet dataset [1]} to train our models for two reasons. First, it is the largest labelled dataset for facial expressions, containing a total of 440,601 labelled images in eight categories: neutral, happy, sad, surprise, fear, disgust, anger, and contempt. Second, it seems to be a difficult dataset to improve on: it was collected by web-based crawling methods, and each image was labeled by one, though expert, human annotator. Therefore, the AffectNet dataset contains in-the-wild images that may be mislabelled or vague. Specifically, out of 36,000 images that were annotated by two human annotators to calculate agreement, their agreement was only 60.7 percent. Moreover, the dataset is highly imbalanced: the largest class, happy, contains 143,991 images while the smallest class, contempt, contains only 5,119 images. This imbalance reflects the real-world proportion of expression occurrences; one does observe happy expressions more often than one observes contempt.
