We perform experiments on the CIFAR-10, CelebA [1]}, and Waterbird [2]} datasets. CIFAR-10 is a standard benchmark for multi-label classification with 10 different classes that we considered as both target and group variables. CelebA contains over 200 thousand facial images with various attribute annotations; similarly to [2]}, we use CelebA as a binary classification dataset by predicting the “blond” label. We used the Cartesian product of “blond” and “binary gender” as the group label,Note that we use the “gender” binary label as a group descriptor for testing purposes, without attempting to ascribe the broader diversity of gender identities to this limiting notion. giving 4 groups total. The Waterbird dataset is a subset from the CUB image dataset [4]}, where the goal is to predict the type of bird (“waterbird” or “landbird”). As in [2]}, we used the Cartesian product of the bird class and the image background ("land" or "water"), leading to a total of 4 groups. We use ResNet-18 and ResNet-34 architectures [6]} as our base classifiers in all cases, and use SGD with cosine annealing learning rate [7]}. Details are provided in Appendix REF .
