Models. We evaluate the calibration of DistilBERT-base-uncased models [1]} that were finetuned on the Amazon Wilds training setWe use the weights provided in [2]}.. All models were fine-tuned using a similar setup (grid search over learning rate and objective-specific parameters, with the rest of the hyperparameters set to standard/default values; see Appendix E in [2]} for a detailed description). The models are standard ERM, IRM [4]} and GroupDRO [5]}.
