@inproceedings{
pfohl2024understanding,
title={Understanding subgroup performance differences of fair predictors using causal models},
author={Stephen Robert Pfohl and Natalie Harris and Chirag Nagpal and David Madras and Vishwali Mhasawade and Olawale Elijah Salaudeen and Katherine A Heller and Sanmi Koyejo and Alexander Nicholas D'Amour},
booktitle={NeurIPS 2023 Workshop on Distribution Shifts: New Frontiers with Foundation Models},
year={2024},
url={https://openreview.net/forum?id=Fd00jISBD0}
}
A common evaluation paradigm compares the performance of a machine learning model across subgroups to assess properties related to fairness. In this work, we argue that distributional differences across subgroups can render this approach to evaluation of fairness misleading. We consider distributional differences across subgroups as a source of confounding that can lead to differences in performance metrics across subgroups even if the relationship between covariates and a label of interest is modeled as well as possible for each subgroup. We show that these differences in model performance can be anticipated and characterized based on the causal structure of the data generating process and the choices made during the model fitting procedure (e.g. whether subgroup membership is used as a predictor). We demonstrate how to construct alternative evaluation procedures that control for this source of confounding during evaluation by implicitly matching the distribution of confounding variables across subgroups. We emphasize that the selection of appropriate control variables requires domain knowledge and selection of contextually inappropriate control variables can produce misleading results.
Understanding subgroup performance differences of fair predictors using causal models
Stephen Robert Pfohl Natalie Harris Chirag Nagpal David Madras Vishwali Mhasawade Olawale Elijah Salaudeen Katherine A Heller Sanmi Koyejo Alexander Nicholas D'Amour