Fairness Transferability Subject to Bounded Distribution Shift
Given an algorithmic predictor that is "fair" on some source distribution,
will it still be fair on an unknown target distribution that differs from the
source within some bound? In this paper, we study the transferability of
statistical group fairness for machine learning predictors (i.e., classifiers
or regressors) subject to bounded distribution shift, a phenomenon frequently
caused by user adaptation to a deployed model or a dynamic environment. Herein,
we develop a bound characterizing such transferability, flagging potentially
inappropriate deployments of machine learning for socially consequential tasks.
We first develop a framework for bounding violations of statistical fairness
subject to distribution shift, formulating a generic upper bound for
transferred fairness violation as our primary result. We then develop bounds
for specific worked examples, adopting two commonly used fairness definitions
(i.e., demographic parity and equalized odds) for two classes of distribution
shift (i.e., covariate shift and label shift). Finally, we compare our
theoretical bounds to deterministic models of distribution shift as well as
real-world data.