Achieving Utility, Fairness, and Compactness via Tunable Information Bottleneck Measures
Designing machine learning algorithms that are accurate yet fair, not
discriminating based on any sensitive attribute, is of paramount importance for
society to accept AI for critical applications. In this article, we propose a
novel fair representation learning method termed the R\'enyi Fair Information
Bottleneck Method (RFIB) which incorporates constraints for utility, fairness,
and compactness of representation, and apply it to image classification. A key
attribute of our approach is that we consider - in contrast to most prior work
- both demographic parity and equalized odds as fairness constraints, allowing
for a more nuanced satisfaction of both criteria. Leveraging a variational
approach, we show that our objectives yield a loss function involving classical
Information Bottleneck (IB) measures and establish an upper bound in terms of
the R\'enyi divergence of order $\alpha$ on the mutual information IB term
measuring compactness between the input and its encoded embedding.
Experimenting on three different image datasets (EyePACS, CelebA, and
FairFace), we study the influence of the $\alpha$ parameter as well as two
other tunable IB parameters on achieving utility/fairness trade-off goals, and
show that the $\alpha$ parameter gives an additional degree of freedom that can
be used to control the compactness of the representation. We evaluate the
performance of our method using various utility, fairness, and compound
utility/fairness metrics, showing that RFIB outperforms current
state-of-the-art approaches.