"You Can't Fix What You Can't Measure": Privately Measuring Demographic Performance Disparities in Federated Learning
Federated learning allows many devices to collaborate in the training of
machine learning models. As in traditional machine learning, there is a growing
concern that models trained with federated learning may exhibit disparate
performance for different demographic groups. Existing solutions to measure and
ensure equal model performance across groups require access to information
about group membership, but this access is not always available or desirable,
especially under the privacy aspirations of federated learning. We study the
feasibility of measuring such performance disparities while protecting the
privacy of the user's group membership and the federated model's performance on
the user's data. Protecting both is essential for privacy, because they may be
correlated, and thus learning one may reveal the other. On the other hand, from
the utility perspective, the privacy-preserved data should maintain the
correlation to ensure the ability to perform accurate measurements of the
performance disparity. We achieve both of these goals by developing locally
differentially private mechanisms that preserve the correlations between group
membership and model performance. To analyze the effectiveness of the
mechanisms, we bound their error in estimating the disparity when optimized for
a given privacy budget, and validate these bounds on synthetic data. Our
results show that the error rapidly decreases for realistic numbers of
participating clients, demonstrating that, contrary to what prior work
suggested, protecting the privacy of protected attributes is not necessarily in
conflict with identifying disparities in the performance of federated models.