FL Games: A federated learning framework for distribution shifts
Federated learning aims to train predictive models for data that is
distributed across clients, under the orchestration of a server. However,
participating clients typically each hold data from a different distribution,
whereby predictive models with strong in-distribution generalization can fail
catastrophically on unseen domains. In this work, we argue that in order to
generalize better across non-i.i.d. clients, it is imperative to only learn
correlations that are stable and invariant across domains. We propose FL Games,
a game-theoretic framework for federated learning for learning causal features
that are invariant across clients. While training to achieve the Nash
equilibrium, the traditional best response strategy suffers from high-frequency
oscillations. We demonstrate that FL Games effectively resolves this challenge
and exhibits smooth performance curves. Further, FL Games scales well in the
number of clients, requires significantly fewer communication rounds, and is
agnostic to device heterogeneity. Through empirical evaluation, we demonstrate
that FL Games achieves high out-of-distribution performance on various
benchmarks.