Mitigating Data Heterogeneity in Federated Learning with Data Augmentation
Federated Learning (FL) is a prominent framework that enables training a
centralized model while securing user privacy by fusing local, decentralized
models. In this setting, one major obstacle is data heterogeneity, i.e., each
client having non-identically and independently distributed (non-IID) data.
This is analogous to the context of Domain Generalization (DG), where each
client can be treated as a different domain. However, while many approaches in
DG tackle data heterogeneity from the algorithmic perspective, recent evidence
suggests that data augmentation can induce equal or greater performance.
Motivated by this connection, we present federated versions of popular DG
algorithms, and show that by applying appropriate data augmentation, we can
mitigate data heterogeneity in the federated setting, and obtain higher
accuracy on unseen clients. Equipped with data augmentation, we can achieve
state-of-the-art performance using even the most basic Federated Averaging
algorithm, with much sparser communication.