An Empirical Study of Personalized Federated Learning
Federated learning is a distributed machine learning approach in which a
single server and multiple clients collaboratively build machine learning
models without sharing datasets on clients. A challenging issue of federated
learning is data heterogeneity (i.e., data distributions may differ across
clients). To cope with this issue, numerous federated learning methods aim at
personalized federated learning and build optimized models for clients. Whereas
existing studies empirically evaluated their own methods, the experimental
settings (e.g., comparison methods, datasets, and client setting) in these
studies differ from each other, and it is unclear which personalized federate
learning method achieves the best performance and how much progress can be made
by using these methods instead of standard (i.e., non-personalized) federated
learning. In this paper, we benchmark the performance of existing personalized
federated learning through comprehensive experiments to evaluate the
characteristics of each method. Our experimental study shows that (1) there are
no champion methods, (2) large data heterogeneity often leads to high accurate
predictions, and (3) standard federated learning methods (e.g. FedAvg) with
fine-tuning often outperform personalized federated learning methods. We open
our benchmark tool FedBench for researchers to conduct experimental studies
with various experimental settings.