Personalized Federated Learning via Variational Bayesian Inference
Federated learning faces huge challenges from model overfitting due to the
lack of data and statistical diversity among clients. To address these
challenges, this paper proposes a novel personalized federated learning method
via Bayesian variational inference named pFedBayes. To alleviate the
overfitting, weight uncertainty is introduced to neural networks for clients
and the server. To achieve personalization, each client updates its local
distribution parameters by balancing its construction error over private data
and its KL divergence with global distribution from the server. Theoretical
analysis gives an upper bound of averaged generalization error and illustrates
that the convergence rate of the generalization error is minimax optimal up to
a logarithmic factor. Experiments show that the proposed method outperforms
other advanced personalized methods on personalized models, e.g., pFedBayes
respectively outperforms other SOTA algorithms by 1.25%, 0.42% and 11.71% on
MNIST, FMNIST and CIFAR-10 under non-i.i.d. limited data.