TITLE: LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasets 

ABSTRACT: Federated learning is a popular distributed machine learning paradigm with
enhanced privacy. Its primary goal is learning a global model that offers good
performance for the participants as many as possible. The technology is rapidly
advancing with many unsolved challenges, among which statistical heterogeneity
(i.e., non-IID) and communication efficiency are two critical ones that hinder
the development of federated learning. In this work, we propose LotteryFL -- a
personalized and communication-efficient federated learning framework via
exploiting the Lottery Ticket hypothesis. In LotteryFL, each client learns a
lottery ticket network (i.e., a subnetwork of the base model) by applying the
Lottery Ticket hypothesis, and only these lottery networks will be communicated
between the server and clients. Rather than learning a shared global model in
classic federated learning, each client learns a personalized model via
LotteryFL; the communication cost can be significantly reduced due to the
compact size of lottery networks. To support the training and evaluation of our
framework, we construct non-IID datasets based on MNIST, CIFAR-10 and EMNIST by
taking feature distribution skew, label distribution skew and quantity skew
into consideration. Experiments on these non-IID datasets demonstrate that
LotteryFL significantly outperforms existing solutions in terms of
personalization and communication cost.
Federated learning is a popular distributed machine learning paradigm with
enhanced privacy. Its primary goal is learning a global model that offers good
performance for the participants as many as possible. The technology is rapidly
advancing with many unsolved challenges, among which statistical heterogeneity
(i.e., non-IID) and communication efficiency are two critical ones that hinder
the development of federated learning. In this work, we propose LotteryFL -- a
personalized and communication-efficient federated learning framework via
exploiting the Lottery Ticket hypothesis. In LotteryFL, each client learns a
lottery ticket network (i.e., a subnetwork of the base model) by applying the
Lottery Ticket hypothesis, and only these lottery networks will be communicated
between the server and clients. Rather than learning a shared global model in
classic federated learning, each client learns a personalized model via
LotteryFL; the communication cost can be significantly reduced due to the
compact size of lottery networks. To support the training and evaluation of our
framework, we construct non-IID datasets based on MNIST, CIFAR-10 and EMNIST by
taking feature distribution skew, label distribution skew and quantity skew
into consideration. Experiments on these non-IID datasets demonstrate that
LotteryFL significantly outperforms existing solutions in terms of
personalization and communication cost.
LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasets
http://arxiv.org/abs/2008.03371v1