@inproceedings{
li2024masked,
title={Masked Random Noise for Communication-Efficient Federated Learning},
author={Shiwei Li and Yingyi Cheng and Haozhao Wang and Xing Tang and Shijie Xu and weihongluo and Yuhua Li and Dugang Liu and xiuqiang He and Ruixuan Li},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=x89a3eRwiO}
}
Federated learning is a promising distributed machine learning paradigm that can effectively protect data privacy. However, it may involve significant communication overhead, thereby potentially impairing training efficiency. In this paper, we aim to enhance communication efficiency from a new perspective. Specifically, we request the distributed clients to find optimal model updates relative to global model parameters within predefined random noise. For this purpose, we propose **Federated Masked Random Noise (FedMRN)**, a novel framework that enables clients to learn a 1-bit mask for each model parameter and apply masked random noise (i.e., the Hadamard product of random noise and masks) to represent model updates. To make FedMRN feasible, we propose an advanced mask training strategy, called progressive stochastic masking (*PSM*). After local training, clients only transmit local masks and a random seed to the server. Additionally, we provide theoretical guarantees for the convergence of FedMRN under both strongly convex and non-convex assumptions. Extensive experiments are conducted on four popular datasets. The results show that FedMRN exhibits superior convergence speed and test accuracy compared to relevant baselines, while attaining a similar level of accuracy as FedAvg.
Masked Random Noise for Communication-Efficient Federated Learning
Shiwei Li Yingyi Cheng Haozhao Wang Xing Tang Shijie Xu weihongluo Yuhua Li Dugang Liu xiuqiang He Ruixuan Li