Friends to Help: Saving Federated Learning from Client Dropout
Federated learning (FL) is an outstanding distributed machine learning
framework due to its benefits on data privacy and communication efficiency.
Since full client participation in many cases is infeasible due to constrained
resources, partial participation FL algorithms have been investigated that
proactively select/sample a subset of clients, aiming to achieve learning
performance close to the full participation case. This paper studies a passive
partial client participation scenario that is much less well understood, where
partial participation is a result of external events, namely client dropout,
rather than a decision of the FL algorithm. We cast FL with client dropout as a
special case of a larger class of FL problems where clients can submit
substitute (possibly inaccurate) local model updates. Based on our convergence
analysis, we develop a new algorithm FL-FDMS that discovers friends of clients
(i.e., clients whose data distributions are similar) on-the-fly and uses
friends' local updates as substitutes for the dropout clients, thereby reducing
the substitution error and improving the convergence performance. A complexity
reduction mechanism is also incorporated into FL-FDMS, making it both
theoretically sound and practically useful. Experiments on MNIST and CIFAR-10
confirmed the superior performance of FL-FDMS in handling client dropout in FL.