AsyncFedED: Asynchronous Federated Learning with Euclidean Distance based Adaptive Weight Aggregation
In an asynchronous federated learning framework, the server updates the
global model once it receives an update from a client instead of waiting for
all the updates to arrive as in the synchronous setting. This allows
heterogeneous devices with varied computing power to train the local models
without pausing, thereby speeding up the training process. However, it
introduces the stale model problem, where the newly arrived update was
calculated based on a set of stale weights that are older than the current
global model, which may hurt the convergence of the model. In this paper, we
present an asynchronous federated learning framework with a proposed adaptive
weight aggregation algorithm, referred to as AsyncFedED. To the best of our
knowledge this aggregation method is the first to take the staleness of the
arrived gradients, measured by the Euclidean distance between the stale model
and the current global model, and the number of local epochs that have been
performed, into account. Assuming general non-convex loss functions, we prove
the convergence of the proposed method theoretically. Numerical results
validate the effectiveness of the proposed AsyncFedED in terms of the
convergence rate and model accuracy compared to the existing methods for three
considered tasks.