Asynchronous Hierarchical Federated Learning
Federated Learning is a rapidly growing area of research and with various
benefits and industry applications. Typical federated patterns have some
intrinsic issues such as heavy server traffic, long periods of convergence, and
unreliable accuracy. In this paper, we address these issues by proposing
asynchronous hierarchical federated learning, in which the central server uses
either the network topology or some clustering algorithm to assign clusters for
workers (i.e., client devices). In each cluster, a special aggregator device is
selected to enable hierarchical learning, leads to efficient communication
between server and workers, so that the burden of the server can be
significantly reduced. In addition, asynchronous federated learning schema is
used to tolerate heterogeneity of the system and achieve fast convergence,
i.e., the server aggregates the gradients from the workers weighted by a
staleness parameter to update the global model, and regularized stochastic
gradient descent is performed in workers, so that the instability of
asynchronous learning can be alleviated. We evaluate the proposed algorithm on
CIFAR-10 image classification task, the experimental results demonstrate the
effectiveness of asynchronous hierarchical federated learning.