Semi-Decentralized Federated Learning with Collaborative Relaying
We present a semi-decentralized federated learning algorithm wherein clients
collaborate by relaying their neighbors' local updates to a central parameter
server (PS). At every communication round to the PS, each client computes a
local consensus of the updates from its neighboring clients and eventually
transmits a weighted average of its own update and those of its neighbors to
the PS. We appropriately optimize these averaging weights to ensure that the
global update at the PS is unbiased and to reduce the variance of the global
update at the PS, consequently improving the rate of convergence. Numerical
simulations substantiate our theoretical claims and demonstrate settings with
intermittent connectivity between the clients and the PS, where our proposed
algorithm shows an improved convergence rate and accuracy in comparison with
the federated averaging algorithm.