Many researchers have put much effort into addressing the expensive communication issue.
Adaptive model aggregation methods adjust the aggregation interval to reduce the total communication cost ([1]}, [2]}).
Gradient (model) compression ([3]}, [4]}), sparsification ([5]}, [6]}), low-rank approximation ([7]}, [8]}), and quantization ([9]}, [10]}, [11]}) techniques directly reduce the local data size.
Employing heterogeneous model architectures across clients is also a communication-efficient approach ([12]}).
While all these works effectively tackle the expensive communication issue from different angles, they commonly assume the underlying periodic full model aggregation.
