The authors in [1]} proposed FedProx, which augments a proximal term to the local training problem to counter possible divergence resulting from statistical heterogeneity. Additionally, this method accommodates system heterogeneity by allowing variable amount of work at each selected client. Nevertheless, the performance of FedProx is sensitive to the selection of the proximal coefficient, whose tuning depends on prior knowledge on statistical distributions and system sizes [1]}.
SCAFFOLD [3]} introduces client and server control variates, which effectively act as tracking variables for local and global information. It is reported to outperform FedAvg/Prox through utilizing the similarity of data across clients, but its communication cost per round is doubled due to the extra control variates.
