Several modifications of FedAvg have been proposed to handle non-IID settings. These variants include using a decreasing learning rate [1]}; modifying device empirical loss dynamically [2]}; or modifying server side updates [3]}, [4]}. Methods that use a decreasing learning rate or customized server side updates still rely on local SGD updates within devices. While these works do recognize the incompatibility of local and global stationary points, their proposed fix is based on inexact minimization. Additionally, in order to establish convergence for non-IID situations, these works impose additional “bounded-non-IID” conditions.
