There are some works that consider the Byzantine-robust centralized stochastic optimization problem, where a central controller aggregates the information from the agents and coordinates the optimization process. The main idea of these works is to modify the stochastic gradient method with robust aggregation rules. To be specific, at every iteration, the central controller sends the current model to all the agents, the regular agents send back their local stochastic gradients, while the Byzantine agents may send back faulty values. When the local data samples are independently and identically distributed (i.i.d.), the local stochastic gradients are also i.i.d. and the central controller can obtain a reliable approximation to the average of the local stochastic gradients through aggregating all the received values with trimmed mean, geometric median, or other robust aggregation rules [1]}, [2]}. However, this idea is not directly applicable to decentralized stochastic optimization. Since there is no central controller to maintain a common model, the regular agents have to evaluate their local stochastic gradients at different points. Therefore, even though the local data samples are i.i.d. the local stochastic gradients are not necessarily so, and thus the robust aggregation rules have no theoretical guarantee in this case.
