This paper develops a Byzantine-robust decentralized stochastic optimization method over static and time-varying networks, where the networks are fully decentralized and contain unknown numbers of Byzantine agents, the local data samples at the regular agents are not necessarily i.i.d. and only one data sample (or a batch of them) is available for every regular agent at every iteration. The key idea is to formulate a total variation (TV) norm-penalized approximation of the Byzantine-free problem, where the penalty term forces the local models of regular agents to be close, but also allows the existence of outliers from the Byzantine agents. A stochastic subgradient method is applied to solve the penalized problem (Section ). Although the TV norm-penalized approximation has been investigated in Byzantine-robust decentralized deterministic [1]}, decentralized dynamic [2]} and centralized stochastic [3]} optimization problems, its application in Byzantine-robust decentralized stochastic optimization is novel. We prove that the proposed method reaches a neighborhood of the Byzantine-free optimal solution under mild assumptions, and the size of neighborhood is determined by the number of Byzantine agents and the network topology (Section ). Numerical experiments corroborate the theoretical analysis and demonstrate the robustness of the proposed method to Byzantine attacks (Section ).
