
where \(\tilde{x} \in {\mathbb {R}}^p\)  is an optimization variable (also called as model), \(F(\tilde{x},\xi _i)\)  is a smooth cost function determined by a random variable \(\xi _i\)  following a distribution \(\mathcal {D}_i\)  and represents an empirical loss related to a randomly chosen data sample in regular agent \(i\) , and \(f_0(\tilde{x})\)  is a smooth regularization term. Instead of the i.i.d. assumption as in [1]}, [2]}, here the random variables \(\lbrace \xi _i, i \in \mathcal {R}\rbrace \)  are only assumed to be mutually independent. We will also assume \(\mathbb {E}[F(\tilde{x}, \xi _i)] + f_0(\tilde{x})\)  to be strongly convex in Assumption REF  in Section . Our goal is to find the optimal solution \(\tilde{x}^*\)  through collaboration of the regular agents. The main challenges are three-fold: (i) the network lacks a central coordinator and is fully decentralized, (ii) only one randomly chosen data sample (or a batch of them) can be used by every regular agent at every iteration, and, (iii) more importantly, the Byzantine agents can send faulty values to their neighbors so as to bias the optimization process, but their identities are unknown.
