In Section  we show that DiNNO is an extremely effective method for distributed training of neural network models. We compare DiNNO against two other commonly referenced stochastic first order distributed optimization methods: distributed stochastic gradient descent (DSGD) [1]}, and distributed stochastic gradient tracking (DSGT) [2]}. Like DiNNO, both methods have each robot maintain a local copy of the optimization variable (neural network weights), and use message passing and locally computed stochastic gradients to collaboratively optimize the neural network. DSGD uses the following simple update equation
\(\theta _i^{k + 1} = \underset{j \in \mathcal {V}}{\sum } w_{ij} \theta _j^{k} - \alpha ^{k} g(\theta _i^{k}) \) 
