Guided by the rules of DPSDA, we design two differentially private distributed online learning algorithms. One employs the circulation-based protocol [1]} for undirected communication (named as DPSDA-C), while the other utilizes the push-sum method [2]} for the directed communication (named as DPSDA-PS). To be more general, the dual averaging steps of both algorithms use the noise-corrupted gradients, i.e., the gradients are evaluated with some random error, instead of the exact gradients. This arises from numerous applications as well, including distributed learning over the network and recursive regression. It is important, but not trivial, to extend this since the stochastic gradient error of each node is spread to other nodes in real time through the communication, making the dynamics statistically dependent on time and nodes.
