The seminal work of [1]} proposed differential private stochastic gradient descent (DP-SGD) algorithm to achieve differential privacy guarantees for deep learning models. Specifically, in each step of the training, DP-SGD adds appropriate noise to the \(\ell _2\) -clipped gradients during the stochastic gradient descent optimization. The incurred privacy budget \(\varepsilon \)  for training is computed using the moment's accountant technique that keeps track of the privacy loss across multiple invocations of the noise addition mechanism applied to random subsets of the input dataset [1]}.
