DP-SGD [1]} is a modification of the stochastic gradient descent algorithm which provides provable privacy guarantees. DP-SGD bounds the sensitivity of each gradient and is paired with a moments accountant algorithm to amplify and track the privacy loss across weight updates. Moments accountant significantly improves on earlier privacy analysis of SGD and allows for meaningful privacy guarantees for deep learning trained on realistically sized datasets [2]}.
