Consequences for theoretical research.
Our work has immediate consequences for theoretical research on differentially private
deep learning.
For a time, given the same algorithm, improvements to the analysis allowed
for researchers to obtain lower and lower values of \(\varepsilon \) .
Our results indicate this trend can continue no further.
We verify that the Moments Accountant [1]}, [2]} on DP-SGD as currently
implemented is tight.
As such, in order to provide better guarantees, either the DP-SGD algorithm will need
to be changed, or additional restrictions must be placed on the adversary.
