Figure REF  summarizes our key results.
When our adversary is given full capabilities, our lower bound
matches the provable upper bound, demonstrating the DP-SGD analysis is tight in the worst-case.
In the past, more sophisticated analysis techniques (without making new assumptions)
were able to establish better and
better upper bounds on essentially the same DP-SGD algorithm [1]}.
This meant that we could train a model once, and over time its privacy would “improve” (so to speak),
as progress in theoretical work on differential privacy yielded a tighter analysis of the same algorithm used to train the model.
Our lower bound implies that this trend has come to an end, and new
assumptions will be necessary to establish tighter bounds.
