Prior work has often observed this phenomenon, and for example trained
models with \(\varepsilon =10^5\) -DP [1]}
despite this offering almost no theoretical privacy, because
empirically the privacy appeared to be strong.
However, as we have already revealed in the introduction,
it turns out that Interpretation 2 is correct:
while, for this weak attack, the bounds are loose,
this does not imply DP-SGD itself is loose when an adversary utilizes all capabilities.
