Figure REF  shows an overview of the results in the paper. We target a specific theoretical epsilon for DPSGD for different datasets. Since all of the training datasets has between \(10^4\)  and \(10^5\)  instances, we choose \(\delta =10^{-5}\)  which means we can leak one instance from the training dataset.
This setting of \(\delta \)  is typical for DP-SGD [1]}.
The dataset attack can achieve a practical epsilon very close to the theoretical bounds. This result suggest that if an adversary has access to all of the assumptions in DPSGD then the theoretical analysis is tight and the researchers should focus more on relaxing some of the assumptions in the analysis. The gradient attack results are also close to the theoretical bounds which means the adversary does not need to modify the dataset. Going from having access to gradient to only input, we can see a noticeable drop in the empirical privacy parameters. When the adversary has access to intermediate step models (e.g., in federated learning) it can still achieve a close privacy parameters close to theoretical values. However, when the adversary does not maliciously modify the training dataset, gradients or input the practical epsilon is much lower than the theoretical ones.
