
Informally, this definition requires the probability any adversary can observe
a difference between the algorithm operating on dataset \(D\)  versus a neighboring dataset \(D^{\prime }\) 
is bounded by \(e^\varepsilon \) , plus a constant additive failure rate of \(\delta \) .
To provide a meaningful theoretical guarantee, \(\varepsilon \)  is typically set to
small single digit numbers, and \(\delta \ll 1/|D|\) .
Other variants of DP use slightly different formulations (e.g., Rényi
differential privacy [1]} and concentrated DP [2]}, [3]}); our paper studies predominantly this
\((\varepsilon ,\delta )\) -definition of DP as it is the most common. Furthermore, it is often possible to translate a property achieved under one formulation to another, in fact
the DP-SGD analysis uses a Rényi bound as an intermediate step to achieving the \((\varepsilon ,\delta )\) 
bounds.
