(ii) DP-SGD with the CLT of GDP Acountant [1]}: GDP, a recently developed DP accounting framework, provides a simple, explicit, and tight privacy accounting CLT bound for the classical DP-SGD with evenly calibrated noise power. It is proved by [2]} that the CLT of GDP provides a tighter composition bound for DP-SGD than the moment accountant method [3]}. As a result, it serves as a standard for our dynamic DP-SGD methods.
