Though [1]} have shown that the CLT of GDP approximate the true privacy cost with negligible error,
[2]} recently discovers that the CLT of GDP may underestimate the privacy cost.
The RDP accountant [3]}, on the other hand, overestimates the true cost.
To evaluate the exact privacy cost, thereby, we plot the privacy cost curves for both the proposed extended CLT of GDP and RDP in Fig. REF . Specifically, we set a target training round and conduct privacy accounting for the dynamic DP-SGD based on the extended CLT of GDP as well as RDPFor RDP, we use autodp library by [4]} for DP accounting. with the advanced translation method between RDP and \((\epsilon , \delta )\) -DP [5]}.
Consequently, the true privacy cost curve must lie somewhere between these two limits. The result of extended CLT for GDP is reasonable because the privacy cost differences between these two limits are small, particularly in the high privacy protection region.
