Privacy Accounting
Each step in the DP-SGD process depletes the total privacy budget.
Following \(T\)  steps of training, DP accounting necessitates the composition of these mechanisms in order to calculate the total privacy cost in terms of \((\epsilon , \delta )\) .
Simple composition [1]}, [2]} with \(\epsilon =\mathcal {O}(T)\)  and advance composition [3]} with \(\epsilon =\mathcal {O}(\sqrt{T})\)  have been proposed in the literature.
If \(M_{1}, M_{2}, \ldots , M_{T}\)  are distinct DP mechanisms such that \(M_{i}\)  is \(\left(\epsilon _{i}, \delta _{i}\right)\) -DP, then it is shown by [4]} that computing the exact DP guarantees for the composition \(M=M_{1} \circ M_{2} \circ \cdots \circ M_{T}\)  is #P-complete.
[5]} propose the moment accountant of DP-SGD. By establishing a privacy cost function, the moment accountant imposes a tight constraint on the estimation of privacy loss, calibrating the noise power down to a practicable level for the first time in the privacy preserving deep learning context.
From a broader perspective, [6]} proposes to use Rényi divergence to determine the distance between the outputs of two adjacent datasets, thereby establishing the RDP concept, and [7]} provide an advance conversion from RDP to \((\epsilon ,\delta )\) -DP.
Due to the inherent use of subsampling in training neural networks, the DP composition with subsampling DP amplification is studied in detail by [8]}, [7]}, which reduce the noise power for the privacy preserving.
[10]} recently propose the concept of f-DP to quantify the privacy cost from a hypothesis testing perspective, with Gaussian DP (GDP) as a major application.
While GDP allows for a tight composition, computing the exact composition of the Gaussian mechanism with subsampling amplification is computationally challenging. It is also demonstrated that a computationally efficient central limit theorem (CLT) can approximate the composition of multiple identical DP mechanisms.
Very recently, [11]}, [12]} propose optimal composition of DP mechanisms via numerical methods, but it is unclear how to calibrate different noise power and clipping value at each step as in the proposed dynamic DP-SGD.
