Abadi et al. [1]} first introduced DP-SGD and the momentum accountant to analyze the privacy bounds. Since then, many works [2]}, [3]}, [4]}, [5]}, [6]}, [7]} improved the analysis of the DP-SGD to get better theoretical privacy bounds. As mentioned before, we used the Rényi Differential privacy (RDP) [5]} to compute the theoretical privacy bounds. Recently Dong et. al. [2]} introduced Gaussian differential privacy (GDP) to show a tight approach to compute the privacy bounds. However, both approaches result in similar privacy bounds. Using the hyperparameters for the MNIST dataset (sampling rate \(\frac{256}{60000}\)  and 60 epochs), we compute the final privacy costs using both Rényi and Gaussian differential privacy which is shown in the Figure REF . As we can see both results in comparable privacy bounds and are within our measurement error bounds.
<FIGURE>