(i) SGD without DP: The SGD method serve as the upper bound of model inference accuracy in the absence of clipping and additive noise.
Please note that the DP-SGD and proposed dynamic DP-SGD can also be used to obtain the DP-Adam counterparts of an Adam optimizer due to DP's postprocessing properties [1]}. For the IMDB data set, we apply the Adam to compute the performance upper bound, and DP-Adam counterparts for comparison.
