We use a 4-layer deep CNN for performing our experiments on MNIST dataset. The reason for using such a small CNN for MNIST is that the dataset contains grayscale images and performing feature extraction on such images is quite easier in comparison to RGB images.
For SVHN dataset,
we utilized the ResNet-18 architecture [1]}.
we use learning rates 0.01 and 0.05 for DP-SGD and PATE, respectively. We train PATE with 50 teachers and 30 students. We train each model 5 times and report the mean and standard deviation of accuracy in our experiments.
