Hyper-parameters  For both CIFAR10 and MNIST, we used cross-entropy as our loss function.
We used four layers convolutional neural network [1]} for both MNIST and CIFAR10 datasets. For Purchase dataset, we used a three layers fully connected neural network. To train the models, we use differenatially private SGD [2]} with 0.15 learning rate for MNIST and CIFAR and 1.1 for Purchase datasets, clipping factor of 1.0 for MNIST and CIFAR and 0.05 for Purchase dataset. We repeat MNIST, CIFAR and Purchase experiments 1000 each and the malicious dataset 1,000,000 times. We evaluated each experiment using noise multipliers (DPSGD parameter) of \(0.5,0.6,0.9,1.0\)  and picked the one which gives us the best practical epsilon. For MNIST, we get accuracy of \(95\%\) ,\(96.0\%\) ,\(97\%\) ,\(98.5\%\) , on CIFAR we get \(53\%, 55\%, 56\%, 59\%\) , and on Purchase \(37\%, 82\%, 88\%, 90\%\)  for epsilons of \(1,2,4,10\)  respectively.
<FIGURE>