We train a simple MLP and CNN on the MNIST data set.
The MLP has one fully-connected hidden layer of 128 neurons and uses ReLU [1]} activation.
The input images are flattened before providing them to the model.
The CNN has two 2D-convolutional layers with kernel size 3 \(\times \)  3, stride 2 and 128 filters, both using ReLU activation.
The fully-connected classification layer takes the flattened feature maps of the second convolutional layer as input.
During training, we use dropout and spatial dropout for fully-connected and convolutional layers, respectively, with a dropout rate of 0.5.
An overview of the architectures is given in the Appendix tab:models.
Both models are trained for 20 epochs using the Adam optimizer [2]} with default parameters and categorical cross-entropy as the loss function.
