1)
Network-A: a 3-layer Deep Neural Network mentioned in SecureML[1]}. (1) A 28\( \times \) 28 fully connected layer, and the activation functions following this layer are ReLU. (2) A 128 fully conneted layer, and the activation fuctions following this layer are ReLU. (3) A 128 fully connected layer, the activation functions are ReLU, and Softmax is used to get a probability distribution.
2)
Network-B: a 4-layer Convolutional Neural Network from MiniONN[2]}.(1) A 2-dimensional convolutional layer with 1 input channel, 16 output channels and a 5\( \times \) 5 filter. The activation functions following this layer are ReLU, followed by a 2\( \times \) 2 Maxpool. (2) A 2-dimensional convolutional layer with 16 input channels, 16 output channels and 5\( \times \) 5 filter.The activation functions following this layer are ReLU and a 2\( \times \) 2 Maxpool followed. (3) A 256\( \times \) 100 fully connected layer, the activation functions are ReLU.(4) A 100\(\times \) 10 fully connected layer, the activation functions are ReLU, and Softmax is used to get a probability distribution.
3)
Network-C: a 4-layer Convolutional Neural Metwork called LeNet network[3]}. (1) A 2-dimensional convolutional layer with 1 input channel, 20 output channels and a 5\( \times \) 5 filter. The activation functions following this layer are ReLU, followed by a 2\( \times \) 2 Maxpool. (2) A 2-dimensional convolutional layer with 20 input channels, 50 output channels and another 5\( \times \) 5 filter. The activation functions following this layer are ReLU and a 2\( \times \) 2 Maxpool followed. (3) An 800\( \times \) 500 fully connected layer. The activation functions are ReLU. (4) A 500\( \times \) 10 fully connected layer, the activation functions are ReLU, and Softmax is used to get a probability distribution.
4)
Network-D: a 3-layer Convolutional Neural Metwork from Chameleon[4]}. (1) a 2-dimensional convolutional layer with a 5\( \times \) 5 filter, stride of 2, and 5 output channels. The activation functions are ReLU. (2) A 980 fully conneted layer, and the activation fuctions following this layer are ReLU. (3) A 100 fully connected layer, the activation functions are ReLU, and Softmax is used to get a probability distribution.

