[noitemsep,topsep=0pt,parsep=0pt,partopsep=0pt,leftmargin=*]
SqueezeNet. RNS-CKKS-based PPNNs [1]}, [2]} adopt the architecture of SqueezeNet [3]} for fast private inferences on CIFAR-10 images. SqueezeNet achieves similar accuracy to AlexNet using 50\(\times \)  fewer parameters. SqueezeNet replaces conventional convolution layers by fire modules shown in Figure REF (b). Assume a conventional convolution layer has the dimension of \(I_i\times 3 \times 3 \times O_i \) , where \(I_i\)  is the input channel number, 3 is the weight filter size, and \(O_i\)  are the output channel number in the \(i\) -th layer. On the contrary, a fire module consists of stacked convolution layers. \(C_i\)  (\(<O_i\) ) is the output channel number of the first convolutional layer. \(O_{i}^{0}+O_{i}^{1}=O_i\)  is enforced to make the \(i_{th}\)  layer output have \(O_i\)  channels. If the \(HW\)  batching technique is used, to compute a regular convolution layer, we need to do \(\mathcal {O}(I_i\times 3 \times 3 )\)  rotations, and \(\mathcal {O}(I_i\times 3 \times 3\times O_i)\)  HE multiplications. In contrast, to compute a fire module of SqueezeNet, we need to perform \(\mathcal {O}(I_i\times 1 +C_i\times (3 \times 3+1))\)  rotations, and \(\mathcal {O}(I_i \times 1 \times 1 \times C_i+ C_i \times 1 \times O_i^0+ C_i \times 3 \times 3\times O_i^1)\)  HE multiplications. However, fire modules of SqueezeNet greatly increase the layer number of the mobile neural network architecture.

InceptionNet. InceptionNet [4]} achieves VGG-like accuracy using \(\sim \) 7\(\times \)  fewer parameters. As Figure REF (c) shows, InceptionNet is built through substituting conventional convolution layers with inception modules. An inception module is composed of four types of filters, each of them has output channels \(O_{i}^{0\sim 3}\) . Although inception modules greatly reduce model parameters, InceptionNet has to almost double the layer number, compared to the VGG CNN.

