Clients feel reluctant to upload their sensitive data, e.g., health or financial records, to untrusted servers in the cloud. To protect clients' privacy, privacy-preserving neural networks (PPNNs) [1]}, [2]}, [3]}, [4]}, [5]} are built to perform inferences directly on encrypted data. An interactive PPNN [1]}, [2]} uses homomorphic encryption (HE) for linear layers, and adopts secure multi-party computation (MPC) to process activation layers. However, huge volumes of data between the client and the server have to be exchanged during an inference of an interactive PPNN. For instance, DELPHI [2]} has to transmit 2GB data for only a ResNet-32 inference on a single encrypted CIFAR-10 image. On the contrary, non-interactive PPNNs [3]}, [5]}, [11]} approximate their activations by a degree-2 polynomial, and compute an entire inference via HE. They do not require high network bandwidth, but still can obtain competitive inference accuracy [11]}. Thereafter (except Section REF ), when we mention a PPNN, we indicate a non-interactive PPNN.
