The use of deep neural networks (DNNs) on computing hardware such as mobile and IoT devices with limited computational and memory resources is becoming increasingly important. This has led to a growing area of research focused on reducing the model size and inference time of DNNs; in this area, the overall goal is to keep the loss of accuracy below an acceptable level compared to floating-point implementations. These methods include, for example, model pruning, neural architecture search, novel efficient architecture design, and low-rank decomposition.
In this work, we focus on network quantization, where weights and activations are quantized to lower bit widths, allowing for efficient fixed-point inference and reduced memory bandwidth usage; see, for example, [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}, [8]} and references therein. Quantized neural networks (QNNs) have attracted many research efforts. Nevertheless, the challenge of closing the accuracy gap between full-precision and quantized networks remains open, especially for extremely low-precision arithmetics (e.g. binary). The task of learning a quantized neural network (QNN) can be formulated as minimising the training loss with quantization constraints on the weights, i.e.,
\(\min _{w\in \mathcal {Q}} \ell (w) \,, \, \ell (w)= \mathbb {E}_{(x, y) \sim p_{\operatorname{data }}}[\ell (f(x, w), y)],\) 
