Quantized Networks
Neural Networks that employ binary weights and activations, also known as binary neural networks (BNNs) [1]}, [2]}, [3]} have been shown to be promising for solving classification problems. Their efficacy has been demonstrated on MNIST [4]} and CIFAR10 [5]} datasets. The XNOR-Net [6]} used a gain term computed from the statistics of the weights to scale the binary weights in order to reduce the quantization error. The XNOR-Net performed relatively better than previous BNN architectures on the ImageNet [7]} dataset. The learned quantization networks (LQ-Net) [8]} was proposed to minimize the quantization error by jointly learning the quantizer and the network. The Bi-Real Network [9]} employs residual connections where full-precision activations from the previous layer affect the quantization in the current layer. On the optimization front [10]} justified the straight-through-estimator (STE) method used for training the BNNs through bayesian learning. The quantization of weights has also been shown to be promising in the context of generative adversarial networks (GANs) [11]}, [12]}. [13]} adapts Neural Architecture Search (NAS) to find an optimized architecture for the binarization of Convolutional Neural Networks (CNNs). [14]} showed that quantized neural networks are generally robust relative to their full precision counterpart against adversarial perturbations.
While the role of quantization has been considered in classification and generation problems, it has been relatively unexplored in the context of optimizing unfolded networks for sparse recovery.
