Model compression aims to reduce the bitwidth and exploit the redundancy and sparsity inherent in neural networks to reduce memory and latency. Han et al. [1]} first showed the concept of pruning, quantization, and model compression jointly in the context of pre-trained deep neural networks (DNN). Pruning [2]} refers to masking redundant weights (i.e., weights lying within a certain activation interval) and representing them in a row form. The network is then retrained to update the weights for other connections. Quantization [3]} accelerates DNN inference latency by rounding off weights to reduce bit width while clustering the similar ones for weight sharing. Encoding (also known as model compression or weight clustering) represents common weights with fewer bits, either through conversion of dense matrices to sparse matrices [1]} or smaller dense matrices through parameter redundancies [5]}. Combining the three techniques can drastically reduce the size of state-of-the-art DNN such as AlexNet (35\(\times \) , 6.9 MB), LeNet-5 (39\(\times \) , 44 kB), LeNet-300-100 (40\(\times \) , 27 kB), and VGG-16 (49\(\times \) , 11.3 MB) without losing accuracy [1]}.
