The task of developing light-weight models for on-device application has drawn a lot of attention. A common methodology is to explore more efficient neural network architectures, e.g., MobileNet[1]}, ShuffleNet[2]}, EfficientNet[3]}, etc.
Another direction is to compress neural network models through parameter pruning[4]}, [5]}, quantization [6]}, [7]}, [8]}, and low-rank decomposition[9]}, [10]}, [11]}. The light-weight models developed with these approaches have small model size and low latency for inference. They are suitable for on-device deployment, but often at the expense of performance degradation.
