DNN models can be trained by well-known frameworks such as PyTorch [1]},
TensorFlow [2]}, and MXNet [3]}. These frameworks achieve
efficient training by taking advantage of well-optimized libraries.
However, those libraries are only tuned for a narrow range of environments with sufficient resources such as GPUs.
These frameworks will not run as well on edge devices, such as smartphones, AR/VR devices, and industrial equipment, or
even less powerful servers, due to limited computational resources and memory. Improving computational capability is often
prohibitively expensive or impractical, be it from material costs, thermal issues, or even power consumption related costs or
environment-friendliness. Inference optimization is thus an important task; with sufficiently fast inference, those devices can be used practically,
without additional computation capability.
