The prohibitive energy cost of running high-performance Convolutional Neural Networks (CNNs) has been limiting their deployment on resource-constrained platforms including mobile and wearable devices. We propose a CNN for energy-aware dynamic routing, called the EnergyNet, that achieves adaptive-complexity inference based on the inputs, leading to an overall reduction of run time energy cost without noticeably losing (or even improving) accuracy. That is achieved by proposing an energy loss that captures both computational and data movement costs. We combine it with the accuracy-oriented loss, and learn a dynamic routing policy for skipping certain layers in the networks, that optimizes the hybrid loss.   Our empirical results demonstrate that, compared to the baseline CNNs, EnergyNetcan trim down the energy cost up to 40% and 65%, during inference on the CIFAR10 and Tiny ImageNet testing sets, respectively, while maintaining the same testing accuracies.   It is further encouraging to observe that the energy awareness might serve as a training regularization and can even improve prediction accuracy:  our models can achieve 0.7% higher top-1 testing accuracy than the baseline on CIFAR-10 when saving up to 27% energy, and 1.0% higher top-5 testing accuracy on Tiny ImageNet when saving up to 50% energy, respectively. While deep learning-powered Internet of Things (IoT) devices promise to dramatically revolutionize the way we live and work by enhancing our ability to recognize, analyze, and classify the world around us, this revolution has yet to be unleashed due to many fundamental challenges. Edge devices, such as smart phones, smart sensors, drones and robots, have limited energy and computation resources since they are battery-powered and have a small form factor. On the other hand, high-performance Convolutional Neural Networks (CNNs) come at a cost of prohibitive energy consumption BID0 . The CNNs with the highest accuracy have hundreds of layers and tens of millions of parameters. When deployed in practice, such networks drain the battery very quickly BID1 . Recently, there have been a number of methods proposed to reduce energy cost in CNNs, while not hampering their predictive power. Most of them aim to reduce the model size or the number of computations BID2 BID3 BID4 BID5 BID6 BID7 BID8 BID9 BID10 BID11 . However, BID1 shows that a smaller model size and fewer operations might not necessarily lead to a lower energy cost. BID1 uses energy cost to guide the pruning process, where the layer with the highest energy cost is pruned first. BID12 formulates the CNN training process as an optimization problem under a certain energy budget constraint. While both methods BID1 BID12 show promising results towards pursuing more energy-efficient CNN models, they do not incorporate energy costs into the training loss function to explicitly learn a more energy-efficient model. Furthermore, once their model structures are learned from training, it can only be fixed during the inference time, and there is no room for input-dependent adaptivity. This paper proposes a new CNN model that combines energy cost with a dynamic routing strategy to enable adaptive energy-efficient inference. Our proposed model, termed as EnergyNet, is a gated CNN architecture which employs conditional computing to route the input data through the network Figure 1 : EnergyNet Structure: each green circle G indicates an RNN gate and each blue square under G indicates one block of layers in the base model. To reduce the energy cost, the RNN gates generate routing strategies dynamically for different input images. By sharing the parameters between all RNN gates, they will have only 0.04% of the energy cost of the base CNN model, which is negligible. In this specific example, only the first and third blocks get executed.in an efficient path. Built on a base network (such as ResNet-34 or ResNet-50 BID13 ), EnergyNet uses an additional gating network BID10 to decide whether the current input should skip certain layers in the network or not. It optimizes a weighted combination of an accuracy loss and an energy loss which captures both the computational and memory data movement costs, under which EnergyNet is trained to find the optimal routing policy to reduce the energy cost of the model without degrading the prediction accuracy. Our empirical results demonstrate that, compared to the base network without gating nor dynamic inference, EnergyNet can trim down the energy cost up to 40% and 65%, during inference on the CIFAR10 and Tiny ImageNet testing sets, respectively, while maintaining almost the same testing accuracy. Interestingly enough, we find the energy-aware EnergyNet can even achieve win-win, by simultaneously improving the prediction accuracy and saving energy, potentially due to its equivalent effect as a training regularization to avoid overfitting. For example, our models achieve 0.7% higher top-1 testing accuracy than the baseline on CIFAR-10 when saving up to 27% energy, and 1.0% higher top-5 accuracy on Tiny ImageNet when saving up to 50% energy, respectively.
This paper proposes a new CNN model that combines energy cost with a dynamic routing strategy to enable adaptive energy-efficient inference.
{"src": "scitldr-AIC_train", "paper_id": "Syxp2bgKoX"}