Convolutional Neural Networks (CNNs) have been widely applied to many tasks and various devices in recent years. However, the network structures are becoming more and more complex, making the training of CNN on large scale datasets very time consuming, especially with limited hardware resources. Some previous researches have shown that CNN training could be finished within minutes on high performance computation platforms [1]}[2]}[3]}, but thousands of GPUs have to be utilized, which is not feasible for many scenarios. Even though there are many existing works on network compressing, most of them are focused on inference [4]}. Our work aims to reduce the training workloads efficiently, enabling large scale training on budgeted computation platforms.
