@inproceedings{
park2024deprune,
title={{DEP}rune: Depth-wise Separable Convolution Pruning for Maximizing {GPU} Parallelism},
author={Cheonjun Park and Mincheol Park and Hyunchan Moon and Myung Kuk Yoon and Seokjin Go and Suhyun Kim and Won Woo Ro},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=MYI443zCvv}
}
Depth-wise Separable Convolution (DSConv) has a powerful representation even with fewer parameters and computation, leading to its adoption by almost all of the state-of-the-art CNN models. 
DSConv models are already compact making it hard to apply pruning, and there are few  previous pruning techniques that target depth-wise convolution (DW-conv).
In this paper, we present Depth-wise Separable Convolution Pruning (DEPrune), a novel pruning method applied to both point-wise and depth-wise convolutions. 
DEPrune is optimized by analyzing the computation of DSConv on GPUs.
DEPrune employs a fine-grained pruning approach, yet it achieves the structured sparsity typically absent in fine-grained pruning, enabling practical hardware acceleration. 
Moreover, this method maintains a high pruning ratio without causing any accuracy drop.
We additionally represent techniques that further enhance DEPrune performance: 1) balanced workload tuning (BWT), and 2) hardware-aware sparsity recalibration (HSR).
Experiment results show that DEPrune achieves up to $3.74\times$ practical speedup in DSConv inference on GPUs while maintaining the accuracy of EfficientNet-B0 on ImageNet.
DEPrune: Depth-wise Separable Convolution Pruning for Maximizing GPU Parallelism
Cheonjun Park Mincheol Park Hyunchan Moon Myung Kuk Yoon Seokjin Go Suhyun Kim Won Woo Ro