In this work we presented an idea of comprehensive pruning of neural networks which leverages from the idea of dropout and residual shortcut connections. We propose pruning at 4 different granularities i.e. filter, layers, branch (sequence of layers) and blocks (multi-branch blocks such as ResBlock, Inception-ResNet [1]} or those found in multi-task learning). Our pruning strategy is based on introducing learnable scaling factors at these granularities, these parameters are learned in an online manner during training and helps in generating a pruned network. In our case instead of using filter pruning as a proxy for layer pruning, we instead model pruning at different granularities by introducing separate sparsity loss for each of the granularity.
