There are two major methodologies in neural net compression i) structured pruning and knowledge distillation. The existing work on structured pruning [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}, [8]}, [9]}, [10]}, [11]} addresses only the width of the layer by removing filters based on their importance. However, selecting what to prune from large CNNs is an NP-Hard problem, to find the optimal solution one would need to rank each filter by turning it off and perform inference using all the samples. Thus, if we are to prune network structures like filters we would need to try out all possibilities of filter configurations and choose the best one that has the least effect on the loss. This problem is addressed in literature by a sparsity loss and learnable dropout parameters.
