Our work is the first to develop theoretical understanding of empirical studies of pruning methods in feed forward neural networks. Based on our theoretical results, we proposed DPP edge pruning which outperforms comparable pruning methods on real data. Our work consolidates the understanding of diversity-based node and edge pruning both theoretically and empirically. When comparing two neural networks, using the number of parameters may not always be the optimal choice, instead, measuring the capacity and expressiveness of neural networks [1]} can provide new insights. Finally, our theoretical and empirical observations can motivate informed and insightful use of pruning methods(especially DPP based), on large networks like CNNs and RNNs in future work.
