Together, Theorem REF , REF  and Conjecture REF  are consistent with empirical observations of  [1]}: sparse networks after edge pruning perform better on the unseen test data than dense networks after node pruning with fixed number of parameters. To the best of our knowledge,  [1]} based their claims from empirical observations of pruning studies in which the pruned networks were not reweighted. This motivated our choice of comparing GE for DPP node pruning and random edge pruning without any reweighting. However, with reweighting from [3]}, GE of DPP node pruning will be less than GE of random edge pruning, highlighting the impact of reweighting proposed by [3]} (proof and details in [proof-of-theorems]appendix C). We find that GE analysis on teacher-student setup is not only flexible for various pruning methods but also produces results that are consistent with empirical observations on real data (Section REF ). This framework can be extended to theoretically understand other pruning methods which are outside the scope of this work. Yet, in its current form, it can only be used to compare pruning methods where networks are not retrained after pruning.
<FIGURE>