We tested our approach on VGG-16 [1]} and ResNet-56 [1]} models which are commonly used in neural network compression literature. In our experiments we used a custom VGG-16 model with three changes i) we used \(0.05\)  L2 Norm applied to every trainable layer to fight weight inflation problem caused by the network to mitigate the effect of shrinking scaling factors, ii) we added BatchNorm layer after each trainable layer to increase training speed and iii) we used a Dropout of 0.5 after 1st FC layer to prevent overfitting. ResNet-56 is chosen as it is a multi-branched network with ResBlocks on one branch and residual connection on the other branch.
