Recall Distortion in Neural Network Pruning and the Undecayed Pruning Algorithm
Pruning techniques have been successfully used in neural networks to trade
accuracy for sparsity. However, the impact of network pruning is not uniform:
prior work has shown that the recall for underrepresented classes in a dataset
may be more negatively affected. In this work, we study such relative
distortions in recall by hypothesizing an intensification effect that is
inherent to the model. Namely, that pruning makes recall relatively worse for a
class with recall below accuracy and, conversely, that it makes recall
relatively better for a class with recall above accuracy. In addition, we
propose a new pruning algorithm aimed at attenuating such effect. Through
statistical analysis, we have observed that intensification is less severe with
our algorithm but nevertheless more pronounced with relatively more difficult
tasks, less complex models, and higher pruning ratios. More surprisingly, we
conversely observe a de-intensification effect with lower pruning ratios.