Pruning neural networks [1]} is in a sense the reverse operation to model completion. [2]} prune individual connections and [3]} prune entire feature maps using different techniques; their findings, lower layers are more important, are compatible with ours. [4]} present empirical evidence for the lottery ticket hypothesis: only a small subnetwork matters (the `lottery ticket') and the rest can be pruned away without loss of performance. The model completion problem for this lottery ticket (which is spread over all layers) would be trivial by definition. All of these works only consider removing parts of the model horizontally.
