Pruning methods.
Depending on when to perform pruning, current methods can generally be divided into three groups [1]}: i) prune pretrained models [2]}, [3]}, [4]}, [5]}, [6]}, [7]}, [8]}, ii) prune at initialization [9]}, [10]}, [11]}, and iii) prune during training [12]}, [13]}, [14]}. Despite notable progresses in the later two streams, pruning pretrained models remains as the most popular paradigm, with structural sparsity favored by off-the-shelf inference platforms such as GPU.
