Data-free compression [1]}, [2]} has been an active research area when the original training dataset for the given pre-trained model is unavailable because of privacy or storage concerns. Given the pre-trained model to be compressed, it is an essential step to reconstruct the original training dataset by inverting representation. For example, accuracy degradation of ultra-low precision quantized models [3]}, [4]}, [5]} is unacceptable without fine-tuning on the reconstructed training dataset. The reconstruction method computes a reconstructed training dataset by leveraging some extra metadata [6]} or by extracting some prior information [7]} from the pre-trained model. Instead of computing the reconstructed training dataset directly [8]}, [2]}, [6]}, recent reconstruction methods [11]}, [12]}, [13]}, [4]}, [7]}, [1]} employ a generator to generate a reconstructed training dataset in an end-to-end manner and show better performance for data-free compression.
