Data-free compression covers data-free quantization and data-free knowledge distillation. Without a generator, the reconstructed training dataset is computed directly in [1]}, [2]}, [3]}, [4]}, [5]}. [1]} present a method for data-free knowledge distillation, where the reconstructed training dataset is computed based on some extra recorded activations statistics from the pre-trained model. DeepInversion [5]} introduces a feature map regularizer based on batch normalization information in the pre-trained model for data-free knowledge distillation. In data-free knowledge distillation [2]}, the class similarities are computed from the pre-trained model and the output space is modeled via Dirichlet Sampling. [3]} calculates the reconstructed training dataset to match the statistics of the batch normalization layers of the pre-trained model and introduces the Pareto frontier to enable mixed-precision quantization. [4]} improves data-free quantization by equalizing the weight ranges and correcting the biased quantization error.
