@inproceedings{martynov-etal-2024-way,
    title = "On the Way to Lossless Compression of Language Transformers: Exploring Cross-Domain Properties of Quantization",
    author = "Martynov, Nikita  and
      Goncharov, Aleksei  and
      Kumichev, Gleb  and
      Egorov, Evgeniy  and
      Pavlov, Stanislav Vladimirovich  and
      Durinov, Mikhail Sergeevich  and
      Zuev, Aleksandr Sergeevich  and
      Filimonov, Egor Anatolievich",
    editor = "Calzolari, Nicoletta  and
      Kan, Min-Yen  and
      Hoste, Veronique  and
      Lenci, Alessandro  and
      Sakti, Sakriani  and
      Xue, Nianwen",
    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.lrec-main.1089",
    pages = "12435--12442",
    abstract = "Modern Transformers achieved impressive results on various Natural Language Processing tasks over the last few years. The one downside of this success is the size of these models. Huge capacity, which sometimes surpasses billions of parameters, improves generalization abilities, but makes it difficult to employ. Developing field of model compression seeks to reduce the model size and inference latency. This research focuses on one of the compression techniques {---} Post-Training Quantization. We present a methodology to effectively quantize at least 95{\%} of Transformer weights and corresponding activations to INT8 without any access to task-specific data so the drop in performance does not exceed 0.02{\%}. Furthermore, we provide intriguing observations that reflect cross-domain nature of some of the quantization properties.",
}

Modern Transformers achieved impressive results on various Natural Language Processing tasks over the last few years. The one downside of this success is the size of these models. Huge capacity, which sometimes surpasses billions of parameters, improves generalization abilities, but makes it difficult to employ. Developing field of model compression seeks to reduce the model size and inference latency. This research focuses on one of the compression techniques {---} Post-Training Quantization. We present a methodology to effectively quantize at least 95{\%} of Transformer weights and corresponding activations to INT8 without any access to task-specific data so the drop in performance does not exceed 0.02{\%}. Furthermore, we provide intriguing observations that reflect cross-domain nature of some of the quantization properties.
Martynov, Nikita Goncharov, Aleksei Kumichev, Gleb Egorov, Evgeniy Pavlov, Stanislav Vladimirovich Durinov, Mikhail Sergeevich Zuev, Aleks r Sergeevich Filimonov, Egor Anatolievich
On the Way to Lossless Compression of Language Transformers: Exploring Cross-Domain Properties of Quantization