* _Efficient fine-tuning enhanced quantization_. For post-training quantization, direct low-bit quantization (_e.g.,_ INT4 quantization) often results in large performance degradation. To overcome this challenge, QLoRA [419] incorporates additional small tunable adapters (16-bit precision) into the quantized models, to achieve an efficient, high-precision model fine-tuning. It combines the merits of LoRA (See Section 5.3.1) and quantization methods. The experiment results show that 4-bit quantized models can achieve the full 16-bit fine-tuning performance by QLoRA. * _Quantization-aware training (QAT) for LLMs_. A recent study [420] explores the effect of QAT methods by applying a data-free distillation method to compress the weights, activations as well as key-value cache. By conducting extensive experiments based on LLaMA, they show promising results with 4-bit quantization on both weights and key-value cache, but not on 4-bit activation quantization, which still needs more exploration.