* _INT8 weight quantization can often yield very good results on LLMs, while the performance of lower precision weight quantization depends on specific methods_[414, 416, 417, 421]. In most cases, INT8 weight quantization can be effectively applied to reduce the memory footprint without performance degradation. While for INT4 (or INT3) weight quantization, existing methods rely on specific strategies to reduce the performance degradation, _e.g.,_ layerwise method [415, 417], activation-aware scaling [416] and low-rank adapter tuning [419]. Interestingly, LLMs seem to be less sensitive to low-bit weight quantization than small-sized language models [421]. In practice, with the same memory cost, it is suggested to use a larger language model with a lower quantization precision rather than a smaller language model with a higher quantization precision. For example, a 4-bit 60GB LLM is demonstrated to have better performance than a 8-bit 30GB LLM [422]. Moreover, focusing on emergent capabilities, the study [423] finds that in-context learning, step-by-step reasoning, and instruction following all seem to be seldom affected with 4-bit weight quantization. This result suggests that INT4 quantization exhibits a favorable trade-off in terms of both total bits and performance of emergent abilities. * _Activations are more difficult to be quantized than weights_[413, 414, 421]. It has been found that large outliers would occur for Transformer language models having a size of 6.7B or above [413]. This issue has been one of the most fundamental difficulties to quantize LLMs. To overcome this issue, various methods, _e.g.,_ mixed-precision decomposition [413], fine-grained quantization [413, 425] and difficulty migration [414], can be applied to alleviate the influence of outlier values. Since large outliers mainly exist in the activations of LLMs, small language models are more resistant to activation quantization [421, 423]. In practice, high-quality INT8 activation quantization is still a difficult task, though several methods can attain satisfying results. Further, lower precision activation quantization has still not been successfully explored, even for QAT methods [420].