**Quantized LLMs**. Compared with original models, quantized language models take a smaller memory footprint, and likely have a faster inference speed [427, 93, 413]. Recently, a number of quantized model copies of several publicly available language models have been released on HuggingFace, including BLOOM, GPT-J, and ChatGLM. In particular, GPTQ [417] has been widely used to quantize generative language models, leading to various quantized variants for LLMaM and OPT. Further, it has been also applied to quantize instruction-tuned models, such as Vicuna and WizardLM. Due to the large number of quantized LLMs, we do not directly incorporate the corresponding links of these models. The readers can easily find them by searching on HuggingFace.