Pretrained large language models (LLMs)  exhibit exceptional general language processing capabilities but come with significant demands on memory and computational resources. As a powerful compression technology, binarization can extremely reduce model weights to a mere 1 bit, lowering the expensive computation and memory requirements. However, existing quantization techniques fall short of maintaining LLM performance under ultra-low bit-widths. In response to this challenge, we presentBiLLM, a groundbreaking 1-bit post-training quantization scheme tailored for pretrained LLMs. Based on the weight distribution of LLMs,BiLLMfirst identifies and structurally selects salient weights, and minimizes the compression loss through an effectivebinary residual approximationstrategy. Moreover, considering the bell-shaped distribution of the non-salient weights, we propose anoptimal splitting searchto group and binarize them accurately.BiLLMachieving for the first time high-accuracy inference (e.g. 8.41 perplexity on LLaMA2-70B)  with only1.08-bitweights across various LLMs families and evaluation metrics, outperforms SOTA quantization methods of LLM by significant margins. Moreover,BiLLMenables the binarization process of the LLM with 7 billion weights within 0.5 hours on a single GPU, demonstrating satisfactory time efficiency.
The code is available athttps://github.com/Aaronhuang-778/BiLLM.