| ### Full Perplexity Comparison Table for Release Documentation | |
| | Quantization Type | PPL(Q) | ln(PPL(Q)/PPL(fp16)) | File Size (G) | | |
| |-------------------|---------|---------------------|---------------| | |
| | IQ2_S | 25.3893 | 0.501266 | 1.6 | | |
| | IQ2_M | 21.6684 | 0.342794 | 1.6 | | |
| | Q3_K_M | 16.8567 | 0.091687 | 1.8 | | |
| | IQ3_M | 16.774 | 0.086769 | 1.7 | | |
| | Q3_K_L | 16.5067 | 0.070705 | 1.8 | | |
| | IQ4_NL | 15.9602 | 0.037037 | 1.9 | | |
| | IQ4_XS | 15.9591 | 0.036968 | 1.8 | | |
| | Q4_K_S | 15.9346 | 0.035431 | 1.9 | | |
| | Q4_K_M | 15.8651 | 0.031060 | 2.0 | | |
| | Q5_K_S | 15.4901 | 0.007140 | 2.1 | | |
| | Q5_K_M | 15.4746 | 0.006139 | 2.2 | | |
| | Q6_K | 15.3961 | 0.001053 | 2.4 | | |
| | Q8_0 | 15.3831 | 0.000208 | 2.7 | | |
| | bf16 | 15.3799 | 0.000000 | 4.2 | | |
| --- | |
| This full table documents all the quantization types tested, showing their respective **Perplexity (PPL)**, **ln(PPL(Q)/PPL(fp16))**, and **file sizes**. | |