Update README.md
Browse files
README.md
CHANGED
|
@@ -34,15 +34,16 @@ state of the art AI models and helping foster innovation for everyone.
|
|
| 34 |
## Quantizon types
|
| 35 |
| quantization method | bits | size | description | recommended |
|
| 36 |
|---------------------|------|----------|-----------------------------------------------------|-------------|
|
| 37 |
-
| Q3_K_S | 3 |
|
| 38 |
-
| Q3_K_L | 3 |
|
| 39 |
-
| Q4_0 | 4 |
|
| 40 |
-
| Q4_K_M | 4 |
|
| 41 |
-
| Q5_0 | 5 |
|
| 42 |
-
| Q5_K_S | 5 |
|
| 43 |
-
| Q5_K_M | 5 |
|
| 44 |
-
| Q6_K | 6 |
|
| 45 |
-
| Q8_0 | 8 |
|
|
|
|
| 46 |
|
| 47 |
## Usage
|
| 48 |
You can use this model with the latest builds of LM Studio and llama.cpp.
|
|
|
|
| 34 |
## Quantizon types
|
| 35 |
| quantization method | bits | size | description | recommended |
|
| 36 |
|---------------------|------|----------|-----------------------------------------------------|-------------|
|
| 37 |
+
| Q3_K_S | 3 | 3.68 GB | very small, high quality loss | ❌ |
|
| 38 |
+
| Q3_K_L | 3 | 4.4 GB | small, substantial quality loss | ❌ |
|
| 39 |
+
| Q4_0 | 4 | 4.81 GB | legacy; small, very high quality loss | ❌ |
|
| 40 |
+
| Q4_K_M | 4 | 5.13 GB | medium, balanced quality | ✅ |
|
| 41 |
+
| Q5_0 | 5 | 5.88 GB | legacy; medium, balanced quality | ❌ |
|
| 42 |
+
| Q5_K_S | 5 | 5.88 GB | large, low quality loss | ✅ |
|
| 43 |
+
| Q5_K_M | 5 | 6.04 GB | large, very low quality loss | ✅ |
|
| 44 |
+
| Q6_K | 6 | 7.01 GB | very large, extremely low quality loss | ❌ |
|
| 45 |
+
| Q8_0 | 8 | 9.08 GB | very large, extremely low quality loss | ❌ |
|
| 46 |
+
| FP16 | 16 | 17.1 GB | enormous, negligible quality loss | ❌ |
|
| 47 |
|
| 48 |
## Usage
|
| 49 |
You can use this model with the latest builds of LM Studio and llama.cpp.
|