Commit
•
f50d898
1
Parent(s):
900a2e2
Update README with quantisation info.
Browse files
README.md
CHANGED
|
@@ -9,7 +9,25 @@ tags:
|
|
| 9 |
- merge
|
| 10 |
license: gemma
|
| 11 |
---
|
| 12 |
-
# Gemma-2-Ataraxy-9B
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
|
| 14 |
|
| 15 |
|
|
|
|
| 9 |
- merge
|
| 10 |
license: gemma
|
| 11 |
---
|
| 12 |
+
# Gemma-2-Ataraxy-9B-exl2
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
This repository contains various EXL2 quantisations of [lemon07r/Gemma-2-Ataraxy-9B](https://huggingface.co/lemon07r/Gemma-2-Ataraxy-9B).
|
| 16 |
+
|
| 17 |
+
Quantisations available:
|
| 18 |
+
| Branch | Description | Recommended |
|
| 19 |
+
| ------ | ------ | ------ |
|
| 20 |
+
| [2.0-bpw](https://huggingface.co/CameronRedmore/Gemma-2-Ataraxy-9B-exl2/tree/2.0-bpw) | 2 bits per weight | Low Quality - Smallest Available Quantisation |
|
| 21 |
+
| [3.0-bpw](https://huggingface.co/CameronRedmore/Gemma-2-Ataraxy-9B-exl2/tree/3.0-bpw) | 3 bits per weight | |
|
| 22 |
+
| [4.0-bpw](https://huggingface.co/CameronRedmore/Gemma-2-Ataraxy-9B-exl2/tree/4.0-bpw) | 4 bits per weight | ✔️ - Recommended for Low-VRAM Environments |
|
| 23 |
+
| [5.0-bpw](https://huggingface.co/CameronRedmore/Gemma-2-Ataraxy-9B-exl2/tree/5.0-bpw) | 5 bits per weight | |
|
| 24 |
+
| [6.0-bpw](https://huggingface.co/CameronRedmore/Gemma-2-Ataraxy-9B-exl2/tree/6.0-bpw) | 6 bits per weight | ✔️ - Best Quality / VRAM Balance |
|
| 25 |
+
| [6.5-bpw](https://huggingface.co/CameronRedmore/Gemma-2-Ataraxy-9B-exl2/tree/6.5-bpw) | 6.5 bits per weight | ✔️ - Near Perfect Quality, Slightly Higher VRAM Usage |
|
| 26 |
+
| [8.0-bpw](https://huggingface.co/CameronRedmore/Gemma-2-Ataraxy-9B-exl2/tree/8.0-bpw) | 8.0 bits per weight | Best Available Quality - Almost always unnecessary |
|
| 27 |
+
|
| 28 |
+
---
|
| 29 |
+
# Original README:
|
| 30 |
+
---
|
| 31 |
|
| 32 |
|
| 33 |
|