Commit
•
38bcb59
1
Parent(s):
269a2cb
Update README.md
Browse files
README.md
CHANGED
|
@@ -45,6 +45,14 @@ Quatizations provided by [TheBloke](https://huggingface.co/TheBloke/laser-dolphi
|
|
| 45 |
|
| 46 |
*Current AWQ [Quantizations](https://huggingface.co/macadeliccc/laser-dolphin-mixtral-2x7b-dpo-AWQ)
|
| 47 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 48 |
## HF Spaces
|
| 49 |
+ GGUF chat available [here](https://huggingface.co/spaces/macadeliccc/laser-dolphin-mixtral-chat-GGUF)
|
| 50 |
+ 4-bit bnb chat available [here](https://huggingface.co/spaces/macadeliccc/laser-dolphin-mixtral-chat)
|
|
|
|
| 45 |
|
| 46 |
*Current AWQ [Quantizations](https://huggingface.co/macadeliccc/laser-dolphin-mixtral-2x7b-dpo-AWQ)
|
| 47 |
|
| 48 |
+
# ExLlamav2
|
| 49 |
+
|
| 50 |
+
Thanks to user [bartowski](https://huggingface.co/bartowski) we now have exllamav2 quantizations in 3.5 through 8 bpw. They are available here:
|
| 51 |
+
|
| 52 |
+
+ [bartowski/laser-dolphin-mixtral-2x7b-dpo-exl2](https://huggingface.co/bartowski/laser-dolphin-mixtral-2x7b-dpo-exl2)
|
| 53 |
+
|
| 54 |
+
His quantizations represent the first ~13B model with GQA support. Check out his repo for more information!
|
| 55 |
+
|
| 56 |
## HF Spaces
|
| 57 |
+ GGUF chat available [here](https://huggingface.co/spaces/macadeliccc/laser-dolphin-mixtral-chat-GGUF)
|
| 58 |
+ 4-bit bnb chat available [here](https://huggingface.co/spaces/macadeliccc/laser-dolphin-mixtral-chat)
|