Update README.md
Browse files
README.md
CHANGED
|
@@ -3,6 +3,8 @@ license: llama2
|
|
| 3 |
---
|
| 4 |
EXL2 quant of alpindale/goliath-120b (https://huggingface.co/alpindale/goliath-120b), to be used on exllamav2. 4.25bpw to being to able to use CFG comfortably on 72GB VRAM. (20,21,22 for gpu split)
|
| 5 |
|
|
|
|
|
|
|
| 6 |
Calibration dataset is a cleaned, fixed pippa RP dataset, which does affect the results (in favor) for RP usage.
|
| 7 |
|
| 8 |
You can find the calibration dataset [here](https://huggingface.co/datasets/royallab/PIPPA-cleaned)
|
|
|
|
| 3 |
---
|
| 4 |
EXL2 quant of alpindale/goliath-120b (https://huggingface.co/alpindale/goliath-120b), to be used on exllamav2. 4.25bpw to being to able to use CFG comfortably on 72GB VRAM. (20,21,22 for gpu split)
|
| 5 |
|
| 6 |
+
Update 06/01/2024: Updated with new quant method after some time, thanks for the measurement [here](https://github.com/turboderp/exllamav2/files/13846439/goliath-120b-rpcal-measurement.json)
|
| 7 |
+
|
| 8 |
Calibration dataset is a cleaned, fixed pippa RP dataset, which does affect the results (in favor) for RP usage.
|
| 9 |
|
| 10 |
You can find the calibration dataset [here](https://huggingface.co/datasets/royallab/PIPPA-cleaned)
|