TeeZee
/

DarkForest-20B-v1.0-bpw8.0-h8-exl2

Text Generation

Not-For-All-Audiences

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Any chance we could get a 6bpw?

#1

by vLEGIONv - opened Feb 18

Feb 18

6bpw is the sweet spot for overhead and quality on 20b's from what I've found. Otherwise, loving the model!

TeeZee

Owner Feb 18

•

6bpw 6h or 6bpw 8h? Both quants are ready, test and please let me know how they work:
https://huggingface.co/TeeZee/DarkForest-20B-v2.0-bpw6.0-h6-exl2
https://huggingface.co/TeeZee/DarkForest-20B-v2.0-bpw6.0-h8-exl2

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment