exllamav2 quant?

#1
by traveltube - opened

Just wanted to say thank you for all the merges; they have been excellent. Would there be a exllamav2 quant of this one as well?

Yep. I would have already uploaded, but the conversion script crashed. Its running again now.

4bpw is uploading now and is linked here, lower quants will be uploaded soon: https://huggingface.co/collections/brucethemoose/most-recent-merge-65742644ca03b6c514afa204

Thank you, it seems really phenomenal on my tries thus far!

Sign up or log in to comment