Does not work. The model size is wrong too. 1.5B x 5.5BPW should be ~1.6GB

by imoc - opened 4 days ago

imoc

4 days ago

So...?

Owner 4 days ago

Oh, I don't know

Owner 4 days ago

I used AutoQuants to quantize it, which himself uses turboderp/exllamav2 to quantize. So I don't know.
I will try to requantize it

Owner 4 days ago

How did you calculate this "1.6GB" ?

Owner 4 days ago

@imoc There was indeed an issue. I requantized it and reuploaded it so now it's corrected.
Thanks for the report

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment