Is it possible to get GPTQ quants in 4 bpw?

#2
by MrHillsss - opened

Or any other quants to fit in a 3090/4090?

I think the amazing @TheBloke is going to convert them and push them as always to his repos!
You can already use load_in_4bit = True for bitsandbytes quantization with transformers format

Sign up or log in to comment