8 bit version

#3
by bullerwins - opened

Hi!

As gptq supports algo 8 bit quantization. Would it be possible to upload it?

For 4x24GB VRAM systems would fit perfectly with enough vram left for high context (~70GB+26GB left for context)

Thanks

Sign up or log in to comment