8 bit version
#3
by
bullerwins
- opened
Hi!
As gptq supports algo 8 bit quantization. Would it be possible to upload it?
For 4x24GB VRAM systems would fit perfectly with enough vram left for high context (~70GB+26GB left for context)
Thanks