hugging-quants

/

Meta-Llama-3.1-70B-Instruct-GPTQ-INT4

like 13

Hi!

As gptq supports algo 8 bit quantization. Would it be possible to upload it?

For 4x24GB VRAM systems would fit perfectly with enough vram left for high context (~70GB+26GB left for context)

Thanks