is there AWQ 4bit for this model?
#9
by
william0014
- opened
is there AWQ 4bit for this model?
I found this quant from @artbred (but haven't tested it): https://huggingface.co/artbred/Llama-3.1-70B-Instruct-lorablated-awq/tree/main