Hugging Face's logo

Neko-Institute-of-Science
/
LLaMA-7B-4bit-32g

Text Generation
Transformers
llama
text-generation-inference
Inference Endpoints
Model card Files Community
1

what is the difference between 128g vs this 32g ?

#1
by bbee - opened
Discussion
bbee

what is the difference between 128g vs this 32g ?

clxy

That's the groupsize used when the model was quantized:

https://github.com/qwopqwop200/GPTQ-for-LLaMa/blob/5731aa11de56affe6e8c88cea66a171045ad1dce/llama.py#L390-L393

Sign up or log in to comment