Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
JustinLin610 
posted an update Feb 26
Post
https://qwen.readthedocs.io/ 🔥 The official doc of Qwen1.5 is coming! This is a bilingual doc (English and Chinese, and it will be multilingual if I have time for them). The doc includes instructions for simple inference, running locally with GGUF, ollama, etc., quantization, finetuning, deployment, etc. We will continue adding more stuff to the doc. Stay tuned!
huntz47

im having trouble with quantization of qwen 0.5b model. i am getting an error while loading the quantized model
" terminate called after throwing an instance of 'std::out_of_range'
what(): _Map_base::at"

will you plz help me

·
JustinLin610

gguf quantization?

In this post