please see dahara1/Qwen2.5-3B-Instruct-gguf-japanese-imatrix-128K
3-bit
4-bit
5-bit
6-bit
8-bit
16-bit