GGUF models

#30
by mariahelenass - opened

Hi everyone!
Are there any GGUF models available for download?

https://huggingface.co/leafspark/Qwen2-VL-7B-Instruct-GGUF

but the model card says:

Note: this is not supported nor will it run in llama.cpp. it's an example of what the model structure may look like if converted to GGUF

https://github.com/ggerganov/llama.cpp/issues/9246

Sign up or log in to comment