How to use with llama-cpp-python?
Is there a way to make a .bin file from this git repo?
You don't use GPTQ models with llama-cpp-python, you use GGUF models with that.
And I have GGUF files for this model, here: https://huggingface.co/TheBloke/tulu-2-7B-GGUF
You can use one of those GGUF model files with llama-cpp-python.
PS. GGUF files usually have the extension .gguf now - .bin is old, from the early days of llama.cpp, back when it used GGML format. GGUF replaced GGML in August.
Thanks!
@TheBloke
Thank you for your contributions and your great work. Could you explain the process to convert from GPTQ to GGUF files? Is that possible or the only way to get GGUF files is from the original files?
Thank you I think I find and explaination
https://github.com/ggerganov/llama.cpp/discussions/3353
If you dont agree, please provide yours. Thanks again.