Llama-2-13B-chat-GPTQ problem
Hi,
I'm trying to use
and I've encountered several problems. My first issue is with llama-cpp-python. I used "pip install llama-cpp-python" to install it, but it appears to have incomplete files. As a result, I had to clone the repository just to build the project.
The second issue is with the command "./main -t 10 -ngl 32 -m llama-2-13b-chat.ggmlv3.q4_0.bin --color -c 4096 --temp 0.7 --repeat_penalty 1.1 -n -1 --in-prefix-bos --in-prefix ' [INST] ' --in-suffix ' [/INST]' -i -p '[INST] <> You are a helpful, respectful, and honest assistant. <> Write a story about llamas. [/INST]'".
Because the pip method installed incomplete files, I had to clone the entire repository. But even after successfully building the project, the main executable is missing. I'm not sure if it's failing to generate or if there's another problem. As a result, I can't execute the command because ./main is missing please refer to the attached screenshot.
llama-cpp-python is a Python library. You use it from Python code, or use it to provide an OpenAI-compatible API server. It doesn't provide a main binary. Check their Github repo for examples and more details.
It's llama.cpp that provides a main binary, either after you compile it yourself, or if you download one of the pre-built releases from its releases page.
i already tried to compile it it was successful, but it didn't generate any main binary