README.md

---
license: other
license_name: qwen-research
license_link: https://huggingface.co/Hack337/WavGPT-1.5-GGUF/blob/main/LICENSE
language:
- en
- ru
pipeline_tag: text-generation
base_model:
- Hack337/WavGPT-1.5
- Qwen/Qwen2.5-3B-Instruct
tags:
- chat
---

# WavGPT-1.5-GGUF

## Quickstart

Check out our [llama.cpp documentation](https://qwen.readthedocs.io/en/latest/run_locally/llama.cpp.html) for more usage guide.

We advise you to clone [`llama.cpp`](https://github.com/ggerganov/llama.cpp) and install it following the official guide. We follow the latest version of llama.cpp. 
In the following demonstration, we assume that you are running commands under the repository `llama.cpp`.

Since cloning the entire repo may be inefficient, you can manually download the GGUF file that you need or use `huggingface-cli`:
1. Install
   ```shell
   pip install -U huggingface_hub
   ```
2. Download:
   ```shell
   huggingface-cli download Hack337/WavGPT-1.5-GGUF WavGPT-1.5.gguf --local-dir . --local-dir-use-symlinks False
   ```

For users, to achieve chatbot-like experience, it is recommended to commence in the conversation mode:

```shell
./llama-cli -m <gguf-file-path> \
    -co -cnv -p "Вы очень полезный помощник." \
    -fa -ngl 80 -n 512
```