Update README.md
Browse files
README.md
CHANGED
@@ -11,16 +11,21 @@ license: mit
|
|
11 |
|
12 |
## Useage
|
13 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
14 |
1. Download model: [qwen1.5-14b-chat-q5_k_m.llamafile](https://huggingface.co/bingal/Qwen1.5-14B-Chat-llamafile/resolve/main/qwen1.5-14b-chat-q5_k_m.llamafile)
|
15 |
2. Run the model
|
16 |
-
|
17 |
-
|
18 |
-
|
19 |
-
|
20 |
-
* Linux / macOS
|
21 |
-
1. Add execution permissions: `chmod +x ./qwen1.5-14b-chat-q5_k_m.llamafile`
|
22 |
-
2. Run in terminal: `./qwen1.5-14b-chat-q5_k_m.llamafile`
|
23 |
-
3. Open browser to http://127.0.0.1:8080 to start chatting
|
24 |
3. Openai api usage
|
25 |
* api url: `http://127.0.0.1:8080/v1`
|
26 |
* Python code:
|
|
|
11 |
|
12 |
## Useage
|
13 |
|
14 |
+
**The Windows system has a limitation where it does not support a single exe file over 4GB, so it is necessary to download the llamafile and gguf models separately and run them individually.**
|
15 |
+
### Windows
|
16 |
+
* Download [llamafile](https://github.com/Mozilla-Ocho/llamafile/releases/download/0.6.2/llamafile-0.6.2).
|
17 |
+
* Rename the file to `llamafile-0.6.2.exe`
|
18 |
+
* Download GGUF model [qwen1_5-14b-chat-q5_k_m.gguf](https://huggingface.co/Qwen/Qwen1.5-14B-Chat-GGUF/resolve/main/qwen1_5-14b-chat-q5_k_m.gguf)
|
19 |
+
* Open terminal window, and run: `\llamafile-0.6.2.exe .\qwen1_5-14b-chat-q5_k_m.gguf -ngl 9999 --host 0.0.0.0 --port 8080`
|
20 |
+
* Open browser to http://127.0.0.1:8080 to start chatting
|
21 |
+
|
22 |
+
### Linux / macOS
|
23 |
1. Download model: [qwen1.5-14b-chat-q5_k_m.llamafile](https://huggingface.co/bingal/Qwen1.5-14B-Chat-llamafile/resolve/main/qwen1.5-14b-chat-q5_k_m.llamafile)
|
24 |
2. Run the model
|
25 |
+
* Add execution permissions: `chmod +x ./qwen1.5-14b-chat-q5_k_m.llamafile`
|
26 |
+
* Run in terminal: `./qwen1.5-14b-chat-q5_k_m.llamafile`
|
27 |
+
* Open browser to http://127.0.0.1:8080 to start chatting
|
28 |
+
|
|
|
|
|
|
|
|
|
29 |
3. Openai api usage
|
30 |
* api url: `http://127.0.0.1:8080/v1`
|
31 |
* Python code:
|