bingal commited on
Commit
3066703
1 Parent(s): f974e68

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -8
README.md CHANGED
@@ -11,16 +11,21 @@ license: mit
11
 
12
  ## Useage
13
 
 
 
 
 
 
 
 
 
 
14
  1. Download model: [qwen1.5-14b-chat-q5_k_m.llamafile](https://huggingface.co/bingal/Qwen1.5-14B-Chat-llamafile/resolve/main/qwen1.5-14b-chat-q5_k_m.llamafile)
15
  2. Run the model
16
- * Windows
17
- 1. Rename the file to `qwen1.5-14b-chat-q5_k_m.exe`
18
- 2. Open terminal window, and run: `\qwen1.5-14b-chat-q5_k_m.exe`
19
- 3. Open browser to http://127.0.0.1:8080 to start chatting
20
- * Linux / macOS
21
- 1. Add execution permissions: `chmod +x ./qwen1.5-14b-chat-q5_k_m.llamafile`
22
- 2. Run in terminal: `./qwen1.5-14b-chat-q5_k_m.llamafile`
23
- 3. Open browser to http://127.0.0.1:8080 to start chatting
24
  3. Openai api usage
25
  * api url: `http://127.0.0.1:8080/v1`
26
  * Python code:
 
11
 
12
  ## Useage
13
 
14
+ **The Windows system has a limitation where it does not support a single exe file over 4GB, so it is necessary to download the llamafile and gguf models separately and run them individually.**
15
+ ### Windows
16
+ * Download [llamafile](https://github.com/Mozilla-Ocho/llamafile/releases/download/0.6.2/llamafile-0.6.2).
17
+ * Rename the file to `llamafile-0.6.2.exe`
18
+ * Download GGUF model [qwen1_5-14b-chat-q5_k_m.gguf](https://huggingface.co/Qwen/Qwen1.5-14B-Chat-GGUF/resolve/main/qwen1_5-14b-chat-q5_k_m.gguf)
19
+ * Open terminal window, and run: `\llamafile-0.6.2.exe .\qwen1_5-14b-chat-q5_k_m.gguf -ngl 9999 --host 0.0.0.0 --port 8080`
20
+ * Open browser to http://127.0.0.1:8080 to start chatting
21
+
22
+ ### Linux / macOS
23
  1. Download model: [qwen1.5-14b-chat-q5_k_m.llamafile](https://huggingface.co/bingal/Qwen1.5-14B-Chat-llamafile/resolve/main/qwen1.5-14b-chat-q5_k_m.llamafile)
24
  2. Run the model
25
+ * Add execution permissions: `chmod +x ./qwen1.5-14b-chat-q5_k_m.llamafile`
26
+ * Run in terminal: `./qwen1.5-14b-chat-q5_k_m.llamafile`
27
+ * Open browser to http://127.0.0.1:8080 to start chatting
28
+
 
 
 
 
29
  3. Openai api usage
30
  * api url: `http://127.0.0.1:8080/v1`
31
  * Python code: