Xin Liu
commited on
Commit
•
8044ba2
1
Parent(s):
b3be26a
Update
Browse filesSigned-off-by: Xin Liu <[email protected]>
README.md
CHANGED
@@ -47,6 +47,8 @@ tags:
|
|
47 |
wasmedge --dir .:. --nn-preload default:GGML:AUTO:CodeLlama-70b-Instruct-hf-Q2_K.gguf llama-api-server.wasm -p codellama-super-instruct -c 1024 --reverse-prompt 'Source: assistant\nEOT: true'
|
48 |
```
|
49 |
|
|
|
|
|
50 |
## Quantized GGUF Models
|
51 |
|
52 |
| Name | Quant method | Bits | Size | Use case |
|
|
|
47 |
wasmedge --dir .:. --nn-preload default:GGML:AUTO:CodeLlama-70b-Instruct-hf-Q2_K.gguf llama-api-server.wasm -p codellama-super-instruct -c 1024 --reverse-prompt 'Source: assistant\nEOT: true'
|
48 |
```
|
49 |
|
50 |
+
**Note that the model only works in the non-streaming mode.**
|
51 |
+
|
52 |
## Quantized GGUF Models
|
53 |
|
54 |
| Name | Quant method | Bits | Size | Use case |
|