Update README.md
Browse files
README.md
CHANGED
@@ -5,14 +5,26 @@ base_model: Felladrin/Minueza-32M-Base
|
|
5 |
|
6 |
GGUF version of [Felladrin/Minueza-32M-Base](https://huggingface.co/Felladrin/Minueza-32M-Base).
|
7 |
|
8 |
-
It was not possible to quantize the model
|
9 |
|
10 |
-
##
|
11 |
|
|
|
|
|
12 |
```
|
13 |
-
|
14 |
-
|
15 |
-
|
16 |
-
|
17 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
18 |
```
|
|
|
5 |
|
6 |
GGUF version of [Felladrin/Minueza-32M-Base](https://huggingface.co/Felladrin/Minueza-32M-Base).
|
7 |
|
8 |
+
It was not possible to quantize the model, so only the F16 and F32 GGUF files are available.
|
9 |
|
10 |
+
## Try it with [llama.cpp](https://github.com/ggerganov/llama.cpp)
|
11 |
|
12 |
+
```sh
|
13 |
+
brew install ggerganov/ggerganov/llama.cpp
|
14 |
```
|
15 |
+
```sh
|
16 |
+
llama-cli \
|
17 |
+
--hf-repo Felladrin/gguf-Minueza-32M-Base \
|
18 |
+
--model Minueza-32M-Base.F32.gguf \
|
19 |
+
--random-prompt \
|
20 |
+
--dynatemp-range 0.1-2.5 \
|
21 |
+
--top-k 0 \
|
22 |
+
--top-p 1 \
|
23 |
+
--min-p 0.1 \
|
24 |
+
--typical 0.85 \
|
25 |
+
--mirostat 2 \
|
26 |
+
--mirostat-ent 3.5 \
|
27 |
+
--repeat-penalty 1.1 \
|
28 |
+
--repeat-last-n -1 \
|
29 |
+
-n 256
|
30 |
```
|