Felladrin commited on
Commit
e20e567
1 Parent(s): 87af300

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +19 -7
README.md CHANGED
@@ -5,14 +5,26 @@ base_model: Felladrin/Minueza-32M-Base
5
 
6
  GGUF version of [Felladrin/Minueza-32M-Base](https://huggingface.co/Felladrin/Minueza-32M-Base).
7
 
8
- It was not possible to quantize the model after converting it to F16/F32 GGUF, so only those versions are available, being F32 the recommended one for having better precision.
9
 
10
- ## Recommended Inference Parameters
11
 
 
 
12
  ```
13
- temp 0.4
14
- min-p 0.1
15
- top_p 1
16
- top_k 0
17
- repeat_penalty 1.0
 
 
 
 
 
 
 
 
 
 
18
  ```
 
5
 
6
  GGUF version of [Felladrin/Minueza-32M-Base](https://huggingface.co/Felladrin/Minueza-32M-Base).
7
 
8
+ It was not possible to quantize the model, so only the F16 and F32 GGUF files are available.
9
 
10
+ ## Try it with [llama.cpp](https://github.com/ggerganov/llama.cpp)
11
 
12
+ ```sh
13
+ brew install ggerganov/ggerganov/llama.cpp
14
  ```
15
+ ```sh
16
+ llama-cli \
17
+ --hf-repo Felladrin/gguf-Minueza-32M-Base \
18
+ --model Minueza-32M-Base.F32.gguf \
19
+ --random-prompt \
20
+ --dynatemp-range 0.1-2.5 \
21
+ --top-k 0 \
22
+ --top-p 1 \
23
+ --min-p 0.1 \
24
+ --typical 0.85 \
25
+ --mirostat 2 \
26
+ --mirostat-ent 3.5 \
27
+ --repeat-penalty 1.1 \
28
+ --repeat-last-n -1 \
29
+ -n 256
30
  ```