Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,30 @@
|
|
1 |
-
---
|
2 |
-
license: apache-2.0
|
3 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: apache-2.0
|
3 |
+
---
|
4 |
+
|
5 |
+
**IMPORTANT**: Make sure you have the latest version of [llama.cpp](https://github.com/ggerganov/llama.cpp) to use these:
|
6 |
+
|
7 |
+
- The [PR that allows multiple control vectors to be loaded](https://github.com/ggerganov/llama.cpp/commit/97877eb10bd8e7f8023420b5b5300bcbdadd62dc) (without zero-padding) just got merged today (27/06/24).
|
8 |
+
- Older versions of `llama.cpp` will just ***silently*** load the first control vector and none of the others!!!
|
9 |
+
|
10 |
+
To use these control vectors effectively you will need to use the "`--control-vector-scaled`" option like this:
|
11 |
+
|
12 |
+
```sh
|
13 |
+
llama-cli --model <model name>.gguf --control-vector-scaled <model name>-dark.gguf 1.0 --control-vector-scaled <model name>-positive.gguf -1.0 --control-vector-scaled <model name>-chaos.gguf 1.0 --control-vector-scaled <model name>-law.gguf -1.0 [the rest of your arguments...]
|
14 |
+
```
|
15 |
+
or:
|
16 |
+
```sh
|
17 |
+
llama-cli --model <model name>.gguf --control-vector-scaled <model name>-dark.gguf 0.5 --control-vector-scaled <model name>-positive.gguf -0.5 --control-vector-scaled <model name>-chaos.gguf 0.5 --control-vector-scaled <model name>-law.gguf -0.5 [the rest of your arguments...]
|
18 |
+
```
|
19 |
+
or:
|
20 |
+
```sh
|
21 |
+
llama-cli --model <model name>.gguf --control-vector-scaled <model name>-dark.gguf 1.0 --control-vector-scaled <model name>-positive.gguf -0.5 --control-vector-scaled <model name>-chaos.gguf 1.0 --control-vector-scaled <model name>-law.gguf -0.5 [the rest of your arguments...]
|
22 |
+
```
|
23 |
+
|
24 |
+
**NOTE:**
|
25 |
+
|
26 |
+
- Use ***negative scale factors*** for traits you ***DON'T*** want and ***positive scale factors*** for traits you ***DO*** want.
|
27 |
+
- The "positive" and "law" traits likely need to use a smaller magnitude of scale factor like `-0.5` or `-0.75` (unless you like ***really*** Grim stories! :D).
|
28 |
+
- The "dark" and "chaos" traits can likely use larger magnitude scale factors like `1.5` or `2.0` before the model starts to repeat gibberish.
|
29 |
+
- You can use some or all of the 4 control vector files, or alternatively set the scale factor to `0.0` for any traits you don't want to use.
|
30 |
+
- You can use the same "`--control-vector-scaled`" command line arguments for "`llama-server`" as in the above "`llama-cli`" examples.
|