File size: 2,633 Bytes
783b7b7 e788d35 14a0e8a e788d35 783b7b7 ad78a65 783b7b7 b1187bf 783b7b7 b1187bf 783b7b7 b1187bf 783b7b7 ad78a65 783b7b7 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
---
license: apache-2.0
---
**NOTE**: See [creative-writing-control-vectors-v3.0](https://huggingface.co/jukofyork/creative-writing-control-vectors-v3.0) for the current main control-vector repo.
> ***Hallo, Darkness, my old friend, I could be going round the bend...***
---
**IMPORTANT**: Make sure you have the latest version of [llama.cpp](https://github.com/ggerganov/llama.cpp) to use these:
- The [PR that allows multiple control vectors to be loaded](https://github.com/ggerganov/llama.cpp/commit/97877eb10bd8e7f8023420b5b5300bcbdadd62dc) (without zero-padding) just got merged today (27/06/24).
- Older versions of `llama.cpp` will just ***silently*** load the first control vector (and none of the others) if the layer index of the final direction in each file does not match...
To use these control vectors effectively you will need to use the "`--control-vector-scaled`" option like this:
```sh
llama-cli --model <model name>.gguf --control-vector-scaled <model name>-positive.gguf -1.0 --control-vector-scaled <model name>-dark.gguf 1.0 --control-vector-scaled <model name>-law.gguf -1.0 --control-vector-scaled <model name>-chaos.gguf 1.0 [the rest of your CLI arguments...]
```
or:
```sh
llama-cli --model <model name>.gguf --control-vector-scaled <model name>-positive.gguf -0.5 --control-vector-scaled <model name>-dark.gguf 0.5 --control-vector-scaled <model name>-law.gguf -0.5 --control-vector-scaled <model name>-chaos.gguf 0.5 [the rest of your CLI arguments...]
```
or:
```sh
llama-cli --model <model name>.gguf --control-vector-scaled <model name>-positive.gguf -0.5 --control-vector-scaled <model name>-dark.gguf 1.0 --control-vector-scaled <model name>-law.gguf -0.5 --control-vector-scaled <model name>-chaos.gguf 1.0 [the rest of your CLI arguments...]
```
**NOTE:**
- Use ***negative scale factors*** for traits you ***DON'T*** want and ***positive scale factors*** for traits you ***DO*** want.
- The "positive" and "law" traits likely need to use a smaller magnitude of scale factor like `-0.5` or `-0.75` (unless you like ***really*** Grim stories! :D).
- The "dark" and "chaos" traits can often use larger magnitude scale factors like `1.5` or `2.0` before the model starts to repeat gibberish.
- You can use some or all of the 4 control vector files, or alternatively set the scale factor to `0.0` for any traits you don't want to use.
- You can use the same "`--control-vector-scaled`" command line arguments for "`llama-server`" as in the above "`llama-cli`" examples. |