README.md

metadata

license: apache-2.0

IMPORTANT: Make sure you have the latest version of llama.cpp to use these:

• The PR that allows multiple control vectors to be loaded (without zero-padding) just got merged today (27/06/24).
• Older versions of llama.cpp will just silently load the first control vector (and none of the others) if the layer index of the final direction in each file does not match...

To use these control vectors effectively you will need to use the "--control-vector-scaled" option like this:

llama-cli --model <model name>.gguf --control-vector-scaled <model name>-dark.gguf 1.0 --control-vector-scaled <model name>-positive.gguf -1.0 --control-vector-scaled <model name>-chaos.gguf 1.0 --control-vector-scaled <model name>-law.gguf -1.0 [the rest of your arguments...]

or:

llama-cli --model <model name>.gguf --control-vector-scaled <model name>-dark.gguf 0.5 --control-vector-scaled <model name>-positive.gguf -0.5 --control-vector-scaled <model name>-chaos.gguf 0.5 --control-vector-scaled <model name>-law.gguf -0.5 [the rest of your arguments...]

or:

llama-cli --model <model name>.gguf --control-vector-scaled <model name>-dark.gguf 1.0 --control-vector-scaled <model name>-positive.gguf -0.5 --control-vector-scaled <model name>-chaos.gguf 1.0 --control-vector-scaled <model name>-law.gguf -0.5 [the rest of your arguments...]

NOTE:

• Use negative scale factors for traits you DON'T want and positive scale factors for traits you DO want.
• The "positive" and "law" traits likely need to use a smaller magnitude of scale factor like -0.5 or -0.75 (unless you like really Grim stories! :D).
• The "dark" and "chaos" traits can often use larger magnitude scale factors like 1.5 or 2.0 before the model starts to repeat gibberish.
• You can use some or all of the 4 control vector files, or alternatively set the scale factor to 0.0 for any traits you don't want to use.
• You can use the same "--control-vector-scaled" command line arguments for "llama-server" as in the above "llama-cli" examples.