|
--- |
|
base_model: |
|
- nothingiisreal/L3.1-8B-Celeste-V1.5 |
|
- Sao10K/Llama-3.1-8B-Stheno-v3.4 |
|
- Sao10K/L3.1-8B-Niitama-v1.1 |
|
- arcee-ai/Llama-3.1-SuperNova-Lite |
|
- akjindal53244/Llama-3.1-Storm-8B |
|
- arcee-ai/Llama-Spark |
|
- grimjim/Llama-3-Instruct-abliteration-LoRA-8B |
|
- crestf411/sunfall-peft |
|
|
|
tags: |
|
- llama |
|
- merge |
|
- llama3 |
|
- mixtral |
|
--- |
|
|
|
# Llama-3.1-Celestial-Stone-2x8B (BF16) |
|
|
|
* *Mixture of Experts (14B).* |
|
|
|
Both experts are used in tandem when generating a token. |
|
|
|
------------------------------------------------------------------------------ |
|
|
|
*The first expert* is Instruct 405B distillation/RP vector merge with a bit of abliteration <b>(Supernova-Lite, Niitama1.1, Storm)</b> |
|
|
|
*The second expert* is ERP/Reddit data merge with a bit of abliteration <b>(Celeste1.5, Stheno3.4, Storm)</b> |
|
|
|
------------------------------------------------------------------------------- |
|
|
|
*The base model* is <b>Sao10k/L3.1-Stheno-3.4</b> with the <b>Sunfall LoRa 0.6.1</b> to make it understand SillyTavern prompts and storywriting better. |
|
|
|
------------------------------------------------------------------------------- |
|
|
|
# Prompt Template: |
|
```bash |
|
<|begin_of_text|><|start_header_id|>system<|end_header_id|> |
|
|
|
{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|> |
|
|
|
{input}<|eot_id|><|start_header_id|>assistant<|end_header_id|> |
|
|
|
{output}<|eot_id|> |
|
|
|
``` |
