README.md

metadata

base_model:
  - nothingiisreal/L3.1-8B-Celeste-V1.5
  - Sao10K/Llama-3.1-8B-Stheno-v3.4
  - Sao10K/L3.1-8B-Niitama-v1.1
  - arcee-ai/Llama-3.1-SuperNova-Lite
  - akjindal53244/Llama-3.1-Storm-8B
  - arcee-ai/Llama-Spark
  - grimjim/Llama-3-Instruct-abliteration-LoRA-8B
  - crestf411/sunfall-peft
tags:
  - llama
  - merge
  - llama3
  - mixtral

Llama-3.1-Celestial-Stone-2x8B (BF16)

• Mixture of Experts (14B).

Both experts are used in tandem when generating a token.

---

The first expert is Instruct 405B distillation/RP vector merge with a bit of abliteration (Supernova-Lite, Niitama1.1, Storm)

The second expert is ERP/Reddit data merge with a bit of abliteration (Celeste1.5, Stheno3.4, Storm)

---

The base model is Sao10k/L3.1-Stheno-3.4 with the Sunfall LoRa 0.6.1 to make it understand SillyTavern prompts and storywriting better.

---

Prompt Template:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>

{input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{output}<|eot_id|>