File size: 2,749 Bytes
4ca5ba4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 |
---
base_model:
- nothingiisreal/L3.1-8B-Celeste-V1.5
- Sao10K/Llama-3.1-8B-Stheno-v3.4
- Sao10K/L3.1-8B-Niitama-v1.1
- arcee-ai/Llama-3.1-SuperNova-Lite
- akjindal53244/Llama-3.1-Storm-8B
- arcee-ai/Llama-Spark
- grimjim/Llama-3-Instruct-abliteration-LoRA-8B
- crestf411/sunfall-peft
- v000000/L3.1-Celestial-Stone-2x8B
library_name: transformers
tags:
- merge
- llama
- mixtral
- dpo
---
[](https://hf.co/QuantFactory)
# QuantFactory/L3.1-Celestial-Stone-2x8B-DPO-GGUF
This is quantized version of [v000000/L3.1-Celestial-Stone-2x8B-DPO](https://huggingface.co/v000000/L3.1-Celestial-Stone-2x8B-DPO) created using llama.cpp
# Original Model Card
> [!WARNING]
> **Sampler:**<br>
> Likes a low temperature due to the MoE architecture. I use 0.3 personally.
# Llama-3.1-Celestial-Stone-2x8B-DPO (BF16)
* *DPO Trained, Mixture of Experts (14B).*
* <b>Direct Preference Optimization run</b>
----> [Q6_K](https://huggingface.co/v000000/L3.1-Celestial-Stone-2x8B-DPO-Q6_K-GGUF)
---------------------------------------------------------------------------------
[L3.1-Celestial-Stone-2x8B](https://huggingface.co/v000000/L3.1-Celestial-Stone-2x8B) Finetuned on Nvidia A100.
0.5 Epoch completed of dataset [jondurbin/gutenberg-dpo-v0.1](https://huggingface.co/datasets/jondurbin/gutenberg-dpo-v0.1) with learning_rate=8e-6
Result seems pretty good. More compliant and verbose, less sloppy and safety aligned.
------------------------------------------------------------------------------
*The first expert* is Instruct 405B distillation/RP vector merge <b>(Supernova-Lite, Niitama1.1, Storm)</b>
*The second expert* is ERP/Reddit data merge <b>(Celeste1.5, Stheno3.4, Storm)</b>
-------------------------------------------------------------------------------
*The base model* is <b>Sao10k/L3.1-Stheno-3.4</b> with the <b>Sunfall LoRa 0.6.1</b> to make it understand SillyTavern prompts and storywriting better.
-------------------------------------------------------------------------------
*Resultant merge finetuned* on [jondurbin/gutenberg-dpo-v0.1](https://huggingface.co/datasets/jondurbin/gutenberg-dpo-v0.1).
# Prompt Template:
```bash
<|begin_of_text|><|start_header_id|>system<|end_header_id|>
{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>
{input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
{output}<|eot_id|>
```
|