---
license: apache-2.0
datasets:
- argilla/distilabel-intel-orca-dpo-pairs
---

# solarized-18B-dpo

DPO'd from vicgalle/franken-SOLAR-18B-v1.0, a SOLAR-like model upscaled to 18B. 
It is a frankenmerge model created using mergekit, alternating layers of Nous-Hermes-2-SOLAR-10.7B and SOLAR-10.7B-Instruct. Then, we applied DPO over a high-quality preference dataset.

![image/png](https://cdn-uploads.huggingface.co/production/uploads/5fad8602b8423e1d80b8a965/rNtaTqTKrAoN5-C5DuPgu.png)