metadata
license: apache-2.0
datasets:
- argilla/distilabel-intel-orca-dpo-pairs
solarized-18B-dpo
DPO'd from vicgalle/franken-SOLAR-18B-v1.0, a SOLAR-like model upscaled to 18B. It is a frankenmerge model created using mergekit, alternating layers of Nous-Hermes-2-SOLAR-10.7B and SOLAR-10.7B-Instruct. Then, we applied DPO over a high-quality preference dataset.
