|
--- |
|
base_model: |
|
- InferenceIllusionist/Excalibur-7b |
|
library_name: transformers |
|
tags: |
|
- finetune |
|
- dpo |
|
- chatml |
|
- gguf |
|
license: apache-2.0 |
|
datasets: |
|
- Intel/orca_dpo_pairs |
|
--- |
|
|
|
|
|
# Excalibur-7b-DPO-GGUF |
|
|
|
<img src="https://i.imgur.com/pbPbqq0.jpeg" width="550"/> |
|
|
|
An initial foray into the world of fine-tuning. The goal of this release was to amplify the quality of the original model's responses, in particular for vision use cases* |
|
|
|
<b>FP16 available [here](https://huggingface.co/InferenceIllusionist/Excalibur-7b-DPO)</b> |
|
|
|
## Notes & Methodology |
|
* [Excalibur-7b](https://huggingface.co/InferenceIllusionist/Excalibur-7b) fine-tuned with Direct Preference Optimization (DPO) using Intel/orca_dpo_pairs |
|
* This is a quick experiment to determine the impact of DPO finetuning on the original base model |
|
* Ran for a little over an hour on a single A100 |
|
* Internal benchmarks showed improvement over base model, awaiting final results |
|
* Precision: bfloat16 |
|
|
|
|
|
## Sample Question - Vision |
|
<img src="https://i.imgur.com/7aRWtzU.jpeg" width="425"/> |
|
|
|
<b>Requires additional [mistral-7b-mmproj-v1.5-Q4_1.gguf](https://huggingface.co/koboldcpp/mmproj/tree/main) file for vision functionality</b> |
|
|
|
Select the gguf file of your choice in Kobold as usual, then make sure to choose the mmproj file above in the LLaVA mmproj field of the model submenu: |
|
<img src="https://i.imgur.com/x8vqH29.png" width="425"/> |
|
|
|
## Prompt Format |
|
* For best results please use ChatML for the prompt format. Alpaca may also work. |
