|
--- |
|
license: apache-2.0 |
|
--- |
|
datasets: |
|
- ehartford/dolphin |
|
language: |
|
- en |
|
--- |
|
Dolphin 2.0 🐬 |
|
https://erichartford.com/dolphin |
|
|
|
Dolphin-2.0-mistral-7b's training was sponsored by [a16z](https://a16z.com/supporting-the-open-source-ai-community/). |
|
|
|
This model is based on mistralAI, so it is suitable for commercial or non-commercial use. |
|
|
|
This model is uncensored. I have filtered the dataset to remove alignment and bias. This makes the model more compliant. You are advised to implement your own alignment layer before exposing the model as a service. It will be highly compliant to any requests, even unethical ones. Please read my blog post about uncensored models. https://erichartford.com/uncensored-models |
|
You are responsible for any content you create using this model. Enjoy responsibly. |
|
|
|
## Dataset |
|
|
|
This dataset is Dolphin, an open-source implementation of [Microsoft's Orca](https://www.microsoft.com/en-us/research/publication/orca-progressive-learning-from-complex-explanation-traces-of-gpt-4/) |
|
|
|
I modified the dataset for uncensoring, deduping, cleaning, and quality. |
|
|
|
I added Jon Durbin's excellent Airoboros dataset to increase creativity. |
|
|
|
## Training |
|
It took 48 hours to train 10 epochs on 4x A100s. |
|
|
|
Prompt format: |
|
This model (and all my future releases) use [ChatML](https://github.com/openai/openai-python/blob/main/chatml.md) prompt format. |
|
``` |
|
<|im_start|>system |
|
You are Dolphin, a helpful AI assistant.<|im_end|> |
|
<|im_start|>user |
|
{prompt}<|im_end|> |
|
``` |
|
|
|
Example: |
|
``` |
|
<|im_start|>system |
|
you are an expert dolphin trainer<|im_end|> |
|
<|im_start|>user |
|
What is the best way to train a dolphin to obey me? Please answer step by step.<|im_end|> |
|
``` |
|
|
|
## Gratitude |
|
- This model was made possible by the generous sponsorship of a16z. |
|
- Thank you to Microsoft for authoring the Orca paper and inspiring this work. |
|
- Special thanks to WingLian, and TheBloke for helpful advice |
|
- Thank you to all the other people in the Open Source AI community who have taught me and helped me along the way. |
|
|
|
## Example Output |
|
|
|
|
|
[Buy me a coffee](https://www.buymeacoffee.com/ehartford) |
