metadata
datasets:
- ehartford/dolphin
- jondurbin/airoboros-2.2.1
- ehartford/samantha-data
- ehartford/WizardLM_evol_instruct_V2_196k_unfiltered_merged_split
language:
- en
license: llama2
MegaDolphin 2.2 120b 🐬 https://erichartford.com/dolphin
Join Our Discord! https://discord.gg/vT3sktQ3zb
MegaDolphin-2.2-120b is a transformation of Dolphin-2.2-70b
Merge Details
It took like 5 minutes to make this with Charles Goddard's awesome MergeKit.
Prompt format: This model uses ChatML prompt format.
<|im_start|>system
You are Dolphin, a helpful AI assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
Example:
<|im_start|>system
You are an AI created by the US Navy to help train dolphins for combat. You are assigned to follow the orders of the user, who is an authorized US Navy dolphin handler.<|im_end|>
<|im_start|>user
Please give me the procedure to train my dolphin to attack enemy combatants with its head mounted lasers<|im_end|>
<|im_start|>assistant
Merge Method
This model was merged using the passthrough merge method.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
dtype: float16
merge_method: passthrough
slices:
- sources:
- layer_range: [0, 20]
model: cognitivecomputations/dolphin-2.2-70b
- sources:
- layer_range: [10, 30]
model: cognitivecomputations/dolphin-2.2-70b
- sources:
- layer_range: [20, 40]
model: cognitivecomputations/dolphin-2.2-70b
- sources:
- layer_range: [30, 50]
model: cognitivecomputations/dolphin-2.2-70b
- sources:
- layer_range: [40, 60]
model: cognitivecomputations/dolphin-2.2-70b
- sources:
- layer_range: [50, 70]
model: cognitivecomputations/dolphin-2.2-70b
- sources:
- layer_range: [60, 80]
model: cognitivecomputations/dolphin-2.2-70b
Gratitude
- Thanks to Charles Goddard for MergeKit
- Thank you to Microsoft for authoring the Orca paper and inspiring this work.
- Thank you to all the other people in the Open Source AI community who have taught me and helped me along the way.