File size: 1,049 Bytes
b5c6898 4edffe7 a9df9e6 5626e8e 1bd7ac9 5626e8e 6a86a7c 5626e8e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
---
license: apache-2.0
---
Contributors: Nicolas Mejia Petit
# Mistral 29b: A New Base Model
The objective of this model is to serve as a new fully open source base model with 29.2 billion parameters.
This model spits out jargon, and needs to be fine tuned, either with qlora, with the adapter attached to every layer, or better yet a full fine tune.
## Model Creation
The model was created by stacking four models: Dolphin, Zephyr, Meta-math7b, and Speechless code, to form a single model.
## Useful Resources
- Model Merging (MergeKit) (https://github.com/cg123/mergekit)
## Source Models
- Dolphin2.1-mistral-7b by Eric Hartford (https://huggingface.co/ehartford/dolphin-2.1-mistral-7b)
- Zephyr-7b-beta by HuggingFace (https://huggingface.co/HuggingFaceH4/zephyr-7b-beta)
- MetaMath-Mistral-7B by meta-math (https://huggingface.co/meta-math/MetaMath-Mistral-7B)
- speechless-code-mistral-7b-v1.0 (https://huggingface.co/uukuguy/speechless-code-mistral-7b-v1.0)
|