Train OpenChat on Mistral-10.7B-v0.2
#12
by
Joseph717171
- opened
OpenChat team, I Depth Up-Scaled Mistral-7B-v0.2, following UpStage’s paper: SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling, if you are interested in training OpenChat on a slightly bigger model.
Joseph717171/Mistral-10.7B-v0.2
- 32K Context Window
- 🚫 Sliding Window Attention