mixtral-soup
For experimental purposes only. Probably not that good.
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the linear merge method.
Models Merged
The following models were included in the merge:
- monology/mixtral-expert7
- monology/mixtral-expert5
- monology/mixtral-expert6
- monology/mixtral-expert0
- monology/mixtral-expert4
- monology/mixtral-expert1
- monology/mixtral-expert3
- monology/mixtral-expert2
Configuration
The following YAML configuration was used to produce this model:
models:
- model: monology/mixtral-expert0
- model: monology/mixtral-expert1
- model: monology/mixtral-expert2
- model: monology/mixtral-expert3
- model: monology/mixtral-expert4
- model: monology/mixtral-expert5
- model: monology/mixtral-expert6
- model: monology/mixtral-expert7
parameters:
weight: 1.0
merge_method: linear
dtype: float16
- Downloads last month
- 73
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.