---
base_model: []
library_name: transformers
tags:
- mergekit
- merge
---

![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/6550b16f7490049d6237f200/WcVRcgD6jSzZbQR3LIQpJ.jpeg)

# Warning: v1.0 seems more stable for now!

## Information
### Description

My main goal is to merge the smartness of the base Instruct Nemo with the better prose from the different roleplaying fine-tunes. This is version v0.2, still to be tested. Not sure if it's better than v1.0.

### Instruct

Mistral Instruct.

```
<s>[INST] {system} [/INST]{assistant}</s>[INST] {user} [/INST]
```

### Settings

Lower Temperature of 0.35 recommended, although I had luck with Temperatures above one (1.0-1.2) if you crank up the Min P (0.01-0.1). Run with base DRY of 0.8/1.75/2/0 and you're good to go.

### GGUF

https://huggingface.co/MarinaraSpaghetti/Nemomix-v2.0-12B-GGUF

### V1.0

https://huggingface.co/MarinaraSpaghetti/Nemomix-v0.1-12B

# Nemomix-v2.0-12B

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

## Merge Details
### Merge Method

This model was merged using the [Model Stock](https://arxiv.org/abs/2403.19522) merge method using F:\mergekit\mistralaiMistral-Nemo-Base-2407 as a base.

### Models Merged

The following models were included in the merge:
* F:\mergekit\intervitens_mini-magnum-12b-v1.1
* F:\mergekit\NeverSleep_Lumimaid-v0.2-12B
* F:\mergekit\mistralaiMistral-Nemo-Instruct-2407

### Configuration

The following YAML configuration was used to produce this model:

```yaml
models:
  - model: F:\mergekit\NeverSleep_Lumimaid-v0.2-12B
  - model: F:\mergekit\intervitens_mini-magnum-12b-v1.1
  - model: F:\mergekit\mistralaiMistral-Nemo-Instruct-2407
merge_method: model_stock
base_model: F:\mergekit\mistralaiMistral-Nemo-Base-2407
parameters:
  filter_wise: false
dtype: bfloat16
```