File size: 7,074 Bytes
af646d0
 
 
 
a1ed8d4
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
af646d0
 
418438f
 
af646d0
 
 
 
 
 
 
 
 
 
1bad3c9
af646d0
51c6d55
8a28f30
 
8754846
 
3b3c301
7191544
 
 
 
11c2e6a
 
7191544
d64f1c7
7191544
79b00b6
7191544
 
8a28f30
4f64b3d
af646d0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
a1ed8d4
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
---
tags:
- merge
license: other
model-index:
- name: QuartetAnemoi-70B-t0.0001
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: AI2 Reasoning Challenge (25-Shot)
      type: ai2_arc
      config: ARC-Challenge
      split: test
      args:
        num_few_shot: 25
    metrics:
    - type: acc_norm
      value: 73.38
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=alchemonaut/QuartetAnemoi-70B-t0.0001
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: HellaSwag (10-Shot)
      type: hellaswag
      split: validation
      args:
        num_few_shot: 10
    metrics:
    - type: acc_norm
      value: 88.9
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=alchemonaut/QuartetAnemoi-70B-t0.0001
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU (5-Shot)
      type: cais/mmlu
      config: all
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 75.42
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=alchemonaut/QuartetAnemoi-70B-t0.0001
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: TruthfulQA (0-shot)
      type: truthful_qa
      config: multiple_choice
      split: validation
      args:
        num_few_shot: 0
    metrics:
    - type: mc2
      value: 69.53
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=alchemonaut/QuartetAnemoi-70B-t0.0001
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: Winogrande (5-shot)
      type: winogrande
      config: winogrande_xl
      split: validation
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 85.32
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=alchemonaut/QuartetAnemoi-70B-t0.0001
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GSM8k (5-shot)
      type: gsm8k
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 68.61
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=alchemonaut/QuartetAnemoi-70B-t0.0001
      name: Open LLM Leaderboard
---

<img src=https://huggingface.co/alchemonaut/QuartetAnemoi-70B-t0.0001/resolve/main/anemoi.png>

# QuartetAnemoi-70B-t0.0001

A sequential merge using a custom algorithm (NearSwap) of:
- [152334H/miqu-1-70b-sf](https://huggingface.co/152334H/miqu-1-70b-sf)
- [Sao10K/WinterGoddess-1.4x-70B-L2](https://huggingface.co/Sao10K/WinterGoddess-1.4x-70B-L2)
- [Aurora-Nights-70B-v1.0](https://huggingface.co/sophosympatheia/Aurora-Nights-70B-v1.0)
- [Xwin-LM-70B-V0.1](https://huggingface.co/Xwin-LM/Xwin-LM-70B-V0.1)

<br/>

In our testing, this model seems like a storyteller, as might be expected, but the changes from this merge are extremely soft. We were impressed that, unlike most models, at the end of a story it did not often use cliches such as "In the end", "And so", "beacon of hope", etc.

<br/>
<br/>

# Quants
Most of the popular quant formats are available now, thanks to community efforts.

| Type  | Misc   | Author        |
| ----- | -----  | -----         |
| [GGUF](https://huggingface.co/alchemonaut/QuartetAnemoi-70B-t0.0001-GGUF/tree/main)  |        | alchemonaut   |
| [GGUF](https://huggingface.co/Nexesenex/alchemonaut_QuartetAnemoi-70B-iMat.GGUF)  | iMat   | Nexesenex      |
| [GGUF](https://huggingface.co/mradermacher/QuartetAnemoi-70B-t0.0001-i1-GGUF)  | iMat   | mradermacher     |
| [GGUF](https://huggingface.co/mradermacher/QuartetAnemoi-70B-t0.0001-GGUF)  | Full Set | mradermacher     |
| [exl2](https://huggingface.co/llmixer/QuartetAnemoi-70B-t0.0001-2.5bpw-h6-exl2)  | 2.5bpw | llmixer       |
| [exl2](https://huggingface.co/altomek/QuartetAnemoi-70B-t0.0001-3.75bpw-EXL2)  | 3.75bpw | altomek       |
| [exl2](https://huggingface.co/llmixer/QuartetAnemoi-70B-t0.0001-4bpw-h6-exl2)  | 4.0bpw| llmixer       |
| [exl2](https://huggingface.co/alchemonaut/QuartetAnemoi-70B-t0.0001-b4.6-h8-exl2)  | 4.6bpw| alchemonaut      |
| [exl2](https://huggingface.co/llmixer/QuartetAnemoi-70B-t0.0001-6.0bpw-h6-exl2) | 6.0bpw | llmixer       |
| [AWQ](https://huggingface.co/tachyphylaxis/QuartetAnemoi-70B-t0.0001-AWQ)   |        | tachyphylaxis |


<br/>
<br/>

# NearSwap Algorithm

NearSwap retains most of the weights of the base model (Miqu), but when a weight is similar between the two, it is interpolated to the secondary model value. A parameter *t* specifies the sameness threshold. When the distance between two values is below *t*, the weight from the secondary model is used.

This version of the model uses *t* = 0.0001. At this *t*, about 0.8% of weights are fully switched to the secondary model during each pass. Model quality rapidly degrades above *t* = 0.0025:

- *t* = 0.0001 (~0.8% full swap): This model
- *t* = 0.0003 (~2%  full swap)
- *t* = 0.001  (~10% full swap): [BoreanGale-70B](https://huggingface.co/alchemonaut/BoreanGale-70B)
- *t* = 0.0025 (~18% full swap): Generates one paragraph okay, but then reverts to garbage
- *t* = 0.005  (~35% full swap): Garbage; semi-related word lists
- *t* = 0.01   (~55% full swap): Garbage; pseudorandom tokens output

For QuartetAnemoi-70B-t0.0001, the three secondary models were each merged sequentially with *t* = 0.0001.

NearSwap implementation:
```
    t: Union[float, np.ndarray],
    v0: Union[np.ndarray, torch.Tensor],
    v1: Union[np.ndarray, torch.Tensor],
...
    lweight = numpy.absolute(v0-v1)
    lweight = t / lweight
    lweight = numpy.nan_to_num(lweight, nan=1.0, posinf=1.0, neginf=1.0)
    numpy.clip(lweight, a_min=0.0, a_max=1.0, out=lweight)
    res = lerp(lweight,v0,v1)
```
<br/>
<br/>


# License and Use

Since the ultimate origin of Miqu is at this time unknown beyond speculation, this model is for noncommercial research use only.

<br/>
<br/>


# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_alchemonaut__QuartetAnemoi-70B-t0.0001)

|             Metric              |Value|
|---------------------------------|----:|
|Avg.                             |76.86|
|AI2 Reasoning Challenge (25-Shot)|73.38|
|HellaSwag (10-Shot)              |88.9|
|MMLU (5-Shot)                    |75.42|
|TruthfulQA (0-shot)              |69.53|
|Winogrande (5-shot)              |85.32|
|GSM8k (5-shot)                   |68.61|