kromeurus commited on
Commit
eda077e
1 Parent(s): a60f50e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +218 -43
README.md CHANGED
@@ -1,43 +1,218 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
- # merge
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the della merge method using parts/himerus.b as a base.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * parts/himerus.c
22
-
23
- ### Configuration
24
-
25
- The following YAML configuration was used to produce this model:
26
-
27
- ```yaml
28
- models:
29
- - model: parts/himerus.c
30
- parameters:
31
- weight: [0.8, 0.32, 0.6]
32
- density: [0.23, 0.5, 0.37]
33
- epsilon: [0.05, 0.06, 0.04]
34
- lambda: 0.07
35
- - model: parts/himerus.b
36
- base_model: parts/himerus.b
37
- parameters:
38
- normalize: false
39
- int8_mask: true
40
- merge_method: della
41
- dtype: bfloat16
42
-
43
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - Sao10K/L3-8B-Niitama-v1
4
+ - Nitral-AI/Hathor_Tahsin-L3-8B-v0.85
5
+ - ArliAI/ArliAI-Llama-3-8B-Formax-v1.0
6
+ - nothingiisreal/L3-8B-Celeste-V1.2
7
+ - ResplendentAI/Nymph_8B
8
+ - TheDrummer/Llama-3SOME-8B-v2
9
+ - nothingiisreal/L3-8B-Instruct-Abliterated-DWP
10
+ library_name: transformers
11
+ tags:
12
+ - mergekit
13
+ - merge
14
+ - not-for-all-audiences
15
+ ---
16
+ ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/667eea5cdebd46a5ec4dcc3d/hFHS4nKKh3sbSUq6MlPaK.jpeg)
17
+
18
+ This did not go in the direction I wanted it to. Why is it always my best models are the ones that happen out of sheer coincidence or stupidity. To start, this model doesn't
19
+ have Crepus Horizon in it despite its name. As of writing this, v0.1 Crepus Horzion is way too broken to merge with and I've taken that repo down until an update is out. This
20
+ model actually works better without it so, a plus. Still keeping the name though, sounds cool lmao.
21
+
22
+ Tangent aside, Antheros was part experement, part 'throw shit at the wall as see what sticks'. I was aiming for a more narritive heavy RP model with some moist and spice thrown
23
+ in, ended up with that and solid intruct following. Took a week and a half, but it's here. Any feedback would be much appreciated.
24
+
25
+ ### Quants
26
+
27
+ [OG Q8 GGUF](https://huggingface.co/kromeurus/L3-Horizon-Anteros-v0.1-13B-Q8-GGUF) by me.
28
+
29
+ Other quants are not available, yet.
30
+
31
+ ### Details & Recommended Settings
32
+
33
+ (Still testing; subject to change)
34
+
35
+ Thrives at more story heavy and narritive RP yet still excels with the basics. Leans vaguely sensual and smooth-tongued with it's description while keeping up human dialogue
36
+ and actions. Fully uncensored and a tad dramatic, so have fun with those eRP scenes.
37
+
38
+ Clings onto whatever instructs you give it and fucking sprints with it (thanks Formax). This model's generation quality heavily depends on the character card given; a shit
39
+ card will give you shit outputs and vise versa. Be careful with your choice of words and the examples you give it.
40
+
41
+ Has a certain tendency to speak for the {user}, but that's easily negated with a few instructs.
42
+
43
+ Rec. Settings:
44
+ ```
45
+ Template: Model Default
46
+ Temperature: 1.25
47
+ Min P: 0.1
48
+ Repeat Penelty: 1.05
49
+ Repeat Penelty Tokens: 256
50
+ ```
51
+
52
+ ### Models Merged & Merge Theory
53
+
54
+ The following models were included in the merge:
55
+ * [ResplendentAI/Nymph_8B](https://huggingface.co/ResplendentAI/Nymph_8B)
56
+ * [TheDrummer/Llama-3SOME-8B-v2](https://huggingface.co/TheDrummer/Llama-3SOME-8B-v2)
57
+ * [nothingiisreal/L3-8B-Instruct-Abliterated-DWP](https://huggingface.co/nothingiisreal/L3-8B-Instruct-Abliterated-DWP)
58
+ * [Sao10K/L3-8B-Niitama-v1](https://huggingface.co/Sao10K/L3-8B-Niitama-v1)
59
+ * [Nitral-AI/Hathor_Tahsin-L3-8B-v0.85](https://huggingface.co/Nitral-AI/Hathor_Tahsin-L3-8B-v0.85)
60
+ * [ArliAI/ArliAI-Llama-3-8B-Formax-v1.0](https://huggingface.co/ArliAI/ArliAI-Llama-3-8B-Formax-v1.0)
61
+ * [nothingiisreal/L3-8B-Celeste-V1.2](https://huggingface.co/nothingiisreal/L3-8B-Celeste-V1.2)
62
+
63
+ too tired rn, will update later
64
+
65
+ ### Config
66
+
67
+ ```yaml
68
+ models:
69
+ slices:
70
+ - sources:
71
+ - layer_range: [0, 4]
72
+ model: ResplendentAI/Nymph_8B
73
+ - sources:
74
+ - layer_range: [3, 9]
75
+ model: TheDrummer/Llama-3SOME-8B-v2
76
+ parameters:
77
+ scale:
78
+ - filter: q_proj
79
+ value: 0.9
80
+ - filter: down_proj
81
+ value: 0.9
82
+ - sources:
83
+ - layer_range: [5, 12]
84
+ model: nothingiisreal/L3-8B-Instruct-Abliterated-DWP
85
+ parameters:
86
+ scale:
87
+ - filter: up_proj
88
+ value: 0.9
89
+ - sources:
90
+ - layer_range: [10, 18]
91
+ model: ResplendentAI/Nymph_8B
92
+ - sources:
93
+ - layer_range: [11, 22]
94
+ model: nothingiisreal/L3-8B-Instruct-Abliterated-DWP
95
+ - sources:
96
+ - layer_range: [17, 19]
97
+ model: TheDrummer/Llama-3SOME-8B-v2
98
+ - sources:
99
+ - layer_range: [19, 24]
100
+ model: ResplendentAI/Nymph_8B
101
+ - sources:
102
+ - layer_range: [22, 26]
103
+ model: nothingiisreal/L3-8B-Instruct-Abliterated-DWP
104
+ - sources:
105
+ - layer_range: [27, 29]
106
+ model: TheDrummer/Llama-3SOME-8B-v2
107
+ parameters:
108
+ scale:
109
+ - filter: v_proj
110
+ value: 0.9
111
+ - sources:
112
+ - layer_range: [29, 30]
113
+ model: ResplendentAI/Nymph_8B
114
+ - sources:
115
+ - layer_range: [26, 32]
116
+ model: nothingiisreal/L3-8B-Instruct-Abliterated-DWP
117
+ parameters:
118
+ int8_mask: true
119
+ merge_method: passthrough
120
+ dtype: bfloat16
121
+ name: anteros.b
122
+ ---
123
+ models:
124
+ slices:
125
+ - sources:
126
+ - layer_range: [14, 20]
127
+ model: nothingiisreal/L3-8B-Celeste-V1.2
128
+ parameters:
129
+ int8_mask: true
130
+ merge_method: passthrough
131
+ dtype: bfloat16
132
+ name: celeste14-20.sl
133
+ ---
134
+ models:
135
+ slices:
136
+ - sources:
137
+ - layer_range: [14, 20]
138
+ model: Sao10K/L3-8B-Niitama-v1
139
+ parameters:
140
+ int8_mask: true
141
+ merge_method: passthrough
142
+ dtype: bfloat16
143
+ name: niitama14-20.sl
144
+ ---
145
+ models:
146
+ - model: celeste14-20.sl
147
+ parameters:
148
+ weight: [1, 0.75, 0.625, 0.5, 0.375, 0.25, 0]
149
+ - model: niitama14-20.sl
150
+ parameters:
151
+ weight: [0, 0.25, 0.375, 0.5, 0.625, 0.75, 1]
152
+ merge_method: dare_linear
153
+ base_model: celeste14-20.sl
154
+ dtype: bfloat16
155
+ name: celeniit14-20.sl
156
+ ---
157
+ models:
158
+ slices:
159
+ - sources:
160
+ - layer_range: [0, 4]
161
+ model: Nitral-AI/Hathor_Tahsin-L3-8B-v0.85
162
+ - sources:
163
+ - layer_range: [1, 5]
164
+ model: ArliAI/ArliAI-Llama-3-8B-Formax-v1.0
165
+ - sources:
166
+ - layer_range: [4, 8]
167
+ model: Nitral-AI/Hathor_Tahsin-L3-8B-v0.85
168
+ - sources:
169
+ - layer_range: [5, 9]
170
+ model: ArliAI/ArliAI-Llama-3-8B-Formax-v1.0
171
+ - sources:
172
+ - layer_range: [8, 10]
173
+ model: Sao10K/L3-8B-Niitama-v1
174
+ - sources:
175
+ - layer_range: [6, 14]
176
+ model: nothingiisreal/L3-8B-Celeste-V1.2
177
+ - sources:
178
+ - layer_range: [0, 6]
179
+ model: celeniit14-20.sl
180
+ - sources:
181
+ - layer_range: [20, 23]
182
+ model: Sao10K/L3-8B-Niitama-v1
183
+ - sources:
184
+ - layer_range: [22, 26]
185
+ model: Nitral-AI/Hathor_Tahsin-L3-8B-v0.85
186
+ - sources:
187
+ - layer_range: [22, 28]
188
+ model: nothingiisreal/L3-8B-Celeste-V1.2
189
+ - sources:
190
+ - layer_range: [25, 27]
191
+ model: Nitral-AI/Hathor_Tahsin-L3-8B-v0.85
192
+ - sources:
193
+ - layer_range: [28, 30]
194
+ model: Sao10K/L3-8B-Niitama-v1
195
+ - sources:
196
+ - layer_range: [25, 32]
197
+ model: nothingiisreal/L3-8B-Celeste-V1.2
198
+ parameters:
199
+ int8_mask: true
200
+ merge_method: passthrough
201
+ dtype: bfloat16
202
+ name: himerus.c # Himerus Basis.C, available on it's own.
203
+ ----
204
+ models:
205
+ - model: himerus.c
206
+ parameters:
207
+ weight: [0.8, 0.32, 0.6]
208
+ density: [0.23, 0.55, 0.37]
209
+ epsilon: [0.05, 0.06, 0.04]
210
+ lambda: 0.07
211
+ - model: anteros.b
212
+ base_model: anteros.b
213
+ parameters:
214
+ normalize: false
215
+ int8_mask: true
216
+ merge_method: della
217
+ dtype: bfloat16
218
+ ```