rAIfle commited on
Commit
7ca921a
1 Parent(s): a51fe3b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +37 -0
README.md CHANGED
@@ -6,7 +6,44 @@ tags:
6
 
7
  ---
8
  # Sloppy-Wingman-8x7B-hf
 
9
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
11
 
12
  ## Merge Details
 
6
 
7
  ---
8
  # Sloppy-Wingman-8x7B-hf
9
+ ![Sloppy Wingman](https://files.catbox.moe/7ay3me.png)
10
 
11
+ Big slop, good model.
12
+ Running better at slightly higher temp (1.1-ish) than usual, along with 0.05 MinP and 0.28 snoot.
13
+ Bog-standard ChatML works best imo, but Alpaca and Mixtral formats work (to some degree) too.
14
+
15
+ Parts:
16
+ ```yaml
17
+ models:
18
+ - model: mistralai/Mixtral-8x7B-v0.1+retrieval-bar/Mixtral-8x7B-v0.1_case-briefs
19
+ parameters:
20
+ weight: 0.33
21
+ - model: mistralai/Mixtral-8x7B-v0.1+wandb/Mixtral-8x7b-Remixtral
22
+ parameters:
23
+ weight: 0.33
24
+ merge_method: task_arithmetic
25
+ base_model: mistralai/Mixtral-8x7B-v0.1
26
+ dtype: float16
27
+ ```
28
+ and
29
+ ```yaml
30
+ models:
31
+ - model: mistralai/Mixtral-8x7B-Instruct-v0.1+/ai/LLM/tmp/pefts/daybreak-peft/mixtral-8x7b
32
+ parameters:
33
+ weight: 0.85
34
+ - model: notstoic/Nous-Hermes-2-Mixtruct-v0.1-8x7B-DPO-DARE_TIES
35
+ parameters:
36
+ weight: 0.25
37
+ - model: ycros/BagelWorldTour-8x7B
38
+ parameters:
39
+ weight: 0.1
40
+ merge_method: task_arithmetic
41
+ base_model: mistralai/Mixtral-8x7B-Instruct-v0.1
42
+ dtype: float16
43
+ ```
44
+ SLERP:ed together as per below.
45
+
46
+ ---
47
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
48
 
49
  ## Merge Details