Hugging Face's logo

Lewdiculous
/
Kunoichi-DPO-v2-7B-GGUF-Imatrix

Text Generation
Transformers
GGUF
mistral
quantized
text-generation-inference
Model card Files Community
2
Lewdiculous commited on
Commit
90992fa
•
1 Parent(s): ecd8dc2

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +93 -0
README.md ADDED
@@ -0,0 +1,93 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - SanjiWatsuki/Kunoichi-DPO-v2-7B
4
+ library_name: transformers
5
+ tags:
6
+ - mistral
7
+ - quantized
8
+ - text-generation-inference
9
+ - merge
10
+ - mergekit
11
+ pipeline_tag: text-generation
12
+ inference: false
13
+ license: cc-by-nc-4.0
14
+ ---
15
+ # **GGUF-Imatrix quantizations for [SanjiWatsuki/Kunoichi-DPO-v2-7B](https://huggingface.co/SanjiWatsuki/Kunoichi-DPO-v2-7B/).**
16
+
17
+ *If you want any specific quantization to be added, feel free to ask.*
18
+
19
+ All credits belong to the [creator](https://huggingface.co/SanjiWatsuki/).
20
+
21
+ `Base⇢ GGUF(F16)⇢ Imatrix-Data(F16)⇢ GGUF(Imatrix-Quants)`
22
+
23
+ The new **IQ3_S** quant-option has shown to be better than the old Q3_K_S, so I added that instead of the later. Only supported in `koboldcpp-1.59.1` or higher.
24
+
25
+ Using [llama.cpp](https://github.com/ggerganov/llama.cpp/)-[b2277](https://github.com/ggerganov/llama.cpp/releases/tag/b2277).
26
+
27
+ For --imatrix data, `imatrix-Kunocchini-7b-128k-test-F16.dat` was used.
28
+
29
+ # Waifu card:
30
+
31
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/65d4cf2693a0a3744a27536c/EVYWQn0osm0eP9xIhWbH4.png)
32
+
33
+
34
+ # Original model information:
35
+
36
+ | Model | MT Bench | EQ Bench | MMLU | Logic Test |
37
+ |----------------------|----------|----------|---------|-------------|
38
+ | GPT-4-Turbo | 9.32 | - | - | - |
39
+ | GPT-4 | 8.99 | 62.52 | 86.4 | 0.86 |
40
+ | **Kunoichi-DPO-v2-7B** | **8.51** | **42.18** | **64.94**| **0.58** |
41
+ | Mixtral-8x7B-Instruct| 8.30 | 44.81 | 70.6 | 0.75 |
42
+ | **Kunoichi-DPO-7B** | **8.29** | **41.60** | **64.83** | **0.59** |
43
+ | **Kunoichi-7B** | **8.14** | **44.32** | **64.9** | **0.58** |
44
+ | Starling-7B | 8.09 | - | 63.9 | 0.51 |
45
+ | Claude-2 | 8.06 | 52.14 | 78.5 | - |
46
+ | Silicon-Maid-7B | 7.96 | 40.44 | 64.7 | 0.54 |
47
+ | Loyal-Macaroni-Maid-7B | 7.95 | 38.66 | 64.9 | 0.57 |
48
+ | GPT-3.5-Turbo | 7.94 | 50.28 | 70 | 0.57 |
49
+ | Claude-1 | 7.9 | - | 77 | - |
50
+ | Openchat-3.5 | 7.81 | 37.08 | 64.3 | 0.39 |
51
+ | Dolphin-2.6-DPO | 7.74 | 42.88 | 61.9 | 0.53 |
52
+ | Zephyr-7B-beta | 7.34 | 38.71 | 61.4 | 0.30 |
53
+ | Llama-2-70b-chat-hf | 6.86 | 51.56 | 63 | - |
54
+ | Neural-chat-7b-v3-1 | 6.84 | 43.61 | 62.4 | 0.30 |
55
+
56
+ | Model | Average | AGIEval | GPT4All | TruthfulQA | Bigbench |
57
+ |---|---:|---:|---:|---:|---:|
58
+ | **Kunoichi-DPO-7B**|**58.4**| 45.08 | 74| 66.99| 47.52|
59
+ | **Kunoichi-DPO-v2-7B**|**58.31**| 44.85| 75.05| 65.69| 47.65|
60
+ | [Kunoichi-7B](https://huggingface.co/SanjiWatsuki/Kunoichi-7B)|57.54| 44.99| 74.86| 63.72| 46.58|
61
+ | [OpenPipe/mistral-ft-optimized-1218](https://huggingface.co/OpenPipe/mistral-ft-optimized-1218)| 56.85 | 44.74 | 75.6 | 59.89 | 47.17 |
62
+ | [Silicon-Maid-7B](https://huggingface.co/SanjiWatsuki/Silicon-Maid-7B) | 56.45| 44.74| 74.26| 61.5| 45.32|
63
+ | [mlabonne/NeuralHermes-2.5-Mistral-7B](https://huggingface.co/mlabonne/NeuralHermes-2.5-Mistral-7B) | 53.51 | 43.67 | 73.24 | 55.37 | 41.76 |
64
+ | [teknium/OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B) | 52.42 | 42.75 | 72.99 | 52.99 | 40.94 |
65
+ | [openchat/openchat_3.5](https://huggingface.co/openchat/openchat_3.5) | 51.34 | 42.67 | 72.92 | 47.27 | 42.51 |
66
+ | [berkeley-nest/Starling-LM-7B-alpha](https://huggingface.co/berkeley-nest/Starling-LM-7B-alpha) | 51.16 | 42.06 | 72.72 | 47.33 | 42.53 |
67
+ | [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) | 50.99 | 37.33 | 71.83 | 55.1 | 39.7 |
68
+
69
+ | Model | AlpacaEval2 | Length |
70
+ | --------------------------- | ----------- | ------ |
71
+ | GPT-4 | 23.58% | 1365 |
72
+ | GPT-4 0314 | 22.07% | 1371 |
73
+ | Mistral Medium | 21.86% | 1500 |
74
+ | Mixtral 8x7B v0.1 | 18.26% | 1465 |
75
+ | **Kunoichi-DPO-v2** | **17.19%** | 1785 |
76
+ | Claude 2 | 17.19% | 1069 |
77
+ | Claude | 16.99% | 1082 |
78
+ | Gemini Pro | 16.85% | 1315 |
79
+ | GPT-4 0613 | 15.76% | 1140 |
80
+ | Claude 2.1 | 15.73% | 1096 |
81
+ | Mistral 7B v0.2 | 14.72% | 1676 |
82
+ | GPT 3.5 Turbo 0613 | 14.13% | 1328 |
83
+ | LLaMA2 Chat 70B | 13.87% | 1790 |
84
+ | LMCocktail-10.7B-v1 | 13.15% | 1203 |
85
+ | WizardLM 13B V1.1 | 11.23% | 1525 |
86
+ | Zephyr 7B Beta | 10.99% | 1444 |
87
+ | OpenHermes-2.5-Mistral (7B) | 10.34% | 1107 |
88
+ | GPT 3.5 Turbo 0301 | 9.62% | 827 |
89
+ | **Kunoichi-7B** | **9.38%** | 1492 |
90
+ | GPT 3.5 Turbo 1106 | 9.18% | 796 |
91
+ | GPT-3.5 | 8.56% | 1018 |
92
+ | Phi-2 DPO | 7.76% | 1687 |
93
+ | LLaMA2 Chat 13B | 7.70% | 1513 |