Hugging Face's logo

XelotX
/
WizardLM-2-8x22B-i1-GGUF

Transformers
GGUF
English
Inference Endpoints
Model card Files Community
XelotX impactframes commited on
Commit
38337c2
•
0 Parent(s):

Duplicate from impactframes/WizardLM-2-8x22B-i1-GGUF

Browse files

Co-authored-by: Impact Frames <[email protected]>

Files changed (38) hide show
  1. .gitattributes +71 -0
  2. README.md +63 -0
  3. WizardLM-2-8x22B.i1-IQ2_M.gguf +3 -0
  4. WizardLM-2-8x22B.i1-IQ2_XS.gguf +3 -0
  5. WizardLM-2-8x22B.i1-IQ2_XXS.gguf +3 -0
  6. WizardLM-2-8x22B.i1-IQ3_M.gguf.part1of2 +3 -0
  7. WizardLM-2-8x22B.i1-IQ3_M.gguf.part2of2 +3 -0
  8. WizardLM-2-8x22B.i1-IQ3_S.gguf.part1of2 +3 -0
  9. WizardLM-2-8x22B.i1-IQ3_S.gguf.part2of2 +3 -0
  10. WizardLM-2-8x22B.i1-IQ3_XS.gguf.part1of2 +3 -0
  11. WizardLM-2-8x22B.i1-IQ3_XS.gguf.part2of2 +3 -0
  12. WizardLM-2-8x22B.i1-IQ3_XXS.gguf.part1of2 +3 -0
  13. WizardLM-2-8x22B.i1-IQ3_XXS.gguf.part2of2 +3 -0
  14. WizardLM-2-8x22B.i1-IQ4_XS.gguf.part1of2 +3 -0
  15. WizardLM-2-8x22B.i1-IQ4_XS.gguf.part2of2 +3 -0
  16. WizardLM-2-8x22B.i1-Q2_K.gguf.part1of2 +3 -0
  17. WizardLM-2-8x22B.i1-Q2_K.gguf.part2of2 +3 -0
  18. WizardLM-2-8x22B.i1-Q3_K_L.gguf.part1of2 +3 -0
  19. WizardLM-2-8x22B.i1-Q3_K_L.gguf.part2of2 +3 -0
  20. WizardLM-2-8x22B.i1-Q3_K_M.gguf.part1of2 +3 -0
  21. WizardLM-2-8x22B.i1-Q3_K_M.gguf.part2of2 +3 -0
  22. WizardLM-2-8x22B.i1-Q3_K_S.gguf.part1of2 +3 -0
  23. WizardLM-2-8x22B.i1-Q3_K_S.gguf.part2of2 +3 -0
  24. WizardLM-2-8x22B.i1-Q4_0.gguf.part1of2 +3 -0
  25. WizardLM-2-8x22B.i1-Q4_0.gguf.part2of2 +3 -0
  26. WizardLM-2-8x22B.i1-Q4_K_M.gguf.part1of2 +3 -0
  27. WizardLM-2-8x22B.i1-Q4_K_M.gguf.part2of2 +3 -0
  28. WizardLM-2-8x22B.i1-Q4_K_S.gguf.part1of2 +3 -0
  29. WizardLM-2-8x22B.i1-Q4_K_S.gguf.part2of2 +3 -0
  30. WizardLM-2-8x22B.i1-Q5_K_M.gguf.part1of3 +3 -0
  31. WizardLM-2-8x22B.i1-Q5_K_M.gguf.part2of3 +3 -0
  32. WizardLM-2-8x22B.i1-Q5_K_M.gguf.part3of3 +3 -0
  33. WizardLM-2-8x22B.i1-Q5_K_S.gguf.part1of2 +3 -0
  34. WizardLM-2-8x22B.i1-Q5_K_S.gguf.part2of2 +3 -0
  35. WizardLM-2-8x22B.i1-Q6_K.gguf.part1of3 +3 -0
  36. WizardLM-2-8x22B.i1-Q6_K.gguf.part2of3 +3 -0
  37. WizardLM-2-8x22B.i1-Q6_K.gguf.part3of3 +3 -0
  38. imatrix.dat +3 -0
.gitattributes ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ imatrix.dat filter=lfs diff=lfs merge=lfs -text
37
+ WizardLM-2-8x22B.i1-Q2_K.gguf.part1of2 filter=lfs diff=lfs merge=lfs -text
38
+ WizardLM-2-8x22B.i1-Q2_K.gguf.part2of2 filter=lfs diff=lfs merge=lfs -text
39
+ WizardLM-2-8x22B.i1-Q4_K_S.gguf.part1of2 filter=lfs diff=lfs merge=lfs -text
40
+ WizardLM-2-8x22B.i1-Q4_K_S.gguf.part2of2 filter=lfs diff=lfs merge=lfs -text
41
+ WizardLM-2-8x22B.i1-IQ3_XXS.gguf.part1of2 filter=lfs diff=lfs merge=lfs -text
42
+ WizardLM-2-8x22B.i1-IQ3_XXS.gguf.part2of2 filter=lfs diff=lfs merge=lfs -text
43
+ WizardLM-2-8x22B.i1-Q3_K_M.gguf.part1of2 filter=lfs diff=lfs merge=lfs -text
44
+ WizardLM-2-8x22B.i1-Q3_K_M.gguf.part2of2 filter=lfs diff=lfs merge=lfs -text
45
+ WizardLM-2-8x22B.i1-Q4_K_M.gguf.part1of2 filter=lfs diff=lfs merge=lfs -text
46
+ WizardLM-2-8x22B.i1-Q4_K_M.gguf.part2of2 filter=lfs diff=lfs merge=lfs -text
47
+ WizardLM-2-8x22B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
48
+ WizardLM-2-8x22B.i1-Q6_K.gguf.part1of3 filter=lfs diff=lfs merge=lfs -text
49
+ WizardLM-2-8x22B.i1-Q6_K.gguf.part2of3 filter=lfs diff=lfs merge=lfs -text
50
+ WizardLM-2-8x22B.i1-Q6_K.gguf.part3of3 filter=lfs diff=lfs merge=lfs -text
51
+ WizardLM-2-8x22B.i1-IQ4_XS.gguf.part1of2 filter=lfs diff=lfs merge=lfs -text
52
+ WizardLM-2-8x22B.i1-IQ4_XS.gguf.part2of2 filter=lfs diff=lfs merge=lfs -text
53
+ WizardLM-2-8x22B.i1-Q3_K_S.gguf.part1of2 filter=lfs diff=lfs merge=lfs -text
54
+ WizardLM-2-8x22B.i1-Q3_K_S.gguf.part2of2 filter=lfs diff=lfs merge=lfs -text
55
+ WizardLM-2-8x22B.i1-Q3_K_L.gguf.part1of2 filter=lfs diff=lfs merge=lfs -text
56
+ WizardLM-2-8x22B.i1-Q3_K_L.gguf.part2of2 filter=lfs diff=lfs merge=lfs -text
57
+ WizardLM-2-8x22B.i1-Q5_K_S.gguf.part1of2 filter=lfs diff=lfs merge=lfs -text
58
+ WizardLM-2-8x22B.i1-Q5_K_S.gguf.part2of2 filter=lfs diff=lfs merge=lfs -text
59
+ WizardLM-2-8x22B.i1-Q5_K_M.gguf.part1of3 filter=lfs diff=lfs merge=lfs -text
60
+ WizardLM-2-8x22B.i1-Q5_K_M.gguf.part2of3 filter=lfs diff=lfs merge=lfs -text
61
+ WizardLM-2-8x22B.i1-Q5_K_M.gguf.part3of3 filter=lfs diff=lfs merge=lfs -text
62
+ WizardLM-2-8x22B.i1-Q4_0.gguf.part1of2 filter=lfs diff=lfs merge=lfs -text
63
+ WizardLM-2-8x22B.i1-Q4_0.gguf.part2of2 filter=lfs diff=lfs merge=lfs -text
64
+ WizardLM-2-8x22B.i1-IQ3_XS.gguf.part1of2 filter=lfs diff=lfs merge=lfs -text
65
+ WizardLM-2-8x22B.i1-IQ3_XS.gguf.part2of2 filter=lfs diff=lfs merge=lfs -text
66
+ WizardLM-2-8x22B.i1-IQ3_S.gguf.part1of2 filter=lfs diff=lfs merge=lfs -text
67
+ WizardLM-2-8x22B.i1-IQ3_S.gguf.part2of2 filter=lfs diff=lfs merge=lfs -text
68
+ WizardLM-2-8x22B.i1-IQ3_M.gguf.part1of2 filter=lfs diff=lfs merge=lfs -text
69
+ WizardLM-2-8x22B.i1-IQ3_M.gguf.part2of2 filter=lfs diff=lfs merge=lfs -text
70
+ WizardLM-2-8x22B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
71
+ WizardLM-2-8x22B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,63 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ exported_from: microsoft/WizardLM-2-8x22B
3
+ language:
4
+ - en
5
+ library_name: transformers
6
+ quantized_by: mradermacher
7
+ ---
8
+ ## About
9
+
10
+ <!-- ### quantize_version: 1 -->
11
+ <!-- ### output_tensor_quantised: 1 -->
12
+ <!-- ### convert_type: -->
13
+ <!-- ### vocab_type: -->
14
+ weighted/imatrix quants of https://huggingface.co/microsoft/WizardLM-2-8x22B
15
+
16
+ <!-- provided-files -->
17
+ static quants are available at https://huggingface.co/mradermacher/WizardLM-2-8x22B-GGUF
18
+ ## Usage
19
+
20
+ If you are unsure how to use GGUF files, refer to one of [TheBloke's
21
+ READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
22
+ more details, including on how to concatenate multi-part files.
23
+
24
+ ## Provided Quants
25
+
26
+ (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
27
+
28
+ | Link | Type | Size/GB | Notes |
29
+ |:-----|:-----|--------:|:------|
30
+ | [GGUF](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 38.0 | |
31
+ | [GGUF](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-IQ2_M.gguf) | i1-IQ2_M | 46.8 | |
32
+ | [PART 1](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q2_K.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q2_K.gguf.part2of2) | i1-Q2_K | 52.2 | IQ3_XXS probably better |
33
+ | [PART 1](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-IQ3_XXS.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-IQ3_XXS.gguf.part2of2) | i1-IQ3_XXS | 55.0 | lower quality |
34
+ | [PART 1](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-IQ3_XS.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-IQ3_XS.gguf.part2of2) | i1-IQ3_XS | 58.3 | |
35
+ | [PART 1](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-IQ3_S.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-IQ3_S.gguf.part2of2) | i1-IQ3_S | 61.6 | beats Q3_K* |
36
+ | [PART 1](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q3_K_S.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q3_K_S.gguf.part2of2) | i1-Q3_K_S | 61.6 | IQ3_XS probably better |
37
+ | [PART 1](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-IQ3_M.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-IQ3_M.gguf.part2of2) | i1-IQ3_M | 64.6 | |
38
+ | [PART 1](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q3_K_M.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q3_K_M.gguf.part2of2) | i1-Q3_K_M | 67.9 | IQ3_S probably better |
39
+ | [PART 1](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q3_K_L.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q3_K_L.gguf.part2of2) | i1-Q3_K_L | 72.7 | IQ3_M probably better |
40
+ | [PART 1](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-IQ4_XS.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-IQ4_XS.gguf.part2of2) | i1-IQ4_XS | 75.6 | |
41
+ | [PART 1](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q4_0.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q4_0.gguf.part2of2) | i1-Q4_0 | 80.0 | fast, low quality |
42
+ | [PART 1](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q4_K_S.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q4_K_S.gguf.part2of2) | i1-Q4_K_S | 80.6 | optimal size/speed/quality |
43
+ | [PART 1](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q4_K_M.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q4_K_M.gguf.part2of2) | i1-Q4_K_M | 85.7 | fast, recommended |
44
+ | [PART 1](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q5_K_S.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q5_K_S.gguf.part2of2) | i1-Q5_K_S | 97.1 | |
45
+ | [PART 1](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q5_K_M.gguf.part1of3) [PART 2](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q5_K_M.gguf.part2of3) [PART 3](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q5_K_M.gguf.part3of3) | i1-Q5_K_M | 100.1 | |
46
+ | [PART 1](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q6_K.gguf.part1of3) [PART 2](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q6_K.gguf.part2of3) [PART 3](https://huggingface.co/mradermacher/WizardLM-2-8x22B-i1-GGUF/resolve/main/WizardLM-2-8x22B.i1-Q6_K.gguf.part3of3) | i1-Q6_K | 115.6 | practically like static Q6_K |
47
+
48
+
49
+ Here is a handy graph by ikawrakow comparing some lower-quality quant
50
+ types (lower is better):
51
+
52
+ ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
53
+
54
+ And here are Artefact2's thoughts on the matter:
55
+ https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
56
+
57
+ ## Thanks
58
+
59
+ I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
60
+ me use its servers and providing upgrades to my workstation to enable
61
+ this work in my free time.
62
+
63
+ <!-- end -->
WizardLM-2-8x22B.i1-IQ2_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:52eba2dc9c805e7ef659c5fcedf8d69f5b262e2a29a1117f405be7afe0bb7ee3
3
+ size 46713106528
WizardLM-2-8x22B.i1-IQ2_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bfb7e91e7d5ecbd67a0906d79b62625f5623a300b3baea09c79df66f771398a5
3
+ size 42002858080
WizardLM-2-8x22B.i1-IQ2_XXS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e6d06bf92cfa96c95a09e16d2c28860092c9cfd68f85e6b1f8136e7c284dc98d
3
+ size 37885100128
WizardLM-2-8x22B.i1-IQ3_M.gguf.part1of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:94e7087049b782de5bc2778a6ed16c8ce8e848af7640c38e65f1ea02aa3f7b2c
3
+ size 33285996544
WizardLM-2-8x22B.i1-IQ3_M.gguf.part2of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8b02983b230d6a8a376f469b2eb818a706246e4239cdf34087f3fe0f91240be5
3
+ size 31212429408
WizardLM-2-8x22B.i1-IQ3_S.gguf.part1of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:74bd5a6e1808688026c5decfeca0e0125d4a9d6e9cb89e2f902f1b6a97f2c4a8
3
+ size 31138512896
WizardLM-2-8x22B.i1-IQ3_S.gguf.part2of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f5dd60fb375cc7200e773c30a05e9f3f8f57cda9cd1a65c2d66c11c1b8ce3bbe
3
+ size 30365180000
WizardLM-2-8x22B.i1-IQ3_XS.gguf.part1of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:93ebdbf1c9e679a10bb2ac4da26e71fd03ef88a1a7329c4fa4f88ee781738e6f
3
+ size 30064771072
WizardLM-2-8x22B.i1-IQ3_XS.gguf.part2of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9b6ebb3a2c61da7fd69c5c5536f2a27c6a22d3c7b94bb007d348b196c7161a3b
3
+ size 28168937568
WizardLM-2-8x22B.i1-IQ3_XXS.gguf.part1of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6df7de40faa5936f70661080f5b45fd1ea29d05d71fcdc1bcb4aead18d0530c3
3
+ size 27917287424
WizardLM-2-8x22B.i1-IQ3_XXS.gguf.part2of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c875423c2c8c4bdf107de9e947ed574f4a89bc81624bfef4047e08996e343124
3
+ size 26987294816
WizardLM-2-8x22B.i1-IQ4_XS.gguf.part1of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a2e7f42da3e2305c17b5de2499dca458361dd475119eedd236491c903f249148
3
+ size 38654705664
WizardLM-2-8x22B.i1-IQ4_XS.gguf.part2of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fa0129c572a6b845439f87c3bda8d4bf36c6be7ad8eb1dfecac924235928abe6
3
+ size 36824191072
WizardLM-2-8x22B.i1-Q2_K.gguf.part1of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:84fa99f34fbbcfdf0a6432a7e9d6a552bba341aa7628352283091fb3e8af85be
3
+ size 26843545600
WizardLM-2-8x22B.i1-Q2_K.gguf.part2of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4ba383191050a21fc818bc9d0ff61959c3f9479df2763395aa9a51d0bf20cf37
3
+ size 25265123424
WizardLM-2-8x22B.i1-Q3_K_L.gguf.part1of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cf044d9e4f6e7e1a378c70ff640eb1f78b115e590a1b8502e7d712911442a1ab
3
+ size 36507222016
WizardLM-2-8x22B.i1-Q3_K_L.gguf.part2of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2854df45214ef1e9c59e81412a99260373a13f0064d9dbf192722e8377522665
3
+ size 36078084192
WizardLM-2-8x22B.i1-Q3_K_M.gguf.part1of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:35f0c52078e4f93757ea858c999d80436d658980f8054544255632b3e31b733d
3
+ size 34359738368
WizardLM-2-8x22B.i1-Q3_K_M.gguf.part2of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3d58d088e410c69cff7f908e99e4351c5fcef8ca3508d0d4ab04b6360dd9c693
3
+ size 33435410528
WizardLM-2-8x22B.i1-Q3_K_S.gguf.part1of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c10ecadf9a8f6bf6d9ca733c5fde3894716296321d1753826427f860f7834cb4
3
+ size 31138512896
WizardLM-2-8x22B.i1-Q3_K_S.gguf.part2of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:088537aa1d703ac5a1bcb1437da466c8f95a2e88a5e5100b2f18eefc9735c3f5
3
+ size 30365180000
WizardLM-2-8x22B.i1-Q4_0.gguf.part1of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ec5556554faaf7c3566208e001c6f63d7e945def29bd7de97706f366c036ad61
3
+ size 40802189312
WizardLM-2-8x22B.i1-Q4_0.gguf.part2of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5164015a118627a45c89ef8e824a9a183a1540a0441ee2c9a164edb8f9c70f29
3
+ size 39064850528
WizardLM-2-8x22B.i1-Q4_K_M.gguf.part1of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3769982967ff11e0da2eb641d48f3e03eb009b8b5fbafca50359ccc048f55dba
3
+ size 42949672960
WizardLM-2-8x22B.i1-Q4_K_M.gguf.part2of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fa4216bf54d2917710e98c3946d1c8bea326978aaaf86ecd2132360b845771cb
3
+ size 42642591840
WizardLM-2-8x22B.i1-Q4_K_S.gguf.part1of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:77b194e655b271f2c508c0df04b373410c15aaf945b864a15c1bedae81d02f85
3
+ size 40802189312
WizardLM-2-8x22B.i1-Q4_K_S.gguf.part2of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0397f8dc72b06f6441b350890df050726c0f23084e92c36c0377ddce87ae9a98
3
+ size 39681413216
WizardLM-2-8x22B.i1-Q5_K_M.gguf.part1of3 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8445a4c24a8d72c8c26bd6f95a96a12ca59ab5b021a4441aa8b9e25aeefc8c6d
3
+ size 34359738368
WizardLM-2-8x22B.i1-Q5_K_M.gguf.part2of3 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8f1ca8a38c171771ec58fa1de58c0306aaee933bb983a5a390e3b84b91c4ecb4
3
+ size 34359738368
WizardLM-2-8x22B.i1-Q5_K_M.gguf.part3of3 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f1e1cda1ff63eb77bf38d586d6fc45383ce6f1c45a38865cd21c1c973941c6b6
3
+ size 31254466656
WizardLM-2-8x22B.i1-Q5_K_S.gguf.part1of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8e6b2ba4ca38ddffc540dec5689f2e96be5a2547dd3c3d8305fa0bef54d499d8
3
+ size 49392123904
WizardLM-2-8x22B.i1-Q5_K_S.gguf.part2of2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e00cf39292e9ffe672a9caae0bd8ea738008e2e06802adcc7b88caf505e48d36
3
+ size 47587086432
WizardLM-2-8x22B.i1-Q6_K.gguf.part1of3 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2f18cefa40e3b14f93a67875cf1c2d4ea8769d09419686d949aa54834da68819
3
+ size 38654705664
WizardLM-2-8x22B.i1-Q6_K.gguf.part2of3 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b5e7a20ca4e31d75da7506b90de9c33ef547232dfa5b0da798dfd26152439aa0
3
+ size 38654705664
WizardLM-2-8x22B.i1-Q6_K.gguf.part3of3 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:64f0f68e67d6fbac54bd5c5ec2e06258c3301cd9da848938b9299cc8f101b1fa
3
+ size 38225821792
imatrix.dat ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cb7738e9dc957318c23beb1c8237ed1d4889e0fbc011d248fd3cd2d08cb6cbc8
3
+ size 58277444