Hugging Face's logo

bartowski
/
Liberated-Qwen1.5-7B-GGUF-old

Text Generation
GGUF
English
Inference Endpoints
conversational
Model card Files Community
2
bartowski commited on
Commit
ed61f41
•
1 Parent(s): 7a32b88

Llamacpp quants

Browse files
Files changed (14) hide show
  1. .gitattributes +12 -0
  2. Liberated-Qwen1.5-7B-Q2_K.gguf +3 -0
  3. Liberated-Qwen1.5-7B-Q3_K_L.gguf +3 -0
  4. Liberated-Qwen1.5-7B-Q3_K_M.gguf +3 -0
  5. Liberated-Qwen1.5-7B-Q3_K_S.gguf +3 -0
  6. Liberated-Qwen1.5-7B-Q4_0.gguf +3 -0
  7. Liberated-Qwen1.5-7B-Q4_K_M.gguf +3 -0
  8. Liberated-Qwen1.5-7B-Q4_K_S.gguf +3 -0
  9. Liberated-Qwen1.5-7B-Q5_0.gguf +3 -0
  10. Liberated-Qwen1.5-7B-Q5_K_M.gguf +3 -0
  11. Liberated-Qwen1.5-7B-Q5_K_S.gguf +3 -0
  12. Liberated-Qwen1.5-7B-Q6_K.gguf +3 -0
  13. Liberated-Qwen1.5-7B-Q8_0.gguf +3 -0
  14. README.md +39 -0
.gitattributes CHANGED
@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Liberated-Qwen1.5-7B-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Liberated-Qwen1.5-7B-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Liberated-Qwen1.5-7B-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Liberated-Qwen1.5-7B-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Liberated-Qwen1.5-7B-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Liberated-Qwen1.5-7B-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Liberated-Qwen1.5-7B-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Liberated-Qwen1.5-7B-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Liberated-Qwen1.5-7B-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Liberated-Qwen1.5-7B-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Liberated-Qwen1.5-7B-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Liberated-Qwen1.5-7B-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
Liberated-Qwen1.5-7B-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f5594a0ecb4c22e0b7063d76a2404f1de2cc6ab7fdf23f29e2325cc7b1a89d13
3
+ size 3101701984
Liberated-Qwen1.5-7B-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3d90c81d28a125fada46f61ca335b766d4c4a3545d95a1a6541f750da67d6d22
3
+ size 4215842656
Liberated-Qwen1.5-7B-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e68448f6fd2bb7751555a490d0d816d5d235633c5dda08ae2695cf97d35d8b6
3
+ size 3916736352
Liberated-Qwen1.5-7B-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3bc1d98943559e1a879ec27d6ced54c760f537e22f2beb26e9b356193828b0ed
3
+ size 3567036256
Liberated-Qwen1.5-7B-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:71a6ba121b389a28b1fc78c9271a57c51026727ced6e40ecb2575392ca1b8985
3
+ size 4509783904
Liberated-Qwen1.5-7B-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4c80a2e218713fb6b0afa3ffc5a3a8a07c131ecf85aa632ef1f2a699fc3c6656
3
+ size 4764981088
Liberated-Qwen1.5-7B-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8a21930136e9a715f14cd0ab5b3b7776ceb70ca1bb394aab9bd53b07a2640b9c
3
+ size 4540716896
Liberated-Qwen1.5-7B-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:19b68985f129d01b2dcf00c24ecbed5e7037b0b6ea16cf920cb1a295dcbf34c2
3
+ size 5397075808
Liberated-Qwen1.5-7B-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d1335ef7ca0e81dcacb7eb858c310c2d72aad54c262dcae8286a8907834503d1
3
+ size 5528541024
Liberated-Qwen1.5-7B-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ab2d5d1a4a32e719783f118c25d13032fa278ee34ae04a473204a5f80b37f5ea
3
+ size 5397075808
Liberated-Qwen1.5-7B-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2c9dcec0d2df35d369e86f0583a243befbb156095b79b8ddfde714fb66f46748
3
+ size 6339823456
Liberated-Qwen1.5-7B-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4aa56da07a2cc418755106817b9d6468844a137ffecb017ee2abca5ab86f501c
3
+ size 8209672032
README.md ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: other
5
+ datasets:
6
+ - teknium/OpenHermes-2.5
7
+ - m-a-p/Code-Feedback
8
+ - m-a-p/CodeFeedback-Filtered-Instruction
9
+ - abacusai/SystemChat
10
+ license_name: tongyi-qianwen
11
+ license_link: https://huggingface.co/Qwen/Qwen1.5-72B/blob/main/LICENSE
12
+ quantized_by: bartowski
13
+ pipeline_tag: text-generation
14
+ ---
15
+
16
+ ## Llamacpp Quantizations of Liberated-Qwen1.5-7B
17
+
18
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2405">b2405</a> for quantization.
19
+
20
+ Original model: https://huggingface.co/abacusai/Liberated-Qwen1.5-7B
21
+
22
+ Download a file (not the whole branch) from below:
23
+
24
+ | Filename | Quant type | File Size | Description |
25
+ | -------- | ---------- | --------- | ----------- |
26
+ | [Liberated-Qwen1.5-7B-Q8_0.gguf](https://huggingface.co/bartowski/Liberated-Qwen1.5-7B-GGUF/blob/main/Liberated-Qwen1.5-7B-Q8_0.gguf) | Q8_0 | 8.20GB | Extremely high quality, generally unneeded but max available quant. |
27
+ | [Liberated-Qwen1.5-7B-Q6_K.gguf](https://huggingface.co/bartowski/Liberated-Qwen1.5-7B-GGUF/blob/main/Liberated-Qwen1.5-7B-Q6_K.gguf) | Q6_K | 6.33GB | Very high quality, near perfect, *recommended*. |
28
+ | [Liberated-Qwen1.5-7B-Q5_K_M.gguf](https://huggingface.co/bartowski/Liberated-Qwen1.5-7B-GGUF/blob/main/Liberated-Qwen1.5-7B-Q5_K_M.gguf) | Q5_K_M | 5.52GB | High quality, very usable. |
29
+ | [Liberated-Qwen1.5-7B-Q5_K_S.gguf](https://huggingface.co/bartowski/Liberated-Qwen1.5-7B-GGUF/blob/main/Liberated-Qwen1.5-7B-Q5_K_S.gguf) | Q5_K_S | 5.39GB | High quality, very usable. |
30
+ | [Liberated-Qwen1.5-7B-Q5_0.gguf](https://huggingface.co/bartowski/Liberated-Qwen1.5-7B-GGUF/blob/main/Liberated-Qwen1.5-7B-Q5_0.gguf) | Q5_0 | 5.39GB | High quality, older format, generally not recommended. |
31
+ | [Liberated-Qwen1.5-7B-Q4_K_M.gguf](https://huggingface.co/bartowski/Liberated-Qwen1.5-7B-GGUF/blob/main/Liberated-Qwen1.5-7B-Q4_K_M.gguf) | Q4_K_M | 4.76GB | Good quality, similar to 4.25 bpw. |
32
+ | [Liberated-Qwen1.5-7B-Q4_K_S.gguf](https://huggingface.co/bartowski/Liberated-Qwen1.5-7B-GGUF/blob/main/Liberated-Qwen1.5-7B-Q4_K_S.gguf) | Q4_K_S | 4.54GB | Slightly lower quality with small space savings. |
33
+ | [Liberated-Qwen1.5-7B-Q4_0.gguf](https://huggingface.co/bartowski/Liberated-Qwen1.5-7B-GGUF/blob/main/Liberated-Qwen1.5-7B-Q4_0.gguf) | Q4_0 | 4.50GB | Decent quality, older format, generally not recommended. |
34
+ | [Liberated-Qwen1.5-7B-Q3_K_L.gguf](https://huggingface.co/bartowski/Liberated-Qwen1.5-7B-GGUF/blob/main/Liberated-Qwen1.5-7B-Q3_K_L.gguf) | Q3_K_L | 4.21GB | Lower quality but usable, good for low RAM availability. |
35
+ | [Liberated-Qwen1.5-7B-Q3_K_M.gguf](https://huggingface.co/bartowski/Liberated-Qwen1.5-7B-GGUF/blob/main/Liberated-Qwen1.5-7B-Q3_K_M.gguf) | Q3_K_M | 3.91GB | Even lower quality. |
36
+ | [Liberated-Qwen1.5-7B-Q3_K_S.gguf](https://huggingface.co/bartowski/Liberated-Qwen1.5-7B-GGUF/blob/main/Liberated-Qwen1.5-7B-Q3_K_S.gguf) | Q3_K_S | 3.56GB | Low quality, not recommended. |
37
+ | [Liberated-Qwen1.5-7B-Q2_K.gguf](https://huggingface.co/bartowski/Liberated-Qwen1.5-7B-GGUF/blob/main/Liberated-Qwen1.5-7B-Q2_K.gguf) | Q2_K | 3.10GB | Extremely low quality, *not* recommended.
38
+
39
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski