Hugging Face's logo

bartowski
/
labradorite-13b-GGUF

Text Generation
GGUF
English
labradorite
llama
llama-2
ibm
lab
labrador
merlinite
Inference Endpoints
Model card Files Community
bartowski commited on
Commit
0969b3f
•
1 Parent(s): ae13772

Llamacpp quants

Browse files
Files changed (15) hide show
  1. .gitattributes +13 -0
  2. README.md +41 -0
  3. labradorite-13b-Q2_K.gguf +3 -0
  4. labradorite-13b-Q3_K_L.gguf +3 -0
  5. labradorite-13b-Q3_K_M.gguf +3 -0
  6. labradorite-13b-Q3_K_S.gguf +3 -0
  7. labradorite-13b-Q4_0.gguf +3 -0
  8. labradorite-13b-Q4_K_M.gguf +3 -0
  9. labradorite-13b-Q4_K_S.gguf +3 -0
  10. labradorite-13b-Q5_0.gguf +3 -0
  11. labradorite-13b-Q5_K_M.gguf +3 -0
  12. labradorite-13b-Q5_K_S.gguf +3 -0
  13. labradorite-13b-Q6_K.gguf +3 -0
  14. labradorite-13b-Q8_0.gguf +3 -0
  15. labradorite-13b-fp16.gguf +3 -0
.gitattributes CHANGED
@@ -33,3 +33,16 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ labradorite-13b-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ labradorite-13b-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
38
+ labradorite-13b-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ labradorite-13b-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
40
+ labradorite-13b-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
41
+ labradorite-13b-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
42
+ labradorite-13b-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
43
+ labradorite-13b-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
44
+ labradorite-13b-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
45
+ labradorite-13b-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
46
+ labradorite-13b-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
47
+ labradorite-13b-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ labradorite-13b-fp16.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: text-generation
3
+ tags:
4
+ - labradorite
5
+ - llama
6
+ - llama-2
7
+ - ibm
8
+ - lab
9
+ - labrador
10
+ - merlinite
11
+ license: llama2
12
+ license_link: https://ai.meta.com/llama/license/
13
+ language:
14
+ - en
15
+ quantized_by: bartowski
16
+ ---
17
+
18
+ ## Llamacpp Quantizations of labradorite-13b
19
+
20
+ Using <a href="https://github.com/ggerganov/llama.cpp/commit/fa974646e1a2024fc7dc9e6f27cf1f2f5d4a3763">llama.cpp commit fa97464</a> for quantization.
21
+
22
+ Original model: https://huggingface.co/ibm/labradorite-13b
23
+
24
+ Download a file (not the whole branch) from below:
25
+
26
+ | Filename | Quant type | File Size | Description |
27
+ | -------- | ---------- | --------- | ----------- |
28
+ | [labradorite-13b-Q8_0.gguf](https://huggingface.co/bartowski/labradorite-13b-GGUF/blob/main/labradorite-13b-Q8_0.gguf) | Q8_0 | 13.83GB | Extremely high quality, generally unneeded but max available quant. |
29
+ | [labradorite-13b-Q6_K.gguf](https://huggingface.co/bartowski/labradorite-13b-GGUF/blob/main/labradorite-13b-Q6_K.gguf) | Q6_K | 10.67GB | Very high quality, near perfect, *recommended*. |
30
+ | [labradorite-13b-Q5_K_M.gguf](https://huggingface.co/bartowski/labradorite-13b-GGUF/blob/main/labradorite-13b-Q5_K_M.gguf) | Q5_K_M | 9.22GB | High quality, very usable. |
31
+ | [labradorite-13b-Q5_K_S.gguf](https://huggingface.co/bartowski/labradorite-13b-GGUF/blob/main/labradorite-13b-Q5_K_S.gguf) | Q5_K_S | 8.97GB | High quality, very usable. |
32
+ | [labradorite-13b-Q5_0.gguf](https://huggingface.co/bartowski/labradorite-13b-GGUF/blob/main/labradorite-13b-Q5_0.gguf) | Q5_0 | 8.97GB | High quality, older format, generally not recommended. |
33
+ | [labradorite-13b-Q4_K_M.gguf](https://huggingface.co/bartowski/labradorite-13b-GGUF/blob/main/labradorite-13b-Q4_K_M.gguf) | Q4_K_M | 7.86GB | Good quality, similar to 4.25 bpw. |
34
+ | [labradorite-13b-Q4_K_S.gguf](https://huggingface.co/bartowski/labradorite-13b-GGUF/blob/main/labradorite-13b-Q4_K_S.gguf) | Q4_K_S | 7.42GB | Slightly lower quality with small space savings. |
35
+ | [labradorite-13b-Q4_0.gguf](https://huggingface.co/bartowski/labradorite-13b-GGUF/blob/main/labradorite-13b-Q4_0.gguf) | Q4_0 | 7.36GB | Decent quality, older format, generally not recommended. |
36
+ | [labradorite-13b-Q3_K_L.gguf](https://huggingface.co/bartowski/labradorite-13b-GGUF/blob/main/labradorite-13b-Q3_K_L.gguf) | Q3_K_L | 6.92GB | Lower quality but usable, good for low RAM availability. |
37
+ | [labradorite-13b-Q3_K_M.gguf](https://huggingface.co/bartowski/labradorite-13b-GGUF/blob/main/labradorite-13b-Q3_K_M.gguf) | Q3_K_M | 6.33GB | Even lower quality. |
38
+ | [labradorite-13b-Q3_K_S.gguf](https://huggingface.co/bartowski/labradorite-13b-GGUF/blob/main/labradorite-13b-Q3_K_S.gguf) | Q3_K_S | 5.65GB | Low quality, not recommended. |
39
+ | [labradorite-13b-Q2_K.gguf](https://huggingface.co/bartowski/labradorite-13b-GGUF/blob/main/labradorite-13b-Q2_K.gguf) | Q2_K | 4.85GB | Extremely low quality, *not* recommended.
40
+
41
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski
labradorite-13b-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:706b55351a2e41bcc13be3a0fea1ae5ef9bb15b9ab22cc1ad87c1232c96e7604
3
+ size 4854317216
labradorite-13b-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e9b19a2aeaf53326a4e8d4366fe895b7360f4e9d10cec3fe088004b33196c9d
3
+ size 6929610976
labradorite-13b-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:569fe4d365dabd3765c11955f424101929a9a03ce2dc0bc5c1ff0d0dda21f54f
3
+ size 6337820896
labradorite-13b-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bc75526a1eedc4383af30f96f29f3e9be903853db08554e37ca8ee8efe216b07
3
+ size 5659031776
labradorite-13b-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:39c2a3cb4eb714825c8e357319a02a7b82a79cd22e229678542036a5263614db
3
+ size 7365891616
labradorite-13b-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:331bb7417d64109629d092a0e151b383954146cb2fd6e94f541b82c26df914ef
3
+ size 7866013216
labradorite-13b-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:59678e86bf91890ab4d0cabddfcebfa7f4b3a3d582c6800c74d0bc1d589d9d95
3
+ size 7423235616
labradorite-13b-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af9850f93e882abb26ba74f8b9b38880f3497e0af9237d932c6521e23164a5d1
3
+ size 8972347936
labradorite-13b-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f6ee72b46c649ed552e4b5cbb8cfd76be464516d665252f56ea6696bc3ec4c0
3
+ size 9229986336
labradorite-13b-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e015e2ab10ffff86c343ce360a7aefa220f06be3c5901b7ab665825567ac29a7
3
+ size 8972347936
labradorite-13b-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cb7fcf834d51d15d76134ad88a920adf0792e39024189db1f0ea1363a2effcde
3
+ size 10679207776
labradorite-13b-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ec5b5ec4aa6807192863837019dd6860403e9166c259d48d638570a68b975124
3
+ size 13831406816
labradorite-13b-fp16.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e9bc81391dccd700e4d3a52a4ccddef4ed10bf2b975f9207414efa6e41498cbd
3
+ size 26033467552