File size: 5,060 Bytes
90ee54c a5fd31e 90ee54c 808ab18 90ee54c 808ab18 34fce3e 808ab18 90ee54c 858b728 90ee54c 34fce3e 4e17c69 1442854 67f94f8 2147365 1442854 3e62292 1442854 67f94f8 1442854 67f94f8 1442854 2147365 67f94f8 0877463 3e62292 2147365 0877463 2147365 90ee54c 5a3207a 858b728 5a3207a 858b728 dda8095 a5fd31e bc01620 90ee54c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 |
---
base_model: Doctor-Shotgun/Nous-Capybara-limarpv3-34B
language:
- en
library_name: transformers
quantized_by: mradermacher
---
## About
weighted/imatrix quants of https://huggingface.co/Doctor-Shotgun/Nous-Capybara-limarpv3-34B
<!-- provided-files -->
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-IQ1_S.gguf) | i1-IQ1_S | 8.2 | for the desperate |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-IQ1_M.gguf) | i1-IQ1_M | 8.9 | mostly desperate |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 10.0 | |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 11.0 | |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-IQ2_S.gguf) | i1-IQ2_S | 11.6 | |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-IQ2_M.gguf) | i1-IQ2_M | 12.5 | |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-Q2_K.gguf) | i1-Q2_K | 13.5 | IQ3_XXS probably better |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 14.3 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-Q3_K_XS.gguf) | i1-Q3_K_XS | 14.8 | |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 14.9 | |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 15.6 | IQ3_XS probably better |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-IQ3_S.gguf) | i1-IQ3_S | 15.7 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-IQ3_M.gguf) | i1-IQ3_M | 16.2 | |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 17.3 | IQ3_S probably better |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 18.8 | IQ3_M probably better |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 19.1 | |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-Q4_0.gguf) | i1-Q4_0 | 20.2 | fast, low quality |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 20.2 | optimal size/speed/quality |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 21.3 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 24.3 | |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 25.0 | |
| [GGUF](https://huggingface.co/mradermacher/Nous-Capybara-limarpv3-34B-i1-GGUF/resolve/main/Nous-Capybara-limarpv3-34B.i1-Q6_K.gguf) | i1-Q6_K | 28.9 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time.
<!-- end -->
|