Hugging Face's logo

MaziyarPanahi
/
mathstral-7B-v0.1-GGUF

Text Generation
Transformers
GGUF
mistral
quantized
2-bit
3-bit
4-bit precision
5-bit
6-bit
8-bit precision
GGUF
imatrix
Model card Files Community
2
MaziyarPanahi commited on
Commit
c4e1df6
•
1 Parent(s): 9d69c68

Upload folder using huggingface_hub (#1)

Browse files

- e266546623f45eb0851454548503ced1edac6b425be9ee7151cbde1c7713e560 (d86040aba0024c0748bd27e158e1885da307a9d3)
- 51e712a3b42d83d2305637b3b3fdb69e0653579d22a71457a78ccd4efe4d7533 (84ae71ba5a3fdf17110a929f3893e35e57592f26)
- 92137493a565dd1182d49f411e1c09d8cd36d8f422ae275b77ebdcd6c277ca68 (a6fc164a60cf1ca75bbae9880b7ea59bfd44e62c)
- 8e312857bf506dddea88e468002363a2959455979035c17d1f9c9d999d016d8c (5ec83ae588a1fa13dc40f681f9c262327820acb3)
- fe85905d7dda85fb7b35c11945ff2278c89bb5e45d4201d7c748084ecf501f8e (78d6dad2cc48eb5acb8f1e9d93868385992d26fe)
- 78c544e2e2fddd93566f2eb240b61f71b061c75b47ab3ea872c44cf439db329a (44d3c97144980dbffb62eaf7b7eb6b557860c85f)
- 1cda21ccaf2fa0054c116e9ee618ff3c1f5b19c9a9bc5df0b5d94e381091e220 (f939bf4bc3d4287d4d709d493d1fb1177373c98a)
- 477888e037f7ec764687f643c9793794a3f5477a210ef9b0eccd7a3ad693e2ab (782ddcb3e4980f68a35c743f6e10b76e0fb64b60)
- 633cbb0385c0e7eb348f3cbf09138447b3e5ad5701a7685ff25dd21be9f8a6df (97f7b4f207bf58cb09cbb10bad47f4dbc160ff0a)
- 9e146b2594990ba200bf525162ab3ab02b4424c192965a03fe55a9236e137b9c (9f126990937e6859aaf777ede359e93e4b0f6836)
- 61a081442fb25db71bc795a966ab2319612ac9769c924a0d3121526ba94f0491 (23a367387cead93e9a50f237253f63a70e9564ba)
- bde366ded56e73059de56eb90b1c4b80abfc5bfa5d5fd3877c7370cdca1de47b (0b8b8bf89dbd000f95b50f5f7da2f9c77c4b8c9b)
- 7cc1e04003720ca860de76d5a3cf0e6385f850c05c073a6a859158351af07b62 (292f7173d593ff06630960e588ab3b29dbcf083e)
- 8a2c4bc08c7355234ed23358655f24d90bf2e95c6cd7222dd7f358f826454751 (f68ecec2933889c5e95082b1cd502e47467b8c78)
- e5f74241ffdb4741699724f33b78cc364cbd2e586d2f8c92dd6d0cb3e517bf45 (ebfe16cedd710fb504a2a9daa8859c4cbf7a0a4a)
- dcdff5cb5ac679e82e0361e03ab2bd7da3b2e406dfb977e713ba58f141a2d288 (2bc558208434bd727fc8412fe5bc13a6d5d0f17f)

Files changed (19) hide show
  1. .gitattributes +17 -0
  2. README.md +46 -3
  3. mathstral-7B-v0.1-GGUF_imatrix.dat +3 -0
  4. mathstral-7B-v0.1.IQ1_M.gguf +3 -0
  5. mathstral-7B-v0.1.IQ1_S.gguf +3 -0
  6. mathstral-7B-v0.1.IQ2_XS.gguf +3 -0
  7. mathstral-7B-v0.1.IQ3_XS.gguf +3 -0
  8. mathstral-7B-v0.1.IQ4_XS.gguf +3 -0
  9. mathstral-7B-v0.1.Q2_K.gguf +3 -0
  10. mathstral-7B-v0.1.Q3_K_L.gguf +3 -0
  11. mathstral-7B-v0.1.Q3_K_M.gguf +3 -0
  12. mathstral-7B-v0.1.Q3_K_S.gguf +3 -0
  13. mathstral-7B-v0.1.Q4_K_M.gguf +3 -0
  14. mathstral-7B-v0.1.Q4_K_S.gguf +3 -0
  15. mathstral-7B-v0.1.Q5_K_M.gguf +3 -0
  16. mathstral-7B-v0.1.Q5_K_S.gguf +3 -0
  17. mathstral-7B-v0.1.Q6_K.gguf +3 -0
  18. mathstral-7B-v0.1.Q8_0.gguf +3 -0
  19. mathstral-7B-v0.1.fp16.gguf +3 -0
.gitattributes CHANGED
@@ -33,3 +33,20 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ mathstral-7B-v0.1-GGUF_imatrix.dat filter=lfs diff=lfs merge=lfs -text
37
+ mathstral-7B-v0.1.IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
38
+ mathstral-7B-v0.1.IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
39
+ mathstral-7B-v0.1.IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
40
+ mathstral-7B-v0.1.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
41
+ mathstral-7B-v0.1.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
42
+ mathstral-7B-v0.1.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
43
+ mathstral-7B-v0.1.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
44
+ mathstral-7B-v0.1.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
45
+ mathstral-7B-v0.1.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ mathstral-7B-v0.1.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
47
+ mathstral-7B-v0.1.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
48
+ mathstral-7B-v0.1.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
49
+ mathstral-7B-v0.1.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
50
+ mathstral-7B-v0.1.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
51
+ mathstral-7B-v0.1.fp16.gguf filter=lfs diff=lfs merge=lfs -text
52
+ mathstral-7B-v0.1.IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -1,3 +1,46 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - quantized
4
+ - 2-bit
5
+ - 3-bit
6
+ - 4-bit
7
+ - 5-bit
8
+ - 6-bit
9
+ - 8-bit
10
+ - GGUF
11
+ - text-generation
12
+ - text-generation
13
+ model_name: mathstral-7B-v0.1-GGUF
14
+ base_model: mistralai/mathstral-7B-v0.1
15
+ inference: false
16
+ model_creator: mistralai
17
+ pipeline_tag: text-generation
18
+ quantized_by: MaziyarPanahi
19
+ ---
20
+ # [MaziyarPanahi/mathstral-7B-v0.1-GGUF](https://huggingface.co/MaziyarPanahi/mathstral-7B-v0.1-GGUF)
21
+ - Model creator: [mistralai](https://huggingface.co/mistralai)
22
+ - Original model: [mistralai/mathstral-7B-v0.1](https://huggingface.co/mistralai/mathstral-7B-v0.1)
23
+
24
+ ## Description
25
+ [MaziyarPanahi/mathstral-7B-v0.1-GGUF](https://huggingface.co/MaziyarPanahi/mathstral-7B-v0.1-GGUF) contains GGUF format model files for [mistralai/mathstral-7B-v0.1](https://huggingface.co/mistralai/mathstral-7B-v0.1).
26
+
27
+ ### About GGUF
28
+
29
+ GGUF is a new format introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp.
30
+
31
+ Here is an incomplete list of clients and libraries that are known to support GGUF:
32
+
33
+ * [llama.cpp](https://github.com/ggerganov/llama.cpp). The source project for GGUF. Offers a CLI and a server option.
34
+ * [llama-cpp-python](https://github.com/abetlen/llama-cpp-python), a Python library with GPU accel, LangChain support, and OpenAI-compatible API server.
35
+ * [LM Studio](https://lmstudio.ai/), an easy-to-use and powerful local GUI for Windows and macOS (Silicon), with GPU acceleration. Linux available, in beta as of 27/11/2023.
36
+ * [text-generation-webui](https://github.com/oobabooga/text-generation-webui), the most widely used web UI, with many features and powerful extensions. Supports GPU acceleration.
37
+ * [KoboldCpp](https://github.com/LostRuins/koboldcpp), a fully featured web UI, with GPU accel across all platforms and GPU architectures. Especially good for story telling.
38
+ * [GPT4All](https://gpt4all.io/index.html), a free and open source local running GUI, supporting Windows, Linux and macOS with full GPU accel.
39
+ * [LoLLMS Web UI](https://github.com/ParisNeo/lollms-webui), a great web UI with many interesting and unique features, including a full model library for easy model selection.
40
+ * [Faraday.dev](https://faraday.dev/), an attractive and easy to use character-based chat GUI for Windows and macOS (both Silicon and Intel), with GPU acceleration.
41
+ * [candle](https://github.com/huggingface/candle), a Rust ML framework with a focus on performance, including GPU support, and ease of use.
42
+ * [ctransformers](https://github.com/marella/ctransformers), a Python library with GPU accel, LangChain support, and OpenAI-compatible AI server. Note, as of time of writing (November 27th 2023), ctransformers has not been updated in a long time and does not support many recent models.
43
+
44
+ ## Special thanks
45
+
46
+ 🙏 Special thanks to [Georgi Gerganov](https://github.com/ggerganov) and the whole team working on [llama.cpp](https://github.com/ggerganov/llama.cpp/) for making all of this possible.
mathstral-7B-v0.1-GGUF_imatrix.dat ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:76d608206705b2e7dda88418bd2d34f7aefa9c5db4336ff9d463fd2e6f54d84c
3
+ size 4988146
mathstral-7B-v0.1.IQ1_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:91ded1760f6971cc2b17c18cf2d3264804e710bb9134579c1ae461cd855c1248
3
+ size 1757663040
mathstral-7B-v0.1.IQ1_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0151710860026369b1ab26e87f0f3953ce56894d07764d8fbd23f984ed02423b
3
+ size 1615318848
mathstral-7B-v0.1.IQ2_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:150de94d14d903f60e818e64f807c9bc631134e7427cd5f1b55662db2f21be66
3
+ size 2201472832
mathstral-7B-v0.1.IQ3_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:08fa6dd35f993d1e576fb8966e40694e8a6ac2581f47b3f996b605e4c90d7f66
3
+ size 3022769984
mathstral-7B-v0.1.IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d1be2bfb3911a6fa211b1b9b9b5c653f0993b805f5e0e7701031805ff12f6500
3
+ size 3911962432
mathstral-7B-v0.1.Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:16def1a096ffcbd66e28b470fd857b7f7f08e078684a6aaa167586cac4b0049b
3
+ size 2722877248
mathstral-7B-v0.1.Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:80dbc2a90bb0b09708604d73c242880cc02c7c0fc4cf975d826c86e37a08da4c
3
+ size 3825979200
mathstral-7B-v0.1.Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a91ced3c7964ce1605e0bb917fb953791221f8a356efad0b103a82780c11591f
3
+ size 3522940736
mathstral-7B-v0.1.Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d2b40e7c4a129ba5e4f291e85087e021f3915fff44e65e0e2b559ddc4c9fdbb6
3
+ size 3168522048
mathstral-7B-v0.1.Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8db1097490577b5147f1d7284ec395614b1ddfd5b3a394f126f6d2e743ddde12
3
+ size 4372811584
mathstral-7B-v0.1.Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:782e69097cfec5a10f85a2fe5dd07e0125a6104c5de44f57d379dfe83ab8b952
3
+ size 4144746304
mathstral-7B-v0.1.Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7322a42c6adb9ce5e8afb2d7878859be478a7bddcd4f3faf686a82e42c113e56
3
+ size 5136174912
mathstral-7B-v0.1.Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8156e2a4398bdf4331e544443cd41344847d950d400335b5009ce420c8f82043
3
+ size 5002481472
mathstral-7B-v0.1.Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8e886b468cc9628b2b965beecec4a3dbdd04b2c6caf179dab63fa2e55b6adad2
3
+ size 5947248448
mathstral-7B-v0.1.Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d02e855eb3cbe5024e15dd795c2609421f71694b57df569fd48ec59fabd0d208
3
+ size 7702564672
mathstral-7B-v0.1.fp16.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6554402b424ff91edbd36f545f529227216707cd6ec51e59801dc476a9014e8b
3
+ size 14497336928