General ExLLamaV2 (exl2) Quants

bigstorm 's Collections

updated Oct 18

Higher end quants to preserve model quality. Built for TabbyAPI and other exllamav2 supporting inference engines. Ready to deploy.

Upvote

bigstorm/dolphin-2.9.2-qwen2-72b-6.0bpw-exl2

Text Generation • Updated Jun 21 • 6 • 1
bigstorm/dolphin-2.9.1-llama-3-70b-6.0bpw-exl2

Text Generation • Updated Jul 4 • 8 • 3
bigstorm/Hermes-2-Theta-Llama-3-8B-8.0bpw-8hb-exl2

Text Generation • Updated Jun 23 • 3
bigstorm/Yi-1.5-34B-Chat-16K-8.0bpw-8hb-exl2

Text Generation • Updated Jun 24 • 5
bigstorm/Codestral-22B-v0.1-8.0bpw-8hb-exl2

Text Generation • Updated Jun 23 • 6
bigstorm/dolphin-2.9.2-qwen2-72b-7.0bpw-8hb-exl2

Text Generation • Updated Jul 1 • 11 • 2
bigstorm/Hermes-2-Theta-Llama-3-70B-7.0bpw-8hb-exl2

Text Generation • Updated Jul 18 • 1
bigstorm/Qwen2-72B-Instruct-7.0bpw-8hb-exl2

Text Generation • Updated Jul 18 • 7
bigstorm/Qwen2-72B-Instruct-8.0bpw-8hb-exl2

Text Generation • Updated Jul 18 • 5
bigstorm/Mixtral-8x22B-Instruct-v0.1-7.0bpw-8hb-exl2

Updated Jun 25
bigstorm/gemma-2-27b-it-8.0bpw-8hb-exl2

Text Generation • Updated Jul 18 • 13
bigstorm/Llama-3-Groq-8B-Tool-Use-8.0bpw-8hb-exl2

Text Generation • Updated Jul 18 • 6
bigstorm/Llama-3-Groq-70B-Tool-Use-8.0bpw-8hb-exl2

Text Generation • Updated Jul 18 • 4 • 1
bigstorm/Mistral-Nemo-Base-2407-8.0bpw-8hb-exl2

Text Generation • Updated Jul 18 • 7
bigstorm/Mistral-Nemo-Instruct-2407-8.0bpw-8hb-exl2

Text Generation • Updated Jul 18 • 6
bigstorm/Meta-Llama-3.1-70B-Instruct-8.0bpw-8hb-exl2

Text Generation • Updated Jul 23 • 10
bigstorm/Meta-Llama-3.1-70B-Instruct-7.0bpw-8hb-exl2

Text Generation • Updated Jul 24 • 9
bigstorm/Meta-Llama-3.1-70B-Instruct-6.5bpw-8hb-exl2

Text Generation • Updated Jul 24 • 7
bigstorm/Hermes-3-Llama-3.1-70B-6.5bpw-8hb-exl2

Updated Aug 22 • 1
bigstorm/Hermes-3-Llama-3.1-70B-7.0bpw-8hb-exl2

Updated Aug 26 • 5
bigstorm/Hermes-3-Llama-3.1-70B-8.0bpw-8hb-exl2

Updated Aug 26 • 7
bigstorm/Phi-3.5-mini-instruct-6.5bpw-8hb-exl2

Text Generation • Updated Aug 27 • 8
bigstorm/dolphin-2.9.4-llama3.1-8b-7.0bpw-8hb-exl2

Updated Aug 28 • 5
bigstorm/Mistral-Large-Instruct-2407-5.5bpw-8hb-exl2

Updated Sep 4 • 3
bigstorm/Mistral-Large-Instruct-2407-5.0bpw-8hb-exl2

Updated Sep 4 • 3
bigstorm/Mistral-Large-Instruct-2407-4.5bpw-8hb-exl2

Updated Sep 4
bigstorm/Mistral-Large-Instruct-2407-6.0bpw-8hb-exl2

Updated Sep 4 • 4
bigstorm/Mistral-Large-Instruct-2407-6.5bpw-8hb-exl2

Updated Sep 4 • 3
bigstorm/Reflections-Llama-3.1-70B-8.0bpw-8hb-exl2

Text Generation • Updated Sep 6 • 6
bigstorm/Reflections-Llama-3.1-70B-6.0bpw-8hb-exl2

Text Generation • Updated Sep 6 • 5 • 1
bigstorm/dolphin-2.9.4-llama3.1-8b-6.0bpw-6hb-exl2

Updated Sep 14 • 3
bigstorm/dolphin-2.9.4-llama3.1-8b-6.0bpw-8hb-exl2

Updated Sep 14 • 2
bigstorm/dolphin-2.9.3-mistral-nemo-12b-6.0bpw-6hb-exl2

Updated Sep 15 • 1
bigstorm/dolphin-2.9.3-llama-3-8b-6.0bpw-6hb-exl2

Updated Sep 16 • 3
bigstorm/dolphin-2.9.1-llama-3-8b-6.0bpw-6hb-exl2

Updated Sep 16 • 4
bigstorm/Qwen2.5-72B-Instruct-6.0bpw-6hb-exl2

Text Generation • Updated Sep 20 • 7
bigstorm/Qwen2.5-72B-Instruct-7.0bpw-8hb-exl2

Text Generation • Updated Sep 20 • 18
bigstorm/Llama-3.2-1B-8.0bpw-8hb-exl2

Text Generation • Updated Oct 2 • 13
bigstorm/Qwama-0.5B-Instruct-8.0bpw-8hb-exl2

Updated Oct 2 • 5
bigstorm/Llama-3.1-Nemotron-70B-Instruct-HF-8.0bpw-8hb-exl2

Text Generation • Updated Oct 16 • 134 • 3
bigstorm/Llama-3.1-Nemotron-70B-Instruct-HF-7.0bpw-8hb-exl2

Text Generation • Updated Oct 16 • 7 • 1
bigstorm/Llama-3.1-Nemotron-70B-Instruct-HF-4.3bpw-6hb-exl2

Text Generation • Updated Oct 16 • 44 • 3
bigstorm/Llama-3.1-Nemotron-70B-Instruct-HF-6.0bpw-8hb-exl2

Text Generation • Updated Oct 16 • 43
bigstorm/Llama-3.1-Nemotron-70B-Instruct-HF-3.0bpw-6hb-exl2

Text Generation • Updated Oct 18 • 10
bigstorm/Llama-3.1-Nemotron-70B-Instruct-HF-3.5bpw-6hb-exl2

Text Generation • Updated Oct 18 • 9

Upvote