ikawrakow
/

various-2bit-sota-gguf

Inference Endpoints

Model card Files Files and versions Community

Resources

View closed (0)

What 2bit quantization approach are you using?

#13 opened 8 months ago by

Not able to use with ctransformers

#12 opened 9 months ago by

NousCapybara 34b

#11 opened 9 months ago by

code-llama-70b

#10 opened 9 months ago by

What -ctx and -chunks parameters did you use to make the iMatrix of the Lllama 2 70b?

#9 opened 10 months ago by

Quantize these amazing models

#8 opened 10 months ago by

mixtral-instruct-8x7b for Q2KS as well

#7 opened 10 months ago by

Would love a deepseekcode 2bit quant. I bet others would love it too :)

#6 opened 10 months ago by

[Model request] Saily 100b, Saily 220b

#5 opened 10 months ago by

Could We combine AWQ and Importance Matrix calculation together to further improve perplexity.

#4 opened 10 months ago by

[Model Request] cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser

#3 opened 10 months ago by

Magic Issues with nous-hermes-2-34b-2.16bpw.gguf (Log Attached...)

#2 opened 10 months ago by

Please 3b model rocket 2bit?

#1 opened 10 months ago by