best speed/perplexity for mobile devices with int8 acceleration

standard iq4xs imatrix quant from bf16 gguf so it has better perplexity

very good quant for speed/perplexity, embedding is at q4k

standard iq4xs imatrix quant from bf16 gguf so it has better perplexity

best speed/perplexity quant for mobile devices with 8bit acceleration

Good speed reference quant for older CPUs, however not much improvement from f16 embedding

Best q8 conversion down from bf16 with slightly better perplexity than f16 based quants

Standard IQ4XS quantizing down from full bf16 ( not from f16)

great quant if your chip has 8bit acceleration, slightly better than 4k embedding

Rename qwen7bq4xsembedding8output8.gguf to qwen7bv2inst_iq4xs_embedding8_output8.gguf

Rename qwen7bf16.gguf to qwen7bv2instruct_bf16.gguf

very good quant for speed/perplexity, embedding is at q4k

Probably best speed to perplexity ratio of any 7b gguf model so far

standard q5km conversions with 8bit output for reference.

Good conversion from bf16 down instead of from f16

calculated imatrix in 8bit, was jsut as good as f16 imatrix

Rename qwen7bq4xs.gguf to qwen7bq4xsoutput6k.gguf

Rename qwen7bq4xsembedding5bitkoutput8bit.gguf to qwen7bq4xsembedding8output8.gguf

Rename qwen7bq4kembeddingbf16outputbf16.gguf to qwen7bq4kembeddingf16outputf16.gguf

Upload 9 files

Update README.md

Update README.md

initial commit