Thank you for this model.

#6
by Alastar-Smith - opened

Just want to say thank you for this model, right now it is one of the best models for CPU only PCs.

-It works really good at 10 t/s on 6 core CPU using Q8.
-It works good on 64k context.
-It can code really fine, about Codestral 22b level.
-It can speak okay on different languages.
-It works good as general chat model.

Keep-up the good work!

agree, happend to be best for twinny vscode copilot with cpu inference, latency dropped to a level, i can manage.

Please make a DeepSeek-Coder-V2 0.5B instruct version! I'm sure that to generate python code DeepSeek-Coder will beat Qwen2-0.5B-instruct.

Sign up or log in to comment