Someone please make GGUF versions (-)

#5
by huggingfacess - opened

Someone please make GGUF versions

The correct way.

python3 llama.cpp/convert-hf-to-gguf.py GemMoE-Beta-1/ --outfile ggml-model-f16.gguf --outtype f16

don't use this.
python3 llama.cpp/convert.py

I will make one - but I am currently updating the model to version 1.1. There was a decently large bug in this one that caused vram use to go out of control.

Sign up or log in to comment