llama_model_load: error loading model: invalid split file
#1
by
Stefanvarunix
- opened
Hi,
I tried to merge the files with
cat dbrx-16x12b-instruct-q6_k-* > dbrx-16x12b-instruct-q6_k.gguf
When running
./server -m ./dbrx-16x12b-instruct-q6_k.gguf
I get the error message
"llama_model_load: error loading model: invalid split file:"
My mistake?
Llama.cpp sharded model support is built-in now, do not concat files in a binary way. You can use gguf-split to merge or simply pass the first split to llama_model_loader.
phymbert
changed discussion status to
closed