ggml-gpt-fourchannel-q4_0.bin does not work on llama.cpp
has checked the file's sha256 to confirm the file is completely correct.
main -m ./models/ggml-gpt-fourchannel-q4_0.bin --n_parts 1 -t 12 -n -1 -c 2048 --temp 0.8 --top_p 0.8 --top_k 160 --repeat_last_n 2048 --repeat_penalty 1.176470588 --ignore-eos -f "../storyboard/answer_question.txt"
main: seed = 1682772219
llama.cpp: loading model from ./models/ggml-gpt-fourchannel-q4_0.bin
error loading model: unexpectedly reached end of file
llama_init_from_file: failed to load model
main: error: failed to load model './models/ggml-gpt-fourchannel-q4_0.bin'
time used: 6.1111602783203125
Does llama CPP support GPT-J models? It is not listed on their github. On koboldCPP it comes up as legacy_gptj_model_load and it works.
-model-q4_1.bin' - please wait ...
legacy_gptj_model_load: n_vocab = 50400
legacy_gptj_model_load: n_ctx = 2048
legacy_gptj_model_load: n_embd = 4096
legacy_gptj_model_load: n_head = 16
legacy_gptj_model_load: n_layer = 28
legacy_gptj_model_load: n_rot = 64
legacy_gptj_model_load: f16 = 3
legacy_gptj_model_load: ggml ctx size = 5226.67 MB
legacy_gptj_model_load: memory_size = 896.00 MB, n_mem = 57344
legacy_gptj_model_load: ................................... done
legacy_gptj_model_load: model size = 4330.60 MB / num tensors = 285
Load Model OK: True
Great! Thanks for the tips. I have tested it in koboldCPP and it works. This leads me to understand why some of the models didn't work on llama.cpp. Thank you!!