Question about convert model to ggml
#4
by
khanhj
- opened
Hi Symanto,
Thanks for provide this model.
I'm using this method to convert it to ggml: https://github.com/ggerganov/llama.cpp/blob/master/convert.py
This is the error:
params: n_vocab:250002 n_embd:768 n_mult:1536 n_head:12 n_layer:12
Traceback (most recent call last):
File "/root/github/llama.cpp/convert.py", line 1270, in <module>
main()
File "/root/github/llama.cpp/convert.py", line 1262, in main
output_type = pick_output_type(model, args.outtype)
File "/root/github/llama.cpp/convert.py", line 1075, in pick_output_type
wq_type = model["layers.0.attention.wq.weight"].data_type
KeyError: 'layers.0.attention.wq.weight'
May i know what is the weight number of this model?
Thank you in advance.
Lee
Hi @khanhj ,
thanks for the interest in this model. I took a look at the convert.py script that you are using and it seems to work only for specific model. From what I understand, llama.cpp does not currently support sentence transformer models. From a quick search however, I have seen that this library should work in principle with this model. Have you tried it already?