Is this supposed to be usable with llama.cpp?

by spanielrassler - opened Apr 15, 2023

Apr 15, 2023

Hi,

I'm wondering if I'm on the wrong track by trying to use this with llama.cpp. I get the following error when trying:
error loading model: unexpectedly reached end of file
llama_init_from_file: failed to load model
main: error: failed to load model

Any suggestions or is this model just not compatible? If not, do you know of a Pyg one that is? Thanks!

spanielrassler changed discussion status to closed Apr 15, 2023

alpindale

Owner Apr 16, 2023

Since you've closed the issue, I imagine you've figured it out, but anyway.

No, you can only use this with my fork of ggml here or (preferably, as it's more advanced) with koboldcpp.

alpindale changed discussion status to open Apr 16, 2023

aberrio

Apr 30, 2023

•

edited Apr 30, 2023

I was wondering why do the weights depend on koboldcpp? How does the conversion differ?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment