model.safetensors.index.json refers to non-existent files?
#6
by
dmanary-pronavigator
- opened
I tried loading this using Transformers and it gave me the error:
OSError: Could not locate model-00001-of-00019.safetensors inside turboderp/Mixtral-8x7B-instruct-exl2
Which makes sense, because there are not that many files. The model.safetensors.index.json references these files.
Is there something I can do or does the author need to change something?
It's an EXL2 model. It won't load directly in Transformers (a Transformers-compatible plugin might be coming, but not just yet.) It will work in ExLlamaV2, TabbyAPI, AphroditeEngine and some other frameworks.