Modl showing up as Llama instead of phi3 in LMstudio
Maybe its a user error on my part. but I wanted to just see if thats how it should be structured. Since when using the system prompt it does not like Phi3 or Llama.
@LZHgrla Yes there is a way to configure chat templates in LM Studio. 😁
Additionally, I'd like to know if this conversion will affect the deployment in LM Studio. Is there a way to manually set chat template?
Also in LM Studio due to the naming of the gguf files the model shows up confusingly in the model selection dropdown
I usually name mine something like "llava-phi-3-mini-Q4_K_M.gguf"
Additionally, I'd like to know if this conversion will affect the deployment in LM Studio. Is there a way to manually set chat template?
Also in LM Studio due to the naming of the gguf files the model shows up confusingly in the model selection dropdown
I usually name mine something like "llava-phi-3-mini-Q4_K_M.gguf"
@saishf
Hi! Thanks for your advice.
I have modified the file names, and can you help me check if it's suitable?
https://huggingface.co/xtuner/llava-phi-3-mini-gguf/tree/main
@saishf
Hi! Thanks for your advice.I have modified the file names, and can you help me check if it's suitable?
https://huggingface.co/xtuner/llava-phi-3-mini-gguf/tree/main
Looks good!
Came to say thanks. Had that issue, too.
I still have to load the model every time I want it to analyze an new image. If not it will talk about fantastic abstract art and pixels, and weirdly about wine bottles a lot of times (there are no bottles in my images). Works after I reload the model. Is that expected behavior? Or is it part of the config and I can correct it somehow?
I've not verified issues about it yet but there are "quirks" inside of the llama.cpp engine which look for strings in the model name, one of those strings would be "phi3".
That changes pretokenization to be phi3 compatible.
I'd assume using llama as model name will cause tokenization errors (handling of newlines, stripping before special tokens, etc)