bartowski

/

Phi-3-medium-128k-instruct-GGUF

like 55

Same as above: getting this error in LMStudio latest version (0.2.23) trying to load the Q6_KM.gguf

"llama.cpp error: 'done_getting_tensors: wrong number of tensors; expected 245, got 243'"

Apple Silicon Macbook.

Same...

same with Q4_K_M

With latest llama.cpp it works (i'm using Q6), but I'm getting awful results with it - 0,41 t\s with dual tesla P40

How do you update llama.cpp in LM Studio?

You can't, you'll have to wait for LM Studio to get an update, they're aware and working on it

newest llamacpp show the same error

Same with 4_K_S

Bummer

wait newest llama.cpp errors?

can you give me the command you're running and the version you're using? Using llama.cpp b3001 I get it to produce with no error with this format:

./main -m /models/Phi-3-medium-128k-instruct-Q4_K_S.gguf -p "<|system|> You are a Helpful Assistant<|end|><|user|> How are you doing today?<|end|><|assistant|>" -ngl 99 -c 1000

I'm doing well, thank you for asking! How can I assist you today? [end of text]

@mirek190 @thevox if you could update me on where you're attempting to run it that would be perf

for me is working already ... just updater llamacpp ;)

Just got an update for LM Studio. Looks good.
Phi-3-medium-128k-instruct.Q8_0.gguf
I'll be testing it out, but so far it's very promising. Thank you!

LM Studio0.2.24is available! Update now or later.
Your current version is 0.2.23. It's recommended to always use the latest version.
### 🎉 What's New in 0.2.24

• 🤖 Support for new GGUF (llama.cpp) models:

• New Phi-3 model from Microsoft: Phi-3 Medium (14B)
• CohereAI's Aya 23 (8B): an impressive Multilingual LLM

The latest version 0.2.24 resolved my issue - thanks