legraphista

/

Yuan2-M32-hf-IMat-GGUF

like 3

Hi!

Unfortunately this currently requires a custom build of llama.cpp to work (and any other software build on top of llama.cpp) which is found here https://github.com/chong000/3rd_party/tree/main

There is currently an issue on the main fork of llama.cpp tracking the support for Yuan2 MoE models https://github.com/ggerganov/llama.cpp/issues/7632

In the meantime, as a workaround, you can try to build the working fork, and replace any llama.cpp executables from your software to make it work. As a word of caution, the working fork is quite out of date.

Hi thanks for the reply i will wait for llama to support it
In the mean time i will check other models