LLAMA.cpp for SnapDragon on Windows ARM64
Hi all. I have LM Studio installed and can run many SLMs on CPU mode, including Llama-v2-7B-Chat and PHi3 and even a 20B model... BUT the model doesn't use the GPU or NPU.
Closest I've come to finding a working llama.cpp file for SnapDragon on windows is https://github.com/ggerganov/llama.cpp/discussions/8336#discussioncomment-10472433
Then I joined Qualcomm tech program and found this document...
https://docs.qualcomm.com/bundle/publicresource/topics/80-62010-1/genai-llama-cpp.html
but there is currently no LLVM ARM version pre-compiled.
I would rather develop my agents and compare SLMs than build these fundamental tools in C++. But it looks like I might have to. If there is a SnapDragon llama.cpp download for windows native or WSL, I'm keen to try it.
We are working on an example app that will demonstrate how to deploy Llama on Windows. Stay tuned!