how to run on ubuntu without a gpu?
I have the cuda toolkit installed but it's failing as followsbuilding ggml-cuda with nvcc -arch=native... nvcc fatal : Value 'native' is not defined for option 'gpu-architecture' /usr/bin/nvcc: returned nonzero exit status building nvidia compute capability detector... cudaGetDeviceCount() failed: no CUDA-capable device is detected
I am simply booting the binary as suggested here
https://simonwillison.net/2023/Nov/29/llamafile/
Thanks in advance
Jack
If CUDA says your video card doesn't support CUDA then there's really not a whole lot we can do on our end. You probably need to buy a different graphics card.
Thanks.
I just now have an update on this situation.
I booted the Llava server on the same box, and it runs just fine.
That leaves me curious: why does the Mistral server fail but the Llava server does not fail?
Chances are you downloaded mistral vs. llava at different times, and you ended up with different release versions. The files on this HF have been updated about five times over the last few days as new bug fix releases have come out. If you've got the bandwidth to spare, I'd say just try redownloading it.
Think I found the cause. Follow https://github.com/Mozilla-Ocho/llamafile/issues/50 for updates.