CUDA extension not installed (even after manual compile and pip install)

#26
by markemicek - opened

Im trying to get the model running fast. i notice that every time i start up the model, it gives me the error (CUDA extension not installed.).
Ive used pip install auto-gptq.
ive also tried to compile it from source.
im in a python environment (not conda).

Screenshot 2024-06-02 175438.png

It does run, its just slow

Sign up or log in to comment