vllm Framework Inference Error

#2
by czqqq - opened

The error message you encountered when using the vllm framework is: "AssertionError: fp8e4nv data type is not supported on CUDA arch < 89".

Sign up or log in to comment