GPU requirement

#1
by peterhuang24 - opened

Hi,
Does GPU required for using this model? Thanks!

Owner

Hi,
Does GPU required for using this model? Thanks!

Hi @peterhuang24 , you can run the model on CPU only (disable flash-attention) but I recommand using a GPU for faster inference.

Sign up or log in to comment