Running in CPU

#9
by hedrergudene - opened

As flash-attn is part of the requirements of this model, is there any chance of having a runtime version of this model in CPU? It would be particularly relevant given its size :)

As flash-attn is part of the requirements of this model, is there any chance of having a runtime version of this model in CPU? It would be particularly relevant given its size :)

did you find a way to run it in CPU

Sign up or log in to comment