Running in CPU
#9
by
hedrergudene
- opened
As flash-attn
is part of the requirements of this model, is there any chance of having a runtime version of this model in CPU? It would be particularly relevant given its size :)
As
flash-attn
is part of the requirements of this model, is there any chance of having a runtime version of this model in CPU? It would be particularly relevant given its size :)
did you find a way to run it in CPU