Come on try quantizing this
#4
by
supercharge19
- opened
Thanks for the model, can you quantize this as well that can run on CPU (and even leverage GPU layers)?
Thanks for the model, can you quantize this as well that can run on CPU (and even leverage GPU layers)?