3bit quantization
#3
by
deleted
- opened
Yes I hope it will and I plan to do that soon
The only reason I've not done it yet is because Falcon40B is so slow atm when quantised, so it's not hugely useful
But I'll do a 3bit anyway and hopefully the slowdown issue might be improved upon soon