i8mm imatrix?
#1
by
EloyOn
- opened
I've been using your models since the first Nymeria. Great models.
Are you considering on releasing i8mm quants generated with imatrix for Ellaria and Rhaenys? They are much faster on smartphones with Snapdragon CPU than regular k quants.
I'm using a i8mm for Ellaria right now, but there is a huge quality downgrade compared to your q4km iGGUF. i8mm imatrix would reduce that difference greatly.
I just noticed that Bartowski is releasing i8mm quants now O_O.
I have my hands full with the current project(Nemo-12B) and dataset creation. It may take several days before I can make the quants. You could approach bartowski or mradermacher in this matter.
Alright, no problem. Thank you for your hard work.
Waiting for that 12B to try it, then.