152334H

/

miqu-1-70b-sf

like 218

I would also really value an AWQ quantisation here as well.

lonestriker has many exl2 exports, if that helps

I can do GPTQ and GGUF if they are not in progress already. (never done AWQ tbh)

@MaziyarPanahi , I would be very appreciative of the GPTQ version. Or maybe just instructions on how to do it.

@ZanMax I use the official script of AutoGPT which TheBloke also uses it. It just requires lots of GPU vRAM. I will start my script for this model

@ZanMax I made the GPTQ version of this model: MaziyarPanahi/miqu-1-70b-sf-GPTQ
PS: it requires 20GB-22GB of A100 and it takes around 4 hours to finish for GPTQ.