Quantized with these parameters:
--bits 4
--group_size 128
--desc_act 1
--damp 0.1
--seqlen 16384
--num_samples 512
Quantization Dataset: Erotiquant XL
- Downloads last month
- 40
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.