shuttleai

/

shuttle-3.1-aesthetic

like 68

Follow

ShuttleAI 122

Hello, we have not fully released the model yet, we are working on the fp8 and the gguf version! 😁

fp8 has been added, gguf will be soon

Hello,

Incredible work! Thanks!

How to use the fp8 with Diffusers? You have a working example?

I am also looking for FP8 with Diffusers demo. Not interested in Comfyi.

Hello @edtjulien @fullsoftwares

To run fp8 with diffusers, I usually just do

from optimum.quanto import freeze, quantize, qint8, qfloat8, qint4
quantize(
    pipe.transformer,
    # weights=qfloat8,
    weights=qint8,
    exclude=[
        "*.norm", "*.norm1", "*.norm2", "*.norm2_context",
        "proj_out", "x_embedder", "norm_out", "context_embedder",
    ],
)
freeze(pipe.transformer)
# pipe.enable_model_cpu_offload()

That's work like a charm. Thanks!