|
--- |
|
license: mit |
|
tags: |
|
- stable-diffusion |
|
- stable-diffusion-diffusers |
|
inference: false |
|
--- |
|
|
|
# Watermarked (weak) VAE for SDXL |
|
|
|
You can use this watermarked VAE for your existing SDXL image generation pipelines |
|
to make sure generated images are watermarked. |
|
|
|
## Usage |
|
|
|
```py |
|
from diffusers.models import AutoencoderKL |
|
from diffusers import StableDiffusionXLPipeline |
|
|
|
model = "stabilityai/sdxl-turbo" |
|
|
|
vae = AutoencoderKL.from_pretrained("imatag/stable-signature-bzh-sdxl-vae-weak") |
|
|
|
pipe = StableDiffusionXLPipeline.from_pretrained(model, vae=vae) |
|
``` |
|
|
|
For more information, please have a look at [the official demo](https://huggingface.co/spaces/imatag/stable-signature-bzh) |
|
|
|
## Purpose |
|
|
|
This model was finetuned from the original StableDiffusion-XL autoencoder with the additional objective of including a weak invisible watermark, following the procedure of [StableSignature](https://ai.meta.com/blog/stable-signature-watermarking-generative-ai/). This watermark is detectable by [IMATAG](https://www.imatag.com/)'s demo BZH decoder, available via API. |
|
|
|
Compared to other watermarking solutions such as [invisible-watermark](https://github.com/ShieldMnt/invisible-watermark): |
|
- the watermark is quite robust to many unintentional editing transforms such as cropping, resizing, changing contrast or brightness, and JPEG compression |
|
- the watermark has zero overhead in terms of computational power |
|
- the watermark is embedded in the model weights and not as a post-processing of the non-watermarked image |
|
- the probability of detecting the watermark by chance on non-watermarked images may be computed analytically and limited to a specific threshold |
|
|
|
## Detecting the watermark |
|
|
|
For security reasons we cannot release the weights of the detector. Instead, you may check for the presence of a watermark with the [detect_api.py](https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/detect_api.py) script like this: |
|
```shell |
|
python detect_api.py test.png |
|
``` |
|
|
|
### Visual |
|
_Visualization of watermark impact on 512x512 images from the COCO2017 validation dataset prompts with SDXL-turbo._ |
|
|
|
<p align="center"> |
|
<br> |
|
<b> |
|
512x512: original (left), watermarked (middle), difference (right)</b> |
|
</p> |
|
|
|
<p align="center"> |
|
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/weak/0001.png /> |
|
</p> |
|
|
|
<p align="center"> |
|
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/weak/0003.png /> |
|
</p> |
|
|
|
<p align="center"> |
|
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/weak/0004.png /> |
|
</p> |
|
|
|
<p align="center"> |
|
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/weak/0005.png /> |
|
</p> |
|
|
|
<p align="center"> |
|
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/weak/0006.png /> |
|
</p> |
|
|
|
### Robustness |
|
|
|
The watermark is robust to most editorial changes to an image, such as cropping, changing brightness or contrast, resizing or JPEG compression: |
|
|
|
<p align="center"> |
|
<br> |
|
<b> |
|
from left to right: original, watermarked, difference, 'crop 50% + brigthen 50% + jpeg 80%' attack, 'downscale 2x + crop 50% + brigthen 50% + jpeg 50%' attack</b> |
|
</p> |
|
|
|
<p align="center"> |
|
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/weak/0000.png /> |
|
</p> |
|
|
|
The Receiver Operating Characteristic curve shows its performance under no modification and when modified by the two scenarios illustrated above: |
|
<p align="center"> |
|
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/weak/roc.png /> |
|
</p> |
|
|
|
More details available in our [announcement](https://www.imatag.com/blog/unlocking-the-future-of-content-authentication-imatags-breakthrough-in-ai-generated-image-watermarking) and our lab's [blog post](https://imatag-lab.medium.com/stable-signature-meets-bzh-53ad0ba13691). |
|
For watermarked models with a different key, support for payload, other perceptual compromises, robustness to other attacks, or faster detection, please [contact IMATAG](https://pages.imatag.com/contact-us-imatag). |
|
|
