File size: 3,537 Bytes
241454a 2716b73 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 |
---
license: mit
tags:
- stable-diffusion
- stable-diffusion-diffusers
inference: false
---
# Watermarked (strong) VAE for SDXL
You can use this watermarked VAE for your existing SDXL image generation pipelines
to make sure generated images are watermarked.
## Usage
```py
from diffusers.models import AutoencoderKL
from diffusers import StableDiffusionXLPipeline
model = "stabilityai/sdxl-turbo"
vae = AutoencoderKL.from_pretrained("imatag/stable-signature-bzh-sdxl-vae-strong")
pipe = StableDiffusionXLPipeline.from_pretrained(model, vae=vae)
```
For more information, please have a look at [the official demo](https://huggingface.co/spaces/imatag/stable-signature-bzh)
## Decoder Finetuning
This model was finetuned from the original StableDiffusion-XL autoencoder with the additional objective of including a strong invisible watermark, following the procedure of [StableSignature](https://ai.meta.com/blog/stable-signature-watermarking-generative-ai/). This watermark is detectable by [IMATAG](https://www.imatag.com/)'s demo BZH decoder, available via API.
## Detecting the watermark
For security reasons we cannot release the weights of the detector. Instead, you may check for the presence of a watermark with the [detect_api.py](https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/detect_api.py) script like this:
```shell
python detect_api.py test.png
```
### Visual
_Visualization of watermark impact on 512x512 images from the COCO2017 validation dataset prompts with SDXL-turbo._
<p align="center">
<br>
<b>
512x512: original (left), watermarked (middle), difference (right)</b>
</p>
<p align="center">
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/strong/0001.png />
</p>
<p align="center">
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/strong/0003.png />
</p>
<p align="center">
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/strong/0004.png />
</p>
<p align="center">
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/strong/0005.png />
</p>
<p align="center">
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/strong/0006.png />
</p>
### Robustness
The watermark is robust to most editorial changes to an image, such as cropping, changing brightness or contrast, resizing or JPEG compression:
<p align="center">
<br>
<b>
from left to right: original, watermarked, difference, 'crop 50% + brigthen 50% + jpeg 80%' attack, 'downscale 2x + crop 50% + brigthen 50% + jpeg 50%' attack</b>
</p>
<p align="center">
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/strong/0000.png />
</p>
The Receiver Operating Characteristic curve shows its performance under no modification and when modified by the two scenarios illustrated above:
<p align="center">
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/strong/roc.png />
</p>
More details available in our [announcement](https://www.imatag.com/blog/unlocking-the-future-of-content-authentication-imatags-breakthrough-in-ai-generated-image-watermarking) and our lab's [blog post](https://imatag-lab.medium.com/stable-signature-meets-bzh-53ad0ba13691).
For watermarked models with a different key, support for payload, other perceptual compromises, robustness to other attacks, or faster detection, please [contact IMATAG](https://pages.imatag.com/contact-us-imatag).
|