nyanko7 commited on
Commit
e09b8a8
1 Parent(s): dc5ed5b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +37 -3
README.md CHANGED
@@ -1,3 +1,37 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ ---
4
+
5
+ # Smoothed Energy Guidance for SDXL
6
+
7
+ https://arxiv.org/abs/2408.00760 | https://colab.research.google.com/github/SusungHong/SEG-SDXL/blob/master/sdxl_seg.ipynb
8
+
9
+ Identical to https://github.com/SusungHong/SEG-SDXL/blob/8d3b2007a5f0660f9dba110a5e83556395f7535f/pipeline_seg.py
10
+
11
+ Implementation of [Smoothed Energy Guidance: Guiding Diffusion Models with Reduced Energy Curvature of Attention](https://arxiv.org/abs/2408.00760) by [Susung Hong](https://susunghong.github.io).
12
+
13
+ <p align="center">
14
+ <img src="teaser.jpg" width="90%">
15
+ </p>
16
+
17
+ ## 🏔️ What is Smoothed Energy Guidance? How does it work?
18
+
19
+ **Smoothed Energy Guidance (SEG)** is a training- and condition-free approach that leverages the energy-based perspective of the self-attention mechanism to improve image generation.
20
+
21
+ **Key points:**
22
+ - Does not rely on the guidance scale parameter that causes side effects when the value becomes large
23
+ - Allows continuous control of the original and maximally attenuated curvature of the energy landscape behind self-attention
24
+ - Introduces a query blurring method, equivalent to blurring the entire attention weights without significant computational cost
25
+
26
+ Please check **[our paper](https://arxiv.org/abs/2408.00760)** for details.
27
+
28
+ ## 🔍 Comparison with other works
29
+
30
+ SEG does not severely suffer from side effects such as making the overall image grayish or significantly changing the original structure, while improving generation quality even without prompts.
31
+
32
+ Unconditional generation without prompts
33
+ <img src="figures/seg_comparison.jpg" width="90%">
34
+
35
+ ControlNet generation without prompts
36
+ <img src="figures/seg_controlnet_comparison.jpg" width="90%">
37
+