DamonDemon commited on
Commit
a793f27
1 Parent(s): ae59f2a
Files changed (1) hide show
  1. src/about.py +32 -21
src/about.py CHANGED
@@ -21,41 +21,52 @@ NUM_FEWSHOT = 0 # Change with your few shot
21
 
22
 
23
  # Your leaderboard name
24
- TITLE = """<h1 align="center" id="space-title">UnlearnDiffAtk Benchmark</h1>"""
25
 
26
  # subtitle
27
- SUB_TITLE = """<h2 align="center" id="space-title">Effective and efficient adversarial prompt generation approach for diffusion models</h2>"""
28
 
29
  # What does your leaderboard evaluate?
30
  INTRODUCTION_TEXT = """
31
- This benchmark evaluates the robustness of safety-driven unlearned diffusion models (DMs)
32
- (i.e., DMs after unlearning undesirable concepts, styles, or objects) across a variety of tasks. For more details, please visit the [project](https://www.optml-group.com/posts/mu_attack),
33
- check the [code](https://github.com/OPTML-Group/Diffusion-MU-Attack), and read the [paper](https://arxiv.org/abs/2310.11868).\\
 
34
  Demo of our offensive method: [UnlearnDiffAtk](https://huggingface.co/spaces/Intel/UnlearnDiffAtk)\\
35
  Demo of our defensive method: [AdvUnlearn](https://huggingface.co/spaces/Intel/AdvUnlearn)
36
  """
37
 
 
 
 
 
 
 
 
 
 
 
 
 
 
38
  # Which evaluations are you running? how can people reproduce what you have?
39
  LLM_BENCHMARKS_TEXT = f"""
40
- For more details of Unlearning Methods used in this benchmarks:\\
41
- (1) [Erased Stable Diffusion (ESD)](https://github.com/rohitgandikota/erasing);\\
42
- (2) [Forget-Me-Not (FMN)](https://github.com/SHI-Labs/Forget-Me-Not);\\
43
- (3) [Ablating Concepts (AC)](https://github.com/nupurkmr9/concept-ablation);\\
44
- (4) [Unified Concept Editing (UCE)](https://github.com/rohitgandikota/unified-concept-editing);\\
45
- (5) [concept-SemiPermeable Membrane (SPM)](https://github.com/Con6924/SPM); \\
46
- (6) [Saliency Unlearning (SalUn)](https://github.com/OPTML-Group/Unlearn-Saliency); \\
47
- (7) [EraseDiff (ED)](https://github.com/JingWu321/EraseDiff); \\
48
- (8) [ScissorHands (SH)](https://github.com/JingWu321/Scissorhands).
 
49
 
 
 
50
  """
51
 
52
- EVALUATION_QUEUE_TEXT = """
53
- Evaluation Metrics: \\
54
- (1) Pre-attack success rate (pre-ASR), lower is better; \\
55
- (2) Post-attack success rate (post-ASR), lower is better; \\
56
- (3) Fréchet inception distance(FID) of images generated by Unlearned Methods, lower is better; \\
57
- (3) CLIP (Contrastive Language-Image Pretraining) Score is to measure contextual alignment with prompt descriptions, higher is better.
58
- """
59
 
60
  CITATION_BUTTON_LABEL = "Copy the following snippet to cite these results"
61
  CITATION_BUTTON_TEXT = r"""
 
21
 
22
 
23
  # Your leaderboard name
24
+ TITLE = """<h1 align="center" id="space-title">UnlearnDiffAtk: Unlearned Diffusion Model Benchmark</h1>"""
25
 
26
  # subtitle
27
+ SUB_TITLE = """<h2 align="center" id="space-title">Effective and efficient adversarial prompt generation approach for unlearned diffusion model evaluations.</h2>"""
28
 
29
  # What does your leaderboard evaluate?
30
  INTRODUCTION_TEXT = """
31
+ This benchmark evaluates the <strong>robustness and utility retaining</strong> of safety-driven unlearned diffusion models (DMs) across a variety of tasks. For more details, please visit the [project](https://www.optml-group.com/posts/mu_attack).
32
+ - The <strong>robustness</strong> of unlearned DM is evaluated through our proposed adversarial prompt attack, [UnlearnDiffAtk](https://github.com/OPTML-Group/Diffusion-MU-Attack), which has been accepted to ECCV 2024.
33
+ - The <strong>utility retaining</strong> of unlearned DM is evaluated through FID and CLIP score on the generated images using [10K randomly sampled COCO caption prompts](https://github.com/OPTML-Group/Diffusion-MU-Attack/blob/main/prompts/coco_10k.csv).
34
+
35
  Demo of our offensive method: [UnlearnDiffAtk](https://huggingface.co/spaces/Intel/UnlearnDiffAtk)\\
36
  Demo of our defensive method: [AdvUnlearn](https://huggingface.co/spaces/Intel/AdvUnlearn)
37
  """
38
 
39
+ EVALUATION_QUEUE_TEXT = """
40
+ <strong>\[Evaluation Metrics\]</strong>:
41
+ - Pre-Attack Success Rate (<strong>Pre-ASR</strong>): lower is better;
42
+ - Post-attack success rate (<strong>Post-ASR</strong>): lower is better;
43
+ - Fréchet inception distance(<strong>FID</strong>): evaluate distributional quality of image generations, lower is better;
44
+ - <strong>CLIP Score</strong>: measure contextual alignment with prompt descriptions, higher is better.
45
+
46
+ <strong>\[DM Unlearning Tasks\]</strong>:
47
+ - NSFW: Nudity
48
+ - Style: Van Gogh
49
+ - Objects: Church, Tench, Parachute, Garbage Truck
50
+ """
51
+
52
  # Which evaluations are you running? how can people reproduce what you have?
53
  LLM_BENCHMARKS_TEXT = f"""
54
+ For more details of Unlearning Methods used in this benchmarks:
55
+ - [Adversarial Unlearning (AdvUnlearn)](https://github.com/OPTML-Group/AdvUnlearn);
56
+ - [Erased Stable Diffusion (ESD)](https://github.com/rohitgandikota/erasing);
57
+ - [Forget-Me-Not (FMN)](https://github.com/SHI-Labs/Forget-Me-Not);
58
+ - [Ablating Concepts (AC)](https://github.com/nupurkmr9/concept-ablation);
59
+ - [Unified Concept Editing (UCE)](https://github.com/rohitgandikota/unified-concept-editing);
60
+ - [concept-SemiPermeable Membrane (SPM)](https://github.com/Con6924/SPM);
61
+ - [Saliency Unlearning (SalUn)](https://github.com/OPTML-Group/Unlearn-Saliency);
62
+ - [EraseDiff (ED)](https://github.com/JingWu321/EraseDiff);
63
+ - [ScissorHands (SH)](https://github.com/JingWu321/Scissorhands).
64
 
65
+ <strong>We will evaluate your model on UnlearnDiffAtk Benchmark!</strong> \\
66
+ Open a [github issue](https://github.com/OPTML-Group/Diffusion-MU-Attack/issues) or email us at [email protected]!
67
  """
68
 
69
+
 
 
 
 
 
 
70
 
71
  CITATION_BUTTON_LABEL = "Copy the following snippet to cite these results"
72
  CITATION_BUTTON_TEXT = r"""