src/display/about.py

# from src.display.utils import ModelType

TITLE = """
<h1 id="space-title">UnlearnCanvas: A Stylized Image Dataset to Benchmark <br> Machine Unlearning for Diffusion Models</h1>"""

INTRODUCTION_TEXT = """

paper: https://arxiv.org/abs/2402.11846

Code: https://github.com/OPTML-Group/UnlearnCanvas

The rapid advancement of diffusion models (DMs) has not only transformed various real- world industries but has also introduced negative societal concerns, including the generation of harmful content, copyright disputes, and the rise of stereotypes and biases. <strong>To mitigate these issues, machine unlearning (MU) has emerged as a potential solution, demonstrating its ability to remove undesired generative capabilities of DMs in various applications.</strong> However, by examining existing MU evaluation methods, we uncover several key challenges that can result in incomplete, inaccurate, or biased evaluations for MU in DMs. 

To address them, we enhance the evaluation metrics for MU, including the introduction of an often-overlooked retainability measurement for DMs post-unlearning. Additionally, we introduce <strong>UnlearnCanvas, a comprehensive high-resolution stylized image dataset</strong> that facilitates us to evaluate the unlearning of artistic painting styles in conjunction with associated image objects. 

We show that this dataset plays a pivotal role in establishing <strong>a standardized and automated evaluation framework for MU techniques on DMs</strong>, featuring <strong>7 quantitative metrics</strong> to address various aspects of unlearning effectiveness. Through extensive experiments, we benchmark <strong>5 state-of- the-art MU methods</strong>, revealing novel insights into their pros and cons, and the underlying unlearning mechanisms. Furthermore, we demonstrate the potential of UnlearnCanvas to benchmark other generative modeling tasks, such as style transfer.

\[Other Related Benchmarks\]
- [<strong>UnlearnDiff Benchmark</strong>](https://github.com/OPTML-Group/Diffusion-MU-Attack): an evaluation benchmark built upon adversarial attacks (also referred to as adversarial prompts), in order to discern the <strong>trustworthiness of these safety-driven unlearned DMs</strong>. 
"""

LLM_BENCHMARKS_TEXT = f"""
# Context

## Evaluated MU Methods

We evaluate 5 most recently proposed MU methods for DMs:

- <a href="https://erasing.baulab.info" target="_blank">  ESD </a>
- <a href="https://arxiv.org/abs/2303.17591" target="_blank">  FMN </a>
- <a href="https://unified.baulab.info" target="_blank">  UCE </a>
- <a href="https://arxiv.org/abs/2303.13516" target="_blank">  CA </a>
- <a href="https://arxiv.org/abs/2310.12508" target="_blank">  Salun </a>


## Metrics
- Unlearning accuracy (<strong>UA</strong>): It represents the proportion of images generated by the unlearned DM using the unlearning target-related prompt, which are not correctly classified into the corresponding class for the case.
- In-domain retain accuracy (<strong>IRA</strong>): It is given by the classification accuracy of images generated by the unlearned DM using innocent prompts (not relevant to the unlearning target) in the same domain.
- Cross-domain retain accuracy (<strong>CRA</strong>): Similar to IRA, it is given by the classification accuracy of images generated by the unlearned DM using innocent prompts but in different domains.
- <strong>FID</strong> score: evaluate the distribution-wise generation quality of the unlearned DM.
- <strong>run-time</strong>
- <strong>storage</strong> space requirements
- <strong>memory</strong> costs



## Impact Statement
This work helps improve the assessment and further promotes the advancement of MU (machine unlearning) methods for DMs (diffusion models), which are known to be effective in relieving or mitigating the various negative societal influences brought by the prevalent usage of DMs, which include but are not limited to the following aspects.
- <strong>Avoiding Copyright Issues:</strong> There is an urgent need for the generative model providers to scrub the influence of certain data on an already-trained model. 
- <strong>Mitigating biases and stereotypes:</strong>  Generative AI systems are known to have tendencies towards bias, stereotypes, and reductionism, when it comes to gender, race and national identities


## Other Related Benchmarks
- [<strong>UnlearnDiff Benchmark</strong>](https://github.com/OPTML-Group/Diffusion-MU-Attack): an evaluation benchmark built upon adversarial attacks (also referred to as adversarial prompts), in order to discern the <strong>trustworthiness of these safety-driven unlearned DMs</strong>.


## Contact
Please feel free to contact Yihua <[email protected]> and Yimeng <[email protected]> if you have any questions.

"""

FAQ_TEXT = """
"""

EVALUATION_QUEUE_TEXT = """
# Evaluation Queue for the UnlearnCanvas Benchmark.

Currently, you have the option to share your checkpoint with us for evaluation, and we will subsequently update the benchmark results. 

In the future, models submitted here will undergo automatic evaluation on our cluster.


"""

CITATION_BUTTON_LABEL = "Copy the following snippet to cite these results"

CITATION_BUTTON_TEXT = r"""
@article{zhang2024unlearncanvas,
  title={UnlearnCanvas: A Stylized Image Dataset to Benchmark Machine Unlearning for Diffusion Models},
  author={Zhang, Yihua and Zhang, Yimeng and Yao, Yuguang and Jia, Jinghan and Liu, Jiancheng and Liu, Xiaoming and Liu, Sijia},
  journal={arXiv preprint arXiv:2402.11846},
  year={2024}
}
"""