license:cc-by-4.0 region:us
---
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
dataset_info:
  features:
  - name: id
    dtype: string
  - name: original_prompt
    dtype: string
  - name: positive_prompt
    dtype: string
  - name: negative_prompt
    dtype: string
  - name: model
    dtype: string
  - name: filepath
    dtype: string
  - name: num_inference_steps
    dtype: int64
  - name: width
    dtype: int64
  - name: height
    dtype: int64
  - name: url
    dtype: string
  - name: image
    dtype: image
  - name: heatmap_labels
    sequence: string
  - name: heatmaps
    sequence:
      sequence:
        sequence: float64
  splits:
  - name: train
    num_bytes: 127788930013
    num_examples: 501000
  download_size: 54902331553
  dataset_size: 127788930013
license: cc-by-4.0
---


# ELSA - Multimedia use case

![daam.gif](https://cdn-uploads.huggingface.co/production/uploads/6380ccd084022715e0d49d4e/a4Sxbr5E69lox_Z9T3gHI.gif)

**ELSA Multimedia is a large collection of Deep Fake images, generated using diffusion models**

### Dataset Summary

This dataset was developed as part of the EU project ELSA. Specifically for the Multimedia use-case.
Official webpage: https://benchmarks.elsa-ai.eu/
This dataset aims to develop effective solutions for detecting and mitigating the spread of deep fake images in multimedia content. Deep fake images, which are highly realistic and deceptive manipulations, pose significant risks to privacy, security, and trust in digital media. This dataset can be used to train robust and accurate models that can identify and flag instances of deep fake images.

### ELSA versions

| Name  | Description | Link |
| ------------- | ------------- | ---------------------| 
| ELSA1M_track1  | Dataset of 1M images generated using diffusion model  | https://huggingface.co/datasets/elsaEU/ELSA1M_track1 |
| ELSA500k_track2  | Dataset of 500k images generated using diffusion model with diffusion attentive attribution maps [1]  | https://huggingface.co/datasets/elsaEU/ELSA500k_track2 |


```python
from daam import WordHeatMap
from datasets import load_dataset
import torch

elsa_data = load_dataset("elsaEU/ELSA500k_track2", split="train", streaming=True)
for sample in elsa_data:
  image = sample.pop("image")
  heatmaps = sample.pop("heatmaps")
  heatmap_labels = sample.pop("heatmap_labels")
  metadata = sample
  for j, (h, l) in enumerate(zip(heatmaps, heatmap_labels)):
    heatmap = WordHeatMap(torch.Tensor(h), word=l)
    heatmap.plot_overlay(image)
    plt.show()
```

Using <a href="https://huggingface.co/docs/datasets/stream">streaming=True</a> lets you work with the dataset without downloading it.

## Dataset Structure

Each parquet file contains nearly 1k images and a JSON file with metadata.

The Metadata for generated images are:

- ID: Laion image ID 
- original_prompt: Laion Prompt
- positive_prompt: positive prompt used for image generation
- negative_prompt: negative prompt used for image generation
- model: model used for the image generation
- nsfw: nsfw tag from Laion
- url_real_image: Url of the real image associated to the same prompt
- filepath: filepath of the fake image
- aspect_ratio: aspect ratio of the generated image
- heatmaps: diffusion attentive attribution maps
- heatmap_labels: words releated to the heatmaps


### Dataset Curators

- Leonardo Labs (rosario.dicarlo.ext@leonardo.com)
- UNIMORE (https://aimagelab.ing.unimore.it/imagelab/)

### References
[1] What the DAAM: Interpreting Stable Diffusion Using Cross Attention, 2023. Tang Raphael  et al.