size_categories:10K<n<100K license:cc-by-4.0 imagenet OOD detection distribution shift arxiv:2105.01879 arxiv:1907.07174 arxiv:2203.10807 region:us
---
license: cc-by-4.0
dataset_info:
  features:
  - name: date_captured
    dtype: string
  - name: coco_url
    dtype: string
  - name: license_name
    dtype: string
  - name: license_url
    dtype: string
  - name: coco_id
    dtype: string
  - name: image
    dtype: image
  - name: label
    dtype: int64
  - name: flickr_url
    dtype: string
  splits:
  - name: clean
    num_bytes: 333801279.747
    num_examples: 36157
  - name: cocomagenet
    num_bytes: 306403223
    num_examples: 2000
  - name: cocomagenet_mono
    num_bytes: 18956338
    num_examples: 2000
  - name: synthetic_gan
    num_bytes: 242598071.454
    num_examples: 24999
  - name: synthetic_diffusion
    num_bytes: 283705025
    num_examples: 25000
  - name: adversarial_autoattack_resnet
    num_bytes: 40058245
    num_examples: 5000
  - name: adversarial_autoattack_vit
    num_bytes: 35610460
    num_examples: 5000
  - name: adversarial_pgd_resnet
    num_bytes: 65806170
    num_examples: 5000
  - name: adversarial_pgd_vit
    num_bytes: 51803590
    num_examples: 5000
  download_size: 1432934722
  dataset_size: 1378742402.201
pretty_name: BROAD
size_categories:
- 10K<n<100K
tags:
- imagenet
- OOD detection
- distribution shift
---
# Partial dataset used to build BROAD (Benchmarking Resilience Over Anomaly Diversity )

Refer to [this repo ](https://github.com/ServiceNow/broad) to build the complete BROAD dataset.

The partial data included here contains synthetica images from BROAD and encoded unrecognizable images given by adversarial perturbations of imagenet samples. Decoding is implemented in the repo referred above.

## Dataset Description

The BROAD dataset was introduced to benchmark OOD detection methods against a broader variety of distribution shifts in the paper
Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection.

Each split of BROAD is designed to be close (but different) to the [ImageNet](https://www.image-net.org/index.php) distribution.

### Dataset Summary

BROAD is comprised of 16 splits, 9 of which can be downloaded from this page. The remaining 7 can be obtained through external links.
We first describe the splits available from this hub, and then specify the external splits and how to get them. Please refer to Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection for more detailed description of the data and its acquisition.

### Included Splits

- **Clean** is comprised of 36157 images from the original validation set of ILSVRC2012. They are used as in-distribution in BROAD.
- **Adversarial Autoattack Resnet**, **Adversarial Autoattack ViT**, **Adversarial PGD Resnet** and **Adversarial PGD ViT** are splits each comprised of 5,000 adversarial perturbations of clean validation images, using a perturbation budget of 0.05 with the L-infinity norm. These attacks are computed against a trained ResNet-50 and a trained ViT-b/16. PGD uses 40 iterations and for Autoattack, only the attack model achieving the most confident misclassification is kept.
- **Synthetic Gan** and **Synthetic Diffusion** are each comprised of 25,000 synthetic images generated to imitate the ImageNet distribution. For Synthetic Gan, a conditional BigGan architecture was used to generate 25 artificial samples from each ImageNet class. For Synthetic diffusion, we leveraged stable diffusion models to generate 25 artificial samples per class using the prompt "High quality image of a {class_name}".
- **CoComageNet** is a novel split from the [CoCo](https://cocodataset.org/#home) dataset comprised of 2000 images, each featuring multiple distinct classes of ImageNet. Each image of CoComageNet thus features multiple objects, at least two of which have distinct ImageNet labels. More details on the construction of CoComageNet can be found in the paper.
- **CoComageNet-mono** is built similarly to CoComageNet, except each image only has one object with ImageNet label. It is designed as an ablation, to isolate the effect of having instances of multiple labels from other distributional shifts in CoComageNet.

### External Splits

- **iNaturalist** is a split of the original [iNaturalist2017 dataset](https://github.com/visipedia/inat_comp/tree/master/2017) designed for OOD detection with ImageNet as in-distribution. It was introduced in [MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space](https://arxiv.org/pdf/2105.01879.pdf) and can be downloaded [here](http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/iNaturalist.tar.gz).
- **ImageNet-O** was introduced in [Natural Adversarial Examples](https://arxiv.org/pdf/1907.07174.pdf) and is comprised of natural examples that were selected for their high classification confidence by CNNs. It can be downloaded [here](https://people.eecs.berkeley.edu/~hendrycks/imagenet-o.tar).
- **OpenImage-O** is a subset of the OpenImage dataset that was built similarly to ImageNet-O in [ViM: Out-Of-Distribution with Virtual-logit Matching](https://arxiv.org/pdf/2203.10807.pdf). The file list can be accessed [here](https://github.com/haoqiwang/vim/blob/master/datalists/openimage_o.txt).
- **Defocus blur**, **Gaussian noise**, **Snow** and **Brightness** are all existing splits of the [ImageNet-C dataset](https://github.com/hendrycks/robustness). For BROAD, only the highest strength of corruption (5/5) is used.

### LICENSE

This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by/4.0/deed.en_US">Creative Commons Attribution 4.0 Unported License</a>.