task_categories:image-classification size_categories:10K<n<100K language:en license:other occlusion arxiv:2306.17848 region:us
SMD is an occluded ImageNet-1K validation set, created to be an additional way to evaluate the impact of occlusion on model performance. This experiment used a variety of occluder objects that are not in the ImageNet-1K label space and are unambiguous in relationship to objects that reside in the label space.
@misc{lee2023hardwiring,
      title={Hardwiring ViT Patch Selectivity into CNNs using Patch Mixing}, 
      author={Ariel N. Lee and Sarah Adel Bargal and Janavi Kasera and Stan Sclaroff and Kate Saenko and Nataniel Ruiz},
      year={2023},
      eprint={2306.17848},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}
---
license: other
task_categories:
- image-classification
language:
- en
tags:
- occlusion
size_categories:
- 10K<n<100K
---

# Superimposed Masked Dataset (SMD)
SMD is an occluded version of the ImageNet-1K validation set, created to serve as an additional way to evaluate the impact of occlusion on model performance. Occluder objects were segmented using Meta's Segment Anything and are not in the ImageNet-1K label space. They were chosen to be unambiguous in relationship to objects that reside in the label space. Additional details about the dataset, including code to generate your own version of SMD, actual occlusion percentage of each image in the dataset, as well as occluder object segmentation masks, will be released shortly.

![SMD_examples](./smd.jpeg)

The occluders shown above from left to right, starting from the top row: <strong>Grogu (baby yoda), bacteria, bacteriophage, airpods, origami heart, drone, diamonds (stones, not setting) and coronavirus</strong>. Occluder object images were obtained through Unsplash.

SMD was created for testing model robustness to occlusion in [Hardwiring ViT Patch Selectivity into CNNs using Patch Mixing](https://arielnlee.github.io/PatchMixing/).

## Citations

```bibtex
@misc{lee2023hardwiring,
      title={Hardwiring ViT Patch Selectivity into CNNs using Patch Mixing}, 
      author={Ariel N. Lee and Sarah Adel Bargal and Janavi Kasera and Stan Sclaroff and Kate Saenko and Nataniel Ruiz},
      year={2023},
      eprint={2306.17848},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}
```
```bibtex
@article{imagenet15russakovsky,
    Author = {Olga Russakovsky and Jia Deng and Hao Su and Jonathan Krause and Sanjeev Satheesh and Sean Ma and Zhiheng Huang and Andrej Karpathy and Aditya Khosla and Michael Bernstein and Alexander C. Berg and Li Fei-Fei},
    Title = { {ImageNet Large Scale Visual Recognition Challenge} },
    Year = {2015},
    journal   = {International Journal of Computer Vision (IJCV)},
    doi = {10.1007/s11263-015-0816-y},
    volume={115},
    number={3},
    pages={211-252}
}
```