language:en license:cc-by-4.0 arxiv:2208.08241 region:us
---
license: cc-by-4.0
language:
- en
pretty_name: Socio Moral Image Rationales
---

# Socio-Moral Image Rationales

This is a collection of machine-generated and human-labeled explanations for immorality in images. 
The images are source from the [Socio-Moral Image Database](https://huggingface.co/datasets/AIML-TUDA/smid) (SMID) and limited to the ones displaying immoral content (SMID moral mean <= 2.0). 
Sampled explanations were generated by vision-language model using the ILLUME paradigm presented in [ILLUME: Rationalizing Vision-Language Models through Human Interactions](https://arxiv.org/abs/2208.08241).
Explanations are rated by human annotators from 1-4 with 1 being the best score. Scores are given according to the following categories:
 1. excellent
 2. sufficient/satisfactory
 3. weak (but right direction)
 4. poor/unrelated

The sample are pre-split into a training and test split. Files contain the following fields:
 - **smid_image_id**: Image identifier from the SMID dataset
 - **smid_moral_mean**: Moral mean score from SMID. Ranging from 1-5 with lower being more immoral. We only include images with moral_mean <=2.0
 - **rationale**: Generated rationale for immmorality of the image. 
 - **human_rating**: Human rating of the rational in accordance with the scoring system above.
 - **fitting_rationale**: Whether a rational is a fitting one, i.e. score <= 2

## Citation
```bibtex
@inproceedings{brack2023illume,
      title={ILLUME: Rationalizing Vision-Language Models through Human Interactions}, 
      author={Manuel Brack and Patrick Schramowski and Björn Deiseroth and Kristian Kersting},
      year={2023},
      booktitle={Proceedings of the 40th International Conference on Machine Learning (ICML)}
}
```