TITLE: SI-Score: An image dataset for fine-grained analysis of robustness to object location, rotation and size 

ABSTRACT: Before deploying machine learning models it is critical to assess their
robustness. In the context of deep neural networks for image understanding,
changing the object location, rotation and size may affect the predictions in
non-trivial ways. In this work we perform a fine-grained analysis of robustness
with respect to these factors of variation using SI-Score, a synthetic dataset.
In particular, we investigate ResNets, Vision Transformers and CLIP, and
identify interesting qualitative differences between these.
Before deploying machine learning models it is critical to assess their
robustness. In the context of deep neural networks for image understanding,
changing the object location, rotation and size may affect the predictions in
non-trivial ways. In this work we perform a fine-grained analysis of robustness
with respect to these factors of variation using SI-Score, a synthetic dataset.
In particular, we investigate ResNets, Vision Transformers and CLIP, and
identify interesting qualitative differences between these.
SI-Score: An image dataset for fine-grained analysis of robustness to object location, rotation and size
http://arxiv.org/abs/2104.04191v1