With the adoption of machine learning models in real-world applications, there is a growing concern about Spurious Patterns (SPs) – when models rely on patterns that do not align with domain knowledge and do not generalize [1]}, [2]}, [3]}, [4]}, [5]}, [6]}.
For example, a model trained to detect tennis rackets on the COCO dataset [7]} learns to rely on the presence of a person, which leads to systemic errors: it is significantly less accurate at detecting tennis rackets for images without people than with people (41.2% vs 86.6%) and only ever has false positives on images with people.
Relying on this SP works well on COCO, where the vast majority of images with tennis rackets also have people, but would not be as effective for other distributions.
In addition to being detrimental to performance, relying on SPs may also lead to serious concerns when they relate to protected attributes such as race or gender [8]}.
