Methods to reduce bias: Several methods have been suggested to reduce bias from a dataset. Some techniques require prior knowledge of the biased variables, for instance, gender bias in vision is addressed by masking related features (e.g., faces) [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}, [8]}, [9]}, [10]}. Also, some techniques require access to the test set to re-balance it [11]}. Additionally, various approaches were proposed for visual question answering [12]}, [13]}, [14]}, [15]}. These methods use a classifier trained only on the question modality to regularize bias directly. [12]} suggest to mask a model's softmax prediction with a softmax prediction of a subset classifier trained on the question modality. [13]} use an ensemble method of a full classifier paired with a question subset classifier using products of experts [18]}. Common to all those approaches is the use of a subset classifier to indicate reliance on subset of the data. In contrast, we assess the perceptiveness of the original model based on permutations of a subset of the data.
