Specifically, for deep neural networks and images, there exist adversarial attacks on saliency map based interpretation of predictions, which can hide the model's bias [1]}, [2]}, [3]}.
[4]} showed that a bias of a neural network could be hidden from post-hoc explanation methods by training a modified classifier that has similar performance to the original one, but the importance of the chosen feature is significantly lower.
