VQA-CP v2 and VQA-CP v1<cite> [10]</cite> were recently introduced as diagnostic datasets containing different answer distributions for each questiontype between train and test splits. Consequentially, models biased towards the question modality fail on these benchmarks. We use the more challenging VQA-CP v2 dataset extensively in order to show the ability of our approach to reduce the learning of biases coming from the question modality.