How sensitive are translation systems to extra contexts? Mitigating gender bias in Neural Machine Translation models through relevant contexts
Neural Machine Translation systems built on top of Transformer-based
architectures are routinely improving the state-of-the-art in translation
quality according to word-overlap metrics. However, a growing number of studies
also highlight the inherent gender bias that these models incorporate during
training, which reflects poorly in their translations. In this work, we
investigate whether these models can be instructed to fix their bias during
inference using targeted, guided instructions as contexts. By translating
relevant contextual sentences during inference along with the input, we observe
large improvements in reducing the gender bias in translations, across three
popular test suites (WinoMT, BUG, SimpleGen). We further propose a novel metric
to assess several large pretrained models (OPUS-MT, M2M-100) on their
sensitivity towards using contexts during translation to correct their biases.
Our approach requires no fine-tuning, and thus can be used easily in production
systems to de-bias translations from stereotypical gender-occupation bias. We
hope our method, along with our metric, can be used to build better, bias-free
translation systems.