@inproceedings{
dong2023probing,
title={Probing Explicit and Implicit Gender Bias through {LLM} Conditional Text Generation},
author={Xiangjue Dong and Yibo Wang and Philip Yu and James Caverlee},
booktitle={Socially Responsible Language Modelling Research},
year={2023},
url={https://openreview.net/forum?id=ZDeEYmKYrR}
}
Large Language Models (LLMs) can generate biased and toxic responses. Yet most prior work on LLM gender bias evaluation requires predefined gender-related phrases or gender stereotypes, which are challenging to be comprehensively collected and are limited to explicit bias evaluation. In addition, we believe that instances devoid of gender-related language or explicit stereotypes can still induce gender bias in LLMs. Thus, in this work, we propose a conditional text generation mechanism without the need for predefined gender phrases and stereotypes. This approach employs three types of inputs generated through three distinct strategies to probe LLMs, aiming to show evidence of explicit and implicit gender biases in LLMs. We also utilize explicit and implicit evaluation metrics to evaluate gender bias in LLMs under different strategies. Our experiments demonstrate that an increased model size does not consistently lead to enhanced fairness and all tested LLMs exhibit explicit and/or implicit gender bias, even when explicit gender stereotypes are absent in the inputs.
Probing Explicit and Implicit Gender Bias through LLM Conditional Text Generation
Xiangjue Dong Yibo Wang Philip Yu James Caverlee