“I’m fully who I am”: Towards Centering Transgender and
Non-Binary Voices to Measure Biases in Open Language
Generation
Anaelia 
"I'm fully who I am": Towards Centering Transgender and Non-Binary Voices to Measure Biases in Open Language Generation
Transgender and non-binary (TGNB) individuals disproportionately experience
discrimination and exclusion from daily life. Given the recent popularity and
adoption of language generation technologies, the potential to further
marginalize this population only grows. Although a multitude of NLP fairness
literature focuses on illuminating and addressing gender biases, assessing
gender harms for TGNB identities requires understanding how such identities
uniquely interact with societal gender norms and how they differ from gender
binary-centric perspectives. Such measurement frameworks inherently require
centering TGNB voices to help guide the alignment between gender-inclusive NLP
and whom they are intended to serve. Towards this goal, we ground our work in
the TGNB community and existing interdisciplinary literature to assess how the
social reality surrounding experienced marginalization of TGNB persons
contributes to and persists within Open Language Generation (OLG). This social
knowledge serves as a guide for evaluating popular large language models (LLMs)
on two key aspects: (1) misgendering and (2) harmful responses to gender
disclosure. To do this, we introduce TANGO, a dataset of template-based
real-world text curated from a TGNB-oriented community. We discover a dominance
of binary gender norms reflected by the models; LLMs least misgendered subjects
in generated text when triggered by prompts whose subjects used binary
pronouns. Meanwhile, misgendering was most prevalent when triggering generation
with singular they and neopronouns. When prompted with gender disclosures, TGNB
disclosure generated the most stigmatizing language and scored most toxic, on
average. Our findings warrant further research on how TGNB harms manifest in
LLMs and serve as a broader case study toward concretely grounding the design
of gender-inclusive AI in community voices and interdisciplinary literature.
Anaelia Ovalle Palash Goyal Jwala Dhamala Zachary Jaggers Kai-Wei Chang Aram Galstyan Richard Zemel Rahul Gupta