Language representations are known to carry certain associations ( e.g. , gendered connotations ) which may lead to invalid and harmful predictions in downstream tasks . While existing methods are effective at mitigating such unwanted associations by linear projection , we argue that they are too aggressive : not only do they remove such associations , they also erase information that should be retained . To address this issue , we propose OS-CAR ( Orthogonal Subspace Correction and Rectification ) , a balanced approach of mitigation that focuses on disentangling associations between concepts that are deemed problematic , instead of removing concepts wholesale . We develop new measurements for evaluating information retention relevant to the debiasing goal . Our experiments on genderoccupation associations show that OSCAR is a well-balanced approach that ensures that semantic information is retained in the embeddings and unwanted associations are also effectively mitigated . Keeping in mind that removing bias and retaining information have to be done in synergy , we present how to obtain aggregated measurements for these two components . We will first describe the de- Word embeddings are used extensively across natural language processing ( NLP ) and succinctly capture not only the syntactic and semantic structure of language , but also word meaning in context . As such , word embeddings are essential building blocks for today 's state-of-the-art in NLP . But they are also known to capture a significant amount of stereotypical associations ( e.g. , Bolukbasi et al . , 2016 ; Zhao et al . , 2017 ; Dev and Phillips , 2019 ; Sun et al . , 2019 ) related to gender , race , nationality , or religion , which can manifest in unwanted and/or potentially harmful ways in downstream tasks ( Webster et al . , 2018 ; Zhao et al . , 2019 ; Dev et al . , 2020 ) . Such potentially problematic associations , when embedded in word representations , can lead to incorrect and unfair decisions about large groups of people . While the term " bias " has many meanings , in this paper we use it to refer to these unwanted stereotypical associations . Existing methods to mitigate these effects either require expensive retraining of vectors ( Lemoine et al . , 2018 ) which can be inefficient , or projecting out information contained along an entire subspace representing a protected concept ( such as gender or race ) in the embedding space ( e.g. , Bolukbasi et al . , 2016 ; Dev and Phillips , 2019 ; Ravfogel et al . , 2020 ) . Projective approaches are difficult to control as they are either insufficient : removing a subspace can still leave residual bias ( Gonen and Goldberg , 2019 ; Lauscher et al . , 2020 ) , or too aggressive : in the case of gender , also unnecessarily altering the association between the word pregnant and words like female and mother . In tasks such as coreference resolution , removing such associations could hinder reference resolution . To quantify how much valid information is retained , we look at the output space of models . Following Dev et al . ( 2020 ) , we use Natural Language Inference ( NLI ) as an effective quantitative probe . Here , we construct the hypothesis by making minimal edits to a premise , and observe model prediction conditioned on such changes . For example , Premise : A matriarch sold a watch . Hypothesis : A woman sold a watch . Here , the objective is to determine if the hypothesis is entailed by the premise , contradicted by it , or neither ( neutral to it ) . A GloVe-based NLI ( Parikh et al . , 2016 ) model , without any explicit form of bias mitigation , predicts label entail with a high probability of 97 % ; the notion of a matriarch being a woman is correctly identified by the model . However , after projective debiasing , the model classifies the pair as neutral with a probability 62 % while the probability of the label entail drops to much lower at 16 % . That is , aggressive mitigation of gender representations erases valid gender associations . Ideally , we should correct problematic associations without erasing valid ones . To this end , we propose OSCAR ( Othogonal Subspace Correction and Rectification ) which orthogonalizes and rectifies identified subspaces of concepts that are incorrectly associated in an embedding space . Embeddings outside the subspace are stretched in a graded manner or untouched . Our contributions are : 1 . We argue that mitigating unwanted stereotypical associations should go beyond information removal ( e.g. , projecting out features ) , and should also preserve pertinent associations . 2 . We present OSCAR1 , a completely different method from the existing projective approaches ; it uses orthogonalization of subspaces desired not to have interdependence , and so minimal change is made to embeddings to prevent loss of desired associations . 3 . We develop a combination of tests based on NLI to evaluate both bias mitigation and information retention . 4 . Our experiments show that OSCAR is a wellbalanced approach that mitigates biases as good as projective approaches while retaining more valid associations . Our contributions , focusing specifically on representations rather than classifiers , are important given the preponderance of distributed representations of text across NLP . Predictions from systems that use such representations , if unchecked , could lead to real-world decisions ( involving , e.g. hiring ) that " systematically and unfairly discriminate against certain individuals or groups of individuals in favor of others " ( Friedman and Nissenbaum , 1996 ) . In an effort to prevent such transition of representational harms into allocational harms ( cf . Crawford , 2017 ; Abbasi et al . , 2019 ; Blodgett et al . , 2020 ) , we look at mitigating stereotyping biases at the source , i.e. the embedding space . 2 Bias , Gender and NLP Bias and Related Work . Social biases in tasks using machine learning have the potential to cause harms ( Barocas and Selbst , 2016 ) and are widely studied . Such biases in language technologies have been detected ( Bolukbasi et al . , 2016 ; Gonen and Goldberg , 2019 ) , measured ( Caliskan et al . , 2017 ; Webster et al . , 2018 ; Lauscher et al . , 2020 ) and mitigated ( Zhao et al . , 2018 ; Ravfogel et al . , 2020 ) . Most work focuses on gender bias , and in particular , the stereotypical associations of occupations to males and females ( Rudinger et al . , 2018 ; De-Arteaga et al . , 2019 ; Gaut et al . , 2019 ; Dev et al . , 2020 ) . In this work too , bias refers to stereotypical associations in language representations . Treatment of Gender . The concept of gender is a complex one ( cf . Larson , 2017 ; Dev et al . , 2021 ) . In our experiments , we identify a subspace in the representation associated with gender , typically derived by terms specifically associated with notions of male and female . This aligns with prominent recent NLP research ( Caliskan et al . , 2017 ; Bolukbasi et al . , 2016 ; Dev and Phillips , 2019 ; Ravfogel et al . , 2020 ) that try to measure and mitigate gender bias ( either implicitly or explicitly ) , directly focusing on male versus female associations . The underlying goal of this work is to disassociate two concepts . We mainly use concepts male-vsfemale and occupations to demonstrate this . There is explicit and measurable bias encoded in word vector representations ( Bolukbasi et al . , 2016 ) by the male-vs-female relation to occupations , and standard evaluation metrics ( Caliskan et al . , 2017 ; Dev et al . , 2020 ) are designed to measure this . As such many of our experiments are geared towards these male-vs-female issues . The notion of gender is not restricted to male and female . But because " conceptualizations of gender as binary are reinforced linguistically in English through pronoun conventions " ( Blodgett , 2021 ) , gender notions beyond male and female are either missing or poorly represented . As a result , their associations with other concepts are also not robust . Quantifying this effect , and evaluating its attenuation , is beyond the scope of this paper ; however the techniques we study for disassociating two concepts are amenable to such tasks in general . We remark that a potential harm towards nonmale / female people arises through omission ( Cao and Daumé III , 2020 ) , and as a result , we explicitly encourage further research on this topic . sign of our evaluation dataset in § 3.1 , then present evaluation metrics . Then , we will discuss how to measure biases in § 3.2 and information retention in § 3.3 . To be comprehensive , we will use metrics that are both intrinsic to embedding geometry and extrinsic ones that focus on downstream NLI model outputs . The propagation of undesirable and stereotypical associations learned from data into decisions made by language models maintains a vicious cycle of biases . Combating biases before deploying representations is thus extremely vital . But this poses its own challenges . Word embeddings capture a lot of information implicitly in relatively few dimensions . These implicit associations are what makes them state-of-the-art at tackling different language modeling tasks . Breaking down these associations for bias rectification , thus , has to be done carefully so as to not destroy the structure of the embeddings . OSCAR 's rectification of associations helps integrate both these aspects , allowing it to be more suitable at making word embeddings more usable . Moreover , being computationally lightweight and sub-differentiable , it is simple to apply adaptively without extensive retraining . OSCAR dissociates concepts which may be otherwise lost or overlapped in distributed representations by re-orthogonalizing them . We envision that this method can be used for many types of unwanted associations ( age-ability , religion-virtue perception , etc ) . Since it only decouples specific associations , informative components of these features will remain unchanged . All the experiments in this paper are based on English embeddings , models and tests ; extending our analysis to other languages is important future work . Further , we be-lieve OSCAR can extend in a straightforward way beyond NLP to other distributed , vectorized representations ( e.g. , of images , graphs , or spatial data ) which can also exhibit stereotypical associations .
Existing methods that remove harmful stereotypical associations from word embeddings either require inefficient retraining or remove information which should be retained.
They propose a method which orthogonalizes and rectifies incorrectly associated subspaces of concepts in an embedding space and a metric for evaluating information retention.
NLI-based evaluation on gender-occupation associations shows that the proposed approach is well-balanced ensuring semantic information is retained in the embeddings while mitigating biases.