In the domain of text, many modern approaches often begin by embedding the input text data into an embedding space that is used as the first layer in a subsequent deep network [4] , [14] . These word embeddings have been shown to contain the same biases <cite>[3]</cite> , due to the source data from which they are trained. In effect, biases from the source data, such as in the differences in representation for men and women, that have been found in many different large-scale studies [5] , [10] , [12] , carry through to the semantic relations in the word embeddings, which become baked into the learning systems that are built on top of them.