Zhao et al. [35] and Basta et al. [1] demonstrate gender bias in ELMo [25] word embeddings, whereas May et al. <cite>[21]</cite> evaluate various models of contextual word representations on a sentential generalization of WEAT.