Frequency of a word has shown to directly influence the semantic representations learnt for that word [1]}, [2]}.
To understand how word frequency influences PLL-based bias evaluation measures, we examine the frequency of words in the advantaged and disadvantaged groups on a corpus that contains Wikipedia articlesWikipedia dump on 2018 Sept is used. & BookCorpus [3]}, popularly used as MLM training data.
This corpus contains a total of 3 billion tokens.
For each bias type in CP, we find the frequency of the words in the corresponding advantaged and disadvantaged groups in this corpus.SS does not split test instances into advantaged vs. disadvantaged groups, hence excluded from this experiment.
Words that have non-stereotypical senses (e.g. white and black are used as colours) are ignored from this analysis.
For words that appear in both groups, we assign them to the group with the higher frequency.
