Next, we evaluated the stability of the direct bias measure proposed in <cite>(Bolukbasi et al. 2016)</cite> . <cite>The authors</cite> assert that the direct bias measure can be used as a metric to conclude how much an embedding is biased. For instance, for word embeddings trained on Google News articles, <cite>they</cite> reported that direct gender bias on 327 profession names is 0.08 and thus <cite>they</cite> concluded that this embedding is biased. However, as we have illustrated in Figure 3a , this bias score is not stable, i.e., direct bias measure decays exponentially with increasing word embedding dimension.