Specifically, we find that the bias metric proposed by <cite>(Bolukbasi et al. 2016 )</cite> is highly sensitive to embedding hyperparameter selection, and that in many cases, the variance due to the selection of some hyper-parameters, notably the embedding space dimensionality, is greater than the variance in the metric due to corpus selection, while in fewer cases, even the relative rankings of the bias measured in the embedding spaces of various corpora varies with hyper-parameter selection.