While it is the case that the bias metrics in <cite>(Bolukbasi et al. 2016 )</cite> may provide meaningful rankings of corpora when controlling for model hyper-parameter configuration, publishing the average absolute value of the metric without a complete account for model configuration is suspect.