As a final stress test, we found that all open language models that use RMSNorm do not use any bias terms (Biderman, 2024). Therefore, we checked that our attack would not give a false positive when applied to a model with RMSNorm but with biases.