so cancels the LayerNorm bias term. Hence, if we apply the Lemma 4.1 attack with this subtraction modification to a model using LayerNorm, then the resultant '\(h\)' output will be smaller by 1 (due to Appendix B.1). This would imply the model used LayerNorm rather than RMSNorm, because RMSNorm does not project onto a smaller subspace and so would not have a decrease in '\(h\)' value if we were to use this subtraction trick.