Therefore finally, we have proven the impossibility result Lemma 4.2(ii) in all common model architectures (all non-residual networks that end with dense layers, and all transformers from Biderman (2024)).