Lemma REF  shows that the transition Eq.(REF ) is a contraction mapping if a small enough scalar \(\gamma \)  is selected for which \(\gamma \Vert A\Vert <1\) . The proof is provided in Appendix REF . However, the operator norm of \(A(k)\)  changes throughout the training process. In general, a fixed scalar \(\gamma \)  cannot guarantee the well-posedness during the entire training. That is the main reason [1]}, [2]}, [3]}, [4]} observe the number of forward iterations required to find a fixed point gradually grows with training epochs. Thus, a simple strategy is to find a sequence \(\lbrace \gamma _k\rbrace _{k=1}^{\infty }\)  of appropriate scalars for each iteration that ensure the equilibrium equation are kept well-posed over all iterations. However, finding an appropriate scalars at each iteration is computationally expensive, since it needs to compute quantities related to the operator norm of the weight matrix \(A(k)\)  (e.g., [5]}, [3]}). Fortunately, we can show \(\Vert A(k)\Vert \)  is uniformly upper bounded by some constant. As a result, there indeed is a small constant for which the forward pass is kept well-posed throughout training.
