We follow the same lines as [1]}. Fix \(L \ge 5 c_A\) . In the proof, we omit the explicit dependence in \(L\) . First, note that we can write the logarithm of the norm of the hidden state as follows:
\(\log \left\Vert h_k\right\Vert  &= \log \left\Vert x\right\Vert  + \frac{1}{2} \sum _{j=1}^k \log \frac{\left\Vert h_j\right\Vert ^2}{\left\Vert h_{j-1}\right\Vert ^2} \\&= \log \left\Vert x\right\Vert  + \frac{1}{2} \sum _{j=1}^k \log \left( 1 + \underbrace{\frac{2\delta _L}{\left\Vert h_{j-1}\right\Vert ^2} \Big \langle h_{j-1}, \sigma (A_j h_{j-1}) \Big \rangle + \delta _L^2 \frac{\left\Vert \sigma (A_j h_{j-1})\right\Vert ^2}{\left\Vert h_{j-1}\right\Vert ^2}}_{\, \Delta _j} \right).\) 
