On the other hand, we also test how the choice of \(\gamma \) 
impacts the number of iteration forward propagation needed to
find the equilibrium point. We stop the forward propagation if
either \(\Vert Z^{\ell +1}-Z^{\ell }\Vert \le 10^{-2}\)  or reaches the
max iteration of 100. Table REF  contains the
averaged number of forward iteration for different \(\gamma \) 
values to find the equilibrium point \(Z\)  on MNIST. Although Figure REF  shows the lower training
and test losses can be obtained by using relatively larger
\(\gamma \) , Table REF  indicates larger \(\gamma \)  results
in more iterations needed in the forward propagation to find the
equilibrium point. Moreover, as the operator norm of \(\gamma A(k)\)  becomes larger as growth of epoch, forward propagation
takes even more iterations to converge. This scenario is also
observed in some previous works
[1]}, [2]}, [3]}. Thus, there
is probably a trade-off between the convergence speed and
well-posedness of implicit neural networks.
