where \(\mathcal {K}_i = \text{span}\left\lbrace  \Phi _0, A\Phi _0, A^2 \Phi _0, \dots , A^{i-1}\Phi _0 \right\rbrace \)  is known as the \(i\) -th Krylov subspace and \(\Phi _0 = \Phi (x_0)\) .
In particular, at each iteration GMRES generates an orthonormal basis of \(\mathcal {K}_i\)  via the Arnoldi's method [1]} and then deploys it to solve (REF ).
Unlike the conjugate gradient method, the orthonormal bases can not be computed with a short recurrence. When \(i\)  increases the number of stored vectors increases like \(i\)  and the number of multiplication like \(0.5 i^2 n\) . A practical variant of GMRES, denoted as GMRES\((i)\) , consists in restarting the algorithm after every \(i\)  iterations.
