Different choices of conditioning matrix \(P_k\)  lead to different algorithms. Newton-Raphson (nr) iterations use \(P_k = [H_n(\theta _k)]^{-1}\)  and \(\gamma _k=1\) ; a damped update uses \(\gamma _k<1\) . For strongly convex problems, nr  enjoys a fast quadratic convergence property, but it comes at the cost of computing both the gradient and the Hessian at each iteration \(k\) .In statistical computing, the convergence of \(\theta _k\)  to \(\hat{\theta }_n\)  is said to be linear if \(\Vert \theta _{k+1}-\hat{\theta }_n\Vert /\Vert \theta _k-\hat{\theta }_n\Vert ^q<r\)  for some \(r\in (0,1)\)  if \(q=1\)  and quadratic if \(q=2\) . Convergence is superlinear if \(\lim _{k\rightarrow \infty } \Vert \theta _{k+1}-\hat{\theta }_n\Vert / \Vert \theta _k-\hat{\theta }_n\Vert =0\) . See [1]} Section 9.3.1 for linear convergence of gradient methods, and [2]} for quadratic convergence of Newton's method when \(\gamma =1\)  or \(\gamma _k \rightarrow 1\)  at an appropriate rate. `Damped Newton' updating with \(\gamma _k \in (0,1)\)  has a linear convergence rate, see [1]} Section 9.5.3 and [4]} Section 1.2.4.
To avoid the cost of computing the \((d_\theta +1)d_\theta /2\)  second-order derivatives in \(H_n\) . Other updates rely on different choices of \(P_k\) . These include quasi-Newton (qn) and gradient-descent (gd).
