Image Restoration for Shift Invariant Blurs-A Survey <s> Steepest Descent Iterative Method <s> This new edition covers the central concepts of practical optimization techniques, with an emphasis on methods that are both state-of-the-art and popular. One major insight is the connection between the purely analytical character of an optimization problem and the behavior of algorithms used to solve a problem. This was a major theme of the first edition of this book and the fourth edition expands and further illustrates this relationship. As in the earlier editions, the material in this fourth edition is organized into three separate parts. Part I is a self-contained introduction to linear programming. The presentation in this part is fairly conventional, covering the main elements of the underlying theory of linear programming, many of the most effective numerical algorithms, and many of its important special applications. Part II, which is independent of Part I, covers the theory of unconstrained optimization, including both derivations of the appropriate optimality conditions and an introduction to basic algorithms. This part of the book explores the general properties of algorithms and defines various notions of convergence. Part III extends the concepts developed in the second part to constrained optimization problems. Except for a few isolated sections, this part is also independent of Part I. It is possible to go directly into Parts II and III omitting Part I, and, in fact, the book has been used in this way in many universities.New to this edition is a chapter devoted to Conic Linear Programming, a powerful generalization of Linear Programming. Indeed, many conic structures are possible and useful in a variety of applications. It must be recognized, however, that conic linear programming is an advanced topic, requiring special study. Another important topic is an accelerated steepest descent method that exhibits superior convergence properties, and for this reason, has become quite popular. The proof of the convergence property for both standard and accelerated steepest descent methods are presented in Chapter 8. As in previous editions, end-of-chapter exercises appear for all chapters.From the reviews of the Third Edition: this very well-written book is a classic textbook in Optimization. It should be present in the bookcase of each student, researcher, and specialist from the host of disciplines from which practical optimization applications are drawn. (Jean-Jacques Strodiot, Zentralblatt MATH, Vol. 1207, 2011) <s> BIB001 </s> Image Restoration for Shift Invariant Blurs-A Survey <s> Steepest Descent Iterative Method <s> A simple iterative technique has been developed for blind deconvolution of two convolved functions. The method is described, and a number of results obtained from a computational implementation are presented. Some further possible applications are indicated. <s> BIB002 </s> Image Restoration for Shift Invariant Blurs-A Survey <s> Steepest Descent Iterative Method <s> Linear space-invariant image restoration algorithms often introduce ringing effects near sharp intensity transitions. It is shown that these artifacts are attributable to the regularization of the ill-posed image restoration problem. Two possible methods to reduce the ringing effects in restored images are proposed. The first method incorporates deterministic a priori knowledge about the original image into the restoration algorithm. The second method locally regulates the severity of the noise magnification and the ringing phenomenon, depending on the edge information in the image. A regularized iterative image restoration algorithm is proposed in which both ringing reduction methods are included by making use of the theory of the projections onto convex sets and the concept of norms in a weighted Hilbert space. Both the numerical performance and the visual evaluation of the results are improved by the use of ringing reduction. > <s> BIB003 </s> Image Restoration for Shift Invariant Blurs-A Survey <s> Steepest Descent Iterative Method <s> A class of iterative signal restoration algorithms is derived based on a representation theorem for the generalized inverse of a matrix. These algorithms exhibit a first or higher order of convergence, and some of them consist of an online and an offline computational part. The conditions for convergence, the rate of convergence of these algorithms, and the computational load required to achieve the same restoration results are derived. An iterative algorithm is also presented which exhibits a higher rate of convergence than the standard quadratic algorithm with no extra computational load. These algorithms can be applied to the restoration of signals of any dimensionality. The presented approach unifies a large number of iterative restoration algorithms. Based on the convergence properties of these algorithms, combined algorithms are proposed that incorporate a priori knowledge about the solution in the form of constraints and converge faster than previously published algorithms. > <s> BIB004
An iterative scheme is used when there is no direct solution or a direct solution is not computationally feasible. For iterative scheme new estimate is given by old estimate plus some function of old estimate. The steepest descent method updates an initial estimate iteratively in the reverse direction of the gradient of the cost function C(f ) BIB004 . An iteration of the steepest descent method is given by; Where is the step size and is the ith estimate. The cost function if regularization is not used is defined as; BIB002 which result in to the iteration; BIB003 which is also known as the Landweber iteration  .The iterations are repeated until a convergence criterion is reached which is; i) the maximum number of iterations; ii) the rate of change in the estimated signal or in cost between two successive iterations. The step size should be small enough to guarantee convergence, on the other hand, the convergence rate would be slow if it is too small BIB001 . To guarantee the convergence , where is the maximum singular value of . The conjugate gradient method is used to have faster convergence. In the steepest descent method the estimate is updated in the direction of the residual where as in the conjugate gradient method the update direction is conjugate to previous directions. Advantage of iterative algorithms are: i) there is no need to explicitly implement the inverse of an operator, ii) the restoration process is monitored as it progresses, iii) termination of the algorithm may take place before convergence & the effects of noise can be controlled in each iteration, iv) the algorithms used can be spatially adaptive.