In this paper, we introduce a class of nonsmooth nonconvex least square optimization problem using convex analysis tools and we propose to use the iterative minimization-majorization (MM) algorithm on a convex set with initializer away from the origin to find an optimal point for the optimization problem. For this, first we use an approach to construct a class of convex majorizers which approximate the value of non-convex cost function on a convex set. The convergence of the iterative algorithm is guaranteed when the initial point @math is away from the origin and the iterative points @math are obtained in a ball centred at @math with small radius. The algorithm converges to a stationary point of cost function when the surregators are strongly convex. For the class of our optimization problems, the proposed penalizer of the cost function is the difference of @math -norm and the Moreau envelope of a convex function, and it is a generalization of GMC non-separable penalty function previously introduced by Ivan Selesnick in IS17 .
The current paper is proposing the use of majorization-minimization (MM) algorithm to solve the class of nonconvex nonsmooth optimization problems of type ). This approach has been used for example in @cite_8 @cite_10 @cite_0 for solving some optimization problems different than what we consider here. There are another types of methods that have been proved effective in solving nonconvex problems. For example, iteratively reweighted least squares (IRLS) method ( @cite_3 ) and iteratively reweighted @math (IRL1) ( @cite_4 ). For a list of other methods, we refer the reader to see @cite_0 and the reference therein.
@cite_4 @cite_8 @cite_3 @cite_0 @cite_10
2531491639 1498400193 2214177029 2549127690
The class of majorization---minimization algorithms is based on the principle of successively minimizing upper bounds of the objective function. Each upper bound, or surrogate function, is locally tight at the current estimate, and each minimization step decreases the value of the objective function. We present a majorization---minimization approach based on a novel convex---nonconvex upper bounding strategy for the solution of a certain class of nonconvex nonsmooth optimization problems. We propose an efficient algorithm for minimizing the (convex) surrogate function based on the alternating direction method of multipliers. A preliminary convergence analysis for the proposed approach is provided. Numerical experiments show the effectiveness of the proposed method for the solution of nonconvex nonsmooth minimization problems. In this paper, we study the global convergence of majorization minimization (MM) algorithms for solving nonconvex regularized optimization problems. MM algorithms have received great attention in machine learning. However, when applied to nonconvex optimization problems, the convergence of MM algorithms is a challenging issue. We introduce theory of the Kurdyka- Lojasiewicz inequality to address this issue. In particular, we show that many nonconvex problems enjoy the Kurdyka- Lojasiewicz property and establish the global convergence result of the corresponding MM procedure. We also extend our result to a well known method that called CCCP (concave-convex procedure). Motivated by the recent developments of nonconvex penalties in sparsity modeling, we propose a nonconvex optimization model for handing the low-rank matrix recovery problem. Different from the famous robust principal component analysis (RPCA), we suggest recovering low-rank and sparse matrices via a nonconvex loss function and a nonconvex penalty. The advantage of the nonconvex approach lies in its stronger robustness. To solve the model, we devise a majorization-minimization augmented Lagrange multiplier (MM-ALM) algorithm which finds the local optimal solutions of the proposed nonconvex model. We also provide an efficient strategy to speedup MM-ALM, which makes the running time comparable with the state-of-the-art algorithm of solving RPCA. Finally, empirical results demonstrate the superiority of our nonconvex approach over RPCA in terms of matrix recovery accuracy. In this paper we study nonconvex and nonsmooth optimization problems with semialgebraic data, where the variables vector is split into several blocks of variables. The problem consists of one smooth function of the entire variables vector and the sum of nonsmooth functions for each block separately. We analyze an inertial version of the proximal alternating linearized minimization algorithm and prove its global convergence to a critical point of the objective function at hand. We illustrate our theoretical findings by presenting numerical experiments on blind image deconvolution, on sparse nonnegative matrix factorization and on dictionary learning, which demonstrate the viability and effectiveness of the proposed method.