Non-convex optimization problems arise from various areas in science and engineering. Although many numerical methods and theories have been developed for unconstrained non-convex problems, the parallel development for constrained non-convex problems remains limited. That restricts the practices of mathematical modeling and quantitative decision making in many disciplines. In this paper, an inexact proximal-point penalty method is proposed for constrained optimization problems where both the objective function and the constraint can be non-convex. The proposed method approximately solves a sequence of subproblems, each of which is formed by adding to the original objective function a proximal term and quadratic penalty terms associated to the constraint functions. Under a weak-convexity assumption, each subproblem is made strongly convex and can be solved effectively to a required accuracy by an optimal gradient-type method. The theoretical property of the proposed method is analyzed in two different cases. In the first case, the objective function is non-convex but the constraint functions are assumed to be convex, while in the second case, both the objective function and the constraint are non-convex. For both cases, we give the complexity results in terms of the number of function value and gradient evaluations to produce near-stationary points. Due to the different structures, different definitions of near-stationary points are given for the two cases. The complexity for producing a nearly @math -stationary point is @math for the first case while it becomes @math for the second case.
Consider the problem of finding the local constrained minimum @math of the function f on the set [ F = x R^n | _i (x) 0, _ j + k (x) = 0; i = 1,2, ,k; j = 1,2, ,l . ]One method of solution is to minimize the associated penalty function [ p_0 (x) = f(x) - _ i = 1 ^k ( 0, _i (x) ) + _ j = 1 ^l | _ j + k (x) | for x R^n , 0. ]Let @math be the minimum of this penalty function. It is known that, provided @math is sufficiently small, @math .However, until recently, a serious drawback to this particular penalty function was that its first order derivatives are not everywhere defined. Thus, well-known gradient type methods usually applied to unconstrained optimization problems were necessarily excluded.This paper presents a method that enables a modified form of the gradient type approaches to be applied to a perturbation of the penalt...  In this paper a new continuously differentiable exact penalty function is introduced for the solution of nonlinear programming problems with compact feasible set. A distinguishing feature of the penalty function is that it is defined on a suitable bounded open set containing the feasible region and that it goes to infinity on the boundary of this set. This allows the construction of an implementable unconstrained minimization algorithm, whose global convergence towards Kuhn-Tucker points of the constrained problem can be established. We develop two new proximal alternating penalty algorithms to solve a wide range class of constrained convex optimization problems. Our approach mainly relies on a novel combination of the classical quadratic penalty, alternating minimization, Nesterov’s acceleration, adaptive strategy for parameters. The first algorithm is designed to solve generic and possibly nonsmooth constrained convex problems without requiring any Lipschitz gradient continuity or strong convexity, while achieving the best-known ( O ( 1 k ) )-convergence rate in a non-ergodic sense, where k is the iteration counter. The second algorithm is also designed to solve non-strongly convex, but semi-strongly convex problems. This algorithm can achieve the best-known ( O ( 1 k^2 ) )-convergence rate on the primal constrained problem. Such a rate is obtained in two cases: (1) averaging only on the iterate sequence of the strongly convex term, or (2) using two proximal operators of this term without averaging. In both algorithms, we allow one to linearize the second subproblem to use the proximal operator of the corresponding objective term. Then, we customize our methods to solve different convex problems, and lead to new variants. As a byproduct, these algorithms preserve the same convergence guarantees as in our main algorithms. We verify our theoretical development via different numerical examples and compare our methods with some existing state-of-the-art algorithms.  The non-linear programming problem seeks to maximize a function f(x) where the n component vector x must satisfy certain constraints gi(x) = 0, i = 1, …, m1 and gi(z) ≧ 0, i = m1 + 1, …, m. The algorithm presented in this paper solves the non-linear programming problem by transforming it into a sequence of unconstrained maximization problems. Essentially, a penalty is imposed whenever x does not satisfy the constraints. Although the algorithm appears most useful in the concave case, the convergence proof holds for non-concave functions as well. The algorithm is especially interesting in the concave case because the programming problem reduces to a single unconstrained maximization problem or, at most, to a finite sequence of unconstrained maximization problems. In addition, the paper presents a new class of dual problems, and the algorithm is shown to be a dual feasible method. Another property of the algorithm is that it appears particularly well suited for large-scale problems with a sizable number of c... An algorithm for solving the problem: minimize @math (a convex function) subject to @math , @math , each @math a concave function, is presented. Specifically, the function [ P [ x,t,r_k ] f( x ) + r_k^ - 1 [ g_i ( x ) - t_i ] ^2 ] is minimized over all x, nonnegative t, for a strictly decreasing null sequence @math . This extends the work of T. Pietrzykowski [5]. It is proved that for every @math , there exists a finite point @math which minimizes P, and which solves the convex programming problem as @math . This algorithm is similar to the Sequential Unconstrained Minimization Technique (SUMT) [1] in that it solves the (Wolfe) dual programming problem [6]. It differs from SUMT in that (1) it approaches the optimum from the region of infeasibility (i.e., it is a relaxation technique), (2) it does not require a nonempty interior to the nonlinearly constrained region, (3) no separate feasibilit... In this paper we present a complete iteration complexity analysis of inexact first-order Lagrangian and penalty methods for solving cone-constrained convex problems that have or may not have optimal Lagrange multipliers that close the duality gap. We first assume the existence of optimal Lagrange multipliers and study primal–dual first-order methods based on inexact information and augmented Lagrangian smoothing or Nesterov-type smoothing. For inexact (fast) gradient augmented Lagrangian methods, we derive an overall computational complexity of O(1 ϵ) projections onto a simple primal set in order to attain an e-optimal solution of the conic convex problem. For the inexact fast gradient method combined with Nesterov-type smoothing, we derive computational complexity O(1 ϵ3 2) projections onto the same set. Then, we assume that optimal Lagrange multipliers might not exist for the cone-constrained convex problem, and analyse the fast gradient method for solving penalty reformulations of the problem. For the ... It is shown that the existence of a strict local minimum satisfying the constraint qualification of [16] or McCormick's [12] second order sufficient optimality condition implies the existence of a class of exact local penalty functions (that is ones with a finite value of the penalty parameter) for a nonlinear programming problem. A lower bound to the penalty parameter is given by a norm of the optimal Lagrange multipliers which is dual to the norm used in the penalty function. The convergence behaviour of a class of iterative methods for solving the constrained minimization problem is analysed. The methods are based on the sequential minimization of a simple differentiable penalty function. They are sufficiently general to ensure global convergence of the iterates to the solution of the problem at an asymptotic (two-step Q-) superlinear rate. The global convergence properties of a class of penalty methods for nonlinear programming are analyzed. These methods include successive linear programming approaches and, more specifically, the successive linear-quadratic programming approach presented by [Math. Program., 100 (2004), pp. 27--48]. Every iteration requires the solution of two trust-region subproblems involving piecewise linear and quadratic models, respectively. It is shown that, for a fixed penalty parameter, the sequence of iterates approaches stationarity of the penalty function. A procedure for dynamically adjusting the penalty parameter is described, and global convergence results for it are established. The main result of the paper consists of the theorem that under certain, natural assumptions the local conditional maximum @math of the function f on the set [ A = x R^n | _i (x) 0, _j (x) = 0,i = 1, ,k,j = 1, ,l ] is identical with the unconditional maximum of the potential function [ p(x, ) = f(x) + _ i = 1 ^k neg ( _i (x)) - _ j = 1 ^l | (x) | , x R^n , 0, ] for @math sufficiently small. There is also provided a draft of a modified gradient procedure for maximizing the potential @math since it is generally nonsmooth even for differentiable f, @math and @math . In their seminal papers Eremin [Soviet Mathematics Doklady, 8 (1966), pp. 459–462] and Zangwill [Management Science, 13 (1967), pp. 344–358] introduce a notion of exact penalization for use in the development of algorithms for constrained optimization. Since that time, exact penalty functions have continued to play a key role in the theory of mathematical programming. In the present paper, this theory is unified by showing how the Eremin–Zangwill exact penalty functions can be used to develop the foundations of the theory of constrained optimization for finite dimensions in an elementary and straightforward way. Regularity conditions, multiplier rules, second-order optimality conditions, and convex programming are all given interpretations relative to the Eremin–Zangwill exact penalty functions. In conclusion, a historical review of those results associated with the existence of an exact penalty parameter is provided. This paper presents a multiplier method for solving optimization problems with equality and inequality constraints. The method realizes all the good features that were foreseen by R. Fletcher for this type of algorithm in the past, but which suffers from none of the drawbacks of the earlier attempts. In this paper, a recursive quadratic programming algorithm for solving equality constrained optimization problems is proposed and studied. The line search functions used are approximations to Fletcher's differentiable exact penalty function. Global convergence and local superlinear convergence results are proved, and some numerical results are given. In this paper it is shown that, given a nonlinear programming problem with inequality constraints, it is possible to construct a continuously differentiable exact penalty function whose global or local unconstrained minimizers correspond to global or local solutions of the constrained problem. This paper considers a special but broad class of convex programming problems whose feasible region is a simple compact convex set intersected with the inverse image of a closed convex cone under an affine transformation. It studies the computational complexity of quadratic penalty based methods for solving the above class of problems. An iteration of these methods, which is simply an iteration of Nesterov’s optimal method (or one of its variants) for approximately solving a smooth penalization subproblem, consists of one or two projections onto the simple convex set. Iteration-complexity bounds expressed in terms of the latter type of iterations are derived for two quadratic penalty based variants, namely: one which applies the quadratic penalty method directly to the original problem and another one which applies the latter method to a perturbation of the original problem obtained by adding a small quadratic term to its objective function.