In this paper, we present a novel yet simple homotopy proximal mapping algorithm for compressive sensing. The algorithm adopts a simple proximal mapping of the @math norm at each iteration and gradually reduces the regularization parameter for the @math norm. We prove a global linear convergence of the proposed homotopy proximal mapping (HPM) algorithm for solving compressive sensing under three different settings (i) sparse signal recovery under noiseless measurements, (ii) sparse signal recovery under noisy measurements, and (iii) nearly-sparse signal recovery under sub-gaussian noisy measurements. In particular, we show that when the measurement matrix satisfies Restricted Isometric Properties (RIP), our theoretical results in settings (i) and (ii) almost recover the best condition on the RIP constants for compressive sensing. In addition, in setting (iii), our results for sparse signal recovery are better than the previous results, and furthermore our analysis explicitly exhibits that more observations lead to not only more accurate recovery but also faster convergence. Compared with previous studies on linear convergence for sparse signal recovery, our algorithm is simple and efficient, and our results are better and provide more insights. Finally our empirical studies provide further support for the proposed homotopy proximal mapping algorithm and verify the theoretical results.
Recently, several algorithms exhibit global linear convergence for the BPDN problem. @cite_2 studied an optimization problem ) for statistical recovery. They used a different update where @math , and @math is a parameter related to the restricted smoothness of the loss function. They proved a global linear convergence of the above update with @math for finding a solution up to the statistical tolerance. @cite_0 studied a proximal-gradient homotopy gradient method for solving ). They iteratively solve the problem ) by the proximal gradient descent with a decreasing regularization parameter @math and an increasing accuracy at each stage, and use the solution obtained at each stage to warm start the next stage. A global linear convergence was also established.
@cite_0 @cite_2
2161227280 2083042020 1668142578 2163985581
We consider solving the @math -regularized least-squares ( @math -LS) problem in the context of sparse recovery for applications such as compressed sensing. The standard proximal gradient method, also known as iterative soft-thresholding when applied to this problem, has low computational cost per iteration but a rather slow convergence rate. Nevertheless, when the solution is sparse, it often exhibits fast linear convergence in the final stage. We exploit the local linear convergence using a homotopy continuation strategy, i.e., we solve the @math -LS problem for a sequence of decreasing values of the regularization parameter, and use an approximate solution at the end of each stage to warm start the next stage. Although similar strategies have been studied in the literature, there have been no theoretical analysis of their global iteration complexity. This paper shows that under suitable assumptions for sparse recovery, the proposed homotopy strategy ensures that all iterates along the homotopy sol... The basis pursuit problem seeks a minimum one-norm solution of an underdetermined least-squares problem. Basis pursuit denoise (BPDN) fits the least-squares problem only approximately, and a single parameter determines a curve that traces the optimal trade-off between the least-squares fit and the one-norm of the solution. We prove that this curve is convex and continuously differentiable over all points of interest, and show that it gives an explicit relationship to two other optimization problems closely related to BPDN. We describe a root-finding algorithm for finding arbitrary points on this curve; the algorithm is suitable for problems that are large scale and for those that are in the complex domain. At each iteration, a spectral gradient-projection method approximately minimizes a least-squares problem with an explicit one-norm constraint. Only matrix-vector operations are required. The primal-dual solution of this problem gives function and derivative information needed for the root-finding method. Numerical experiments on a comprehensive set of test problems demonstrate that the method scales well to large problems. The decoding of Low-Density Parity-Check codes by the Belief Propagation (BP) algorithm is revisited. We check the iterative algorithm for its convergence to a codeword (termination), we run Monte Carlo simulations to find the probability distribution function of the termination time, n_it. Tested on an example [155, 64, 20] code, this termination curve shows a maximum and an extended algebraic tail at the highest values of n_it. Aiming to reduce the tail of the termination curve we consider a family of iterative algorithms modifying the standard BP by means of a simple relaxation. The relaxation parameter controls the convergence of the modified BP algorithm to a minimum of the Bethe free energy. The improvement is experimentally demonstrated for Additive-White-Gaussian-Noise channel in some range of the signal-to-noise ratios. We also discuss the trade-off between the relaxation parameter of the improved iterative scheme and the number of iterations. We address the problem of estimating spectral lines from irregularly sampled data within the framework of sparse representations. Spectral analysis is formulated as a linear inverse problem, which is solved by minimizing an l1-norm penalized cost function. This approach can be viewed as a basis pursuit de-noising (BPDN) problem using a dictionary of cisoids with high frequency resolution. In the studied case, however, usual BPDN characterizations of uniqueness and sparsity do not apply. This paper deals with the l1-norm penalization of complex-valued variables, that brings satisfactory prior modeling for the estimation of spectral lines. An analytical characterization of the minimizer of the criterion is given and geometrical properties are derived about the uniqueness and the sparsity of the solution. An efficient optimization strategy is proposed. Convergence properties of the iterative coordinate descent (ICD) and iterative reweighted least-squares (IRLS) algorithms are first examined. Then, both strategies are merged in a convergent procedure, that takes advantage of the specificities of ICD and IRLS, considerably improving the convergence speed. The computation of the resulting spectrum estimator can be implemented efficiently for any sampling scheme. Algorithm performance and estimation quality are illustrated throughout the paper using an artificial data set, typical of some astrophysical problems, where sampling irregularities are caused by day night alternation. We show that accurate frequency location is achieved with high resolution. In particular, compared with sequential Matching Pursuit methods, the proposed approach is shown to achieve more robustness regarding sampling artifacts.