We analyze the decomposition problem of multivariate polynomial-exponential functions from truncated series and present new algorithms to compute their decomposition. Using the duality between polynomials and formal power series, we first show how the elements in the dual of an Artinian algebra correspond to polynomial-exponential functions. They are also the solutions of systems of partial differential equations with constant coefficients. We relate their representation to the inverse system of the roots of the characteristic variety. Using the properties of Hankel operators, we establish a correspondence between polynomial exponential series and Artinian Gorenstein algebras. We generalize Kronecker theorem to the multivariate case, by showing that the symbol of a Hankel operator of finite rank is a polynomial-exponential series and by connecting the rank of the Hankel operator with the decomposition of the symbol. A generalization of Prony's approach to multivariate decomposition problems is presented , exploiting eigenvector methods for solving polynomial equations. We show how to compute the frequencies and weights of a minimal polynomial-exponential decomposition , using the first coefficients of the series. A key ingredient of the approach is the flat extension criteria, which leads to a multivariate generalization of a rank condition for a Carath 'e odory-Fej 'e r decomposition of multivariate Hankel matrices. A new algorithm is given to compute a basis of the Artinian Gorenstein algebra, based on a Gram-Schmidt orthogonalization process and to decompose polynomial-exponential series. A general framework for the applications of this approach is described and illustrated in different problems. We provide Kronecker-type theorems for convolution operators, showing that a convolution operator (or a cross-correlation operator) is of finite rank, if and only if, its symbol is a polynomial-exponential function, and we relate its rank to the decomposition of its symbol. We also present Kronecker-type theorems for the reconstruction of measures as weighted sums of Dirac measures from moments and for the decomposition of polynomial-exponential functions from values. Finally, we describe an application of this method for the sparse interpolation of polylog functions from values.
A completely different approach, known as compressive sensing , has been developed over the last decades to compute sparse decompositions of functions (see e.g. @cite_1 ). In this approach, a (large) dictionary of functions is chosen and a sparse combination with few non-zero coefficients is computed from some observations. This boils to find a sparse solution @math of an underdetermined linear system @math . Such a solution, which minimizes the @math norm'' can be computed by @math minimization, under some hypothesis.
@cite_1
2164452299
In this paper, a fast algorithm for overcomplete sparse decomposition, called SL0, is proposed. The algorithm is essentially a method for obtaining sparse solutions of underdetermined systems of linear equations, and its applications include underdetermined sparse component analysis (SCA), atomic decomposition on overcomplete dictionaries, compressed sensing, and decoding real field codes. Contrary to previous methods, which usually solve this problem by minimizing the l 1 norm using linear programming (LP) techniques, our algorithm tries to directly minimize the l 1 norm. It is experimentally shown that the proposed algorithm is about two to three orders of magnitude faster than the state-of-the-art interior-point LP solvers, while providing the same (or better) accuracy. Consider the recovery of an unknown signal @math from quantized linear measurements. In the one-bit compressive sensing setting, one typically assumes that @math is sparse, and that the measurements are of the form @math . Since such measurements give no information on the norm of @math , recovery methods typically assume that @math . We show that if one allows more generally for quantized affine measurements of the form @math , and if the vectors @math are random, an appropriate choice of the affine shifts @math allows norm recovery to be easily incorporated into existing methods for one-bit compressive sensing. In addition, we show that for arbitrary fixed @math in the annulus @math , one may estimate the norm @math up to additive error @math from @math such binary measurements through a single evaluation of the inverse Gaussian error function. Finally, all of our recovery guarantees can be made universal over sparse vectors in the sense that with high probability, one set of measurements and thresholds can successfully estimate all sparse vectors @math in a Euclidean ball of known radius. Consider the recovery of an unknown signal @math from quantized linear measurements. In the one-bit compressive sensing setting, one typically assumes that @math is sparse, and that the measurements are of the form @math . Since such measurements give no information on the norm of @math , recovery methods from such measurements typically assume that @math . We show that if one allows more generally for quantized affine measurements of the form @math , and if the vectors @math are random, an appropriate choice of the affine shifts @math allows norm recovery to be easily incorporated into existing methods for one-bit compressive sensing. Additionally, we show that for arbitrary fixed @math in the annulus @math , one may estimate the norm @math up to additive error @math from @math such binary measurements through a single evaluation of the inverse Gaussian error function. Finally, all of our recovery guarantees can be made universal over sparse vectors, in the sense that with high probability, one set of measurements and thresholds can successfully estimate all sparse vectors @math within a Euclidean ball of known radius. We consider a linear stochastic bandit problem where the dimension @math of the unknown parameter @math is larger than the sampling budget @math . In such cases, it is in general impossible to derive sub-linear regret bounds since usual linear bandit algorithms have a regret in @math . In this paper we assume that @math is @math sparse, i.e. has at most @math non-zero components, and that the space of arms is the unit ball for the @math norm. We combine ideas from Compressed Sensing and Bandit Theory and derive algorithms with regret bounds in @math . A full-rank matrix @math with @math generates an underdetermined system of linear equations @math having infinitely many solutions. Suppose we seek the sparsest solution, i.e., the one with the fewest nonzero entries. Can it ever be unique? If so, when? As optimization of sparsity is combinatorial in nature, are there efficient methods for finding the sparsest solution? These questions have been answered positively and constructively in recent years, exposing a wide variety of surprising phenomena, in particular the existence of easily verifiable conditions under which optimally sparse solutions can be found by concrete, effective computational methods. Such theoretical results inspire a bold perspective on some important practical problems in signal and image processing. Several well-known signal and image processing problems can be cast as demanding solutions of undetermined systems of equations. Such problems have previously seemed, to many, intractable, but there is considerable evidence that these problems often have sparse solutions. Hence, advances in finding sparse solutions to underdetermined systems have energized research on such signal and image processing problems—to striking effect. In this paper we review the theoretical results on sparse solutions of linear systems, empirical results on sparse modeling of signals and images, and recent applications in inverse problems and compression in image processing. This work lies at the intersection of signal processing and applied mathematics, and arose initially from the wavelets and harmonic analysis research communities. The aim of this paper is to introduce a few key notions and applications connected to sparsity, targeting newcomers interested in either the mathematical aspects of this area or its applications. Compressed sensing of simultaneously sparse and rank-one matrices enables recovery of sparse signals from a few linear measurements of their bilinear form. One important question is how many measurements are needed for a stable reconstruction in the presence of measurement noise. Unlike the conventional compressed sensing for sparse vectors, where convex relaxation via the @math -norm achieves near optimal performance, for compressed sensing of sparse and rank-one matrices, recently it has been shown by that convex programmings using the nuclear norm and the mixed norm are highly suboptimal even in the noise-free scenario. We propose an alternating minimization algorithm called sparse power factorization (SPF) for compressed sensing of sparse rank-one matrices. Starting from a particular initialization, SPF achieves stable recovery and requires number of measurements within a logarithmic factor of the information-theoretic fundamental limit. For fast-decaying sparse signals, SPF starting from an initialization with low computational cost also achieves stable reconstruction with the same number of measurements. Numerical results show that SPF empirically outperforms the best known combinations of mixed norm and nuclear norm. The minimum @math -norm solution to an underdetermined system of linear equations @math , is often, remarkably, also the sparsest solution to that system. In this paper, we provide a and condition for @math to be identifiable for a large set of matrices @math ; that is to be the unique sparsest solution to the @math -norm minimization problem. Furthermore, we prove that this sparsest solution is stable under a reasonable perturbation of the observations @math . We also propose an efficient semi-greedy algorithm to check our condition for any vector @math . We present numerical experiments showing that our condition is able to predict almost perfectly all identifiable solutions @math , whereas other previously proposed criteria are too pessimistic and fail to identify properly some identifiable vectors @math . Beside the theoretical proof, this provides empirical evidence to support the sharpness of our condition. Two compressive sensing inspired approaches for the solution of non-linear inverse scattering problems are introduced and discussed. Differently from the sparsity promoting approaches proposed in most of the papers published in the literature, the two methods here tackle the problem in its full non-linearity, by adopting a contrast source inversion scheme. In the first approach, the @math -norm of the unknown is added as a weighted penalty term to the contrast source cost functional. The second, and (to the best of our knowledge) completely original, approach enforces sparsity by constraining the solution of the non-linear problem into a convex set defined by the @math -norm of the unknown. A numerical assessment against a widely used benchmark example (the “Austria” profile) is given to assess the capabilities of the proposed approaches. Notably, the two approaches can be applied to any kind of basis functions and they can successfully tackle both reduced number of data (with respect to Nyquist sampling) and or overcomplete dictionaries. In sparse recovery we are given a matrix @math (the dictionary) and a vector of the form @math where @math is sparse, and the goal is to recover @math . This is a central notion in signal processing, statistics and machine learning. But in applications such as sparse coding, edge detection, compression and super resolution, the dictionary @math is unknown and has to be learned from random examples of the form @math where @math is drawn from an appropriate distribution --- this is the dictionary learning problem. In most settings, @math is overcomplete: it has more columns than rows. This paper presents a polynomial-time algorithm for learning overcomplete dictionaries; the only previously known algorithm with provable guarantees is the recent work of Spielman, Wang and Wright who gave an algorithm for the full-rank case, which is rarely the case in applications. Our algorithm applies to incoherent dictionaries which have been a central object of study since they were introduced in seminal work of Donoho and Huo. In particular, a dictionary is @math -incoherent if each pair of columns has inner product at most @math . The algorithm makes natural stochastic assumptions about the unknown sparse vector @math , which can contain @math non-zero entries (for any @math ). This is close to the best @math allowable by the best sparse recovery algorithms even if one knows the dictionary @math exactly. Moreover, both the running time and sample complexity depend on @math , where @math is the target accuracy, and so our algorithms converge very quickly to the true dictionary. Our algorithm can also tolerate substantial amounts of noise provided it is incoherent with respect to the dictionary (e.g., Gaussian). In the noisy setting, our running time and sample complexity depend polynomially on @math , and this is necessary. Phase retrieval problems involve solving linear equations, but with missing sign (or phase, for complex numbers) information. More than four decades after it was first proposed, the seminal error reduction algorithm of (Gerchberg and Saxton 1972) and (Fienup 1982) is still the popular choice for solving many variants of this problem. The algorithm is based on alternating minimization; i.e. it alternates between estimating the missing phase information, and the candidate solution. Despite its wide usage in practice, no global convergence guarantees for this algorithm are known. In this paper, we show that a (resampling) variant of this approach converges geometrically to the solution of one such problem -- finding a vector @math from @math , where @math and @math denotes a vector of element-wise magnitudes of @math -- under the assumption that @math is Gaussian. Empirically, we demonstrate that alternating minimization performs similar to recently proposed convex techniques for this problem (which are based on "lifting" to a convex matrix problem) in sample complexity and robustness to noise. However, it is much more efficient and can scale to large problems. Analytically, for a resampling version of alternating minimization, we show geometric convergence to the solution, and sample complexity that is off by log factors from obvious lower bounds. We also establish close to optimal scaling for the case when the unknown vector is sparse. Our work represents the first theoretical guarantee for alternating minimization (albeit with resampling) for any variant of phase retrieval problems in the non-convex setting. We consider solving the @math -regularized least-squares ( @math -LS) problem in the context of sparse recovery for applications such as compressed sensing. The standard proximal gradient method, also known as iterative soft-thresholding when applied to this problem, has low computational cost per iteration but a rather slow convergence rate. Nevertheless, when the solution is sparse, it often exhibits fast linear convergence in the final stage. We exploit the local linear convergence using a homotopy continuation strategy, i.e., we solve the @math -LS problem for a sequence of decreasing values of the regularization parameter, and use an approximate solution at the end of each stage to warm start the next stage. Although similar strategies have been studied in the literature, there have been no theoretical analysis of their global iteration complexity. This paper shows that under suitable assumptions for sparse recovery, the proposed homotopy strategy ensures that all iterates along the homotopy sol... We investigate the minimization of Newton's functional for the problem of the body of minimal resistance with maximal height @math cite butt in the class of convex developable functions defined in a disc. This class is a natural candidate to find a (non-radial) minimizer in accordance with the results of cite lrp2. We prove that the minimizer in this class has a minimal set in the form of a regular polygon with @math sides centered in the disc, and numerical experiments indicate that the natural number @math is a non-decreasing function of @math . The corresponding functions all achieve a lower value of the functional than the optimal radially symmetric function with the same height @math . We consider the problem of estimating a deterministic finite alphabet vector @math from underdetermined measurements @math , where @math is a given (random) @math matrix. Two new convex optimization methods are introduced for the recovery of finite alphabet signals via @math -norm minimization. The first method is based on regularization. In the second approach, the problem is formulated as the recovery of sparse signals after a suitable sparse transform. The regularization-based method is less complex than the transform-based one. When the alphabet size @math equals 2 and @math grows proportionally, the conditions under which the signal will be recovered with high probability are the same for the two methods. When @math , the behavior of the transform-based method is established. Experimental results support this theoretical result and show that the transform method outperforms the regularization-based one. Let @math be an arbitrary polynomial of degree @math with @math non-zero integer coefficients of absolute value less than @math . In this paper, we answer the open question whether the real roots of @math can be computed with a number of arithmetic operations over the rational numbers that is polynomial in the input size of the sparse representation of @math . More precisely, we give a deterministic, complete, and certified algorithm that determines isolating intervals for all real roots of @math with @math many exact arithmetic operations over the rational numbers. When using approximate but certified arithmetic, the bit complexity of our algorithm is bounded by @math , where @math means that we ignore logarithmic. Hence, for sufficiently sparse polynomials (i.e. @math for a positive constant @math ), the bit complexity is @math . We also prove that the latter bound is optimal up to logarithmic factors. We develop fast and memory efficient numerical methods for learning functions of many variables that admit sparse representations in terms of general bounded orthonormal tensor product bases. Such functions appear in many applications including, e.g., various Uncertainty Quantification(UQ) problems involving the solution of parametric PDE that are approximately sparse in Chebyshev or Legendre product bases. We expect that our results provide a starting point for a new line of research on sublinear-time solution techniques for UQ applications of the type above which will eventually be able to scale to significantly higher-dimensional problems than what are currently computationally feasible. More concretely, let @math be a finite Bounded Orthonormal Product Basis (BOPB) of cardinality @math . We will develop methods that approximate any function @math that is sparse in the BOPB, that is, @math of the form @math with @math of cardinality @math . Our method has a runtime of just @math , uses only @math function evaluations on a fixed and nonadaptive grid, and not more than @math bits of memory. For @math , the runtime @math will be less than what is required to simply enumerate the elements of the basis @math ; thus our method is the first approach applicable in a general BOPB framework that falls into the class referred to as "sublinear-time". This and the similarly reduced sample and memory requirements set our algorithm apart from previous works based on standard compressive sensing algorithms such as basis pursuit which typically store and utilize full intermediate basis representations of size @math . We describe a parallel iterative least squares solver named LSRN that is based on random normal projection. LSRN computes the min-length solution to @math , where @math with @math or @math , and where @math may be rank-deficient. Tikhonov regularization may also be included. Since @math is only involved in matrix-matrix and matrix-vector multiplications, it can be a dense or sparse matrix or a linear operator, and LSRN automatically speeds up when @math is sparse or a fast linear operator. The preconditioning phase consists of a random normal projection, which is embarrassingly parallel, and a singular value decomposition of size @math , where @math is moderately larger than 1, e.g., @math . We prove that the preconditioned system is well-conditioned, with a strong concentration result on the extreme singular values, and hence that the number of iterations is fully predictable when we apply LSQR or the Chebyshev semi-iterative method. As we demonstrate, the Chebyshev method is particularly efficient for solving large problems on clusters with high communication cost. Numerical results demonstrate that on a shared-memory machine, LSRN outperforms LAPACK's DGELSD on large dense problems, and MATLAB's backslash (SuiteSparseQR) on sparse problems. Further experiments demonstrate that LSRN scales well on an Amazon Elastic Compute Cloud cluster. In this paper we consider a system of quadratic equations @math , where @math is unknown while normal random vectors @math and quadratic measurements @math are known. The system is assumed to be underdetermined, i.e., @math . We prove that if there exists a sparse solution @math , i.e., at most @math components of @math are nonzero, then by solving a convex optimization program, we can solve for @math up to a multiplicative constant with high probability, provided that @math . On the other hand, we prove that @math is necessary for a class of natural convex relaxations to be exact. Recovering a low-rank tensor from incomplete information is a recurring problem in signal processing and machine learning. The most popular convex relaxation of this problem minimizes the sum of the nuclear norms of the unfoldings of the tensor. We show that this approach can be substantially suboptimal: reliably recovering a @math -way tensor of length @math and Tucker rank @math from Gaussian measurements requires @math observations. In contrast, a certain (intractable) nonconvex formulation needs only @math observations. We introduce a very simple, new convex relaxation, which partially bridges this gap. Our new formulation succeeds with @math observations. While these results pertain to Gaussian measurements, simulations strongly suggest that the new norm also outperforms the sum of nuclear norms for tensor completion from a random subset of entries. Our lower bound for the sum-of-nuclear-norms model follows from a new result on recovering signals with multiple sparse structures (e.g. sparse, low rank), which perhaps surprisingly demonstrates the significant suboptimality of the commonly used recovery approach via minimizing the sum of individual sparsity inducing norms (e.g. @math , nuclear norm). Our new formulation for low-rank tensor recovery however opens the possibility in reducing the sample complexity by exploiting several structures jointly. In this paper, motivated by network inference and tomography applications, we study the problem of compressive sensing for sparse signal vectors over graphs. In particular, we are interested in recovering sparse vectors representing the properties of the edges from a graph. Unlike existing compressive sensing results, the collective additive measurements we are allowed to take must follow connected paths over the underlying graph. For a sufficiently connected graph with @math nodes, it is shown that, using @math path measurements, we are able to recover any @math -sparse link vector (with no more than @math nonzero elements), even though the measurements have to follow the graph path constraints. We further show that the computationally efficient @math minimization can provide theoretical guarantees for inferring such @math -sparse vectors with @math path measurements from the graph. This paper studies sparse spikes deconvolution over the space of measures. We focus on the recovery properties of the support of the measure (i.e., the location of the Dirac masses) using total variation of measures (TV) regularization. This regularization is the natural extension of the @math l1 norm of vectors to the setting of measures. We show that support identification is governed by a specific solution of the dual problem (a so-called dual certificate) having minimum @math L2 norm. Our main result shows that if this certificate is non-degenerate (see the definition below), when the signal-to-noise ratio is large enough TV regularization recovers the exact same number of Diracs. We show that both the locations and the amplitudes of these Diracs converge toward those of the input measure when the noise drops to zero. Moreover, the non-degeneracy of this certificate can be checked by computing a so-called vanishing derivative pre-certificate. This proxy can be computed in closed form by solving a linear system. Lastly, we draw connections between the support of the recovered measure on a continuous domain and on a discretized grid. We show that when the signal-to-noise level is large enough, and provided the aforementioned dual certificate is non-degenerate, the solution of the discretized problem is supported on pairs of Diracs which are neighbors of the Diracs of the input measure. This gives a precise description of the convergence of the solution of the discretized problem toward the solution of the continuous grid-free problem, as the grid size tends to zero.