The Paulsen problem is a basic open problem in operator theory: Given vectors @math that are @math -nearly satisfying the Parseval's condition and the equal norm condition, is it close to a set of vectors @math that exactly satisfy the Parseval's condition and the equal norm condition? Given @math , the squared distance (to the set of exact solutions) is defined as @math where the infimum is over the set of exact solutions. Previous results show that the squared distance of any @math -nearly solution is at most @math and there are @math -nearly solutions with squared distance at least @math . The fundamental open question is whether the squared distance can be independent of the number of vectors @math . We answer this question affirmatively by proving that the squared distance of any @math -nearly solution is @math . Our approach is based on a continuous version of the operator scaling algorithm and consists of two parts. First, we define a dynamical system based on operator scaling and use it to prove that the squared distance of any @math -nearly solution is @math . Then, we show that by randomly perturbing the input vectors, the dynamical system will converge faster and the squared distance of an @math -nearly solution is @math when @math is large enough and @math is small enough. To analyze the convergence of the dynamical system, we develop some new techniques in lower bounding the operator capacity, a concept introduced by Gurvits to analyze the operator scaling algorithm.
Matrix scaling @cite_28 is a well-studied special case of operator scaling. It has applications in numerical analysis, in approximating permanents @cite_9 and in combinatorial geometry @cite_19 . Very recently, much faster algorithms are developed for matrix scaling by two independent research groups @cite_18 @cite_15 . Cohen, Madry, Tsipras and Vladu @cite_18 obtain an algorithm for matrix scaling with running time @math , where @math is the number of nonzeros in the input matrix, @math is the ratio between the largest and the smallest entries in the optimal scaling solution, and @math is the error parameter of the output. Note that the algorithm is near linear time when @math is bounded by a polynomial in @math , but in general it could be exponentially large. Not much is known about upper bounding @math for specific instances, except when the input matrix is strictly positive @cite_31 . Our techniques for smoothed analysis in provides a new way to bound @math ; see Remark . In particular, this implies that the algorithm in @cite_18 is near linear time in a pseudorandom instance as defined in Definition (not necessarily strictly positive).
@cite_18 @cite_31 @cite_28 @cite_9 @cite_19 @cite_15
2605569246 2077575254 2099811697 2089135543
In this paper, we study matrix scaling and balancing, which are fundamental problems in scientific computing, with a long line of work on them that dates back to the 1960s. We provide algorithms for both these problems that, ignoring logarithmic factors involving the dimension of the input matrix and the size of its entries, both run in time @math where @math is the amount of error we are willing to tolerate. Here, @math represents the ratio between the largest and the smallest entries of the optimal scalings. This implies that our algorithms run in nearly-linear time whenever @math is quasi-polynomial, which includes, in particular, the case of strictly positive matrices. We complement our results by providing a separate algorithm that uses an interior-point method and runs in time @math . In order to establish these results, we develop a new second-order optimization framework that enables us to treat both problems in a unified and principled manner. This framework identifies a certain generalization of linear system solving that we can use to efficiently minimize a broad class of functions, which we call second-order robust. We then show that in the context of the specific functions capturing matrix scaling and balancing, we can leverage and generalize the work on Laplacian system solving to make the algorithms obtained via this framework very efficient. Given an n × m nonnegative matrix A = (a ij ) and positive integral vectors r e ℜ n and c e Âm having a common one-norm h, the (r,c)-scaling problem is to obtain positive diagonal matrices X and Y, if they exist, such that XAY has row and column sums equal to r and c, respectively. The entropy minimization problem corresponding to A is to find an n × m matrix z = (z ij ) having the same zero pattern as A, the sum of whose entries is a given number h, its row and column sums are within given integral vectors of lower and upper bounds, and such that the entropy function consisting of the sum of the terms z ij ln (z ij a ij ) is minimized. When the lower and upper bounds coincide, matrix scaling and entropy minimization are closely related. In this paper we present several complexity bounds for the e-approximate (r,c)-scaling problem, polynomial in n,m,h, 1 e, and ln V v, where V and v are the largest and the smallest positive entries of A, respectively. These bounds, although not polynomial in ln(1 1 e), not only provide alternative complexities for the polynomial time algorithms, but could result in better overall complexities. In particular, our theoretical analysis supports the practicality of the well-known RAS algorithm. In our analysis we obtain bounds on the norm of scaling vectors which will be used in deriving not only some of the above complexities, but also a complexity for square nonnegative matrices having positive permanent. In particular, our results extend, nontrivially, many bounds for the doubly stochastic scaling of square nonnegative matrices previously given by Kalantari and Khachiyan to the case of general (r,c)-scaling. Finally, we study a more general entropy minimization problem where row and column sums are constrained to lie in prescribed intervals, and the sum of all entries is also prescribed. Balinski and Demange described an RAS type algorithm for its continuous version, but did not analyze its complexity. We show that this algorithm produces an 1 e-approximate solution within complexity polynomial in n, m, h, V v and 1 1 e. Updating probabilistic belief matrices as new observations arrive, in the presence of noise, is a critical part of many algorithms for target tracking in sensor networks. These updates have to be carried out while preserving sum constraints, arising for example, from probabilities. This paper addresses the problem of updating belief matrices to satisfy sum constraints using scaling algorithms. We show that the convergence behavior of the Sinkhorn scaling process, used for scaling belief matrices, can vary dramatically depending on whether the prior unscaled matrix is exactly scalable or only almost scalable. We give an efficient polynomial-time algorithm based on the maximum-flow algorithm that determines whether a given matrix is exactly scalable, thus determining the convergence properties of the Sinkhorn scaling process. We prove that the Sinkhorn scaling process always provides a solution to the problem of minimizing the Kullback-Leibler distance of the physically feasible scaled matrix from the prior constraint-violating matrix, even when the matrices are not exactly scalable. We pose the scaling process as a linearly constrained convex optimization problem, and solve it using an interior-point method. We prove that even in cases in which the matrices are not exactly scalable, the problem can be solved to e-optimality in strongly polynomial time, improving the best known bound for the problem of scaling arbitrary nonnegative rectangular matrices to prescribed row and column sums. We give efficient algorithms for volume sampling, i.e., for picking @math -subsets of the rows of any given matrix with probabilities proportional to the squared volumes of the simplices defined by them and the origin (or the squared volumes of the parallelepipeds defined by these subsets of rows). In other words, we can efficiently sample @math -subsets of @math with probabilities proportional to the corresponding @math by @math principal minors of any given @math by @math positive semi definite matrix. This solves an open problem from the monograph on spectral algorithms by Kannan and Vempala (see Section @math of KV , also implicit in BDM, DRVW ). Our first algorithm for volume sampling @math -subsets of rows from an @math -by- @math matrix runs in @math arithmetic operations (where @math is the exponent of matrix multiplication) and a second variant of it for @math -approximate volume sampling runs in @math arithmetic operations, which is almost linear in the size of the input (i.e., the number of entries) for small @math . Our efficient volume sampling algorithms imply the following results for low-rank matrix approximation: (1) Given @math , in @math arithmetic operations we can find @math of its rows such that projecting onto their span gives a @math -approximation to the matrix of rank @math closest to @math under the Frobenius norm. This improves the @math -approximation of Boutsidis, Drineas and Mahoney BDM and matches the lower bound shown in DRVW . The method of conditional expectations gives a algorithm with the same complexity. The running time can be improved to @math at the cost of losing an extra @math in the approximation factor. (2) The same rows and projection as in the previous point give a @math -approximation to the matrix of rank @math closest to @math under the spectral norm. In this paper, we show an almost matching lower bound of @math , even for @math .