We study a classical iterative algorithm for balancing matrices in the @math norm via a scaling transformation. This algorithm, which goes back to Osborne and Parlett & Reinsch in the 1960s, is implemented as a standard preconditioner in many numerical linear algebra packages. Surprisingly, despite its widespread use over several decades, no bounds were known on its rate of convergence. In this paper we prove that, for any irreducible @math (real or complex) input matrix @math , a natural variant of the algorithm converges in @math elementary balancing operations, where @math measures the initial imbalance of @math and @math is the target imbalance of the output matrix. (The imbalance of @math is @math , where @math are the maximum entries in magnitude in the @math th row and column respectively.) This bound is tight up to the @math factor. A balancing operation scales the @math th row and column so that their maximum entries are equal, and requires @math arithmetic operations on average, where @math is the number of non-zero elements in @math . Thus the running time of the iterative algorithm is @math . This is the first time bound of any kind on any variant of the Osborne-Parlett-Reinsch algorithm. We also prove a conjecture of Chen that characterizes those matrices for which the limit of the balancing process is independent of the order in which balancing operations are performed.
A different notion of matrix balancing is often known as Sinkhorn balancing after Sinkhorn @cite_18 , who proposed a natural iterative algorithm analogous to that of Osborne-Parlett-Reinsch. The goal here is to find a scaling matrix @math such that @math has prescribed row and column sums. A polynomial time algorithm for this problem was given by Kalantari and Khachian @cite_11 , while the convergence rate of Sinkhorn's iterative algorithm was studied by several authors @cite_1 @cite_2 @cite_17 @cite_16 . In particular Linial, Samorodnitsky and Wigderson @cite_17 gave strongly polynomial bounds (after a non-Sinkhorn preprocessing step) and derived a surprising approximation scheme for the permanent of a non-negative matrix.
@cite_18 @cite_11 @cite_1 @cite_2 @cite_16 @cite_17
2099811697 2037552299 2605569246 2045361520
Updating probabilistic belief matrices as new observations arrive, in the presence of noise, is a critical part of many algorithms for target tracking in sensor networks. These updates have to be carried out while preserving sum constraints, arising for example, from probabilities. This paper addresses the problem of updating belief matrices to satisfy sum constraints using scaling algorithms. We show that the convergence behavior of the Sinkhorn scaling process, used for scaling belief matrices, can vary dramatically depending on whether the prior unscaled matrix is exactly scalable or only almost scalable. We give an efficient polynomial-time algorithm based on the maximum-flow algorithm that determines whether a given matrix is exactly scalable, thus determining the convergence properties of the Sinkhorn scaling process. We prove that the Sinkhorn scaling process always provides a solution to the problem of minimizing the Kullback-Leibler distance of the physically feasible scaled matrix from the prior constraint-violating matrix, even when the matrices are not exactly scalable. We pose the scaling process as a linearly constrained convex optimization problem, and solve it using an interior-point method. We prove that even in cases in which the matrices are not exactly scalable, the problem can be solved to e-optimality in strongly polynomial time, improving the best known bound for the problem of scaling arbitrary nonnegative rectangular matrices to prescribed row and column sums. The Sudoku puzzle is a discrete constraint satisfaction problem, as is the error correction decoding problem. We propose here an algorithm for solution to the Sinkhorn puzzle based on Sinkhorn balancing. Sinkhorn balancing is an algorithm for projecting a matrix onto the space of doubly stochastic matrices. The Sinkhorn balancing solver is capable of solving all but the most difficult puzzles. A proof of convergence is presented, with some information theoretic connections. A random generalization of the Sudoku puzzle is presented, for which the Sinkhorn-based solver is also very effective. In this paper, we study matrix scaling and balancing, which are fundamental problems in scientific computing, with a long line of work on them that dates back to the 1960s. We provide algorithms for both these problems that, ignoring logarithmic factors involving the dimension of the input matrix and the size of its entries, both run in time @math where @math is the amount of error we are willing to tolerate. Here, @math represents the ratio between the largest and the smallest entries of the optimal scalings. This implies that our algorithms run in nearly-linear time whenever @math is quasi-polynomial, which includes, in particular, the case of strictly positive matrices. We complement our results by providing a separate algorithm that uses an interior-point method and runs in time @math . In order to establish these results, we develop a new second-order optimization framework that enables us to treat both problems in a unified and principled manner. This framework identifies a certain generalization of linear system solving that we can use to efficiently minimize a broad class of functions, which we call second-order robust. We then show that in the context of the specific functions capturing matrix scaling and balancing, we can leverage and generalize the work on Laplacian system solving to make the algorithms obtained via this framework very efficient. Abstract Elementary proofs are given for theorems of Bapat and Raghavan on the scaling of nonnegative multidimensional matrices. Theorems of Sinkhorn and of Brualdi, Parter, and Schneider are derived as corollaries. For positive two-dimensional matrices, Hilbert's projective metric and a theorem of G. Birkhoff are used to prove that Sinkhorn's original iterative procedure converges geometrically; the ratio of convergence is estimated from the given data.