A random matrix is likely to be well conditioned, and motivated by this well known property we employ random matrix multipliers to advance some fundamental matrix computations. This includes numerical stabilization of Gaussian elimination with no pivoting as well as block Gaussian elimination, approximation of the leading and trailing singular spaces of an ill conditioned matrix, associated with its largest and smallest singular values, respectively, and approximation of this matrix by low-rank matrices, with further extensions to computing numerical ranks and the approximation of tensor decomposition. We formally support the efficiency of the proposed techniques where we employ Gaussian random multipliers, but our extensive tests have consistently produced the same outcome where instead we used sparse and structured random multipliers, defined by much fewer random parameters compared to the number of their entries.
Preconditioning of linear systems of equations is a classical subject @cite_42 , @cite_18 , @cite_11 . Randomized multiplicative preconditioning for numerical stabilization of GENP was proposed in [Section 12.2] PGMQ and @cite_35 , but with no formal support for this approach. On low-rank approximation we refer the reader to the survey @cite_39 . We cite these and other related works throughout the paper and refer to [Section 11] PQZb on further bibliography. For a natural extension of our present work, one can combine randomized matrix multiplication with randomized augmentation and additive preprocessing of @cite_0 , @cite_7 , @cite_12 , @cite_1 , @cite_25 , @cite_36 , @cite_46 .
@cite_35 @cite_18 @cite_7 @cite_36 @cite_42 @cite_1 @cite_39 @cite_0 @cite_46 @cite_25 @cite_12 @cite_11
2025033760 2019588162 2952294252 2073656615
An algebraic multilevel (ML) preconditioner is presented for the Helmholtz equation in heterogeneous media. It is based on a multilevel incomplete @math factorization and preserves the inherent (complex) symmetry of the Helmholtz equation. The ML preconditioner incorporates two key components for efficiency and numerical stability: symmetric maximum weight matchings and an inverse-based pivoting strategy. The former increases the block-diagonal dominance of the system, whereas the latter controls @math for numerical stability. When applied recursively, their combined effect yields an algebraic coarsening strategy, similar to algebraic multigrid methods, even for highly indefinite matrices. The ML preconditioner is combined with a Krylov subspace method and applied as a “black-box” solver to a series of challenging two- and three-dimensional test problems, mainly from geophysical seismic imaging. The numerical results demonstrate the robustness and efficiency of the ML preconditioner, even at higher frequency regimes. The construction of suitable preconditioners for the solution of linear systems by iterative methods continues to receive a lot of interest. Traditionally, preconditioners are designed to accelerate convergence of iterative methods to the solution of the linear system. However, when truncated iterative methods are used as regularized solvers of ill-posed problems, the rate of convergence is seldom an issue, and traditional preconditioners are of little use. Here, we present a new approach to the design of preconditioners for ill-posed linear systems, suitable when statistical information about the desired solution or a collection of typical solutions is available. The preconditioners are constructed from the covariance matrix of the solution viewed as a random variable. Since the construction is based on available prior information, these preconditioners are called priorconditioners. A statistical truncation index selection is also presented. Computed examples illustrate how effective such priorconditioners can be. This paper describes a novel approach of packing sparse convolutional neural networks for their efficient systolic array implementations. By combining subsets of columns in the original filter matrix associated with a convolutional layer, we increase the utilization efficiency of the systolic array substantially (e.g., 4x) due to the increased density of nonzeros in the resulting packed filter matrix. In combining columns, for each row, all filter weights but one with the largest magnitude are pruned. We retrain the remaining weights to preserve high accuracy. We demonstrate that in mitigating data privacy concerns the retraining can be accomplished with only fractions of the original dataset (e.g., 10 for CIFAR-10). We study the effectiveness of this joint optimization for both high utilization and classification accuracy with ASIC and FPGA designs based on efficient bit-serial implementations of multiplier-accumulators. We present analysis and empirical evidence on the superior performance of our column combining approach against prior arts under metrics such as energy efficiency (3x) and inference latency (12x). In the present paper we analyze a class of tensor-structured preconditioners for the multidimensional second-order elliptic operators in ℝ d , d≥2. For equations in a bounded domain, the construction is based on the rank-R tensor-product approximation of the elliptic resolvent ℬ R ≈(ℒ−λ I)−1, where ℒ is the sum of univariate elliptic operators. We prove the explicit estimate on the tensor rank R that ensures the spectral equivalence. For equations in an unbounded domain, one can utilize the tensor-structured approximation of Green’s kernel for the shifted Laplacian in ℝ d , which is well developed in the case of nonoscillatory potentials. For the oscillating kernels e −i κ‖x‖ ‖x‖, x∈ℝ d , κ∈ℝ+, we give constructive proof of the rank-O(κ) separable approximation. This leads to the tensor representation for the discretized 3D Helmholtz kernel on an n×n×n grid that requires only O(κ |log e|2 n) reals for storage. Such representations can be applied to both the 3D volume and boundary calculations with sublinear cost O(n 2), even in the case κ=O(n).