A Deep Gradient Correction Method for Iteratively Solving Linear Systems
We present a novel deep learning approach to approximate the solution of
large, sparse, symmetric, positive-definite linear systems of equations. These
systems arise from many problems in applied science, e.g., in numerical methods
for partial differential equations. Algorithms for approximating the solution
to these systems are often the bottleneck in problems that require their
solution, particularly for modern applications that require many millions of
unknowns. Indeed, numerical linear algebra techniques have been investigated
for many decades to alleviate this computational burden. Recently, data-driven
techniques have also shown promise for these problems. Motivated by the
conjugate gradients algorithm that iteratively selects search directions for
minimizing the matrix norm of the approximation error, we design an approach
that utilizes a deep neural network to accelerate convergence via data-driven
improvement of the search directions. Our method leverages a carefully chosen
convolutional network to approximate the action of the inverse of the linear
operator up to an arbitrary constant. We train the network using unsupervised
learning with a loss function equal to the $L^2$ difference between an input
and the system matrix times the network evaluation, where the unspecified
constant in the approximate inverse is accounted for. We demonstrate the
efficacy of our approach on spatially discretized Poisson equations with
millions of degrees of freedom arising in computational fluid dynamics
applications. Unlike state-of-the-art learning approaches, our algorithm is
capable of reducing the linear system residual to a given tolerance in a small
number of iterations, independent of the problem size. Moreover, our method
generalizes effectively to various systems beyond those encountered during
training.