Neural networks have reached outstanding performance for solving various ill-posed inverse problems in imaging. However, drawbacks of end-to-end learning approaches in comparison to classical variational methods are the requirement of expensive retraining for even slightly different problem statements and the lack of provable error bounds during inference. Recent works tackled the first problem by using networks trained for Gaussian image denoising as generic plug-and-play regularizers in energy minimization algorithms. Even though this obtains state-of-the-art results on many tasks, heavy restrictions on the network architecture have to be made if provable convergence of the underlying fixed point iteration is a requirement. More recent work has proposed to train networks to output descent directions with respect to a given energy function with a provable guarantee of convergence to a minimizer of that energy. However, each problem and energy requires the training of a separate network.
 In this paper we consider the combination of both approaches by projecting the outputs of a plug-and-play denoising network onto the cone of descent directions to a given energy. This way, a single pre-trained network can be used for a wide variety of reconstruction tasks. Our results show improvements compared to classical energy minimization methods while still having provable convergence guarantees. In many image processing tasks an observed image f is modeled as the result of the transformation of a clean imageû under a known (linear) operator A and unknown noise ξ, f = Aû + ξ. ( In most cases, the problem of reconstructingû from f and A is ill-posed and can thus not be solved by a simple inversion of A, giving rise to the field of regularization theory with iterative or variational methods, see e.g. [2] for an overview. In recent years neural networks were very successful in learning a direct mapping G(f ) ≈û for a variety of problems such as deblurring [32, 28] , denoising [34] , super-resolution [8] , demosaicing [9] and MRI-or CT-reconstruction [33, 14] . Even though this works well in practice, there are rarely any guarantees on the behaviour of neural networks on unseen data, making them difficult to use in safety-critical applications. Moreover, for each problem and type of noise a separate network has to be trained. In contrast, classical variational methods try to find the solution by the minimization of a suitable energy function of the formû where H f is a data fidelity term, for example commonly chosen as H f (u) = 1 2 ||Au − f || 2 , and R is a regularization function that models prior knowledge about the solution, e.g. the popular total variation (TV) regularization, R(u) = ∇u 1 , [24] . While minimizers of (2) come with many desirable theoretical guarantees, regularizations like the TV often cannot perfectly capture the complex structure of the space of natural images. To combine the advantages of powerful feed-forward networks and model-based approaches like (2), authors have considered various hybrid models like learning regularizers (e.g. [23, 1, 11, 5] ), designing networks architectures that resemble the structure of minimization algorithms or differential equations, e.g. [25, 36, 15, 6] , interleaving networks with classical optimization steps [16, 17] , or using the parametrization of networks as a regularization for (2), see e.g. [29, 12] . A particularly flexible approach arises from [7, 37, 30, 13] , where proximal operators with respect to the regularizer are replaced by arbitrary denoising operators, with recent works focusing on the use of denoising networks [18, 4, 35] . While such approaches allow to tackle different inverse problems with the same neural network, the derivation of theoretical guarantees -even in terms of the convergence of the resulting algorithmic scheme -remains difficult, see [3, 27] or some discussion in [20] , unless the denoiser satisfies particular properties [22] . We combine deep learning and energy minimization methods for solving inverse problems in image reconstruction into a provably convergent algorithmic scheme. Still, our approach is able to generalize to different problems with a single denoising network and without the need to retrain if that problem changes. We were able to reach better results than the energy minimization baseline in our experiments, and are happy to elaborate on the above aspects in the NeurIPS workshop.
We use neural networks trained for image denoising as plug-and-play priors in energy minimization algorithms for image reconstruction problems with provable convergence.
{"src": "scitldr-AIC_train", "paper_id": "SJxRjQncLH"}