A generic framework for the solution of PDE-constrained optimisation problems based on the FEniCS system is presented. Its main features are an intuitive mathematical interface, a high degree of automation, and an efficient implementation of the generated adjoint model. The framework is based upon the extension of a domain-specific language for variational problems to cleanly express complex optimisation problems in a compact, high-level syntax. For example, optimisation problems constrained by the time-dependent Navier-Stokes equations can be written in tens of lines of code. Based on this high-level representation, the framework derives the associated adjoint equations in the same domain-specific language, and uses the FEniCS code generation technology to emit parallel optimised low-level C++ code for the solution of the forward and adjoint systems. The functional and gradient information so computed is then passed to the optimisation algorithm to update the parameter values. This approach works both for steady-state as well as transient, and for linear as well as nonlinear governing PDEs and a wide range of functionals and control parameters. We demonstrate the applicability and efficiency of this approach on classical textbook optimisation problems and advanced examples.
A closely related project is developed by waanders2002 , with the goal of creating an optimisation framework based on the finite-element software Sundance @cite_1 . Sundance is similar to in that it also operates on variational forms: in particular, it can automatically differentiate and adjoin individual variational forms. However, the built-in automatic adjoint derivation of Sundance does not currently extend to cases where the forward model consists of a sequence of variational problems, which is typically the case for time-dependent problems.
@cite_1
1985302366 2607406448 1510651616 2151454335
Sundance is a package in the Trilinos suite designed to provide high-level components for the development of high-performance PDE simulators with built-in capabilities for PDE-constrained optimization. We review the implications of PDE-constrained optimization on simulator design requirements, then survey the architecture of the Sundance problem specification components. These components allow immediate extension of a forward simulator for use in an optimization context. We show examples of the use of these components to develop full-space and reduced-space codes for linear and nonlinear PDE-constrained inverse problems. We propose a partially learned approach for the solution of ill-posed inverse problems with not necessarily linear forward operators. The method builds on ideas from classical regularisation theory and recent advances in deep learning to perform learning while making use of prior information about the inverse problem encoded in the forward operator, noise model and a regularising functional. The method results in a gradient-like iterative scheme, where the 'gradient' component is learned using a convolutional network that includes the gradients of the data discrepancy and regulariser as input in each iteration. We present results of such a partially learned gradient scheme on a non-linear tomographic inversion problem with simulated data from both the Sheep-Logan phantom as well as a head CT. The outcome is compared against filtered backprojection and total variation reconstruction and the proposed method provides a 5.4 dB PSNR improvement over the total variation reconstruction while being significantly faster, giving reconstructions of pixel images in about 0.4 s using a single graphics processing unit (GPU). A variational approach to finite connectivity spin-glass-like models is developed and applied to describe the structure of optimal solutions in random satisfiability problems. Our variational scheme accurately reproduces the known replica symmetric results and also allows for the inclusion of replica symmetry breaking effects. For the 3-SAT problem, we find two transitions as the ratio α of logical clauses per Boolean variables increases. At the first one ( a_s 3.96 ), a non-trivial organization of the solution space in geometrically separated clusters emerges. The multiplicity of these clusters as well as the typical distances between different solutions are calculated. At the second threshold ( a_c 4.48 ), satisfying assignments disappear and a finite fraction ( B_o 0.13 ) of variables are overconstrained and take the same values in all optimal (though unsatisfying) assignments. These values have to be compared to ( a_c 4.27 ), ( B_o 0.4 ) obtained from numerical experiments on small instances. Within the present variational approach, the SAT-UNSAT transition naturally appears as a mixture of a first and a second order transition. For the mixed 2+p-SAT with p<2 5, the behavior is as expected much simpler: a unique smooth transition from SAT to UNSAT takes place at ( a_c = 1 (1 - p) ). We present an algorithm that infers the model structure of a mixture of factor analysers using an efficient and deterministic variational approximation to full Bayesian integration over model parameters. This procedure can automatically determine the optimal number of components and the local dimensionality of each component (i.e. the number of factors in each factor analyser). Alternatively it can be used to infer posterior distributions over number of components and dimensionalities. Since all parameters are integrated out the method is not prone to overfitting. Using a stochastic procedure for adding components it is possible to perform the variational optimisation incrementally and to avoid local maxima. Results show that the method works very well in practice and correctly infers the number and dimensionality of nontrivial synthetic examples. By importance sampling from the variational approximation we show how to obtain unbiased estimates of the true evidence, the exact predictive density, and the KL divergence between the variational posterior and the true posterior, not only in this model but for variational approximations in general.