Inverse problems are ubiquitous in natural sciences and refer to the challenging task of inferring complex and potentially multi-modal posterior distributions over hidden parameters given a set of observations. Typically, a model of the physical process in the form of differential equations is available but leads to intractable inference over its parameters. While the forward propagation of parameters through the model simulates the evolution of the system, the inverse problem of finding the parameters given the sequence of states is not unique. In this work, we propose a generalisation of the Bayesian optimisation framework to approximate inference. The resulting method learns approximations to the posterior distribution by applying Stein variational gradient descent on top of estimates from a Gaussian process model. Preliminary results demonstrate the method's performance on likelihood-free inference for reinforcement learning environments. We consider the problem of estimating parameters θ of a physical system according to observed data y. The forward model of the system is approximated by a computational model that generates dataŷ θ based on the given parameter settings θ. In many cases, the corresponding likelihood function p(ŷ θ |θ) is not available, and one resorts to likelihoodfree methods, such as approximate Bayesian computation (ABC) (Robert, 2016) , conditional density estimation (Papamakarios and Murray, 2016) , etc. For certain applications in robotics and reinforcement learning, however, the number of simulations might be limited by resource constraints, imposing challenges to current approaches. Recent methods address the problem of efficiency in the use of simulations by either constructing conditional density estimators from joint data {θ i ,ŷ i } N i=1 , using, for example, mixture density networks (Papamakarios and Murray, 2016; Ramos et al., 2019) , or by sequentially learning approximations to the likelihood function (Gutmann and Corander, 2016; Papamakarios et al., 2019) and then running Markov chain Monte Carlo (MCMC). In particular, Gutmann and Corander (2016) derive an active learning approach using Bayesian optimisation (BO) (Shahriari et al., 2016) to propose parameters for simulations. Their approach reduces the number of simulator runs from the typical thousands to a few hundreds. This paper investigates an approach to combine the flexible representative power of variational inference methods (Liu and Wang, 2016) with the data efficiency of Bayesian optimisation. We present a Thompson sampling strategy (Russo and Van Roy, 2016) to sequentially refine variational approximations to a black-box posterior. Parameters for new simulations are proposed by running Stein variational gradient descent (SVGD) (Liu and Wang, 2016) over samples from a Gaussian process (GP) (Rasmussen and Williams, 2006) . The approach is also equipped with a method to optimally subsample the variational approximations for batch evaluations of the simulator models at each round. In the following, we present the derivation of our approach and preliminary experimental results. This paper presented a Bayesian optimisation approach to inverse problems on simulator parameters. Preliminary results demonstrated the potential of the method for reinforcement learning applications. In particular, results show that distributional Bayesian optimisation is able to provide a more sample-efficient approach than other likelihood-free inference methods when inferring parameters of a classical reinforcement learning environment. Future work includes further scalability and theoretical analysis of the method. 3. OpenAI Gym: https://gym.openai.com 4. Code available at: https://github.com/rafaol/dbo-aabi2019 , instead.
An approach to combine variational inference and Bayesian optimisation to solve complicated inverse problems
{"src": "scitldr-AIC_train", "paper_id": "HJxvcJhVYS"}