The extended Kalman filter (EKF) is a classical signal processing algorithm which performs efficient approximate Bayesian inference in non-conjugate models by linearising the local measurement function, avoiding the need to compute intractable integrals when calculating the posterior. In some cases the EKF outperforms methods which rely on cubature to solve such integrals, especially in time-critical real-world problems. The drawback of the EKF is its local nature, whereas state-of-the-art methods such as variational inference or expectation propagation (EP) are considered global approximations. We formulate power EP as a nonlinear Kalman filter, before showing that linearisation results in a globally iterated algorithm that exactly matches the EKF on the first pass through the data, and iteratively improves the linearisation on subsequent passes. An additional benefit is the ability to calculate the limit as the EP power tends to zero, which removes the instability of the EP-like algorithm. The resulting inference scheme solves non-conjugate temporal Gaussian process models in linear time, $\mathcal{O}(n)$, and in closed form. Temporal Gaussian process (GP, Rasmussen and Williams, 2006 ) models can be solved in linear computational scaling, O(n), in the number of data n (Hartikainen and Särkkä, 2010) . However, non-conjugate (i.e., non-Gaussian likelihood) GP models introduce a computational problem in that they generally involve approximating intractable integrals in order to update the posterior distribution when data is observed. The most common numerical method used in such scenarios is sigma-point integration (Kokkala et al., 2016) , with Gauss-Hermite cubature being a popular way to choose the sigma-point locations and weights. A drawback of this method is that the number of cubature points scales exponentially with the dimensionality d. Lower-order sigma-point methods allow accuracy to be traded off for scalability, for example the unscented transform (which forms the basis for the unscented Kalman filter, see Särkkä, 2013) requires only 2d + 1 cubature points. One significant alternative to cubature methods is linearisation. Although such an approach has gone out of fashion lately, García-Fernández et al. (2015) showed that a globally iterated version of the statistically linearised filter (SLF, Särkkä, 2013) , which performs linearisation w.r.t. the posterior rather than the prior, performs in line with expectation propagation (EP, Minka, 2001 ) in many modelling scenarios, whilst also providing local convergence guarantees (Appendix D explains the connection to our proposed method). Crucially, linearisation guarantees that the integrals required to calculate the posterior have a closed form solution, which results in significant computational savings if d is large. Motivated by these observations, and with the aim of illustrating the connections between classical filtering methods and EP, we formulate power EP (PEP, Minka, 2004) as a Gaussian filter parametrised by a set of local likelihood approximations. The linearisations used to calculate these approximations are then refined during multiple passes through the data. We show that a single iteration of our approach is identical to the extended Kalman filter (EKF, Jazwinski, 1970) , and furthermore that we are able to calculate exactly the limit as the EP power tends to zero, since there are no longer any intractable integrals that depend on the power. The result is a global approximate inference algorithm for temporal GPs that is efficient and stable, easy to implement, scales to problems with large data and high-dimensional latent states, and consistently outperforms the EKF. In Fig. 2 , we compare our approach (EKF-PEP, α = 1) to EP and the EKF on two nonconjugate GP tasks (see Appendix E for the full formulations). Whilst our method is suited to large datasets, we focus here on small time series for ease of comparison. In the left-hand plot, a log-Gaussian Cox process (approximated with a Poisson model for 200 equal time interval bins) is used to model the intensity of coal mining accidents. EKF-PEP and the EKF match the EP posterior well, with EKF-PEP obtaining an even tighter match to both the mean and marginal variances. The right-hand plot shows a similar comparison for 133 accelerometer readings in a simulated motorcycle crash, using a heteroscedastic noise model. Linearisation in this model is a crude approximation to the true likelihood, but we observe that iteratively refining the linearisation vastly improves the posterior is some regions. This new perspective on linearisation in approximate inference unifies the PEP and EKF paradigms for temporal data, and provides an improvement to the EKF that requires no additional implementation effort. Key areas for further exploration are the effect of adjusting α (i.e., changing the cavity and the linearisation point), and the use of statistical linearisation as an alternative method for obtaining the local approximations. Appendix A. The proposed globally iterated EKF-PEP algorithm Algorithm 1 Globally iterated extended Kalman filter with power EP-style updates and discretised state space model h, H, J x , J r , α measurement model, Jacobian and EP power m 0 ← 0, P 0 ← P ∞ , e 1:n = 0 initial state while not converged do iterated EP-style loop for k = 1 to n do forward pass (FILTERING) evaluate Jacobian
We unify the extended Kalman filter (EKF) and the state space approach to power expectation propagation (PEP) by solving the intractable moment matching integrals in PEP via linearisation. This leads to a globally iterated extension of the EKF.
{"src": "scitldr-AIC_train", "paper_id": "HkxNKk2VKS"}