Taking expectation corresponds to integrating out Gaussian distributed \(\mathbf {z}_{k,i}\) . The conditional expectation of the first term is easily determined since it linearly depends on \(\mathbf {z}_{k,i}\) . However, the expectation of the second term, which requires integrating \(\mathbf {z}_{k,i}\)  over the \(\text{lse}\)  function, is intractable to compute in a closed form. To overcome this issue, we perform quadratic surrogate optimization transfer, in which a quadratic approximation to the \(\text{lse}\)  function [1]} is applied. This results in an upper bound on the multinomial log likelihood. This approximation uses the second order Taylor series expansion with a fixed Hessian matrix.
Particularly, the quadratic upper bound takes the following form:
\( \text{lse}(\Theta _{kl} \mathbf {z}_{k,i}) \le \frac{1}{2} \mathbf {z}_{k,i}^T \Theta _{kl}^T \mathbf {A}_{l} \Theta _{kl} \mathbf {z}_{k,i} - \mathbf {b}_{kl,i}^T \Theta _{kl} \mathbf {z}_{k,i} + \mathbf {c}_{kl,i},\) 
