"Marginalizing out" a parameter in a PDF Say I have a variable $x$ whose PDF is a von Mises distribution with a single parameter, $\kappa$, that determines the distribution's concentration: $VM(x; \kappa)$.

But suppose $\kappa$ itself is not fixed, but drawn from a Gamma distribution: $G(\kappa; \bar{\kappa}, \tau)$, where the mean is $\bar{\kappa}$ and the variance is $\bar{\kappa}\tau$.

An author whose work I'm replicating has this to say about this situation: "The predictions [for each $x$ value's probability] have to be averaged over all possible values of precision $\kappa$ (mathematically: we need to 'marginalize out $\kappa$').  We do this by discretizing the (gamma) distribution over $\kappa$ into 50 bins with equal probability masses, computing the model [von Mises] prediction at each bin center, and then averaging the predictions.  This solution is accurate as long as the number of bins is not too small (in the limit of an infinite amount of bins, the solution is equal to an analytical marginalization)" (emphasis mine).

I've read elsewhere that probabilities should be summed, not averaged, in marginalization.  Another confusion I have regards this author's statement that the average should be "over all possible values of $\kappa$."  This seems to imply that each possible value of $\kappa$ should be treated equally -- but then we're talking about a uniform distribution, not a Gamma one.  Clearly, the author was using language loosely.

Can someone shed some light on this, maybe with language that is more precise?
Suppose you  have:
\begin{align}
\Pr(X = 0 \mid k=0) &amp; = 2/3 &amp; &amp; &amp; \Pr(X=1\mid k=0) &amp; = 1/3 \\[10pt]
\Pr(X = 0 \mid k=1) &amp; = 1/2 &amp; &amp; &amp; \Pr(X=1\mid k=1) &amp; = 1/2
\end{align}
If you want the marginal distribution of $X$, you average the two rows above, with weight equal to $\Pr(k=0)$ for the first row and to $\Pr(k=1)$ for the second row.  Let's suppose those weights are each $1/2$.  Then the joint distribution is:
\begin{align}
\Pr(X=0\ \&amp;\ k=0) &amp; = 1/3 &amp; &amp; &amp; \Pr(X=1\ \&amp;\ k=0) &amp; 1/6 \\[10pt]
\Pr(X=0\ \&amp;\ k=1) &amp; = 1/4 &amp; &amp; &amp; \Pr(X=1\ \&amp;\ k=1) &amp; 1/4
\end{align}
Then rather than averaging, you sum, to get
\begin{align}
\Pr(X=0) = &amp; \frac 7 {12}, &amp; &amp; &amp; \Pr(X=1) &amp; = \frac 5 {12}.
\end{align}

In one case you have a conditional distribution given $k$, and you find an average; in the other case you have a joint distribution, and you find the sum.  Either way you get the same marginal distribution.

In the situation you describe, the von Mises distribution is a conditional distribution given the value of $\kappa$.

The average should be a weighted average with weights given by the distribution of $\kappa$.
You have a conditional density distribution: $f_{X\mid K}(x\mid k) = VM(x;\kappa)$

By "analytic marginalisation" the author means:

$$\begin{align}f_X(x) ~=~ &amp; \mathsf E\big(f_{X\mid K}(x\mid K)\big) \\[1ex] ~=~ &amp; \int_0^\infty f_{X\mid K}(x\mid \kappa)~f_K(\kappa) ~\operatorname d \kappa\end{align}$$

However the author claims that as long as the number of bins is not too small, this can approximated by "averaging out".