Inconsistency in two-sided hypothesis testing Suppose you have two sets of data with known population variances and want to test the null hypothesis that two means are equal, ie. $H_{0}: \mu_{1} = \mu_{2}$ against $H_{1}: \mu_{1} > \mu_{2}$. There's a certain way I want to think about it, which is the following:
\begin{align}
P(\mu_{1} > \mu_{2}) &= P(-(\mu_{1} - \mu_{2}) < 0) \\
&= P\left(\frac{\bar{x}_{1} - \bar{x}_{2} - (\mu_{1} - \mu_{2})}{\sigma_{\delta \bar{x}}}<\frac{\bar{x}_{1} - \bar{x}_{2}}{\sigma_{\delta \bar{x}}} \right) \\
&=P\left(z < \frac{\bar{x}_{1} - \bar{x}_{2}}{\sigma_{\delta \bar{x}}}  \right)
\end{align}
To me, this 'derivation' makes it perfectly clear what's actually going on. You're actually calculating the probability that $H_{1}$ is true and not just blindly looking up some $z$-score. However, now suppose that $H_{1}: \mu_{1} \neq \mu_{2}$. The problem with this is that the method I just described doesn't seem to work. If I write
$$
P(\mu_{1} \neq \mu_{2}) =  P(\mu_{1} < \mu_{2}) + P(\mu_{1} > \mu_{2})
$$
Then all that happens is $P(\mu_{1} \neq \mu_{2}) = 1$. I think I'm probably not interpreting the above equation correctly.

You have
$$
\frac{\bar X_1-\bar X_2}{\sigma_{\delta \bar X}}\sim N(0,1). 
$$
If you want to use $\bar X_1-\bar X_2$ as the test statistic, then you would reject the null hypothesis if $\bar X_1-\bar X_2>c$, for some critical value $c$.  Often one chooses $c$ by deciding how much probability of Type I error one is willing to tolerate --- a subjective economic decision.  One can then think about probability of Type II error as a function of $\mu_1-\mu_2$, and that probability would depend on the value of $c$ and on the two sample sizes.
None of this gives you a probability distribution of $\mu_1-\mu_2$.  For that you'd need a prior distribution, and then you'd have to find the posterior distribution, i.e. the conditional distribution given the two samples.
PS: A bit of back-of-the-envelope scratchwork suggests that the likelihood-ratio test statistic may be:
$$
\left.\begin{cases} \left(\dfrac{\bar X_1 - \bar X}{\sigma_2/\sqrt{n_1}}\right)^2 + \left(\dfrac{\bar X_2-\bar X}{\sigma_2/\sqrt{n_2}}\right)^2 & \text{if }\bar X_1\ge \bar X_2, \\[10pt]
{}\qquad 0 & \text{otherwise},
\end{cases}\right\}
$$
where $\bar X$ is simply the grand mean $\dfrac{n_1 \bar X_1 + n_2 \bar X_2}{n_1+n_2}$.
