How to calculate the angle between three points? If I have a circle with the center O(o1, o2), and three points A(x1, y1), B(x2, y2) and C(x3, y3) lying on its circumference, how can I find the angle AOC? The thing is, that there are two angles and I need the one which "contains" the point B.

I can use the law of cosines but I won't be able to tell if this is the angle that I need.

Edit:

I can check if OB and AC intersect. If yes, I need the angle that is less than pi, if no, I need the other one. Would that be correct?
We can use the two-dimensional analog of cross product here:
$$\sin\left(\angle AOC\right) = \frac{\vec{OA} \times \vec{OB}}{\lvert\vec{OA}\rvert \lvert\vec{OB}\rvert}$$
which is positive if $B$ is counterclockwise from $A$ in a right-handed coordinate system, negative if clockwise: $$-1 \le \sin\left(\angle AOB\right) \le +1$$or$$-180° \le \angle AOB \le +180°$$

Let $\vec{a}$, $\vec{b}$, and $\vec{c}$ be the vectors from the center of the circle to the respective points on the circumference:
$$\begin{array}
\; \vec{a} &amp;= ( x_a, y_a ) = (x_1 - x_0, y_1 - y_0) \\
\vec{b} &amp;= ( x_b, y_b ) = (x_2 - x_0, y_2 - y_0) \\
\vec{c} &amp;= ( x_c, y_c ) = (x_3 - x_0, y_3 - y_0)
\end{array}$$
where $(x_0, y_0)$ is the center of the circle, and $\lvert\vec{a}\rvert = \lvert\vec{b}\rvert = \lvert\vec{c}\rvert = r$. Then,
$$\begin{array}
\; \sin\left(\angle AOB\right) &amp;= \frac{x_a y_b - x_b y_a}{r^2} \\
\sin\left(\angle AOC\right) &amp;= \frac{x_a y_c - x_c y_a}{r^2} \\
\sin\left(\angle BOC\right) &amp;= \frac{x_b y_c - x_c y_b}{r^2}
\end{array}$$
So, if we calculate just
$$\begin{array}
\; \lambda_{AB} = r^2 \sin(\angle AOB) = x_a y_b - x_b y_a \\
\lambda_{AC} = r^2 \sin(\angle AOC) = x_a y_c - x_c y_a \\
\lambda_{BC} = r^2 \sin(\angle BOC) = x_b y_c - x_c y_b
\end{array}$$
we can use their signs to determine if $B$ is inside $\angle AOC$ or not. (Note that if all vectors have a nonzero length, the length multiplier only affects the scale; thus, this works, even if the points are not on the circumference of a circle.)

$B$ is inside $\angle AOC$ if and only if
$$\lambda_{AC} \ge 0 \text{ and } \lambda_{AB} \ge 0 \text{ and } \lambda_{BC} \le 0$$
or
$$\lambda_{AC} \le 0 \text{ and } \lambda_{AB} \le 0 \text{ and } \lambda_{BC} \ge 0$$
Otherwise $B$ is outside $\angle AOC$.

In pseudocode:

function inside(x_a, y_a, x_b, y_b, x_c, y_b):
    ab = x_a*y_b - x_b*y_a
    ac = x_a*y_c - x_c*y_a
    bc = x_b*y_c - x_c*y_b
    return ( (ac &gt;= 0 &amp;&amp; ab &gt;= 0 &amp;&amp; bc &lt;= 0) ||
             (ac &lt;= 0 &amp;&amp; ab &lt;= 0 &amp;&amp; bc &gt;= 0) )


or

function inside(x_0, y_0, x_1, y_1, x_2, y_2, x_3, y_3)
    ab = (x_1 - x_0)*(y_2 - y_0) - (x_2 - x_0)*(y_1 - y_0)
    ac = (x_1 - x_0)*(y_3 - y_0) - (x_3 - x_0)*(y_1 - y_0)
    bc = (x_2 - x_0)*(y_3 - y_0) - (x_3 - x_0)*(y_2 - y_0)
    return ( (ac &gt;= 0 &amp;&amp; ab &gt;= 0 &amp;&amp; bc &lt;= 0) ||
             (ac &lt;= 0 &amp;&amp; ab &lt;= 0 &amp;&amp; bc &gt;= 0) )
A formula you can use is as follows. Let $\alpha$ be the angle $\angle AOC$, then
$$cos(\alpha)=\frac{x_1x_3+y_1y_3}{\surd(x_1^2+y_{1}^2) \cdot \surd(x_3^2+y_{3}^2)}$$