Determining angle for rotation of conics I am working on rotation of conic sections and I'm having trouble determining the angle of rotation from the coefficients of the general conic equation. I'm given $$11x^2-24xy+4y^2+20=0$$ From this equation I know that $$cot(2\theta)= \frac{A-C}B=\frac{11-4}{-24}=\frac{7}{-24}$$ But when I draw a triangle with angle $2\theta$ to find $cos(2\theta)$, I end up with $$cos(2\theta)=\frac{7}{25}$$ Using this value for $cos(2\theta)$ with the half-angle formulas results in an incorrect solution, which does not eliminate the original $xy$-term. However, if I rewrite $$cot(2\theta)=\frac{-7}{24}$$ and draw a new triangle with angle $2\theta$, I end up with $$cos(2\theta)=\frac{-7}{25}$$ which yields the solution given in the back of the book after substitution and simplification. I keep encountering problems of this sort and am having trouble understanding why changing the sign of $$cot(2\theta)=\frac{7}{-24}$$ is allowed. I'm sure I am missing something with respect to the trig functions. I would be grateful for any clarification.

The matrix $\begin{bmatrix}\cos(\theta)&-\sin(\theta)\\\sin(\theta)&\cos(\theta)\end{bmatrix}$ rotates $\begin{bmatrix}x&y\end{bmatrix}$ clockwise.
Therefore,
$$
\begin{align}
&\begin{bmatrix}x&y\end{bmatrix}\begin{bmatrix}\cos(\theta)&-\sin(\theta)\\\sin(\theta)&\cos(\theta)\end{bmatrix}\begin{bmatrix}a&0\\0&b\end{bmatrix}\begin{bmatrix}\cos(\theta)&\sin(\theta)\\-\sin(\theta)&\cos(\theta)\end{bmatrix}\begin{bmatrix}x\\y\end{bmatrix}\\[6pt]
&=\begin{bmatrix}x&y\end{bmatrix}\begin{bmatrix}\color{#C00000}{a\cos^2(\theta)+b\sin^2(\theta)}&\color{#00A000}{(a-b)\sin(\theta)\cos(\theta)}\\\color{#00A000}{(a-b)\sin(\theta)\cos(\theta)}&\color{#0000F0}{a\sin^2(\theta)+b\cos^2(\theta)}\end{bmatrix}\begin{bmatrix}x\\y\end{bmatrix}\\[6pt]
&=\begin{bmatrix}x&y\end{bmatrix}\begin{bmatrix}\color{#C00000}{11}&\color{#00A000}{-12}\\\color{#00A000}{-12}&\color{#0000F0}{4}\end{bmatrix}\begin{bmatrix}x\\y\end{bmatrix}
\end{align}
$$
says that if we rotate $\begin{bmatrix}x&y\end{bmatrix}$ clockwise by $\theta$ we get an upright conic (one whose axes are aligned with the coordinate axes).
To compute $\theta$, note that $11-4=(a-b)\cos(2\theta)$ and $-24=(a-b)\sin(2\theta)$. Therefore,
$$
\tan(2\theta)=-\frac{24}{7}=\tan(-1.287)
$$
Thus, if we rotate the graph of $11x^2-24xy+4y^2+20=0$ counterclockwise $0.6435$ radians, it will be upright.
Note that knowing $\tan(2\theta)$ only tells what $\theta$ is mod $\pi/2$. This is fine, however, since rotating an upright conic by $\pi/2$ also gives an upright conic.
