Show that equation $\det(A+xB)=0$ has real solutions if and only if $\det(A^{2}+B^{2})\geq(\det(A)+\det(B))^{2}$ We have $A,B$ two $2×2$ matrices with real values and we know $\det(AB-BA)=0$. Show that equation $\det(A+xB)=0$ has real solutions if and only if $$\det(A^{2}+B^{2})\geq(\det(A)+\det(B))^{2}.$$
I used the formula:
$$\det(A+xB)=\det(A)+x^{2}\det(B)+x(\det(A+B)-\det(A)-\det(B)).$$ To have real solutions for $x$, we need to have: $$(\det(A+B)-(\det(A)+\det(B)))^{2}\geq4\det(A)\det(B).$$
Now I dont know how to use the fact that $\det(AB-BA)=0$ Is there any formula to rewrite $\det(AB-BA)$ in terms of $\det(A+B)$, $\det(A)$, $\det(B)$?

As I have mentioned in my comment under your question, the statement is false, e.g. when $A=I$ and $B=0$. It can be salvaged by assuming that $\det(B)\ne0$.
The other answer has probably explained where this question came from. Yet, being an Olympiad-typed question, I guess it was intended to be solved by using more familiar identities for real $2\times2$ matrices. Here I think we need the following two:

*

*$\det(X+Y)=\det(X)+\det(Y)+\operatorname{tr}\left(Y\operatorname{adj}(X)\right)$;

*$|\det(X+iY)|^2=\det(X^2+Y^2)-\det(XY-YX)$.

(While the quantity $\operatorname{tr}\left(Y\operatorname{adj}(X)\right)$ in the first identity does not appear symmetric at the outset, with entrywise computation one can easily verify that $\operatorname{tr}\left(X\operatorname{adj}(Y)\right)=\operatorname{tr}\left(Y\operatorname{adj}(X)\right)$.)
Now let $a=\det(A),b=\det(B)$ and $t=\operatorname{tr}\big(B\operatorname{adj}(A)\big)$. Using the two identities above with the condition that $\det(AB-BA)=0$, we obtain
$$
\begin{cases}
\det(A+xB)=bx^2+tx+a,\\
\det(A^2+B^2)= |\det(A+iB)|^2=|-b+it+a|^2=(a-b)^2+t^2.
\end{cases}
$$
So, essentially, you are asked to show that $bx^2+tx+a=0$ has a real root if and only if $(a-b)^2+t^2\ge(a+b)^2$. But this is true when $b\ne0$ because both propositions are equivalent to $t^2\ge4ab$.
