Let $a,b \in \mathbb R$ and $f(x)=a\cos x+b\cos3x$. Prove that $|b|\le 1$. Let $a,b \in \mathbb R$ and $f(x)=a\cos x+b\cos3x$. It is known that $f(x)>1$ has no real solutions. Prove that $|b|\le 1$. We can write the given equality as 
\begin{align}
&f(x)=(a-3b)\cos x+4b\cos^3x\\[2ex]
\Rightarrow\quad&f'(x)=(3b-a)\sin x-12b\cos^2x\sin x=0\\[1ex]
\Rightarrow\quad&\sin x=0,\quad \cos^2x=\frac {3b-a}{12b}
\end{align}
Therefore $f(x)$ has minima or maxima when $\sin x=0$, so $\cos x=\pm 1$ or when $\cos x=\pm\sqrt{\frac {3b-a}{12b}}$. Hence $f(x)=\pm(a+b)$ or $\pm\frac 23(3b-a)$. 
Now what should I do?

By plugging in for $x$, we obtain:
$$f(0)=b+a,\qquad f\left(\frac{2\pi}{3}\right)=b-\frac{a}{2}$$
Since $f$ is continuous, it takes all values between $b+a$ and $b-a/2$ (regardless of which is larger/smaller). In particular, it takes on the value $b$. Since $f(x+\pi)=-f(x)$, it follows that $f$ also takes on the value $-b$. Since $f(x)>1$ has no real solutions, it follows that neither $b$ nor $-b$ exceed 1, i.e. $|b|\leq 1$.
