If $\cos^6(x)+\sin^6(x)=a+b\cos(4x)$. Find $ab$? As after solving above equation, I get $1-3\sin^2(x)\cos^2(x)$= $a+b\cos(4x)$.
Please help me to solve this problem.
Hint : What happens if you replace $x$ by $0$ or $\pi/4$?
Hint:

$$4\sin^2x\cos^2x=(2\sin x\cos x)^2=(\sin2x)^2$$

$$\cos2y=1-2\sin^2y$$