let $f(x)$ be rational non constant polynomial and $f\circ f(x)=3f(x)^4-1$ then find the $f(x)$ let $f(x)$ be rational non constant polynomial and $f\circ f(x)=3f(x)^4-1$
then find the $f(x)$ .
My Try :
$$f(f(x))=3f(x)^4-1$$
Let $f(x)=ax^n+g(x)$ so :
$$a(ax^n+g(x))^n+g(ax^n+g(x))=3(ax^n+g(x))^4-1$$
$$a^2x^{n^2}+h(x)+k(x)=3ax^{4n}+l(x)-1$$
$$3ax^{4n}-a^2x^{n^2}-1=h(x)+k(x)-l(x)$$
Now what ?

Note that, because I think that you supposed deg $g(x)\leq n-1$, from
$$
a_n(a_n x^n+g(x))^n+g(a_n x^n+g(x))=
3(a_n x^n+g(x))^4-1$$
the degree of the LHS is $n^2$, while on the other hand the degree of the RHS is $4n$. This is an equality between two polynomials, hence their degree must be the same. Then $n=4$. Moreover the two polynomials must have the same leading coefficient. The coefficient of $x^{16}$ in the LHS is $a_4^{5}$, while the coefficient in the RHS is $3a_4^{4}$, hence $a_4=3$.
Now I think that you have to put $f(x)=3x^4+\sum_{i=0}^{3} a_i x^i $ in the equation and compute
