Need Help With Calc Homework Question Suppose that $x^*$ means $5x^2 - x$. Then what does $(y+3)^*$ mean?

By 5x2 do you mean $5x^2$? In such a case, simply replace every instance of $x$ with $(y+3)$, and then expand.
For example, $f(x) = 2x+2 \Longrightarrow f(y+3) = 2(y+3)+2 = 2y+6+2 = 2y+8$.
