How to integrate $\int(x\pi-x^2)\cos(kx)dx$ My question is can I solve by integrating by parts if I do $u = (x\pi-x^2)$,or first I have to do $∫[x\pi \cos(kx)-x^2\cos(kx)]dx$ and then split it in two integrals $\int x\pi\cos(kx)dx-\int x^2\cos(kx)dx$ ?

Here is an approach that does not use integration by parts.  We observe that 
$$\frac{d^2}{dk^2}\int \cos (kx)\,dx=-\int x^2\cos (kx)\,dx$$
and
$$\frac{d}{dk}\int \sin (kx)\,dx=\int x\cos (kx)\,dx$$
Therefore,
$$\int x^2\cos (kx)\,dx=-\frac{d^2}{dk^2}\left(\frac{\sin kx}{k}\right)=\frac{x^2\sin kx}{k}+\frac{2x\cos kx}{k^2}-\frac{2\sin kx}{k^3}$$
and
$$\int x\cos (kx)\,dx=-\frac{d}{dk}\left(\frac{\cos kx}{k}\right)=\frac{x\sin kx}{k}+\frac{x\sin kx}{k^2}$$
