What is $\lim_{x\to\infty} xe^{-x^2}$? 
What is $\lim_{x\to\infty} xe^{-x^2}$?

I am calculating $\int_0^\infty x^2e^{-x^2}\,dx$. By integration by parts, I have
$$I = -\frac{1}{2}xe^{-x^2} |_{0}^\infty+\frac{1}{2}\int_0^\infty e^{-x^2}\,dx$$
The second integration is just $\frac{\sqrt{\pi}}{2}$. Now I want to know how to calculate $\lim_{x\to \infty} xe^{-x^2}$. It is in the form of $\infty \cdot 0$.

For the limit itself, it's useful to remember that exponentials will always win out over polynomials. Even for something absurd like:
$$\lim_{x\to\infty} x^{100,000}e^{-x} = 0$$
Based off of the nature of your question, I'm guessing you are currently in an introductory calculus course. You'll later see that you can express:
$e^x = 1 + x + \frac{x^2}{2!}+\frac{x^3}{3!}+...$
which I think makes it a bit easier to see why exponentials always dominate polynomials. 
As for the direct calculation of the integral, there's a neat trick that might come in handy someday. Consider instead the integral:
$\int_{0}^{\infty}x^2e^{-ax^2}dx$
This can be rewritten as:
$-\frac{d}{da}\int_{0}^{\infty}e^{-ax^2}dx$
The integral is a standard Gaussian now, so if we remember that:
$\int_{0}^{\infty}e^{-ax^2}dx = \frac{\sqrt{\pi}}{2\sqrt{a}}$
Thus,
$\int_{0}^{\infty}x^2e^{-ax^2}dx = -\frac{d}{da}\frac{\sqrt{\pi}}{2\sqrt{a}} = \frac{\pi}{4a^{3/2}}$
and setting $a = 1$ as we have in our problem yields the answer $\frac{\sqrt{\pi}}{4}$.
