Are all functions that have an inverse bijective functions? To have an inverse, a function must be injective i.e one-one.

Now, I believe the function must be surjective i.e. onto, to have an inverse, since if it is not surjective, the function's inverse's domain will have some elements left out which are not mapped to any element in the range of the function's inverse. 

So is it true that all functions that have an inverse must be bijective? 

Thank you.
All the answers point to yes, but you need to be careful as what you mean by inverse (of course, mathematics always requires thinking). I will try not to get into set-theoretic issues and appeal to your intuition.

If $f : X \to Y$ is a map of sets which is injective, then for each $x \in X$, we have an element $y = f(x)$ uniquely determined by $x$, so we can define $g : Y \to X$ by sending those $y \in f(X)$ to that element $x$ for which $f(x) = y$, and the fact that $f$ is injective will show that $g$ will be well-defined ; for those $y \in Y \backslash f(X)$, just send them wherever you want (this would require this axiom of choice, but let's not worry about that). The function $g$ satisfies $g(f(x)) = g(y) = x$, so that $g \circ f$ is the identity map ; that is, $f$ admits a left inverse. 

Conversely, suppose $f$ admits a left inverse $g$, and assume $f(x_1) = f(x_2)$. Then $x_1 = g(f(x_1)) = g(f(x_2)) = x_2$, so $f$ is injective. That is, 

A function $f : X \to Y$ is injective if and only if it admits a left-inverse $g : Y \to X$ such that $g \circ f = \mathrm{id}_X$. 

Similarly, it is not hard to show that $f$ is surjective if and only if it has a right inverse, that is, a function $g : Y \to X$ with $f \circ g = \mathrm{id}_Y$. I'll let you ponder on this one. 

If you're looking for a little more fun, feel free to look at this ; it is a bit harder though, but again if you don't worry about the foundations of set theory you can still get some good intuition out of it.

Hope that helps,
The claim that every function with an inverse is bijective is false.  A simple counter-example is $f(x)=1/x$, which has an inverse but is not bijective.  $f$ is not bijective because although it is one-to-one, it is not onto (due to the number $0$ being missing from its range).

For additional correct discussion on this topic, see this duplicate question rather than the other answers on this page.