Are all functions that have an inverse bijective functions? To have an inverse, a function must be injective i.e one-one.
Now, I believe the function must be surjective i.e. onto, to have an inverse, since if it is not surjective, the function's inverse's domain will have some elements left out which are not mapped to any element in the range of the function's inverse. 
So is it true that all functions that have an inverse must be bijective? 
Thank you.

Yes. A function is invertible if and only if the function is bijective. 
For a pairing between X and Y (where Y need not be different from X) to be a bijection, four properties must hold:
(1) Each element of X must be paired with at least one element of Y,
(2) No element of X may be paired with more than one element of Y,
(3) Each element of Y must be paired with at least one element of X, and
(4) No element of Y may be paired with more than one element of X.

A bijection f with domain X (indicated by f: X → Y in functional notation) also defines a relation starting in Y and going to X. Properties (3) and (4) of a bijection say that this inverse relation is a function with domain Y. Moreover, properties (1) and (2) then say that this inverse function is a surjection and an injection, that is, the inverse function exists and is also a bijection. 
The 'counterexample' given in the other answer, i.e. $f: X \to Y$ via $f(x) = \frac{1}{x}$ which maps $\mathbb{R} - \{0\} \to \mathbb{R} - \{0\}$ is actually bijective.
Let $f(x_1) = f(x_2) \implies \frac{1}{x_1} = \frac{1}{x_2}$, then it follows that $x_1 = x_2$, so f is injective.
Let $x = \frac{1}{y}$. Then, $\forall \ y \in Y, f(x) = \frac{1}{\frac{1}{y}} = y$. So f is surjective.
