Is the space of continuous (may be unbounded) functions complete? I know that the space of continuous bounded function is complete in the sup norm.
However, I am confused that if $X\subset R$ is not compact, then is the space of continuous function on $X$, denoting by $C(X)$ still complete? 

The issue is not whether the norm is complete but whether it is defined at all.  If $f$ is a continuous function on $X$ that is not bounded, then the sup norm of $f$ is infinite (by definition of $f$ not being bounded), so the sup norm does not even define a norm on all of $C(X)$.
However, this isn't an insurmountable problem for some purposes.  You can still talk about convergence of sequences, or Cauchy sequences, or the topology induced by a "norm", even if that "norm" is sometimes infinite.  And in fact the sup "norm" on $C(X)$ is complete in the sense that every Cauchy sequence converges, even if you allow unbounded functions: the proof is exactly the same as it is when you restrict to bounded functions.
(Note, however, that the topology induced by a possibly infinite "norm" can be quite pathological compared to what you are used to in genuine Banach spaces.  For instance, scalar multiplication is not continuous on $C(X)$: if $f$ is an unbounded continuous function on $X$, then $cf$ is also unbounded for any nonzero scalar $c$, and so has infinite norm.  This means that $cf$ does not converge to $0\cdot f=0$ as $c$ approaches $0$!)
