How to recognize an improper integral? I think $\displaystyle \int ^{4} _0 \frac{\sin x}{x} dx$ is not an improper integral. Is this true? If not  true, then  how can one recongnise improper integrals in general?

To put it simply, an improper integral is one where there is some problem with the integrand on the interval, or where the interval is unbounded. Thus, since 
$$
\frac{\sin(x)}{x}
$$
is undefined at $x=0$, your integral is indeed improper. What you are really evaluating is the limit
$$
\lim_{a\rightarrow 0+} \int_a^4 \frac{\sin(x)}{x} dx
$$
That the undefined point is on the boundary does not matter, if you had been asked to evaluate an integral like
$$
\int_{-1}^4 \frac{\sin(x)}{x} dx
$$
you'd really be evaluating
$$
\lim_{a\rightarrow 0-} \int_{-1}^a \frac{\sin(x)}{x} dx+
\lim_{b\rightarrow 0+} \int_b^4 \frac{\sin(x)}{x} dx
$$
(note that the limits are independent; here it does not really matter as the improper integrals evaluate to finite numbers, but in other cases, it does).
Another example of an improper integral would be
$$
\int_{1}^\infty \frac{\sin(x)}{x} dx
$$
which is shorthand for
$$
\lim_{b\rightarrow \infty} \int_1^b \frac{\sin(x)}{x} dx
$$
Some improper integrals involve integrands which are unbounded; these should also be evaluated as above, with limits. Sometimes they yield something finite, sometimes they don't.
