What does the letter epsilon signify in mathematics? This letter "$\varepsilon$" is called epsilon right ? What does it signify in mathematics ?

The greek letter epsilon, written $\epsilon$ or $\varepsilon$, is just another variable, like $x$, $n$ or $T$.
Conventionally it's used to denote a small quantity, like an error, or perhaps a term which will be taken to zero in some limit.
It's possible that you are confusing it with the set membership symbol $\in$, which is something different. When you see $x\in X$ it means that $X$ is a set, and $x$ is a member of the set. For example,
$$1\in \{1,2,3\}$$
is true, but
$$4\in\{1,2,3\}$$
is false.
