What is the mathematical distinction between closed and open sets? If you wanted me to spell out the difference between closed and open sets, the best I could do is to draw you a circle one with dotted circumference the other with continuous circumference. Or I would give you an example with a set of numbers $(1, 2)$ vs $[1,2]$ and tell you which bracket signifies open or closed.
But in many theorems the author is dead set about using either closed or open sets. What is the strict mathematical difference that distinguish between the two sets and signifies the importance for such distinction?
Can someone demonstrate with an example where using closed set for a theorem associated with open set would cause some sort of a problem?

The question you asked has answers at various levels of sophistication. You really want to be thinking about intervals rather than finite sets of points, and think also about regions in the plane. Here is one way of looking at things.
A closed set is one which contains all its limit points - so when you are working with closed sets you can be confident abut taking limits because you know that the limit points exist.
An open set can be thought of as one in which every point is an interior point, at least that is a useful guide when you are thinking about sets on the line or in the plane. So the idea is that if you pick a point in an open set, you have enough points close to it in the set to work with, and that there are no points close to it which are outside the set (every point in an open set has a neighbourhood of points wholly within the open set). This cashes out particularly when thinking about continuous functions - the classic epsilon-delta definition is essentially talking about the relationships between nearby points.
These ideas can be considerably generalised and made precise as part of the machinery of topology. Note it is possible to have a set which is both open and closed -- the whole of the real line for example -- or to have a set that is neither open nor closed, such as the set of all rational numbers.
The basic properties of closed and open sets are not the only useful things about them. For example a closed and bounded subset of the real line has a useful property called "compactness", which enables us to reduce some infinite problems to finite ones, and hence get better results.
