Are the letters $O$ and $\infty $ homeomorphic? I feel they are homeomorphic
But if we remove the intersection point from $\infty $ we get two disconnected components, right ?

No, they are not. The fundamental group of "O" is isomorphic to the group of integers. However, using Van Kampen's theorem, one can see that the fundamental group of the figure eight is isomorphic to the free product on two sets of integers: Z*Z. The first group is abelian, while the second is not. Therefore they do not have isomorphic fundamental groups and the spaces cannot be homeomorphic. 
Another proof - using only point-set topology:  Assume they are homeomorphic. Then the figure eight minus the center point p must be homeomorphic to the circle O minus some point q. However, removing the point from the figure eight results in a non-connected space, while the circle minus a point is still connected. Contradiction, as homeomorphism preserves connectedness. 
