Are the letters $O$ and $\infty $ homeomorphic? I feel they are homeomorphic
But if we remove the intersection point from $\infty $ we get two disconnected components, right ?

As you have realized, a useful topological invariant is the set of path components $\pi_0$. A continuous map $f\colon X\rightarrow Y$ must induce a map on path components $f\colon \pi_0(X)\rightarrow \pi_0(Y)$. This is because it respects the equivalence relation which defines path components; if $\beta\colon I\rightarrow X$ is a path from $x$ to $x'$ (both points in $X$), then $f\circ \beta \colon I\rightarrow Y$ is a path from $f(x)$ to $f(x')$. Importantly, homeomorphisms induce bijections on $\pi_0$ (this is an easy exercise). 
This is often useful in "local" considerations like the one you make in your question. Indeed, if $x\in X$ then $f$ must induce a homeomorphism from $X\setminus x$ to $Y\setminus f(x)$. But this will then induce a bijection between $\pi_0 (X\setminus x)$ and $\pi_0 (Y\setminus f(x))$.
