Is my career over if thesis went bad? I wrote a master thesis which went quite bad, due to communication problems with advisor, and my advisor said he is not willing to continue guiding me for PhD studies.

Specifically, I was asked to prove that the elementary theory of finite field is decidable, and I had no background on model theory. I had difficulty understanding, for example, the proofs of Riemann hypothesis for curves and Chebotarev's theorem. As such, I made mistakes when I wrote the thesis. It felt like the subject is too large for one thesis so I was unable to fill some details in some proofs. When I asked for guidance for some of the proofs, he did provide assistance, but only after six months time.

However, I have studied math on my own and I know that I am able to understand math by working on my own. Is there any hope for my math career if my thesis advisor said so? I have done couple of small new proofs for some old results, but they are so small that I guess I won't get a publication to any journal.
It's very hard to switch to a different advisor or even a comparable school when your thesis goes bad.  This is because the first question a potential advisor would ask is what happened with your old advisor ... if both you and the advisor don't have a reasonable answer to that, you will have a very hard time.

That said, there are alternatives:


You could try working for a year or two and then returning to PhD studies.
You could try switching to a different area.  There are lots of areas where a student with good mathematics abilities could find interesting problems (economics, computer science, physics, ...)
Switch to a less respectable school that might be willing to take a gamble on an unknown student with a good background.


In short, one of the most important decisions a student can make in his/her graduate career is an advisor.