Is finding an interest in Mathematics in late teens not a good sign for amazing research capability? I am 19 years old.  I was average in high school in Mathematics. I never participated in IMO or related competitions.  As a result I didn't  join a prestigious undergraduate school. However during 2nd year in abstract algebra courses I developed an obsession for the subject. My advisor told me to look over some graduate level abstract algebra textbooks which I certainly will. 

My school can offer me some grad level courses in my fourth year like Class Field Theory and Modular Forms. Nonetheless, not being able to attend a prestigious undergraduate still haunts me. I know it will certainly affect my graduate school admissions.

In the long term, do late bloomers in Mathematics have a chance to pursue good research careers in Mathematics?
It is somewhat common to find highly intelligent, promising students that find it difficult to apply themselves in topics they are not interested in. Upon finding that one topic, these people excel quickly because it consumes them. I actually know scientists at top universities that previously flunked out of college for just that reason. So the fact that you’re experiencing this now is not surprising; in fact you’re fortunate that you’re still so young - plenty of time to dive in. 

On your less-than-reputable undergrad institution, this may hurt you a bit as names do matter, sadly. Fortunately, if you mean to go academic in life, this is less a problem for undergraduate institution than graduate. So, you should set your sights on a good graduate school and start working on it. The best way to do this is to gain the respect of a professor you know to secure a strong letter and also start reaching out to specific people you’d like to work with. Take a very impassioned, personal approach, and you’ll be given a bit more consideration than the nameless others that apply. 

Good luck, future mathematician!