Assessing quality of research groups within math departments as a prospective PhD student? First, let me quickly say that I've read as many related questions on this site as I can find, and I believe my questions haven't been directly addressed yet. 

I am currently selecting between a few PhD programs in mathematics and have read many different accounts of how to best approach the process. There are several top answers here that suggest that one should try hard to attend the program with highest prestige, as long as this doesn't directly compromise your ability to pursue research in your field of interest (or land you in an area in which you are unhappy). Most posts then qualify that statement carefully, reminding the OPs that measuring prestige is not the same as comparing ranks on USNews (in multiple threads it is suggested that there is little difference between schools that differ by 10 points).

My questions are:

1) Are the AMS groupings generally a better indicator of a program's quality than differences in USNews ranks? Is it an unwise move to attend a Group II school if you are admitted into Group I schools, given that your ultimate goal is to work as a research mathematician? 

2) How does one start to compare programs' strengths, specifically in their subfield of interest, without resorting to some kind of online ranking? Which metrics are reliable indicators for the strength of the specific research group in a particular department?

Honestly, I wish I wasn't thinking so much about my future in these terms. I come from a humble undergraduate institution, and many of my options represent a positive leap forward, for which I am very grateful. However, I can't help but feel the pressure of the competitiveness of the academic job market (all hearsay to me). When I read that hiring committees pay close attention to these numbers, I feel like I ought to do the same.

Thanks for any insight
For both your questions, I will posit that the key factor is whether the group in question is doing "high impact" work. For example, are their papers interesting and often cited? Are they working in very "popular" areas? Do the respected researchers in your (sub)field know the names of the senior group members?

To your questions:


I am not a mathematician, but I do know that the rankings will be at the level of the "department average", where faculty hires will be based on the "group" or even the individual. So, a high-power group in a third-tier school is better than a lower-power group in a higher-ranked school. 
The factors listed above are perhaps the "true" criteria -- but, as an undergraduate, you are probably not in a position to judge your prospective advisors' work. This is why the rankings are so useful. If you have anyone at your current institution that you trust to be well-informed as to the state-of-the-art in your (sub)field, you may want to ask that person for some advice as well.