Should I prefer PhD programs with "higher quality" students? In choosing between PhD programs A and B (for pure math in the US), some professors have remarked to me that A is favorable because it is more selective and consequently has stronger students than B. Is this a factor I should consider? Both institutions have several of the top researchers in my subfield so quality of advisors is not an issue.
It's worth considering in your case because your graduate experience will in part be shaped by your peers. Your peer group in graduate school will help (or hurt) your research and teaching experiences --- at some schools your peers will help lift you up and inspire you, at others they will underwhelm you.

Your peers will also help with networking. The people who are finishing when you arrive in grad school will be useful people to get advice from when you go on the job market, as they will have recently been through it themselves.

I'm in the finishing stages of a PhD right now and I've benefited a lot from knowing the top-notch students who had come before me. I learnt from them, they have given me lots of feedback over the years, they've helped me with networking, etc.