Can a grad student take undergrad classes? Maybe a dumb question, but for example: say I'm doing a PhD in Theoretical Physics (I'm not) but I need to have taken certain undergrad math courses to understand the material, which I never took in undergrad. Do grad students ever end up in undergrad classes for this reason, or do they just learn the material themselves out of books?
Depends on institution policies, but usually yes.
That said, the grade you get usually won't matter, and besides you might already know some of the things in the course. You might be better off just focusing on the interesting part then, by self-educating.