Attending Conferences as an unaffiliated listener I will be graduating soon with my PhD in mathematics, but do not intend to pursue an academic career.  I do, however want to continue to dabble and toy with the ideas I have invested so much to immerse myself in.  I love the idea that I may come up with a result or two in my lifetime.  But I just don't jive well with research pressures.

It is a faux pas to attend a conference independently, and just pay the registration fee myself?

It would seem that registration is open to anyone technically, but I have never enjoyed making a fool out of myself.  I appreciate the clarification.
Go for it ... probably.

There are no doubt aspects of conferences that are specific to individual fields, so my answer based on experience in Computer Science may not apply to Mathematics.  But I'm guessing the answer to this is somewhat general, in that some conferences are small (and may be largely limited to people who are actively participating/presenting) while others are larger.  If you pick a large enough conference, with many more attendees than presenters, no one will think twice of who you are or why you're there.