I am having difficult first year in my part-time PhD program; How can I figure out whether I should leave it or not? I have a full-time job, and am a part-time PhD student who is experiencing tremendous struggles throughout the first year, compared to my previous schooling which was extremely easy. I want to know if I'm psyching myself out / over-thinking things, or if I'm being rational or not, given all the circumstances.

My academic career began with a degree in Computer Science, but I took as many side courses in Mathematics as I could.  I did very well, loved it, and thought to do graduate studies. A couple of my professors thought it was a great idea, considering how well I did in their classes, and suggested going for a Ph.D., part-time as I have a full-time job, in the same school. They wrote me stellar recommendation letters and I got in without any issues, whatsoever (unfunded, but that was not a concern to me).

I am nearing the end of my first year in the program, and it has been an unbelievable struggle. A family/personal emergency caused me to have to withdraw from my classes for the first semester; despite how embarrassed I was, my professors told me it was okay and that things happen, and that they still had complete faith in me. Now it's the second semester, and I am struggling to keep up in my classes.  I have always approached math via my intuition, but now I feel I am able to follow my professors in lectures and participate, but when it comes to studying or applying it that's when I struggle. Self-study takes so long and is very arduous and as I'm not used to it; I get extremely distracted. 

Compounded with this, is that I want to discuss this with others, but I find it almost impossible to face my professors that recommended me, anymore. I haven't spoken to them since the winter due to shame. They had so much faith and confidence in me (and probably still do), but I feel that as a PhD I should do better, but I'm not. My performance is shocking and appalling to me, considering what I was used to doing. Talking to my adviser is out of the question as well, as when I discussed my last-semester withdrawal situation with him, he made a loose implication that "he wasn't necessarily sure he would've admitted me, in hindsight," and he wondered aloud who was the one that did admit me. 

I am still in a position where I feel that I would not be happy unless pursuing graduate study in mathematics, as I want to learn this subject inside and out, be surrounded by a community that I find interesting and exciting, and researching and publishing new ideas. I just have no idea if my struggle is either completely normal, if I'm horribly under-qualified, or if I am just psyching myself out.  How can I figure out whether I should stick with my program or not?
I'll keep this answer fairly general, because what you describe sounds like a fairly general situation. 

The transition from undergraduate to Ph.D work is difficult because you're going from a very structured environment (take classes; get grades) to a very unstructured one (take classes, but focus on learning rather than grades; deal with material that's more cutting-edge and therefore open-ended; start getting into research, which is really open-ended). 

This is hard, and it's normal to expect a period of transition where things seem out of whack. 

Coupled with that is the shame: people going to grad school are usually the ones who do well in undergraduate and have done well all their life in school. You've received praise from your teachers throughout, and have received constant encouragement. 

Now you're in grad school. Everyone there is like you, so the competition is more fierce. There are no pseudo-objective measures like grades so there's no way to compare yourself to others (which for high achievers often means that you compare yourself negatively). And above all, all those nice professors who used to encourage you have turned into scowly curmudgeons who expect more and more and more...

It's a wonder everyone isn't depressed. And on top of that you have a full-time job !!

The recommendation that I will give, and that everyone here is giving you, is to ease up on yourself and try separate out your feelings of shame and inadequacy from the reality. You should find a support network of friends, hang out in study groups, do extracurricular stuff. The friends you make in grad school will be friends for life - such is the nature of this crucible. Since you're working full time this is going to be even harder, but maybe you can find others who also work full time and are in a similar position. Also see if you can find senior grad students who are part-time: their experience will be very valuable. 

And then in a year you'll be back year asking about how to avoid discouragement in research, and how to avoid procrastinating, without realizing that this is a sign you've made it through the first year successfully :)