Is OTHER wrong in the post ? POST : Using a throwaway because I know he lurks on Reddit.

So, when I was younger I'd always get teased by my bother. My sisters never said or did things as harsh as he did but being the only boy I thought it was just something boys did. I'd also like to mention there is a 7 year age gap. One day he'd say or do something that made me cry and the next we'd play video games. I was the only other sibling that really enjoyed playing the games he liked so we'd play together often. 

As I grew up he'd become meaner to me. Not just the teasing kind but actually making me feel bad for being interested in certain things. At this point I had started high school and like some kids, the change was hard on me and I developed some form of anxiety. I started seeing a counselor at school because sometimes going to class was difficult for me and just seeing a counselor was helpful. 

On the first day of my junior year, I had an appointment to see the counselor during my English class. They had called my mom's cell later on in the day, which I was okay with since I wanted my parents knowing that I was having a tough time. Unknowingly my brother had answered. He used this against me saying horrible things about me, things along the lines of me being a psycho. Even going as far as to use that exact word. This was the straw that broke the camels back.  This was five years ago and I've only spoken to him since a handful of times. Each time is a different argument, that has me out of the house for hours at a time. 

I haven't since gone back to get help for the anxiety that I still get to this day. This is mainly due to a fear I have of him finding out about me needing help and being made fun of again. Something I couldn't handle the first time.

 A few days ago I was getting my car fixed from a minor accident and my mom had to pick me up from work. I was in a bad mood since work hadn't been that easy on me, and she teased me about always having an attitude. I kind of stayed silent and she asked If I wanted to look into going back for help with my anxiety? I told her why I didn't want to, as I had before. She got upset at me asking why I couldn't forgive him if it had been such a long time ago. That I hurt her for not wanting anything to do with my brother. She was upset that I constantly exclude myself from family events because I don't want to be around him. 

This event has left me conflicted. I am not ready to be on talking terms with him, but if it makes my mother so upset. Should I try to? This also causes my sisters to get annoyed with me because I wont attend events if he is there. They say I'm acting immature but they also don't have the full story so I don't blame them. So Reddit, Am I the Asshole for not wanting anything to do with my brother and causing my mother to be upset? 