Before I get on with the story I want to point out that I have BPD, generalized anxiety disorder, and depression. This makes my emotions switch on a dime and causes me to see through something called "black and white" vision (basically I can go from hating someone to adoring them for no apparent reason).

So I've never really dated anyone mostly because I'm a flaming homosexual and the only people attracted to me are guys. A few months ago one of my friends openly stated she had a crush on me. This kind of shocked me because I had a but of a crush on her as well, but she had a boyfriend. 

We had talked a bit and a few days later I told her that I get really anxious and a relationship wouldnt be good for either of us mentally. (She has depression aswell) Besides, I didnt want her to cheat on her boyfriend. 

She said that she understood and I thought we would go back to being friends. I was wrong. Whenever I'd go get lunch and sit with a few of my friends she would get really affectionate with me; aka hold my hand, hug me, etc. This would be fine but 1: we weren't dating and 2: touching me made me really uncomfortable.

I didn't say anything but I didnt really share the affections. That night she texted me, asking if I was mad at her. I had explained that I wasnt mad but just got really anxious when people touch me. I also went on a long tangent about how we shouldn't date because my BPD makes it extremly difficult for both people in the relationship. She seemed understanding so I thought that was it.

A few days after I was in a very bad place mentally and it showed. She talked with me and made me feel a lot better, which in turn led me to get clingy that day. I wanted to hold her and hug her, even if that was very odd for me. I needed someone to hug that day, and she was that person. I guess she assumed that was the normal and for the next few weeks tried holding and kissing me. I was visibly anxious once again and told her that it made me uncomfortable. 

She took this as anger and ended up avoiding me for the next week because she thought I was mad at her. This was not true and I had confronted her about it. After this things went back to normal, and by normal it meant she was clingy and close to me again. 

Now I knew she had her own problems and I really didn't want to cause things to go worse for her. This meant I let her hold me close no matter how uncomfortable it made me. I wanted to help her and by avoiding her efforts it obviously affected her negativly.

This went on for a while and I kind of let it happen. It took its toll on me, getting me into a worse state by the day. The only time I drew the line was when she invited me to her house for a dinner the very day. I got anxious at the thought and frantically made any excuse I could. She told me how important it was to her and I told that I couldn't. This led to a mini fight and she ghosted me for a week. Again she thought I was mad at her and had again explained that situations made me very anxious and could send me into a very bad panic attack.

Again things back to normal. Up until I had snapped. I got upset and told her to please stop touching me and to stop getting upset after I dont answer her constant calls or texted. I got mad because she wouldnt listen to me and we technically broke up. She was really hurt by that, and avoided me. I had come back to re-breakup, but being much more polite about it. I told her that it just wouldn't work, and continuing a relationship would only hurt us both. I apologized for earlier but said I still didnt want a relationship.

Am I an asshole for breaking it off with her?

TLDR: My mental illnesses make relationships very hard. Girl tries to date me anyway even though I am openly uncomfortable. After not listening to my requests I decide to break up with her. 