I think a little background is necessary. My coworker and I, let’s call him Tom, have never directly worked with each other. A position I used to hold was responsible for bringing in new people and making sure they were getting their proper training and answering any questions they may have. This is how we were acquainted. On many occasions I would stop and talk with him, I’ve consulted him and helped him learn and do things to advance his career. During these interactions we got fairly comfortable and would speak freely. I know he is gay and we’ve even joked on the subject before. He knows my brother is gay and I literally got ordained so I could marry him to his now husband.

The incident..
A couple of weeks ago me and two other coworkers were chatting and joking around. Tom came walking through the area and stopped and joined our conversation. Him and I began to kind of trade jokes with each other. One of the people there stopped us and said “calm down guys, no need to be like that.” Tom replied “Oh, we do this all the time. We just see who is the bigger asshole.” I replied “Something tells me you would win that battle.” Tom got a super serious look on his face and says “On that note, this is where I walk away.”
I’ve seen him numerous times since then and always say hi or try to spark conversation but he gives me a cold shoulder or a quick reply. I’ve even seen him kind of go out of his was to avoid going by me.
AITA for making that joke?