Is NOBODY wrong in the post ? POST : So I'm fairly socially inept and I have no issue with anyone based on the color of their skin. I'm white as can be.

This happened awhile ago, but I was only recently made aware of the fallout of this interaction.

My SO (now ex), me, and our (at the time) 17mo daughter were invited to dinner with one of his friends and his buddy's wife. The wife is black. We went to their house and had a nice, if slightly awkward, dinner and throughout our visit my daughter was absolutely infatuated with his wife. She made a comment about my daughter being so interested in her and by way of explaining I told her she's the first "Person of color" my daughter's ever met. That was the only comment related to her race I made all night, it wasn't a big deal to me, and I didn't think anything of the comment. If it was racially insensitive I wasn't aware and I'd like to apologize. I was informed (about 3 months after the fact) that we were no longer welcome in their house, because his wife thinks I'm racist and that I don't like her because of the color of her skin.

Should I apologize? And if so, how? Im not sure it even matters anymore since I have no relationship with these people outside of my (now defunct) relationship with my ex. I feel like if I made some big social gaffe I should at least try to apologize. I don't like the idea that I made someone feel bad about their race. So AITA? And if I am how do I make it right? 