So to make the relationship backstory short my ex-wife had a &gt;1 year affair and divorced me because she wasn't happy and wanted to be with him. After a lot of on again off again stuff its final that she has chosen to be in a relationship with the affair partner. Per our custody arrangement we have 50/50 split custody (one week on, one week off) of our two girls aged 5 and 7. We are currently living together and co-parenting until my ex-wife can afford to move out.

My ex-wife has a friend, let's call her Mary, who has cancer and lives close to my ex-wife's work and with whom she stays most nights when it's my week to watch the girls. Not too long ago I found out that she had lied to me about staying with Mary when she was really spending the night with the affair partner. It is currently her week to watch the kids and tonight she asked me if I was okay with taking care of the girls overnight so she could stay with Mary. I told her that if she was actually staying with Mary I was fine with it. She asked me what I meant and I told her that I was not okay with doing a favor for her that directly benefited the guy she had cheated on me with and who I obviously hate. She stated that it shouldn't matter who she was staying with if I'm okay with watching them for her to stay with Mary then I should be fine with watching them for whatever reason and that I was being controlling. I think there's a big difference and told her that putting conditions on a favor is not controlling since the other option is just a straight no.  

I've been pretty messed up by everything that's gone down and don't have a ton of people who I can ask for advice, so reddit, AITA?