I (m27) was at a bar/restaurant with my wife (f26) and our neighbor (m42). Me and my wife have been married for 2 years but together for 10. I have known the neighbor for just over a year. 

Although we have been friendly - I never quite liked him. He is 42 years old and as immature as a 16 year old. Always making jokes about "doing my mom" and "hit it and quit it" type of humor. 

So I reluctantly go to a bar/restaurant with him and my wife and he keeps going on and on about "clits". It's an inside thing that is not even worth explaining. So he keeps yelling "clits" and my wife even joins in. He gets the waiter involved talking about clits, stops the busboy talking about clits - both tables by us have turned their heads looking at us. I am embarrassed. 

I start to pretty much berate both of them asking them to stop. They don't. After pleading with my wife, she stops. He doesn't. I then, psuedo-regretfully demean him. Saying, "wow I couldn't imagine my dad at 42 years old making a scene like this at a restaurant." This upsets him.

We get into a verbal argument with threats of going outside to handle this. He is super passive aggressive about how I'm being a bitch and uptight and dick up my ass. Uh, I just want to eat my food and not talk about clits. 

I drove us all there and he just got his check and left. In the car ride home, who gets yelled at? Me. My wife says she was embarrassed because we were arguing and making a scene but wont even comprehend the idea that I was embarrassed about them making a scene long before. My wife admits that he was too much nut is still upset with me. I have absolutely no interest in being his friend moving forward but it seems my wife wants to keep thing friendly. 

What are your thoughts on this? Am I the asshole? 

TL;DR: my neighbor was an embarrassment at a restaurant and wants me to feel bad about it. I get yelled at by my wife - but that's nothing new.