How do I talk about my abusive former advisor if people ask? I was just at a conference where I met another grad student who - upon learning about my specialization and university affiliation - excitedly asked me if I knew Dr. -- and if he was my advisor (she's a fan). Deer in the headlights moment for me, as I did know Dr. --; he had been my advisor - one that until recently I admired and was very fond of. Only he's not my advisor anymore, because our once enviable mentor/mentee relationship is now in ruins. (In sum: he crossed a professional line with me, I called him out on it, he started punishing me for it in various ways, and it culminated in him stepping down from my committee and me reporting him to HR (human resources) when my dept chair dismissed my complaints about his abusive behavior. After effects: we're not speaking to each other, avoid each other as much as possible, I get panic attacks/have some sort of PTSD (Post-Traumatic Stress Disorder) whenever I have run-ins with him, and I've slowly become isolated from the rest of my department because he practically runs it and no student or faculty member wants/dares to step on his toes). 

The thing is, I came to this university primarily to work with him...he and everyone in the dept knows this..and pretty much anyone in our field of research would assume that he and I are connected somehow, given that we're at the same university, and how much our research interests align. So, as I've discovered from this conference incident, I need to come up with a response for if/when people ask me if I know him/work with him/why am I not working with him. Personally, I'd love to just answer truthfully and throw him under the bus since that's what he's done to me several times over these last few months, but I know doing so will hurt me more than it'll hurt him. Any suggestions on how to handle those types of questions?

EDIT:
First, thank you so much to everyone for taking the time to respond. I really appreciate everyone's feedback and different opinions. You've definitely given me a lot to think about. From reading the responses, I think I can clarify a few items if you think it'll alter your suggestions:


"crossed a professional line" - admittedly, I intentionally used that vague term because it's very complicated, and also because he crossed that line in a few ways. One is of a sexual harassment nature, but I was reluctant to use that term because it happened in a very subtle and covert manner - so much that the Title IX (gender equality) office at my university said that technically his behavior (both initial line crossing and his reaction to me standing up to him) - however unprofessional it was - did not rise to the level of sexual harassment. But as I said, he was unprofessional towards me in other non-sexual ways as well. Ultimately, for the scope of my question, I don't think it matters much, because regardless of what he did to me initially, regardless of whether or not I had any right to be offended, and regardless of which specific incident he took issue with when I confronted him, nothing justifies how he subsequently treated me/how he handled his anger, which was essentially bullying me and then later gaslighting me and becoming verbally abusive/hostile when I tried to address the bullying.
HR investigation: the investigation is ongoing; I filed my official complaint about a month ago, so hopefully I'll hear back soon re: next steps/resolution.
I'm an American student in the U.S.
The extent to which others know: a few of the the graduate students/TAs/my friends do know about everything that's happened. Some of the faculty do as well (including my other two committee members, one of whom is my new advisor; the other was also willing to take over as advisor) Outside my department: HR, Title IX office, the graduate ombudsperson, assistant dean of the college of arts &amp; sciences, university police (Dr. - didn't do anything criminal...it was just a formality that HR had to observe when I filed my complaint) so his behavior has definitely been publicized a bit, but in a very hushed/behind closed doors sort of way.
First, let me say that I'm sorry to hear that your relationship with your original advisor has deteriorated to such a state that you have been emotionally scarred by the relationship. That is certainly not the desired outcome.
That said, you have a very important cautionary story to tell. Unlike cactus_pardner's answer, I don't think you should &quot;jokingly defer,&quot; but instead take a very serious tone about the situation:

I wish I could be more positive about my relationship with Prof. X.
If you are serious about working with Prof. X, then there are some things that you should be aware of.
This may not happen to you, but I had some significant issues during my time working in that lab.

You can then offer to talk with the asker in more detail, if she so wishes, and leave the choice up to the asker. But do not blow off the discussion. Hiding the situation will just allow it  to continue unchecked, which is exactly what you do not want to see happen in the long run.