After getting hired, department matter I am a new tenure-track faculty.  I did my phd in a very weird way -- my advisor was in a different dept.  So even for my committees, I did not have anyone from my department -- I even do not know anyone in my department.  

I got hired for a tenure-track faculty.  I had a social meeting -- three people asked, "what department did you graduate?".  For many reasons, I said, my advisors department -- not mine....

However, now I am questioning it is ok...  May I get your advice?  Do I need to approach them to correct?  

I feel panicky as a new kid here now
I seriously doubt that it matters, but you might want to develop the story with your new colleagues at some point. The story is interesting, not terrible. It is a bit unusual, that is all. 

In fact, it is in a lot of ways the most honest answer rather than the most bureaucratic answer. 

Relax. Develop a nice story and become a legend.