Yes, I definitely have this problem too at times! It depends a bit on what kind of people there are, like for instance if they all know each other already, or have something else strongly in common...