Those packets are, in some firewalls, logged.
It is possible for anyone to grab your IP Address by looking at the logged packets and in/out signals that every computer sends when connected to another computer.