Before invading, make sure you have your Passive stacked so you can win the trade. If they invade you, be prepared to fight, but only do so if your Passive is stacked and you can win, otherwise just back off.