A good time to fight Brand is when he's out of mana or has an ability on cooldown. This will make trading much better for you as he will not be able to dish out a lot of damage (or possibly stun you if his Q is on cooldown) as he'll have 1 less ability to use. Use your early game strength to poke and harass the enemy with your W as often as you can.