you served us loyally , it 's true ... but varys tells me that your brother rode with renly and then stannis , whilst your lord father chose not to call his banners at all and remained behind the walls of stonehelm all through the fighting . ''