the city 's walls were much more formidable than those of his father 's castle , and had thousands of men to defend them , as opposed to the handful lord randyll would have left at horn hill when he marched to highgarden to answer his liege lord 's summons .