They meant to kill him somehow, and the defense of him was growing too costly unless there were positive results.
It was too expensive to keep him from getting killed.
Over the generations the castle has been expanded, but it is still essentially a family home, full of furniture, books, and family portraits.