Not including Neidpath, who owned the other castle that Sir Simon Fraser had for most of its existence?
Simon Fraser (d. 1306): Sir Simon Fraser of Oliver and Neidpath was a Scottish knight who fought in the Wars of Scottish Independence, for which he was hanged, drawn, and quartered in 1306.
i don't know