on the contrary , documents indicated that it had always been a private castle , built in the eleventh century by someone named armand de clery , and later heavily rebuilt early in the fourteenth century , with another ring of outer walls , and additional drawbridges .