Eventually you will reach Foog 's Gate , but take a right before you travel through the gate and follow the path around to the Castle Vaults .
The path goes around to the vaults where they keep the guns .