the soldiers were mostly on the walls , sweating as they worked the guns , and a great crowd of settlers , men and women and children together , huddled in the shelter of the barracks waiting for the boats to return .