The American retreat from Fort Ticonderoga began late on July 5 after British cannons was seen on top of high ground , Mount Defiance ( aka Rattlesnake Mountain and Sugar Loaf Hill ) that commanded the fort . The bulk of General Arthur St. Clair 's army retreated through Hubbardton to Castleton , while the rear guard , commanded by Seth Warner , stopped at Hubbardton to rest and pick up stragglers .