Settlers also had some continuing conflicts with Indians and insurgents using the recent Confederate cause as excuse for robbery . By the spring of 1873 , the US Army withdrew the troops from Fort Scott for good . From 1873 to 1965 , the buildings of the fort were left unattended , and slowly deteriorated . Many military buildings were razed and replaced with structures built for civilian use .