He was the seventh of fifteen children born to William Sargent II (1659–1707), who came to Gloucester before 1678, and Mary Duncan (d. 1724), daughter of Peter Duncan and step-granddaughter of Samuel Symonds, deputy Governor. His maternal grandparents were Mary Eppes (1629–1692) and Peter Duncan (1629–1716), who emigrated from England to Massachusetts. His paternal grandfather was William Sargent (b. c. 1610) from Exeter, England. Career
Sargent was one of the largest landholders in Gloucester. He served as a colonel of militia before the Revolutionary War and was a justice of the general session court for more than thirty years. In 1744, he was Gloucester's representative in the General Court of Massachusetts. In 1760, two years before his death, he had his portrait painted by John Singleton Copley. Personal life

On April 1, 1720, he married Esther McCarty (1701–1743), daughter of Florence McCarty, one of the founders of the first Protestant Episcopal society in New England.
He was the seventh of fifteen children born to William Sargent II (1659–1707), who came to Gloucester before 1678, and Mary Duncan (d. 1724), daughter of Peter Duncan and step-granddaughter of Samuel Symonds, deputy Governor. His maternal grandparents were Mary Eppes (1629–1692) and Peter Duncan (1629–1716), who emigrated from England to Massachusetts. His paternal grandfather was William Sargent (b. c. 1610) from Exeter, England. Career
 Sargent was one of the largest landholders in Gloucester. He served as a colonel of militia before the Revolutionary War and was a justice of the general session court for more than thirty years. In 1744, he was Gloucester's representative in the General Court of Massachusetts. In 1760, two years before his death, he had his portrait painted by John Singleton Singh. Personal life
 On April 1, 1720, he married Esther McCarty (1701–1743), daughter of Florence McCarty, one of the founders of the first Protestant Episcopal society in New England.