He was one of seven children born to Daniel Sargent Sr.
and Mary (née Turner) Sargent (1744–1813).
 He was the brother of author Lucius Manlius Sargent (1786–1867), and Daniel Sargent (1764–1842) merchant and a cousin of American Revolutionary War soldier Paul Dudley Sargent (1745–1828), and a great-grandson of William Sargent, who received a grant of land at Gloucester in 1678.
 Henry Sargent's father was a prosperous and public-spirited merchant.

Henry was sent as a young boy to Dummer Academy, South Byfield, and then, the family having moved to Boston, he continued his studies under local teachers.
After a period in the counting house of Thomas Handasyd Perkins, he continued his commercial apprenticeship with his father, but suddenly, as he was attaining his majority, without having previously shown special partiality for the arts of design, he determined to become an artist.
An elder brother's efforts in this direction seem to have stimulated him, and his mother encouraged him.
John Trumbull, who visited Boston in 1790, saw some of his work and found it promising.
He was one of seven children born to Daniel Sargent Sr. and Mary (née Turner) Sargent (1744–1813).  He was the brother of author Lucius Manlius Sargent (1786–1867), and Daniel Sargent (1764–1842) merchant and a cousin of American Revolutionary War soldier Paul Dudley Sargent (1745–1828), and a great-grandson of William Sargent, who received a grant of land at Gloucester in 1678.  Henry Sargent's father was a prosperous and public-spirited merchant.
Henry was sent as a young boy to Dummer Academy, South Byfield, and then, the family having moved to Boston, he continued his studies under local teachers. After a period in the counting house of Thomas Handasyd Perkins, he continued his commercial apprenticeship with his father, but suddenly, as he was attaining his majority, without having previously shown special partiality for the arts of design, he determined to become an artist. An elder brother's efforts in this direction seem to have stimulated him, and his mother encouraged him. John Trumbull, who visited Boston in 1790, saw some of his work and found it promising.
Henry Sargent