Addison Brown was born on February 21, 1830, in West Newbury, Massachusetts, the oldest of five children of Addison Brown Sr.,
a shoemaker, and Catherine Babson Griffin, both descended from Massachusetts' earliest Pilgrim settlers.
He attended West Newbury's one-room school until he had exhausted its offerings at age 12.
In 1843 he began more advanced studies in such areas as Latin, physics, algebra, and philosophy.

In 1848 Brown entered Amherst College, intending from the start to transfer to Harvard University in his sophomore year.
While at Harvard, Brown earned money as the college organist and unhappily spent some summer months as a village school teacher.
Brown befriended and roomed with his Harvard classmate Horatio Alger and counted Ephraim Whitman Gurney (who became a professor of philosophy and history and dean of the Harvard faculty) as his closest college friend.

Brown received an Artium Baccalaureus degree in 1852 from Harvard University, ranked second in his class.
Joseph Hodges Choate, who became a lawyer and diplomat, was ranked third.
Joseph's brother, William Gardner Choate, who preceded Brown in the United States District Court for the Southern District of New York, was valedictorian; George Bethune Adams, Brown's successor on the bench, was also a member of the class of 1852.
To restore his health after years dedicated to study, Brown spent the summer of 1852 working aboard a fishing boat, sailing out of Gloucester, Massachusetts to Prince Edward Island.


In his Autobiographical Notes, Brown wrote that a college graduate in his circumstances had three career choices: the ministry, medicine or the law.
He "knew nothing of the law, or of lawyers personally, ... and disliked the life of a physician."
After consideration, he deemed himself ill-suited for the ministry: "Only law remained."
Upon learning that he could save almost half of the cost of a  Harvard law degree by working and studying in a law office for a year, Brown returned home to the law offices of John James Marsh.
Brown entered Harvard Law School in 1853, receiving a Bachelor of Laws in late 1854.
Addison Brown was born on February 21, 1830, in West Newbury, Massachusetts, the oldest of five children of Addison Brown Sr., a shoemaker, and Catherine Babson Griffin, both descended from Massachusetts' earliest Pilgrim settlers. He attended West Newbury's one-room school until he had exhausted its offerings at age 12. In 1843 he began more advanced studies in such areas as Latin, physics, algebra, and philosophy.
In 1848 Brown entered Amherst College, intending from the start to transfer to Harvard University in his sophomore year. While at Harvard, Brown earned money as the college organist and unhappily spent some summer months as a village school teacher. Brown befriended and roomed with his Harvard classmate Horatio Alger and counted Ephraim Whitman Gurney (who became a professor of philosophy and history and dean of the Harvard faculty) as his closest college friend. 
Brown received an Artium Baccalaureus degree in 1852 from Harvard University, ranked second in his class. Joseph Hodges Choate, who became a lawyer and diplomat, was ranked third. Joseph's brother, William Gardner Choate, who preceded Brown in the United States District Court for the Southern District of New York, was valedictorian; George Bethune Adams, Brown's successor on the bench, was also a member of the class of 1852. To restore his health after years dedicated to study, Brown spent the summer of 1852 working aboard a fishing boat, sailing out of Gloucester, Massachusetts to Prince Edward Island. 

In his Autobiographical Notes, Brown wrote that a college graduate in his circumstances had three career choices: the ministry, medicine or the law. He "knew nothing of the law, or of lawyers personally, ... and disliked the life of a physician." After consideration, he deemed himself ill-suited for the ministry: "Only law remained." Upon learning that he could save almost half of the cost of a  Harvard law degree by working and studying in a law office for a year, Brown returned home to the law offices of John James Marsh. Brown entered Harvard Law School in 1853, receiving a Bachelor of Laws in late 1854.
Addison Brown