While in Hawaiʻi, Pitman and his older sister Mary attended Mrs. Wetmore's children's school in Hilo.
The school was located at the Wetmores' residence on Church Street.
Taught by Lucy Sheldon Taylor Wetmore, the wife of American missionary doctor and government physician Charles Hinckley Wetmore, who had come to Hawaii in 1848 with the American Board of Commissioners for Foreign Missions (ABCFM), the two elder Pitman children received their education in English rather than Hawaiian.
This was unusual since Hawaiian was the official language of the Kingdom of Hawaiʻi, and all other schools in Hilo were conducted in the Hawaiian language.
Mrs. Wetmore taught the children reading, writing, spelling, arithmetic and singing, while also reinforcing the curriculum with a strong adherence to the principles of the Protestant faith.
Like the Pitman siblings, many of their classmates were also of half-Hawaiian (hapa-kanaka) descent with a majority of them being Chinese-Hawaiians (hapa-pake).

After the death of his mother Kinoʻole in 1855, Pitman's father remarried to Maria Louisa Walsworth Kinney, the widow of American missionary Rev. Henry Kinney.
The Kinneys were part of the Twelfth Company of missionaries from the ABCFM to arrive in 1848.
The marriage aligned the Pitman children with the American missionary community.
They were called "cousins" by the children of the missionaries and considered part of the extended missionary family of Hawaiʻi.
This first stepmother died in 1858 after giving birth to their father's fourth child, a daughter named Maria Kinoʻole (1858–1892).
The Pitman family returned to Massachusetts in 1861 after his father remarried to his third wife Martha Ball, giving his four children another stepmother.
Letters by traveler Sophia Cracroft, niece of Lady Jane Franklin, indicated that the Pitman family left for San Francisco, on June 25, 1861, aboard the ship Comet with Cracroft and Lady Franklin and that the elder Pitman "now has a third wife with a baby [Charles Brook Pitman]."
According to an 1887 biography written by Robert G. Carter, a private who would later serve in the same company as Pitman, he was neglected after his mother's death by his father and stepmother, who "subjected [him] to neglect and treatment, that with his sensitive nature he could not bear".
He continued his education in the public schools of Roxbury, where the Pitman family lived for a period of time.
He have proceeded the family in leaving Hawaii.
The 1860 United States Census registered Pitman under his teacher Solomon Adams as residing and presumably being educated in Newton, also in the Boston area.

Growing into adolescence, he was said to strongly resemble his Hawaiian mother.
His enlistment card during the war noted he had a dark complexion, hazel colored eyes, black hair and stood five feet eight inches.
Robert G. Carter gave a brief description of his appearance in wartime letters first published in 1897:
[A] tall, slim boy, straight as an arrow.
His face was a perfect oval, his hair was as black as a raven's wing, and his eyes were large and of that peculiar soft, melting blackness, which excites pity when one is in distress.
His skin was a clear, dark olive, bordering on the swarthy, and this, with his high cheek bones, would have led us to suppose that his nationality was different from our own, had we not known that his name was plain Henry P. There was an air of good breeding and refinement about him, that, with his small hands and feet, would have set us to thinking, had it not been that in our youth and intensely enthusiastic natures, we gave no thought to our comrades' personal appearance.
We can look back now and see the shy, reserved nature of the boy, the dark, melancholy eyes, the sad smile, the sensitive twitching of the lips.
While in Hawaiʻi, Pitman and his older sister Mary attended Mrs. Wetmore's children's school in Hilo. The school was located at the Wetmores' residence on Church Street. Taught by Lucy Sheldon Taylor Wetmore, the wife of American missionary doctor and government physician Charles Hinckley Wetmore, who had come to Hawaii in 1848 with the American Board of Commissioners for Foreign Missions (ABCFM), the two elder Pitman children received their education in English rather than Hawaiian. This was unusual since Hawaiian was the official language of the Kingdom of Hawaiʻi, and all other schools in Hilo were conducted in the Hawaiian language. Mrs. Wetmore taught the children reading, writing, spelling, arithmetic and singing, while also reinforcing the curriculum with a strong adherence to the principles of the Protestant faith. Like the Pitman siblings, many of their classmates were also of half-Hawaiian (hapa-kanaka) descent with a majority of them being Chinese-Hawaiians (hapa-pake).
After the death of his mother Kinoʻole in 1855, Pitman's father remarried to Maria Louisa Walsworth Kinney, the widow of American missionary Rev. Henry Kinney. The Kinneys were part of the Twelfth Company of missionaries from the ABCFM to arrive in 1848. The marriage aligned the Pitman children with the American missionary community. They were called "cousins" by the children of the missionaries and considered part of the extended missionary family of Hawaiʻi. This first stepmother died in 1858 after giving birth to their father's fourth child, a daughter named Maria Kinoʻole (1858–1892). The Pitman family returned to Massachusetts in 1861 after his father remarried to his third wife Martha Ball, giving his four children another stepmother. Letters by traveler Sophia Cracroft, niece of Lady Jane Franklin, indicated that the Pitman family left for San Francisco, on June 25, 1861, aboard the ship Comet with Cracroft and Lady Franklin and that the elder Pitman "now has a third wife with a baby [Charles Brook Pitman]." According to an 1887 biography written by Robert G. Carter, a private who would later serve in the same company as Pitman, he was neglected after his mother's death by his father and stepmother, who "subjected [him] to neglect and treatment, that with his sensitive nature he could not bear". He continued his education in the public schools of Roxbury, where the Pitman family lived for a period of time. He have proceeded the family in leaving Hawaii. The 1860 United States Census registered Pitman under his teacher Solomon Adams as residing and presumably being educated in Newton, also in the Boston area.
Growing into adolescence, he was said to strongly resemble his Hawaiian mother. His enlistment card during the war noted he had a dark complexion, hazel colored eyes, black hair and stood five feet eight inches. Robert G. Carter gave a brief description of his appearance in wartime letters first published in 1897:
[A] tall, slim boy, straight as an arrow. His face was a perfect oval, his hair was as black as a raven's wing, and his eyes were large and of that peculiar soft, melting blackness, which excites pity when one is in distress. His skin was a clear, dark olive, bordering on the swarthy, and this, with his high cheek bones, would have led us to suppose that his nationality was different from our own, had we not known that his name was plain Henry P. There was an air of good breeding and refinement about him, that, with his small hands and feet, would have set us to thinking, had it not been that in our youth and intensely enthusiastic natures, we gave no thought to our comrades' personal appearance. We can look back now and see the shy, reserved nature of the boy, the dark, melancholy eyes, the sad smile, the sensitive twitching of the lips.
Henry Hoʻolulu Pitman