On October 24, 1837, William married his cousin, Esther Phillips Wetmore of Middletown, Connecticut, at Gloucester Lodge, Regent's Park, London.
She was the daughter of his uncle Samuel Wetmore and a sister of Mary Cryder.
The following year, a daughter was born in New York City, who died at birth or soon thereafter on October 12, 1838.
Esther died on October 26, 1838.

After her death, he married 21-year-old Anstiss Derby Rogers in Salem, Massachusetts, on September 5, 1843, daughter of John Wittingham Rogers.
They had three children:
William Shepard Wetmore Jr. (1844–1858), who died of scarlet fever in June 1858.

George Peabody Wetmore (1846–1921), later the Governor of Rhode Island and a U.S. Senator.

Annie Derby Rogers Wetmore (1848–1884), who married William Watts Sherman on July 7, 1871.

In Newport, he built Chateau-sur-Mer, the first of the grand Bellevue Avenue mansions of the Gilded Age in Newport, Rhode Island.
It is now open to the public as a museum.
 He furnished it with "strange and interesting" Chinese lacquer furniture and porcelain.
In 1860 he added a massive stone moon gate on the grounds from designs brought back from China.

However, there is no mention of his wife, Anstiss, in a series of letters written between Annie, George, and their father, corresponding between Newport and New York City during the years 1856-1860.
It is believed she lived at the Merlano Cottage in Jamaica Plain, Massachusetts.
According to the 1860 and 1865 census, Mrs. Wetmore is not residing at Chateau-sur-Mer.
Servants living at the house in the 1860 census were a butler, cook, laundress and chambermaid.
Another resident in the 1860 census at Chateau-sur-Mer was a Wetmore cousin, twenty-year-old Lucy Dennison.

William S. Wetmore conducted an active community life in Newport as a founder and incorporator of the Newport Historical Society in 1854, as the first and continuing chairman of the Newport Reading Room, 1854–1861, and as a Director of the Redwood Library from 1856-1862.
Wetmore was one of sixteen Newporters who joined to contribute $16,500.00 to Touro Park.
He was also elected Vice President of the innovative Butler Hospital for the insane in 1858, which initiated reform of treatment for the poor and insane.

Upon William Wetmore's death on June 16, 1862, sixteen-year-old George and fourteen-year-old
Annie were master and mistress of Chateau-sur-Mer.
On October 24, 1837, William married his cousin, Esther Phillips Wetmore of Middletown, Connecticut, at Gloucester Lodge, Regent's Park, London. She was the daughter of his uncle Samuel Wetmore and a sister of Mary Cryder. The following year, a daughter was born in New York City, who died at birth or soon thereafter on October 12, 1838. Esther died on October 26, 1838.
After her death, he married 21-year-old Anstiss Derby Rogers in Salem, Massachusetts, on September 5, 1843, daughter of John Wittingham Rogers. They had three children:
William Shepard Wetmore Jr. (1844–1858), who died of scarlet fever in June 1858.
George Peabody Wetmore (1846–1921), later the Governor of Rhode Island and a U.S. Senator.
Annie Derby Rogers Wetmore (1848–1884), who married William Watts Sherman on July 7, 1871.
In Newport, he built Chateau-sur-Mer, the first of the grand Bellevue Avenue mansions of the Gilded Age in Newport, Rhode Island. It is now open to the public as a museum.  He furnished it with "strange and interesting" Chinese lacquer furniture and porcelain. In 1860 he added a massive stone moon gate on the grounds from designs brought back from China.
However, there is no mention of his wife, Anstiss, in a series of letters written between Annie, George, and their father, corresponding between Newport and New York City during the years 1856-1860. It is believed she lived at the Merlano Cottage in Jamaica Plain, Massachusetts. According to the 1860 and 1865 census, Mrs. Wetmore is not residing at Chateau-sur-Mer. Servants living at the house in the 1860 census were a butler, cook, laundress and chambermaid. Another resident in the 1860 census at Chateau-sur-Mer was a Wetmore cousin, twenty-year-old Lucy Dennison.
William S. Wetmore conducted an active community life in Newport as a founder and incorporator of the Newport Historical Society in 1854, as the first and continuing chairman of the Newport Reading Room, 1854–1861, and as a Director of the Redwood Library from 1856-1862. Wetmore was one of sixteen Newporters who joined to contribute $16,500.00 to Touro Park. He was also elected Vice President of the innovative Butler Hospital for the insane in 1858, which initiated reform of treatment for the poor and insane.
Upon William Wetmore's death on June 16, 1862, sixteen-year-old George and fourteen-year-old
Annie were master and mistress of Chateau-sur-Mer.
William Shepard Wetmore