He married Ann Ogle (1775–1855) in 1792.
 She was the daughter of Provincial Maryland Governor Benjamin Ogle and Henrietta Margaret (Hill) Ogle, of "Belair", and granddaughter of Samuel Ogle, Proprietary Governor of Maryland.
The Tayloes raised a family of 15 children: sons Benjamin, Edward, Charles, William Henry, John, Henry Augustine, George, Lloyd, Robert Carter; daughters Henrietta, Catherine, Rebecca, Ann, Virginia, Ann Ogle, Elizabeth.


The memorial by one of his sons, Benjamin Ogle Tayloe, says that "his manners were refined and elegant.
He was distinguished for his nice sense of honor, and a scrupulous regard to his word at all times.
His wife was esteemed for sincerity and kindness of heart, graceful and dignified manners, and true and unaffected piety."

His coat of arms, Purpure a sword palewise proper between two lions rampant addorsed [Argent?],
is differenced from the arms displayed by his third-great uncle William Tayloe, grandfather Col.
John Tayloe and those of Teylow in Gloucester, England.
The sword John III displayed was an epee; the only child of Sarah Knowles Bolton, librarian Charles Knowles Bolton, considered that the lions were likely Ermine rather than Argent.
His son Benjamin displayed the arms of his father.
He married Ann Ogle (1775–1855) in 1792.  She was the daughter of Provincial Maryland Governor Benjamin Ogle and Henrietta Margaret (Hill) Ogle, of "Belair", and granddaughter of Samuel Ogle, Proprietary Governor of Maryland. The Tayloes raised a family of 15 children: sons Benjamin, Edward, Charles, William Henry, John, Henry Augustine, George, Lloyd, Robert Carter; daughters Henrietta, Catherine, Rebecca, Ann, Virginia, Ann Ogle, Elizabeth.

The memorial by one of his sons, Benjamin Ogle Tayloe, says that "his manners were refined and elegant. He was distinguished for his nice sense of honor, and a scrupulous regard to his word at all times. His wife was esteemed for sincerity and kindness of heart, graceful and dignified manners, and true and unaffected piety." 
His coat of arms, Purpure a sword palewise proper between two lions rampant addorsed [Argent?], is differenced from the arms displayed by his third-great uncle William Tayloe, grandfather Col. John Tayloe and those of Teylow in Gloucester, England. The sword John III displayed was an epee; the only child of Sarah Knowles Bolton, librarian Charles Knowles Bolton, considered that the lions were likely Ermine rather than Argent. His son Benjamin displayed the arms of his father.
John Tayloe III