Lord Nugent married firstly, on 14 July 1730, Emilia (died in childbirth 16 August 1731), daughter of Peter Plunkett, 4th Earl of Fingall.
They had one son, Edmund, who became a Lieutenant-Colonel and the father of two illegitimate sons (later Field Marshal) Sir George Nugent, 1st Baronet, and Admiral of the Fleet Sir Charles Edmund Nugent) before dying in 1771.

Nugent married secondly, on 23 March 1736, Anna Knight (died 22 November 1756), widow of John Knight and daughter of James Craggs and sister of the Right Honourable James Craggs, the secretary of state.
Nugent adopted the surname of Craggs-Nugent.
She had already been widowed twice, but Robert, who was born a Roman Catholic, had abandoned his Church very early in life.

He married thirdly, on 2 January 1757, Elizabeth (died 29 January 1792), the widow of the fourth Earl of Berkeley, who brought him a large fortune and with whom he had two daughters.

The earldom descended by special remainder to the earl's son-in-law, George Nugent-Temple-Grenville, 1st Marquess of Buckingham, and so to his successors, the dukes of Buckingham and Chandos.
Lord Nugent married firstly, on 14 July 1730, Emilia (died in childbirth 16 August 1731), daughter of Peter Plunkett, 4th Earl of Fingall. They had one son, Edmund, who became a Lieutenant-Colonel and the father of two illegitimate sons (later Field Marshal) Sir George Nugent, 1st Baronet, and Admiral of the Fleet Sir Charles Edmund Nugent) before dying in 1771. 
Nugent married secondly, on 23 March 1736, Anna Knight (died 22 November 1756), widow of John Knight and daughter of James Craggs and sister of the Right Honourable James Craggs, the secretary of state. Nugent adopted the surname of Craggs-Nugent. She had already been widowed twice, but Robert, who was born a Roman Catholic, had abandoned his Church very early in life. 
He married thirdly, on 2 January 1757, Elizabeth (died 29 January 1792), the widow of the fourth Earl of Berkeley, who brought him a large fortune and with whom he had two daughters.
The earldom descended by special remainder to the earl's son-in-law, George Nugent-Temple-Grenville, 1st Marquess of Buckingham, and so to his successors, the dukes of Buckingham and Chandos.
Robert Nugent, 1st Earl Nugent