Born in New York City in 1924, Vanderbilt was the great-great granddaughter of railroad tycoon Cornelius Vanderbilt, who left $100 million USD to the family when he died in 1877 at age 82 . Following her father's death when she was just a baby, Gloria was thrust into the center of a fierce and highly-publicized battle over her custody (one that earned her the nickname "poor little rich girl") between her mother and her aunt, Gertrude Vanderbilt Whitney. Gertrude, who herself was worth $78 million USD and founded the Whitney Museum of American Art, became Gloria's guardian.