His paternal grandfather was the namesake of Remsen, New York, the original proprietor of the township and the inheritor of the Remsenburgh patent, which embraced some 48000 acres in Oneida and Herkimer counties and was granted in 1766 (later re-granted by the Legislature in 1787) to Remsen and four other New York merchants.