the four-story stone house above the waters of the keizersgracht in amsterdam was a monument to , if not a remembrance of , the splendor that was the port city 's wealthiest years at the turn of the century .