1. A shape measuring device, comprising: an imaging device that receives light from a surface of a measurement object illuminated by an illuminating section, and capturing a surface image of the measurement object, a relative moving section that causes a relative movement between the measurement object and the imaging device along the optical axis of an optical system constituting the imaging device, an image processing section that processes a plurality of images of the surface of the measurement object captured by the imaging device; wherein, the image processing section comprises: a first processing section that extracts one target image and at least one reference image other than the target image from among the plurality of images, and uses a differential operator to act on the target image and the reference image to calculate feature quantities of the target image based on each extracted pair of the target image and the reference image; and a second processing section that calculates the surface height of the measurement object on the basis of the maximum relative movement position of each pixel from among a plurality of the feature quantities calculated on a pixel by pixel basis; where the differential operator has coefficients which, in calculation of the feature quantities, are weighted with respect to the pixel values of observed pixel of the target image and the pixel values of adjacent pixels located at different pixel locations from the observed pixels in the reference image; and the feature quantities are derivatives of the pixel values of the observed pixels and the pixel values of the adjacent pixels acted upon by the differential operator.