1. A pattern dimension measuring device that measures a dimension of a pattern formed on a specimen or between different patterns with the use of an acquired image, comprising: an arithmetic device that moves a field of view with reference to a first pattern formed on the specimen on the basis of predetermined first distance information, acquires a first image, executes template matching with the use of the first image and a matching template, and calculates a distance between a second pattern included in the first image and the first pattern on the basis of second distance information obtained by the template matching, and the first distance information.