1. An image processing device, comprising: a subject frame setting section which, by operating a subject detector which detects a subject captured in an image, sets a subject frame which surrounds a predetermined range of the subject detected from the image; an acceptance frame setting section which sets an acceptance frame with a range wider than the subject frame according to the context of the image; a position detecting section which detects a specified position on an image; and a recognizing section which recognizes a subject which is a tracking target based on the acceptance frame set by the acceptance frame setting section and the specified position detected by the position detecting section, wherein the subject frame setting section has a plurality of subject detectors to respectively detect a plurality of types of subjects, and, in a case where the recognizing section is unable to recognize the subject which is the tracking target, the operation of the subject detectors other than the subject detector which was operating is started and other types of the subjects are detected.