1. A subject tracking device comprising: a processor; and a memory which stores a plurality of instructions, which when executed by the processor, cause the processor to execute, detecting at least one subject candidate area in which it is probable that a tracking target subject appears on an image that is received from an imaging unit; calculating a degree of blur of the subject candidate area for each of the subject candidate areas; determining that the subject appears in a subject candidate area having a degree of blur in accordance with a moving speed of the subject, out of the subject candidate areas; and deciding movement of the subject depending on a movement direction from an area in which the subject appears on a previous image that is captured by the imaging unit before capturing the image, to the subject candidate area in which the subject appears on the image.