1. A photographic subject tracking device, comprising: a search area setting unit that sets a search area in an input image; a first degree-of-similarity calculation unit that calculates a degree of similarity between a template image for tracking and an image in the search area; a photographic subject position identification unit that identifies a tracked photographic subject position in the input image based on the degree of similarity calculated by the first degree-of-similarity calculation unit; a second degree-of-similarity calculation unit that calculates a degree of similarity between each of multiple template images for resizing determination, which are generated based on the template image for tracking, and the image in the search area; a matching position identification unit that identifies matching positions of the multiple template images for resizing determination, respectively, in the input image based on degrees of similarity calculated by the second degree-of-similarity calculation unit; and a size changing unit that changes an image size of the template image for tracking and the template images for resizing determination based on a density of the plurality of matching positions identified by the matching position identification unit.