1. An object tracking device which tracks a target object in a time-series image including a plurality of frames, the object tracking device comprising: a location information acquisition unit that acquires location information of a target object in a first frame, the target object being a tracked target; a detailed contour model generation unit that generates a detailed contour model in the first frame, on the basis of the location information, the detailed contour model being formed with a plurality of contour points representing a contour of the target object; a search location setting unit that sets a plurality of different search locations in a second frame, the second frame being any one of frames following the first frame; a rough contour model generation unit that identifies a plurality of first associated points at each search location in the second frame which correspond to locations of the plurality of contour points, when the detailed contour model is applied to each search location, the rough contour model generation unit selecting a first predetermined number of points out of the plurality of first associated points as second associated points, the first predetermined number being fewer than the first associated points, and generating a rough contour model formed with the second associated points; a likelihood assessment unit that compares, at each search location, a second image feature amount related to the second associated points of the rough contour model in an image of the second frame with a first image feature amount related to one or more of the contour points of the detailed contour model in an image of the first frame which correspond to the second associated points, thereby assessing likelihoods that the individual search locations in the second frame are a location of the target object; and an object location identification unit that identifies the location of the target object from the plurality of search locations, on the basis of the likelihoods determined by the likelihood assessment unit.