1. An object tracking device that obtains image data, detects an object, and tracks a target object comprising: a matching unit that is configured to collate image data of a tracking object, which is a tracking target, with image data that has been obtained, and output information about a candidate region of the tracking object; a region determination unit that is configured to determine an object region relating to the tracking object from the candidate region that is output by the matching unit; a depth information calculation unit that is configured to calculate depth information of the object; and a depth range setting unit that is configured to obtain information about the object region that has been determined by the region determination unit at a time earlier than the current time and the depth information that has been calculated by the depth information calculation unit, and set a depth range on the basis of a depth of the tracking object, wherein the region determination unit determines an object region relating to the tracking object based on an image region falling within the depth range set by the depth range setting unit and a candidate region that is output by the matching unit.