1. A tracking apparatus comprising: a tracking subject setting unit which sets a tracking subject in image data obtained by imaging; a tracking subject search unit which searches for the set tracking subject by using at least one of luminance of the image data, color of the image data, a result of face detection, and a result of focus detection; an obstructing detection unit which detects that the tracking subject is obstructed; and an obstructing area detection unit which detects an obstructing area which is obstructing the tracking subject when the obstructing of the tracking subject is detected, wherein the obstructing detection unit detects that the tracking subject is obstructed by comparing a standard focus detection information which is a standard information of the tracking subject with a focus detection information obtained at a tracking position, and wherein the tracking subject search unit exclusively sets a search area around the obstructing area and searches the set search area for the tracking subject by using information different from the result of focus detection, when the obstructing of the tracking subject is detected.