1. An image processing apparatus comprising: an object detection unit configured to detect an object from an image; a tracking unit configured to track the detected object; a trajectory management unit configured to manage a trajectory of the object being tracked; and a specific object detection unit configured to detect a specific object from the image, wherein, in a case where the specific object detection unit detects the object being tracked by the object tracking unit to be the specific object, the trajectory management unit manages a trajectory of the object being tracked at a time point before the time point the object being tracked is detected to be the specific object as the trajectory of the specific object.