1. An image processing apparatus comprising: an object defining unit configured to define a specific object that can occlude a target object to be observed; an image obtaining unit configured to obtain a captured image captured by an image pickup apparatus; a region detector configured to detect a region including the specific object from the obtained captured image; an image feature detector configured to detect image features of the target object from the captured image; and a calculating unit configured to calculate a position or orientation of the image pickup apparatus that captured the captured image or a position or orientation of the target object in the captured image on the basis of the image features detected by the image feature detector, wherein the image processing apparatus is configured not to take into account features from the detected region in determining the position and/or orientation of the image-pickup apparatus that captured the image or a position and/or orientation of the target object in the captured image.