1. An image processing apparatus comprising: an acquisition unit configured to acquire an image from an imaging apparatus that captures an image of a physical space; a calculating unit configured to obtain location information of each image feature in the physical space based on image coordinates of the image feature in the captured image; a selection unit configured to select location information usable to calculate a position and an orientation of the imaging apparatus among the location information obtained by the calculating unit; and a calculation unit configured to calculate the position and the orientation of the imaging apparatus based on the location information selected by the selection unit and an image feature corresponding to the selected location information among the image features included in the captured image.