1. A position/orientation measurement apparatus comprising: a model holding unit configured to hold a three-dimensional shape model representing a geometric feature of an object; an approximate value acquisition unit configured to acquire an approximate value indicating a position and an orientation of the object; an image acquisition unit configured to acquire a two-dimensional image of the object captured by an image capturing apparatus; a projection unit configured to project the geometric feature of the three-dimensional shape model on the two-dimensional image based on the approximate value, and to calculate the direction of the geometric feature of the three-dimensional shape model projected on the two-dimensional image; a detection unit configured to detect an image feature based on the two-dimensional image and to calculate the direction of the image feature; an association unit configured to associate the image feature and the geometric feature by comparing the direction of the image feature calculated based on the two-dimensional image and the direction of the geometric feature calculated based on the three-dimensional shape model; and a position/orientation calculation unit configured to calculate the position and the orientation of the object by correcting the approximate value based on the distance between the geometric feature and the image feature associated therewith by the association unit, wherein for each geometric feature in the three-dimensional shape model projected on the two-dimensional image by the projection unit, the detection unit detects a plurality of image features based on the two-dimensional image in correspondence with the geometric feature, and calculates the direction of each of the image features, wherein the association unit comprises a calculation unit configured to, for each geometric feature of the three-dimensional shape model projected on the two-dimensional image by the projection unit, calculate the difference between the direction of the geometric feature and the direction of each of the corresponding image features, and to calculate a reference value based on the obtained differences in direction of the image features at the geometric features, and a selection unit configured to, for each of the geometric features, select any image feature whose difference in direction is within a predetermined range from the reference value, from among the plurality of image features detected in correspondence with the geometric feature, and wherein the association unit associates the selected image features with the corresponding geometric features.