1. A position/orientation detecting device for detecting a three-dimensional position/orientation of an object based on two-dimensional images of the object taken from a plurality of different visual points, comprising: defining means for defining two-dimensional taught models of the object and at least three reference points on the respective two-dimensional taught models; image detecting means for detecting images of the object in each of the two-dimensional pictures by matching using said two-dimensional taught models; corresponding image determining means for determining correspondence between the image detected in one of the two-dimensional pictures and the image detected in another one of the two-dimensional pictures; reference point position calculating means for obtaining a position of each of the reference points on each of the detected images using positions of the reference points defined on the two-dimensional taught models; and three-dimensional position/orientation determining means for obtaining three-dimensional position of each of the reference points based on the obtained position of each of the reference points on each of the detected images, and determining the three-dimensional position/orientation of the object using the obtained three-dimensional positions of the reference points.