1. Field of the Invention
The present invention relates to a three-dimensional-position measuring method and a three-dimensional-position measuring apparatus.
The present invention also relates to a three-dimensional shape measuring method and a three-dimentional-shape measuring apparatus for measuring the position of an object and a three-dimensional-shape of the object using visual information of the three-dimensional object.
The present invention also relates to an image pickup (or picture taking) apparatus and a method for synthesizing respective picked-up images (or taken pictures) into one image (or picture), the respective picked-up images (or taken pictures) having been picked up (or taken) in a manner in which a portion of each image corresponds to a portion of another image, these images (or pictures) being images (pictures) present on an object plane to be input. Such an apparatus and a method enable picking up an image (or taking a picture) in a case where the object plane, an image (or picture) of which is to be input, is large or in a case where an object having the object plane, an image (or picture) of which is to be input, cannot be moved.
2. Descriptions of the Related Art
In various fields such as surveying, production of a CAD model and so forth, measurement of a three-dimensional shape of an object has been demanded. As a method for measuring a three-dimensional shape, there is an active method in which a reference pattern or the like is projected on an object, and an image of distortion of the pattern is picked up. However, according to such a method, the object may be affected by projection of the reference pattern, and the effect of the reference pattern varies in accordance with the environment. Because such demerits exist, application of such a method is significantly limited. A passive method, by which an object is not affected, for measuring the shape of an object has been also used. Such a method for measuring a three-dimensional shape is a method in which, basically, corresponding points are extracted from images which have been picked up from two different points of view. Then, the three-dimensional position of the corresponding points is calculated in accordance with the principle of triangulation. In this method, two image pickup apparatuses are fixed and the images of an object are obtained from two different points of view. In another method, one image pickup apparatus is moved on rails a predetermined distance and images of an object are obtained from two different points of view. In these two methods, the relationship of the positions of the two different points of view and the change of the orientation of the image pickup apparatus(es) are treated as already-known information. Instead, there is another method in which an image pickup apparatus is moved in a condition where the apparatus is held by hands of a person, and the relationship of the positions of two different points of view and the change of the orientation of the image pickup apparatus(es) are treated as not-already-known information.
In methods such as those described above in which the relationship of the positions of two different points of view and the change of the orientation of image pickup apparatus(es) are treated as already-known information, a measuring apparatus therefor should be a large-scale apparatus. Therefore, application of such a method is limited. In the method in which the relationship of the positions of two different points of view and the change of the orientation of image pickup apparatus(es) are treated as not-already-known information, the method can be applied to various uses because an object is not affected by such a method and a measuring apparatus therefor can be a small-scale apparatus. As disclosed in, for example, Japanese Laid-Open Patent Application Nos.5-196437, 6-129833, 7-181024, 7-244735 and so forth, various measuring methods and measuring apparatuses have been developed.
In the method disclosed in Japanese Laid-Open Patent Application No.5-196437, an image of a measurement point on an object is picked up by a camera in orthogonal projection, the orientation of the camera at the current time is obtained from a three-axis gyro fixed to the camera, and the thus-obtained information is supplied to a three-dimensional-information measuring apparatus. Thus, the three-dimensional information of the object is measured. In the method disclosed in Japanese Laid-Open Patent Application No.6-129833, using moving image information of an object, images of which have been picked up by a monocular camera, the motion of the camera is obtained, and the three-dimensional shape of the object is calculated. In the method disclosed in Japanese Laid-Open Patent Application No.7-181024, an optical flow, which indicates movement between images, is obtained from a plurality of images of an object, picked up by image pickup means in a manner in which the points of view of the image pickup means are different. Then, based on the optical flow, distance information is calculated. Further, in this method, the distance information is directly calculated by using a formula for a relationship between the time difference and the space gradient of the images and by using the movement information of the image pickup means itself. Then, these ways for obtaining the distance information are combined, and the three-dimensional shape is measured. The apparatus disclosed in the Japanese Laid-Open Patent Application 7-244735 obtains three-dimensional information as a result of temporally combining distance information obtained by using moving image information and movement of a camera.
However, in the case where an integral value of output from an acceleration sensor or a gyro is used for obtaining position change or orientation change as disclosed in Japanese Laid-Open Patent Application Nos.5-196437 and 7-181024, because of use of the integral value, reliability of the information of the thus-obtained position change or orientation change is low. In the case where only image information is used as disclosed in Japanese Laid-Open Patent Application No.6-129833, it is difficult to extract precise corresponding points. In the case where, as disclosed in Japanese Laid-Open Patent Application No.7-244735, the movement speed of the image pickup apparatus is sensed, and the thus-obtained information is used in order to improve the measurement accuracy, the orientation change of the image pickup apparatus is not sensed. Because information concerning the orientation change is important for extracting corresponding points and shape measurement, significant improvement of the measurement accuracy cannot be expected.
An apparatus for inputting and restoring characters or a photograph present on a plane simply and with high accuracy is demanded. When an image of an ordinary size such as an image drawn on a paper sheet of A4 size, A3 size or the like is input, inputting the image with high accuracy can be performed by a copying machine, a scanner or the like. However, a copying machine, a scanner and so forth cannot be used for inputting, with high accuracy, a larger image, an image drawn on a medium which cannot be moved or the like, such as information drawn on a larger paper sheet, information drawn on a wall, information drawn on a panel or the like. As apparatuses and a method for inputting a larger image, an image drawn on a medium which cannot be moved, or the like, using an electronic camera, an electronic camera disclosed in Japanese Laid-Open Patent Application No.6-141228 and a panorama image synthesizing apparatus and a method for producing a panorama image disclosed in Japanese Laid-Open Patent Application No.9-90530 have been developed.
The electronic camera disclosed in Japanese Laid-Open Patent Application No.6-141228 forms the image of an object on an image pickup element via an image pickup lens and a mirror, picks up an image of the object intermittently several times in a manner in which a portion of a previously picked-up image is the same as a portion of a currently picked-up image. The electronic camera moves in a manner in which the currently picked-up image is appropriately connected with the previously picked-up image. In the panorama image synthesizing apparatus and the method for producing a panorama image disclosed in Japanese Laid-Open Patent Application No.9-90530, parameters for synthesizing images are calculated, and the images are synthesized based on the calculated parameters. Thereby, it is attempted to obtain a synthesized image with high accuracy using separate images picked up by the electronic camera.
However, in the electronic camera disclosed in Japanese Laid-Open Patent Application No.6-141228, the mirror and a mechanism for driving the mirror are needed. Thereby, the scale of an apparatus therefor is large. Accordingly, it is difficult to install the apparatus in an electronic camera of an ordinary size.
Further, when an image on a paper sheet or a panel is picked up, the distance between a camera and an object is short. Therefore, in the panorama image synthesizing apparatus and the method for producing a panorama image disclosed in Japanese Laid-Open Patent Application No.9-90530, due to influence of perspective projection due to translation operation of the camera, distortion of the synthesized image occurs. Further, an image rotates because a person holds the electronic camera, and the image rotation causes distortion of the synthesized image.