Many computerized systems utilize position and orientation sensors. Most notable are virtual and augmented reality systems, computerized recognition systems, robotics, etc. Those sensors are used to learn the location (X, Y, and Z), and orientation (yaw, pitch, and roll) of the sensor.
To increase clarity and brevity, these specifications will relate interchangeably to a combination of location and orientation as ‘LOR’, and to the combination of yaw, pitch, and roll as ‘YPR’.
Many LOR sensors exist that provide location and orientation parameters. The most common ones for location are based on GPS or inertial navigation technologies. For orientation, magnetic and gyroscopic, as well as light based systems are widely used. LOR sensors, or YPR sensors may be divided into two broad categories: absolute sensors and relative sensors. Absolute sensors provide location and/or orientation information in absolute terms, by sensing the location relative to an external reference such as longitude, latitude, bearing, elevation, etc, or by sensing the absolute orientation using earth's magnetic and gravitational, and the like. Relative sensors are placed in a known local position, with a known orientation. Thereafter, the sensor collects and integrates motion information to derive the present position and/or orientation. While the representation may be either relative or absolute, the relative sensor measurement is based on accumulated displacement from a known origin point. Such relative position systems will be alternatively referred to as delta based systems in these specifications.
Many relative sensors suffer from drift errors that stem primarily from cumulative errors in the motion measurements. Therefore, solutions based on relative sensors can operate only for a limited amount of time before the errors become excessive and system accuracy drops below acceptable levels.
The existing absolute sensors suffer from compromises as well: Fast and accurate sensors are prohibitively expensive for many applications. While inexpensive sensors generally provide a position with sufficient accuracy, they require a long time to stabilize and thus are unsuitable for many applications that involve rapid motion in one or more axis. Vehicular, marine and aviation applications are especially vulnerable to the slow resolution time of those inexpensive sensors.
There exist in the art several solutions for fast and precise LOR sensing. Those may be divided into three categories: fiduciaries based, image memorization, and triangulation. Fiduciary based systems rely on the placement of markers—either objects or marked up points such as a grid, in the imaged scene. Identification of the fiduciaries by image analysis allows finding both the location and the orientation of the sensor. Naturally, this type of solution requires pre-placement of the fiduciaries, an act often impractical for many applications, most specifically in environments hostile to the placement of such fiduciaries, such as in military augmented reality systems.
Memorization based sensors utilize a library of memorized images taken at an earlier time, with LOR information attached to the images. Comparisons between current image and one or more images stored in the library provide for LOR information. In some cases, a present image may provide location information, from which a relative sensor can derive an initial placement. Relative motion sensing provides for LOR thereafter, until the errors discussed above make the information unusable. Memorization requires video knowledge of the environment prior to use. Such knowledge is not always available and prevents the use of memorization sensors in a new environment.
Triangulation sensors use devices such as RF transponders, mirrors that reflect light emanating from the system such as laser light and the like, to receive information from the environment. This again requires placement of the transponders or mirrors in the environment which oftentimes makes the solution unusable.
It should be noted that most often the speed of location change tend to be far slower than the speed of YPR change. Moreover, for many practical applications the rate of location change is sufficiently small to permit the use of an inexpensive location sensor as the only means for location information. However changes in YPR are far faster, and thus require fast resolution. This is especially the case when the system is used to provide visual information such as in an augmented reality system, where the user perception is severely hampered by registration errors with the environment.
U.S. Pat. No. 4,802,757 to Pleitner et al. describes a system for determining attitude of a moving imaging sensor platform by utilizing a digital image correlator for comparing successive images of the terrain taken from a second image sensor in known LOR. The second image sensor may be located on a satellite or the like. Clearly this solution is only viable as long as the second image source is available, which limit the system use. However Pleitner et al. provides some algorithmic solution to finding YPR information from video image comparison.
In U.S. Pat. No. 6,453,223 to Kelly et al. and in US published application US2002/0069013n to Navab et al. a computer assisted methods for determining position and orientation are provided. The methods include sensing an image of the scene and comparing the sensed image to previously captured images of the scene. This method requires the use of an image map composed of previously captured images of the environment and is not practicable in many applications.
U.S. Pat. No. 4,672,562 is representative of the fiduciary based system, where a target point array is placed in fixed relation to an object. Spatial information of the object is derived from an image in correspondence to the target points.
U.S. Pat. No. 6,285,930 to Dickson et al. teaches an image based method for sensing the orientation of crop rows relative to agricultural machinery and controlling the machinery or some aspects of its operation responsive to that orientation.
An algorithmic source of computing orientation relative to a ground beacon array (fiduciaries) may be found in Egli's U.S. Pat. No. 4,866,626. These algorithms provide an example of possible methods to derive YPR information by comparing known elements in an image to a newly acquired image.
All those solutions suffer the disadvantage of requiring training or placing objects in the environment. If for example one of the uses of the system is entering into an environment for the first time, such as by a ship entering a new port, or an airplane flying over new terrain, or a combat team entering a hostile environment, the solutions provided are impractical. There is therefore a clear and unanswered need for a system that will provide LOR data in a fast and efficient manner and at reasonably low cost. The present invention aim at providing an apparatus and a method for answering this need.