The present invention relates generally to various arrangements of optical and electronic components to form a high-resolution helmet mounted display (HMD) or other compact display device. In particular, the current invention is designed in such a way as to allow it to operate utilizing several different kinds of image generating devices and to incorporate many different features including the option of xe2x80x9ceye trackingxe2x80x9d.
Recently there have been several major breakthroughs in display generation technology. These breakthroughs have occurred in the fields of active matrix liquid crystal display (AMLCD), ferroelectric display (FED) technology and a new technology known as liquid crystal on silicon (LCOS). Display devices will soon be emerging with extended graphics array (XGA) and super extended graphics array (SXGA) performance. At this time, it is uncertain which technology will prevail in the HMD market.
The HMD design of this invention works equally well with both transmissive and reflective technologies. When used with a transmissive AMLCD type display device, the HMD design allows the generation of a full stereoscopic (separate image to each eye) image from a single display device. This is achieved by xe2x80x9ctime-multiplexingxe2x80x9d the use of the display device between the two eyes. When used with a reflective display technology such as FED or LCD on silicon device, a separate display chip is required for each eye. In addition to the versatility of this display configuration to utilize either of these display technologies, it is also possible to utilize both of these display technologies simultaneously. In such a situation, the images produced would consist of an xe2x80x9coptical overlayxe2x80x9d of the images generated by the FED and AMLCD devices. Under certain circumstances this configuration can yield great benefits. For example, it would be possible to provide a large panoramic field of view in relatively low resolution (perhaps in monochrome) and use the reflective displays to provide a smaller high-quality field of view as a subset of this larger field image. It is also possible utilizing this unique HMD design to combine multiple fields of view utilizing pairs of similar display technology. For example, one can utilize a pair of transmissive AMLCD displays (2 displays for each eye) or a pair of FED displays for each eye.
With the growing interest in xe2x80x9c-augmented reality-xe2x80x9d and virtual reality based CAD there is a growing demand for new and more efficient man-machine interface tools. One of the most sophisticated user interface tools is that of the xe2x80x9ceye cursorxe2x80x9d or xe2x80x9ceye tracking device.xe2x80x9d Such xe2x80x9ceye cursorxe2x80x9d technology calculates the precise direction that each eye is pointing and mathematicallly calculates the xe2x80x9ctargetxe2x80x9d that is being viewed. This technology is inherently three-dimensional and represents an ideal interface tool for 3-D virtual worlds and augumented reality. The unique design of this HMD allows the incorporation of eye tracking in addition to the display tasks normally associated with an HMD. In addition, the imaging technology that comprises the eye tracking system can be used to overcome one of the major problems with lens-based HMD designs, which is the ability to automatically accommodate viewers with different inter-ocular spacing (spacing between the eyes). Most off-the-shelf HMD""s which are utilizing a lens-based design require manual inter-ocular adjustment for each viewer. This adjustment is often difficult and time-consuming, both of which are factors that make such HMD""s inappropriate for public use. By utilizing the imaging hardware of the eye tracking system as feedback, an automatic or servo-controlled inter-ocular adjustment can be readily achieved. Interestingly, such precise inter-ocular adjustment are an essential requirement for the correct operation of most eye tracking hardware. In this way, both hardware requirements are met and are mutually symbiotic.
In certain situations, a xe2x80x9csee-throughxe2x80x9d augmented reality HMD design is preferable to a fully enclosed virtual reality configuration. In such a situation it is possible to configure the HMD design described in this specification to xe2x80x9csee-through-modexe2x80x9d by replacing the eye tracking video cameras with corrective optics. This will allow the user to see virtual objects superimposed upon the real world images at the cost of losing the eye tracking and automatic inter-ocular adjustment features of the design. An alternative configuration which achieves many of the same objectives is simply to include two miniature video cameras as part of the HMD design. These cameras would provide xe2x80x9creal-timexe2x80x9d video feeds which can be digitally combined with the xe2x80x9cvirtual objectsxe2x80x9d displayed in the HMD. It would of course be possible to provide the same monoscopic video signal from a single miniature video camera to both eyes of the HMD. However, this would preclude the use of eye tracking for all but the virtual objects displayed in the system because eye-tracking inherently requires the stereoscopic images to provide the required three-dimensional information.
In many ways, this electronic equivalent of xe2x80x9csee-through modexe2x80x9d is actually superior to the simplistic optical approach as it inherently avoids the problems of xe2x80x9ctransport lagxe2x80x9d and xe2x80x9c-positional inaccuracies-xe2x80x9d associated with optical superimposing of virtual objects. This is possible because the same xe2x80x9clive video feedsxe2x80x9d are being used to determine the spatial points of reference for the projection of the virtual objects. Therefore, any xe2x80x9ctransport lagxe2x80x9d and/or xe2x80x9cpositional inaccuracies between the xe2x80x9creal worldxe2x80x9d and the xe2x80x9cvirtual worldxe2x80x9d are not perceived by the user because they are common to both the background and the virtual objects projected upon it. In effect, the user is working entirely within a xe2x80x9cvirtual environmentxe2x80x9d.