INCORPORATION BY REFERENCE

This application incorporates by reference the entirety of each of the following patent applications: U.S. application Ser. No. 14/555,585 filed on Nov. 27, 2014; U.S. application Ser. No. 14/690,401 filed on Apr. 18, 2015; U.S. application Ser. No. 14/212,961 filed on Mar. 14, 2014; U.S. application Ser. No. 14/331,218 filed on Jul. 14, 2014; U.S. application Ser. No. 15/072,290 filed on Mar. 16, 2016; and U.S. Provisional Application No. 62/156,809, filed on May 4, 2015.

BACKGROUND

Field

The present disclosure relates to optical devices, including augmented reality and virtual reality imaging and visualization systems.

Description of the Related Art

Modern computing and display technologies have facilitated the development of systems for so called “virtual reality” or “augmented reality” experiences, wherein digitally reproduced images or portions thereof are presented to a user in a manner wherein they seem to be, or may be perceived as, real. A virtual reality, or “VR”, scenario typically involves presentation of digital or virtual image information without transparency to other actual real-world visual input; an augmented reality, or “AR”, scenario typically involves presentation of digital or virtual image information as an augmentation to visualization of the actual world around the user. A mixed reality, or “MR”, scenario is a type of AR scenario and typically involves virtual objects that are integrated into, and responsive to, the natural world. For example, in an MR scenario, AR image content may be blocked by or otherwise be perceived as interacting with objects in the real world.

Referring toFIG. 1, an augmented reality scene10is depicted wherein a user of an AR technology sees a real-world park-like setting20featuring people, trees, buildings in the background, and a concrete platform30. In addition to these items, the user of the AR technology also perceives that he “sees” “virtual content” such as a robot statue40standing upon the real-world platform30, and a cartoon-like avatar character50flying by, which seems to be a personification of a bumble bee, even though these elements40,50do not exist in the real world. Because the human visual perception system is complex, it is challenging to produce an AR technology that facilitates a comfortable, natural-feeling, rich presentation of virtual image elements amongst other virtual or real-world imagery elements.

Systems and methods disclosed herein address various challenges related to AR and VR technology.

SUMMARY

In some embodiments, a head-mounted display system is provided. The display system comprises a frame configured to mount on a viewer; a light source; a spatial light modulator configured to modulate light from the light source; and projection optics mounted on the frame and configured to direct light from the spatial light modulator into an eye of a viewer. The display system is configured to display a virtual object on a depth plane by injecting a set of parallactically-disparate intra-pupil images of the object into the eye.

In some other embodiments, a method is provided for displaying image content. The method comprises providing a spatial light modulator; providing a light source configured to output light to the spatial light modulator from a plurality of different light output locations; and displaying a virtual object on a depth plane by temporally sequentially injecting a set of parallactically-disparate intra-pupil images of the virtual object into an eye of a viewer. Each of the intra-pupil images is formed by outputting light from the light source to the spatial light modulator, wherein the light is outputted from one or more associated light output locations of the light source; modulating the light with the spatial light modulator to form an intra-pupil image corresponding to the one or more associated light output locations; and propagating the modulated light to the eye. The one or more associated light output locations for each intra-pupil image is distinct from the one or more associated light output locations for others of the intra-pupil images.

In yet other embodiments, a display system is provided. The display system comprises a light source comprising a plurality of spatially distinct light output locations; a spatial light modulator configured to modulate light from the light source; and projection optics mounted on the frame and configured to direct light from the spatial light modulator into an eye of a viewer. The display system is configured to display a virtual object on a depth plane by temporally sequentially injecting a set of parallactically-disparate intra-pupil images of the object into the eye.

In some other embodiments, a method is provided for displaying image content. The method comprises providing a head-mounted display comprising a light source and a spatial light modulator. The method further comprises displaying a virtual object on a depth plane by injecting, within a flicker fusion threshold, a set of parallactically-disparate intra-pupil images of the virtual object from the display into an eye of a viewer.

In addition, various innovative aspects of the subject matter described in this disclosure may be implemented in the following embodiments:

1. A method for displaying image content, the method comprising:providing a spatial light modulator;providing a light source configured to output light to the spatial light modulator from a plurality of different light output locations; anddisplaying a virtual object on a depth plane by temporally sequentially injecting a set of parallactically-disparate intra-pupil images of the virtual object into an eye of a viewer, wherein each of the intra-pupil images is formed by:outputting light from the light source to the spatial light modulator, wherein the light is outputted from one or more associated light output locations of the light source;modulating the light with the spatial light modulator to form an intra-pupil image corresponding to the one or more associated light output locations; andpropagating the modulated light to the eye,wherein the one or more associated light output locations for each intra-pupil image is distinct from the one or more associated light output locations for others of the intra-pupil images.

2. The method of Embodiment 1, wherein activating the one or more associated light-emitting regions comprises selecting the one or more associated light-emitting regions based upon the depth plane, wherein a physical separation between light-emitting regions for the intra-pupil images increases with decreasing distance of the depth plane to the viewer.

3. The method of any of Embodiments 1-2, wherein light rays forming each of the parallactically-disparate image are collimated, wherein the depth plane is at less than optical infinity.

4. The method of any of Embodiments 1-3, wherein injecting the set of parallactically-disparate intra-pupil images is conducted within a timeframe below the flicker fusion threshold of the viewer.

5. The method of Embodiment 4, wherein the flicker fusion threshold is 1/60 of a second.

6. The method of any of Embodiments 1-5, further comprising an eye tracking sensor configured to track a gaze of the eye, wherein displaying the virtual object comprises:determining a gaze of the eye using the eye tracking sensor; andselecting content for the intra-pupil images based upon the determined gaze of the eye.

7. The method of any of Embodiments 1-6, further comprising projection optics configured to direct modulated light from the spatial light modulator to the eye.

8. The method of any of Embodiments 1-7, wherein the one or more associated light-emitting regions for the intra-pupil images partially overlap.

9. The method of any of Embodiments 1-8, further comprising changing a position of the one or more associated light-emitting regions during injection of at least one of the intra-pupil images into the eye.

10. A display system configured to perform the method of any of Embodiments 1-9.

11. A method for displaying image content, the method comprising:providing a head-mounted display comprising:a light source; anda spatial light modulator; anddisplaying a virtual object on a depth plane by injecting, within a flicker fusion threshold, a set of parallactically-disparate intra-pupil images of the virtual object from the display into an eye of a viewer.

12. The method of Embodiment 11, wherein injecting the set of parallactically-disparate intra-pupil images comprises temporally sequentially injecting individual ones of the intra-pupil images into an eye of a viewer.

13. The method of Embodiment 11, wherein injecting the set of parallactically-disparate intra-pupil images comprises simultaneously injecting multiple ones of the intra-pupil images.

14. The method of Embodiment 13, wherein injecting the set of parallactically-disparate intra-pupil images comprises temporally sequentially injecting multiple intra-pupil images at a time.

15. The method of any of Embodiments 11-14, wherein the light beams forming the intra-pupil images are collimated.

16. The method of any of Embodiments 11-14, wherein the light beams forming the intra-pupil images have divergent wavefronts.

17. The method of any of Embodiments 11-16, wherein the light source comprises a plurality of selectively activated light-emitting regions, wherein injecting the set of parallactically-disparate intra-pupil images comprises activating a different light emitting region for each intra-pupil image.

18. The method of any of Embodiments 11-17, wherein the light source is configured to output light from a plurality of distinct light output locations, further comprising jittering the light output locations during injection of at least one of the intra-pupil images into the eye.

19. A display system configured to perform the method of any of Embodiments 11-18.

20. A head-mounted display system comprising:a frame configured to mount on a viewer;a light source;a spatial light modulator configured to modulate light from the light source; andprojection optics mounted on the frame and configured to direct light from the spatial light modulator into an eye of a viewer,wherein the display system is configured to display a virtual object on a depth plane by injecting a set of parallactically-disparate intra-pupil images of the object into the eye.

21. The display system of Embodiment 20, wherein the display system is configured to temporally multiplex display of individual intra-pupil images.

22. The display system of any of Embodiments 20-21, wherein the display system is configured to spatially multiplex display of the intra-pupil images.

23. The display system of any of Embodiments 20-22, wherein the display system is configured to temporally multiplex display of a plurality of spatially-multiplexed intra-pupil images.

24. The display system of any of Embodiments 20-23, wherein the projection optics comprises a waveguide comprising incoupling optical elements and outcoupling optical elements.

25. The display system of Embodiment 24, wherein the projection optics comprises a plurality of waveguides, wherein each waveguide is configured to output light of a different component color than other waveguides of the plurality of waveguides.

26. The display system of any of Embodiments 20-25, wherein the light source comprises a plurality of selectively-activated light-emitting regions.

27. The display system of Embodiment 26, wherein the light source comprises at least one of a light-emitting diode array and a spatial light modulator.

28. The display system of Embodiment 8, wherein the light-emitting diode array comprises an organic light-emitting diode array or an inorganic light-emitting diode array.

29. The display system of Embodiment 27, wherein the spatial light modulator light source comprises a liquid crystal array or a digital light processing (DLP) chip.

30. The display system of any of Embodiments 20-29, wherein the display system is configured to change a position of activated light-emitting regions during injection of at least one of the intra-pupil images into the eye.

31. The display system of any of Embodiments 20-25, wherein the light source comprises:a light emitter; andan actuator configured to direct light to the spatial light modulator along different paths.

32. The display system of Embodiment 31, wherein the actuator is a dual-axis galvanometer.

33. The display system of Embodiment 31, wherein the light source is a fiber scanner.

34. The display system of any of Embodiments 20-33, wherein the spatial light modulator configured to modulate light from the light source comprises an LCOS panel.

35. The display system of any of Embodiments 20-34, further comprising an eye tracking sensor configured to track a gaze of the eye, wherein the display system is configured to:determine a gaze of the eye using the eye tracking sensor; andselect content for the intra-pupil images based upon the determined gaze of the eye.

36. The display system of any of Embodiments 20-35, wherein the display system is configured to synchronize a light output location of the light source with image content provided by the spatial light modulator.

37. The display system of any of Embodiments 20-36, further comprising an optical mechanism between the spatial light modulator and the projection optics, wherein the optical mechanism is configured to direct light from different locations of the spatial light modulator to projection optics at different angles.

38. The display system of Embodiment 37, wherein the optical mechanism comprises one or more of a prism or a lens structure.

39. The display system of Embodiment 38, wherein the lens structure is a lenslet array.

40. A display system comprising:a light source comprising a plurality of spatially distinct light output locations;a spatial light modulator configured to modulate light from the light source; andprojection optics mounted on the frame and configured to direct light from the spatial light modulator into an eye of a viewer,wherein the display system is configured to display a virtual object on a depth plane by temporally sequentially injecting a set of parallactically-disparate intra-pupil images of the object into the eye.

41. The display system of Embodiment 40, configured to output light from different light output locations of the light source for different intra-pupil images.

42. The display system of Embodiment 41, configured to vary a lateral separation between the light output locations based upon a distance of the depth plane from the eye of the viewer.

43. The display system of any of Embodiments 41-42, configured to increase the lateral separation between light output locations with increases in the distance of the depth plane from the eye of the viewer.

44. The display system of any of Embodiments 41-42, wherein the display system is configured to change the light output locations during injection of at least one of the intra-pupil images into the eye.

