CROSS-REFERENCE TO RELATED APPLICATIONS

This application incorporates by reference the entirety of each of the following patent applications: U.S. application Ser. No. 14/555,585 filed on Nov. 27, 2014; U.S. application Ser. No. 14/690,401 filed on Apr. 18, 2015; U.S. application Ser. No. 14/212,961 filed on Mar. 14, 2014; and U.S. application Ser. No. 14/331,218 filed on Jul. 14, 2014.

BACKGROUND

Field

The present disclosure relates to display systems, including augmented reality imaging and visualization systems.

Description of the Related Art

Modern computing and display technologies have facilitated the development of systems for so called “virtual reality” or “augmented reality” experiences, in which digitally reproduced images or portions thereof are presented to a user in a manner wherein they seem to be, or may be perceived as, real. A virtual reality, or “VR”, scenario typically involves the presentation of digital or virtual image information without transparency to other actual real-world visual input; an augmented reality, or “AR”, scenario typically involves presentation of digital or virtual image information as an augmentation to visualization of the actual world around the user. A mixed reality, or “MR”, scenario is a type of AR scenario and typically involves virtual objects that are integrated into, and responsive to, the natural world. For example, an MR scenario may include AR image content that appears to be blocked by or is otherwise perceived to interact with objects in the real world.

Referring toFIG. 1, an augmented reality scene10is depicted. The user of an AR technology sees a real-world park-like setting20featuring people, trees, buildings in the background, and a concrete platform30. The user also perceives that he/she “sees” “virtual content” such as a robot statue40standing upon the real-world platform30, and a flying cartoon-like avatar character50which seems to be a personification of a bumble bee. These elements50,40are “virtual” in that they do not exist in the real world. Because the human visual perception system is complex, it is challenging to produce AR technology that facilitates a comfortable, natural-feeling, rich presentation of virtual image elements amongst other virtual or real-world imagery elements.

Systems and methods disclosed herein address various challenges related to AR and VR technology.

SUMMARY

In some embodiments, a display system comprises a head-mountable display having a hyperfocal distance. The head-mountable display comprises one or more waveguides each having optical power and an associated depth plane. The one or more waveguides are configured to project light to a viewer to display image content on the associated depth planes. Each of the depth planes is at less than optical infinity.

In some other embodiments, a display system comprises a head-mountable display. The head-mountable display comprises one or more waveguides each having optical power and an associated depth plane. The one or more waveguides are configured to project light to a viewer to display image content on the associated depth planes. A farthest one of the depth planes is within about 0.33 dpt of optical infinity.

In yet other embodiments, a display system comprises a head-mountable display. The head-mountable display comprises a plurality of waveguides forming a stack of waveguides. Each waveguide has optical power and an associated depth plane, wherein the waveguides are configured to project light to a viewer to display image content on the associated depth planes. A farthest one of the depth planes is within a mismatch tolerance of optical infinity. The mismatch tolerance is about 0.5 dpt.

In some other embodiments, a method for displaying image content on a head-mounted display is provided. The method comprises determining whether an accommodation-vergence mismatch for the image content exceeds a threshold; and modifying the image content if the combination-vergence mismatch exceeds the threshold.

In yet other embodiments, a method for displaying image content on a head-mounted display worn by a user is provided. The method comprises determining a presence of eye strain in the user; and modifying the image content if eyestrain is determined to be present.

In some embodiments, a wearable head-mounted display system comprises a frame configured to be worn by a user. A display is attached to the frame. The display system also comprises a support structure configured to extend from one side of a head of the user to an other side of the head. The support structure is mechanically coupled to the frame.

In some embodiments, a display system comprises a display device, one or more processors, and computer storage media. The display system performs operations that comprise determining a fixation depth, wherein the fixation depth is a depth at which eyes of the user are fixating. The operations also include determining whether to adjust selection of a selected depth plane at which a virtual object is being presented to the user's eyes is determined based on whether the fixation depth is within a depth plane range that is: (1) solely encompassed by the selected depth plane; or (2) encompassed by both the selected depth plane and an adjacent depth plane. Presentation of the virtual object is caused at a particular depth plane selected based on determinations made in determining whether to adjust selection of a selected depth plane.

In some embodiments, a display system comprises a display device, one or more processors, and computer storage media. The display system performs operations that comprise determining a fixation depth, wherein the fixation depth is a depth at which eyes of the user are fixating. A particular depth plane of the plurality of depth planes at which a virtual object is to be presented to the user is determined, the determination based on the fixation depth and depth plane ranges encompassed by each of the depth planes, wherein adjacent depth planes both encompass a depth overlap region. Presentation of the virtual object is caused at the particular depth plane.

In some embodiments, a method comprises determining a fixation depth, wherein the fixation depth is a depth at which eyes of the user are fixating. Whether to adjust selection of a selected depth plane at which a virtual object is being presented to the user's eyes is determined based on whether the fixation depth is within a depth plane range that is: (1) solely encompassed by the selected depth plane; or (2) encompassed by both the selected depth plane and an adjacent depth plane. Presentation of the virtual object is caused at a particular depth plane selected based on the determination of whether to adjust selection of the selected depth plane.

In some embodiments, a display system comprises a display device, one or more processors, and computer storage media. The display system performs operations that comprise determining a fixation point of a user indicating a three-dimensional location upon which the user is fixating. The operations also include determining whether to switch a depth plane at which a virtual object is to be presented is determined, the determination based, at least in part, on a depth of the determined fixation point. The depth plane at which the virtual object is to be presented is switched, wherein switching the depth plane is triggered by a user perception limiting event.

Additional examples of embodiments are provided below.

Example 1

A display system comprising:a head-mountable display comprising:one or more waveguides each having optical power and configured to provide content on an associated depth plane, wherein the one or more waveguides are configured to project light to a viewer to display image content on the associated depth plane of the one or more waveguides,wherein each of the depth planes is at less than optical infinity.

Example 2

The display system of Claim1, wherein a farthest one of the depth planes is within about 0.50 dpt of optical infinity.

Example 3

The display system of Claim2, wherein a farthest one of the depth planes is within about 0.33 dpt of optical infinity.

Example 4

The display system of Claim3, wherein a farthest one of the depth planes is within about 0.25 dpt of optical infinity.

Example 5

The display system of any of Claims1-4, wherein the depth planes are separated by no more than about 0.7 dpt.

Example 6

The display system of any of Claims1-6, wherein the depth planes are separated by no more than about 0.5 dpt.

Example 7

The display system of any of Claims1-7, wherein the display is configured to display image information on only two depth planes, wherein the two depth planes are each at less than optical infinity.

Example 8

The display system of Claim1, wherein the display is configured to display image information on only one depth plane, wherein the one depth plane is at less than optical infinity.

Example 9

A display system comprising:a head-mountable display comprising:one or more waveguides each having optical power and an associated depth plane, wherein the one or more waveguides are configured to project light to a viewer to display image content on the associated depth planes,wherein a farthest one of the depth planes is within about 0.33 dpt of optical infinity.

Example 10

The display system of Claim10, wherein a next farthest one of the depth planes is within about 0.66 dpt of the farthest one of the depth planes.

Example 11

The display system of any of Claims10-11wherein a total number of depth planes is two.

Example 12

The display system of any of Claims10-12, wherein a total number of depth planes is greater than two, wherein a separation between immediately neighboring depth planes is less than about 0.66 dpt.

Example 13

The display system of any of Claims10-13, wherein a total number of depth planes is less than four.

Example 14

The display system of any of Claims10-13, wherein the one or more waveguides form a stack of the waveguides, wherein each waveguide comprises incoupling optical elements configured to redirect incident light to propagate by total internal reflection inside the waveguide.

Example 15

The display system of Claim15, wherein the incoupling optical elements of each waveguide are configured to selectively redirect light having wavelengths corresponding to a single component color.

Example 16

The display system of Claim15, wherein, as seen in a top-down plan view, the incoupling optical elements of each waveguide are laterally spaced-apart from the incoupling optical elements of other waveguides.

Example 17

The display system of any of Claims15-17, wherein each waveguide further comprises outcoupling optical elements configured to redirect light propagating within each waveguide out of the waveguide.

Example 18

The display system of any of Claims15-18, further comprising a light projector system configured to direct image content to the incoupling optical elements of the waveguides, the light projector system comprising:a light emitter; anda spatial light modulator.

Example 19

The display system of any of Claims10-19, wherein each waveguide has optical power for creating only a single depth plane.

Example 20

A display system comprising:a head-mountable display comprising:a plurality of waveguides forming a stack of waveguides, wherein each waveguide has optical power and is configured to provide content on an associated depth plane, wherein the waveguides are configured to project light to a viewer to display image content on the associated depth planes,wherein a farthest one of the depth planes is within a mismatch tolerance of optical infinity, wherein the mismatch tolerance is about 0.5 dpt.

Example 21

The display system of Claim21, wherein the mismatch tolerance is about 0.33 dpt.

Example 22

The display system of any of Claims21-22, wherein the separation between an associated depth plane and a nearest associated depth plane of the stack is about twice the mismatch tolerance or less.

Example 23

The display system of any of Claims21-23, wherein a total number of depth planes is four or less.

Example 24

The display system of Claim24, wherein the total number of depth planes is two.

Example 25

A method for displaying image content on a head-mounted display, the method comprising:determining whether an accommodation-vergence mismatch for the image content exceeds a threshold; andmodifying the image content if the accommodation-vergence mismatch exceeds the threshold.

Example 26

The method of Claim26, wherein the accommodation-vergence mismatch threshold is 0.5 dpt or less.

Example 27

The method of Claim27, wherein the accommodation-vergence mismatch threshold is 0.33 dpt or less.

Example 28

The method of any of Claims26-28, wherein modifying the image content comprises fading the image content.

Example 29

The method of Claim29, wherein fading the image content comprises reducing a resolution of the image content.

Example 30

The method of Claim30, wherein a reduction in the resolution of the image content increases with increasing accommodation-vergence mismatch.

Example 31

The method of any of Claims26-31, wherein modifying the image content comprises not displaying the image content.

Example 32

A display system comprising:a processor; andcomputer storage media storing instructions that when executed by the display system, cause the display system to perform operations comprising:the method of any of Claims26-32.

Example 33

The display system of Claim33, wherein the display system is configured to display image information on only one depth plane, wherein the one depth plane is at less than optical infinity.

Example 34

The display system of Claim33, wherein the display system is configured to display image information on only two depth planes, wherein the two depth planes are each at less than optical infinity.

Example 35

A method for displaying image content on a head-mounted display worn by a user, the method comprising:determining a presence of eye strain in the user; andmodifying the image content if eyestrain is determined to be present.

Example 36

The method of Claim36, wherein determining the presence of eyestrain comprises imaging one or both eyes of the user.

Example 37

The method of any of Claims36-37, wherein determining the presence of eyestrain comprises detecting one or more of pupil dilation, convergence oscillation, and pupil oscillation.

Example 38

The method of any of Claims36-38, wherein determining the presence of eyestrain comprises measuring a galvanic skin response.

Example 39

The method of any of Claims36-39, wherein determining the presence of eyestrain comprises detecting a duration of exposure to image content having an accommodation-vergence mismatch greater than 0.25 dpt.

Example 40

The method of Claim40, wherein the accommodation-vergence mismatch is greater than 0.33 dpt.

Example 41

The method of Claim41, wherein the accommodation-vergence mismatch is greater than 0.50 dpt.

Example 42

The method of any of Claims36-42, wherein modifying image content comprises one or more of:increasing a size of features of the image content;reducing a resolution of the image content; anddisplaying the image content on a depth plane farther from the viewer than originally specified for the image content.

Example 43

The method of any of Claims36-43, wherein were modifying image content is performed until the presence of eyestrain is no longer detected in the user.

Example 44

The method of any of Claims36-44, wherein were modifying image content is performed for a set duration.

Example 45

A display system comprising a processor and computer storage media storing instructions that, when executed by the display system, cause the display system to perform the method of any of Claims36-44.

Example 46

The display system of Claim45, wherein the display system is configured to display image information on only one depth plane, wherein the one depth plane is at less than optical infinity.

Example 47

The display system of Claim45, wherein the display system is configured to display image information on only two depth planes, wherein the two depth planes are each at less than optical infinity.

Example 48

A wearable head-mounted display system comprising:a frame configured to be worn by a user;a display attached to the frame; anda support structure configured to extend from one side of a head of the user to an other side of the head,wherein the support structure is mechanically coupled to the frame.

Example 49

The wearable head-mounted display system of Claim48, further comprising a sound transducer attached to the support structure and configured to direct sound into an ear of the user.

Example 50

The wearable head-mounted display system of Claim49, wherein the sound transducer is a speaker.

Example 51

The wearable head-mounted display system of any of Claims48-50, wherein the support structure is a band that is configured to extend from one side of the head to an other side of the head.

Example 52

The wearable head-mounted display system of Claim51, wherein the band crosses the head of the user at between a 35° and a 55° angle relative to a plane intersecting the eyes and ears of the user.

Example 53

The wearable head-mounted display system of Claim51, wherein the band crosses the head of the user at between an 80° and a 100° angle relative to a plane intersecting the eyes and ears of the user.

Example 54

The wearable head-mounted display system of any of Claims52-53, wherein an angle of the band relative to a plane intersecting the eye and the first and second ear of the user is adjustable.

Example 55

The wearable head-mounted display system of any of Claims48-54, further comprising a waveguide comprising:an incoupling optical element configured to selectively incouple incident light into the waveguide based upon a property of the incident light; andan outcoupling optical element configured to project light to an eye of the viewer by outcoupling the light incoupled into the waveguide.

Example 56

The wearable head-mounted display system of Claim55, wherein the outcoupling optical element has optical power and is configured to project light to the eye to display image content on an associated depth plane, wherein the associated depth plane is at less than optical infinity.

Example 57

The wearable head-mounted display system of Claim56, wherein the waveguide is part of a stack of waveguides, wherein at least some waveguides of the stack of waveguides comprises different associated depth planes and outcoupling optical elements having different optical power so as to provide different divergence of exiting light for each of the at least some waveguides.

Example 58

A display system comprising:a display device configured to present virtual objects to a user at a plurality of depth planes;one or more processors; andcomputer storage media storing instructions that, when executed by the display system, cause the display system to perform operations comprising:determining a fixation depth, wherein the fixation depth is a depth at which eyes of the user are fixating;determining whether to adjust selection of a selected depth plane at which a virtual object is being presented to the user's eyes based on whether the fixation depth is within a depth plane range that is:(1) solely encompassed by the selected depth plane; or(2) encompassed by both the selected depth plane and an adjacent depth plane; andcausing presentation of the virtual object at a particular depth plane selected based on determining whether to adjust selection of a selected depth plane.

Example 59

The display system of claim58, wherein the fixation depth is within a depth plane range solely encompassed by the selected depth plane, wherein the determination of whether to adjust selection of a depth plane is negative.

Example 60

The display system of claim58, wherein the fixation depth is within a depth plane range that is encompassed by both the selected depth plane and an adjacent depth plane, wherein the determination of whether to adjust selection of a depth plane is negative.

Example 61

The display system of claim58, wherein the fixation depth is within a particular depth plane range outside of a depth plane range that is:(1) solely encompassed by the selected depth plane; and(2) encompassed by both the selected depth plane and an adjacent depth plane, and wherein the determination of whether to adjust selection of the depth plane is positive.

Example 62

The display system of claim61, wherein the virtual object is presented at a depth plane that encompasses the particular depth plane range.

Example 63

The display system of claim58, wherein the fixation depth is within a depth plane range solely encompassed by the adjacent depth plane, and wherein the determination of whether to adjust selection of the depth plane is positive.

Example 64

The display system of claim63, wherein the virtual object is presented at the adjacent depth plane.

Example 65

The display system of claim58, wherein, in response to a positive determination to adjust selection of a depth plane, triggering presentation at the adjusted depth plane in response to detecting performance of a blink or a saccade by the user.

Example 66

The display system of claim58, wherein the operations further comprise:determining a fixation point of a user indicating a three-dimensional location upon which the user is fixating, the three-dimensional location indicating the fixation depth.

Example 67

A display system comprising:a display device configured to present virtual objects to a user at a plurality of depth planes;one or more processors; andcomputer storage media storing instructions that when executed by the display system, cause the display system to perform operations comprising:determining a fixation depth, wherein the fixation depth is a depth at which eyes of the user are fixating;determining a particular depth plane of the plurality of depth planes at which a virtual object is to be presented to the user, the determination based on the fixation depth and depth plane ranges encompassed by each of the depth planes, wherein adjacent depth planes both encompass a depth overlap region; andcausing presentation of the virtual object at the particular depth plane.

Example 68

The display system of claim66, wherein a depth plane range encompassed by a depth plane indicates a range of depths from the user that, when fixated at, cause presentation of the virtual object at the depth plane.

Example 69

The display system of claim66, wherein presentation at the particular depth plane comprises presenting the virtual object with accommodation cues associated with a nominal focal depth of the particular depth plane and vergence cues based on location information associated with the virtual object.

Example 70

The display system of claim69, wherein a size of the depth plane range encompassed by the particular depth plane is based on a accommodation-vergence mismatch tolerance, the accommodation-vergence mismatch toleranceindicating a maximum difference between a perceived depth associated with vergence cues of a presented virtual object and a perceived depth associated with accommodation cues of the virtual object.

Example 71

The display system of claim66, wherein a size of the depth overlap region is based on an error associated with determining fixation depths.

Example 72

The display system of claim66, wherein the fixation depth is solely encompassed by the particular depth plane, such that the depth falls in the depth plane range.

Example 73

The display system of claim66, wherein the fixation depth is within a depth overlap region encompassed by the particular depth plane and an adjacent depth plane, and wherein determining the particular depth plane is based on prior fixation depths of the user.

Example 74

The display system of claim73, wherein the operations further comprise:identifying, based on the prior fixation depths, that the user fixated at a fixation depth solely encompassed by the particular depth plane prior to fixating at one or more fixation depths within the depth overlap region.

Example 75

The display system of claim73, wherein the operations further comprise:determining a subsequent fixation depth of the user that is within the depth overlap region; andcausing presentation of the virtual object at the particular depth plane.

Example 76

The display system of claim73, wherein the operations further comprise:determining a subsequent fixation depth of the user that is encompassed by the adjacent depth plane and outside of the depth overlap region; andcausing presentation of the virtual object at the adjacent depth plane.

Example 77

The display system of claim76, wherein the display system is configured to determine that the user has performed a (1) blink or a (2) saccade, and in response cause presentation at the adjacent depth plane.

Example 78

A method implemented, at least in part, by a display device configured to present virtual objects to a user at a plurality of depth planes, the method comprising:determining a fixation depth, wherein the fixation depth is a depth at which eyes of the user are fixating;determining whether to adjust selection of a selected depth plane at which a virtual object is being presented to the user's eyes based on whether the fixation depth is within a depth plane range that is:(1) solely encompassed by the selected depth plane; or(2) encompassed by both the selected depth plane and an adjacent depth plane; andcausing presentation of the virtual object at a particular depth plane selected based on the determination whether to adjust selection of the selected depth plane.

Example 79

The method of claim78, wherein the fixation depth is within a particular depth plane range outside of a depth plane range that is:(1) solely encompassed by the selected depth plane; and(2) encompassed by both the selected depth plane and an adjacent depth plane,and wherein the determination of whether to adjust selection of the depth plane is positive.

Example 80

The method of claim78, wherein in response to a positive determination to adjust selection of a depth plane, triggering presentation at the adjusted depth plane in response to detecting performance of a blink or a saccade by the user.

Example 81

A display system comprising:a display device configured to present virtual objects to a user at a plurality of depth planes;processors; andcomputer storage media storing instructions that when executed by the display system, cause the display system to perform operations comprising:determining a fixation point of a user indicating a three-dimensional location upon which the user is fixating;determining whether to switch a depth plane at which a virtual object is to be presented, the determination based, at least in part, on a depth of the determined fixation point; andswitching the depth plane at which the virtual object is to be presented, wherein switching the depth plane is triggered by a user perception limiting event.

Example 82

The display system of claim81, wherein the operations further comprise:monitoring the user's eyes to detect the user perception limiting event comprising one or more of a blink or a saccade being performed.

Example 83

The display system of claim82, wherein monitoring the user's eyes comprises monitoring the user's pupils, and wherein detecting a saccade is based on a rotational velocity of the pupils exceeding a threshold velocity.

Example 84

The display system of claim83, wherein detecting the saccade is further based on movement information associated with the user's head.

Example 85

The display system of claim81, wherein switching the depth plane comprises:detecting a blink being performed by the user; andin response, switching the depth plane.

Example 86

The display system of claim81, wherein switching the depth plane comprises:detecting a saccade being performed by the user; andin response, switching the depth plane.

Example 87

The display system of claim81, wherein switching the depth plane comprises:in response to not detecting performance of a blink or a saccade after a threshold amount of time, switching the depth plane.

Example 88

The display system of claim81, wherein determining whether to switch a depth plane comprises:determining that the depth of the determined fixation point is encompassed by the switched depth plane.

Example 89

The display system of claim81, wherein the operations further comprise:storing information indicating that the depth plane is to be switched, and monitoring eyes of the user to determine the user perception limiting event.

Example 90

The display system of claim81,wherein the display device comprises a plurality of stacked waveguides forming a display area and providing a view of an ambient environment through the display area, wherein at least some waveguides of the plurality of waveguides are configured to output light with different wavefront divergence than other waveguides, each waveguide being associated with a depth plane; andwherein presenting the virtual object at the switched depth plane comprises a waveguide associated with the switched depth plane outputting light for forming the virtual object.

Example 91

A method implemented, at least in part, by a display device configured to present virtual objects to a user at a plurality of depth planes, the method comprising:determining a fixation point of a user indicating a three-dimensional location upon which the user is fixating;determining whether to switch a depth plane at which a virtual object is to be presented, the determination based, at least in part, on a depth of the determined fixation point; andswitching the depth plane at which the virtual object is to be presented, wherein switching the depth plane is triggered by a user perception limiting event.

Example 92

The method of claim91, further comprising: monitoring the user's eyes to detect the user perception limiting event comprising one or more of a blink or a saccade being performed.

Example 93

The method of claim92, wherein monitoring the user's eyes comprises monitoring the user's pupils, and wherein detecting a saccade is based on a rotational velocity of the pupils exceeding a threshold velocity.

Example 94

The method of claim93, wherein detecting the saccade is further based on movement information associated with the user's head.

Example 95

The method of claim91, wherein switching the depth plane comprises:detecting a blink being performed by the user; andin response, switching the depth plane.

Example 96

The method of claim91, wherein switching the depth plane comprises:detecting a saccade being performed by the user; andin response, switching the depth plane.

Example 97

A display system comprising a display device, processors, and computer storage media storing instructions that, when executed by the processors, cause the display system to perform operations comprising:presenting, by the display device, frames including virtual content to a user at a plurality of depth planes, wherein for each frame presented to the user, the virtual content is presented at a same depth plane selected based on user fixation information; andin response to identifying that selection of a selected depth plane is to be adjusted, storing information indicating that, upon detection of a blink or a saccade being performed by the user, one or more frames of the virtual content are to be presented, by the display device, at the adjusted depth plane.

Example 98

The display system of claim97, wherein the operations further comprise:detecting performance of a blink or a saccade; andin response to the determination, presenting the virtual content at the adjusted depth plane.

Example 99

The display system of claim97, wherein the operations further comprise:determining that the user has not performed a blink or a saccade for greater than a threshold time; andin response to the determination, presenting the virtual content at the adjusted depth plane.

Example 100

The display system of claim97, wherein the operations further comprise:monitoring the user's eyes to detect a blink or a saccade, and while monitoring, presenting one or more frames of virtual content at the selected depth plane.

Example 101

The display device of claim97, wherein while waiting for detection of a blink or a saccade, determining that the user is fixating at a depth associated with a particular depth plane different from the adjusted depth plane, and storing information indicating that, upon detection of a blink or a saccade, the virtual content is to be presented at the particular depth plane.

