BACKGROUND

Virtual reality (VR) allows users to experience and/or interact with an immersive artificial environment, such that the user feels as if they were physically in that environment. For example, virtual reality systems may display stereoscopic scenes to users in order to create an illusion of depth, and a computer may adjust the scene content in real-time to provide the illusion of the user moving within the scene. When the user views images through a virtual reality system, the user may thus feel as if they are moving within the scenes from a first-person point of view. Similarly, mixed reality (MR) or augmented reality (AR) systems combine computer generated information (referred to as virtual content) with real world images or a real world view to augment, or add content to, a user's view of the world. The simulated environments of VR and/or the mixed environments of MR may thus be utilized to provide an interactive user experience for multiple applications, such as applications that add virtual content to a real-time view of the viewer's environment, interacting with virtual training environments, gaming, remotely controlling drones or other mechanical systems, viewing digital media content, interacting with the Internet, or the like.

SUMMARY

Various embodiments of methods and apparatus for focusing in virtual reality (VR) or augmented reality (AR) devices based on gaze tracking information are described. Embodiments of a VR/AR device such as a headset, helmet, goggles, or glasses (referred to herein as a head-mounted display (HMD)) are described that include a display, left and right optical lenses (referred to herein as eye lenses) located between the display and the user's eyes, and a controller. For AR applications, the HMD may include or be coupled to one or more external video cameras that capture video of the user's environment for display. The external cameras may include an autofocus mechanism that allows the cameras to automatically focus on objects or surfaces in the environment. A gaze tracking system may be included in the HMD for detecting position and movement of the user's eyes.

In conventional AR HMDs, the autofocus mechanism may focus on something that the user is not looking at. In embodiments of an HMD as described herein, for AR applications, the controller may use the gaze tracking information obtained from the gaze tracking system to direct the autofocus mechanism of the external cameras to focus in the direction of the user's gaze so that the external cameras focus on objects in the environment at which the user is currently looking.

In embodiments, for AR or VR applications, the eye lenses may be focusable lenses, and the HMD may use the gaze tracking information to adjust the focus of the eye lenses so that the virtual content that the user is currently looking at has the proper vergence to match the convergence of the user's eyes. The controller may leverage the gaze tracking information to direct the eye lenses to adjust focus so that close objects that the user is looking at appear at the right distance. For closed-circuit AR applications, the eye lenses can be focused to adjust the display vergence to agree with focus of the external cameras. For VR applications, the controller may obtain distance information for virtual content to be displayed on the display panels, and may use this distance information to direct the eye lenses to adjust focus according to the distance of virtual content that the user is currently looking at according to the gaze tracking information.

In some embodiments, adjusting focus of the eye lenses may be applied during playback of recorded video. Depth information may be recorded with the video, or may be derived from the computer graphics. The gaze tracking information may be used to determine the direction of the user's gaze during playback of the video, and the gaze direction can be used to determine depth at the place where the user's gaze is directed. The eye lenses can then be adjusted to provide the appropriate vergence for the part of the scene that the user is looking at.

