CROSS-REFERENCE TO RELATED APPLICATION

The present application claims benefit to Swedish patent application No. 1951422-3, filed Dec. 19, 2019, entitled “EYE EVENT DETECTION”, and is hereby incorporated by reference in its entirety.

FIELD

The present disclosure generally relates to detecting eye events related to a user of an eye tracking system. More specifically, the present disclosure relates to a method and system for detecting an eye event of a user and performing an appropriate action.

BACKGROUND

Eye tracking systems monitor the eyes of a user to determine a gaze position and/or direction of the user, and then use this information to control an apparatus. For example, such systems can detect if a user is looking at an icon on a screen, and can interact with that icon on a screen if the user's gaze settles on the icon for a given period of time.

One of the drawbacks of current eye tracking systems is latency. This is the time interval between the occurrence of an eye event, for example a movement of the eyeball or a blink, and the time at which the event is processed by the eye tracking system. In current systems, latency can be due to the frame rates of cameras used to monitor the eyes, data transfer between different parts of the system, processing of data to calculate relevant outputs, and other delays inherent to eye tracking systems.

As a user's gaze moves around, information on when and where the gaze is moving is fundamental to providing responsive and reliable eye tracking functionality. Quick and accurate processing of eye events can significantly increase the performance of an eye tracking system. This is particularly true in systems that use foveated rendering, where rendering quality is reduced outside of a particular gaze zone of the user in order to reduce the workload of the system.

Therefore, a system and method is required that can reduce the latency present in current eye tracking systems.

SUMMARY

The present disclosure provides a method and system for detecting an eye event using an eye tracking system that reduces the latency between the occurrence of the eye event and a corresponding action performed by the system. This is achieved by interlacing image capture from one eye with image capture from the other eye.

In current eye tracking systems, images of both left and right eyes are captured simultaneously. Successive images are required to determine if an eye event has occurred, based on detecting a difference between the images. Therefore, the speed at which eye events can be detected is limited by the frame rates of cameras that are used to monitor the eyes. However, it has been determined that the left and right eye move generally in sync. That is to say, when an eye event occurs, for example, a movement of the eyeball or a blink, the event occurs in both the left and right eyes simultaneously. As such, the eye event need only be detected in one of the two eyes for relevant information to be captured and used by the system.

In the present disclosure, this information is used to implement a system where the left and right eyes are monitored separately, and image capture from one eye is interlaced with image capture from the other eye. Therefore, successive images of a given eye can be captured earlier than in cases where the eyes are imaged simultaneously, and eye events can be detected sooner after they occur. By taking this approach, the frame rate of image capture can be effectively doubled, and so the time from an eye event occurring to it being detected can be reduced. On average, the time from an eye event occurring to it being detected is reduced by a factor of two, although higher or lower factors of reduction are also possible. This in turn reduces the overall latency of the system.

In accordance with an aspect of the disclosure, there is provided a method for detecting an eye event of a user using an eye tracking system, the method comprising capturing a first image of a first eye of a user, capturing an image of a second eye of the user a first period after capturing the first image of the first eye and a second period before capturing a next image of the first eye, capturing a second image of the first eye the second period after capturing the image of the second eye, determining that an eye event has occurred based on a difference between the first and second images of the first eye, and performing at least one action if it is determined that that an eye event has occurred.

Optionally, the difference between first and second images comprises a difference in at least one characteristic for use in determining a gaze direction of the user. Optionally, the characteristic is related to the position of at least one corneal reflection from the eye. Optionally, the characteristic is related to the position of a pupil of the user. Optionally, the method further comprises determining that the eye event is the start of a saccade based on a difference between the first and second images.

Optionally, performing at least one action comprises determining an anticipated gaze point of the user. Optionally, the method comprises determining the anticipated gaze point based on at least one image captured after the start of the saccade. Optionally, the anticipated gaze point is determined based on captured images of the first and second eyes. Optionally, determining the anticipated gaze point comprises weighting later-captured images more heavily than earlier-captured images.

Optionally, the difference between first and second images comprises a difference in at least one characteristic for use in determining an eye openness of the user. Optionally, the characteristic is at least one of a pixel intensity of the first and second images, a number of corneal reflections shown in the first and second images, an obscured area of the pupil of the user, or an obscuration of the centre of the pupil of the user. Optionally, the method further comprises determining that the eye event is a blink based on an eye openness of the user. Optionally, performing at least one action comprises adjusting a power level of an associated device based on the eye openness.

Optionally, determining that an eye event has occurred comprises comparing the difference between the first and second images of the first eye to a threshold, and determining that an eye event has occurred if the difference is above the threshold. Optionally, capturing the images of the first eye is performed using at least one first image capture device, and capturing the image of the second eye is performed using at least one second image capture device.

In accordance with another aspect of the disclosure, there is provided an eye tracking system comprising at least one first image capture device configured to capture images of a first eye of a user, at least one second image capture device configured to capture images of a second eye of the user, and a processor, wherein the system is configured to capture a first image of the first eye, capture a first image of the second eye a first period after capturing the first image of the first eye and a second period before capturing a next image of the first eye, capture a second image of the first eye the second period after capturing the first image of the second eye, determine that an eye event has occurred based on a difference between the first and second images of the first eye, and perform at least one action if it is determined that that an eye event has occurred. Optionally, a head-mountable apparatus comprises the eye tracking system.

In accordance with another aspect of the disclosure, there is provided a computer program comprising instructions which, when executed on at least one processor, cause the at least one processor to control an eye tracking system to capture a first image of a first eye of a user, capture a first image of a second eye of the user a first period after capturing the first image of the first eye and a second period before capturing a next image of the first eye, capture a second image of the first eye the second period after capturing the first image of the second eye, determine that an eye event has occurred based on a difference between the first and second images of the first eye, and perform at least one action if it is determined that that an eye event has occurred. Optionally, a carrier containing the computer program is one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

By taking this approach, many advantages are realised. For example, faster detection of eye events can improve the performance of gaze prediction and foveated rendering. The recovery time after a blink occurs, when a gaze point cannot be detected, is also reduced. In some cases, the frame rates of the individual cameras can be reduced without reducing the overall frame rate of the system. This can reduce the power consumption and processing load of the system. In some cases, a balance can be struck between a reduction of latency and a reduction of power consumption and processing load by setting the frame rates of the cameras appropriately.

