BACKGROUND

The present invention relates to methods and apparatus for diagnosing cognitive impairment of a subject. In particular, the present invention relates to methods and an apparatus for acquisition of eye movement data. More particularly, the present invention provides methods and an apparatus for acquisition of eye movement data and gaze information.

According to embodiments of the present invention, techniques for processing information associated with eye movement using web based image-capturing devices is disclosed. Merely by way of example, the invention can be applied to analysis of information for determining cognitive performance of subjects.

Historically, recognition memory of a subject has been assessed through conventional paper-pencil based task paradigms. Such tests typically occur in a controlled environment (e.g. laboratory, doctor's office, etc.) under the guidance of a test administrator using expensive (e.g. $10 k-$80K) systems. Such tests also require the subject to travel to the laboratory and spend over an hour preparing for and taking such tests. Typically, a test administrator shows a series of visual stimuli to subjects at a certain frequency and rate. After the exposure phase the user waits for a time delay of over twenty-five minutes before the test administrator tests the subject's recall of the visual stimuli. In addition to the visual stimuli and the test administrator, visual recognition memory paradigms also require response sheets to facilitate administrator scoring. Although effective, conventional paradigms are expensive, cumbersome, and subjective.

From the above, it is seen that techniques for improving acquisition of eye movement data are highly desired.

SUMMARY

According to the present invention, techniques for processing information associated with eye movement using web based image-capturing devices. Merely by way of example, the invention can be applied to analysis of information for determining cognitive performance of subjects.

In an example, the present invention provides a method for identifying a feature of an eye of a human user. The method includes initiating an image capturing device, such as a camera or other imaging device. In an example, the image capturing device comprises a plurality of sensors arranged in an array. In an example, the method includes capturing video information from a facial region of the human user using the image capturing device. In an example, the video information is from a stream of video comprising a plurality of frames.

In an example, the method includes processing the video information to parse the video information into the plurality of images. In an example, each of the plurality of images has a time stamp from a first time stamp, a second time stamp, to an Nth time stamp, where N is greater than 10, or other number.

In an example, the method includes processing each of images to identify a location of the facial region and processing each of the images with the location of the facial region to identify a plurality of landmarks associated with the facial region. In an example, the method includes processing each of the images with the location of the facial regions and the plurality of landmarks to isolation a region including each of the eyes. The method includes processing each of the regions, frame by frame, to identify a pupil region for each of the eyes. In an example, the region is configured as a rectangular region having an x-axis and a y-axis to border each of the eyes of the human user.

In an example, the processing comprises at least: processing the region using a grayscale conversion to output a grayscale image; processing the grayscale image using an equalization process to output an equalized image; processing the equalized image using a thresholding process to output a thresholded image; processing the thresholded image using a dilation and erosion process to output a dilated and eroded image; and processing the dilated and eroded image using a contour and moment process to output a finalized processed image. Of course, there can be other variations, modifications, and alternatives.

In an example, the method includes processing the finalized processed image to identify a spatial location of each pupil in the region, each pupil being identified by a two-dimensional spatial coordinate. The method includes processing information associated with each pupil identified by the two d-dimensional spatial coordinate to output a plurality of two-dimensional spatial coordinates, each of which is in reference to a time, in a two-dimensional space. The method then includes outputting a gaze information about the human user. The gaze information includes the two dimensional spatial coordinates each of which is in reference to a time in a two dimensional space.

According to one aspect of the invention a method of processing information including aligning eye movement with an image capturing device for detection of cognitive anomalies is described. One method includes initiating an application, under control of a processor, to output an image of a frame on a display device to a user, the display device being coupled to the processor, the processor being coupled to a communication device coupled to a network of computers, the network of computers being coupled to a server device, initiating a camera coupled to the application to capture a video image of a face of a user, the face of the user being positioned by the user viewing the display device, and displaying the video of the image of the face of the user (e.g. including their eyes, pupils, etc.) on the display device within a vicinity of the frame being displayed. A process includes positioning the face of the user within the frame to align the face to the frame, and capturing an image of the face of the user, processing captured information regarding the image of the face to initiate an image capturing process of eye movement of the user, and outputting an indication on a display after initiation of the image capturing process; and moving the indication spatially to one of a plurality of images being displayed on the display device. A technique includes capturing a video of at least one eye of the human user, while the user's head/face is maintained within the viewing display of the device (e.g. visible to the camera), and one or both eyes of the user moves to track the position of the indication of the display, the image of each eye comprising a sclera portion, an iris portion, and a pupil portion, parsing the video to determine a first reference image corresponding to a first eye position for a first spatial position for the indication; and a second reference image corresponding to a second eye position for a second spatial position of the indication, and correlating each of the other plurality of images to either the first reference image or the second reference image. In various embodiments, the parsing steps may be performed on a user's computing device, or by a remote server.

According to another aspect of the invention, a method for processing information using a web camera, the web camera being coupled to a computing system is disclosed. One technique includes placing a user in front of a display device coupled to the computing system, the computing system being coupled to a worldwide network of computers, initiating a Neurotrack application stored on a memory device of the computing system and initiating the web camera by transferring a selected command from the Neurotrack application. A process may include capturing an image of a facial region of the user positioned in front of the display device, retrieving a plurality of test images from the memory device coupled to the computing system, the plurality of test images comprising a first pair of images, a second pair of images, a third pair of images, etc. (e.g. twentieth pair of images), each of the pair of images being related to each other, and displaying the first pair of the test images on the display device to be viewed by the user. A method may include capturing a plurality of first images associated with a first eye location while the user is viewing the first pair of test images, repeating the displaying of the pairs of images, while replacing one of the previous pairs of images, and capturing of images for a plurality of second pair of images to the twentieth pair of images, each of which while the user is viewing the display device, and capturing a fixation from an initial point having four regions within a vicinity of the initial point during the displaying of the pairs of images, while replacing the previous pair of images, each of the four regions within about one degree visual angle from the initial point, and an associated saccade with the fixation. A process may include processing information to filter the saccade, determining a visual preference using the fixation on the replaced image from the plurality of images; and using the visual preference information to provide the user with feedback.

According to another aspect of the invention, a method for playing a matching game on a host computer is disclosed. One technique may include uploading from the host computer to a remote computer system, a computer network address for a plurality of static images, wherein the plurality of static images comprises a first plurality of static images and a second plurality of static images. A method may include uploading from the host computer to the remote computer system, remote computer system executable software code including: first remote computer system executable software code that directs the remote computer system to display on a display of the remote computer system to a player, only static images from the first plurality of static images but not static images from the second plurality of static images, wherein each of the static images from the first plurality of static images is displayed upon at most half of the display for a first predetermined amount of time, second remote computer system executable software code that directs the remote computer system to inhibit displaying on the display of the remote computer system to the player, at least one static image from the first plurality of static images to the player, for a second predetermined amount of time, third remote computer system executable software code that directs the remote computer system to simultaneously display on the display of the remote computer system to the player, a first static image from the first plurality of static images and a second static image from the second plurality of static images, wherein the first static image and the second static image are displayed upon at most half of the display for a third predetermined amount of time, fourth remote computer system executable software code that directs the remote computer system to capture using a web camera of the remote computer system video data of the player, wherein the video data captures eye movements of the player while the display of the remote computer system is displaying to the player the first static image and the second static image, fifth remote computer system executable software code that directs the remote computer system to create edited video data from a subset of the video data in response to a pre-defined two dimensional area of interest from the video data, wherein the edited video data has a lower resolution than the video data, and sixth remote computer system executable software code that directs the remote computer to provide to the host computer, the edited video data. A process may include determining with the host computer a first amount of time representing an amount of time the player views the first static image and a second amount of time representing an amount of time the player views the second static image, in response to the edited video data, determining with the host computer a viewing relationship for the player between the second amount of time and the first amount of time, in response to the first amount of time and the second amount of time, and determining with the host computer whether the viewing relationship for the player between the second amount of time and the first amount of time exceeds a first threshold and generating a success flag in response thereto. A technique may include providing from the host computer to the remote computer system, an indication that the player is successful, in response to the success flag.

The above embodiments and implementations are not necessarily inclusive or exclusive of each other and may be combined in any manner that is non-conflicting and otherwise possible, whether they be presented in association with a same, or a different, embodiment or implementation. The description of one embodiment or implementation is not intended to be limiting with respect to other embodiments and/or implementations. Also, any one or more function, step, operation, or technique described elsewhere in this specification may, in alternative implementations, be combined with any one or more function, step, operation, or technique described in the summary. Thus, the above embodiment implementations are illustrative, rather than limiting.

