The Importance of Visual Attention in Improving the 3D-TV Viewing Experience: Overview and New Perspectives <s> D. Discussion on Issues and Challenges <s> Using eye tracking for the investigation of visual attention has become increasingly popular during the last few decades. Nevertheless, only a small number of eye tracking studies have employed 3D displays, although such displays would closely resemble our natural visual environment. Besides higher cost and effort for the experimental setup, the main reason for the avoidance of 3D displays is the problem of computing a subject's current 3D gaze position based on the measured binocular gaze angles. The geometrical approaches to this problem that have been studied so far involved substantial error in the measurement of 3D gaze trajectories. In order to tackle this problem, we developed an anaglyph-based 3D calibration procedure and used a well-suited type of artificial neural network—a parametrized self-organizing map (PSOM)—to estimate the 3D gaze point from a subject's binocular eye-position data. We report an experiment in which the accuracy of the PSOM gaze-point estimation is compared to a geometrical ... <s> BIB001 </s> The Importance of Visual Attention in Improving the 3D-TV Viewing Experience: Overview and New Perspectives <s> D. Discussion on Issues and Challenges <s> This paper describes a novel real-time 3D gaze estimation system. The system consists of two cameras and two IR light sources. There are three novelties in this method. First, in our system, two IR lights are mounted near the centers of the stereo cameras, respectively. Based on this specific configuration, the 3D position of the corneal center can be simply derived by the 3D reconstruction technique. Then, after extracting the 3D position of the "virtual pupil" correctly, the optical axis of the eye can be obtained directly by connecting the "virtual pupil" with the corneal center. Second, we systematically analyze the noise in our 3D gaze estimation algorithm and propose an effective constraint to reduce this noise. Third, to estimate the user-dependent parameters (i.e. the constraint parameters and the eye parameters), a simple calibration method is proposed by gazing at four positions on the screen. Experimental results show that our system can accurately estimate and track eye gaze under natural head movement. <s> BIB002
Several research works, which have examined the influence of disparity and depth on visual attention, support the fact that depth provides a salient image feature. More importantly, these works provide some evidence that results of other past studies using 2D stimuli cannot be automatically generalized to 3D stimuli as the introduction of disparity information may change the deployment of visual attention. A few computational models of 3D visual attention have been proposed. However, most of these works usually do not report results of formal subjective experiments to evaluate the performance of the proposed models. This raises an important question concerning the collection of ground-truth data of eye movements in the context of 3D-TV. From the existing literature describing experimental work examining visual attention, it is apparent that there is a lack of standard protocols for conducting eye-tracking experiments in studies related to visual attention modeling and a lack of agreed procedures for post-hoc analysis of eye-tracking data. These issues already exist for the case of 2D video and still apply in the case of 3D. Furthermore, additional hurdles related to the recording of binocular data and their interpretation in the 3D sense have to be considered in the study of 3D visual attention. Although some researchers have started to discuss modeling of 3D visual attention, the very first challenge that needs to be addressed is how to reliably collect and interpret ground-truth data. Most studies on 2D visual attention have been using monocular eye-trackers or binocular eye-trackers with the option of using the data from only one eye. In other words, past studies on visual attention have considered a similar behavior for each eye when recording eye movements of a participant viewing a 2D image or video. Although it is recognized that both eyes will not necessarily produce an identical gaze point on the screen when viewing a 2D content, this difference may not be relevant when considering a 2D saliency map. However, this difference of gaze location of each eye in a plane and in depth may prove to be more critical when trying to identify the 3D gaze point of both eyes. For the recording of 3D gaze, binocular recordings are necessary. Yet, such eye-tracking equipment can only provide a two-dimensional spatial gaze location individually for each eye. These data then need to be extrapolated or processed to provide a notion of depth in relation with gaze direction or location BIB001 - BIB002 . The principles are similar to the ones involved in retinal disparity. By using two images of the same scene obtained from slightly different angles, it is possible to triangulate the distance to an object with a high degree of accuracy. If an object is far away, the disparity of that image falling on both retinas will be small. If the object is close or near, the disparity will be large. However, the triangulation of the two 2D gaze points from both eyes to produce a single 3D gaze point is not straightforward and is also dependent on the calibration of the system. For an experiment using 2D stimuli, calibration points are typically shown at different spatial locations on the screen and the observer is required to look at these points in order to calibrate the eye-tracker. In this case, it is easy to determine if the observer is looking accurately at the point since the 2D coordinates are known and the 2D gaze can be accurately tracked on the plane of the screen. However, in the case of an experiment using 3D stimuli, this calibration procedure now requires a volumetric calibration, e.g., by showing points at different locations and different depth planes. Therefore, the problem is to find a procedure that ensures that the observer has accurately looked at the point at the given depth plane. Past studies examining 2D visual attention on video tend to show that the HVS is more sensitive to motion contrasts compared to intensity or orientation contrasts. However it is not clear whether this is true in 3D. Further studies are required to examine visual attention in situations of opposing contrasts of motion, color and disparity. For example, where is visual attention drawn to in a scenario where motion contrasts are strong in the background but where there is a pop-out effect with a static object in the foreground? Finally, existing studies on 3D visual attention have not discussed a 3D representation of saliency maps, as saliency maps are still currently considered purely in the 2D sense, i.e., saliency maps are still represented as flat maps.