The Importance of Visual Attention in Improving the 3D-TV Viewing Experience: Overview and New Perspectives <s> D. Content Repurposing and Depth Adaptation <s> Watching TV shows on cell phones is starting to become a reality. Nevertheless, there still exist some significant issues due to the small size of cell phone screens. The direct transfer of contents that are not specifically shot for the mobile device will provide indistinguishable objects. An automated way, delivering the best viewing experience is proposed in this paper. This solution significantly improves the visual comfort, by zooming in on the regions of interest. The relevance of this solution rests on its capability to preserve the visually important areas as well as the temporal stability. Eye-tracking experiments are one metric to assess the reframing quality. Involving 16 observers, they show that more than 90% of the visually important regions are kept in the reframed clip. <s> BIB001 </s> The Importance of Visual Attention in Improving the 3D-TV Viewing Experience: Overview and New Perspectives <s> D. Content Repurposing and Depth Adaptation <s> The automatic content repurposing solution proposed, enables broadcasters to adapt traditional content to the specific constraints of hand- held devices, both automatically and in realtime . The goal is to provide customers with the best possible viewing experience. Automatic content repurposing solution rests mainly on the dynamic reframing (also called retargeting) of content. It aims at identifying the regions of interest and zooming in on these regions. Although initially dedicated to mobile TV, this solution could also target broadcast, PDAs, portable devices, or internet and web publishing. Thanks to this new and unique approach, broadcasters and content owners now have the ability to adapt content for multiple distribution platforms in a cost-effective way. Compared to the classical approach, the proposed reframing solution delivers a superior viewing experience by focusing on the most visually interesting areas. Based on a human visual model, this unique solution dramatically reduces the repurposing tasks. <s> BIB002 </s> The Importance of Visual Attention in Improving the 3D-TV Viewing Experience: Overview and New Perspectives <s> D. Content Repurposing and Depth Adaptation <s> 3D processing techniques are really promising. However, several hurdles have to be overcome. In this paper, two of ::: them are examined. The first is related to the high disparity management. It is currently not well mastered and its impact ::: is strong for viewing 3D scene on stereoscopic screens. The second concerns the salient regions of the scene. These areas ::: are commonly called Region-Of-Interest (RoI) in the image processing domain. The problem appears when there are ::: more than one region-of-interest in a video scene. Indeed, it is then complicated for the eyes to scan them and especially ::: if the depth difference between them is high. In this contribution, the 3D experience is improved by applying some ::: effects related to RoIs. The shift between the two views is adaptively adjusted in order to have a null disparity on a given ::: area in the scene. In the proposed approach, these areas are the visually interesting areas. A constant disparity on the ::: salient areas improves the viewing experience over the video sequence. <s> BIB003
Content repurposing (also termed reframing) of 2D video content is used to address the problem of aspect ratio difference between the content and the screen (e.g., content with cinema aspect ratio shown on a display with a different aspect ratio). Without content repurposing, either black borders have to be inserted around the image (top/bottom or left/right) to fit the aspect ratio of the new target screen or geometric distortion will have to be applied on the content to fit the screen. Either approach will worsen the visual experience. To adapt 2D video content to a display with a different aspect ratio, content repurposing usually involves a combination of cropping and zooming (re-scaling), especially for viewing on small devices. In this case, cropping is done around a region of interest selected to preserve the most important information in the content, i.e., the most visually important areas. Currently, determining the coordinates of the cropping window for content repurposing is a highly manual process, which can prove to be very time-consuming and expensive, or is performed simply to retain the center of the picture regardless of the content. A visual attention model could be used to make this process faster, more automated, and adaptive to the content as it would analyze the content to predict the main area of interest to retain in the reframing process. Such tool would be particularly very useful for the broadcasting of live events. Attention-based video reframing has already been proposed in the literature for 2D video BIB001 - BIB002 but a similar technology would be useful in the framework of 3D-TV broadcasting. However, repurposing of 3D content needs to address two additional important issues. The first one is the border effect, which needs to be avoided as cutting objects appearing in front of the screen inhibits the perception of depth. The second one is depth adaptation. Creation of 3D stereoscopic content cannot be disconnected from the display and viewing conditions because both depth perception and visual comfort are highly dependent on the screen size and viewing distance, for a given content disparity. Therefore, a given content is currently produced for a given set of screen size and viewing distance. However, these factors are extremely different in cinema, home television, and on a portable device. Therefore, depth adaptation of stereoscopic content initially produced for a given viewing scenario is necessary for usage in a different one. Content adaptation from cinema environment with its large field of view to the home environment with a narrower viewing angle and shorter viewing distance is currently technically very challenging. It is also a very time-consuming and manual process. More automated content-based post-production or post-processing tools to help adapt 3D content to television are required. Again, a 3D visual attention model would provide the area of interest and convergence plane to drive the content repurposing of stereoscopic content. In addition to the necessary depth adaptation for 3D content repurposing, the adaptation of the scene depth can be used in order to improve visual comfort. The adaptive rendering of a 3D stereoscopic video based on identification of a main region of interest has been proposed using a 2D visual attention model BIB003 . The adaptation of the convergence plane of the main area of interest is used to reduce visual fatigue induced by a continuous need to change the plane of convergence when the main area of interest is moving across different depth levels. A way to reduce such strain is to modify the convergence plane of the main area of interest to place it on the screen plane, i.e., by adapting the content disparity. In order to achieve this, two steps are used. Firstly, a visual attention model is used to compute the saliency maps that indicate the importance of visual interest of all pixels in each view. Secondly, disparity information (depth or disparity map) is used to refine these saliency maps in order to select one dominant area of interest. The disparity information is also used to compute the necessary shift in disparity that has to be applied between the stereo-views to place the main area of interest on the zero-parallax plane. A succession of shifting, cropping and scaling is necessary to achieve this. The drawback of the proposed approach is that cropping may introduce border effects if an object appearing in front of the screen and initially placed very near a border is cut in the reframing process. Scaling may also slightly deform the objects. The effectiveness of this approach also depends on the availability and quality of the disparity map. The computation of high-quality disparity maps is still a difficult problem to be solved. Therefore, the approach based on the combination of computed disparity maps and a 2D visual attention model may suffer from the errors in these disparity maps. A proper 3D visual attention model may overcome this drawback.