The Importance of Visual Attention in Improving the 3D-TV Viewing Experience: Overview and New Perspectives <s> F. Decoding and Rendering of 3D Video <s> The transmission of 3DTV sequences over packet based networks may result in degradations of the video quality due to packet loss. In the conventional 2D case, several different strategies are known for extrapolating the missing information and thus concealing the error. In 3D however, the residual error after concealment of one view might leads to binocular rivalry with the correctly received second view. In this paper, three simple alternatives are presented: frame freezing, a reduced playback speed, and displaying only a single view for both eyes, thus effectively switching to 2D presentation. In a subjective experiment the performance in terms of quality of experience of the three methods is evaluated for different packet loss scenarios. Error-free encoded videos at different bit rates have been included as anchor conditions. The subjective experiment method contains special precautions for measuring the Quality of Experience (QoE) for 3D content and also contains an indicator for visual discomfort. The results indicate that switching to 2D is currently the best choice but difficulties with visual discomfort should be expected even for this method. <s> BIB001 </s> The Importance of Visual Attention in Improving the 3D-TV Viewing Experience: Overview and New Perspectives <s> F. Decoding and Rendering of 3D Video <s> This paper investigates the subjective impact resulting from different concealment methods for coping with lost frames in 3D video communication systems. It is assumed that a high priority channel is assigned to the main view and only the auxiliary view is subject to either transmission errors or packet loss, leading to missing frames at the decoder output. Three methods are used for frame concealment under different loss ratios. The results show that depth is well perceived by users and the subjective impact of frame loss not only depends on the concealment method but also exhibits high correlation with the disparity of the original sequence. It is also shown that under heavy loss conditions it is better to switch from 3D to 2D rather than presenting concealed 3D video to users. <s> BIB002
At the display side, another format conversion may occur depending on the signal representation used for transmission or if a different viewpoint needs to be rendered. Depth Image Based Rendering (DIBR) approaches that rely on depth/disparity maps are frequently used. These render the stereo pair before the display, producing a dedicated image for the left and the right eye. Because at least one viewpoint differs slightly from the transmitted view, inpainting algorithms are needed to fill the previously occluded image regions. The inaccuracy of the inpainting often produces artifacts around the edges of objects. 3D rendering may require an estimation of the depth or disparity. Estimation of depth from at least two views is likely to produce artifacts, mostly because of the ambiguity of image features. Existing 2D decoders usually employ some kind of error concealment techniques or freeze the last correctly decoded video frame when transmission errors occur. In the case of stereoscopic video transmitted as two streams, there is also the crucial issue of keeping the streams completely synchronized, especially if the decoder employs the frame freezing strategy when transmission errors occur. Indeed, a strong effect of binocular rivalry occurs in the case of simulcast transmission if the error concealment method leads to a temporal de-synchronization: the two stereoscopic images would belong to different object or camera positions and thus fusion might be difficult or impossible to achieve. Besides temporal de-synchronization, spatial error concealment strategies may have a major impact on the perceived quality. In the 2D transmission case, error concealment methods are applied to predict the content of the missing image regions using spatial and temporal information available from the bitstream. Recently, it has been shown that such an error concealment strategy does not necessarily improve the quality of experience (QoE) for 3D videos in the same way. For transmission outages that affect the content over more than a few frames, switching back to 2D presentation seems to be preferred to either concealing the erroneous frames or to pausing the playback while staying in 3D presentation mode BIB001 . Another study subjectively evaluated the quality drop due to frames lost frequently, for example every other frame was lost in one view. In this case, it seems better to stay in the 3D presentation mode and to pause for one to three frames rather than switching to 2D for a single correctly received frame every two to four frames BIB002 . It may be anticipated that the switching between 2D and 3D presentation mode has an important impact on the annoyance of the viewer. This effect may be limited by determining the main region of interest and aligning the corresponding object to the display plane. Another alternative would be to apply an adapted error concealment method that conceals the erroneous regions in 2D or 3D in function of their visual importance.