BACKGROUND

Engineers use compression (also called source coding or source encoding) to reduce the bit rate of digital video. Compression decreases the cost of storing and transmitting video information by converting the information into a lower bit rate form. Decompression (also called decoding) reconstructs a version of the original information from the compressed form. A “codec” is an encoder/decoder system.

Over the last two decades, various video codec standards have been adopted, including the ITU-T H.261, H.262 (MPEG-2 or ISO/IEC 13818-2), H.263 and H.264 (MPEG-4 AVC or ISO/IEC 14496-10) standards, the MPEG-1 (ISO/IEC 11172-2) and MPEG-4 Visual (ISO/IEC 14496-2) standards, and the SMPTE 421M (VC-1) standard. More recently, the H.265/HEVC standard (ITU-T H.265 or ISO/IEC 23008-2) has been approved. Extensions to the H.265/HEVC standard (e.g., for scalable video coding/decoding, for coding/decoding of video with higher fidelity in terms of sample bit depth or chroma sampling rate, for screen capture content, or for multi-view coding/decoding) are currently under development. A video codec standard typically defines options for the syntax of an encoded video bitstream, detailing parameters in the bitstream when particular features are used in encoding and decoding. In many cases, a video codec standard also provides details about the decoding operations a decoder should perform to achieve conforming results in decoding. Aside from codec standards, various proprietary codec formats define other options for the syntax of an encoded video bitstream and corresponding decoding operations.

During decoding, a video codec can apply one or more filters to reduce the difference between the reconstructed pixels and the original pixels. For example, for the H.265/HEVC standard, two processing steps—a deblocking filter (DBF) followed by a sample adaptive offset (SAO) operation—are available for application to the reconstructed samples before writing them into the decoded picture buffer in the decoder loop. The deblocking filter is intended to reduce the blocking artifacts due to block-based coding. The H.265/HEVC deblocking filter is similar to the deblocking filter of the H.264/MPEG-4 AVC standard, whereas the SAO filter is newly introduced in the H.265/HEVC standard. The deblocking filter is only applied to the samples located at block boundaries. By contrast, the SAO filtering operation is applied adaptively to all samples satisfying certain conditions, such as gradient.

Whether the SAO filtering operation is performed for a reconstructed picture can be controlled by a syntax element (sample_adaptive_offset_enabled_flag) that is part of a given sequence parameter set More specifically, sample_adaptive_offset_enabled_flag equal to 1 specifies that the sample adaptive offset process is applied to the reconstructed picture after the deblocking filter process, whereas sample_adaptive_offset_enabled_flag equal to 0 specifies that the sample adaptive offset process is not applied to the reconstructed picture after the deblocking filter process. If sample adaptive offset filtering is specified to be applied, additional flags are used to separately indicate whether SAO filtering is enabled for picture portions at a smaller level of granularity as well as for luma components and chroma components. This additional signaling can be performed, for instance, on a slice-by-slice basis using the slice_sao_luma_flag and the slice_sao_chroma_flag. Specifically, slice_sao_luma_flag equal to 1 specifies that SAO is enabled for the luma component in the current slice; slice_sao_luma_flag equal to 0 specifies that SAO is disabled for the luma component in the current slice. When the slice_sao_luma_flag is not present, it is inferred to be equal to 0. Further, slice_sao_chroma_flag equal to 1 specifies that SAO is enabled for the chroma component in the current slice; slice_sao_chroma_flag equal to 0 specifies that SAO is disabled for the chroma component in the current slice. When the slice_sao_chroma_flag is not present, it is inferred to be equal to 0. Still further, SAO filtering can be specified at the coding-tree-unit (CTU) level as well. For example, the sao_type_idx_luma and sao_type_idx_chroma syntax elements can be used to enable (or disable) SAO filtering and select the type of SAO filtering to perform for a given CTU. Specifically, for either luma or chrominance CTU, an SAO_type_idx value of 0 indicates that no SAO filtering is enabled, a value of 1 indicates that band-offset-type SAO filtering (BO) is enabled, and a value of 2 indicates that edge-offset-type SAO filtering (EO) is enabled.

The H.265/HEVC standard provides little to no guidance as to when and how SAO filtering should be enabled (e.g., by specifying the sample_adaptive_offset_enabled_flag, the slice_sao_luma_flag, slice_sao_chroma_flag, sao_type_idx_luma, and/or sao_type_idx_chroma syntax elements) or how and when associated SAO parameters should be set (e.g., by specifying the sao_offset_abs, sao_offset_sign, sao_band_position, sao_eo_class_luma, sao_eo_class_chroma, and/or SaoOffsetVal syntax elements). On the other hand, the use of SAO filtering can adversely affect the computational efficiency and processing time for encoding and decoding operations. Accordingly, there are many conditions and situations where SAO is desirably not enabled. This application discloses several approaches for determining when and how to enable SAO filtering operations.

SUMMARY

In summary, the detailed description presents innovations in the area of encoding pictures or portions of pictures (e.g., slices, coding tree units, or coding units) and determining whether and how certain filtering operation should be performed and specified for performance by the decoder in the bitstream. In particular examples, various implementations for setting the sample adaptive offset (SAO) syntax elements in the H.265/HEVC standard are disclosed. Although these examples concern the H.265/HEVC standard and its SAO filtering process, the disclosed technology is more widely applicable to other video codecs that involve filtering operations (particularly multi-stage filtering operations) as part of their encoding and decoding processes.

In one example embodiment, a multi-stage filtering process is applied to a picture portion currently being encoded. The multi-stage filtering process comprises a deblocking filter and a sample adaptive offset (SAO) process that is applied to post-deblocking pixels of the picture portion. In this embodiment, one or more SAO parameters that control application of the SAO process are specified using one or more pre-deblocking pixels of the picture portion.

In another example embodiment, for a picture portion currently being encoded, a determination is made as to whether to apply a second stage of filtering (e.g., SAO filtering) for a multi-stage filtering process based at least in part on data that is indicative of the amount of residual data for the picture portion resulting from the encoding. For example, the amount of residue may be used or a syntax element that is indicative of the residual but not directly associated with SAO filtering may be used. For example, the syntax element controlling whether a picture portion is to be encoded using a “skip mode” can be used.

In a further example embodiment, for a picture portion currently being encoded, a determination is made as to whether to apply a second stage of filtering (e.g., SAO filtering) for a multi-stage filtering process based at least in part on the encoding efficiency of the video encoder or image encoder. For instance, the processing time or number of processing cycles for encoding the current picture or picture portion can be used as the basis for enabling or disabling SAO filtering. The encoding time or number of processing cycles for previously encoded pictures or picture portions can alternatively or additionally be used.

In still a further example embodiment, for a series of pictures or picture portions being encoded, the second stage of filtering (e.g., SAO filtering) of a multi-stage filtering process is periodically applied. For instance, SAO filtering can be applied every n pictures or picture portions, where n is any integer.

In yet a further example embodiment, an edge-offset (EO) process is applied to a picture portion currently being encoded, including applying an edge filter to the picture portion in order to identify one of multiple available EO classes to use. In particular implementations, each of the multiple available EO classes is associated with a different direction along which pixel values will be evaluated during subsequent EO processing. The subsequent EO processing can comprise, for example, determining whether the pixels in the picture portion along the direction of the identified EO class include a local valley, a local peak, a concave corner, or a convex corner, and thereby determining whether positive offset values or negative offset values are to be applied during EO processing.

The innovations can be implemented as part of a method, as part of a computing device adapted to perform the method or as part of a tangible computer-readable media storing computer-executable instructions for causing a computing device to perform the method. The various innovations can be used in combination or separately.

The foregoing and other objects, features, and advantages of the invention will become more apparent from the following detailed description, which proceeds with reference to the accompanying figures.

