TECHNICAL FIELD

The present principles relate generally to video encoding and decoding and, more particularly, to methods and apparatus for adaptive geometric partitioning for video encoding and decoding.

BACKGROUND

Most video coding techniques use prediction plus residual coding to model video images. Other approaches may also consider prediction as a step into some process of signal transformation, like when lifting schemes are used to generate wavelet transform (with or without motion compensation). Prediction is performed on each frame on a partition basis. That is, each frame is partitioned into blocks or sets of nested blocks in a tree structure, and then each block partition is coded by using an intra or inter predictor plus some residual coding. Frame partitioning into blocks is performed by defining a grid of regions, which are normally blocks (called macroblocks) all over the frame and then each of the macroblocks may also be further partitioned in smaller blocks (also called subblocks or sub-macroblocks). Typically, macroblocks on the boundary of objects and/or frame regions with different textures, color, smoothness and/or different motion, tend to be further divided into subblocks in order to make the coding of the macroblock as efficient as possible, with as much objective and/or subjective quality as possible.

In recent studies, tree structures have been shown to be sub-optimal for coding image information. These studies sustain that tree-based coding of images is unable to optimally code heterogeneous regions (here, regions are considered to have a well-defined and uniform characteristic, such as a flat, smooth, or stationary texture) separated by a regular edge or contour. This problem arises from the fact that tree structures are not able to optimally catch the geometrical redundancy existing along edges, contours or oriented textures. This concept implies that adaptive tree partitioning of macroblocks, even if better than simple fixed-size frame partitioning, is still not optimal enough to capture the geometric information included in 2D data for coding purposes in an efficient manner.

Frame partitioning is a process of key importance in efficient video coding. Recent video compression technologies such as the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group-4 (MPEG-4) Part 10 Advanced Video Coding (AVC) standard/International Telecommunication Union, Telecommunication Sector (ITU-T) H.264 recommendation (hereinafter the “MPEG-4 AVC standard”), use a tree-based frame partition. This seems to be more efficient than a simple uniform block partition, typically used in older video coding standards and recommendations such as the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group-2 (MPEG-2) standard and the International Telecommunication Union, Telecommunication Sector (ITU-T) H.263 recommendation (hereinafter the “H.263 Recommendation”). However, tree based frame partitioning still does not code the video information as efficiently as possible, as it is unable to efficiently capture the geometric structure of two-dimensional (2D) data.

Tree-structured macroblock partitioning is adopted in current major video coding standards. The International Telecommunication Union, Telecommunication Sector (ITU-T) H.261 recommendation (hereinafter the “H.261 Recommendation”), the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group-1 (MPEG-1) standard, and the ISO/IEC MPEG-2 standard/ITU-T H.263 recommendation (hereinafter the “MPEG-2 Standard”) support only 16×16 macroblock (MB) partition. The ISO/IEC Moving Picture Experts Group-4 (MPEG-4) Part 10 Advanced Video Coding (AVC) standard/ITU-T H.264 recommendation (hereinafter the “MPEG-4 AVC standard”) simple profile or ITU-T H.263(+) Recommendation support both 16×16 and 8×8 partitions for a 16×16 MB. The MPEG-4 AVC standard supports tree-structured hierarchical macroblock partitions. A 16×16 MB can be partitioned into macroblock partitions of sizes 16×8, 8×16, or 8×8. 8×8 partitions are also known as sub-macroblocks. Sub-macroblocks can be further broken into sub-macroblock partitions of sizes 8×4, 4×8, and 4×4. Turning toFIG. 1, MPEG-4 AVC standard macroblock division sets are indicated generally by the reference numeral100. In particular, macroblock partitions are indicated by the reference numeral110, and sub-macroblock partitions are indicated by the reference numeral120. In recent studies, tree structures have been shown to be sub-optimal for coding image information. Some of these studies demonstrate that tree-based coding systems are unable to optimally code heterogeneous regions separated by a regular edge or contour.

Some prior work on the subject experimentally identified the need for other types of block partitioning than that supplied by simple tree based partitioning for motion compensation. These techniques propose, in addition to tree based block partition, the use of some additional macroblock partitions able to better adapt to motion edges for motion estimation and compensation.

In one prior art approach (hereinafter “the first prior art approach”) within the framework of a H.263 codec, it is proposed to use two additional diagonal motion compensation modes. When one of these modes is selected, concerned macroblocks are partitioned into two similar triangles divided by a diagonal segment. Depending on the coding mode, this goes from lower left corner to upper right corner for one mode, and from upper-left corner to the lower-right one for the second mode. Turning toFIGS. 2A and 2B, additional motion compensation coding modes corresponding to the designated “first prior art approach” described herein are indicated generally by the reference numerals200and250, respectively. The motion compensation coding mode200corresponds to a right-up diagonal edge coding mode, and the motion compensation coding mode250corresponds to a left-up diagonal edge coding mode.

The first prior art approach is very limited in the sense that these modes are simple variations of the 16×8 or 8×16 motion compensation modes by a fixed diagonal direction. The edge they define is very coarse and it is not precise enough to fit the rich variety of edges found in video frames. There is no explicit coding of geometric information, which impairs from having an adapted treatment of this information in the encoder. Two modes are introduced in the list of coding modes, which increases the coding overhead of other coding modes located after these two in the list of modes.

A direct evolution from the first prior art approach relates to three other prior art approaches, respectively referred to herein as the second, third, and fourth prior art approaches. Collectively in these works, a larger set of motion compensation coding modes are introduced than that described in the first prior art approach. The systems described with respect to the second, third, and fourth prior art approaches introduce a large collection of additional coding modes including oriented partitions. These modes are different translated versions of the 16×8, 8×16 modes as well as different translated versions of the modes proposed in the first prior art approach with a zigzag profile. Turning toFIG. 3, motion compensation coding modes relating to the designated “second”, “third”, and “fourth prior art approaches” are indicated generally by the reference numeral300. Eighteen motion compensation coding modes are shown.

As in the case of the first prior art approach, the partitions defined in the second, third, and fourth prior art approaches for motion compensation are very gross and imprecise with video frames content. Even if the set of oriented partitions outnumber those in the first prior art approach, they are still not precise enough for efficient coding of the rich variety of edges found in video frames. In this case, there is no explicit coding of geometric information, which impairs to have an adapted treatment of the geometric information in the encoder. Moreover, the overhead introduced in order to code the much larger set of modes has an even worse effect on the non-directional modes that follow the oriented modes in the list of modes.

A fifth prior art approach proposes the use of intra prediction within the partitions of the oriented modes from the second, third, and fourth prior art approaches, in addition to their former purpose for motion compensation based prediction. The limitations of the fifth prior art approach are inherited from the second, third, and fourth prior art approaches, hence all those stated in previous paragraphs also apply to the fifth prior art approach.

A sixth prior art approaches proposes the most flexible framework from the works found in the literature. The sixth prior art approach proposes the introduction of only 2 modes where segments connecting two boundary points are used to generate block partitions. The first of the proposed motion compensation coding modes divides a macroblock into two partitions separated by a segment connecting two macroblock boundary points. Turning toFIG. 4A, macroblock partitioning according to a first motion compensation coding mode of the designated “sixth prior art approach” described herein is indicated generally by the reference numeral400.

The second proposed mode is based on a primary division of the macroblock into subblocks, and then, each subblock is divided using a segment connecting two points on the boundary of each subblock. Turning toFIG. 4B, macroblock partitioning according to a second motion compensation coding mode of the designated “sixth prior art approach” described herein is indicated generally by the reference numeral450.

Several limitations still exist with respect to the scheme outlined in the sixth prior art approach, and include the following.

In a first limitation related to the sixth prior art approach, block partitioning defined as the connection of two boundary points by a segment is not able to handle, efficiently, cases of more complex boundaries or contours. For this, the sixth prior art approach proposes the division of macroblocks into subblocks, and the use of points connecting segments in every subblock, in order to approximate more complex shapes, which is inefficient.

In a second limitation related to the sixth prior art approach, partitions are only conceived for motion compensation, disregarding the use of some intra coding technique within the generated partitions. This disables the proposed technique to handle uncovering effects (situations where new data appears from behind an object during a sequence), or simply to code information in a non-temporally predictive way in any of the video frames.

In a third limitation related to the sixth prior art approach, partition coding by coding boundary points is not efficient enough in terms of distortion and coding cost. This is because they are not able to properly represent the geometric characteristics of the partitions boundary; hence, they do not properly show the geometric characteristics of the data in the video frame. Indeed, data in video frames typically presents different statistics for geometric information like local orientations and local positions of different video components and/or objects. The simple use of boundary points is unable to reflect such information. Thus, one cannot exploit such statistics for coding purposes.

In a fourth limitation related to the sixth prior art approach, different video compression qualities have different geometric information precision requirements in order to achieve the best distortion versus coding cost trade-off. The sixth prior art approach does not adapt the information sent to encode the block partitions depending on the video compression quality. Moreover, as the sixth prior art approach does not have and/or otherwise describe a proper representation of partition geometric information, the sixth prior art approach cannot favor, if needed, the encoding of some kind of geometric information with higher precision than some other kind of geometric information.

In a fifth limitation related to the sixth prior art approach, the sixth prior art approach does not appear to handle those pixels lying on the boundary of the partitions which are partly on one side of the boundary, and partly on the other side. These pixels should be able, when needed, to mix information from both partition sides.

Turning toFIG. 8, a video encoder capable of performing video encoding in accordance with the MPEG-4 AVC standard is indicated generally by the reference numeral800.

The video encoder800includes a frame ordering buffer810having an output in signal communication with a non-inverting input of a combiner885. An output of the combiner885is connected in signal communication with a first input of a transformer and quantizer825. An output of the transformer and quantizer825is connected in signal communication with a first input of an entropy coder845and a first input of an inverse transformer and inverse quantizer850. An output of the entropy coder845is connected in signal communication with a first non-inverting input of a combiner890. An output of the combiner890is connected in signal communication with a first input of an output buffer835.

A first output of an encoder controller805is connected in signal communication with a second input of the frame ordering buffer810, a second input of the inverse transformer and inverse quantizer850, an input of a picture-type decision module815, an input of a macroblock-type (MB-type) decision module820, a second input of an intra prediction module860, a second input of a deblocking filter865, a first input of a motion compensator870, a first input of a motion estimator875, and a second input of a reference picture buffer880.

A second output of the encoder controller805is connected in signal communication with a first input of a Supplemental Enhancement Information (SEI) inserter830, a second input of the transformer and quantizer825, a second input of the entropy coder845, a second input of the output buffer835, and an input of the Sequence Parameter Set (SPS) and Picture Parameter Set (PPS) inserter840.

A first output of the picture-type decision module815is connected in signal communication with a third input of a frame ordering buffer810. A second output of the picture-type decision module815is connected in signal communication with a second input of a macroblock-type decision module820.

An output of the Sequence Parameter Set (SPS) and Picture Parameter Set (PPS) inserter840is connected in signal communication with a third non-inverting input of the combiner890.

An output of the inverse quantizer and inverse transformer850is connected in signal communication with a first non-inverting input of a combiner825. An output of the combiner825is connected in signal communication with a first input of the intra prediction module860and a first input of the deblocking filter865. An output of the deblocking filter865is connected in signal communication with a first input of a reference picture buffer880. An output of the reference picture buffer880is connected in signal communication with a second input of the motion estimator875. A first output of the motion estimator875is connected in signal communication with a second input of the motion compensator870. A second output of the motion estimator875is connected in signal communication with a third input of the entropy coder845.

An output of the motion compensator870is connected in signal communication with a first input of a switch897. An output of the intra prediction module860is connected in signal communication with a second input of the switch897. An output of the macroblock-type decision module820is connected in signal communication with a third input of the switch897. An output of the switch897is connected in signal communication with a second non-inverting input of the combiner825.

Inputs of the frame ordering buffer810and the encoder controller805are available as input of the encoder800, for receiving an input picture801. Moreover, an input of the Supplemental Enhancement Information (SEI) inserter830is available as an input of the encoder800, for receiving metadata. An output of the output buffer835is available as an output of the encoder800, for outputting a bitstream.

Turning toFIG. 10, a video decoder capable of performing video decoding in accordance with the MPEG-4 AVC standard is indicated generally by the reference numeral1000.

The video decoder1000includes an input buffer1010having an output connected in signal communication with a first input of an entropy decoder1045. A first output of the entropy decoder1045is connected in signal communication with a first input of an inverse transformer and inverse quantizer1050. An output of the inverse transformer and inverse quantizer1050is connected in signal communication with a second non-inverting input of a combiner1025. An output of the combiner1025is connected in signal communication with a second input of a deblocking filter1065and a first input of an intra prediction module1060. A second output of the deblocking filter1065is connected in signal communication with a first input of a reference picture buffer1080. An output of the reference picture buffer1080is connected in signal communication with a second input of a motion compensator1070.

A second output of the entropy decoder1045is connected in signal communication with a third input of the motion compensator1070and a first input of the deblocking filter1065. A third output of the entropy decoder1045is connected in signal communication with an input of a decoder controller1005. A first output of the decoder controller1005is connected in signal communication with a second input of the entropy decoder1045. A second output of the decoder controller1005is connected in signal communication with a second input of the inverse transformer and inverse quantizer1050. A third output of the decoder controller1005is connected in signal communication with a third input of the deblocking filter1065. A fourth output of the decoder controller1005is connected in signal communication with a second input of the intra prediction module1060, with a first input of the motion compensator1070, and with a second input of the reference picture buffer1080.

An output of the motion compensator1070is connected in signal communication with a first input of a switch1097. An output of the intra prediction module1060is connected in signal communication with a second input of the switch1097. An output of the switch1097is connected in signal communication with a first non-inverting input of the combiner1025.

An input of the input buffer1010is available as an input of the decoder1000, for receiving an input bitstream. A first output of the deblocking filter1065is available as an output of the decoder1000, for outputting an output picture.

SUMMARY

These and other drawbacks and disadvantages of the prior art are addressed by the present principles, which are directed to methods and apparatus for adaptive geometric partitioning for video encoding and decoding.

According to an aspect of the present principles, there is provided an apparatus. The apparatus includes an encoder for encoding image data corresponding to pictures by adaptively partitioning at least portions of the pictures responsive to at least one parametric model. The at least one parametric model involves at least one of implicit and explicit formulation of at least one curve.

According to another aspect of the present principles, there is provided a method. The method includes encoding image data corresponding to pictures by adaptively partitioning at least portions of the pictures responsive to at least one parametric model. The at least one parametric model involves at least one of implicit and explicit formulation of at least one curve.

According to yet another aspect of the present principles, there is provided an apparatus. The apparatus includes a decoder for decoding image data corresponding to pictures by reconstructing at least portions of the pictures partitioned using at least one parametric model. The at least one parametric model involves at least one of implicit and explicit formulation of at least one curve.

According to still another aspect of the present principles, there is provided a method. The method includes decoding image data corresponding to pictures by reconstructing at least portions of the pictures partitioned using at least one parametric model. The at least one parametric model involves at least one of implicit and explicit formulation of at least one curve.

These and other aspects, features and advantages of the present principles will become apparent from the following detailed description of exemplary embodiments, which is to be read in connection with the accompanying drawings.

