FIELD OF THE INVENTION

The present invention relates to video coding with affine motion compensation. In particular, the present invention relates to buffer management for a video coding system implementing affine motion compensation and techniques to enable adaptive motion vector resolution for affine motion compensation.

BACKGROUND AND RELATED ART

Video data requires a large storage space to buffer intermediate data during encoding and decoding. Along with the growing high resolution and higher frame rates, as well as more powerful coding techniques are developed to achieve better coding performance, the storage requirement for video encoder and decoder increase significantly. One of the newly developed coding techniques is affine motion prediction and compensation, which effectively tracks more complicated motion such as rotation, zooming, and the deformation of moving objects. Inter frame motion prediction methods applied in the recently developed coding standards such as High Efficiency Video Coding (HEVC) only consider two-dimensional (2D) translational motion, where all the pixels in an area of interest follow the same motion direction and magnitude. Affine motion prediction is capable of describing 2D block rotations and scaling according to a four-parameter affine model. Affine motion prediction is also capable of capturing 2D deformations according to a six-parameter affine model which transforms a square or a rectangle into a parallelogram. There are two major modes for affine motion prediction proposed in the literatures, including affine Merge mode and affine Inter mode. Affine Merge mode allows the inheritance of affine motion information from a spatially neighboring block whereas affine Inter mode constructs several most probable candidates by combining motion information of spatially neighboring blocks. Affine Inter mode is also referred to affine advance motion vector prediction (AMVP) mode.

Motion occurs across pictures along temporal axis can be described by a four-parameter affine motion model as shown in Equation (1). Assuming A(x, y) is an original pixel at location (x, y) under consideration, and A′(x′, y′) is a corresponding reference pixel at location (x′, y′) in a reference picture for the original pixel A(x, y).
x′=a0*x+a1*y+a2, and
y′=−a1*x+a0*y+a3.  (1)
where a0, a1, a2 and a3 are the four parameters in the four-parameter affine motion model.

The motion vector (vx, vy) between this original pixel A(x,y) and its corresponding reference pixel A′(x′,y′) in a block coded in an affine mode is described as:
vx=(1−a0)*x−a1*y−a2, and
vy=(1−a0)*y+a1*x−a3.  (2)

An exemplary four-parameter affine motion model is shown inFIG. 1A. Two corner pixels110and112are located at upper-left and upper-right corners of a current block102, and these two corner pixels are also called control points for the current block102in the four-parameter affine motion model. Motion vectors Mv0 and Mv1 of the two control points110and112map the current block102to a reference block104in a reference picture. The motion vector field of each pixel A(x,y) in the current block102may be derived based on the motion vectors Mv0 and Mv1 of the control points110and112according to Equation (3).

{vx=(v1⁢x-v0x)w⁢x-(v1⁢y-v0y)w⁢y+v0xvy=(v1⁢y-v0y)w⁢x+(v1⁢x-v0x)w⁢y+v0y(3)
where (v0x, v0y) represents the motion vector Mv0 at the upper-left corner110, (v1x, v1y) represents the motion vector Mv1 at the upper-right corner112, and w represents a width of the current block. For block-based affine motion compensation, when the motion vectors Mv0 and Mv1 of the two control points are decoded, the motion vector of each 4×4 block of the current block102can be determined according to Equation (3). In other words, the four-parameter affine motion model for the current block102can be specified by the two motion vectors Mv0 and Mv1 at the two control points. Furthermore, while the upper-left corner and the upper-right corner of the block are used as the two control points, other two control points may also be used.

A six-parameter affine motion model can be described by Equation (4). In this model, a total of six parameters a0, a1, a2, b0, b1 and b2 and three control points are used. For each pixel A(x, y), the motion vector (vx, vy) between this pixel A(x, y) and its corresponding reference pixel A′(x′, y′) is shown in Equation (5).
x′=a0+a1*x+a2*y, and
y′=b0+b1*x+b2*y.(4)
vx=(a1−1)*x+a2*y+a0, and
vy=(b2−1)*y+b1*x+b0.  (5)

The motion vector for each pixel predicted by the six-parameter affine motion model is also location dependent.FIG. 1Billustrates an example of affine motion compensation according to the six-parameter affine motion model, where a current block122is mapped to a reference block124in a reference picture. The correspondences between three corner pixels130,132, and134of the current block122and three corner pixels of the reference block124can be determined by the three arrows as shown inFIG. 1B. The six parameters for the affine motion model can be derived based on three known motion vectors Mv0, Mv1, Mv2 of the upper-left, upper-right, and lower-left control points of the current block122. Parameter derivation for the affine motion model is known in the field and the details are omitted here.

Various implementations of affine Inter mode and affine Merge mode have been discussed, for example, an affine flag is used to indicate whether the affine Inter mode is applied, and this affine flag is signaled for each Inter-coded coding unit (CU) when the CU is equal to or larger than 16×16. A candidate motion vector predictor (MVP) pair list is constructed for a current CU using valid neighboring coded blocks if the current CU is coded or to be coded in affine Inter mode.FIG. 2illustrates an example of the candidate MVP pair derivation for a current block20coded in affine Inter mode or affine Merge mode. As shown inFIG. 2, a motion vector predictor (MVP) for the motion vector Mv0 at an upper-left control point of the current block20is selected from motion vectors of upper-left neighboring coded blocks A0, A1, or A2; and a MVP for Mv1 at an upper-right control point of the current block20is selected from motion vectors of upper-right neighboring coded blocks B0 and B1. An MVP index for the candidate MVP pair list is signaled in a video bitstream and motion vector differences (MVDs) of the two control points are coded in the video bitstream.

For a current block20coded in Merge mode, five neighboring coded sub-blocks C0 (referred as the left-bottom block), B0 (referred as the top-right block), B1 (referred as the upper-right corner block), C1 (referred as the lower-left corner block), and A0 (referred as the upper-left corner block) inFIG. 2are sequentially checked to determines whether any of the neighboring coded sub-blocks is coded in affine Inter mode or affine Merge mode. The current block20in this example is a prediction unit (PU). An affine flag is signaled to indicate whether the current block20is coded in affine Merge mode only if any of the neighboring coded sub-blocks is coded in affine Inter mode or affine Merge mode. When encoding or decoding the current block20according to affine Merge mode, a first available affine-coded neighboring block is determined by selecting from the five neighboring coded sub-blocks. The first available affine-coded neighboring block including the selected neighboring coded sub-block is used to derive an affine Merge candidate. The affine Merge candidate is used for deriving a predictor in a reference picture for the current block. The selection order for selecting one of the neighboring coded sub-blocks is from left-bottom block, top-right block, upper-right corner block, lower-left corner block to upper-left corner block (C0→B0→B1→C1→A0) as shown inFIG. 2. The affine Merge candidate for the current block20is derived from MVs at the control points of the first available affine-coded neighboring block, for example, the MVs of a top-left N×N sub-block and a top-right N×N sub-block of the first available affine-coded neighboring block are used to derive the affine Merge candidate if a four-parameter affine motion model is applied. The MV of a bottom-left N×N sub-block of the first available affine-coded neighboring block is also used to derive the affine Merge candidate when a third control point is included for a six-parameter affine motion model.

BRIEF SUMMARY OF THE INVENTION

A method and apparatus for video encoding and decoding with affine motion compensation in a video coding system are disclosed. Embodiments of a video encoder according to the present invention receive input data associated with a current block in a current picture and embodiments of a video decoder according to the present invention receive a video bitstream corresponding to compressed data including the current block in the current picture. The current block is coded or to be coded in an affine mode according to an affine motion model. Various embodiments of the present invention reduce the buffer requirement of a temporal buffer for affine candidate derivation. The temporal buffer stores selective motion vectors less than all motion vectors of previous coded blocks in the current picture. Embodiments of the method retrieve two or more motion vectors (MVs) of a neighboring block from the temporal buffer, where at least one of the retrieved motion vectors is not corresponding to a sub-block in a first neighboring N×N block row closest to a top boundary of the current block or a first neighboring N×N block column closest to a left boundary of the current block, and N×N is a block size for storing one motion vector in the temporal buffer. The method further derives an affine candidate including affine motion vectors using the retrieved two or more motion vectors of the neighboring block, and encodes or decodes the current block by predicting the current block using one or more motion compensated blocks derived by the derived affine candidate. Each of the affine motion vectors predicts a movement between a point of the current block and a corresponding point of one or more motion compensated blocks.

In some embodiments, the temporal buffer stores MVs of the first neighboring N×N block row and a second neighboring N×N block row that are closest to the top boundary of the current block, and the temporal buffer stores MVs of the first neighboring N×N block column and a second neighboring N×N block column that are closest to the left boundary of the current block.

In one embodiment, the retrieved MVs includes first and second MVs for a four-parameter affine motion model, for example, the first and second MVs are replacing MVs replacing original MVs at an upper-left corner and an upper-right corner of the neighboring block respectively if the neighboring block is above or adjacent to an upper-left corner of the current block. The first MV is a replacing MV replacing an original MV at an upper-left corner of the neighboring block and the second MV is an original MV at an upper-right corner of the neighboring block if the neighboring block is on the left side of the current block. In an embodiment, the first and second MVs correspond to first and second sub-blocks in the neighboring block, and a first affine MV of the affine MVs is derived using the first and second MVs, a pixel position of the current block relative to the current picture, a pixel position of the first sub-block relative to the current picture, and a width of the neighboring block. A second affine MV of the affine MVs is derived using the first and second MVs, the width of the neighboring block, and a width of the current block.

In another embodiment, the retrieved MVs includes first, second and third MVs for a six-parameter affine motion model, for example, the three MVs are for first, second and third sub-blocks in the neighboring block, and a first affine MV is derived using at least two of the three MVs, a pixel position of the current block relative to the current picture, a pixel position of the first sub-block relative to the current picture, and a width of the neighboring block. A second affine MV is derived using the first and second motion vectors, the width of the neighboring block, and a width of the current block, and a third affine MV is derived using the first and third motion vectors and at least one of a height of the neighboring block and a height of the current block. In this embodiment, the first, second MVs are replacing MVs replacing original MVs at an upper-left corner and an upper-right corner of the neighboring block, and the third MV is an original MV at a lower-left corner of the neighboring block if the neighboring block is above the current block. The first, second, and third MVs are replacing MVs replacing original MVs at an upper-left corner, upper-right corner, and lower left corner of the neighboring block if the neighboring block is adjacent to an upper-left corner of the current block. The first and third MVs are replacing MVs replacing original MVs at an upper-left corner and lower-left corner of the neighboring block, and the second MV is an original MV at an upper-right corner of the neighboring block if the neighboring block is on the left side of the current block.

The current block is predicted by the motion compensated blocks by the affine motion vectors in the affine candidate according to a four-parameter affine motion model with two control points or according to a six-parameter affine motion model with three control points. The temporal buffer stores the selective motion vectors, for example, the temporal buffer stores MVs of M N×N block rows above the current block, and M is less than a largest coding unit height divided by N (CTU_height/N). In another embodiment, the temporal buffer stores MVs of K N×N block columns on a left side of the current block, and K is less than a largest coding unit width divided by N (CTU_width/N).

In some embodiments of the method, the retrieved MVs are original MVs at two or more control points of the neighboring block. The control points include at least two of an upper-left corner, an upper-right corner, a bottom-right corner, and a bottom-left corner of the neighboring block. In an embodiment, the temporal buffer stores MVs of 2 N×N block rows and 2 N×N block columns, including original MVs of a first neighboring N×N block row, a first neighboring N×N block column, a top N×N block row set and a left-most N×N block column set. The first neighboring N×N block row is a last row in upper neighboring blocks that is closest to the top boundary of the current block, the first neighboring N×N block column is a last column in left neighboring blocks that is closest to the left boundary of the current block, the top N×N block row set is the first rows in the upper neighboring blocks, and the left-most N×N block column set is the first columns in the left neighboring blocks.

In yet another embodiment, the method comprises receiving input data associated with a current block coded or to be coded in an affine mode, calculating and storing affine parameters for a plurality of coded blocks in the current picture, retrieving the affine parameter of one or more coded block from the temporal buffer corresponding to a neighboring block of the current block to derive an affine candidate including affine MVs. The current block is encoded or decoded by predicting the current block using one or more motion compensated blocks by the derived affine candidate. Each of the affine motion vectors predicts a movement between a point of the current block and a corresponding point of the one or more motion compensated blocks.

The affine parameters in an embodiment include a horizontal direction motion vector (MV) offset and one motion vector in the neighboring block when the affine motion model is a four-parameter affine motion model using two of an upper-left corner, an upper-right corner, an lower-left corner, and a lower-right corner as the control points, or the affine parameters include a vertical direction motion vector (MV) offset and one motion vector in the neighboring block when the affine motion model is a four-parameter affine motion model using two of an upper-left corner, an upper-right corner, a lower-left corner, and a lower-right corner as control points. The affine parameters in another embodiment include a horizontal direction MV offset, a vertical direction MV offset, and a motion vector in the neighboring block when the affine motion model is a six-parameter affine motion model. An example of the motion vector in the neighboring block is a motion vector at an upper-left corner, upper-right corner, lower-right corner, or lower-left corner of the neighboring block. The affine parameters in another embodiment include a scaled MV offset for the coded block. The affine parameters in yet another embodiment include two or three affine motion vectors representing motion vectors at two or three control points, and the temporal buffer stores the two or three affine motion vectors for the coded block.

Another embodiment of the method comprises receive input data associated with a current block coded or to be coded in an affine mode, retrieving two or more MVs of a valid neighboring block from a temporal buffer for the current block, deriving an affine candidate including affine MVs using the retrieved two or more MVs of the valid neighboring block, and encoding or decoding the current block by predicting the current block using one or more compensated blocks by the derived affine candidate. The valid neighboring block does not include a neighboring block adjacent to an upper-left corner of the current block. The temporal buffer stores MVs of one neighboring N×N block row and one neighboring N×N block column of the current block, where N×N is a block size for storing one MV in the temporal buffer. The affine MVs predict a movement between a point of the current block and a corresponding point of the one or more motion compensated blocks.

Aspects of the disclosure further provide an apparatus comprising one or more electronic circuits configured for performing the video coding methods with affine motion compensation. Other aspects and features of the invention will become apparent to those with ordinary skill in the art upon review of the following descriptions of specific embodiments.

