TECHNICAL FIELD

This disclosure relates to video encoding and/or decoding.

BACKGROUND

This disclosure relates to the encoding and/or decoding of a video sequence, which consists of a series of images (a.k.a., pictures). Each image of the video sequence consists of one or more components. Each component can be described as a two-dimensional rectangular array of sample values. It is common that an image in a video sequence consists of three components: one luma component (Y) where the sample values are luma values, a first chroma component (Cb) where the sample values are chroma values and a second chroma component (Cr) where the sample values are chroma values. Other examples include Y′ Cb Cr, Yuv and ICTCP. In ICTCP, I is the “intensity luma” component. For the remainder of this document we will refer to any luma component Y′, Y or I as Y or simply luma. It is common that the dimensions of the chroma components are smaller than the luma components by a factor of two in each dimension. For example, the size of the luma component of an HD image would be 1920×1080 and the chroma components would each have the dimension of 960×540. Components are sometimes referred to as color components.

A block is a two-dimensional array of samples. In video coding, each component is split into blocks and the coded video bitstream is a series of blocks. It is common in video coding that the image is split into units that cover a specific area of the image. Each unit consist of all blocks from all components that make up that specific area and each block belongs fully to one unit. The macroblock in H.264 and the Coding unit (CU) in HEVC are examples of units.

In HEVC, each picture is partitioned into coding tree units (CTU). A CTU consist of an N×N block of luma samples and two M×M corresponding chroma blocks. A CTU in HEVC is like a macroblock in H.264 and earlier standards but in contrast to macroblocks the CTU size is configurable. Most often, however, the CTU size in HEVC is set to 64×64 luma samples. Each CTU can be recursively quadtree split. The root of the quadtree is then associated with the CTU. The quadtree is split until a leaf is reached, which is referred to as the coding unit (CU). A CU in HEVC always consist of a luma block with equal height and width. How each CTU is split is conveyed in the bitstream. The CU is further the root node of two other trees, the prediction tree that has got prediction units (PUs) as nodes and the transform tree that has got transform units (TUs) as nodes. Some decoding processes in HEVC are done on the CU level, some are done on the PU level and some on TU level. Boundaries between PUs and boundaries between TUs are deblocked—i.e., filtered by a deblocking filter, to reduce discontinuities between TUs and PUs. In HEVC there exist two kinds of prediction types for a PU, intra-prediction which only uses prediction from previously decoded samples of the current picture for prediction, and inter-prediction which uses prediction form at least one previously decoded picture.

In HEVC, deblocking is first applied on vertical boundaries and then on horizontal boundaries. The boundaries are either TU boundaries or PU boundaries. To enable parallel friendly deblocking, the deblocking is performed on an 8×8 sample grid.

A deblocking filter strength parameter (BS) is set for each boundary. If the value of BS is larger than 0, then deblocking may be applied. The larger the boundary strength is, the stronger filtering is applied. First it is checked if any of the blocks at a PU boundary between the blocks is an intra-predicted block then (BS is set to=2), or if both blocks use inter-prediction but and they use different reference frames or have significantly different motion vectors then (BS is set to=1). It is also checked if a TU boundary between the blocks has non-zero transform coefficients in at least one of the blocks (code block flag CBF equal to 1), then (BS is set to=1). This first checks sets a boundary strength (BS) which is larger than 0 to indicate that deblocking should be applied. The larger the boundary strength, the stronger filtering is applied. To reduce/avoid removing natural structures when deblocking, a check that there are not any natural structures on respective sides of the boundary is then applied for luma. In HEVC, gradient calculations are used on respective sides of the boundary using the following inequality: abs(p0−2*p1+p2)+abs(q0−2*q1+q2)<beta, where beta is a parameter based on the quantization parameter for the block and p0, p1, to p2 are samples on one side of the block boundary and q0, q1, to q2 are samples on the other side of the block boundary. The condition is checked at two positions along the boundary, and if both conditions are fulfilled, then the luma samples are deblocked for that 4 sample part of the boundary. Chroma boundaries may always be filtered if one any of the neighbouring blocks are intra coded.

In the current draft of the specification for H.266 (VVC draft 3 JVET-L1001 (see reference [1]) (hereafter “VVC” for short)) a coding tree unit (CTU) is similar to the CTU in HEVC with the difference that the CTU in H.266 has a size of 128×128 luma samples. In VVC, the CTU can be split more flexibly such that a resulting CUs may consist of a rectangular luma block. In VVC, there is no prediction tree as in HEVC. However, a CU in VVC can implicitly be divided into a multiple of TUs. Implicit TUs can however only appear when the CU size has a width or height larger than the maximum transform size. If that is not the case, the CU does not have any prediction tree nor transform tree.

In VVC, deblocking is first applied on vertical CU boundaries and vertical implicit TU boundaries and vertical sub-block boundaries then on horizontal CU boundaries and horizontal implicit TU boundaries and horizontal sub-block boundaries. The deblocking is based on HEVC deblocking.

In VVC two sub-block prediction tools are included AFFINE and ATMVP. A sub-block prediction tool can be characterized as a prediction tool that has more than one set of motion parameters for different parts of a block, e.g. sub-blocks. AFFINE operates on sub-block size 4×4 and ATMVP on sub-block size 8×8. Each sub-block is predicted by specific motion parameters such as at least one motion vector and one indicator of reference frame to predict from. When a CU uses sub-block motion compensation like ATMVP and AFFINE, sub-block boundaries can be created inside CU due to different motion parameters are used in at least some of the sub-blocks. In VVC (JVET-L1001) from Macau, in addition to deblock CU and implicit TU boundaries on 8×8 grid also sub-block boundaries are deblocked on 8×8 grid when motion parameters on respective side of a sub-block boundary differs sufficiently. Both CU, implicit TU and sub-block boundaries can be deblocked in parallel since deblocking is based on HEVC luma deblocking that reads at most 4 samples and writes at most 3 samples, on each side of the boundary, which is sufficient for when deblocking on an 8×8 grid.

In VVC a tool that combines intra-predication and inter-prediction has been adopted (originates from JVET-L0100 test CE10.1.1.c). When the combination is used for vertical or horizontal intra-prediction the prediction block is formed by a weighted addition of the intra-prediction and the inter-prediction where 4 weighting factors (w={ 6/8, ⅝, ⅜, 2/8} for intra and 1-w for inter) are used. Each weighting factor is used for ¼ of the width for horizontal prediction and ¼ of the height for vertical intra-prediction.

FIG. 1Ashows an example when the intra-prediction is horizontal prediction. The block is sub-partitioned into 4 sub-blocks: #0, #1, #2 and #3. The weight {w_intra, w_inter} for each part is {6/8, 2/8}, {5/8, 3/8}, {3/8, 5/8} and {2/8, 6/8}. Since each sub-block uses different weights to combine intra-inter prediction, one can say that the magnitude modification for each-sub-blocks are different.

FIG. 1Bshows another example when the intra-prediction is vertical prediction. The block is sub-partitioned into 4 sub-blocks where each sub-block has its height equals to ¼ of the height of the original block. Similarly, each sub-block uses different weights to combine intra-inter prediction.

Using different weights to combine intra-inter prediction generates sub-block boundaries.FIG. 1Cbelow shows an example of vertical boundaries from combined intra-inter prediction using horizontal intra prediction.

The direction of cu boundary could be different to the sub-block boundaries, for example, when the sub-block boundaries are horizontal, the cu boundary is vertical.FIG. 1Dshows an example of horizontal boundaries from combined intra-inter prediction using vertical intra prediction.

The cu boundary could exist when using equal weights to combine intra and inter prediction. When the combined intra prediction is planar or dc prediction, the combine weights for intra and inter are equal.

SUMMARY

Certain challenges currently exist. For example, no deblocking of boundaries between combined intra-inter prediction and other CUs in case motion parameters are identical and no CU is intra, and no deblocking of internal boundaries originating from sub-blocks of the combined intra-inter prediction block that are weighted differently. Accordingly, this disclosure describes embodiments for overcoming these and/or other challenges.

Example 1

Prediction Block Boundary

In one embodiment, a boundary between a first block (denoted block P) and a second block (denoted block Q) is deblocked, where at least block P or block Q is based on a weighted combination of an intra-prediction block and an inter-prediction block. In one embodiment, the inter-prediction block is a block predicted from reconstructed samples of the current picture, e.g. intra block copy also referred to as current picture referencing.

Example 2

Internal Prediction Block Boundary

In another embodiment, a sub-block boundary inside a block is deblocked, where the block is based on a prediction block that was created using one magnitude modification on one side of the sub-block boundary and another magnitude modification on the other side of the sub-block boundary. Example of magnitude modification is by scaling a sub-block by a weighting factor or an additive offset such that the magnitude of the samples of the sub-block is changed.

Accordingly, in on aspect there is a method for deblocking a boundary between a first block of samples, block P, and a second block of samples, block Q. The method includes determining whether at least one of the P block or the Q block is a combined intra-inter prediction block. The method also includes, as a result of determining that at least one of the P block or the Q block is a combined intra-inter prediction block, deblocking (804) the boundary between the P block and the Q block.

In some embodiments, the step of deblocking the boundary between the P block and the Q block comprises reducing a discontinuity between the P block and the Q block. In some embodiments, reducing the discontinuity between P and Q comprises filtering the P block using a deblocking filter and filtering the Q block using the deblocking filter.

In some embodiments the process also includes setting a boundary strength, bs, variable for the boundary to a value that is greater than zero as a result of determining that at least one of the P block or the Q block is a combined intra-inter prediction block.

In some embodiments the boundary is a vertical boundary or a horizontal boundary.

In some embodiments, the process also includes generating a block of samples based on a weighted combination of an intra-prediction block and an inter-prediction block, wherein one of the P block or the Q block is the generated block of samples. In some embodiments, the process also includes generating the intra-prediction block from a line of samples or generating the intra-prediction block from referencing a block in a current picture. In some embodiments, the process also includes generating the inter-prediction block from referencing at least one block in a previous decoded picture or generating the inter-prediction block from referencing a block in a current picture.

In some embodiments, the process also includes deblocking a boundary between a first sub-block and a second sub-block if a first condition is true, wherein the first condition is true if: the first sub-block is a weighted combination of a first intra prediction sub-block and a first inter prediction sub-block, the second sub-block is a weighted combination of a second intra prediction sub-block and a second inter prediction sub-block, and the weight for the first intra prediction sub-block is different from the weight for the second intra prediction sub-block.

An advantage of the embodiments is that they enable deblocking with reduction in blocking artifacts originating from combined intra-inter prediction.

