TECHNICAL FIELD

This disclosure relates to video encoding and/or decoding.

BACKGROUND

This disclosure relates to the encoding and/or decoding of a video sequence, which consists of a sequence of pictures.

Components

Each picture of the video sequence consists of one or more components. Each component can be described as a two-dimensional rectangular array of sample values. It is common that a picture in a video sequence consists of three components: i) a luma component (Y), where the sample values are luma values; ii) a first chroma component (Cb), where the sample values are chroma values; and iii) a second chroma component (Cr), where the sample values are chroma values.

Other examples include Y′ Cb Cr, YUV and ICTCP. In ICTCP, I is the “intensity luma” component. For the remainder of this document we may refer to any luma component Y′, Y or I as Y or simply luma. It is common that the dimensions of the chroma components are smaller than the luma components by a factor of two in each dimension. For example, the size of the luma component of an HD picture would be 1920×1080 and the chroma components would each have the dimension of 960×540. Components are sometimes referred to as color components.

Blocks and Units

A block is one two-dimensional array of samples corresponding to an area of a picture, and a unit consists of one or more blocks. In video coding, each component is split into blocks and the coded video bit stream consist of a series of blocks. A “transform block” is a block to which a transform is applied. A “prediction block” is a block to which a single prediction mode is applied.

It is common in video coding that the picture is split into units that cover a specific area of the picture, referred to as a “picture area.” Each unit consists of all blocks that make up that picture area and each block belongs fully to one unit. The Coding Unit (CU) in HEVC is an example of a unit.

In HEVC, some decoding processes are done at the Coding Unit level, some are done at the prediction blocks, and some at the transform blocks.

In HEVC, there are two kinds of prediction types for a prediction block: intra prediction, which only uses prediction from previous decoded samples of the current picture for prediction, and inter prediction, which uses prediction from at least one previously decoded picture. A prediction block that is generated using intra prediction can be called an “intra block,” and a prediction block generated using inter prediction can be called an “inter block.”

In the current versatile video coding (VVC) standardization development, a CU can implicitly be divided into multiple of transform blocks. Implicit transform blocks can however only appear when the CU size has a width or height that is larger than the maximum transform size. When a CU prediction type is intra prediction, the same transform block implicit split mechanism is applied to prediction block. The size of transform block is the same as the size of prediction block when CU prediction type is intra prediction. In other words, the transform and prediction operate on the same two-dimensional array when CU prediction type is intra prediction.

Intra Prediction

In HEVC, the intra prediction generates the samples for a prediction block based on samples from previously decoded CUs of the same picture. These samples are referred to as “reference samples.” When a previously decoded CU is not available, the corresponding reference samples are padded from the nearest available left-bottom or right-top reference samples.

In HEVC, there are 35 intra prediction modes: DC, planar and 33 angular modes. Each of these 33 angular modes represent a pre-defined prediction direction. The intra prediction then follows the direction to predict samples for a prediction block corresponding to an area of the picture based on spatial extrapolation of reference samples corresponding to a neighboring area of the picture.

The current VVC development defines a set of tools called bench mark sets (BMS). The BMS consists of tools that are likely for inclusion in the standard. The BMS extends the existing HEVC's intra directions to support up to 67 intra prediction modes.

Residual, Primary Transform and Quantization

A residual block consists of residual samples that are obtained by taking the sample value difference between the original samples (“original block”) corresponding to an area of the picture and the predicted samples corresponding to the area of the picture (i.e., the predicted block for the area of the picture). The residual block is further processed by using a spatial core transform to produce primary-transformed (DCT/DST transformed) transform coefficients. The transform is a process to decorrelate the information of the residual block into frequency domain. In HEVC, the types of spatial core transform include DCT-II and 4×4 DST-VII. The spatial core transform is sometimes referred to as primary transform.

The resulting primary-transformed (DCT/DST transformed) transform coefficients (or simply “transform coefficients” for short) are then quantized according to a quantization parameter (QP) which controls the precision of the quantized coefficients. The quantized coefficients can be referred to as residual coefficients. A high QP would result in low precision of the coefficients and thus low fidelity of the residual block. The non-zero residual coefficients are signaled in the video bit stream. A decoder then receives the non-zero residual coefficients, applies inverse quantization and inverse transform to derive the residual block. If all residual coefficients are 0 after quantization, a flag is signaled in the video bit stream, therefore a decoder does not need to apply inverse quantization and inverse transform for the block.

Non-Separable Secondary Transform (NSST)

NSST is applied for intra blocks (i.e., is applied when the residual block is generated using a predicted block that was generated suing intra prediction). It is a secondary transform which operates on the transform coefficients. On the encoder side, a forward NSST is applied between forward core transform and quantization. On the decoder side, an inverse NSST is applied between de-quantization and inverse core transform. In the current VVC, the NSST is applied only on the low-frequency transformed coefficients, i.e. top-left 4×4 transformed coefficients.

BMS tools improve the adaptability of transform process since the DCT/DST-based transform cannot fully decorrelate the residual signal, especially when there are strong directional information present.

For each of the angular modes, a NSST transform set of 3 different matrices is defined. For DC or planar mode, a NSST transform set of 2 different matrices is defined. In BMS, each NSST transform set is assigned an index, which identifies the transform set. A mapping between the intra prediction modes and the NSST transform set indices is defined using a look-up table. That is, a look-up table is used to map each of the intra prediction modes to an transform set index, thereby mapping each of the intra prediction modes to the transform set identified by the index to which the intra prediction mode is mapped.

Cross-Component Prediction

BMS includes cross-component linear model (CCLM) prediction modes. CCLM is a special intra prediction method for the chroma component. The chroma samples are predicted based on the reconstructed luma samples by using a linear model as follows: pred_C(i,j)=α·rec_L′(i,j)+β, where pred_C(i,j) represents the predicted chroma samples in a CU and rec_L′(i,j) represents the downsampled reconstructed luma samples (in case chroma has reduced resolution compared to luma). Parameters α and β are derived by minimizing the regression error between the neighboring reconstructed luma and chroma samples around the prediction block.

SUMMARY

When CCLM is used to generate a particular prediction block comprising chroma components (i.e., a chroma component prediction block), the current solution uses the intra planar mode as the entry for selecting the transform set (e.g., NSST transform set) for the particular prediction block—i.e., the transform set that is used to transform the transform coefficients that were generated by transforming the residual block that was generated based on the particular prediction block. The selected transform set, however, might not be well suitable because there can be a directional pattern present in the prediction block (e.g., in the chroma component).

To overcome this problem, this disclosure describes an improved process for selecting the transform set for a prediction block. The process can be used in both an encoder and a decoder. For example, the process can be used in both an encoder and a decoder for a prediction block that has been predicted from a reference block. In some embodiments, both the prediction block and the reference block are intra blocks.

In one embodiment (i.e., an encoder embodiment), the process includes determining a directional pattern in a prediction block of one of the pictures in the video sequence, wherein the determining comprises using information from a reference block from which the prediction block is generated to determine the directional pattern. The method also includes selecting a transform set (e.g., an NSST transform set) for the prediction block based on the determined directional pattern. The method further includes using the transform set selected for the prediction block based on the determined directional pattern transform transforming data derived from the prediction block (e.g., a residual block derived from the prediction block and an original block or transform coefficients generated as a result of transforming the residual block).

In another embodiment (i.e., a decoder embodiment), the process includes determining a directional pattern in a prediction block of one of the pictures in the video sequence, wherein the determining comprises using information from a reference block from which the prediction block is generated to determine the directional pattern. The process also includes selecting an inverse transform set (e.g., an inverse NSST transform set) for the prediction block based on the determined directional pattern. The process further includes inverse transforming data (e.g., a transform coefficient block after de-quantization) using the inverse transform set selected based on the determined directional pattern.

In some embodiments, the reference block has an intra prediction mode, and determining the directional pattern comprises or consists of determining the reference block's intra prediction mode. In some embodiments, the process further comprises maintaining mapping information that maps each intra prediction mode included in a particular set of intra prediction modes to a transform set index, the reference block's intra prediction mode is included in the particular set of intra prediction modes, and selecting the transform set or inverse transform set comprises using the mapping information to identify the transform set index to which the reference block's intra prediction mode is mapped.

In some embodiments, the prediction block belongs to a chroma color component and the reference block belongs to a luma color component.

In some embodiments, the process further comprises generating the prediction block using the reference block and a cross-component linear model (CCLM) prediction mode.

In some embodiments, the prediction block belongs to a color component and the reference block belongs to the same color component.

In some embodiments, the process further includes generating the prediction block using the reference block and an intra block copy (IBC) tool.

In some embodiments, determining the directional pattern includes the following steps: defining a set of two or more intra prediction modes, the set of set of two or more intra prediction modes comprising the intra prediction mode of the reference block and a second intra prediction mode; for each intra prediction mode included in the set of intra prediction modes, generating a temporary block; and using the generated temporary blocks and the prediction block to select one of the intra prediction modes from the set of intra prediction modes, wherein the selected intra prediction mode represents the directional pattern such that the transform set is selected based on the selected intra prediction mode.

In some embodiments, the encoding process further includes determining that a set of conditions is satisfied, wherein the step of transforming the data using the transform set selected based on the determined directional pattern is performed as a result of determining that the set of conditions is satisfied.

In some embodiments, the decoding process further includes determining that a set of conditions is satisfied, wherein the step of transforming the data using the transform set selected based on the determined directional pattern is performed as a result of determining that the set of conditions is satisfied.

In some embodiments, determining that the set of conditions is satisfied comprises: determining the prediction block's intra prediction mode; and determining that the prediction block's intra prediction mode satisifies a certain condition (e.g., is above a first threshold (T1), where T1may equal 66).

In some embodiments, determining that the set of conditions is satisfied comprises determining that the prediction block was generated using CCLM.

In some embodiments, determining that the set of conditions is satisfied comprises: determining the number of non-zero transform coefficients (N) of the reference block; and determining that N satisfies a certain condition (e.g., N is at or below a certain threshold (T2)).

In some embodiments, determining that the set of conditions is satisfied comprises: determining the QP used for the reference block; and determining that the QP satisfies a certain condition (e.g., QP is at or above a certain threshold (T3)).

In some embodiments, determining that the set of conditions is satisfied comprises determining that a particular flag received from an encoder is set to a certain value. Thus, in some embodiments, a block level flag can be signaled in the video bit stream. In such an embodiment, the reference block is used to determine the directional pattern only when the flag is set to 1 (or other predefined value).

In some embodiments, transforming data using the transform set selected based on the determined directional pattern comprises transforming a residual block using the selected transform set.

In some embodiments, transforming data using the transform set selected based on the determined directional pattern comprises using a primary transform unit to transform a residual block to produce the data and using the selected transform set to transform the data.

