TECHNICAL FIELD

This patent document relates to video coding techniques, devices and systems.

BACKGROUND

In spite of the advances in video compression, digital video still accounts for the largest bandwidth use on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, it is expected that the bandwidth demand for digital video usage will continue to grow.

SUMMARY

Devices, systems and methods related to digital video coding, and specifically, matrix-based intra prediction methods for video coding are described. The described methods may be applied to both the existing video coding standards (e.g., High Efficiency Video Coding (HEVC)) and future video coding standards (e.g., Versatile Video Coding (VVC)) or codecs.

A first example video processing method includes generating, for a conversion between a current video block of a video and a coded representation of the current video block, a first most probable mode (MPM) list using a first procedure based on a rule; and performing the conversion between the current video block and the coded representation of the current video block using the first MPM list, where the conversion of the current video block uses a matrix based intra prediction (MIP) mode in which a prediction block of the current video block is determined by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation, where the rule specifies that the first procedure used for generation of the first MPM list is same as a second procedure used for generation of a second MPM list for conversion of other video blocks of the video that are coded using a non-MIP intra mode that is different from the MIP mode, and where at least a portion of the first MPM list is generated based on at least a portion of the second MPM list.

A second example video processing method includes generating, for a conversion between a current video block of a video and a coded representation of the current video block, a most probable mode (MPM) list based on a rule, where the rule is based on whether a neighboring video block of the current video block is coded with a matrix based intra prediction (MIP) mode in which a prediction block of the neighboring video block is determined by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation; and performing the conversion between the current video block and the coded representation of the current video block using the MPM list, where the conversion applies a non-MIP mode to the current video block, and where the non-MIP mode is different from the MIP mode.

A third example video processing method includes decoding a current video block of a video that is coded in a coded representation of the current video block using a matrix based intra prediction (MIP) mode in which a prediction block of the current video block is determined by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation; and updating a line buffer associated with the decoding without storing in the line buffer information indicating whether the current video block is coded using the MIP mode.

A fourth example video processing method includes performing a conversion between a current video block and a bitstream representation of the current video block, where the current video block is coded using a matrix based intra prediction (MIP) mode in which a prediction block of the current video block is determined by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation, where a flag is coded in the bitstream presentation using at most K contexts in an arithmetic coding or a decoding process, where the flag indicates whether the current video block is coded using the MIP mode, and where K is greater than or equal to zero.

A fifth example video processing method includes generating, for a conversion between a current video block of a video and a bitstream representation of the current video block, an intra prediction mode for the current video block coded with a matrix based intra prediction (MIP) mode in which a prediction block of the current video block is determined by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation; determining a rule for storing information indicative of the intra prediction mode based on whether the current video block is coded with the MIP mode; and performing the conversion according to the rule, where the rule defines that a syntax element for the intra prediction mode is stored in the bitstream representation for the current video block, and where the rule defines that a mode index for the MIP mode for the current video block is not stored in the bitstream representation.

A sixth example video processing method includes performing a first determination that a luma video block of a video is coded using a matrix based intra prediction (MIP) mode in which a prediction block of the luma video block is determined by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation; performing, based on the first determination, a second determination about a chroma intra mode to be used for a chroma video block associated with the luma video block; and performing, based on the second determination, a conversion between the chroma video block and a bitstream representation of the chroma video block.

A seventh example video processing method includes performing a conversion between a current video block of a video and a coded representation of the current video block, where the conversion is based on a determination whether to code the current video block using a matrix based intra prediction (MIP) mode in which a prediction block of the current video block is determined by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation.

An eight example video processing method includes determining, for encoding a current video block of a video into a bitstream representation of the current video block, according to a rule whether to use a matrix based intra prediction (MIP) mode and a coding mode that is different from the MIP mode for encoding the current video block, where the MIP mode includes determining a prediction block of the current video block by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation; and adding, based on the determining, an encoded representation of the current video block to the bitstream representation.

A ninth example video processing method includes determining that a current video block of a video is encoded in a bitstream representation using a matrix based intra prediction (MIP) mode and a coding mode that is different from the MIP mode, where the MIP mode includes determining a prediction block of the current video block by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation; and generating a decoded representation of the current video block by parsing and decoding the bitstream representation.

A tenth example video processing method includes making a determination about an applicability of a loop filter to a reconstructed block of a current video block of a video in a conversion between the video and a coded representation of the current video block, where the current video block is coded using a matrix based intra prediction (MIP) mode in which a prediction block of the current video block is determined by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation; and processing the current video block according to the determination.

An eleventh example video processing method includes determining, for encoding a current video block of a video into a bitstream representation of the current video block, a type of neighboring samples of the current video block to be used according to a rule; and adding, based on the determining, an encoded representation of the current video block to the bitstream representation, where the current video block is encoded using a matrix based intra prediction (MIP) mode in which a prediction block of the current video block is determined by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation.

A twelfth example video processing method includes determining that a current video block of a video is encoded in a bitstream representation using a matrix based intra prediction (MIP) mode and using a type of neighboring samples of the current video block according to a rule, where the MIP mode includes determining a prediction block of the current video block by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation; and generating a decoded representation of the current video block by parsing and decoding the bitstream representation.

A thirteenth example video processing method includes performing a conversion between a current video block of a video and a bitstream representation of the current video block, where the conversion includes generating, by using a matrix based intra prediction (MIP) mode, a prediction block for the current video block by selecting and applying matrix multiplication using matrices of samples and/or by selecting and adding an offset using offset vectors for the current video block, where the samples are obtained from a row and column wise averaging of previously coded samples of the video, and where the selecting is based on reshaping information associated with applying a luma mapping with chroma scaling (LMCS) technique on a reference picture of the current video block.

A fourteenth example video processing method includes determining that a current video block is to be coded using a matrix based intra prediction (MIP) mode in which a prediction block of the current video block is determined by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation; and performing, based on the determining, a conversion between the current video block and a bitstream representation of the current video block, where the performing the conversion is based on a rule of co-application of the MIP mode and another coding technique.

A fifteenth example video processing method includes performing a conversion between a current video block of a video and a bitstream representation of the current video block using a matrix based intra prediction (MIP) mode, where the performing the conversion using the MIP mode includes generating a prediction block by applying matrix multiplication using matrices of samples obtained from a row and column wise averaging of previously coded samples of the video, and where the matrices depend on a bit-depth of a sample.

A sixteenth example video processing method includes generating, for a current video block of a video, an intermediate prediction signal using a matrix based intra prediction (MIP) mode in which a prediction block of the current video block is determined by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation; generating, based on the intermediate prediction signal, a final prediction signal; and performing, based on the final prediction signal, a conversion between the current video block and a bitstream representation of the current video block.

A seventeenth example video processing method includes performing a conversion between a current video block of a video and a bitstream representation of the current video block using a matrix based intra prediction (MIP) mode, where the performing the conversion includes using an interpolation filter in an upsampling process for the MIP mode, where, in the MIP mode, a matrix multiplication is applied to a first set of samples obtained from a row and column wise averaging of previously coded samples of the video, and the interpolation filter is applied to a second set of samples obtained from the matrix multiplication, and where interpolation filter excludes a bilinear interpolation filter.

An eighteenth example video processing method includes performing a conversion between a current video block of a video and a bitstream representation of the current video block according to a rule, where the rule specifies a relationship between applicability of a matrix based intra prediction (MIP) mode or a transform mode during the conversion, where the MIP mode includes determining a prediction block of the current video block by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation, and where the transform mode specifies use of a transform operation for the determining the prediction block for the current video block.

A nineteenth example video processing method includes performing a conversion between a current video block of a video and a bitstream representation of the current video block using a matrix based intra prediction (MIP) mode in which a prediction block of the current video block is determined by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation, where the performing the conversion includes deriving, according to a rule, boundary samples by applying a left bit shift operation or a right bit shift operation on a sum of at least one reference boundary sample, and where the rule determines whether to apply the left bit shift operation or the right bit shift operation.

A twentieth example video processing method includes performing a conversion between a current video block of a video and a bitstream representation of the current video block using a matrix based intra prediction (MIP) mode in which a prediction block of the current video block is determined by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation, where the prediction samples predSamples[xHor+dX][yHor] are determined according to following formula in the upsampling operation: predSamples[xHor+dX IL yHor]=((upHor−dX)*predSamples[xHor][yHor]+dX*predSamples[xHor+upHor][yHor]+offsetHor)/upHor, and predSamples[xVer][yVer+dY]=((upVer−dY)*predSamples[xVer][yVer]+dY*predSamples[xVer][yVer+upVer]+offsetVer)/upVer, where offsetHor and offsetVer are integers, where upHor is a function of the current video block width and a pre-defined value based on the current video block size, where upVer is a function of the current video block height and the pre-defined value based on the current video block size; where dX is 1 . . . upHor−1, where dY is 1 . . . upVer−1, and where xHor are positions based on the upHor and yHor are positions based on the upVer.

A twenty-first example video processing method includes generating, for a conversion between a current video block of a video and a bitstream representation of the current video block, an intra prediction mode for the current video block coded with a matrix based intra prediction (MIP) mode in which a prediction block of the current video block is determined by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation; determining a rule for storing information indicative of the intra prediction mode based on whether the current video block is coded with the MIP mode; and performing the conversion according to the rule, where the rule defines that the bitstream representation excludes a storage of information indicative of the MIP mode associated with the current video block.

In yet another representative aspect, the disclosed technology may be used to provide a method for video processing. This exemplary method includes determining that a current video block is coded using an affine linear weighted intra prediction (ALWIP) mode, constructing, based on the determining, at least a portion of a most probable mode (MPM) list for the ALWIP mode based on an at least a portion of an MPM list for a non-ALWIP intra mode, and performing, based on the MPM list for the ALWIP mode, a conversion between the current video block and a bitstream representation of the current video block.

In yet another representative aspect, the disclosed technology may be used to provide a method for video processing. This exemplary method includes determining that a luma component of a current video block is coded using an affine linear weighted intra prediction (ALWIP) mode, inferring, based on the determining, a chroma intra mode, and performing, based on the chroma intra mode, a conversion between the current video block and a bitstream representation of the current video block.

In yet another representative aspect, the disclosed technology may be used to provide a method for video processing. This exemplary method includes determining that a current video block is coded using an affine linear weighted intra prediction (ALWIP) mode, and performing, based on the determining, a conversion between the current video block and a bitstream representation of the current video block.

In yet another representative aspect, the disclosed technology may be used to provide a method for video processing. This exemplary method includes determining that a current video block is coded using a coding mode different from an affine linear weighted intra prediction (ALWIP) mode, and performing, based on the determining, a conversion between the current video block and a bitstream representation of the current video block.

In yet another representative aspect, the above-described method is embodied in the form of processor-executable code and stored in a computer-readable program medium.

In yet another representative aspect, a device that is configured or operable to perform the above-described method is disclosed. The device may include a processor that is programmed to implement this method.

In yet another representative aspect, a video decoder apparatus may implement a method as described herein.

The above and other aspects and features of the disclosed technology are described in greater detail in the drawings, the description, and the claims.

