FIELD

This application is related to video coding and compression. More specifically, this application relates to systems and methods of performing improved local illumination compensation.

BACKGROUND

Many devices and systems allow video data to be processed and output for consumption. Digital video data includes large amounts of data to meet the demands of consumers and video providers. For example, consumers of video data desire video of the utmost quality, with high fidelity, resolutions, frame rates, and the like. As a result, the large amount of video data that is required to meet these demands places a burden on communication networks and devices that process and store the video data.

Various video coding techniques may be used to compress video data. Video coding is performed according to one or more video coding standards. For example, video coding standards include high-efficiency video coding (HEVC), advanced video coding (AVC), MPEG-2 Part 2 coding (MPEG stands for moving picture experts group), VP9, Alliance of Open Media (AOMedia) Video 1 (AV1), Essential Video Coding (EVC), or the like. Video coding generally utilizes prediction methods (e.g., inter-prediction, intra-prediction, or the like) that take advantage of redundancy present in video images or sequences. An important goal of video coding techniques is to compress video data into a form that uses a lower bit rate, while avoiding or minimizing degradations to video quality. With ever-evolving video services becoming available, encoding techniques with better coding efficiency are needed.

BRIEF SUMMARY

Illumination compensation can be used to efficiently compensate variations in illumination between one or more pictures. Techniques are described herein to provide more efficient and less complex techniques for performing illumination compensation when a virtual pipeline data unit (VPDU) processing structure is used.

According to at least one example, a method of processing video data is provided. The method can include obtaining a current block of a picture of the video data, and determining the current block includes more than one virtual pipeline data unit (VPDU). The method can further include obtaining, for illumination compensation, current neighbor samples for the current block. The current neighbor samples include samples from one or more of a first row of samples from a first neighboring block of a first VPDU included in the current block or a first column of samples from a second neighboring block of the first VPDU. The method can further include obtaining, for illumination compensation, reference neighbor samples for the current block. The reference neighbor samples include samples from one or more of a first row of samples from a first neighboring block of a reference block or a first column of samples from a second neighboring block of the reference block. The method can further include obtaining, for illumination compensation, additional neighbor samples for the current block. The additional neighbor samples include samples from one or more of a second row of samples from the first neighboring block of the first VPDU, a second column of samples from the second neighboring block of the first VPDU, a second row of samples from the first neighboring block of the reference block, a second column of samples from the second neighboring block of the reference block, or samples from a neighboring block of a second VPDU included in the current block. The method can further include determining one or more illumination compensation parameters for the current block using the current neighbor samples, the reference neighbor samples, and the additional neighbor samples. The additional neighbor samples are used for determining the one or more illumination compensation parameters based on the current block covering more than one VPDU. The method can further include performing illumination compensation for the current block of the picture using the one or more illumination compensation parameters.

According to another example, an apparatus for processing video data is provided. The apparatus can include at least one memory and one or more processors implemented in circuitry. The one or more processors are configured to obtain a current block of a picture of the video data, and to determine the current block includes more than one virtual pipeline data unit (VPDU). The one or more processors are configured to obtain, for illumination compensation, current neighbor samples for the current block. The current neighbor samples include samples from one or more of a first row of samples from a first neighboring block of a first VPDU included in the current block or a first column of samples from a second neighboring block of the first VPDU. The one or more processors are configured to obtain, for illumination compensation, reference neighbor samples for the current block. The reference neighbor samples include samples from one or more of a first row of samples from a first neighboring block of a reference block or a first column of samples from a second neighboring block of the reference block. The one or more processors are configured to obtain, for illumination compensation, additional neighbor samples for the current block. The additional neighbor samples include samples from one or more of a second row of samples from the first neighboring block of the first VPDU, a second column of samples from the second neighboring block of the first VPDU, a second row of samples from the first neighboring block of the reference block, a second column of samples from the second neighboring block of the reference block, or samples from a neighboring block of a second VPDU included in the current block. The one or more processors are configured to determine one or more illumination compensation parameters for the current block using the current neighbor samples, the reference neighbor samples, and the additional neighbor samples. The additional neighbor samples are used for determining the one or more illumination compensation parameters based on the current block covering more than one VPDU. The one or more processors are configured to perform illumination compensation for the current block of the picture using the one or more illumination compensation parameters.

According to another example, a non-transitory computer-readable medium for coding video data is provided. The non-transitory computer-readable medium can include instructions stored thereon that, when executed by one or more processors, cause the one or more processors to: obtain a current block of a picture of the video data; determine the current block includes more than one virtual pipeline data unit (VPDU); obtain, for illumination compensation, current neighbor samples for the current block, the current neighbor samples including samples from one or more of a first row of samples from a first neighboring block of a first VPDU included in the current block or a first column of samples from a second neighboring block of the first VPDU; obtain, for illumination compensation, reference neighbor samples for the current block, the reference neighbor samples including samples from one or more of a first row of samples from a first neighboring block of a reference block or a first column of samples from a second neighboring block of the reference block; obtain, for illumination compensation, additional neighbor samples for the current block, the additional neighbor samples including samples from one or more of a second row of samples from the first neighboring block of the first VPDU, a second column of samples from the second neighboring block of the first VPDU, a second row of samples from the first neighboring block of the reference block, a second column of samples from the second neighboring block of the reference block, or samples from a neighboring block of a second VPDU included in the current block; determine one or more illumination compensation parameters for the current block using the current neighbor samples, the reference neighbor samples, and the additional neighbor samples, the additional neighbor samples being used for determining the one or more illumination compensation parameters based on the current block covering more than one VPDU; and perform illumination compensation for the current block of the picture using the one or more illumination compensation parameters.

According to another example, an apparatus comprising means for coding video data is provided. The apparatus can include means for obtaining a current block of a picture of the video data, and means for determining the current block includes more than one virtual pipeline data unit (VPDU). The apparatus can include means obtaining, for illumination compensation, current neighbor samples for the current block. The current neighbor samples include samples from one or more of a first row of samples from a first neighboring block of a first VPDU included in the current block or a first column of samples from a second neighboring block of the first VPDU. The apparatus can include means obtaining, for illumination compensation, reference neighbor samples for the current block. The reference neighbor samples include samples from one or more of a first row of samples from a first neighboring block of a reference block or a first column of samples from a second neighboring block of the reference block. The apparatus can include means obtaining, for illumination compensation, additional neighbor samples for the current block. The additional neighbor samples include samples from one or more of a second row of samples from the first neighboring block of the first VPDU, a second column of samples from the second neighboring block of the first VPDU, a second row of samples from the first neighboring block of the reference block, a second column of samples from the second neighboring block of the reference block, or samples from a neighboring block of a second VPDU included in the current block. The apparatus can include means determining one or more illumination compensation parameters for the current block using the current neighbor samples, the reference neighbor samples, and the additional neighbor samples. The additional neighbor samples are used for determining the one or more illumination compensation parameters based on the current block covering more than one VPDU. The apparatus can include means performing illumination compensation for the current block of the picture using the one or more illumination compensation parameters.

In some aspects, the methods, apparatuses, and computer-readable media described above can include: determining a size of a VPDU; determining one or more of a width or a height of the current block; determining one or more of the width or the height of the current block is larger than the size of the VPDU; and determining the current block includes more than one VPDU based on determining one or more of the width or the height of the current block is larger than the size of the VPDU.

In some aspects, the first neighboring block of the first VPDU includes a top neighboring block, the second neighboring block of the first VPDU includes a left neighboring block, the first neighboring block of the reference block includes a top neighboring block, and the second neighboring block of the reference block includes a left neighboring block.

In some aspects, subsampling is not applied to samples from the first neighboring block of the first VPDU, samples from the second neighboring block of the first VPDU, samples from the first neighboring block of the reference block, and samples from the second neighboring block of the reference block when performing illumination compensation for the current block.

In some aspects, the additional neighbor samples include samples from the second row of samples from the first neighboring block of the first VPDU and samples from the second column of samples from the second neighboring block of the first VPDU. In some cases, the second row of samples from the first neighboring block of the first VPDU includes at least two rows of samples, and the second column of samples from the second neighboring block of the first VPDU includes at least two columns of samples.

In some aspects, the additional neighbor samples include samples from the second row of samples from the first neighboring block of the first VPDU, samples from second column of samples from the second neighboring block of the first VPDU, samples from the second row of samples from the first neighboring block of the reference block, and samples from the second column of samples from the second neighboring block of the reference block. In some cases, the second row of samples from the first neighboring block of the first VPDU includes at least two rows of samples, the second column of samples from the second neighboring block of the first VPDU includes at least two columns of samples, the second row of samples from the first neighboring block of the reference block includes at least two rows of samples, and the second column of samples from the second neighboring block of the reference block includes at least two columns of samples.

In some aspects, the additional neighbor samples include the samples from the neighboring block of the second VPDU included in the current block. In some cases, the additional neighbor samples include samples from a neighboring block of a third VPDU included in the current block. In some cases, the samples from the neighboring block of the second VPDU include a row of samples from the neighboring block of the second VPDU, and the samples from the neighboring block of the third VPDU include a column of samples from the neighboring block of the third VPDU. In some examples, the row of samples from the first neighboring block of the second VPDU includes at least two rows of samples, and the column of samples from the neighboring block of the third VPDU includes at least two columns of samples.

In some aspects, the additional neighbor samples include the samples from the neighboring block of the second VPDU, samples from the second row of samples from the first neighboring block of the reference block, and samples from the second column of samples from the second neighboring block of the reference block. In some cases, the samples from the neighboring block of the second VPDU include a row of samples from the neighboring block of the second VPDU and a column of samples from a neighboring block of a third VPDU.

In some aspects, the additional neighbor samples include one or more of samples to a right of a top neighboring row of the current block, samples below a left neighboring column of the current block, samples to a right of a top neighboring row of the reference block, or samples below a left neighboring column of the reference block.

In some aspects, determining the one or more illumination compensation parameters for the current block includes minimizing a difference between one or more of the samples from the first row of samples from the first neighboring block of the first VPDU or the samples from the first column of samples from the second neighboring block of the first VPDU and one or more of the samples from the first row of samples from the first neighboring block of the reference block or the samples from the first column of samples from the second neighboring block of the reference block.

In some aspects, the one or more illumination compensation parameters include at least one scaling factor and at least one offset.

In some aspects, the methods, apparatuses, and computer-readable media described above can include decoding the current block of video data based on performing the illumination compensation for the current block. In some cases, the methods, apparatuses, and computer-readable media described above can include: determining a residual value for the current block; performing a prediction mode for the current block; and reconstructing at least one sample of the current block based on the illumination compensation performed for the current block, the residual value for the current block, and the prediction mode performed for the current block.

In some examples, the methods, apparatuses, and computer-readable media described above can include generating an encoded video bitstream. The encoded video bitstream can include at least a portion of the current block of video data.

In some examples, the methods, apparatuses, and computer-readable media described above can include signaling the encoded video bitstream.

In some examples, the methods, apparatuses, and computer-readable media described above can include signaling the one or more illumination compensation parameters in the encoded video bitstream.

In some aspects, the apparatuses described above can include a mobile device. In some examples, the apparatus can include a camera for capturing one or more pictures. In some examples, the apparatus can include a display for displaying one or more pictures. For example, the apparatus can include a mobile device (or other device) with a camera for capturing one or more pictures. In another example, the apparatus can include a mobile device (or other device) with a display for displaying one or more pictures. In another example, the apparatus can include a mobile device (or other device) with a camera for capturing one or more pictures and a display for displaying the one or more pictures.

This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used in isolation to determine the scope of the claimed subject matter. The subject matter should be understood by reference to appropriate portions of the entire specification of this patent, any or all drawings, and each claim.

The foregoing, together with other features and embodiments, will become more apparent upon referring to the following specification, claims, and accompanying drawings.

