TECHNICAL FIELD

This disclosure generally relates to data compression, and, more specifically, to rate controlled image and texture data compression.

BACKGROUND

Textures may generally include two-dimensional (2D) images that map to a three-dimensional (3D) surface, in which the individual pixels of the texture images may be referred to as “texels” (e.g., texture elements). For example, during graphics rendering, the textures of visible objects are sampled to generate a final image for display. However, artificial reality (e.g., augmented reality [AR], virtual reality [VR]) headsets may include one or more display lenses that causes different wavelengths of light to have different focal lengths. This may lead to visual artifacts, such as chromatic aberration becoming apparent on the displays of the artificial reality headset. For example, chromatic aberration may include different color channels of a pixel in the original rendered image not being projected onto the same corresponding location in the user's eyes when the image is displayed. Thus, it may be useful to provide techniques to compensate for chromatic aberration.

SUMMARY OF PARTICULAR EMBODIMENTS

The present embodiments are directed to techniques for (1) encoding individual color components separately and (2) providing a compression rate control method of allocating available bits to different (R)ed, (G)reen, (B)lue color components based on the amount of color information included in each of the RGB color components of each pixel per pixel block. In certain embodiments, an encoder device may access color components of a pixel region in an image. For example, in one embodiment, the image may include an N-bit color image (e.g., 8-bit color image), which may include RGB color components to be compressed and stored and/or transmitted. The encoder device may then determine a color variance for each of the color components of the pixel region. For example, in certain embodiments, the encoder device may perform a pre-analysis of the of the N-bit color image (e.g., on a pixel region by pixel region basis) and determine a mean (e.g., an average of the total pixel value) and a variance (e.g., a measure of the average degree to which each RGB color component is different from the mean value) with respect to each of the RGB color components of each pixel per pixel region.

In certain embodiments, the encoder device may then determine a desired bit allocation for each of the color components of the pixel region based on the color variance associated with that color component. For example, in some embodiments, the encoder device may determine a desired bit allocation [x, y, z] (e.g., x—corresponding to the all of the red color components of the pixel region; y—corresponding to the all of the green color components of the pixel region; and z—corresponding to the all of the blue color components of the pixel region) for encoding each of the RGB color components of the pixel region based on the color variance associated with each of the RGB color components. The encoder device may then determine a total bit allocation budget for the pixel region based on the desired bit allocation for the color components of the pixel region and a number of unallocated bits available for allocation.

For example, in some embodiments, the encoder device may determine the total bit allocation budget based on a sum (e.g., N=bit allocation budget+number of unallocated bits available for allocation) of the desired bit allocations for each of the RGB color components of the pixel region (e.g., x, y, z) and the number of unallocated bits available for allocation (e.g., bits). The encoder device may then determine a final bit allocation for each of the color components by allocating the total bit allocation for the pixel region to each of the color components of the pixel region according to the desired bit allocation for each of the color components. For example, in certain embodiments, the encoder device may determine the final bit allocation by comparing, for example, the number of unallocated bits available for allocation (e.g., N bits) to the desired bit allocation [x, y, z]. Specifically, in some embodiments, the number of unallocated bits available for allocation may correspond to a predetermined bit allocation budget (e.g., N bpp) for the pixel region.

In some embodiments, in response to determining that a sum of the desired bit allocations (e.g., x, y, z) for each of the RGB color components is less than or equal to the bit allocation budget (e.g., N bpp) for the pixel region, the encoder device may encode each of the color components of the pixel region using the final bit allocation. For example, when the sum of the desired bit allocations (e.g., x, y, z) for each of the RGB color components is less than or equal to the bit allocation budget (e.g., N bpp), the final bit allocation may correspond to the desired bit allocation [x, y, z]. Accordingly, the bit allocation budget of N bits may be distributed to each of the RGB color components for the pixel region to correspond to the desired bit allocation [x, y, z].

The present embodiments further provide techniques for efficiently and optimally selecting quantization levels that provide the least amount of quantization error to improve visual quality. In certain embodiments, the encoder device may determine a bit depth for encoding one or more color components of a pixel region in an image, in which the one or more color components includes a color range. For example, in one embodiment, the image may include an N-bit color image, which may include RGB color components to be separately and independently encoded and compressed. The encoder device may then determine a quantization range having a first quantization endpoint and a second quantization endpoint.

In some embodiments, for calculation of the first quantization endpoint, the encoder device may fix the second quantization endpoint to an initial value determined based on the color range, and then select one of a plurality of first candidate values for the first quantization endpoint based on a plurality of corresponding first quantization errors over an entire pixel block. Similarly, for calculation of the second quantization endpoint, the encoder device may fix the first quantization endpoint to the selected first candidate value for the first quantization endpoint, and then select one of a plurality of second candidate values for the second quantization endpoint based on a plurality of corresponding second quantization errors. For example, in certain embodiments, the encoder device may perform, for example, a divide-and-conquer algorithm to identify the most optimal quantization value for the first quantization endpoint (e.g., most optimal lowest quantization value for the quantization range) while the second quantization endpoint (e.g., most optimal highest quantization value for the quantization range) is fixed to a predetermined value. Specifically, in some embodiments, performing the divide-and-conquer algorithm, for example, may include first determining a minimum color value and a maximum color value for the color range of the red color component, the green color component, or the blue color component. The encoder device may then fix the second quantization endpoint to the maximum color value of the particular color component range (e.g., separate color range for the red color component, separate color range for the green component, or separate color range for the blue component).

In certain embodiments, the encoder device may then enumerate all possible pixel values of the first quantization endpoint to identify the most optimal quantization value for the first quantization endpoint. The encoder device may then calculate the quantization error for each of the identified possible quantization values for the first quantization endpoint, and then select the quantization value with the least amount of quantization error. For example, in one embodiment, the quantization value of the first quantization endpoint may be determined and selected based on an M number of trials of differing quantization values for the first quantization endpoint while the second quantization endpoint remains fixed to the maximum color value. In one embodiment, the M number of trials of differing quantization values for the first quantization endpoint may be spaced by a calculated quantization step size.

In some embodiments, upon determining and selecting the most optimal pixel value of the first quantization endpoint, the encoder device may then proceed in determining and selecting the most optimal quantization value of the second quantization endpoint by enumerating all possible quantization values for the second quantization endpoint while the first quantization endpoint remains fixed at the determined and selected the most optimal quantization value of the first quantization endpoint. In one embodiment, the second quantization endpoint may be determined and selected based on N−1 number of trials of differing pixel values for the second quantization endpoint.

In other embodiments, to determine the first quantization endpoint and the second quantization endpoint, the encoder device may perform, for example, an exhaustive search algorithm (e.g., brute force) in which all possible combinations for the first quantization endpoint and the second quantization endpoint are enumerated to identify the most optimal quantization values for the first quantization endpoint and the second quantization endpoint. Specifically, if there are M possible selections for the first quantization endpoint and N possible selections for the second quantization endpoint, the encoder device may select the most optimal quantization value of each of the first quantization endpoint and the second quantization endpoint by calculating the quantization error for all M*N combinations, and then selecting the particular M*N combination with the least amount of quantization error.

In certain embodiments, following the selection and determination of the most optimal quantization values for the first quantization endpoint (e.g., most optimal lowest quantization value for the quantization range) and the second quantization endpoint (e.g., most optimal highest quantization value for the quantization range), the encoder device may define quantization levels corresponding to the bit depth using the quantization range defined by the first quantization endpoint and the second quantization endpoint. For example, in one embodiment, the intervening quantization levels may then be selected as equally spaced quantization values (e.g., equally spaced based on a calculated quantization step size) between the first quantization endpoint and the second quantization endpoint, forming the range of quantization levels corresponding to the bit depth for the one or more color components. The encoder device may then encode the one or more color components of the pixel region using the determined quantization levels.

The present embodiments further include techniques for selectively performing a compression rate control method of allocating available bits to different alpha components that may be utilized to reduce memory consumption and decrease network bandwidth and reduce data transmission latency. In certain embodiments, the encoder device may access first alpha values associated with first pixels in a first pixel region of an N-bit color image (e.g., 8-bit color image). The encoder device may then determine a bit budget for encoding the first alpha values. For example, in some embodiments, a fixed-point representation may be used to determine a bit budget for each alpha channel component per pixel block. For example, because the encoder device may not utilize many bits to, for example, mask out foreground and background pixels, only quantization values up to 4 bits may be utilized. Thus, in some embodiments, the encoder device may determine, for example, an average fixed-point an alpha allocation budget of 2.67bpp.

In certain embodiments, the encoder device may then select a first alpha-encoding mode for the first alpha values to reflect a determination that the first alpha values are all fully transparent or all fully opaque. For example, in some embodiments, for alpha-encoding modes corresponding to a predetermined “Transparent mode” or an “Opaque mode”, all of the alpha components in the pixel block may be determined by the encoder device to be “0” alpha value when in the “Transparent mode”) and “255” alpha value when in “Opaque mode”). The encoder device may then encode the first alpha values by storing the selected first alpha-encoding mode as part of metadata associated with the first pixel region without using the bit budget to encode the first alpha values individually. For example, when the encoder device determines that the first alpha values are all fully transparent or all fully opaque, the encoder device may not encode the alpha components, and may instead store as metadata one or more 2-bit alpha-encoding mode signifiers indicating that the first alpha values are all fully transparent or all fully opaque.

In certain embodiments, the encoder device may then update a record of unallocated bits available for allocation based on the bit budget unused in the encoding of the first alpha values. For example, the encoder device may increment the record of unallocated bits alpha allocation budget of 2.67 bits (e.g., bits unused from the determined alpha allocation budget of 2.67bpp). The encoder device may then allocate, based on the record of unallocated bits, one or more bits to encode a set of alpha values different from the first alpha values. For example, the bits unused (e.g., 2.67 bits) from the determined alpha allocation budget of 2.67bpp may be recorded and allocated for encoding the alpha values of one or more other color components, for example.

The embodiments disclosed herein are only examples, and the scope of this disclosure is not limited to them. Particular embodiments may include all, some, or none of the components, elements, features, functions, operations, or steps of the embodiments disclosed above. Embodiments according to the invention are in particular disclosed in the attached claims directed to a method, a storage medium, a system and a computer program product, wherein any feature mentioned in one claim category, e.g. method, can be claimed in another claim category, e.g. system, as well. The dependencies or references back in the attached claims are chosen for formal reasons only. However, any subject matter resulting from a deliberate reference back to any previous claims (in particular multiple dependencies) can be claimed as well, so that any combination of claims and the features thereof are disclosed and can be claimed regardless of the dependencies chosen in the attached claims. The subject-matter which can be claimed comprises not only the combinations of features as set out in the attached claims but also any other combination of features in the claims, wherein each feature mentioned in the claims can be combined with any other feature or combination of other features in the claims. Furthermore, any of the embodiments and features described or depicted herein can be claimed in a separate claim and/or in any combination with any embodiment or feature described or depicted herein or with any of the features of the attached claims.

