The invention is directed to optimizing the quantization of transform coefficients with minimal bit rate overhead by scaling a uniform quantization parameter for the entire transform block, obviating the need for a quantization matrix. Although the invention is described in the context of video coding, those skilled in the art will understand that one can apply the invention to other areas, such as image and audio coding.
The basic essence of a video transmission is a sequence of pictures transmitted at a relatively fixed time sequence for reproduction at a receiving site. For digital transmissions, such sequences of pictures are transmitted in the form of a digital bit stream that is stored at the receiving site for reproduction in some form. In practice, such digitized video transmissions have accompanying audio and synchronization data that together add up to a large amount of data. The video and audio data can occupy a vast amount of storage space as well as transmission bandwidth.
In order to save transmission bandwidth and storage space, video data is compressed at the transmission end, and decompressed at the receiving end. Video compression typically involves taking the differences between adjacent pictures in a stream of pictures or frames and then coding most frames as differences relative to neighboring pictures. This may be done in several ways through the process of motion estimation and compensation by the encoder, and motion compensation at the decoder. An encoder at the beginning of the transmission process is required to determine the way in which a picture is compressed, solely at its own discretion, and then calculate the displacement or motion vectors. This is done frequently through code sequences represented by a long decision tree. In contrast, the decoder at the receiving end is configured to merely perform decoding operations according to discrete operational processes performed by the encoder, or “does what it is told to do.” To serve as a basis of prediction of other frames and to provide functionalities such as random access to the compressed bitstream, the encoder will occasionally encode input video frames independent of other frames. Such frames are termed “Intra” coded frames. In contrast, other frames that are encoded as the difference between the input and the motion compensated predicted information are termed “Inter” coded frames. An encoder sometimes uses information from “future” or subsequent frames in a sequence of frames to decode current frames. Thus, the coding order, the order in which compressed frames are transmitted, is not the same as the display order, which is the order in which the frames are presented to a viewer. Frames encoded with reference to both future and past frames are termed “B” (B-directional) frames.
MPEG (Movie Picture Expert Group) is a standard specifically engineered for inter-frame (motion) compression of video sequences. FIGS. 1 and 2 illustrate, respectively, a group of pictures in display order in FIG. 1 and in coding order in FIG. 2. In FIGS. 1 and 2 “I” represents intra coded frames, “B” represents bidirectional coded pictures, and “P” represents predicted pictures. FIG. 3 illustrates the use of a forward prediction reference pictures and backward prediction reference pictures to generate a current picture. Specifically, FIG. 3 illustrates motion compensation, that is, how future pictures are predicted from subsequent pictures (and future pictures). If motion occurs in a sequence of frames, prediction is carried out by coding differences relative to areas that are shifted with respect to the area being coded. This is known as “motion compensation,” and the process of determining the motion vectors is called “motion estimation.” The resulting motion vectors, describing the direction and amount of motion of a macroblock, are stored and transmitted to the decoder as part of the MPEG bitstream. In operation, the decoder uses the origin and length of the motion vector to reconstruct the frame.
In intra-coding a single frame, the basic building block is the macroblock. Typically, the macroblock is a 16×16 sample array of luminance (gray scale) samples together with one 8×8 block of samples for each of the two chrominance (color) components. Next in the hierarchy is what is known as the “slice,” a contiguous sequence of macroblocks in raster scan order. The slice starts at a specific address or position in the picture, and the address is specified in a slice header.
Intercoding and intracoding are both built on the Discrete Cosine Transform (hereinafter the “DCT”), representing the prediction error after motion compensation (in the case of Inter coding) of the input signal itself (in the case of Intra coding) as a linear combination of spatial frequencies. Each spatial frequency pattern has a corresponding DCT coefficient, that is, the amplitude needed to represent the contribution of the specific spatial frequency to the block of data being represented.
DCT coefficients are then quantized by a scalar quantizer via division by a non-zero “quantization step size” and thereafter either truncating the quantized DCT coefficient or rounding the quantized DCT quotient to the nearest integer, termed quantization levels. At the decoder, the inverse operation (“de-quantization”) is performed by multiplying the quantization level by the same quantization step size used by the cnoder. Both the quantization step size and the quantization levels for each DCT coefficient are signaled in the compressed bitstreams. The reconstruction values, as determined by the above process will always be a multiple of the quantization step size of the corresponding coefficient used by the encoder.
It is to be noted that, the larger the quantization value, the lower the precision of the quantized DCT coefficient, and the smaller the quantization level. Physiologically, large quantization values for high spatial frequencies allows the encoder to discard high frequency activity that are of lower perceptability to the human eye. This saves bandwidth and storage space by discarding data that can not be detected by the human eye.
To exploit the difference is perceptability of different DCT frequencies by the human visual system and to improve compression performance, in image and video compression standards such as JPEG and MPEG-2, quantization matrices were designed so that different quantization step sizes could be applied to different DCT coefficients by signaling only the matrix itself, instead of each of the 8×8=64 different quantization step sizes, one for each DCT coefficient. Standards such as JPEG further allows the signaling of customized quantization matrices from the encoder to the decoder to take advantage of the differences in statistics of the input image and video signal at hand. For example, a JPEG encoder used for media images may use a different quantization matrix than one used for consumer digital cameras.
It should be noted that, however, not all image and video coding standards use quantization matrices or allow the usage of custom quantization matrices. Also, due to numerous profiles and levels for all image and video coding standards which define the subset of image and video coding tools used by compliant encoder and decoders, not all standard compliant image and video decoders support the usage of custom quantization matrices, even if the matrices are transmitted in the bitstreams in a standard-compliant manner. In addition, when the original image and video signal to be compressed exhibit strong time-varying characteristics, adapting the quantization matrix to the varying input may require a large overhead of extra data associated with sending the quantization matrices. For some standards or configured CODECs (components having combined analog to digital and digital to analog converters), dynamic adjustment of quantization matrices may not even be possible.
Therefore, it is highly desirable to design a system that can benefit from the capability of differentiated quantization for DCT coefficients without the overhead of sending the possibly time varying quantization matrices in the bitstreams. As will be seen, the invention accomplishes this and overcomes the shortcomings of the prior art in an elegant manner.