Vector quantization of image subbands: a survey <s> Subband vector quantization methods for color images <s> High-quality coding of images at low bit rates depends critically on matching the characteristics of the coding algorithm to the properties of the human visual system. We describe two recent examples of perceptually optimized codmg algorithms. For transparent coding of images m a 2-D sub-band coding framework, we use a model in which the quantization threshold for just noticeable distortion is a function of three parameters: center frequency (sub-band number), background intensity and background texture. Use of this model with variable rate DPCM coding of sub-bands leads to transparent coding of color images at rates ranging from 0.2 to 2.0 hits per pixel, depending on input signal. For high-quality coding of sparse images, such as highfrequency sub-bands in 3-D coding of video signals, we use a geometrical vector quantizer whose codevectors are edgerelated shapes rather than traditional designs based on minimum mean squared error. In the coding of teleconference scenes, average bit rates for high-frequency sub-bands are in the range of 0.01 to 0.3 bits per pixel. Combinations of the above principles are expected to provide efficient coder designs for a broad spectrum of applications, including image transmission for ISDN and HDTV. <s> BIB001 </s> Vector quantization of image subbands: a survey <s> Subband vector quantization methods for color images <s> This paper deals with vector quantization of sub-band video color images with a high visual quality. If lots of papers have already been published about the use of both techniques for monochrome still images, few of them work with color images. Our work concerns the use of some important human visual system properties to derive an optimized vector quantization of sub-band decomposed TV images. Two major properties are used. First, spatial frequency dependence in the fovea filtering is pushed into a distortion measure for vector quantization. Secondly, and perhaps the most important is that the codebook design is performed in the splitting algorithm by incorporating two complementary thresholds in the global evaluation of the distortion for each class of the code book : a visibility threshold and an annoyance threshold. With these techniques we designed optimized codebooks (with or without classification) which allowed visual vector quantizing of the chrominance component of color video digital signal without any visible impairment and with a high compression ratio. <s> BIB002 </s> Vector quantization of image subbands: a survey <s> Subband vector quantization methods for color images <s> A subband/VQ coder that combines the three color components of each subband pixel into a length 3 vector and then vector quantizes each subband is presented. Perceptual weights derived from subjective visual experiments are used along with the rate-distortion curves to determine a perceptually optimal bit allocation. Advantages of this method are that fewer codebooks need to be designed and the resulting bit allocation problem is simplified. Preliminary results show that this method achieves results comparable to the more standard subband/VQ method where the color components of each subband are vector quantized separately using the perceptually optimal bit allocation algorithm. To obtain the required compression ratio and the high color fidelity required for HDTV applications, the vector quantization is done in a perceptually uniform color space. > <s> BIB003 </s> Vector quantization of image subbands: a survey <s> Subband vector quantization methods for color images <s> Subband coding and vector quantization combined can provide a powerful method for compressing color images. The use of properties of the human visual system can increase the performance of such a system and allow one to achieve very high quality reconstructed images at compression ratios exceeding 10:1. The authors design a color subband-vector quantization system, and formulate the bit allocation problem as an optimization problem where the objective function depends on the distortion-rate curves of the quantizers and on a set of perceptual weights. These weights are derived from data provided by measurements of the mean detection threshold of the human visual system for color transitions along the luminance, red-green, and blue-yellow directions. Minimization of the objective function constrained by the desired bit rate gives a perceptually optimal bit allocation. Three subband/VQ cases are examined. In the first two cases (case 1 and case 2), the color components of the lowest frequency subband are scalar quantized. Case 1 combines the three color components of each pixel of the higher frequency subbands into a three-dimensional vector, while case 2 creates four-dimensional vectors from 2/spl times/2 blocks in each subband color component. The third case (case 3) is the same as case 2, except that the chrominance components of the lowest frequency subband are also vector quantized with 2/spl times/2 blocks in each component. To obtain the required compression ratio and the high color fidelity required for HDTV applications, the vector quantization is done in C.I.E. L*a*b* space and AC/sub 1/C/sub 2/ space. > <s> BIB004 </s> Vector quantization of image subbands: a survey <s> Subband vector quantization methods for color images <s> Abstract The combination of subband coding and vector quantization can provide a powerful and flexible method for compressing color images, since this type of system can easily incorporate properties of the human visual system (HVS). Most of the previous work in image compression using subband coding has filtered the input image so that the frequency domain is partitioned into rectangular regions. While this rectangular decomposition allows one to use separable filters resulting in an easy implementation, it does not take into account the higher sensitivity of the human eye to horizontal and vertical edges than to diagonal ones. In this work, we present a new method for coding high resolution color images using a separable diamond subband coder and vector quantization. By first diagonally interpolating the input image and then filtering with one-dimensional filters, a decomposition into subbands that more closely match the orientation of the human visual system is obtained. Simulation results are given for several vector quantizer cases. Performance evaluations show that for compression ratios in the range of 10:1 to 20:1, a five band system with diamond-shaped subbands yields results that are comparable to those of a seven band rectangular system. <s> BIB005
The great potential for e cient exploitation of properties of the human visual system for monochrome subband/VQ systems is even greater for color subband/VQ systems. The earliest color subband/VQ system was by Kim, et al. 86] , and it did not explicitly incorporate models of human vision, although some such ideas were employed, for example for deciding which frequency bands to drop. Later subband/VQ systems for color images began to explicitly exploit models of human vision BIB001 BIB002 and color vision BIB003 BIB004 BIB005 . The rst color subband/VQ system used DPCM together with either CVQ or FSVQ 86] . RGB images were transformed to the YIQ color system, and each color plane was ltered by symmetrically extended QMFs to produce 16 uniform frequency bands. For Q and I planes, only bands Q 11 , I 11 , I 12 and I 21 were retained. The 12 lowest frequency bands of the luminance plane were retained, producing a total of 16 bands to be encoded. The low bands (Y 11 , I 11 , Q 11 ) were encoded by DPCM. Di erent approaches were tried for the higher bands. Each 4 4 vector was coarsely classi ed as being shade, high variance, or one of several edges. This classi cation was sent to the decoder as side information, and the appropriate codebook used. An improved version used an FSVQ to select the codebook, in a manner very similar to that of Aravind and Gersho 9] . In a continuation of this work, Kim, at al, used a three-component multirate representation of the image, composed of a chrominance subimage pair, a medium-to low-frequency luminance image, and a high-frequency luminance residual. The method used FSVQ and DPCM in coding these components, with noticeable improvement over the earlier work 87]. Safranek et al. presented two examples of perceptually optimized coding algorithms 132]. Variable rate DPCM coding together with a model in which the quantization threshold for just noticeable distortion is a function of three parameters (center frequency = subband number, background intensity and background texture) led to transparent coding of color images at rates ranging from 0.2 to 2.0 bpp, depending on the input. For high-quality coding of sparse images, such as highfrequency sub-bands in 3-D coding of video signals, a geometrical VQ was used whose codevectors were edge-related shapes rather than traditional designs based on minimum MSE. In the coding of teleconference scenes, average bit rates for high-frequency subbands were in the range of 0.01 to 0.3 bpp. Spatial frequency dependence of the human visual system was incorporated into the VQ distortion measure in 13]. This work used two complementary thresholds, called the visibility threshold and the annoyance threshold, in the global evaluation of the distortion during the splitting design of the codebook for each class. These techniques were claimed to yield visual VQ of the chrominance component of color video digital signal without any visible impairment and with a high compression ratio. Full search of an unstructured codebook was examined for color images in several studies. In 56], 2-D uniform band decomposition was applied to the error image obtained after motion compensated interframe prediction. The luminance error image was decomposed into 16 subbands and the chrominance error image into 8 subbands. These were limited to 16 bands total by taking 10 luminance bands and 6 chrominance bands, discarding the lower energy ones. 16-D crossband vectors were formed as in Figure 7 , and quantized by full-search of an unstructured codebook designed by the GLA. Each subband location has an \area of in uence" of size 5 5 in the original image. The subband location was considered signi cant if the percentage of non-zero pixels in its area of in uence was greater than a pre-determined threshold. The insigni cant subband locations were run-length coded and the subband coe cients from the signi cant locations were vector quantized. This sample selection scheme can help prevent bu er over ow. In another study using full-search unstructured VQ, Van Dyck and Rajala 53, 55] formulated the bit allocation problem as an optimization problem where the objective function depended on the distortion-rate curves of the quantizers and on a set of perceptual weights. These weights were derived from data provided by measurements of the mean detection threshold of the human visual system for color transitions along the luminance, red-green, and blue-yellow directions. Minimization of the objective function constrained by the desired bit rate gave a perceptually optimal bit allocation. Three subband/VQ cases were examined. In all cases, the color image was decomposed 2 levels by a QMF (32-tap lter D) to form 7 subbands. In the rst two cases the color components of the lowest frequency subband were scalar quantized. Case 1 combined the three color components of each pixel of the higher frequency subbands into a 3-D vector, while case 2 created 4-D vectors from 2 2 blocks in each subband color component. In the third case, the chrominance components of the lowest frequency subband were also vector quantized with 2 2 blocks. In all cases, a full-search VQ was generated by the GLA. To obtain the required compression ratio and the high color delity required for HDTV applications, the VQ was done in the perceptually uniform C.I.E. L a b 35] space and AC 1 C 2 space. Subsequently, Van Dyck and Rajala used VQ with a separable diamond subband coder in order to take into account the higher sensitivity of the human eye to horizontal and vertical edges than to diagonal ones 54]. By rst diagonally interpolating the input image and then ltering with one-dimensional lters, a decomposition into subbands that more closely match the orientation of the human visual system was obtained. Performance evaluations showed that for compression ratios in the range of 10:1 to 20:1, a 5-band system with diamond-shaped subbands yielded results comparable to those of a 7-band rectangular system. A comparison of ECVQ and ECTCQ for color video sequences was conducted in 52]. The RGB format was converted to YUV. The luminance component underwent a 7-band decomposition, and each chrominance component was decomposed into 4 bands. The lowest frequency subband used a variant of the H.261 coder, and the higher frequency bands were coded either with ECVQ on 2 2 blocks from a particular subband color component, or with ECTCQ. The authors concluded that the system could provide good quality video for moderately complex scenes at around 400 kb/s at 10 frames/sec. In 105], a \same orientation" crossband vector was formed after three levels of decomposition with a Gabor-like wavelet lter applied to a color image in the YUV color space. For each color plane, the lowest band was scalar quantized. For the next higher bands, 15-D vectors were formed by taking 5 coe cients from each color plane: one coe cient from the lower band combined with the 2 2 block from the next higher band of the same orientation. At the highest level, the UV color planes were discarded, and the luminance plane was encoded as 4 4 intra-band blocks. Unstructured full-search codebooks were designed by the GLA for each group of vectors. A PSNR of 28.2 dB was achieved at 1.13 bpp for the color LENA test image.