Local Perceptual Weighting in JPEG2000 for Color Images
CC Attribution - NoDerivatives 2.0 UK: England & Wales:
You are free to use, copy, distribute and transmit the work or content in unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Moreno, Jaime
Otazu, Xavier
Vanrell, Maria
The aim of this work is to explain how to apply perceptual concepts to define a perceptual pre-quantizer and to improve JPEG2000 compressor. The approach consists in quantizing wavelet transform coefficients using some of the human visual system behavior properties. Noise is fatal to image compression performance, because it can be both annoying for the observer and consumes excessive bandwidth when the imagery is transmitted. Perceptual pre-quantization reduces unperceivable details and thus improve both visual impression and transmission properties. The comparison between JPEG2000 without and with perceptual pre-quantization shows that the latter is not favorable in PSNR, but the recovered image is more compressed at the same or even better visual quality measured with a weighted PSNR. Perceptual criteria were taken from the CIWaM (Chromatic Induction Wavelet Model).
 Thank you very much. I want to apologize for the delay. I wanted to use the computer of my wife, but I think that she prepares a plot against me. So sorry. The aim of my research is to try to improve the image, the JPEC compression, applying a method of color perception. But what is the problem in JPEC 2000 or any algorithm of compression? The problem is that just information theory criteria are applied. So when we send the bits to the decoder, we send first the most important bits of the most important coefficients. So it produce visible artifacts, since important perceptual information is discarded. Then our solution is not used. The information theory criteria is used perceptual criteria. And we need a color perception model which quantize the coefficients perceptually. And the chromatic induction wavelet model is suitable for it, or C1. But what is C1? C1, first of all, decompose the image in different wavelet planes and different spatial orientations. So with these coefficients, try to emulate some features of the human visual system, applying a weighting function. In this case, it's the C prime of the entire image, but the residual plane. So C1 combined three important features. That is spatial orientation, spatial frequency, and surround contrast. When we quantize these coefficients, we obtain a matrix with weighted coefficient, I want to say. So with these coefficients, if we decontize, excuse me, inverse wavelet transform, we obtain an image that is the representation that is forming in our brain. In these cases, a LENA image looks at two methods. I want to present through examples how works B1. In these cases, for grayscale images, the question is, what is the brightest rectangle here? We know this example, but obviously are the same, since the rectangle B is surrounded by darker checkers, while the rectangle A is surrounded by brighter checkers. So in a column brightness profile, C1 predicts that is the dash function, that the rectangle A is darker than the rectangle B, while the digital values, that is the continuous function, said that have the same color. In this case, C1 tried to predict what is the color that we perceive. And in these cases, it's a grayscale. But in color, this is the example for color, and the question is, what is the color of the tiles? The tile at the top is a darker brown, and the tile at the side is orange, but it's just our perceptual reality. But the physical reality says that it's the same darker brown color. So we can say that our brain is save us. Briefly, C1 tried to predict what I said, taking into account all the neighborhood. So in this case, the first tile, the darker, the brown, I won't say. The brown is almost the same, but the other one, the tile at the side, because it's inside a shadow, we, the C1 predicts that is almost orange. Going back with our problem, in the entire process, this is the general block for JPEG 2000 in color, but it's important to say that just the quantization is the block that introduced distortion in the process. Since we reduce the dynamic range just dividing by a number, which is called the quantization step, it's just a waiting. It's a fixed waiting. So the standard JPEG 2000 includes some visual frequency waiting, but it's just for an entire subband. In this case, we try to depict that in an entire band, apply the same value. So it's not important to transmit, because both sides know this value. But our proposal is to introduce, previously to the quantization step, introduce a block of C1, which produce one wait by any coefficient. So in this case, it's lean at 400 dots per inch, and we know the difference. We say that JPEG 2000 is global, because it's for an entire subband, and our approach is for a single coefficient. The problem is that we don't know what is the value, because it depends on the distance and for the image, too. Our results was tested of all these 16 images obtained from the image database of the University of Southern California. And I present, first of all, three individual results. For example, in this case, the image at the left, the JPEG 2000 has better numerical quality, since in this case, it's 31 decibels. And the other one, in our results, we have 27. But you can see that there are a lot of artifacts in the edges, and our image is soft. The result is because to pick signal to noise radio, check just pixel by pixel the difference, not global. So in this case, we change quantizing, or we change weighting every coefficient. So we change the statistical properties of the image. So that's why we obtain this kind of image. Even we have less numerical quality, we have better perceptual quality. The same result we obtained, for example, in this image, the airplane. And we can say that, for example, the shape of the montage here is imperceivable, and here is the shape of the mountains. We can see also the letters in the airplane. There are some special cases, like in this image, like Babun, that we obtain better, both compression ratio and perceptual quality. This image is with the same numerical quality. So we have the same numerical quality, but better perceptual quality. And we can say that the JPEG 2000 image have some visible artifacts. On average, this figure represents, in the horizontal axis, a set of coefficients that we select by a certain threshold. In this case, 32, 16, 8, and so on. And this is the horizontal. The vertical is the amount of bits that now, on average, are pixelated represented. So for example, at the last bit plane, we have almost six bits per pixel. The original pixel was 24 bit depth, so we reduce. The blue function is the result obtained by JPEG 2000. And the green one is the result obtained by our approach. So we can say that the lower the function, the better the compression, because we need less bits to represent the same coefficients. So our ongoing work, which is not included in the proceedings, is to try to predict with this when we receive this image, this soft image, try to predict what were the weightings that we apply. So we obtain, for example, that it's almost the same numerical quality, but we have better perceptual one at the same bit rate. We try to combine another entropy colors. And with this entropy color, we obtain better results, both subjectic and objectic quality. So in conclusion, SIGWAM perceptual quantization compressed more or less 20% higher. The numerical quality is worse, but the perceptual quality is better. So when we introduce the inverse perceptual quantization, we can obtain both numerical and perceptual quality. Thanks for your attention. Muchas gracias. Thank you. OK. Thank you. Just one announcement before we go to the questions. We have some practical announcements from the hosts, so don't leave immediately when we finish with the questions. OK. Now if we have questions, we have five minutes, maybe to answer them. OK. Do we have one here? And thank you for the nice talk. Have you conducted any psychophysical tests to judge the performance of the method? No, not yet. Not yet, just with our partners of the group. But for more people, we don't try these images. Do you compare your work with other usual waiting that like not know from the rest? Waiting, for example? I can't hear you. Excuse me. OK. Do you compare your work with other usual waiting of GPAC 2000, like, for example, the first one, not know waiting? No. Not know from EPFL? What? It was the first to propose some waiting in GPAC 2000? No, we don't compare our results with this. We try to improve the quantization. But we don't try. We don't prove with another. But there are a lot of other proposals for such kind of waiting. Yeah, no, we don't know. We have time for more questions. OK. So thank you again. Thank you. Thank you. Thank you.
 Thank you very much.  I want to apologize for the delay.  I wanted to use the computer of my wife,  but I think that she prepares a plot against me.  So sorry.  The aim of my research is to try to improve the image,  the JPEC compression, applying a method of color perception.  But what is the problem in JPEC 2000 or any algorithm  of compression?  The problem is that just information theory criteria  are applied.  So when we send the bits to the decoder,  we send first the most important bits  of the most important coefficients.  So it produce visible artifacts, since important perceptual  information is discarded.  Then our solution is not used.  The information theory criteria is used perceptual criteria.  And we need a color perception model which quantize  the coefficients perceptually.  And the chromatic induction wavelet model  is suitable for it, or C1.  But what is C1?  C1, first of all, decompose the image  in different wavelet planes and different spatial orientations.  So with these coefficients, try to emulate some features  of the human visual system, applying a weighting function.  In this case, it's the C prime of the entire image,  but the residual plane.  So C1 combined three important features.  That is spatial orientation, spatial frequency,  and surround contrast.  When we quantize these coefficients,  we obtain a matrix with weighted coefficient, I want to say.  So with these coefficients, if we decontize,  excuse me, inverse wavelet transform,  we obtain an image that is the representation that  is forming in our brain.  In these cases, a LENA image looks at two methods.  I want to present through examples  how works B1.  In these cases, for grayscale images,  the question is, what is the brightest rectangle here?  We know this example, but obviously are the same,  since the rectangle B is surrounded by darker checkers,  while the rectangle A is surrounded by brighter checkers.  So in a column brightness profile,  C1 predicts that is the dash function,  that the rectangle A is darker than the rectangle B,  while the digital values, that is the continuous function,  said that have the same color.  In this case, C1 tried to predict  what is the color that we perceive.  And in these cases, it's a grayscale.  But in color, this is the example for color,  and the question is, what is the color of the tiles?  The tile at the top is a darker brown,  and the tile at the side is orange,  but it's just our perceptual reality.  But the physical reality says that it's  the same darker brown color.  So we can say that our brain is save us.  Briefly, C1 tried to predict what I said,  taking into account all the neighborhood.  So in this case, the first tile, the darker, the brown,  I won't say.  The brown is almost the same, but the other one,  the tile at the side, because it's inside a shadow,  we, the C1 predicts that is almost orange.  Going back with our problem, in the entire process,  this is the general block for JPEG 2000 in color,  but it's important to say that just the quantization  is the block that introduced distortion in the process.  Since we reduce the dynamic range just  dividing by a number, which is called the quantization step,  it's just a waiting.  It's a fixed waiting.  So the standard JPEG 2000 includes some visual frequency  waiting, but it's just for an entire subband.  In this case, we try to depict that in an entire band,  apply the same value.  So it's not important to transmit,  because both sides know this value.  But our proposal is to introduce,  previously to the quantization step,  introduce a block of C1, which produce one wait  by any coefficient.  So in this case, it's lean at 400 dots per inch,  and we know the difference.  We say that JPEG 2000 is global, because it's  for an entire subband, and our approach  is for a single coefficient.  The problem is that we don't know what is the value,  because it depends on the distance  and for the image, too.  Our results was tested of all these 16 images  obtained from the image database of the University  of Southern California.  And I present, first of all, three individual results.  For example, in this case, the image at the left,  the JPEG 2000 has better numerical quality,  since in this case, it's 31 decibels.  And the other one, in our results, we have 27.  But you can see that there are a lot of artifacts in the edges,  and our image is soft.  The result is because to pick signal to noise radio,  check just pixel by pixel the difference, not global.  So in this case, we change quantizing,  or we change weighting every coefficient.  So we change the statistical properties of the image.  So that's why we obtain this kind of image.  Even we have less numerical quality,  we have better perceptual quality.  The same result we obtained, for example,  in this image, the airplane.  And we can say that, for example, the shape of the montage  here is imperceivable, and here is the shape of the mountains.  We can see also the letters in the airplane.  There are some special cases, like in this image,  like Babun, that we obtain better,  both compression ratio and perceptual quality.  This image is with the same numerical quality.  So we have the same numerical quality,  but better perceptual quality.  And we can say that the JPEG 2000 image  have some visible artifacts.  On average, this figure represents,  in the horizontal axis, a set of coefficients  that we select by a certain threshold.  In this case, 32, 16, 8, and so on.  And this is the horizontal.  The vertical is the amount of bits that now, on average,  are pixelated represented.  So for example, at the last bit plane,  we have almost six bits per pixel.  The original pixel was 24 bit depth, so we reduce.  The blue function is the result obtained by JPEG 2000.  And the green one is the result obtained by our approach.  So we can say that the lower the function, the better  the compression, because we need less bits  to represent the same coefficients.  So our ongoing work, which is not included in the proceedings,  is to try to predict with this when we receive  this image, this soft image, try to predict  what were the weightings that we apply.  So we obtain, for example, that it's  almost the same numerical quality,  but we have better perceptual one at the same bit rate.  We try to combine another entropy colors.  And with this entropy color, we obtain better results,  both subjectic and objectic quality.  So in conclusion, SIGWAM perceptual quantization  compressed more or less 20% higher.  The numerical quality is worse, but the perceptual quality  is better.  So when we introduce the inverse perceptual quantization,  we can obtain both numerical and perceptual quality.  Thanks for your attention.  Muchas gracias.  Thank you.  OK.  Thank you.  Just one announcement before we go to the questions.  We have some practical announcements from the hosts,  so don't leave immediately when we finish with the questions.  OK.  Now if we have questions, we have five minutes,  maybe to answer them.  OK.  Do we have one here?  And thank you for the nice talk.  Have you conducted any psychophysical tests  to judge the performance of the method?  No, not yet.  Not yet, just with our partners of the group.  But for more people, we don't try these images.  Do you compare your work with other usual waiting  that like not know from the rest?  Waiting, for example?  I can't hear you.  Excuse me.  OK.  Do you compare your work with other usual waiting of GPAC  2000, like, for example, the first one, not know waiting?  No.  Not know from EPFL?  What?  It was the first to propose some waiting in GPAC 2000?  No, we don't compare our results with this.  We try to improve the quantization.  But we don't try.  We don't prove with another.  But there are a lot of other proposals  for such kind of waiting.  Yeah, no, we don't know.  We have time for more questions.  OK.  So thank you again.  Thank you.  Thank you.  Thank you.
Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_0_59.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_60_191.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_192_731.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_732_1607.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_1608_1871.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_1872_2339.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_2340_2783.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_2784_3959.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_3960_4511.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_4512_5447.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_5448_6275.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_6276_7211.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_7212_8159.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_8160_8627.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_8628_9095.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_9096_9887.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_9888_10091.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_10092_11111.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_11112_11567.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_11568_14855.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_14856_16883.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_16884_17483.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_17484_18431.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_18432_19043.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_19044_20135.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_20136_22763.png Local_Perceptual_Weighting_in_JPEG2000_for_Color_Images_22764_22804.png