1. An image quality enhancement apparatus comprising: a peaking filter for performing a peaking operation on an image signal to generate a peaking signal; a selecting circuit coupled to the peaking filter for selecting a plurality of pixels from the peaking signal and the image signal; and a luminance setting circuit coupled to the selecting circuit for deciding a luminance value for a target pixel of the image signal in accordance with luminance values of the plurality of pixels, wherein the luminance setting circuit comprises: a median filter for identifying a median of luminance values of the plurality of pixels; and a decision circuit coupled to the median filter for determining whether an initial luminance value of the target pixel is between the median and a luminance value of a reference pixel, which corresponds to the target pixel, from the peaking signal; wherein if the initial luminance value is not between the median and the luminance value of the reference pixel, the decision circuit sets the luminance of the target pixel to be the median.