1. An image processing apparatus, comprising: an initial interpolation unit, arranged for receiving image data from a frame buffer, wherein each pixel of the image data has only one color information, and for a target pixel of the image data, the initial interpolation unit estimates four second color information corresponding to up, down, left and right sides of the target pixel respectively according to first color information of the target pixel and color information of neighboring pixels, where a color to which the first color information corresponds is different from colors to which the four second color information corresponds; a color difference gradient estimating unit, coupled to the initial interpolation unit, the color difference gradient estimating unit arranged for calculating four color difference gradients corresponding to the up, down, left, and right sides of the target pixel respectively according to the four second color information of the target pixel; an edge texture characteristic determining unit, coupled to the color difference gradient estimating unit, the edge texture characteristic determining unit arranged for determining an edge texture characteristic of the target pixel according to the four color difference gradients of the target pixel; and an edge indication factor recording unit, coupled to the edge texture characteristic determining unit, the edge indication factor recording unit arranged for determining whether to modify a bit value of the first color information of the target pixel that is stored in a frame buffer according to the edge texture characteristic of the target pixel.