1. An image processing apparatus for performing filling processing for filling a part of an image with a specific color, comprising: a holding unit which holds compressed image data that is generated by compressing input image data, wherein the compressed image data have, for each block formed from a predetermined number of pixels included in the input image data, a representative color of the block, a color layout of pixels included in the block, and an interpolation color which is a color other than the representative color included in the block; and a fill unit which performs, without decompressing the compressed image data, filling processing for filling the part of the compressed image data with the specific color, wherein, when filling the block corresponding to the part of the compressed image data with the specific color, said fill unit replaces the representative color and the interpolation color of the block corresponding to the part of the compressed image data with the specific color.