1. An image processing apparatus to perform an enlarging process for an original image constituted by C colors (C≧2), comprising: a target color selecting unit that selects two target colors C 1 and C 2 for each local region of the original image; a sorting unit that sorts the local region on the basis of a layout of the target colors selected by the target color selecting unit; and a processing unit that determines pixel values after enlargement from the target colors C 1 and C 2 in accordance with a result of the sorting performed by the sorting unit.