1. An image processing apparatus comprising: an extraction unit that extracts a color boundary of an image; an influence value calculation unit that calculates an influence value indicating the degree of an influence of the color boundary on a pixel; a setting unit that sets a coefficient for a convolution operation in accordance with the color boundary extracted in the extraction unit and the influence value calculated in the influence value calculation unit; and an operation unit that performs a convolution operation using the coefficient set by the setting unit.