1. An image processing apparatus comprising: a storage that stores a coefficient set designed through learning for each position of pixel sets referred to in a convolution such that a result of the convolution referring to a second image with the coefficient set is brought nearer to a first image, the second image being obtained by deteriorating the first image through a specific deterioration process; and a processor programmed to: acquire an input image captured via an optical system including a color filter array made UP from at least one unit color filter array; divide the input image into a plurality of regions; and perform the convolution by: determining a position of a target pixel set in the input image based on a region of the plurality of regions that is associated with a target pixel, a location within the at least one unit color filter array and associated with the target pixel set, and one or more colors associated with the at least one unit color filter array, reading the coefficient set from the storage corresponding to the determined position of the target pixel set, and generating an output image by convoluting the target pixel set with the coefficient set.