1. An image processing apparatus comprising: an acquisition unit configured to acquire an input image captured via an optical system; a storage unit configured to store therein coefficient set designed through learning for each position of pixel sets referred to in a convolution such that a result of the convolution referring to a second image with the coefficient set is brought nearer to a first image, the second image being obtained by deteriorating the first image through a specific deterioration process; and a convolution unit configured to read the coefficient set from the storage unit correspondingly to a position of a target pixel set in the input image, and generate an output image by convoluting the target pixel set with the coefficient set.