1. An image processing device, comprising: an edge determination unit that determines an edge in an image; a color estimation unit that identifies a non-target image to remove the non-target image from the image by selecting a block of interest region in the image based on a neighborhood of a target pixel on which a non-target image removing process is to be performed and estimates an original color of the image which is not affected by another image read together as a non-target image, according to a result of the edge determination performed by the edge determination unit; and a color value candidate selecting unit that substitutes a color of a target pixel on which a non-target image removing process is to be performed and performs a quadratic differential on a histogram within a partial color space from a position within a color space which corresponds to the color of the target pixel; wherein a length of the partial color space is determined by the following relationships: L =f[B(P)]:f[x] is a monotone increasing function of x L =g[E(P)]:g[x] is a monotone decreasing function of x; where f(x)=a×x (n) , g(x)=−a×x (n) , B(P) is a brightness of a pixel of interest, E(P) is a degree of edge of the pixel of interest, and a is a positive constant and n is a real value which is equal to or larger than 1.