1. An image processing device for performing median processing in a time direction on a plurality of temporally consecutive input images, the device comprising: a pixel value extraction unit that extracts pixel values of a pixel of interest from the plurality of input images; a pixel value selection unit that rearranges a plurality of pixel values of a certain channel extracted for the pixel of interest by the pixel value extraction unit and selects a pixel value corresponding to a predetermined position after rearrangement; a pixel value obtaining unit that obtains a pixel value of another channel in the input image corresponding to the pixel value selected by the pixel value selection unit; and an output pixel value determination unit that, based on the pixel values obtained by the pixel value selection unit and the pixel value obtaining unit, determines an output pixel value of the pixel of interest.