1. A moving object detecting device that detects a moving object by using an image, the device comprising: a motion degree obtaining portion that obtains a motion degree of a pixel between image frames; a color obtaining portion that obtains a color of the pixel included in the image frame; an evaluation score calculating portion that calculates an evaluation score indicating a motion level of the color on the basis of the motion degree for each color obtained by the color obtaining portion; and a moving object detecting portion that detects the moving object on the basis of the evaluation score for each color.