1. A moving object detecting device that detects a moving object by using an image, the device comprising: a motion degree obtaining portion configured to obtain a motion degree of a pixel between image frames; a color obtaining portion configured to obtain a color of the pixel included in the image frame; an evaluation score calculating portion configured to calculate an evaluation score indicating a motion level of the color on the basis of the motion degree for each color obtained by the color obtaining portion; and a moving object detecting portion configured to detect the moving object on the basis of the evaluation score for each color; wherein one or more of the motion degree obtaining portion, the color obtaining portion, the evaluation score calculation portion and the moving object detection portion are implemented via a CPU.