1. An environment recognition device comprising: an on-board camera mounted on a vehicle; an object specifying unit that specifies a target object to be recognized; an image data acquiring unit that acquires image data based on a color image in front of the vehicle imaged by the on-board camera; an average color calculating unit that calculates, based on the image data acquired by the image data acquiring unit, an average color of each of pixel blocks produced by splitting the color image by a preset size; a color information adding unit that determines to which of color indexes provided in advance the average color calculated in the average color calculating unit belongs, and stores a determination result in association with the pixel block as color information; a color histogram creating unit that accumulates along one direction on the color image the number of the pixel blocks that aimed color indexes match the color information associated by the color information adding unit, and creates a color histogram formed by one-dimensionally arranging accumulated values, for each of the color indexes and in time series in units of frames of the image data; and an object information adding unit that creates information about a target object specified by the object specifying unit on the basis of the color histograms created by the color histogram creating unit, and stores the created information as object information in association with the image data acquired by the image data acquiring unit.