1. An object detecting system comprising: stereo-image taking means for simultaneously taking images of an object by a pair of right and left cameras mounted at the same height and outputting the images as a reference image and a comparative image; stereo matching means for performing stereo matching so that a reference pixel block having a predetermined number of pixels is set in the reference image and differences in luminance pattern between the reference pixel block and comparative pixel blocks are calculated according to a predetermined calculation expression, the comparative pixel blocks having the same shape as that of the reference pixel block on an epipolar line in the comparative image and being corresponding to the reference pixel block, and so that a comparative pixel block having the smallest difference is specified, the stereo matching means calculating a parallax from a position of the specified comparative pixel block in the comparative image and a position of the reference pixel block in the reference image, the parallax being calculated for each reference pixel block; grouping means for grouping the parallaxes regarded as corresponding to the same object on the basis of the parallaxes of the reference pixel blocks; and determination means for detecting objects in the reference image on the basis of the grouped parallaxes and setting regions including images of the objects, selecting a reference pixel block belonging to a left end portion or a right end portion of each of the regions, performing stereo matching again for an area on the right or left side of a comparative pixel block, which is specified in the comparative image corresponding to the reference pixel block, on an epipolar line including the specified comparative pixel block, and determining that the object in the region is mismatched when a comparative pixel block that is different from the specified comparative pixel block and provides the minimum difference less than or equal to a preset threshold value is detected.