1. A three-dimensional object detection device comprising: an image capturing unit arranged to capture images rearward of a host vehicle equipped with the three-dimensional object detection device; a detection area setting unit programmed to set a detection area for detecting a three-dimensional object rearward of the host vehicle; an image conversion unit programmed to convert a viewpoint of the images obtained by the image capturing unit to create bird's-eye view images; a three-dimensional object detection unit programmed to detect a presence of a three-dimensional object within the detection area in which the bird's-eye view images obtained at different times by the image conversion unit are aligned, and difference waveform information is generated by counting and creating a frequency distribution of a number of pixels that indicate a predetermined difference having a difference threshold value or greater in a difference image of the bird's-eye view images that were aligned to detect the presence of the three-dimensional object within the detection area; a reference image area setting unit programmed to set, as a reference image area, an image area that is different from the detection area and in which extraction of contour information of a subject is predicted; a contour information extraction unit programmed to extract contour information of a subject in the reference image area and the detection area; a reference sharpness calculation unit programmed to calculates an amount of brightness of an image in the reference image area as a reference sharpness, based on the contour information extracted in the reference image area; and a subject sharpness calculation unit programmed to calculates an amount of brightness of an image in the detection area as a subject sharpness, based on the contour information extracted in the detection area, the three-dimensional object detection unit calculating a final sharpness based on the reference sharpness and the subject sharpness, and setting the difference threshold value based on the final sharpness that was calculated.