1. A three-dimensional object detection device comprising: an image capturing unit configured to capture images of an area including a right-side detection area or a left-side detection area rearward of a vehicle; a three-dimensional object detection unit programmed to detect a three-dimensional object based on the images acquired by the image capturing unit; a high-luminance area assessment unit programmed to access a first detection area including a high-luminance area complying with a predetermined reference on either the right-side detection area or the left-side detection area; and a controller programmed to suppress detection of the three-dimensional object based on image information of the first detection area that was detected, and to maintain or promote detection of the three-dimensional object based on image information of a second detection area other than the first detection area within the right-side detection area or the left-side detection area.