1. A vehicle position detection system comprising: a reference data database in which reference data is stored, wherein the reference data is generated by associating image feature point data generated by extracting image feature points from a captured image obtained by capturing a scene viewed from a vehicle, with an image-capturing position at which an image is captured to obtain the captured image corresponding to the image feature point data, and a recorded vehicle event that indicates a vehicle event at when the image is captured to obtain the captured image; a captured image processing unit to which an actually-captured image is input, and which outputs data for matching generated by extracting image feature points from the actually-captured image, wherein the actually-captured image is obtained by capturing an image of a scene using a vehicle-mounted camera; a vehicle event evaluation unit that generates information on an actual vehicle event that is a vehicle event detected by the vehicle; a first reference data output unit that extracts first reference data whose image-capturing position is located in a vicinity of an estimated position of the vehicle, from among a set of the reference data stored in the reference data database, based on the estimated position of the vehicle, and outputs the extracted first reference data as a matching candidate for the data for matching; a second reference data output unit that extracts second reference data that includes the recorded vehicle event that matches the actual vehicle event, from among the set of the reference data stored in the reference data database, based on the vehicle event, and outputs the extracted second reference data as the matching candidate for the data for matching, an extraction preference being set so that the extracting of the first reference data by the first reference data output unit is performed in preference to the extracting of the second reference data by the second reference data output unit; a matching performing unit that: performs matching between the first reference data and the data for matching to determine whether the first reference data matches the data for matching; and when the first reference data does not match the data for matching, performs matching between the second reference data and the data for matching to determine whether the second reference data matches the data for matching; and a vehicle position determination unit that determines a position of the vehicle, based on the image-capturing position associated with the one of the first reference data or the second reference data that is determined to match the data for matching.