1. A sound source detection system for detecting a sound source based on sound collected by a sound collecting device, comprising: a classifying unit configured to extract a plurality of characteristic amounts from the sound collected by the sound collecting device, and to set a plurality of classes according to a position of the sound source by a multi-class pattern recognition, using the characteristic amounts; a class determining unit configured to extract the characteristic amounts from the sound collected by the sound collecting device when detecting the sound source, and to determine a class to which the extracted characteristic amounts belong, from the plurality of classes set by the classifying unit; a sound source estimating unit configured to estimate the sound source, based on the class determined by the class determining unit; a sound source detecting unit configured to calculate a correlation value of sounds respectively collected by a plurality of microphones that constitute the sound collecting device, and to detect the sound source by comparing the calculated correlation value with a threshold value; and a correcting unit configured to correct the threshold value of the sound source detecting unit, based on a detection result obtained by the sound source detecting unit and an estimation result obtained by the sound source estimating unit, wherein the sound source detecting unit detects the sound source by comparing the correlation value with the threshold value corrected by the correcting unit.