1. A sound source detection system for detecting a sound source based on sound collected by a sound collecting device, comprising: a classifying unit configured to extract a plurality of characteristic amounts from the sound collected by the sound collecting device, and to set a plurality of classes according to a position of the sound source by a multi-class pattern recognition method, using the characteristic amounts; a class determining unit configured to extract the characteristic amounts from the sound collected by the sound collecting device when detecting the sound source, and to determine a class to which the extracted characteristic amounts belong, from the plurality of classes set by the classifying unit; and a sound source estimating unit configured to estimate the sound source, based on the class determined by the class determining unit.