1. A sound and image segment sorting device comprising: a sound segment sorting unit that sorts a plurality of sound segments contained in a video based on an arbitrary operation condition and generates sound segment sorting results, each of said sound segments sorting results being a collection of sound clusters including at least one sound cluster as an element, and each of said at least one sound cluster being a collection of sound segments determined to be sound phenomena representing a same object; an image segment sorting unit that sorts a plurality of image segments contained in the video based on an arbitrary operation condition and generates image segment sorting results, each of said image segment sorting results being a collection of image clusters including at least one image cluster as an element, and each of said at least one image cluster being a collection of image segments determined to be image phenomena representing a same object; a multiple sorting result generation unit that generates at least one of a plurality of sound segment sorting results and a plurality of image segment sorting results by applying a plurality of different operation conditions to at least one of said sound segment sorting unit and said image segment sorting unit; a sorting result pair generation unit that generates a plurality of sorting result pairs each including one sound segment sorting result and one image segment sorting result based on the plurality of sound segment sorting results and the plurality of image segment sorting results obtained by said multiple sorting result generation unit; a sorting result comparative score calculation unit that calculates, for each sorting result pair, a sorting result comparative score representing a fitness between a sound segment sorting result and an image segment sorting result included in the sorting result pair; and a sorting result output unit that selects a sorting result pair having a high fitness among said plurality of sorting result pairs based on the sorting result comparative score and outputs a sound segment sorting result and an image segment sorting result included in the sorting result pair.