The Clotho Dataset [1]} is by far the dataset most researchers have examined and experimented on. The Clotho Dataset consists of 4981 audio sample of duration from 15 to 30 seconds. Each audio sample are matched to 5 crowdsourced English audio annotations, or captions, that describe the events in that audio. The Clotho Dataset was created to address the critiques of the larger Audiocaps dataset [2]}, which were found to contain biases in the annotation process. There are also niche datasets like the Hospital Scene and Car Scene dataset [3]} which are Mandarin annotated.
