Speaker diarization refers to the task of segmenting a given recording into homogeneous speaker-specific regions [1]}, [2]}. The conventional approach for diarization [3]}, [4]} involves applying speech activity detection (SAD) followed by clustering of fixed-dimensional speaker embeddings – usually i-vectors [5]} or neural embeddings [6]}, [7]} — extracted from small subsegments of the speech regions. This may optionally be followed by a resegmentation step [8]}, [9]}. However, this framework inherently makes a single speaker assumption, since every subsegment can only be assigned a single label through hard clustering.
